IPFS problems: Inefficiency

Imagine you have two IPFS nodes and unique content, created by you, in the first one. From the second, you can connect to the first and everyhing looks right. You then try to fetch that content. After some seconds it starts coming, the progress bar begins to move, that’s slow, very slow, doing an rsync would have been 20 times faster.

The progress bar halts. You investigate, the second node is not connected to the first anymore. Why, if that was the only source for the file we’re trying to fetch? It remains a mistery to this day. You reconnect manually, the progress bar moves again, halts, you’re disconnected again. Instead of reconnecting you decide to add the second node to the first node’s “Bootstrap” list.

I once tried to run an IPFS node on a VPS and store content on S3. There are two S3 datastore plugins available. After fixing some issues in one of them, recompiling go-ipfs, figuring out how to read settings from the IPFS config file, creating an init profile and recompiling again I got the node running. It worked. My idea was to host a bunch of data on that node. Data would be fetched from S3 on demand so there would be cheap and fast access to it from any IPFS node or gateway.

IPFS started doing hundreds of calls to S3 per minute – something I wouldn’t have known about if I hadn’t inserted some log statements in the plugin code, I mean before the huge AWS bill arrived. Apparently that was part of participation on the DHT. Adjusting some settings turned my node into a listen-only thing as I intended, but I’m not 100% sure it would work as an efficient content provider, and I’ll never know, as the memory and CPU usage got too high for my humble VPS and I had to turn it down.