bug: sometimes serves index.html files within a folder, sometimes does not

[bguiz]

System information

Swarm version: 0.5.8
OS & Version: Linux - Ubuntu 18.04
Commit hash : 6faff7f

$ swarm version
Swarm
Version: 0.5.8-unstable
Git Commit: 6faff7fcb6f25c706e75d8d3c8945c4231663b93
Go Version: go1.14.3
OS: linux

Expected behaviour

• when visiting a bzz:/ URL in a browser, the corresponding index.html page gets rendered all the time
  • e.g. http://localhost:8500/bzz:/some-hash/foo/ serves up foo/index.html within the uploaded manifest

Actual behaviour

• when visiting a bzz:/ URL in a browser, some pages do not get routed properly
  • this occurs when the URL is nested within one of more subfolders, and needs to render the index.html page within it
  • most pages like this work, but some do not
  • for pages in which this happens, interestingly, if you suffix /index to the URL, the correct file is served
    • e.g. when a request to foo/ fails, a request to foo/index does indeed serve up what foo/ was supposed to serve up
• An error gets thrown, visible in the log output of the local swarm node
  • stack trace indicates api.loadManifest and storage.TreeJoin
• the ones that don't work appear to be deterministic, but I have not been able to establish whtta the the "deciding factor" is. hypotheses include:
  • when file size is under a certain threshold?
    • are there some differences in routing when the file fits within a chunk, or when it is composed from multiple chunks?
  • manifest-related?
    • does the manifest encode any redirects/ fallbacks?

Steps to reproduce the behaviour

mkdir -p "./bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/"
swarm down --recursive bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0 "./bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/"
npx http-server -c-1 -p 2323 .

• now visit these URLs and observe that both work
  • http-server, works: http://localhost:2323/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/
  • swarm, works: http://localhost:8500/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/
• now visit these URLs and observe that one does not work
  • http-server, works: http://localhost:2323/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/
  • swarm, does not work: http://localhost:8500/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/
• trying out Ilan's suggestion to append index, both do indeed work, but this isn't really a solution, as the links shouldn't be changed for this
  • http-server, works: http://localhost:2323/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/index
  • swarm, works: http://localhost:8500/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/index

For the failing request, the HTTP traffic copied from my browser's dev tools is:

Request URL:http://localhost:8500/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/
Request Method:GET
Remote Address:127.0.0.1:8500
Status Code:200
Version:HTTP/1.1

Response Headers

HTTP/1.1 200 OK
Cache-Control: max-age=2147483648, immutable
Vary: Origin
Date: Fri, 22 May 2020 02:45:13 GMT
Content-Length: 0

Request Headers

Host: localhost:8500
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: _ga=GA1.1.1499329452.1573198589; _fbp=fb.0.1573198589580.1789094661; _pk_id.1.1fff=e2a89cbb2372268d.1576200674.3.1576215177.1576215091.; _hp2_id.2213510609=%7B%22userId%22%3A%221085954645012780%22%2C%22pageviewId%22%3A%226386709616043043%22%2C%22sessionId%22%3A%227532691733158163%22%2C%22identity%22%3Anull%2C%22trackerVersion%22%3A%224.0%22%7D; io=mfejrShcnl0mq0K3AAAN; _gid=GA1.1.723625548.1589709085
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0

Which can be summarised as: Returned a "200 OK" status with zero-length response body.

Backtrace

The corresponding output from my local swarm:

INFO [05-22|10:44:18.118] created ruid for request                 ruid=249a275d   method=GET  url=/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/
INFO [05-22|10:44:18.118] setting request host                     ruid=249a275d   host=localhost:8500
ERROR[05-22|10:44:18.121] panic recovery!                          stack trace="<REDACTED>"

The ERROR stack trace, which appears as a single line, and is redacted above, is shown below, parsed to add new lines:

goroutine 51140678 [running]:\nruntime/debug.Stack(0xc003f70548, 0xf787e0, 0x1c811f0)
  runtime/debug/stack.go:24 +0x9d\ngithub.com/ethersphere/swarm/api/http.RecoverPanic.func1.1(0xc002eb4000)
  github.com/ethersphere/swarm/api/http/middleware.go:194 +0x5d\npanic(0xf787e0, 0x1c811f0)
  runtime/panic.go:969 +0x166\ngithub.com/ethersphere/swarm/storage.NewTreeJoiner(...)
  github.com/ethersphere/swarm/storage/chunker.go:164\ngithub.com/ethersphere/swarm/storage.TreeJoin(0x1330240, 0xc018607500, 0x22f2cf8, 0x0, 0x0, 0x130f5e0, 0xc004a583f0, 0x0, 0x2)
  github.com/ethersphere/swarm/storage/chunker.go:139 +0x327\ngithub.com/ethersphere/swarm/storage.(*FileStore).Retrieve(0xc01797b380, 0x1330240, 0xc018607500, 0x22f2cf8, 0x0, 0x0, 0x19de3b8, 0x1cc8440)
  github.com/ethersphere/swarm/storage/filestore.go:98 +0x12f\ngithub.com/ethersphere/swarm/api.loadManifest(0x1330240, 0xc018607500, 0xc01797b380, 0x22f2cf8, 0x0, 0x0, 0x0, 0xc008c5ecc0, 0x10542c0, 0xd2ae01, ...)
  github.com/ethersphere/swarm/api/manifest.go:234 +0x125\ngithub.com/ethersphere/swarm/api.(*API).Get(0xc0179aa480, 0x1330240, 0xc018607500, 0xc008c5ecc0, 0x22f2cf8, 0x0, 0x0, 0xc0161c2964, 0x4, 0x0, ...)
  github.com/ethersphere/swarm/api/api.go:342 +0x1d9\ngithub.com/ethersphere/swarm/api.(*API).Get(0xc0179aa480, 0x1330240, 0xc018607500, 0xc008c5ecc0, 0xc00a830f40, 0x20, 0x40, 0xc003cae94b, 0x8, 0x1a5a26d, ...)
  github.com/ethersphere/swarm/api/api.go:361 +0x155b\ngithub.com/ethersphere/swarm/api/http.(*Server).HandleGetFile(0xc0187cef30, 0x1328cc0, 0xc00b8dc500, 0xc002eb4400)
  github.com/ethersphere/swarm/api/http/server.go:923 +0x57e\ngithub.com/ethersphere/swarm/api/http.(*Server).HandleBzzGet(0xc0187cef30, 0x1328cc0, 0xc00b8dc500, 0xc002eb4400)
  github.com/ethersphere/swarm/api/http/server.go:270 +0x21a\nnet/http.HandlerFunc.ServeHTTP(0xc0179b9640, 0x1328cc0, 0xc00b8dc500, 0xc002eb4400)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.InstrumentOpenTracing.func1(0x1328cc0, 0xc00b8dc500, 0xc002eb4300)
  github.com/ethersphere/swarm/api/http/middleware.go:161 +0x321\nnet/http.HandlerFunc.ServeHTTP(0xc0187cb860, 0x1328cc0, 0xc00b8dc500, 0xc002eb4300)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.ParseURI.func1(0x1328cc0, 0xc00b8dc500, 0xc002eb4200)
  github.com/ethersphere/swarm/api/http/middleware.go:80 +0x700\nnet/http.HandlerFunc.ServeHTTP(0xc0187cb880, 0x1328cc0, 0xc00b8dc500, 0xc002eb4200)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.InitLoggingResponseWriter.func1(0x132ab40, 0xc003cf0000, 0xc002eb4200)
  github.com/ethersphere/swarm/api/http/middleware.go:91 +0xdf\nnet/http.HandlerFunc.ServeHTTP(0xc0187cb8a0, 0x132ab40, 0xc003cf0000, 0xc002eb4200)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.SetRequestHost.func1(0x132ab40, 0xc003cf0000, 0xc002eb4100)
  github.com/ethersphere/swarm/api/http/middleware.go:52 +0x39e\nnet/http.HandlerFunc.ServeHTTP(0xc0187cb8c0, 0x132ab40, 0xc003cf0000, 0xc002eb4100)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.SetRequestID.func1(0x132ab40, 0xc003cf0000, 0xc002eb4000)
  github.com/ethersphere/swarm/api/http/middleware.go:41 +0x46e\nnet/http.HandlerFunc.ServeHTTP(0xc0187cb8e0, 0x132ab40, 0xc003cf0000, 0xc002eb4000)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.RecoverPanic.func1(0x132ab40, 0xc003cf0000, 0xc002eb4000)
  github.com/ethersphere/swarm/api/http/middleware.go:197 +0x74\nnet/http.HandlerFunc.ServeHTTP(0xc0187cb900, 0x132ab40, 0xc003cf0000, 0xc002eb4000)
  net/http/server.go:2012 +0x44\ngithub.com/ethersphere/swarm/api/http.methodHandler.ServeHTTP(0xc0187cef60, 0x132ab40, 0xc003cf0000, 0xc002eb4000)
  github.com/ethersphere/swarm/api/http/server.go:92 +0x8c\nnet/http.(*ServeMux).ServeHTTP(0xc018780a80, 0x132ab40, 0xc003cf0000, 0xc002eb4000)
  net/http/server.go:2387 +0x1a5\ngithub.com/rs/cors.(*Cors).Handler.func1(0x132ab40, 0xc003cf0000, 0xc002eb4000)
  github.com/rs/cors@v0.0.0-20160617231935-a62a804a8a00/cors.go:190 +0x1a4\nnet/http.HandlerFunc.ServeHTTP(0xc0187d64e0, 0x132ab40, 0xc003cf0000, 0xc002eb4000)
  net/http/server.go:2012 +0x44\nnet/http.serverHandler.ServeHTTP(0xc0187d2380, 0x132ab40, 0xc003cf0000, 0xc002eb4000)
  net/http/server.go:2807 +0xa3\nnet/http.(*conn).serve(0xc01861a3c0, 0x1330180, 0xc008c5ea40)
  net/http/server.go:1895 +0x86c\ncreated by net/http.(*Server).Serve
  net/http/server.go:2933 +0x35c\n" url=/bzz:/c5e6c48ddbb7a419b379fdd8fea776dc7e39373147ca568bf479e93fbfcab0c0/rif/rns/ headers="map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8] Accept-Encoding:[gzip, deflate] Accept-Language:[en-US,en;q=0.5] Connection:[keep-alive] Cookie:[_ga=GA1.1.1499329452.1573198589; _fbp=fb.0.1573198589580.1789094661; _pk_id.1.1fff=e2a89cbb2372268d.1576200674.3.1576215177.1576215091.; _hp2_id.2213510609=%7B%22userId%22%3A%221085954645012780%22%2C%22pageviewId%22%3A%226386709616043043%22%2C%22sessionId%22%3A%227532691733158163%22%2C%22identity%22%3Anull%2C%22trackerVersion%22%3A%224.0%22%7D; io=mfejrShcnl0mq0K3AAAN; _gid=GA1.1.723625548.1589709085] Upgrade-Insecure-Requests:[1] User-Agent:[Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0]]

... so it looks like it is something related to api.loadManifest and storage.TreeJoin
being unable to handle this particular type of file.

It appears that most files do indeed work, however, the ones that do not work now are the ones involving a redirect.
Perhaps the router that is built in is attempting to do something "smart"
by attempting to detect redirects and handle them internally, but winds up with a panic when doing so?

cc @vojtechsimetka