pull through cache (proxy): docker pull error "unexpected EOF" due to common blob expiration

This happens consistently when trying to pull an image that shares a layer with another image that has been expired.  Because the TTL is 7 days we have images which we are unable to pull for multiple days until that image hits the TTL.
Steps to reproduce:

Patch the latest version of distribution to set the expiry to something small (10 minutes).  This is to make the problem reproducible quickly

At https://github.com/docker/distribution/blob/06fa77aa11a3913096efcb9b5bd25db8ef55a939/registry/proxy/proxymanifeststore.go#L15
Change to: const repositoryTTL = time.Duration(10 * time.Minute)

Build and run distribution configured as proxy to registry-1.docker.io

docker build -t registry .
docker run -d -p5000:5000 \
  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
  -e REGISTRY_STORAGE_CACHE_BLOBDESCRIPTOR=inmemory \
  --name registry registry

Pull the library/python image from the local registry proxy

docker rmi localhost:5000/library/python
docker pull localhost:5000/library/python

Wait a few minutes and pull the library/node image

docker rmi localhost:5000/library/python
docker rmi localhost:5000/library/node
docker pull localhost:5000/library/node

After waiting 10 minutes from step 3 you should see a message in registry logs about deleting blobs:

time="2017-08-15T17:25:21.516625079Z" level=debug msg="filesystem.Stat("/docker/registry/v2/blobs/sha256/81/81a1ab6955f8a17045fb4c2866b21c91445837507e86d489b03744557e6ed209/data")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=32.961µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Stat" trace.id=c635fc90-4397-4fa3-b10c-f32ed9578170 trace.line=138 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:21.516820044Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_manifests/revisions/sha256/81a1ab6955f8a17045fb4c2866b21c91445837507e86d489b03744557e6ed209/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=107.87µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=0ff8eb93-21c9-4200-822d-01cd733c4d91 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:21.516941798Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/81a1ab6955f8a17045fb4c2866b21c91445837507e86d489b03744557e6ed209/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=54.834µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=e8c98a94-f16e-435e-b1ff-05a6db79329c trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:22.411773597Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/968120d8cbe8c99ab47504ffeb49e16ce3b773d14ba4eb78c7a5eca6ad34e00c/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=167.598µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=ef753f71-1268-4194-9dd4-44a5d1c52e8b trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:22.411921019Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/96/968120d8cbe8c99ab47504ffeb49e16ce3b773d14ba4eb78c7a5eca6ad34e00c" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:22.412218761Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/96/968120d8cbe8c99ab47504ffeb49e16ce3b773d14ba4eb78c7a5eca6ad34e00c")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=246.403µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=1c013292-4444-4b5c-b308-23ca41022912 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:22.564524978Z" level=debug msg="filesystem.PutContent("/scheduler-state.json")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=30.512342ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).PutContent" trace.id=37717880-31f6-434b-b53b-8d995df33d65 trace.line=96 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:22.870933249Z" level=debug msg="filesystem.Stat("/")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=46.644µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Stat" trace.id=64602e0e-95de-411c-8874-7149fcc69d3b trace.line=138 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:25.771801383Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/2b032b8bbe8bc215ad3337035d0183fc353841ec6478d1c481e6e13628ad9e00/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=167.598µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=e372ab76-7ee4-4f64-a39a-d32d69cc3ec1 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:25.771924435Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/2b/2b032b8bbe8bc215ad3337035d0183fc353841ec6478d1c481e6e13628ad9e00" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:25.779106183Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/2b/2b032b8bbe8bc215ad3337035d0183fc353841ec6478d1c481e6e13628ad9e00")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=7.123118ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=ec999c6f-bf75-4eae-87e8-2502741a661f trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:27.561749439Z" level=debug msg="filesystem.PutContent("/scheduler-state.json")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=27.048315ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).PutContent" trace.id=3419b00d-1ea6-42e2-8f4f-6dadca82d361 trace.line=96 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:32.872650715Z" level=debug msg="filesystem.Stat("/")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=52.137µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Stat" trace.id=0b9a662b-29f2-4b19-bc23-ae0c13957819 trace.line=138 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:33.955121366Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/032924b710ba95b71e9a2d971e705bab895ddf7a219994c2cf8761f6959e62ed/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=177.186µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=310fddc4-a281-4e1b-8386-4512515c1786 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:33.955241721Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/03/032924b710ba95b71e9a2d971e705bab895ddf7a219994c2cf8761f6959e62ed" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:33.956808833Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/03/032924b710ba95b71e9a2d971e705bab895ddf7a219994c2cf8761f6959e62ed")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=1.512178ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=b6b3f5c5-ab15-4b2f-ab65-7e0310b58ea3 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:34.708783931Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=156.311µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=bc8112d8-f7cc-4ae8-b20b-8ddd0b191504 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:34.708953327Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/a9/a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:34.723095091Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/a9/a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=14.092723ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=1351974e-bb2b-459a-a5c6-112ef82d163d trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:37.126646541Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/450f5ea3e7eafb6df380c997fa891c13693af5ef46eb7a8ac6c54b58d00fecbb/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=166.699µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=f878de6c-a418-4c27-aa03-13e137fe9d53 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:37.12677958Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/45/450f5ea3e7eafb6df380c997fa891c13693af5ef46eb7a8ac6c54b58d00fecbb" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:37.127005209Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/45/450f5ea3e7eafb6df380c997fa891c13693af5ef46eb7a8ac6c54b58d00fecbb")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=172.193µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=e4e7a4ac-2356-4a5c-a4ed-d1e0682175d5 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:37.565964096Z" level=debug msg="filesystem.PutContent("/scheduler-state.json")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=30.373808ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).PutContent" trace.id=95f3b9b9-0b4a-4fac-8aba-07092f4d68e9 trace.line=96 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:38.457333866Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/b1dca9769b8b2f680ff0ed0de230c1d8984715ea8608e8a7df08b44a982f1fb3/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=161.006µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=06ada939-1974-4378-8b58-24afcbb8424e trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:38.45747999Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/b1/b1dca9769b8b2f680ff0ed0de230c1d8984715ea8608e8a7df08b44a982f1fb3" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:38.458513945Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/b1/b1dca9769b8b2f680ff0ed0de230c1d8984715ea8608e8a7df08b44a982f1fb3")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=973.127µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=25f52868-4bd1-49f1-a9cd-453c12d29162 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:39.423246535Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/0d7cffe4bfd71fa93d626a4a0761dc02083eda97bfc757feafdc168a9b24fbda/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=162.504µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=2567d11e-85a8-467d-9521-edc039c08729 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:39.423393558Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/0d/0d7cffe4bfd71fa93d626a4a0761dc02083eda97bfc757feafdc168a9b24fbda" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:39.430487012Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/0d/0d7cffe4bfd71fa93d626a4a0761dc02083eda97bfc757feafdc168a9b24fbda")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=7.03922ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=901fd0c0-2812-4164-8d08-50006117bfcd trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:39.604583279Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/ad74af05f5a24bcf9459ae1cf7718628c2aeb6b587eb51b6eeaf639aca3e566f/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=159.508µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=fedb294a-0a8b-426f-8569-15704de2eae6 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:39.604724709Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/ad/ad74af05f5a24bcf9459ae1cf7718628c2aeb6b587eb51b6eeaf639aca3e566f" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:39.620825712Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/ad/ad74af05f5a24bcf9459ae1cf7718628c2aeb6b587eb51b6eeaf639aca3e566f")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=16.030289ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=973070df-b3b6-44c4-8bbb-b41a73c29417 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:42.561907601Z" level=debug msg="filesystem.PutContent("/scheduler-state.json")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=27.883109ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).PutContent" trace.id=7570e3b4-5552-4673-a9a3-a4b9064b2e95 trace.line=96 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:42.871661741Z" level=debug msg="filesystem.Stat("/")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=55.633µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Stat" trace.id=513ab546-9e48-4ec8-a111-b154db454013 trace.line=138 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:44.872937411Z" level=debug msg="filesystem.Delete("/docker/registry/v2/repositories/library/python/_layers/sha256/3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e/link")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=163.802µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=6b8af6d0-ba88-4fb8-a41d-6c758d3df506 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:44.873077842Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/32/3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry version=v2.6.0-rc.1-213-g06fa77aa.m
time="2017-08-15T17:25:44.907699933Z" level=debug msg="filesystem.Delete("/docker/registry/v2/blobs/sha256/32/3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e")" environment=development go.version=go1.8.3 instance.id=cb5c9bf9-bc57-422c-9da2-51eefab7bfc6 service=registry trace.duration=34.561764ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Delete" trace.id=75af8054-f141-4211-9fa8-8182d8fd3a08 trace.line=178 version=v2.6.0-rc.1-213-g06fa77aa.m

Try to pull library/node again and see an error

docker rmi localhost:5000/library/python
docker rmi localhost:5000/library/node
docker pull localhost:5000/library/node # <--- this causes an error

Using default tag: latest
latest: Pulling from library/node
ad74af05f5a2: Retrying in 1 second
2b032b8bbe8b: Retrying in 1 second
a9a5b35f6ead: Downloading [==================================================>]  43.23MB/43.23MB
3245b5a1c52c: Waiting
afa075743392: Waiting
9fb9f21641cd: Waiting
2e38fbae54d2: Waiting
64e77472dc48: Waiting
unexpected EOF

Logs from registry on failed pull:
distribution-logs-EOF-pull.txt
Repeatedly trying docker pull localhost:5000/library/node fails until 10 minutes after the first pull at which point it succeeds.
If you hit the API directly for one of the blobs that is stuck retrying you get back a 200 OK but with an empty body:
$ curl -svO localhost:5000/v2/library/node/blobs/sha256:3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 5000 (#0)
> GET /v2/library/node/blobs/sha256:3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: max-age=31536000
< Content-Length: 131858814
< Content-Type: application/octet-stream
< Docker-Content-Digest: sha256:3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e
< Docker-Distribution-Api-Version: registry/2.0
< Etag: "sha256:3245b5a1c52cbf0ae23d948fb94ef7b321e3dc54e13c3f6cf79951ed8237f03e"
< X-Content-Type-Options: nosniff
< Date: Tue, 15 Aug 2017 17:49:08 GMT
<
{ [0 bytes data]
* transfer closed with 131858814 bytes remaining to read
* Closing connection 0

Docker version/info
$ docker version
Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:31:53 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:51:55 2017
 OS/Arch:      linux/amd64
 Experimental: true

$ docker info
Containers: 4
 Running: 1
 Paused: 0
 Stopped: 3
Images: 55
Server Version: 17.06.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.36-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.855GiB
Name: moby
ID: NWWG:VGBP:CL62:HAK7:WMUH:AVT2:7AN2:4DCR:DDT5:ST3A:HDSW:VNZI
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 30
 Goroutines: 41
 System Time: 2017-08-15T17:08:28.029292033Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

Logs from when I did the curl that returned an empty body:
time="2017-08-15T17:28:22.199275121Z" level=debug msg="authorizing request" go.version=go1.8.3 http.request.host="localhost:5000" http.request.id=46c49bff-038f-49db-b12a-a971b42b1df5 http.request.method=GET http.request.remoteaddr="<IP_ADDRESS>:56350" http.request.uri="/v2/library/node/blobs/sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" http.request.useragent="curl/7.54.0" vars.digest="sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" vars.name="library/node"
time="2017-08-15T17:28:22.201419438Z" level=debug msg=GetBlob go.version=go1.8.3 http.request.host="localhost:5000" http.request.id=46c49bff-038f-49db-b12a-a971b42b1df5 http.request.method=GET http.request.remoteaddr="<IP_ADDRESS>:56350" http.request.uri="/v2/library/node/blobs/sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" http.request.useragent="curl/7.54.0" vars.digest="sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" vars.name="library/node"
time="2017-08-15T17:28:22.201562765Z" level=debug msg="filesystem.URLFor("/docker/registry/v2/blobs/sha256/a9/a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba/data")" go.version=go1.8.3 http.request.host="localhost:5000" http.request.id=46c49bff-038f-49db-b12a-a971b42b1df5 http.request.method=GET http.request.remoteaddr="<IP_ADDRESS>:56350" http.request.uri="/v2/library/node/blobs/sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" http.request.useragent="curl/7.54.0" trace.duration=18.477µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).URLFor" trace.id=8c659a30-a431-4395-8e40-ccc974c0f720 trace.line=190 vars.digest="sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" vars.name="library/node"
time="2017-08-15T17:28:22.201742648Z" level=debug msg="filesystem.Reader("/docker/registry/v2/blobs/sha256/a9/a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba/data", 0)" go.version=go1.8.3 http.request.host="localhost:5000" http.request.id=46c49bff-038f-49db-b12a-a971b42b1df5 http.request.method=GET http.request.remoteaddr="<IP_ADDRESS>:56350" http.request.uri="/v2/library/node/blobs/sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" http.request.useragent="curl/7.54.0" trace.duration=34.658µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).Reader" trace.id=e8eb926b-da05-41c3-bc48-e6feab143b96 trace.line=108 vars.digest="sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" vars.name="library/node"
time="2017-08-15T17:28:22.201901257Z" level=info msg="response completed" go.version=go1.8.3 http.request.host="localhost:5000" http.request.id=46c49bff-038f-49db-b12a-a971b42b1df5 http.request.method=GET http.request.remoteaddr="<IP_ADDRESS>:56350" http.request.uri="/v2/library/node/blobs/sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba" http.request.useragent="curl/7.54.0" http.response.contenttype="application/octet-stream" http.response.duration=3.638817ms http.response.status=200 http.response.written=0
<IP_ADDRESS> - - [15/Aug/2017:17:28:22 +0000] "GET /v2/library/node/blobs/sha256:a9a5b35f6ead105e66a9af969454ac09b5772eeb0c6281972c48d2ce882e8eba HTTP/1.1" 200 0 "" "curl/7.54.0"

Reproduced again to capture pprof output for goroutines.  Attached logs and pprof capture taken immediately after the curl returned an empty body.
distribution_logs.with_pprof.txt
pprof_goroutines.txt

The workaround is to restart and delete the cache before the 7 day ttl. Not ideal at all.

I solve the problem by just rebuild the image and push it to the registry.

We also had to work around it by restarting our Docker Registry systemd unit before the cache expires:
RuntimeMaxSec=604740

https://www.freedesktop.org/software/systemd/man/systemd.service.html#RuntimeMaxSec=

Is there anything I can do to get this fixed sooner rather than later?
This is really annoying and hard to track down.
I've been searching for months, off and on.

@docwhat Until the bug is fixed your best best is to clean the cache before the expiry time.

any plan or status update on the issue fixing?
The issue has been raised 16 month...

I think remove  the STORAGE_CACHE_BLOBDESCRIPTOR setting in proxy mode can help fix this problem

Stumbled while researching pull-through cache TTL. Am I reading this correctly that pull-through cache as described in https://docs.docker.com/registry/recipes/mirror/#run-a-registry-as-a-pull-through-cache will return bad results after 7 days due cache expiration not working properly? This issue has been open since 2017 and there is no mention of the problem in the documentation so I am probably missing some nuance here. If not is the guidance not to use pull-through cache or is this an intermittent thing? Thanks.

Stumbled while researching pull-through cache TTL. Am I reading this correctly that pull-through cache as described in https://docs.docker.com/registry/recipes/mirror/#run-a-registry-as-a-pull-through-cache will return bad results after 7 days due cache expiration not working properly? This issue has been open since 2017 and there is no mention of the problem in the documentation so I am probably missing some nuance here. If not is the guidance not to use pull-through cache or is this an intermittent thing? Thanks.

Stumbled over this while researching pull-through cache TTL. Am I reading this correctly that pull-through cache as described in https://docs.docker.com/registry/recipes/mirror/#run-a-registry-as-a-pull-through-cache will return bad results after 7 days due cache expiration not working properly? This issue has been open since 2017 and there is no mention of the problem in the documentation so I am probably missing some nuance here. If not is the guidance not to use pull-through cache or is this an intermittent thing? Thanks.

my workaround is set cache expiration as 20 years, never clean up. We have 8TB cache, only need to clean up once a year.

Stumbled over this while researching pull-through cache TTL. Am I reading this correctly that pull-through cache as described in https://docs.docker.com/registry/recipes/mirror/#run-a-registry-as-a-pull-through-cache will return bad results after 7 days due cache expiration not working properly? This issue has been open since 2017 and there is no mention of the problem in the documentation so I am probably missing some nuance here. If not is the guidance not to use pull-through cache or is this an intermittent thing? Thanks.

my workaround is set cache expiration as 20 years, never clean up. We have 8TB cache, only need to clean up once a year.

Does this cache expiration also happen when garbage collections do not run, we are running into a very similar problem but don't seem to find any deletions of blobs in the debugging logs. Only storage deletions that we can find in the logs are like s3aws.Delete("/docker/registry/v2/repositories/docker/.../.../_uploads/.
At a certain moment after a push (not even sure if this is instant, or if this happens after a while due to expiration like in this case) is pushed blobs that are referenced to by manifests are just not in the storage.

my workaround is set cache expiration as 20 years, never clean up automatically. We have 8TB cache, only need to clean up once a year.

how do you change expiration and turn off auto cleanup?

https://github.com/docker/distribution/blob/06fa77aa11a3913096efcb9b5bd25db8ef55a939/registry/proxy/proxymanifeststore.go#L15

@messiahUA just hardcode this num: https://github.com/docker/distribution/blob/06fa77aa11a3913096efcb9b5bd25db8ef55a939/registry/proxy/proxymanifeststore.go#L15, and then rebuild image.

@messiahUA you could use this image andyzhangx/registry:v2.7.0-nottl, I have set up a docker registry cache service based on this image, and works stable for more than 3 years, I use a 8TB disk to store image caches, only need to clean up once every two years, that's quite easy to maintain.

Just hit this as well, trying to pull the rust image today failed:
Jun 05 08:37:12 <..>: time="2023-06-05T08:37:12.398719924Z" level=info msg="Adding new scheduler entry for library/alpine@sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6 with ttl=167h59m59.99999796s" go.version=go1.19.9 instance.id=bebddc3d-0119-4f00-b361-94be3293a797 service=registry version=2.8.2
Jun 05 14:00:40 <..>: time="2023-06-05T14:00:40.637893868Z" level=info msg="Adding new scheduler entry for library/python@sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6 with ttl=167h59m59.999997453s" go.version=go1.19.9 instance.id=bebddc3d-0119-4f00-b361-94be3293a797 service=registry version=2.8.2
Jun 07 11:46:50 <..>: time="2023-06-07T11:46:50.002688528Z" level=info msg="Adding new scheduler entry for library/rust@sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6 with ttl=167h59m59.999981317s" go.version=go1.19.9 instance.id=b4f642eb-3a2d-4532-967e-bfd11a9e124d service=registry version=2.8.2

[...]

Jun 12 08:37:12 <..>: time="2023-06-12T08:37:12.401076752Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/8a/8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6" go.version=go1.19.9 instance.id=17a33ca7-e103-466c-86b3-fa79c93c4763 service=registry version=2.8.2
Jun 12 14:00:40 <..>: time="2023-06-12T14:00:40.638837146Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/8a/8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6" go.version=go1.19.9 instance.id=17a33ca7-e103-466c-86b3-fa79c93c4763 service=registry version=2.8.2
Jun 12 14:00:40 <..>: time="2023-06-12T14:00:40.638909171Z" level=error msg="Scheduler error returned from OnExpire(library/python@sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6): filesystem: Path not found: /docker/registry/v2/blobs/sha256/8a/8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6" go.version=go1.19.9 instance.id=17a33ca7-e103-466c-86b3-fa79c93c4763 service=registry version=2.8.2

[..]

Jun 14 08:02:59 <..>: time="2023-06-14T08:02:59.418672611Z" level=info msg="response completed" go.version=go1.19.9 http.request.host=<..> http.request.id=4da103aa-ab31-44d2-8c14-ecbbf60b4a7f http.request.method=GET http.request.remoteaddr="<IP_ADDRESS>:44766" http.request.uri="/v2/library/rust/blobs/sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6" http.request.useragent="go-containerregistry/v0.15.2" http.response.contenttype="application/octet-stream" http.response.duration=2.734889ms http.response.status=200 http.response.written=0
Jun 14 08:02:59 <..>: <IP_ADDRESS> - - [14/Jun/2023:08:02:59 +0000] "GET /v2/library/rust/blobs/sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6 HTTP/1.1" 200 0 "" "go-containerregistry/v0.15.2"

Apparently it schedules three separate removals of the the same layer; one for the alpine image, one for python and one for rust. The first (alpine) removed the file when it fired, the second one (python) failed since the file was already missing, and the third (rust) had not yet executed.
I think the reason seems pretty clear; the code obviously attempts to remove any existing timers and only keep the last one for the layer - effectively with the longest remaining TTL - but entries.Key also contains the image name (see the "Adding new scheduler" log entry above). So while they share the same file on disk, they have separate scheduler entries:
func (ttles *TTLExpirationScheduler) add(r reference.Reference, ttl time.Duration, eType int) {
	entry := &schedulerEntry{
		Key:       r.String(),
		Expiry:    time.Now().Add(ttl),
		EntryType: eType,
	}
	dcontext.GetLogger(ttles.ctx).Infof("Adding new scheduler entry for %s with ttl=%s", entry.Key, time.Until(entry.Expiry))
	if oldEntry, present := ttles.entries[entry.Key]; present && oldEntry.timer != nil {
		oldEntry.timer.Stop()
	}
[..]

I don't know if it's as easy as just fixing the keys used for the scheduler entries, or if this has wider repercussions.

If #3880 only allows you to change when the cached entries expire, that would not appear to fix the underlying issue reported here (as far as I understand it) beyond allowing you to configure how often the bug should be potentially triggered. Am I misunderstanding the issue or the fix?

Sorry, I accidentally closwd the wrong issue 🫠 will reopen

It would be great to cut a new release with this (and likely other recent changes) :pray:
I just cherry-picked #3880 onto 2.8.2 to hopefully get some handle on this invalidation quirk

@joshlay I don't think #3880 does anything to actually solve this bug, it only allows you to ignore it for longer (or shorter) by configuring how long to wait before the faulty logic executes. Or never, if you disable cleanup completely.
That might be an OK workaround for some, but let's not close this bug prematurely :-)
Maybe I didn't get all the details correct in i my comment above, but I think there's a real bug there that should be fixed.
Maybe some other recent change has addressed the issue?

Indeed, I've edited my post to try to be more clear @dhedberg
The invalidation problem is definitely real - we'd hit it weekly, consistently! The TTL just buys us some time; we can afford the growth, so I'm disabling it.
2.8.2 is the latest base I've tried, I can't speak to HEAD
I will have results on Tuesday about how disabling the TTL holds. That'll be the longest we've gone without requiring wiping the caches to fix unexpected EOF

Hello,
Upon a brief review of the source code, I discovered that cleanup can be implemented at the proxy level - it requires adding a call to the MarkAndSweep function with parameters specified in the config.
However, invoking this function directly from the proxy code could result in a large number of queries to the storage, which is particularly costly for Cloud Storages that charge per request. In my opinion, based on the current realization of retention, this cleanup should be external to the proxy code, except for the manifests.
Additionally, I have prepared a test build (a docker image compatible with amd64/arm64) from the latest main branch (commit 13fe08d87b766610775fc92ee022ed7c271bfcc4) with patches as described in https://github.com/zagaria/distribution/pull/1/files
If anyone wishes to test it, you can use the following command: docker pull ghcr.io/zagaria/distribution/registry:main-with-proxy-retention-patches-v1-test

I also ran into this issue using the current edge version (3.0.0-alpha.1) and dug into this a bit. I'll dump my findings:
Reviewing the logs and the scheduler state file (/var/lib/registry/scheduler-state.json), I came to the same conclusion as @dhedberg - that the cleanup scheduler keys are scoped to repositories (e.g. node@sha256:aabbcc, python@sha256:aabbcc ) whereas the underlying blob store uniquely identifies blobs with only the digest (e.g. sha256:aabbcc).
I took a stab at changing the scheduler key to just be the digest, however, the blob descriptor cache's keys are also scoped to repos: https://github.com/distribution/distribution/blob/main/registry/storage/cache/memory/memory.go#L23-L26. So we get the inverse problem. To illustrate this new flow:
> pull ref node@sha256:aabbcc scheduler entry for key sha256:aabbcc added
> pull ref python@sha256:aabbcc existing scheduler entry for key sha256:aabbcc is replaced ***
[ blob TTL expires ]
> scheduler removes blob sha256:aabbcc and cache entry for python@sha256:aabbcc
> pull node@sha256:aabbcc ... EOF error because cache entry for node@sha256:aabbcc still exists ***

So as a quick hack I also de-scoped the cache keys to always use the empty string as the repo to effectively make the digest as the key. (Side note: what is the reason to have repo-scoped cache instances besides having a request-scoped reference to the real cache? Seems like the repo-scoped cache just delegates to the global cache anyways. I see there was a large refactor in the past to add this interface: https://github.com/distribution/distribution/blob/main/registry/storage/cache/cache.go#L16).
Anyways, doing these two things gets us past these two cases, but I still saw some blips of the EOF error (albeit more rare than before) when putting some load on the proxy with a small TTL.
The changes I made so far are here: https://github.com/distribution/distribution/pull/4236/files
My next guess is that there are race conditions when updating the cache and the underlying storage driver, specifically an interleaving where a cache entry is added at the same time as the scheduler removes it. Guessing that blocks such as:

https://github.com/distribution/distribution/blob/main/registry/storage/cache/cachedblobdescriptorstore.go#L45-L62
https://github.com/distribution/distribution/blob/main/registry/proxy/proxyregistry.go#L72-L81

should be made into critical sections to ensure consistency b/w cache and store.
At this point I think a proper fix likely requires refactor w.r.t the repo-scoped blob cache and/or mutex on cache+store operations. Happy to receive guidance or have others take over.
TL;DR: For now I think the easiest workaround is still to just disable the blob descriptor cache by setting REGISTRY_STORAGE_CACHE_BLOBDESCRIPTOR=blah where blah is not inmemory or redis (see https://github.com/distribution/distribution/blob/main/registry/handlers/app.go#L282) at the cost of additional reads against the storage driver. This is probably not too bad for filesystem-based drivers.

This has also been a recurring issue for my team's deployment as well.
We just added the REGISTRY_STORAGE_CACHE_BLOBDESCRIPTOR=blah workaround that @yndai suggested.
I'm hoping a fix for this issue can get prioritized soon.

@andyxning Schema1 is disabled by default in Registry 2.7. You will need to enable it in config file:
compatibility:
  schema1:
    enabled: true

@manishtomar Thank you for suggesting this, i m facing this during image push.
Please suggest, Is this compatibility to be enabled in the deployment yaml of registry?

Schema1 will be deprecated in v3 release and v2 release is essentially in maintenance mode. If I were you I'd strongly advise you to rebuild your v1 images into v2.


Validation technique

What's the proper way to report errors back to the client?  If I'm using regexs - I don't think the user would like to see the regex in his error.

this library isn't aimed at making user-facing errors, if needed you can of course catch the error and redisplay it in whatever way you see fit


Error messages for Visual Studio

I downloaded this launcher to try it out but downloaded visual studio to make it an EXE file. When I started the project, I received a lot of errors and I could not convert it to an exe file.
The designer for this file could not be displayed because none of the classes in this file could be designed. The designer examined the following classes in the file: BrowserForm --- Unable to design base class 'System.Void'.
and I have more error messages.
Adding an updated exe file to this project may be useful to many users. Thank you.

check out the release page
https://github.com/jsoctocat/BDO-Launcher/releases
I update the executable every time there is a code change


fixes for community releases

fixed email address to bsig list
changed the URLs to original-repository.jboss.org since repository.jboss.org appears empty
removed link that was provided twice
added hidden variable to additional tests to assure the right URL building (see https://github.com/kiegroup/droolsjbpm-build-bootstrap/blob/master/script/release/kie-wbSmokeTestsMatrix.sh - the target has to be specified)

ad point 2) what do you mean by empty? I thought that jbossorg team recommended to not use that origin URL.

@mareknovotny yes - but after uploading the articts o Nexus you don't see the artifacts there:
https://repository.jboss.org/nexus/content/groups/kie-group/org/kie/business-central/ look for 7.29.0.Final - and now look at https://origin-repository.jboss.org/nexus/content/groups/kie-group/org/kie/business-central/ - I am a kind of tired about this - I don't know when exactly the dir tree is updated

ok, that is then explained, just as a side effect I would report it to jbossorg team in parallel.


Setting topic permission under load

Expected behavior
Do a REST call to admin interface for setting topic permissions like:
https://localhost:8081/admin/v2/persistent/tt1/testns1/t4%5Cdata/permissions/testrole

body:
{'testrole': ['consume', 'produce']}

Should work every time and return status 204.
Actual behavior
When this is done with heavy load setting up permission for many topics in parallel. Some will result in the below logged exception and status code 500.
Steps to reproduce
Set up tenant and namespace, then run the set permission for many different topics in that namespace in parallel. I use the same role and actions for all of them.
System configuration
pulsar: 2.2.0
This error was when running the broker in standalone mode in a docker container supplied on docker hub, with authorisation and authentication via client certs. Have not tested on our full pulsar setup yet.
Logs
12:27:31.613 [pulsar-web-53-28] INFO  org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [superUser] Successfully granted access for role testrole: [produce, consume] - topic persistent://tt1/testns1/t5\meta
12:27:31.587 [pulsar-web-53-30] ERROR org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [superUser] Failed to grant permissions for topic persistent://tt1/testns1/t0\data
org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /admin/policies/tt1/testns1
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[org.apache.pulsar-pulsar-zookeeper-2.2.0.jar:2.2.0]
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[org.apache.pulsar-pulsar-zookeeper-2.2.0.jar:2.2.0]
	at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1336) ~[org.apache.pulsar-pulsar-zookeeper-2.2.0.jar:2.2.0]
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient.access$3101(ZooKeeperClient.java:70) ~[org.apache.bookkeeper-bookkeeper-server-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient$21.call(ZooKeeperClient.java:1065) ~[org.apache.bookkeeper-bookkeeper-server-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient$21.call(ZooKeeperClient.java:1059) ~[org.apache.bookkeeper-bookkeeper-server-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.zookeeper.ZooWorker.syncCallWithRetries(ZooWorker.java:140) ~[org.apache.bookkeeper-bookkeeper-server-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient.setData(ZooKeeperClient.java:1059) ~[org.apache.bookkeeper-bookkeeper-server-4.7.2.jar:4.7.2]
	at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalGrantPermissionsOnTopic(PersistentTopicsBase.java:274) [org.apache.pulsar-pulsar-broker-2.2.0.jar:2.2.0]
	at org.apache.pulsar.broker.admin.v2.PersistentTopics.grantPermissionsOnTopic(PersistentTopics.java:107) [org.apache.pulsar-pulsar-broker-2.2.0.jar:2.2.0]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_171]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_171]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_171]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_171]
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:143) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [org.glassfish.jersey.core-jersey-common-2.25.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [org.glassfish.jersey.core-jersey-common-2.25.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [org.glassfish.jersey.core-jersey-common-2.25.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [org.glassfish.jersey.core-jersey-common-2.25.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [org.glassfish.jersey.core-jersey-common-2.25.jar:?]
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [org.glassfish.jersey.core-jersey-common-2.25.jar:?]
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [org.glassfish.jersey.core-jersey-server-2.25.jar:?]
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.25.jar:?]
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.25.jar:?]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.25.jar:?]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.25.jar:?]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [org.glassfish.jersey.containers-jersey-container-servlet-core-2.25.jar:?]
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) [org.eclipse.jetty-jetty-servlet-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1689) [org.eclipse.jetty-jetty-servlet-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:53) [org.apache.pulsar-pulsar-broker-2.2.0.jar:2.2.0]
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [org.eclipse.jetty-jetty-servlet-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.apache.pulsar.broker.web.AuthenticationFilter.doFilter(AuthenticationFilter.java:74) [org.apache.pulsar-pulsar-broker-common-2.2.0.jar:2.2.0]
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [org.eclipse.jetty-jetty-servlet-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) [org.eclipse.jetty-jetty-servlet-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) [org.eclipse.jetty-jetty-servlet-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.Server.handle(Server.java:524) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253) [org.eclipse.jetty-jetty-server-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [org.eclipse.jetty-jetty-io-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [org.eclipse.jetty-jetty-io-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202) [org.eclipse.jetty-jetty-io-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [org.eclipse.jetty-jetty-io-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [org.eclipse.jetty-jetty-io-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [org.eclipse.jetty-jetty-io-9.3.11.v20160721.jar:9.3.11.v20160721]
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-all-4.1.22.Final.jar:4.1.22.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
12:27:31.669 [pulsar-web-53-27] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [07/Nov/2018:12:27:31 +0000] "POST https://localhost:8081/admin/v2/persistent/tt1/testns1/t6%5Cdata/permissions/testrole HTTP/1.1" 500 6238 "-" "Python/3.7 aiohttp/3.4.2" 385
12:27:31.669 [pulsar-web-53-31] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [07/Nov/2018:12:27:31 +0000] "POST https://localhost:8081/admin/v2/persistent/tt1/testns1/t7%5Cmeta/permissions/testrole HTTP/1.1" 500 5652 "-" "Python/3.7 aiohttp/3.4.2" 402
12:27:31.668 [pulsar-web-53-29] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [07/Nov/2018:12:27:31 +0000] "POST https://localhost:8081/admin/v2/persistent/tt1/testns1/t8%5Cdata/permissions/testrole HTTP/1.1" 500 6238 "-" "Python/3.7 aiohttp/3.4.2" 371
12:27:31.643 [pulsar-web-53-28] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [07/Nov/2018:12:27:31 +0000] "POST https://localhost:8081/admin/v2/persistent/tt1/testns1/t5%5Cmeta/permissions/testrole HTTP/1.1" 204 0 "-" "Python/3.7 aiohttp/3.4.2" 173
12:27:31.642 [pulsar-web-53-24] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [07/Nov/2018:12:27:31 +0000] "POST https://localhost:8081/admin/v2/persistent/tt1/testns1/t4%5Cdata/permissions/testrole HTTP/1.1" 500 6238 "-" "Python/3.7 aiohttp/3.4.2" 341

Now verified same issue on a full Pulsar setup.

I think the problem might be related to concurrent updates on namespace policy. we can probably handle the bad version and retry, or put all the updates on same namespace in ordered executor.

Yes, the zookeeper bad version implies that pulsar have concurrent updates.
At least a retry is needed. Don't know if a queue would help, do you only
have one entry point? Otherwise, you would still have potential concurrent
updates.
Harald

Hi,
Just a comment is it not normal to give status service unavailable 503, for
a retry scenario. Status conflict implies that the client has sent a faulty
request, which is not the case, and sometimes it could be hard to
distinguish this conflict from an actual conflict, and a need to still
parse response text. Or is this zookeeper conflict handled inside pulsar
with a retry.
Harald
On Fri, 28 Dec 2018, 19:15 Sijie Guo<EMAIL_ADDRESS>wrote:

Closed #2952 https://github.com/apache/pulsar/issues/2952 via 1fa81ca
https://github.com/apache/pulsar/commit/1fa81ca54f688938c83a0be3c98bebc7589be668
.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/pulsar/issues/2952#event-2047561931, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADKqXSU-BC9QxLxFu9KkjppUfPNwcDxzks5u9l_dgaJpZM4YSXhe
.

@HaraldGustafsson At the time of bad version happening, it means there are concurrent updates to namespace policy. It is hard to determine what is the real conflict and what conflict it is. so returning CONFLICT status code sounds a right approach to me. IMO, Pulsar applications which are using pulsar admin api should be able to catch this exception and retries, because pulsar applications are better places to resolve the conflict. Does that align with what you are thinking?

I understand the reasoning, but from the api it is not obvious that when
you change permissions on different topics in the same namespace that you
could get this concurrent conflict (and should retry), if you don't know
that all topics permissions are stored in the namespace. At least I did not
know it before digging a bit deeper. When realising this I needed to queue
such operations on the client side and add retry since multiple workers
might attempt to do changes. So when setting up many topics, with several
configurations like retention, ttl, permissions, etc, they all need to be
queued, and get rtt delay. In my setup, for each admin request we get we
need to queue around 10 admin requests to pulsar, with retries even more.
Would it be possible to introduce a larger namespace policy update method
to change multiple things simultaneously, since in my case the conflict was
often due to changing several things in the ns policy.
On Fri, 28 Dec 2018, 19:49 Sijie Guo<EMAIL_ADDRESS>wrote:

@HaraldGustafsson https://github.com/HaraldGustafsson At the time of
bad version happening, it means there are concurrent updates to namespace
policy. It is hard to determine what is the real conflict and what conflict
it is. so returning CONFLICT status code sounds a right approach to me.
IMO, Pulsar applications which are using pulsar admin api should be able to
catch this exception and retries, because pulsar applications are better
places to resolve the conflict. Does that align with what you are thinking?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/pulsar/issues/2952#issuecomment-450408555, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADKqXf-Nb3u2i7X88mY23dJtaugbUCD_ks5u9metgaJpZM4YSXhe
.

@HaraldGustafsson I see your point now. Let me reopen this issue and think through a better solution on addressing such conflicts.

@HaraldGustafsson for my point of view, retry could not a good solution, due to retry also could fail again.  Maybe a queue/or only one request at one time is a better idea, what I know for kafka-manager  to change policy does like this .
@sijie what do you think about this ?

@foreversunyao -

Maybe a queue/or only one request at one time is a better idea, what I know for kafka-manager to change policy does like this .

I was thinking about same thing. that sgtm

@foreversunyao sgtm

Any news here? I have topics where there are ~10 different roles with different permissions. I manage them with pulsar terraform provider which basically does 10 POSTs in a row. In a new cluster with no load it was almost certain that already the second POST would fail. Also, the terraform provider uses internally pulsarctl which does not have any retry mechanism, although it could, at least while the internals of the server are not fixed?

I am in the exact same situation as @jaak-pruulmann-sympower - have you found any workaround? I just terraform apply repeatedly until all the changes go through, which under many circumstances takes a long time.
I'm interested in contributing a fix to this since it's a pretty frequent pain-point for me; I like the idea of handling the BadVersionException by retrying with a back-off. For consistency-oriented systems like ZooKeeper I'm not sure there's another workaround besides removing the point of contention entirely. This'd be my first contribution so any guidance would be appreciated


While adding patient, on keyboard when click on next button, keyboard gets closed instead of next text-box

Describe the bug
While adding patient, on keyboard when click on next button, keyboard gets closed instead of next text-box
Component
Demo app
To Reproduce
Steps to reproduce the behavior:

Go to 'Demo' app
Click on '+'
Add first name and click on next button from keyboard
Observe behavior

Expected behavior
keyboard should not get closed, it should go to next text-box or next filed in the form
Smartphone (please complete the following information):

Device: Lenovo tab
Android version: 10

hey is this issue still active if yes then can i get assigned to this?


Problem with default enum value

I have two proto files
alignment.proto:
...
syntax = "proto2";
package alignment;
enum VerticalAlignment {
CENTER_VERTICALLY = 0;
BOTTOM = 1;
TOP = 2;
}
...
widget.proto:
...
syntax = "proto2";
package widget;
import "alignment.proto";
message Text {
required string text = 1;
optional alignment.VerticalAlignment vertical_alignment = 2 [default = CENTER_VERTICALLY];
...
}
...
Result execution:
D:\path_to\generator-bin>nanopb_generator.exe widget.pb
Traceback (most recent call last):
File "", line 6, in
File "main.py", line 128, in
File "main__nanopb_generator.py", line 2029, in
File "main__nanopb_generator.py", line 1929, in main_cli
File "main__nanopb_generator.py", line 1866, in process_file
File "main__nanopb_generator.py", line 1523, in generate_header
File "main__nanopb_generator.py", line 1092, in fields_declaration
File "main__nanopb_generator.py", line 1225, in default_value
KeyError: 'alignment_HorizontalAlignment'
If I delete [default = CENTER_VERTICALLY], it's all right.
Help me understand the problem.

Hmm, looks like yet another problem in finding the dependent files.
Could you try this:
nanopb_generator.exe alignment.pb widget.pb
i.e. give both files at once to make sure it can find them.

The result is the same :(

Could you try this:
nanopb_generator.exe alignment.pb widget.pb

Yes, it works!
Thank you!


Q: Unsigned clamping for multiply accumulator

@cxd4 I am trying to understand how these 2 unsigned vector multiplication instructions clamp the result to unsigned  16 bits.
The doc describes the accumulator bits from 31 to 16 are passed into the clamp function.  With only reading in 16 bits, it can't be possible to check if the value is overflowing 0xFFFF or underflowing 0 without reading proper bits from other parts of the accumulator.
I have already attempted to check bits over the boundary but it doesn't always work.
Thanks.

The unsigned clamp in VMULU and VMACU extracts the upper 32 bits of each 48-bit element in the accumulator.  Also, so does the signed clamp, although the documentation in my C implementation of this is extremely obfuscated due to the vectorization-friendly for loops I wrote.

Note that because VMULU re-sets the accumulator (rather than accumulates to its existing value from previous multiplication instructions) that the odds of exceeding the boundaries of 0 or +65535 as a result of the instruction's computation are much, much lower.
VMUDL also uses an unsigned clamp on the hardware's circuitry, but again, this is highly irrelevant mathematically to an emulator's implementation of the instruction in software because of the fact that VMUDL always stores the upper 16-bits of the unsigned 32-bit product to the low 16 bits of each accumulator element, thereby utterly eliminating any logical chance of overflow or underflow.  Therefore, while VMUDL technically "does" an unsigned clamp on the hardware, it's never useful or able to change the final mathematical result, so only VMACU needs a complete unsigned clamp function.  VMULU could call a complete unsigned clamp function, but only in corner case combinations of VS * VT is overflow or underflow possible.

Thanks for the feed back here.


Notify a user if an option is defined without the 'default' prefix and onOptionChanged is not defined

Issue Type
Suggestion
Description
We have a lot of issues related to the fact the the "default" prefix is not specified and onOptionChanged is not handled. I suggest throwing a warning message to the browser console, so users can manually identify the cause of the issue
Steps to Reproduce

Environment Details

devextreme version: 18.1
devextreme-react version: 18.1
react version: 16.2

We will consider this enhancement in the future, but I can't provide any timeframe.


[RFC] Introducing DeclBuffer

This is a follow-up of https://github.com/apache/tvm/pull/9727 and RFC#63. Currently buffer can be implicitly declared and then used. The implicit behavior can be error prone and makes analysis more difficult. This RFC introduces DeclBuffer, a new IR construct as an explicit statement for buffer declaration.
Rendered version: https://github.com/vinx13/tvm-rfcs/blob/decl-buffer/rfcs/0070-introducing-decl-buffer.md
cc @tqchen @Lunderberg @junrushao1994 @csullivan @mbs-octoml @jroesch @areusch @wrongtest @Hzfengsy

Thanks a lot!  I think then we can handle buffer related issues in customized passes with more explicit and robust way.
I have one question on tir script, for certain algorithms in DL workloads, users may want to write non-stir formed script like
x = T.allocate((), "int32", "")
x[()] = 0
while x[()] < 128:
    x[()] = x[()] + 1
    # ...

Could the parser support still write things like that (though underlying IR structure changed) instead of
x_data = T.allocate((), "int32", "")
x = T.decl_buffer(data=x_data,)
x[()] = 0
# ...

@wrongtest Thanks for bringing up this. There are a few options for the behavior in TVM script, I'm open to discussion.

A1: The original behavior before https://github.com/apache/tvm/pull/9727:
T.allocate returns a Var, which can be later used in T.load / T.store.

A2: Current behavior:
T.allocate returns a buffer (Var is implicitly created inside). The buffer can be accessed via subscripts, which will be translated to BufferLoad / BufferStore.

A3: Potential behavior for this RFC:
T.allocate returns a Var, use T.decl_buffer to create buffer and access it (because T.load / T.store are deprecated). This follows the translation to TIR nodes closely.

A4: Potential behavior for this RFC (the one @wrongtest is proposing):
If there are no buffer aliases (in most of the cases), it is tempting to avoid boilerplate code of T.decl_buffer.
T.allocate returns a buffer, where buffer->data is implicitly created and allocated inside. The buffer can be accessed via subscripts. Subsequent buffer aliases can be created by referring to buffer->data. This requires DeclBuffer to be created implicitly, it is okay to have some discrepancy between parser side and the TIR nodes.

@wrongtest I've thought about the option A3 vs A4. From the parsing / translation from TVM script to TIR, it is acceptable to have T.allocate translated to Allocate + DeclBuffer two nodes. But it will be tricky for TVMScriptPrinter. We will need to find both Allocate and DeclBuffer nodes and then print T.allocate, and these two nodes do not have to be parent/child of each other. I'm not sure if this behavior, which breaks 1-to-1 mapping between TVM script and TIR, is desirable. Alternatively, we can add an option to T.allocate, such as def allocate(..., return_buffer: bool). What do you think?

A gentle ping. Would be great to have a status checkin and let us move to make this happen

Seems we all agree that introducing DeclBuffer is helpful. The only unresolved question is how shall the TVMScript be updated as @wrongtest mentioned. As discussed above, we have the options:

B1: In TVMScript, T.allocate and T.decl_buffer strictly map to the corresponding TIR nodes. To allocate and declare a buffer (in lowered TIR), there will be two separate steps:

data = T.allocate(physical_shape)
buffer = T.decl_buffer(data=data, shape=physical_shape)

B2: provide some syntax sugar for T.allocate (adding a returns_buffer option), which is translated to AllocateNode + DeclBufferNode in TIR.

buffer = T.allocate(physical_shape, returns_buffer=True)

When printing them from TIR to TVMScript, we still print in unsugared form in B1.
Note that in B2, it is not feasible to make T.allocate always return the created buffer, because

there might be still need to directly use the buffer var in lower level TIR
when printing the TIR to TVMScript, AllocateNode and DeclBufferNode do not always appear in the same place, it is difficult (and not preferred) to map two TIR nodes into one TVMScript statement

would love to know what you think @wrongtest @Hzfengsy

We always need the original syntax to enable bidirectional property. It does not hurt though to enable sugars that allows us to have a combination on top. How about we start with the original syntax and then discuss sugar as a separate topic

reuse T.alloc_buffer seems good，as long as there is no ambiguity for parser impl ：）

@areusch @Hzfengsy I've updated the RFC. It is ready for another look


Fix Switch cursor issue (fixes #1530)

When switch is on, cursor is no longer pointer. It is set back to default. fixed it by removing the overriding cursor: auto

LGTM. Thanks @narendrashetty. Waiting for a second ACK before merging.

fixes issue #1530.

Good catch, @narendrashetty! LGTM, merging.


[question] Some of the example datapacks use forge tags, how does it differ from fabric?

Namely, i want to deny spawns in a modded dimension, but doubt forge:dimension_loaded works in 1.18.2 fabric

See example in fabric branch.
You don't have to fill condition block, so you can remove condition or fabric:load_conditions block if you don't need.
For more information, see the examples.
Forge provides only these conditions. There isn't something related to dimension.
Fabric provides these.

thanks very much!


Infinite loop "Found"/"Not Found" messages

From this setup :
$ gopass team-secu/provider/url/
team-secu/provider/url/
├── README
└── secret (yaml)

I can access the yaml  file using the extension :
$ gopass team-secu/provider/url/secret.yaml
foo:
  bar:      abc
  bar2:   xyz

But the auto-complete returns it without the extension and gopass starts to loop between "Found"/"Not Found" entry messages :
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
Found exact match in 'team-secu/provider/url/secret'
Entry 'team-secu/provider/url/secret' not found. Starting search...
^C

gopass: 1.3.2
os: ubuntu 16.04

Yes, sorry. This is a silly bug. Found it. Will fix it ASAP :slightly_frowning_face:

Thanks for the fix.
I believe the auto-complete returning the file it without the extension should also be fixed within this issue.
But I can create a new one if you prefer.
$ gopass team-secu/provider/url/sec▒
[HIT TAB]
$ gopass team-secu/provider/url/secret ▒

I realize may be your fix allow the usage of both with and without the extension...
If so my previous comment doesn't make sense anymore.

We should remove the special case handling for .yaml files in the tree output.


"attribute missing" error when using TOML config file

When I try to run a diff using a TOML config file to compare two tables within one postgres database, I receive this error:
(datadiff) ➜  ~ data-diff --conf ~/config_files/datadiff.toml --run diff_test -v -k activity_id -w "event_timestamp < '2022-10-10'" -d
Traceback (most recent call last):
  File "/Users/leoebfolsom/virtualenvs/datadiff/bin/data-diff", line 8, in <module>
    sys.exit(main())
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/data_diff/__main__.py", line 194, in main
    kw = apply_config_from_file(conf, run, kw)
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/data_diff/config.py", line 77, in apply_config_from_file
    return _apply_config(toml.load(f), run_name, kw)
  File "/Users/leoebfolsom/virtualenvs/datadiff/lib/python3.9/site-packages/data_diff/config.py", line 42, in _apply_config
    raise ConfigParseError(f"Running 'run.{run_name}': Connection #{index} is missing attribute '{attr}'.")
data_diff.config.ConfigParseError: Running 'run.diff_test': Connection #1 is missing attribute 'database'.

Contents of TOML file:
# POSTGRES
# Specify the connection params to database 1.
[database.diff_test]
driver = "postgresql"
database = "diff_test"
user = "leoebfolsom"
password = "$PW_POSTGRES"

# Specify the default run params
[run.default]
# update_column = "timestamp"
verbose = true

[run.diff_test]
# Source 1 ("left")
database1 = "diff_test"
table1 = "org_activity_stream"

# Source 2 ("right")
database2 = "diff_test"
table2 = "org_activity_stream_backup"

I realized this problem is related to the args object being created as an empty dict ({}) because the run_args dictionary doesn't have a key of "1" (or "2").
I mocked up a superficial fix here, which gets my diff to run locally, but it doesn't pass CI/CD, indicating that there's a deeper problem.
Environment

installed data-diff from github master at https://github.com/datafold/data-diff/commit/b9ce7ed1b7e4cd2017eef5ada58006a074b4fa4b
macOS Monterey 12.6
Python 3.9.6

I don't think it's a bug. The correct syntax is
1.database = "diff_test"
1.table = "org_activity_stream"

# Source 2 ("right")
2.database = "diff_test"
2.table = "org_activity_stream_backup"

But we definitely need a better error.

That did it, thanks!


NullPointerException OutputConfiguration.getSurfaces()

Using camerax_version = '1.0.0-alpha06'
After
CameraCaptureSession.onConfigured() mState=RELEASING
CameraCaptureSession.onReady() RELEASING
CameraCaptureSession.onClosed()

I'm getting
E/AndroidRuntime: FATAL EXCEPTION: LegacyCameraCallback
    Process: com.levelup.synk, PID: 20926
    java.lang.NullPointerException: Attempt to invoke virtual method 'java.util.List android.hardware.camera2.params.OutputConfiguration.getSurfaces()' on a null object reference
        at android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onCaptureErrorLocked(CameraDeviceImpl.java:2276)
        at android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onDeviceError(CameraDeviceImpl.java:1914)
        at android.hardware.camera2.legacy.CameraDeviceUserShim$CameraCallbackThread$CallbackHandler.handleMessage(CameraDeviceUserShim.java:308)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:201)
        at android.os.HandlerThread.run(HandlerThread.java:65)

This happens very randomly but it definitely happens on exiting the camera fragment.

Sample has been updated to alpha06, please reference the latest version and reopen if you find any issues with the sample as-is before modifying any code.

It's crashing on alpha06 as well.

Did you ever resolve this?


Command in guide Deploying Docker to Azure App Service no longer working

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
Deploying Docker to Azure App Service
What part(s) of the article would you like to see updated?
When running the action in the section Creating the workflow there is an issue with the Step "Deploy to Azure Web App".
When running the follwing error message is returned:
Run azure/webapps-deploy@05ac4e98bfa0f856e6669624239291c73ca27698
node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module '@actions/core'
Require stack:
- /home/runner/work/_actions/azure/webapps-deploy/05ac4e98bfa0f856e6669624239291c73ca27698/lib/main.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/home/runner/work/_actions/azure/webapps-deploy/05ac4e98bfa0f856e6669624239291c73ca27698/lib/main.js:19:27)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/runner/work/_actions/azure/webapps-deploy/05ac4e98bfa0f856e6669624239291c73ca27698/lib/main.js'
  ]
}

Additional information
No response

@samuel-coderhead Thanks for opening an issue and submitting a PR! I'll get this triaged for review! :zap:

dccv


How to use c++11 syntax check instead

The auto syntax check done by one of the plugins does not include c+11 features.

You might want to check Google, e.g.
https://stackoverflow.com/questions/2977174/is-there-a-c11-syntax-file-for-vim


Added version field to manifest.json

Can you change the version to 0.0.5 please as that's the version number that is current in the github releases then I'll merge the pull. Thanks

ok, done

Thanks @brewston

@gazoodle No worries - how does this now find its way into a HACs update ?

@gazoodle No worries - how does this now find its way into a HACs update ?

I'm not sure. It might be that the change of version number will be sufficient - I had noticed that previously it gave the version as a 32bit hex number, so perhaps this will be sufficient.
I need to take some time to find out how to control the update mechanism. Perhaps I have to release a new version (and obviously update the version number in the manifest :-) ).
This is my first custom component so I'm learning at the same time!


General: Add more paths to PATH

Description
This PR adds more paths to $PATH.

/opt: Standalone programs are installed here.
/usr/local/bin: Local programs are installed here.
/cygdrive/c/Windows/*: Windows cmd.exe and powershell.exe programs.

On second thought, this new PATH change might not be a good idea as the user might have something non-standard on their machine. I think we should go back to the old pacman fix.

On second thought, this new PATH change might not be a good idea as the user might have something non-standard on their machine.

Wait, what do you mean by this? Since when Neofetch interacts with outside programs, all (or most, as I don't know about where Cygwin keeps wmic) of them, one way or another are inside /usr/bin, /usr/sbin, /bin, or /sbin. Well, maybe count /usr/local.

The user might have programs we use in Neofetch installed outside of /usr/sbin, /usr/bin, /sbin:/bin and etc. This would cause Neofetch to break since it can't find the programs.


Debian package and system library dep. build

You can download Debian package as an artifact of CI build here https://github.com/MapsHD/HDMapping/actions/runs/10748497744/artifacts/1903927070


[VL] Fix High Precision Rounding

What changes were proposed in this pull request?

Gluten Result: Round(0.19324999999999998, 4) = 0.1933
Vanilla Spark Result: Round(0.19324999999999998, 4) = 0.1932

Currently gluten round has std::nextafter to offset the small error coming from multiply result. However, for high precision numbers the nextafter leads to a bigger delta than the expected value.
This eventually causes wrong results for high precision rounding (starts occurring beyond 15 decimals).
Currently we use double in the best case for intermediate calculations. Now moved to long double for holding the intermediate results more accurately.
In the future, we can use boost multi-precision to support arbitrary precision at runtime, which will more closely match the Java BigDecimal and give better performance.

Also fixes: #5366.
How was this patch tested?

Unit Tests
Local Spark shell

After the fix:
Gluten Result: Round(0.19324999999999998, 4) = 0.1932
Vanilla Spark Result: Round(0.19324999999999998, 4) = 0.1932
The following query was also tested for the attached issue: (This can be flaky since we still use double which will lose some precision)
select round(avg(cast(col as double)), 4) as topic_2 from VALUES (0.188), (0.194),  (0.194), (0.194), (0.194), (0.194), (0.194), (0.194) AS tab(col);

cc @PHILO-HE @rui-mo

cc @zhztheplayer could you please help triggering the workflow. Thanks!

What changes were proposed in this pull request?

Gluten Result: Round(0.19324999999999998, 4) = 0.1933
Vanilla Spark Result: Round(0.19324999999999998, 4) = 0.1932

Currently gluten round has std::nextafter to offset the small error coming from multiply result. However, for high precision numbers the nextafter leads to a bigger delta than the expected value.
This eventually causes wrong results for high precision rounding (starts occurring beyond 15 decimals).
Currently we use double in the best case for intermediate calculations. Now moved to long double for holding the intermediate results more accurately.
In the future, we can use boost multi-precision to support arbitrary precision at runtime, which will more closely match the Java BigDecimal and give better performance.

Also fixes: #5366.
How was this patch tested?

Unit Tests
Local Spark shell

After the fix: Gluten Result: Round(0.19324999999999998, 4) = 0.1932 Vanilla Spark Result: Round(0.19324999999999998, 4) = 0.1932
The following query was also tested for the attached issue: (This can be flaky since we still use double which will lose some precision) select round(avg(cast(col as double)), 4) as topic_2 from VALUES (0.188), (0.194), (0.194), (0.194), (0.194), (0.194), (0.194), (0.194) AS tab(col);

Excellent description!

@kecookier can you confirm if the PR can fix your issue? Looks good to me

@kecookier If there is any other relevant issue please feel free to comment.

Thanks, I'll test it.

@rui-mo I found an issue, but I can't tell whether the issue is from round.
SELECT round(3/32*CAST(5.92 AS DECIMAL(20,2)),2);

gluten returns 0.56, but vanilla returns 0.55.
The SQLs below have the same result between gluten and vanilla, which is 0.5549999999999999 and 0.55:
SELECT 3/32*CAST(5.92 AS DECIMAL(20,2));
SELECT round(0.5549999999999999, 2);

@rui-mo I found an issue, but I can't tell whether the issue is from round.
SELECT round(3/32*CAST(5.92 AS DECIMAL(20,2)),2);

gluten returns 0.56, but vanilla returns 0.55. The SQLs below have the same results between gluten and vanilla, which are 0.5549999999999999 and 0.55:
SELECT 3/32*CAST(5.92 AS DECIMAL(20,2));
SELECT round(0.5549999999999999, 2);

I find that the argument passed to round is 0.555. I will check it.

@rui-mo I found an issue, but I can't tell whether the issue is from round.
SELECT round(3/32*CAST(5.92 AS DECIMAL(20,2)),2);

gluten returns 0.56, but vanilla returns 0.55. The SQLs below have the same results between gluten and vanilla, which are 0.5549999999999999 and 0.55:
SELECT 3/32*CAST(5.92 AS DECIMAL(20,2));
SELECT round(0.5549999999999999, 2);

I find that the argument passed to round is 0.555. I will check it.

I think there will always be an issue with decimals because of how c++ represents floating point, there is a negligible loss when representing high precision with double. Java's bigdecimal does not pose this and can scale upto whatever the memory will support with dynamic precision. As long as we continue to use double in c++ as an alternative to bigdecimal, we will see issues with arithmetic. One of the long term fixes could be using mpfr/boost throughout the code but it might require significant effort. Thanks!

@rui-mo I found an issue, but I can't tell whether the issue is from round.
SELECT round(3/32*CAST(5.92 AS DECIMAL(20,2)),2);

gluten returns 0.56, but vanilla returns 0.55. The SQLs below have the same results between gluten and vanilla, which are 0.5549999999999999 and 0.55:
SELECT 3/32*CAST(5.92 AS DECIMAL(20,2));
SELECT round(0.5549999999999999, 2);

I find that the argument passed to round is 0.555. I will check it.

I think there will always be an issue with decimals because of how c++ represents floating point, there is a negligible loss when representing high precision using double. Java's bigdecimal does not pose this and can scale upto whatever the memory will support with dynamic precision. As long as we continue to use double in c++ as an alternative to bigdecimal, we will see issues with arithmetic. One of the long term fixes could be using mpfr/boost throughout the code but it might require significant effort. Thanks!

Thanks for your reply!
I have a minimal example that doesn't produce the right answer:
SELECT round(cast(0.5549999999999999 as double), 2);

gluten returns 0.56, vanilla returns 0.55.
Besides, I run the following SQL and I get 0.1933:
SELECT round(cast(0.19324999999999998 AS DOUBLE), 4);

@jiangjiangtian Would you like to open an issue in Gluten with a repro? Perhaps also add to the issue tracker https://github.com/apache/incubator-gluten/issues/4652.

@jiangjiangtian could you please also send your hardware/os specs, I'll double check the change and try to fix it, thanks for testing this!

@jiangjiangtian could you please also send your hardware/os specs, I'll double check the change and try to fix it, thanks for testing this!

OS is RHEL 8.1(4.18.0-147).
CPU is Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz.
gcc is g++ (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11).
Thanks.


Peek Definition not working

I installed today two VS Code extensions from VS Code extensions button:

Modern Fortran 0.4.5
FORTRAN IntelliSense 0.6.0

I got the Syntax highlighting from the Modern Fortran but the Peek Definition from Fortran IntelliSense is not working.
What is happening?
My system:
Windows 10 x64
VS Code 1.38.1

Did you install the underlying language server? This package relies on the language server for all of its functionality.

Yes...it is installed! But not working yet...
What more I have to do?

In the Fortran IntelliSense extension settings, did you properly set the path to the fortls executable?

Thank you @michaelkonecny ! Now it's working great!

I'm using this extension in VS Code.
Is possible to use it in Visual Studio Enterprise 2019? How can i configure it in this case?

Not yet, AFAIK.
I have created an extension for Visual Studio 2017: https://github.com/michaelkonecny/vs-fortran-ls-client
but it needs a lot of improvement
(unfortunately, my work setup didn't allow me to install a fresh enough version of Visual Studio which would support these extensions, so I couldn't work on it)
I'm planning on updating it for VS2019, but I won't be able to do it until I get a bit of free time at work, so probably some time around the beginning of next year...

Thank you a lot @michaelkonecny !


[AUTOPATCHER-CORE] Upgrade ntopng to 6.0 Upgrade for 3.0

[AUTOPATCHER-CORE] Upgrade ntopng to 6.0 Upgrade for 3.0
Upgrade pipeline run -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=600124&view=results
buddy build -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=600125&view=results

Abandoning upgrade.  CVE's were patched instead


Fix create repo bugs

Errors on repository creation were out of scope of the handling function, so Gitea showed the 500 page (because err was nil).
When an error occured, templateID was set to int default 0 and the JS was showing template units even though no template was chosen.

Codecov Report

Merging #9606 into master will decrease coverage by <.01%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master    #9606      +/-   ##
==========================================
- Coverage   42.17%   42.17%   -0.01%
==========================================
  Files         578      578
  Lines       76065    76066       +1
==========================================
- Hits        32083    32081       -2
- Misses      40001    40002       +1
- Partials     3981     3983       +2

Impacted Files
Coverage Δ

routers/repo/repo.go
29.22% <66.66%> (+0.17%)
:arrow_up:

models/unit.go
62.16% <0%> (-5.41%)
:arrow_down:

modules/repository/repo.go
33.55% <0%> (-1.98%)
:arrow_down:

modules/migrations/migrate.go
24.21% <0%> (-1.58%)
:arrow_down:

modules/log/event.go
64.61% <0%> (-1.03%)
:arrow_down:

routers/repo/view.go
38.59% <0%> (-0.88%)
:arrow_down:

modules/migrations/gitea.go
9.03% <0%> (-0.64%)
:arrow_down:

models/repo.go
48.08% <0%> (+0.1%)
:arrow_up:

models/error.go
33.82% <0%> (+1.1%)
:arrow_up:

services/pull/temp_repo.go
34.18% <0%> (+2.56%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 03d59bc...5f805c0. Read the comment docs.

@jolheiser can you backport this to 1.10.3?

No, this bug doesn't exist in 1.10.x

OK :)


Create an issue in Homebrew/brew about MANPATH

failing build https://circleci.com/gh/lynncyrin/base-image/68
failing line

eval export 'HOMEBREW_PREFIX="/root/.linuxbrew"' export 'HOMEBREW_CELLAR="/root/.linuxbrew/Cellar"' export 'HOMEBREW_REPOSITORY="/root/.linuxbrew/Homebrew"' export 'PATH="/root/.linuxbrew/bin:/root/.linuxbrew/sbin:$PATH"' export 'MANPATH="/root/.linuxbrew/share/man:$MANPATH"' export 'INFOPATH="/root/.linuxbrew/share/info:$INFOPATH"'
/bin/bash: MANPATH: unbound variable

specifically

MANPATH: unbound variable

this issue went away when I made this change https://github.com/lynncyrin/base-image/pull/43/commits/3e204adb658049c0ccec4383251a8e2d9a9756db


Build error: java.util.zip.ZipException:

On adding
maven { url "https://jitpack.io" } in root build.gradle
and
compile 'com.github.chrisbanes:PhotoView:1.3.0' in  module build.gradle
Sync success and found multiple jar files in  ...\app\build\intermediates\exploded-aar folder (in Screen shot attached).

Even though the build succeeds, while trying to run the project in debug mode, I get the following error in gradle console:

Error:Execution failed for task ':app:packageAllDebugClassesForMultiDex'.
java.util.zip.ZipException: duplicate entry: uk/co/senab/photoview/PhotoViewAttacher$1.class

Looks like you might be importing the library twice? Can you verify that this is not the case?

Found issue
We are using image picker library that is internally using photoview library.


FLINK-13106][doc-zh]Translate Parallel Execution page into Chinese

What is the purpose of the change
This pull request completes the Chinese translation of parallel executionpage from flink official doucuments "parallel.md"
Brief change log
This pull request completes the Chinese translation of parallel executionpage from flink official doucuments "parallel.md"
Verifying this change
This change is to add a new translated document.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): ( no )
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

CI report for commit ac5a550b8dfe8ebde32b295ec2b6990fd2c580ed: FAILURE Travis

@klion26 Hi,I have updated all suggestion,pls check,thanks.

@klion26  Hi,I hava updated all again,please review,thanks.

@klion26  maybe this time  is ok.

@wuchong Hi,is there any problem in your mind?if not,can you merge it? thanks.

@wuchong @klion26  Hi，I have updated all suggetions in this commit.please check,thanks.


Allow the retrieving of an entire entity by ID

I'd like to be able to clone entire entities, including components that the cloning code may not be aware of. The purpose is to be able to instantiate entity prototypes that contain mod-defined components relating to mod-defined behavior.
@philpax mentioned adding an entity::get_all_components guest API call that could do this, and this definitely works for my use case, especially since I can look up non-mod components before they are spawned into the world and trigger spawn systems.
This is pretty high-priority because this design pattern will be very common in Flowerpot.

Dupe of #90, but I'll close both once it's done


Bad inferences with error TS2365

TypeScript Version:  2.2.1
Code
class Foo {
  _elements: string[];

  constructor(input: string) {
    this._elements = input.split("");
  }

  get top(): string {
    return (this._elements.length > 0) ?
      this._elements[this._elements.length - 1] :
      "";
  }

  strip(): number {
    if (this.top == "$") {
      this._elements.pop(); // $
      let count = 0;
      while (this.top != "^") { // XXX TS2365
        ++count;
        this._elements.pop();
      }
      this._elements.pop(); // ^
      return count;
    }

    return 0;
  }
}

let f = new Foo("ba^z$");
console.log(f.strip());

Expected behavior:
Clean compilation, with no errors or warnings.
Actual behavior:

foo.ts(18,14): error TS2365: Operator '!=' cannot be applied to types '"$"' and '"^"'.

This may be related to issue #12794, or maybe not, but even if so, it looks like there's a lot of noise and casted aspersions wrt the testcases in that issue, rather than engineering.

It is. See https://github.com/Microsoft/TypeScript/issues/9998#issuecomment-235963457

Duplicate #13904 / we don't keep open issues from :ghost: s, too spooky


Qwen/Qwen1.5-14B-Chat not supported yet!

I want to load the Qwen model with :
from unsloth import FastLanguageModel
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen1.5-14B-Chat")
model = FastLanguageModel.from_pretrained( model_path)
but I got this error:
--> 116             raise NotImplementedError(
117                 f"Unsloth: {model_name} not supported yet!\n"
118                 "Make an issue to https://github.com/unslothai/unsloth!",
NotImplementedError: Unsloth: Qwen/Qwen1.5-14B-Chat not supported yet!
Make an issue to https://github.com/unslothai/unsloth!

@mouhsineguet You can try lllamfying it via llama-factory also try searching on HF models for qwen 14b llama and there might be some llamafied versions

@danielhanchen Thanks for the reply! actually I finetuned Qwen 14B with LoRa, I and I want to load the adapter with unsolth FastLanguageModel so I can convert it to GGUF-Q4 format using model.save_pretrained_gguf("model", tokenizer, quantization_method = "q4_k_m"); so please if you have any suggestions?

So sorry @mouhsineguet I did not reply!! (This slipped past me!)
I managed to make Qwen 1.5 work if you're interested (with GGUF support)
If you're on a local machine, please update Unsloth via
pip uninstall unsloth -y
pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git

Colab and Kaggle is fine (just restart it)