[jaeger-v2] add elasticsearch & opensearch e2e integration test

Which problem is this PR solving?

part of #5254

Description of the changes

Utilizing existing StorageIntegration to test the jaeger-v2 OTel Collector and gRPC storage backend with the provided config file at cmd/jaeger/config-elasticsearch.yaml.

How was this change tested?

Start a elasticsearch or opensearch docker instance.
Run STORAGE=elasticsearch SPAN_STORAGE_TYPE=elasticsearch make jaeger-v2-storage-integration-test

Checklist

[x] I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
[x] I have signed all commits
[x] I have added unit tests for the new functionality
[x] I have run lint and test steps successfully

for jaeger: make lint test
for jaeger-ui: yarn lint and yarn test

Currently, the Traces are stored correctly in the database but the test is not able to read data from it.
ElasticSearch logs
{"took":768,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1105,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"jaeger-main-jaeger-span-write","_type":"_doc","_id":"NBW1x44Bj8WH9-LVviap","_score":1.0,"_source":{"traceID":"0000000000000512","spanID":"0000000000000005","operationName":"query23-operation","references":[],"startTime":1712681191639875,"startTimeMillis":1712681191639,"duration":1,"tags":[{"key":"sameplacetag1","type":"string","value":"same*"}],"logs":[{"timestamp":1712681191639875,"fields":null},{"timestamp":1712681191639875,"fields":null}],"process":{"serviceName":"query23-service","tags":[]}}},{"_index":"jaeger-main-jaeger-span-write","_type":"_doc","_id":"NRW1x44Bj8WH9-LVvib_","_score":1.0,"_source":{"traceID":"0000000000000a12","spanID":"0000000000000004","operationName":"","references":[],"startTime":1712681191639875,"startTimeMillis":1712681191639,"duration":2,"tags":[{"key":"sameplacetag1","type":"string","value":"sameplacevalue1"}],"logs":[],"process":{"serviceName":"query24-service","tags":[]}}},{"_index":"jaeger-main-jaeger-span-write","_type":"_doc","_id":"NhW1x44Bj8WH9-LVvyYJ","_score":1.0,"_source":{"traceID":"0000000000001212","spanID":"0000000000000004","operationName":"","references":[],"startTime":1712681191639875,"startTimeMillis":1712681191639,"duration":2,"tags":[{"key":"sameplacetag1","type":"string","value":"sameplacevalue2"}],"logs":[],"process":{"serviceName":"query24-service","tags":[]}}},{"_index":"jaeger-main-jaeger-span-write","_type":"_doc","_id":"NxW1x44Bj8WH9-LVvyYi","_score":1.0,"_source":{"traceID":"0000000000000001","spanID":"0000000000000002","operationName":"some-operation","references":[],"startTime":1712681191639875,"startTimeMillis":1712681191639,"duration":7,"tags":[{"key":"sameplacetag1","type":"string","value":"sameplacevalue"},{"key":"sameplacetag2","type":"int64","value":"123"},{"key":"sameplacetag4","type":"bool","value":"true"},{"key":"sameplacetag3","type":"float64","value":"72.5"}]

Error in test
github.com/jaegertracing/jaeger/cmd/query/app.(*GRPCHandler).GetTrace
        /home/pushkar/jaeger/cmd/query/app/grpc_handler.go:105
github.com/jaegertracing/jaeger/proto-gen/api_v2._QueryService_GetTrace_Handler
        /home/pushkar/jaeger/proto-gen/api_v2/query.pb.go:1009
google.golang.org/grpc.(*Server).processStreamingRPC
        /home/pushkar/jaeger/vendor/google.golang.org/grpc/server.go:1663
google.golang.org/grpc.(*Server).handleStream
        /home/pushkar/jaeger/vendor/google.golang.org/grpc/server.go:1784
google.golang.org/grpc.(*Server).serveStreams.func2.1
        /home/pushkar/jaeger/vendor/google.golang.org/grpc/server.go:1019
    integration.go:276: trace not found
    integration.go:143: Waiting for storage backend to update documents, iteration 2 out of 100
2024-04-10T17:30:57.541+0530    error   app/grpc_handler.go:105 trace not found {"kind": "extension", "name": "jaeger_query", "error": "trace not found"}

Do you have a plan how to solve it?

The ES client has a TraceLogger, which we do not initialize today. I suggest implementing some flag somewhere that would initialize this logger to be able to see which queries ES Storage is making to the ES Backend (this would be a separate PR).

do we need something like this?

{"level":"info","ts":1712921641.4783256,"caller":"zapgrpc/zapgrpc.go:128","msg":"HTTP/1.1 200 OK\r\nContent-Type: application/json; charset=UTF-8\r\nX-Elastic-Product: Elasticsearch\r\n\r\n{\"took\":12,\"errors\":false,\"items\":[{\"index\":{\"_index\":\"jaeger-main-jaeger-service-write\",\"_type\":\"_doc\",\"_id\":\"5c78c55a6b1b86a1\",\"_version\":1,\"result\":\"created\",\"_shards\":{\"total\":2,\"successful\":1,\"failed\":0},\"_seq_no\":3,\"_primary_term\":1,\"status\":201}}]}\n"}
{"level":"info","ts":1712921641.4787142,"caller":"zapgrpc/zapgrpc.go:128","msg":"POST /_bulk HTTP/1.1\r\nHost: localhost:9200\r\nUser-Agent: elastic/6.2.37 (linux-amd64)\r\nContent-Length: 459\r\nAccept: application/json\r\nContent-Type: application/x-ndjson\r\nAccept-Encoding: gzip\r\n\r\n{\"index\":{\"_index\":\"jaeger-main-jaeger-span-write\"}}\n{\"traceID\":\"0000000000000011\",\"spanID\":\"0000000000000006\",\"operationName\":\"example-operation-3\",\"references\":[],\"startTime\":1712853991639875,\"startTimeMillis\":1712853991639,\"duration\":100,\"tags\":[{\"key\":\"span.kind\",\"type\":\"string\",\"value\":\"server\"}],\"logs\":[{\"timestamp\":1712853991639875,\"fields\":null},{\"timestamp\":1712853991639875,\"fields\":null}],\"process\":{\"serviceName\":\"example-service-1\",\"tags\":[]}}\n\n"}

    integration.go:276: trace not found

{"level":"info","ts":1712921641.5270474,"caller":"zapgrpc/zapgrpc.go:128","msg":"GET /_msearch?rest_total_hits_as_int=true HTTP/1.1\r\nHost: localhost:9200\r\nUser-Agent: elastic/6.2.37 (linux-amd64)\r\nContent-Length: 403\r\nAccept: application/json\r\nContent-Type: application/json\r\nAccept-Encoding: gzip\r\n\r\n{\"ignore_unavailable\":true,\"index\":\"jaeger-main-jaeger-span-read\"}\n{\"query\":{\"bool\":{\"must\":[{\"bool\":{\"should\":[{\"term\":{\"traceID\":{\"boost\":2,\"value\":\"0000000000000011\"}}},{\"term\":{\"traceID\":\"11\"}}]}},{\"range\":{\"startTimeMillis\":{\"from\":136035241526,\"include_lower\":true,\"include_upper\":true,\"to\":1713008041526}}}]}},\"search_after\":[136118041526656],\"size\":10000,\"sort\":[{\"startTime\":{\"order\":\"asc\"}}]}\n\n"}
{"level":"info","ts":1712921641.5347877,"caller":"zapgrpc/zapgrpc.go:128","msg":"HTTP/1.1 200 OK\r\nContent-Type: application/json; charset=UTF-8\r\nX-Elastic-Product: Elasticsearch\r\n\r\n{\"took\":0,\"responses\":[{\"took\":0,\"timed_out\":false,\"_shards\":{\"total\":0,\"successful\":0,\"skipped\":0,\"failed\":0},\"hits\":{\"total\":0,\"max_score\":0.0,\"hits\":[]},\"status\":200}]}\n"}
2024-04-12T17:04:01.534+0530	error	app/grpc_handler.go:105	trace not found	{"kind": "extension", "name": "jaeger_query", "error": "trace not found"}

here, in log, we can see the client is trying to perform get query in jaeger-main-jaeger-span-read index. but actually, it doesn't exists in elasticsearch.

health status index                            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .geoip_databases                 0z0sUHE3TaedeUallyGkMA   1   0         35            0     32.4mb         32.4mb
yellow open   jaeger-main-jaeger-span-write    2AEUrJFkSEKX9SWzgv8wKg   1   1         63            0      152kb          152kb
yellow open   jaeger-main-jaeger-service-write m8pO7MKwSXuOgtv9R5Tvlw   1   1          7            0     13.8kb         13.8kb```

do we need something like this?

Yes! Can you make a PR to support an option to turn this on?

here, in log, we can see the client is trying to perform get query in jaeger-main-jaeger-span-read index. but actually, it doesn't exists in elasticsearch.

Yep, this seems to be the root cause. Why doesn't it exist? When using read/write name it sounds like the aliasing solution with ILM enabled, maybe it's incorrectly enabled for the tests?

Yes! Can you make a PR to support an option to turn this on?

Yes, I can add it. But I see; we can already enable this by setting log_level to debug. But if we still need it, we can create another option in config.

Yep, this seems to be the root cause. Why doesn't it exist? When using read/write name it sounds like the aliasing solution with ILM enabled, maybe it's incorrectly enabled for the tests?

will take look into it.

If it's already can be enabled via existing flags then don't change it

one more interesting finding:
if we enable useilm for v1 integration, we can see similar results (jaeger-read index didn't exists & test fails).

got some progress :
--- ❌ FAIL: TestESStorage (202.01s)
    --- ✅ PASS: TestESStorage/GetServices (2.71s)
    --- ❌ FAIL: TestESStorage/GetOperations (101.32s)
    --- ✅ PASS: TestESStorage/GetTrace (3.27s)
        --- ✅ PASS: TestESStorage/GetTrace/NotFound_error (0.01s)
    --- ✅ PASS: TestESStorage/GetLargeSpans (89.15s)
    --- ❌ FAIL: TestESStorage/FindTraces (4.44s)
        --- ❌ FAIL: TestESStorage/FindTraces/Tag_escaped_operator_+_Operation_name_+_max_Duration (1.53s)
        --- ❌ FAIL: TestESStorage/FindTraces/Tag_wildcard_regex (0.03s)
        --- ❌ FAIL: TestESStorage/FindTraces/Tags_in_one_spot_-_Tags (0.02s)
        --- ❌ FAIL: TestESStorage/FindTraces/Tags_in_one_spot_-_Logs (0.03s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_in_one_spot_-_Process (1.64s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_in_different_spots (0.06s)
        --- ✅ PASS: TestESStorage/FindTraces/Trace_spans_over_multiple_indices (0.05s)
        --- ✅ PASS: TestESStorage/FindTraces/Operation_name (0.03s)
        --- ✅ PASS: TestESStorage/FindTraces/Operation_name_+_max_Duration (0.05s)
        --- ✅ PASS: TestESStorage/FindTraces/Operation_name_+_Duration_range (0.04s)
        --- ✅ PASS: TestESStorage/FindTraces/Duration_range (0.02s)
        --- ✅ PASS: TestESStorage/FindTraces/max_Duration (0.04s)
        --- ✅ PASS: TestESStorage/FindTraces/default (0.02s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_+_Operation_name (0.05s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_+_Operation_name_+_max_Duration (0.04s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_+_Operation_name_+_Duration_range (0.03s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_+_Duration_range (0.06s)
        --- ✅ PASS: TestESStorage/FindTraces/Tags_+_max_Duration (0.12s)
        --- ✅ PASS: TestESStorage/FindTraces/Multi-spot_Tags_+_Operation_name (0.03s)
        --- ✅ PASS: TestESStorage/FindTraces/Multi-spot_Tags_+_Operation_name_+_max_Duration (0.05s)
        --- ✅ PASS: TestESStorage/FindTraces/Multi-spot_Tags_+_Operation_name_+_Duration_range (0.04s)
        --- ✅ PASS: TestESStorage/FindTraces/Multi-spot_Tags_+_Duration_range (0.03s)
        --- ✅ PASS: TestESStorage/FindTraces/Multi-spot_Tags_+_max_Duration (0.03s)
        --- ✅ PASS: TestESStorage/FindTraces/Multiple_Traces (0.02s)

@james-ryans getting same issue here also

--- ❌ FAIL: TestESStorage (202.01s)
--- ✅ PASS: TestESStorage/GetServices (2.71s)
--- ❌ FAIL: TestESStorage/GetOperations (101.32s)
--- ✅ PASS: TestESStorage/GetTrace (3.27s)
--- ✅ PASS: TestESStorage/GetLargeSpans (89.15s)
--- ✅ FAIL: TestESStorage/FindTraces (4.44s)

still not able to fix this.
when i run services and operation test indiviually, they both pass. but together, the one which is kept second always fail.

when i run services and operation test indiviually, they both pass. but together, the one which is kept second always fail.

that sounds like you either have data pollution across tests, or the opposite - a dependency of one test on another's input data (although in that case tests would not succeed individually).

@james-ryans which line are you referring to?

the format of this error message is strange - components inside OTEL collector usually get Loggers that are annotated with labels indicating the component name (e.g. "otlpexporter")

fixed the issue, by running the operation test in a seperate test:
like this
func TestESStorage(t *testing.T) {
	integration.SkipUnlessEnv(t, "elasticsearch", "opensearch")

	s := &ESStorageIntegration{}
	s.initializeES(t)
	s.Fixtures = integration.LoadAndParseQueryTestCases(t, "fixtures/queries_es.json")
	s.ConfigFile = "cmd/jaeger/config-elasticsearch.yaml"
	s.SkipBinaryAttrs = true
	s.e2eInitialize(t)
	t.Cleanup(func() {
		s.e2eCleanUp(t)
	})
	s.RunSpanStoreTests(t)
}

func TestESStorage2(t *testing.T) {
	integration.SkipUnlessEnv(t, "elasticsearch", "opensearch")

	s := &ESStorageIntegration{}
	s.initializeES(t)
	s.Fixtures = integration.LoadAndParseQueryTestCases(t, "fixtures/queries_es.json")
	s.ConfigFile = "cmd/jaeger/config-elasticsearch.yaml"
	s.SkipBinaryAttrs = true
	s.e2eInitialize(t)
	t.Cleanup(func() {
		s.e2eCleanUp(t)
	})
	s.RunOperationTest(t)
}

I will refactor it and add this to the cleanup.

I come to bring a good news; I've found the root cause. It is sourced from ES storage ServiceOperationStorage.Write(), which checks if the service name was stored before. But because it is stored in memory cache, I haven't found a way to fix this.
https://github.com/jaegertracing/jaeger/blob/5434892718d574dc1576e65fc4d35154d98fcf3e/plugin/storage/es/spanstore/service_operation.go#L73-L77
My argument is also strengthened by findings from the Elasticsearch logs. In the ES spanstore implementation, it writes the service first and then the span.
https://github.com/jaegertracing/jaeger/blob/5434892718d574dc1576e65fc4d35154d98fcf3e/plugin/storage/es/spanstore/writer.go#L154-L157
But because ES storage checked that the service was already stored, the ES storage skipped it. This is why in the GetOperations test the creation of jaeger-main-jaeger-service- index appeared after the jaeger-main-jaeger-span-, it was created because the test was trying to retrieve the operations from the service index.
// GetServices
{"@timestamp":"2024-04-19T16:21:24.058Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-service], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:21:24.071Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-span], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:21:26.032Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-/QDZv-bbDRxeBXzkcfhNu_w] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:21:26.033Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-/Ke2nkUDrReKW0AFsTJ9CEA] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}

// GetOperations
{"@timestamp":"2024-04-19T16:21:26.229Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-span], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:21:29.043Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-service], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:08.462Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-/SBikBdplTCGUe4aceY9m5Q] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:08.465Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-/yxVhrUghQiSX-8UufwU1mg] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}

// GetTrace
{"@timestamp":"2024-04-19T16:23:08.845Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-service], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:08.850Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-span], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:10.787Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-/RVxFKQcXRcWGY0N1jrMmkA] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:10.787Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-/QltgMtEXQd6PQX9NkI1CTA] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}

// GetLargeSpans
{"@timestamp":"2024-04-19T16:23:11.032Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-span], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:27.285Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-/7SK_IhEWQbiG2-GhmVSmvQ] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}

// FindTraces
{"@timestamp":"2024-04-19T16:23:27.427Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-service], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:27.431Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-] creating index, cause [auto(bulk api)], templates [jaeger-main-jaeger-span], shards [5]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:29.940Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-span-/ms3jNLOtScO0f2GQfe__UQ] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-04-19T16:23:29.940Z", "log.level": "INFO", "message":"[jaeger-main-jaeger-service-/3U8IdyZuTYuphhlYGT2L1A] deleting index", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[d43e5fa14561][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.metadata.MetadataDeleteIndexService","elasticsearch.cluster.uuid":"lnq124DFRIOv2kONkSdzgQ","elasticsearch.node.id":"m2t6zhtbSpqLRtoo-x7ZiA","elasticsearch.node.name":"d43e5fa14561","elasticsearch.cluster.name":"docker-cluster"}

But because ES storage checked that the service was already stored, the ES storage skipped it.

So we discovered another edge case where purging the storage from outside of the storage implementation causes issues with the data, somewhat similar to the Badger use case. I suggest we follow the same path as with Badger - implement factory.Purge function that will both delete the indices and clean-up the implementation's cache, and invoke Purge via an HTTP endpoint in the collector served from a test-only extension.


Enable the node to use different modes according to a configuration

Enable relay mode to get rid of relay.ts

done #45


[ALBEF] model refactoring

Summary:
This PR refactors ALBEFModel for different downstream tasks.

ALBEFModel: the core model that outputs unimodal and multimodal embeddings (momentum embeddings too).
(VQA can use this class because it only needs mlm loss)

ALBEFModelWithSimilarity: outputs unimodal and multimodal embeddings (no momentum embeddings), negative image-text multimodal embeddings (for ITM loss), and similarity outputs (for ITC loss).
(Retrieval can use this class because it needs ITM and ITC losses)

Test plan:
Passed unit tests in test/models/test_albef.py

Codecov Report

Merging #128 (05e999e) into main (358c2b9) will decrease coverage by 0.40%.
The diff coverage is 82.25%.

@@            Coverage Diff             @@
##             main     #128      +/-   ##
==========================================
- Coverage   90.66%   90.26%   -0.41%
==========================================
  Files          45       46       +1
  Lines        2336     2414      +78
==========================================
+ Hits         2118     2179      +61
- Misses        218      235      +17

Impacted Files
Coverage Δ

torchmultimodal/models/albef.py
90.69% <82.25%> (-9.31%)
:arrow_down:

...hmultimodal/modules/encoders/mdetr_text_encoder.py
92.06% <0.00%> (ø)

torchmultimodal/models/flava/flava_model.py
92.96% <0.00%> (+0.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 358c2b9...05e999e. Read the comment docs.

Re @YosuaMichael's point about inference mode, an alternative option to the if/else approach you describe is to create a separate bare-bones class containing only the non-momentum models, e.g. ALBEFForInference. But then there are a couple follow-up questions:
(1) What does the forward look like? It will  depend on the task a bit more, but perhaps we could just return image, text, and multimodal embeddings. Then let the specific task handle the remaining pieces (similarity for retrieval and text decoder for VQA).
(2) In a training and eval loop, do we then use a separate class for each of these? That could be clunky (we'd have to do lots of param copying between the two).

Re @YosuaMichael's point about inference mode, an alternative option to the if/else approach you describe is to create a separate bare-bones class containing only the non-momentum models, e.g. ALBEFForInference. But then there are a couple follow-up questions:
(1) What does the forward look like? It will depend on the task a bit more, but perhaps we could just return image, text, and multimodal embeddings. Then let the specific task handle the remaining pieces (similarity for retrieval and text decoder for VQA). (2) In a training and eval loop, do we then use a separate class for each of these? That could be clunky (we'd have to do lots of param copying between the two).

I agree, I think this is also a good approach. From @ebsmothers idea, here is what I think:
class AlbefBaseModel(nn.Module):
    def __init__(
        self,
        vision_encoder: nn.Module,
        text_encoder: nn.Module,
        multimodal_encoder: nn.Module,
    ) -> None:
        super().__init__()
        self.vision_encoder = vision_encoder
        self.text_encoder = text_encoder
        self.multimodal_encoder = multimodal_encoder

    def forward(
        self,
        image: Tensor,
        text: Tensor,
        text_atts: Tensor,
    ) -> ALBEFOutput:
        image_embeds = self.vision_encoder(image)
        text_embeds = self.text_encoder(text, text_atts)
        multimodal_embeddings = self.multimodal_encoder(
            image_embeds, text_embeds, text_atts
        )

        return ALBEFBaseOutput(
            image_embeddings=image_embeds,
            text_embeddings=text_embeds,
            multimodal_embeddings=multimodal_embeddings,
        )

class AlbefModel(nn.Module):
    def __init__(
        self,
        base_model: nn.Module,
        momentum: float
    ) -> None:
        super().__init__()
        self.model = base_model
        self.model_m = copy.deepcopy(base_model).eval()
        self.momentum = momentum

  def forward(
        self,
        image: Tensor,
        text: Tensor,
        text_atts: Tensor,
    ) -> ALBEFOutput:
        momentum_update(self.model, self.model_m, self.momentum)
        base_output = self.model(image, text, text_atts)
        momentum_output = self.model_m(image, text, text_atts)
        return ALBEFOutput(
            base_output=base_output,
            momentum_output=momentum_output,
        )


Zsh shut down mac

Is zsh can shut down macbook pro? Without any additional plugins, just randomly turn off

If you're asking for a way to shut down macOS you can use the shutdown command, which works anywhere, not just zsh. See http://osxdaily.com/2017/08/13/shutdown-mac-command-line/


Flexible Element Search

Fix https://github.com/openstreetmap/iD/issues/7627
Accept a variety of formats in Feature Search when specifying an element by type and id.
For example: n123, n 123, node123, node 123 or even https://www.openstreetmap.org/node/123/history#map=19/43.66140/-70.25415

I ran npm test and got errors that didn't look related to my changes.

We can add some tests for all the cases we want to match/don't match too so that this improved functionality never gets accidentally broken 👍
Don't worry about that in this PR though, I'd be happy to do that kind of maintenance after merge

I wanted to add tests, but didn't know where to put them, or how best to extract the function for testing. Thanks for handling it.
Positive Cases:
n1, N1, n01, n11, w1, r1, node1, way1, relation1, n 1, node 1, ?n1, n?1, n1?, ?n?1, ?n1?, n?1?, ?n?1?, 1?n?1?n,
www.example.com/node/1/history?lat=100&lon=100
Negative Cases:
nn1, n1n, 1n1, 1n, n??1, n0, n, 1, [empty string]

@blackboxlogic Thanks for working on this! Welcome to the dev side of iD 🔨
@SilentSpike Thanks a bunch for taking an interest. The code looks reasonable to me at first glance, but I'll defer to you. Feel free to merge and tweak this as needed. (Tests are always stellar too, since you mentioned them.)
Oh, and if anyone's feeling inspired it might be a good idea to break out some of this functionality into separate functions instead of having everything in features().

@blackboxlogic I've tested this and it's good to go, will merge as is now 👍
If you'd like to learn how to add the tests yourself, let me know and I an point you in the right directions 😃 If not, I don't mind taking care of it.


Get rid of Chatterino.pro in favor of CMakeFiles.txt

QMake will be removed from Qt so we might as well start simplifying the build process and get rid of Chatterino.pro. This could also remove oddities like "need to have library x in this specific folder". Also allows using things like vcpkg to get libraries.

🦀 qmake is gone 🦀


JBTM-2749 Upgrade jboss-transaction-spi to 7.4.0.Final

https://issues.jboss.org/browse/JBTM-2749

out of date


error: failed to run custom build command for chromiumoxide_cdp v0.3.0

Hello folks,
How i can fix this issue?
Regards :)

that's probably a dupe of #10


Examples of REST Server no longer working

The examples given here: https://github.com/cosmos/sdk-application-tutorial/blob/master/tutorial/run-rest.md
no longer work. If one adjusts the curl commands as posted here:
https://github.com/cosmos/sdk-application-tutorial/issues/76
then the results are unsigned jsons and no working buy-name is broadcasted. It would be nice to adjust the run-rest.md so that a working example is given.

@patrickwieth thanks for the catch! Would you mind pushing a PR correct this?

I don't know exactly how you intend to have this example. You have changed quite a lot here. Before it was done with https and the password was sent to the server and the server signed the message. Now you get an unsigned tx from the REST server and have to sign it with nscli tx sign and then you have to broadcast it. This is the way I got it working, but I don't know if you intend it to be like that?
Especially broadcasting with nscli tx broadcast does not really make sense, there should be a REST endpoint for that right? So if you clarify how the intended usage is, I can write the readme, but currently how I got it working does not seem optimal.

@patrickwieth due to security concerns about having the gaiacli rest-server managing keys we removed that functionality and haven't updated the example here.

@patrickwieth due to security concerns about having the gaiacli rest-server managing keys we removed that functionality and haven't updated the example here.
The new flow requires a singer on the client (secure key management FTW) which might be hard to show in a concise example. Maybe just show transaction generation?

@jackzampolin I had the same issue for buying via REST, could you please update the tutorial with latest example of how to use a singer?

my PR fixes this

Will close issue, since PR regarding this issue, has been merged


Soundness issue in MvccRwLock

Hi there, we (Rust group @sslab-gatech) are scanning crates on crates.io for potential soundness bugs. We noticed that MvccRwLock implements Send and Sync for all types:
https://github.com/pipedown/noise/blob/5a582a7d0af1d7038646bfacaced40c3c19d4421/src/index.rs#L418-L419
However, this should probably have tighter bounds on its Send and Sync traits, otherwise its possible to create data-races from safe rust code by using non-Sync types like Cell across threads or sending non-Send types across like Rc. Here's a little proof-of-concept using Rc.
#![forbid(unsafe_code)]

use noise_search::index::MvccRwLock;

use std::rc::Rc;

fn main() {
    let rc = Rc::new(42);

    let lock = MvccRwLock::new(rc.clone());
    std::thread::spawn(move || {
        let smuggled_rc = lock.read();

        println!("Thread: {:p}", *smuggled_rc);
        for _ in 0..100_000_000 {
            (*smuggled_rc).clone();
        }
    });

    println!("Main:   {:p}", rc);
    for _ in 0..100_000_000 {
        rc.clone();
    }
}

This outputs:
Main:   0x561539bdcd40
Thread: 0x561539bdcd40

Terminated with signal 4 (SIGILL)

It seems like this class also potentially allows for aliasing violations, in this case maybe it would be better to mark the methods as unsafe and maybe not expose the class outside the crate?

Heads up: this issue has been included in the RustSec advisory database. It will be surfaced by tools such as cargo-audit or cargo-deny from now on.
Once a fix is released to crates.io, please open a pull request to update the advisory with the patched version, or file an issue on the advisory database repository.


Using jdk.incubator.vector leads to degraded performance

[ ] auto-unrolling  could be improved (see EncoderSimd.sumAbs); partial manual unrolling makes code faster
[ ] manually unrolled loops unexpectedly gets worse from 2x to 4x
[ ] vectorised version is slower than non-vectorized (see EncoderSimd.updateGeGeneric); reason: conversion not inlined and happens inside call to jdk.incubator.vector.AbstractVector::convert

@PaulSandoz Hi! I've filed an issue to OpenJDK about problems with jdk.incubator.vector, but perhaps it is stuck in moderation process. That is why I'm trying to reach you directly.
Long story short - using vector API makes calculations slower, not faster. In one case I've succeeded by manually unrolling (one-line) loop. In the other case convert(VectorOperators.F2I, 0) works unbelievably slow.
Could provide more details on reproduction of the problems, if you are interested.
Thanks in advance.

@eustas the quickest approach is to email panama-dev (if you don't join the email it will require moderation, but CC me, and i can reply quicker), presenting code examples showing the problem, preferably using JMH.
You may be encountering a regression that was recently identified, see https://mail.openjdk.java.net/pipermail/panama-dev/2021-March/012355.html, in other cases it may be the method has yet to be optimized.


circleci build fails

mocha tests fail with the following error: Cannot find module 'babel-core/register'

Did you run npm install first?

On 19 Sep 2017, at 07:24, Biswajit Dutta Baruah<EMAIL_ADDRESS>wrote:
mocha tests fail with the following error: Cannot find module 'babel-core/register'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

npm install works locally and the tests pass locally.
But still this error on circle ci. Tried all of these:

Change cache name in order to do a fresh install of the npm dependencies
Built with ssh. Deleted the node_modules on the build box and a reinstall
Saved exact versions of the babel packages and re-install

None of these strategies worked.

can you paste the link to the full circleci log where things are failing?

Will this web summary do? Its a public repo.
Didn't see any logs under artifacts to download.

that link helps, it's exactly what i asked for.
this error is interesting.
it seems that when doing npm install, the file package.json changes and that files is used as a hash key for circleci cache, and because of this the node_modules does not get saved.
i see this line
npm notice created a lockfile as package-lock.json. You should commit this file.
this used to work but it seems node/npm changed something again.
i'll look into it (early next week) and make sure it works.
thanks for the report

package-lock.json was released with npm v5 & node v8
Am still following node LTS, which is still on npm v3. So package-lock.json is neither generated nor committed.
So changed the cicrcle config file to use a specific node version. And the tests passed.
Don't know if node latest os preferred or better over LTS!!

reopening this to remember to "lock" the node version (not that important to be on the last version since node is only used for tests)


feat(automerge): Add reference support

copilot:all

Current dependencies on/for this PR:

main

PR #4786   👈

This stack of pull requests is managed by Graphite.


Update Get ACLs Section and add parameters list

A number of customers have mentioned that they don't understand what the '-f my-file-system' parameter should be. It would be nice to mention in the Get ACLs example section that the -file-system -f parameter should be the container they are trying to get back or access.

@AndresPad : Thanks for your contribution! The author(s) have been notified to review your proposed change.


run_job_for_testing should validate its data argument

As ...
Austin - Network Automation Engineer
I want ...
run_job_for_testing to validate that the data argument conforms to the schema of the job
So that ...
my unit tests put out clear and predictable error messages.
I know this is done when...

I use run_job_for_testing as part of my unit tests
I omit some of the fields
run_job_for_testing tells me that I have omitted some of the fields and explicitly fails the unit test

Optional - Feature groups this request pertains to.

[ ] Automation
[ ] Circuits
[ ] DCIM
[ ] IPAM
[ ] Misc (including Data Sources)
[ ] Organization
[X] Plugins (and other Extensibility)
[ ] Security (Secrets, etc)
[ ] Image Management
[ ] UI/UX
[ ] Documentation
[ ] Other (not directly a platform feature)

Database Changes
None
External Dependencies
None

@Kircheneer What are the errors you get when data isn't valid.

I can detail some of the error states, sure.
Imagine a job that takes in a site ObjectVar and a description TextVar. A very easy check would be whether all keys (incl. the non-required ones) are present in the data argument. Now what could also make sense is to check if the type matches, i.e. the ObjectVar is a nautobot.dcim.models.Site instance and the TextVar is indeed a string, but I'm not sure that is worth the effort. As to the actual error this throws - that depends on the job code. Since jobs can reasonably expect that all the keys will be in the dictionary with the correct types (or None) I have not seen these cases handled in job code so far.


ui: Waivers - Compliance node detail page displays waived controls

User Story
As an Automate User, I will use this work when I want to see a waived control on a particular node.
Problem Statement
We need a way to show which controls were waived at the node level, plus show the reason, expiration date or status if applicable.
Definition of Done

A user can navigate to a compliance node detail screen and see which controls were waived.
A user can see see the reason why a control was waived
A user can see see the expiration of a waived control if applicable
A user can see the status of the waived control if it was set to run.
A user can filter by waived controls

Mock Ups
Showing a control on the node that had run, but was waived.

Row showing a control that did not run, but was skipped.

this work involves:

ensure the status logic recognizes "waived" status and applies it to controls with waived status
show waiver information in the control results section, as shown in designs above
(note: only need the expires and reason details, not the node status ones - the api will not return node status stats)

load data by using the load_compliance_reports command from your studio dev env
sample api response for a waived control:


roles: ceph-rgw: Enable the ceph-radosgw target

If the ceph-radosgw target is not enabled, then enabling the
ceph-radosgw@ service has no effect since nothing will pull
it on the next reboot. As such, we need to ensure that the
target is enabled.
Signed-off-by: Markos Chandras<EMAIL_ADDRESS>

@leseb hello. Do you have any thoughts on that? This fixes in the openSUSE Ceph deployments with OpenStack-Ansible.

@hwoarang sorry for the late reply, sounds good to me.

I guess I was just wondering if we needed a backport or not, but we do.


How to use my own data set model

hi, i have some questions in predcit speaker embedding. i used "train_speaker_embeddings.py" train a model with chinese dataset. But the checkpoint result had not hyperparams.yaml file like huggingface open source model. that make i use "from speechbrain.pretrained import SpeakerRecognition" have some error . so how could i get this file or create it?

now i use huggingface hyperparams.yaml and change out_n_neurons. but the checkpoint result have not mean_var_norm_emb.ckpt. so how to solve this question

now i use huggingface hyperparams.yaml and change out_n_neurons. but the checkpoint result have not mean_var_norm_emb.ckpt. so how to solve this question


handle disappearing folders (eg. samba shares) in listdir

Last pull request had some garbage commits I thought I had removed. Let's try again.
The daemon will check for the existence of folders listed in listdir, but if they disappear after the config validation, then the listdir plugin will throw an OSError and abort the task (same for non-daemon runs). I haven't really tested my changes but it should work.
Log:
2015-01-12 12:00 ERROR    task          Move tvshows    BUG: Unhandled error in plugin configure_series: [Errno 112] Host is down: '/media/network/seagate/staging/tv'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 444, in __run_plugin
    return method(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/event.py", line 22, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/plugin_configure_series.py", line 69, in on_task_start
    result = method(task, input_config)
  File "/usr/local/lib/python2.7/dist-packages/flexget/event.py", line 22, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/listdir.py", line 34, in on_task_input
    for filepath in folder.listdir():
  File "/usr/local/lib/python2.7/dist-packages/path.py", line 506, in listdir
    for child in map(self._always_unicode, os.listdir(self))
OSError: [Errno 112] Host is down: '/media/network/seagate/staging/tv'
2015-01-12 12:00 WARNING  task          Move tvshows    Aborting task (plugin: configure_series)

@paranoidi Okay, I have changed the log message and actually tested it. It logged the following:
2015-01-13 15:47 INFO     scheduler                     Starting scheduler
2015-01-13 15:47 INFO     manager                       There is a FlexGet process already running for this config, sending execution there.
2015-01-13 15:47 ERROR    listdir       move            Path \\RASPBMC\devices\media could not be accessed: The network path was not found
2015-01-13 15:47 WARNING  configure_series move            Input listdir did not return anything
2015-01-13 15:47 INFO     configure_series move            Did not get any series to generate series configuration

Should I just go ahead and merge it or do you have any other objections?

Seems good to me.


Upgrade node (nodejs) to at least 14

I have encountered this issue https://github.com/alexbosworth/balanceofsatoshis/issues/85 with Balance of Satoshi and author commented that I should upgrade nodejs.
Raspi runs with verson:
node -v
v12.16.3

Last LTS is 14.15, newest is 15.1 (https://nodejs.org/en/download/releases/)
Is it possible / good idea to upgrade?

go for it: https://github.com/rootzoll/raspiblitz/pull/1748#issue-517123777

PR merged. v1.6.2 will ship with 14.15 - closing issue.

Thanks <3


[Fix-7512][UI] dependent node ui dislocation

Purpose of the pull request
close #7512
Brief change log

Codecov Report

Merging #7716 (afe9768) into dev (0bc4f9b) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev    #7716      +/-   ##
============================================
- Coverage     41.10%   41.09%   -0.01%
+ Complexity     3677     3676       -1
============================================
  Files           642      642
  Lines         26665    26665
  Branches       2993     2993
============================================
- Hits          10960    10959       -1
  Misses        14656    14656
- Partials       1049     1050       +1

Impacted Files
Coverage Δ

...er/master/dispatch/host/assign/RandomSelector.java
77.77% <0.00%> (-5.56%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bc4f9b...afe9768. Read the comment docs.

@songjianet @Wangyizhi1 please review.thx.


Feat/spherebody

Description
WIP - SphereBody creation method. Currently it creates the body on the server side, but client side is not aware of the shape of the body, so we may need to discuss how to wrap this up.
Issue linked
Please mention the issue number or describe the problem this pull request addresses.
Checklist

[x] I have tested my changes locally.
[ ] I have added necessary documentation or updated existing documentation.
[x] I have followed the coding style guidelines of this project.
[ ] I have added appropriate unit tests.
[ ] I have reviewed my changes before submitting this pull request.
[ ] I have linked the issue or issues that are solved to the PR if any.
[x] I have assigned this PR to myself.
[ ] I have added the minimum version decorator to any new backend method implemented.
[ ] I have made sure that the title of my PR follows Conventional commits style (e.g. feat: extrude circle to cylinder)

@jonahrb This creates a spherebody on backend as we discussed last time. But client side is not aware of its shape for evaluation reasons. Do this need to wait that work you are working on? Let me know if I miss anything here?

@jonahrb This creates a spherebody on backend as we discussed last time. But client side is not aware of its shape for evaluation reasons. Do this need to wait that work you are working on? Let me know if I miss anything here?

@umutsoysalansys totally separate concerns, you're fine to merge this before #655

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 89.96%. Comparing base (21196a2) to head (d4f25ff).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1035      +/-   ##
==========================================
+ Coverage   89.94%   89.96%   +0.01%
==========================================
  Files          93       93
  Lines        6794     6805      +11
==========================================
+ Hits         6111     6122      +11
  Misses        683      683

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


50 initial classes then 50 increments of 1 class

Have you try (using your terminology) B50 50 steps?
It's quite challenging and models follow there a different regimes, I'd be interested to know your results.

Hey, thanks for your attention.
For the DER(w/o pruning) based on modified resnet-32, the average accuracy is 64.31 and the final accuracy is 56.11.

Hey, thanks for your attention.
We  tried our method on CIFAR100-B50 50 steps and the results are as follows
For the DER(w/o pruning) based on modified ResNet-32, the average accuracy is 64.31 and the final accuracy is 56.11.
For the DER(w/o pruning) based on ResNet-18, the average accuracy is 72.39 and the final accuracy is 64.84.

Thanks for the info, and with pruning?

The sparsity loss coefficient needs to be tuned to make the average number of parameters comparable with other methods. If there is an update of the result, I will update it in time here.

So your sparsity loss is tuned for every setting?

@arthurdouillard

Thanks for your advice. The class ordering is stated in the dataset.py for each dataset. Specifically, we adopt the same 100 classes as PODNet for ImageNet100.

The results w/ pruning
resnet18 w/ pruning
avg #paras:  5.0625M   avg acc:71.22
modified resnet32 w/pruning:
avg #paras: 0.856M      avg acc: 64.83

Sparsity loss coefficient:
Yes. We tune the sparsity loss coefficient for each dataset to make the number of parameters comparable. Concrete configurations can refer to our supplementary material.

Thanks for your answers!


Calculates observed bulk density

A new function to calculate the observed bulk density of the planet.
This is done by taking the total mass (= atm + int) and comparing it to the observed radius. Since we don't have optical depth stored, this is taken to be observed at the 1 mbar level and is a grey quantity.
This function is called by PROTEUS at runtime, so we can track how this quantity changes as the planet cools.

Thanks!


"Can't identify the notebook name", jupyterlab version 3.0.14, Windows

Hi all,
Discovered this wonderful utility a couple days ago, but a team mate using Windows is having trouble.
--jupyter-lab version: 3.0.14
...\ipynbname\__init__.py in name()
      80      if path:
      81          return path.stem
--->  82      raise FileNotFoundError(FILE_ERROR.format('name'))
      83
      84

FileNotFoundError: Can't identify the notebook name

Opening new issue as none of the opened ones mention Windows.

This works on jupyterlab and Windows for me, version 3.6.3. If you still have problems please raise another issue.


Logging refactoring

https://github.com/MassTransit/MassTransit/issues/1439

Like I said, this will be after 5.5 is released, likely a 6.0 version update.

I totally agree with you, I had free time and I did it :)

@NooNameR why would I be forced to use Microsoft.Extensions.Logging? They even have no way to instantiate the logger without using a container and it will only be fixed in .NET Core 3.

@alexeyzimarev you are not forced. It’s only an abstraction, same as MT did before. It also includes providers for well known loggers, such as NLog and Serilog. Only Log4Net will be dropped.

@alexeyzimarev you are not forced. It’s only an abstraction, same as MT did before

Ok, but what problem are you solving? You change one abstraction (internal) in favour of another (external) and drop support for log4net, so what is the benefit?

@alexeyzimarev,

Microsoft introduces a lot of general abstraction and most of developers start using them as a base.
I do not see any sense to support own abstraction. Microsoft’s is really nice and flexible. And there are so many providers to use with it.

I’m not a decision maker. I’m only the person who wants make product better. As I know @phatboyg has plans to migrate to Microsoft abstractions

Fine by me, I just want to understand the reason. Not long ago, I wanted to create a simple app and I had to use a library that decided to use Microsoft logging abstraction. I might be a complete nub but I was unable to find a way to just use Serilog. The Serilog adapter for Microsoft logging can only be added to the logger factory or web host builder. It forced me to create a container, add the logging there and use it just for the sake of adding some logs! The internet is full of complaints.
For me, it is enough to look at this: https://github.com/aspnet/Extensions/blob/master/src/Logging/Logging/src/Microsoft.Extensions.Logging.csproj
All of it will become dependencies for the MassTransit core library, which now depends on Newtonsoft.Json, NewId and GreenPipes, where two of those dependencies are part of the MassTransit organisation.
If I'd be to decide what logging to use, I'd just go for LibLog. It has no dependencies itself and it is also not a dependency, it is a source package. And it supports whatever logging library that exists without any additional configuration.

You can look inside my commit, there are couple ways to create logging factory. It isn’t a big issue. Almost every logger provider has support for Microsoft’s

You can look inside my commit, there are couple ways to create logging factory. It isn’t a big issue. Almost every logger provider has support for Microsoft’s

Cool, will do.
Denys, don't get me wrong.  I am not challenging the code or your intentions, job well done. At the same time, as a heavy MassTransit user, I don't want to bring more and more dependencies to it and to our apps with it.
To finalise, I did a small experiment. I created an empty console app and added two packages that are added to the MassTransit.csproj by this PR:
  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="2.2.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.2.0" />
  </ItemGroup>

Then, I published the app and I see this in the output:
ConsoleApp1.deps.json
ConsoleApp1.dll
ConsoleApp1.pdb
ConsoleApp1.runtimeconfig.json
Microsoft.Extensions.Configuration.Abstractions.dll
Microsoft.Extensions.Configuration.Binder.dll
Microsoft.Extensions.Configuration.dll
Microsoft.Extensions.DependencyInjection.Abstractions.dll
Microsoft.Extensions.Logging.Abstractions.dll
Microsoft.Extensions.Logging.Configuration.dll
Microsoft.Extensions.Logging.Console.dll
Microsoft.Extensions.Logging.dll
Microsoft.Extensions.Options.ConfigurationExtensions.dll
Microsoft.Extensions.Options.dll
Microsoft.Extensions.Primitives.dll
System.Runtime.CompilerServices.Unsafe.dll

I seriously challenge all these files to be added to my projects.

@alexeyzimarev it was Microsoft.Extensions.Logging.Console. I've removed it

My understanding is that the only library which would be referenced is this one:
Microsoft.Extensions.Logging.Abstractions
And that has no dependencies.

yeap Chris, you are right:
https://github.com/MassTransit/MassTransit/pull/1443/files#diff-84983e65b5f42b4207bfe5cc81c9ec62R32

No longer needed, based upon https://github.com/MassTransit/MassTransit/pull/1549


Editing for clubs + Picture uploading/static serving from dropbox

Resolves #91 FINALLY!

Resolves #83 as well

Resolves #83 as well


Docker Build Arg unable to consume poor error output

Output of docker version:
docker version
Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      linux/amd64

Output of docker info:
docker info
Containers: 4
 Running: 1
 Paused: 0
 Stopped: 3
Images: 3203
Server Version: 1.10.3
Storage Driver: devicemapper
 Pool Name: docker-202:2-1124190117-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 59.79 GB
 Data Space Total: 107.4 GB
 Data Space Available: 47.59 GB
 Metadata Space Used: 58.47 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.089 GB
 Udev Sync Supported: false
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.82 (2013-10-04)
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 3.10.0-327.3.1.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.26 GiB
Name: ip-xx-xxx-xxx-xxx.xxx.xxx.xxx.xxx
ID: 4ZQV:TN5A:7PIJ:OFEX:46CS:MUNI:RK6C:H4AV:5RF4:GTQY:OPTN:O6XN
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

JENKINS
Steps to reproduce the issue:

Do not provide an environment variable for consumption during docker build --build-arg

Step 5 : ARG BUILDER_HOST
 ---> Running in 794a78698fd3
 ---> edc128bddd1a
Removing intermediate container 794a78698fd3
Step 6 : ARG DOCKER_IMAGE
 ---> Running in 15ce05d7c755
 ---> 4f2d9989e3c7
Removing intermediate container 15ce05d7c755
Step 7 : ARG GIT_REPO
 ---> Running in 0e37b61a7390
 ---> 80865f5bf341
Removing intermediate container 0e37b61a7390
Step 8 : ARG GIT_BRANCH
 ---> Running in 8e7ec51219af
 ---> c3ef61652819
Removing intermediate container 8e7ec51219af
Step 9 : ARG GIT_HASH
 ---> Running in 864b64a55ae1
 ---> ba7d139d2099
Removing intermediate container 864b64a55ae1
Step 10 : ARG DOCKER_TAG
 ---> Running in 94c341a68baa
 ---> 03e8174ca405
Removing intermediate container 94c341a68baa
Step 11 : ARG BUILD_DATE
 ---> Running in cc94c6e5412c
 ---> f3777cbd2b2c
Removing intermediate container cc94c6e5412c
Step 12 : ARG BASE_SHA
 ---> Running in 01a9509d9d9a
 ---> 94bd7c0d2af0
Removing intermediate container 01a9509d9d9a
Step 13 : ARG CREATED
 ---> Running in f6e49015e2a2
 ---> 32f96ac234a8
Removing intermediate container f6e49015e2a2
Step 14 : ARG VENDOR
 ---> Running in c16888a5476e
 ---> 15881d74d012
Removing intermediate container c16888a5476e
Step 15 : ARG VERSION
 ---> Running in 05cebf9acf03
 ---> a7fbac45da15
Removing intermediate container 05cebf9acf03
Step 16 : LABEL com.docker.hub.base.version "${VERSION:-UNKNOWN}"
 ---> Running in 7e06250f73d5
 ---> 355006c391da
Removing intermediate container 7e06250f73d5
Step 17 : LABEL com.docker.hub.base.image "${VENDOR:-UNKNOWN}"
 ---> Running in 8d37a6e826a7
 ---> f5a2962823b5
Removing intermediate container 8d37a6e826a7
Step 18 : LABEL com.docker.hub.base.build-date "${CREATED:-UNKNOWN}"
 ---> Running in ab0be94badce
 ---> eb4441bd6cb3
Removing intermediate container ab0be94badce
Step 19 : LABEL com.docker.hub.base.digest "${BASE_SHA:-UNKNOWN}"
 ---> Running in 7f07d5e2fa9a
 ---> 792394d36027
Removing intermediate container 7f07d5e2fa9a
Step 20 : LABEL a "${DOCKER_IMAGE:-my_img_name}"
 ---> Running in 22658a934415
 ---> 662fb6169fb0
Removing intermediate container 22658a934415
Step 21 : LABEL b "${BUILDER_HOST:-UNKNOWN}"
 ---> Running in 4b8ec79c5a5f
 ---> deac7842e72b
Removing intermediate container 4b8ec79c5a5f
Step 22 : LABEL c "${GIT_REPO:-git/my_git_repo}"
 ---> Running in 28f0459195a9
 ---> f0c002382cf8
Removing intermediate container 28f0459195a9
Step 23 : LABEL d "${GIT_BRANCH:-UNKNOWN}"
 ---> Running in 34d5f96496af
 ---> b0b1d534d4ef
Removing intermediate container 34d5f96496af
Step 24 : LABEL f "${GIT_HASH:-UNKNOWN}"
 ---> Running in c1498cd739e5
 ---> d135a3f16e2d
Removing intermediate container c1498cd739e5
Step 25 : LABEL g "${DOCKER_TAG:-0.0.x}"
 ---> Running in c248d76d5c81
 ---> ec4acba1e60a
Removing intermediate container c248d76d5c81
Step 26 : LABEL h "${BUILD_DATE:-UNKNOWN}"
 ---> Running in 08801055ef6e
 ---> bcdd36987524
Removing intermediate container 08801055ef6e
One or more build-args [[BUILD_DATE [DOCKER_IMAGE [VERSION [http_proxy [https_proxy [BUILDER_HOST [CREATED [DOCKER_TAG [GIT_BRANCH [GIT_HASH [GIT_REPO [VENDOR [BASE_SHA] were not consumed, failing build.

The output of this error should point exactly to the MISSING build-arg

can you give us a minimal Dockerfile that shows the issue and the complete command line you ran?

FROM centos:7.2.151

ARG BUILDER_HOST
ARG DOCKER_IMAGE
ARG GIT_REPO
ARG GIT_BRANCH
ARG GIT_HASH
ARG DOCKER_TAG
ARG BUILD_DATE
ARG BASE_SHA
ARG CREATED
ARG VENDOR
ARG VERSION

LABEL com.docker.hub.base.version="${VERSION:-MANUAL}"
LABEL com.docker.hub.base.image="${VENDOR:-MANUAL}"
LABEL com.docker.hub.base.build-date="${CREATED:-MANUAL}"
LABEL com.docker.hub.base.digest="${BASE_SHA:-MANUAL}"
LABEL gov.dhs.uscis.elis2.base.image="${DOCKER_IMAGE:-base_image}"
LABEL gov.dhs.uscis.elis2.base.builder="${BUILDER_HOST:-MANUAL}"
LABEL gov.dhs.uscis.elis2.base.git.repo="${GIT_REPO:-USCIS/docker_CentOS7_BaseImage}"
LABEL gov.dhs.uscis.elis2.base.git.branch="${GIT_BRANCH:-MANUAL}"
LABEL gov.dhs.uscis.elis2.base.git.sha="${GIT_HASH:-MANUAL}"
LABEL gov.dhs.uscis.elis2.base.version="${DOCKER_TAG:-0.0.x}"
LABEL gov.dhs.uscis.elis2.base.build-date="${BUILD_DATE:-MANUAL}"

In the above case if the ENV variable for say, VERSION does not exist or is empty, Docker would give that error even when a default variable is provided.

One or more build-args [[BUILD_DATE [DOCKER_IMAGE [VERSION [http_proxy [https_proxy [BUILDER_HOST [CREATED [DOCKER_TAG [GIT_BRANCH [GIT_HASH [GIT_REPO [VENDOR [BASE_SHA] were not consumed, failing build.

Something is really weird here, all the build-arg the docker build reported are prepended with a [, probably why it fails on all of them.
With the following Dockerfile…
FROM busybox

ARG VERSION
LABEL com.docker.hub.base.version="${VERSION:-MANUAL}"

… this is some output examples :
λ docker build --build-arg VERSION -t test
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM busybox
 ---> 47bcc53f74dc
Step 2 : ARG VERSION
 ---> Using cache
 ---> 3dfe425e7ef5
Step 3 : LABEL com.docker.hub.base.version "${VERSION:-MANUAL}"
 ---> Using cache
 ---> 5f2cde124b1b
Successfully built 5f2cde124b1b
# […]
λ docker build --build-arg FOO --build-arg BAR --build-arg VERSION -t test
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM busybox
 ---> 47bcc53f74dc
Step 2 : ARG VERSION
 ---> Using cache
 ---> 3dfe425e7ef5
Step 3 : LABEL com.docker.hub.base.version "${VERSION:-MANUAL}"
 ---> Using cache
 ---> 5f2cde124b1b
One or more build-args [BAR FOO] were not consumed, failing build.

As @duglin said, it reports any build-arg specified using the cli but not consumed by the Dockerfile… So in your example, somehow, you specified it in some way (and in a way all of them were prepended with a [).

@kiawin please show us the 'docker build' command used. I have feeling its not what you expect.

Let me close this ticket for now, as it looks like it went stale.


Add JAX-RS Callback

Goal
JAX-RS is a very popular API which is yet to be out-of-the-box supported by Bugsnag.
Design
The design very highly mimics the Java Servlet API Callback implementation.
Minimal Java version: 8
Testing
I've been using this in production for about two weeks as of today.

Somewhy Travis build was not triggered

Well... okay. If this repository wants to be java 7 compatible I have nothing to offer :(


Fix the Validate Examples GitHub Action

The GitHub Action is not running through at the moment and this Pull Request is to fix the issue
Changes made:

Remove the git fetch command from the "Identify and copy changed .json and .xml files"

Zumindest gibt es Reviewer, die das bemerken. Danke


feat: update dependencies, improve dot-prop guard type

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Files
Coverage Δ

src/index.ts
100.00% <100.00%> (ø)

:loudspeaker: Thoughts on this report? Let us know!.


[HeadlessInAppWebView] Error Domain=RBSServiceErrorDomain Code=1 on iOS 15.2

InAppWebView - dealloc
PullToRefreshControl - dealloc
[assertion] Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}>
[ProcessSuspension] 0x10e9f8ba0 - ProcessAssertion: Failed to acquire RBS assertion 'ConnectionTerminationWatchdog' for process with PID=3743, error: Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}

I am having this issue and i'm not using the inappwebview. I wonder if it is related?

I have the same problem.
flutter_inappwebview 5.3.2
log
FlutterWebViewController - dealloc
LeakAvoider - dealloc
InAppWebViewMethodHandler - dealloc
PullToRefreshControl - dealloc
InAppWebView - dealloc
[assertion] Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}>
[ProcessSuspension] 0x10a6f9380 - ProcessAssertion: Failed to acquire RBS assertion 'ConnectionTerminationWatchdog' for process with PID=784, error: Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}

[✓] Flutter (Channel stable, 2.10.0, on macOS 11.6 20G165 darwin-x64, locale zh-Hans-CN)
    • Flutter version 2.10.0 at /Users/admin/flutter
    • Upstream repository https://github.com/flutter/flutter/
    • Framework revision 5f105a6ca7 (8 days ago), 2022-02-01 14:15:42 -0800
    • Engine revision 776efd2034
    • Dart version 2.16.0
    • DevTools version 2.9.2
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at /Users/admin/android/sdk
    • Platform android-30, build-tools 32.0.0
    • ANDROID_HOME = /Users/admin/android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.10.0

[✗] Chrome - develop for the web (Cannot find Chrome executable at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)

[✓] Connected device (1 available)
    • XXX iPad (mobile) • 30b361fa0d77c8f3358d86e61b300fc0b819028b • ios • iOS 15.3 19D50

Any updates on this?

I am facing this error too. any update on this ?

Any updates?

Any updates?
I'm getting this error too in InAppWebView (I'm not using HeadlessInAppWebView) and this leads to a blank white page instead of the requested page.
Moreover, it happens randomly - sometimes it works as expected and load the requested page and sometimes I get this error.

Any updates? I'm getting this error in InAppWebView without using HeadlessInAppWebView

I've stopped using a global key for the webview.
Afaict inappwebview (or the underlying webview for thst matter) doesn't notice when the view (or some part of it, there are multiple processes) gets garbage collected due to the device going to sleep etc. I don't think there's a way to prevent it either.
Since I use bloc I increment a "generation" counter which causes the builder to create a new inappwebview. I do this whenever I detect a problem, so the user gets a new webview quickly. For a smooth user experience this requires that you maintain most of the state outside of the webview, and that you have a way to transfer the state back into the webview.
I did all my testing on Flutter 2.x, not re-tested all the problems after upgrading to 3.x

I've stopped using a global key for the webview.
Afaict inappwebview (or the underlying webview for that matter) doesn't notice when the view (or some part of it, there are multiple processes) gets garbage collected due to the device going to sleep etc. I don't think there's a way to prevent it either.
You get the white page when the render process associated with the webview has been killed.
Since I use bloc I increment a "generation" counter which causes the builder to create a new inappwebview. I do this whenever I detect a problem, so the user gets a new webview quickly. For a smooth user experience this requires that you maintain most of the state outside of the webview, and that you have a way to transfer the state back into the webview.
I did all my testing on Flutter 2.x, not re-tested all the problems after upgrading to 3.x

The webview seems to be deallocated when error occurs so the onLoadError method is not called. How do you detect problems?

Implementing androidOnRenderProcessGone and iosOnWebContentProcessDidTerminate , and have them trigger mapGenerationCubit.increment() probably cover 95%
My app frequently sends updates to the webview, so I also have watchdog timer there. Every time I send a message to the webview, if I don't see an ack within X seconds I also kill the webview.

Implementing androidOnRenderProcessGone and iosOnWebContentProcessDidTerminate , and have them trigger mapGenerationCubit.increment() probably covers 95% of the cases.
My app frequently sends updates to the webview, so I also have watchdog timer there. Every time I send a message to the webview, if I don't see an ack within X seconds I start over.

Thanks! In my case iosOnWebContentProcessDidTerminate is not called. The problem occurs high frequently after the app launches in release mode on Flutter 3.0.1. When I switch tab and back to the webview, all content show without loading progress. It acts normally in debug or profile mode.

(No need to quote when you are replying to the comment directly above.)
I think you need a physical device not attached to power to trigger this. Lock the screen and wait 30 sec.
If you run the iOS simulator the webview process is visible in mac os, so you can kill it manually to simulate.

I will try it.
It's odd that If I run directly in vscode on my iphone xr, it's hard to reproduce the problem. But if I build general app through fastlane, the problem is easy to reproduce.

Any solution yet on this?

Hello Team,
Any solution to this?
Facing the same issue on iOS 16.0.2 devices. It's happening only in the builds. The page goes blank white.
FlutterWebViewController - dealloc
LeakAvoider - dealloc
InAppWebViewMethodHandler - dealloc
InAppWebView - dealloc
PullToRefreshControl - dealloc
[assertion] Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}>
[ProcessSuspension] 0x11601c720 - ProcessAssertion::acquireSync Failed to acquire RBS assertion 'ConnectionTerminationWatchdog' for process with PID=3814, error: Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}
[✓] Flutter (Channel stable, 3.3.10, on macOS 12.6 21G115 darwin-arm (Rosetta), locale en-IN)
• Flutter version 3.3.10 on channel stable at /Users/jk/Documents/developer/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 135454af32 (7 weeks ago), 2022-12-15 07:36:55 -0800
• Engine revision 3316dd8728
• Dart version 2.18.6
• DevTools version 2.15.0
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0-rc1)
• Android SDK at /Users/jk/Library/Android/sdk
• Platform android-33, build-tools 33.0.0-rc1
• Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 14A400
• CocoaPods version 1.11.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
[✓] VS Code (version 1.74.3)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension can be installed from:
🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[✓] Connected device (3 available)
• JK's iPhone (mobile)      • ios                       • iOS 16.0.2 20A380
• macOS (desktop)           • macos                     • darwin-arm64   • macOS 12.6 21G115 darwin-arm (Rosetta)
• Chrome (web)                • chrome                    • web-javascript • Google Chrome 109.0.5414.119

Any solution of this?

Facing same issue

Same here, on my side this happens in combination using Azure B2C on the InAppWebView and users try to "Sign in with Apple". So this is kinda mission critical as it prevents users from signing in. Upgrading to the latest beta version and enabling "Background processing" within "Background modes" on "Signing & Capabilities" in Xcode did help. Recent iOS-Updates did change something so my assumption is that the underlaying WebKit implementation now recognises apple.com URLs and tries to launch the native "Sign in with Apple" flow. I'm 100% sure that this was not the case couple months ago and our users were redirected within the web view to Apples Web-OAuth-Flow.
Changed version from flutter_inappwebview: ^5.7.1+2 to flutter_inappwebview: 6.0.0-beta.22. Not sure what's the key difference in 6.0.0, so maybe a dependency was updated or a bug was fixed that lead to the disposal of the WebView.
Anyhow, after the changes the native "Sign in with Apple" is launched properly and the WebView continues when getting redirected. Before it lead to an "onLoadError" call what basically ended the journey.
Flutter doctor -v:
[✓] Flutter (Channel stable, 3.7.7, on macOS 13.0.1 22A400 darwin-arm64, locale en-CH)
    • Flutter version 3.7.7 on channel stable at /Users/thomas/Flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2ad6cd72c0 (5 days ago), 2023-03-08 09:41:59 -0800
    • Engine revision 1837b5be5f
    • Dart version 2.19.4
    • DevTools version 2.20.1

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, please use
      `flutter config --android-sdk` to update to that location.

[✓] Xcode - develop for iOS and macOS (Xcode 14.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14B47b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).

[✓] VS Code (version 1.75.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.58.0

[✓] Connected device (3 available)
    • iPhone (mobile) • 00008030-001D54393C3A802E • ios            • iOS 16.3.1 20D67
    • macOS (desktop) • macos                     • darwin-arm64   • macOS 13.0.1 22A400 darwin-arm64
    • Chrome (web)    • chrome                    • web-javascript • Google Chrome 111.0.5563.64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

any solution ?

Any updates on this?

I am getting the same issue. Does anyone know if it is ok to have these issues in the app. Will this cause memory leaks or any other problems with a production application?
FlutterWebViewController - dealloc
LeakAvoider - dealloc
InAppWebViewMethodHandler - dealloc
InAppWebView - dealloc
PullToRefreshControl - dealloc

+1

error: Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}

This problem occur when I'm dispose the widget that contain inappwebview before the it can display @pichillilorenzo


Use vcpkg cache from 2020.11 version on CI

This change doesn't really require Central-EngSys .. It can be done at any time by updating https://github.com/Azure/azure-sdk-for-cpp/blob/master/eng/vcpkg-commit.txt
cc @vhvb1989
You'll want to run all test pipelines by adding comments like /azp run cpp - core in the PR to kick off all of the pipelines. To be most confident I recommend running live tests.

@antkmsft Lets make this change early this week, please prioritize this.

@vhvb1989, we are now on the latest release, 2020.11: https://github.com/Azure/azure-sdk-for-cpp/blob/master/eng/vcpkg-commit.txt + https://github.com/microsoft/vcpkg/releases

@antkmsft yes, that's correct, but we are still using the build cache from Azure-DevOps: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=822762&view=logs&j=0fa9045b-9773-5caa-2289-e9e87f13f3cb&t=880ba5cd-f074-5f56-2cef-dc78056bb3a5
The Azure devops caches uses 4 strings to build a key for the cache:
Resolving key:
- Validate Linux_x64_gcc8   [string]
- "tags/2020.11"            [string]
- Linux                     [string]
- curl[ssl] libxml2 openssl [string]
This issue was created to ask Eng Sys (@danieljurek ) to remove the azure devops cache and use the VCPKG cache that is available in the version we are using. We need to set it up to where the cache will be persisted.

@danieljurek, my understanding is that we are either where we want to be already, then please close this issue. Otherwise, please either fix it, or let us know how we can get it fixed (and if we should).

This the binary caching feature in vcpkg: https://vcpkg.readthedocs.io/en/stable/users/binarycaching/
It isn't required for GA.


Firestore screenshots or exported json needed for running the project without null error

The common error everyone is facing is because I think it is trying to fetch something from database and since we have a new database with no collections, null error is there, can you please provide your firebase firestore database screenshots, or you can export it in json format using this and share it here somewhere?

when I'm logging into google account it shows like this. How  to resolve this?

when I'm logging into google account it shows like this. How to resolve this?

You cannot pretty much do anything, you need a copy of database that the original owner of this repo is using and then only you can run the application without any null errors and that is what I have asked for in this issue but I think the person is inactive since a long time.

This issue is due to an out dated firebase dependencies, not related to my(owners) firebase storage. When you add the package name and SHA-1 key to your firebase project, it uses your firebase storage not mine.


import and cloning problems between versions

it is normal to have serious import and cloning problems between the master version and dev version ?

There may be issues if your master version is not 1.14.3 (the latest).

I solved using master -> master , so  I don't know if the thing really works, but maybe you can leave open this here for a while, so people who have problems of this kind can read here and see if it works. I saw in hubchart-tarine.rhcloud.com a lot  of 1.14.2 hubs...


Redis - Caching System (very fast)

We should use Redis for Caching (Redis is the fastest key storage).
http://redis.io/

We will analyse if is good for us. @Nenji can you say the features of Redis ?

I don't think that we need Redis. NHibernate already got First Level Caching by default and we can additionally activate second level caching, when we get around to do so.
Yupi just uses very few ram on startup, because the cache is a real cache:
On first request, it WILL result in a cache miss, but on subsequent requests (by id), NHibernate should return the data from cache (cache hit).

As we see, isn't necessary now. Closing Issue.
If anyone want to open again feel free.


Getting error while creating restart service rule in Stackstorm

SUMMARY
We tried to restart linux service using stackstorm and created linux.service. When we tried to run that getting below error.
Error:: Traceback (most recent call last):\n  File "/tmp/6244491ccb55c0f4c7922066/service.py", line 32, in \n    from st2common.util.shell import quote_unix\nImportError: No module named st2common.util.shell
Command to restart service was: systemctl restart httpd.service
Please suggest what changes we need to do for this error.
STACKSTORM VERSION
Paste the output of st2 --version:
OS, environment, install method
Post what OS you are running this on, along with any other relevant information/

e.g. Docker, Vagrant, Kubernetes, etc. Describe how you installed ST2
e.g. one-line install, custom install, etc -->

Steps to reproduce the problem
Show how to reproduce the problem, using a minimal test-case. Make sure to include any content
(pack content - workflows, actions, etc.) which are needed to reproduce the problem.
Expected Results
What did you expect to happen when running the steps above?
Actual Results
What happened? What output did you get?
Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!

Duplicate of https://github.com/StackStorm/st2/issues/5004

$ st2 --version
st2 3.6.0, on Python 3.6.8
Installed on Linux server and it was one line install.

@nzlosh  we look into the error 5004 , we still did not find any resolution regarding our issue. We are getting same error while running linux.service. Kindly provide any detail , what changes we should do for running this service. I have provided error logs below:
{
"hostname**": {
"stdout": "",
"stderr": "Traceback (most recent call last):\n  File "/tmp/6244491ccb55c0f4c7922066/service.py", line 32, in \n    from st2common.util.shell import quote_unix\nImportError: No module named st2common.util.shell",
"return_code": 1,
"succeeded": false,
"failed": true
}
}


Add HACKING.md

Added initial HACKING.md file to provide development instructions and
guidelines.
Signed-off-by: Don Penney<EMAIL_ADDRESS>

/retest

/retest

/retest


Create a settings page for changing LLM/TTS model settings

Page should be accessed via the navbar
A field to change LLM model (Selection)
A field to change TTS Voice (Selection)

Frontend Completed

Settings for GPT/LLM complete