Null Ref Exception when InstrumentationKey is null

We are building some middleware for ASP.NET Core which takes advantage of AI's handling of the Request-Id header. While testing the middleware we wanted to enable AI but did not want to report any telemetry. We therefore created:
var appInsightsOptions =
                new ApplicationInsightsServiceOptions
                {
                    DeveloperMode = Environment.IsDevelopment()
                };

Note that there is no Instrumentation  Key. Invoking services.AddApplicationInsightsTelemetry(appInsightsOptions); with these options succeeds. However, our component under test later makes Web Requests via an HttpClient. These invocations result in a NRE with no useful debugging information.
We worked around this issue by adding an empty-string Instrumentation Key to our AI Options:
var appInsightsOptions =
                new ApplicationInsightsServiceOptions
                {
                    DeveloperMode = Environment.IsDevelopment(),
                    InstrumentationKey = ""
                };

Expected:
There are two obvious options:

Enable HttpClient to work when AI is enabled but an Instrumentation Key is not provided. This is preferred, since AI seems to work [e.g. it reads an incoming request's Request-Id header and propagates it to Activity.Current.Id] so long as we don't invoke an HttpClient.
Provide an actionable exception. This would probably be some sort of argument exception when invoking AddApplicationInsightsTelemetry without an Instrumentation Key.

@lmolkova can you please take a look?

Here is a stack trace from @piotrpMSFT
{"message":"An unhandled exception occurred.","exception":"System.NullReferenceException: Object reference not set to an instance of an object.\n   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.ApplicationInsightsUrlFilter.get_EndpointLeftPart()\n   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.ApplicationInsightsUrlFilter.IsApplicationInsightsUrlImpl(String url)\n   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.HttpCoreDiagnosticSourceListener.OnActivityStart(HttpRequestMessage request)\n   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.HttpCoreDiagnosticSourceListener.OnNext(KeyValuePair`2 evnt)\n   at System.Diagnostics.DiagnosticListener.Write(String name, Object value)\n   at System.Diagnostics.DiagnosticSource.StartActivity(Activity activity, Object args)\n   at System.Net.Http.DiagnosticsHandler.<SendAsync>d__2.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\n   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()\n   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\n

So the issue happens on netcoreapp 2.0
According to @piotrpMSFT it frequently reproduces on Linux and  sometimes on Mac.
I was not able to repro it on Windows: no exception happens with null or empty ikey.
Potentially the issue is some static initialization race condition that happens when dependency call is done before the AI fully initialized
Perhaps exception happens here: https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/Shared/Implementation/ApplicationInsightsUrlFilter.cs#L34, however it's not clear how TelemetryChannel could be null.
        private string EndpointLeftPart
        {
            get
            {
                string currentEndpointAddressValue = null;

                // Cache AI endpoint URI
                if (this.telemetryConfiguration != null)
                {
                    // EXCEPTION ???
                    string endpoint = this.telemetryConfiguration.TelemetryChannel.EndpointAddress;
                    if (!string.IsNullOrEmpty(endpoint))
                    {
                        // The TelemetryChannel is used which defines the production endpoint in ApplicationInsights.config.
                        currentEndpointAddressValue = endpoint;
                    }
                }

                if (this.cachedEndpointLeftPart.Key != currentEndpointAddressValue)
                {
                    this.cachedEndpointLeftPart = this.GetEndpointLeftPart(currentEndpointAddressValue);
                }

                return this.cachedEndpointLeftPart.Value;
            }
        }

@cijothomas do you know if we have a way to collect event source traces on Mac?
As a workaround, problem should be solved by using real AI resource for free plan.

Some updates on this issue:

I have a fairly reliable repro on Windows, but not in Visual Studio. Running dotnet test from the command line reproduces the issue with similar frequency to Mac.
Adding a InstrumentationKey for test scenarios, as suggested above, did not resolve the issue. I do not have a viable workaround at this time.

@piotrpMSFT
Since the workaround did not help, it seems absence of ikey is not the problem.
for the code you shared with me previously, for the test scenarios you have services.AddTransient<TelemetryClient>()
Any particular reason why it's transient?
Can you repro the issue if you add it as singleton?

I changed this to singleton with no observed difference.

In addition to the change above, we've changed the affected tests to execute serially. This led to a new callstack that may yield some hints:
System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
     at Microsoft.ApplicationInsights.Extensibility.Implementation.TransmissionProcessor.Process(ITelemetry item)
     at Microsoft.ApplicationInsights.TelemetryClient.Track(ITelemetry telemetry)
     at Microsoft.ApplicationInsights.AspNetCore.DiagnosticListeners.HostingDiagnosticListener.EndRequest(HttpContext httpContext, Int64 timestamp)
     at Proxy_Method_From_<>f__AnonymousType2`1_To_Void OnHttpRequestInStop(Microsoft.AspNetCore.Http.HttpContext)(Object , Object , IProxyFactory )
     at Microsoft.Extensions.DiagnosticAdapter.DiagnosticSourceAdapter.Write(String diagnosticName, Object parameters)
     at Microsoft.Extensions.DiagnosticAdapter.DiagnosticSourceAdapter.System.IObserver<System.Collections.Generic.KeyValuePair<System.String,System.Object>>.OnNext(KeyValuePair`2 value)
     at System.Diagnostics.DiagnosticListener.Write(String name, Object value)
     at System.Diagnostics.DiagnosticSource.StopActivity(Activity activity, Object args)
     at Microsoft.AspNetCore.Hosting.Internal.HostingApplicationDiagnostics.StopActivity(HttpContext httpContext, Activity activity)
     at Microsoft.AspNetCore.Hosting.Internal.HostingApplication.DisposeContext(Context context, Exception exception)
     at Microsoft.AspNetCore.TestHost.TestServer.ApplicationWrapper`1.DisposeContext(TContext context, Exception exception)
     at Microsoft.AspNetCore.TestHost.ClientHandler.<>c__DisplayClass3_0.<<SendAsync>b__0>d.MoveNext()

@piotrpMSFT
based on the traces I collected running your tests, I see that you have multiple subscriptions active to the same DiagnosticsSources.
It means that between the tests ApplicationInsights is not properly disposed.
We will check further why it does not happen, but running your tests one by one (in separate processes, not serially) works fine.

Thanks @lmolkova! Are there ways in which we may be creating and not disposing these subscriptions in e.g. logging infrastructure? Any tell-tale APIs we should be looking for?

It appears that web host was not disposed properly by tests, i.e. AppInsights objects were not disposed and every test created a new AI modules.
The exception is misleading and unrelated to the issue.
@dnduffy @SergeyKanzhelev
Please prioritize it.
Could we detect that AI was not disposed and is being initialized again? Could we prevent it to happen?
I remember spending at least a day debugging a similar disposing-related issue in AspNetCore SDK tests and I think it's not critical, but important to quickly diagnose such problems.

I believe the .NET Core SDK prevents duplicate initialization from calling both AddApplicationInsights and/or UseApplicationInsights more than once but I don't recall anything in the base SDK like that.  It should be safe to make multiple TelemetryClient and TelemetryConfiguration instances and/or to reuse the default Active config instance.  There used to be a check in the base to make sure a null or empty instrumentation key resulted in logging a message to the console instead of trying to actually send data, I'm not sure what the code is currently like though after the changes for multiple channels.

@dnduffy
the problem I have seen is NOT related to ikey, but to the fact that AddApplicationInsights is called multiple times in the same process. So this is not safe at least under some conditions.

And it reporduces with AI SDK 2.4, so multi channel is not related

Sounds like the base SDK init needs to be made idempotent the way the .NET Core SDK was.
@cijothomas

FYI - This may be purely anecdotal, but I recently set "DeveloperMode" in my AI config to true, I started receiving the get_EndpointLeftPart null reference exception about every other time while my app was initializing.  I then turned off that feature and that error seems to have gone away.  At least the last 20 or so times I tried starting my app.

With more testing, it doesn't seem related to that, I still haven't tracked down the original source but found that if I start/stop a WebHost, it does something nasty to the internals that are connecting to remote servers via HttpClient.  Anyway, I found a work around, but still don't understand how/why this was happening.

Any news on making AddApplicationInsights idempotent? Multiple calls to AddApplicationInsights is a common practice in Service Fabric... in many different kind of scenarios including: SharedProcess hosting model, multiple listeners...

It seems I have found the root cause for this issue.
It looks like the problem is not AddApplicationInsights  not being idempoted - it is.
The problem is multiple WebHosts running in the same process. I.e. there are several instances of ServiceProvider and each of them has full and valid set of ApplicationInsights services.
However since there are a lot of singletons (i.e. static instance, not the DI kind of singleton), their lifetime is not limited to the WebHost.
It's easy to repro with following tests:

Sequential web host creation  - seems to work just fine

        public async Task TwoWebHostsSequentially()
        {
            using (var host1 = BuildWebHost(new string[0], 5000))
            {
                Task.Run(() => host1.Run());
                await hc.SendAsync(request);
            }

            using (var host2 = BuildWebHost(new string[0], 5001))
            {
                Task.Run(() => host2.Run());
                await hc.SendAsync(request);
            }
        }

Parallel web host creation - works, however, dependencies are reported twice for each host ( perhaps because of DiagnosticListener subscriptions)

        [Fact]
        public async Task TwoWebHostsInParallel()
        {
            using (var host1 = BuildWebHost(new string[0], 5000))
            using (var host2 = BuildWebHost(new string[0], 5001))
            {
                Task.Run(() => host1.Run());
                Task.Run(() => host2.Run());

                var request1 = new HttpRequestMessage(HttpMethod.Get, "http://localhost:5000/api/values");
                await hc.SendAsync(request1);

                var request2 = new HttpRequestMessage(HttpMethod.Get, "http://localhost:5001/api/values");
                await hc.SendAsync(request2);
            }
        }

one of the hosts is disposed while another is active

        [Fact]
        public async Task TwoWebHostsAfterOneIsDisposed()
        {
            var host1 = BuildWebHost(new string[0], 5000);
            var host2 = BuildWebHost(new string[0], 5001);

            Task.Run(() => host1.Run());
            Task.Run(() => host2.Run());

            var request1 = new HttpRequestMessage(HttpMethod.Get, "http://localhost:5000/api/values");
            await hc.SendAsync(request1);
            host1.Dispose();

            var request2 = new HttpRequestMessage(HttpMethod.Get, "http://localhost:5001/api/values");
            await hc.SendAsync(request2);
        }

It results in
Test Name:	testrepro.UnitTest1.TwoWebHostsAfterOneIsDisposed
Test FullName:	testrepro.UnitTest1.TwoWebHostsAfterOneIsDisposed
Test Source:	c:\repo\delme\aspnetcore\testrepro\UnitTest1.cs : line 71

Result StackTrace:
at Microsoft.ApplicationInsights.DependencyCollector.Implementation.ApplicationInsightsUrlFilter.get_EndpointLeftPart()
   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.ApplicationInsightsUrlFilter.IsApplicationInsightsUrlImpl(String url)
   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.HttpCoreDiagnosticSourceListener.HttpCoreDiagnosticSourceSubscriber.<OnNext>b__6_0(String evnt, Object r, Object _)
   at System.Diagnostics.DiagnosticListener.IsEnabled(String name, Object arg1, Object arg2)
   at System.Net.Http.DiagnosticsHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at testrepro.UnitTest1.<TwoWebHostsAfterOneIsDisposed>d__5.MoveNext() in c:\repo\delme\aspnetcore\testrepro\UnitTest1.cs:line 103
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Result Message:	System.NullReferenceException : Object reference not set to an instance of an object.

@mkosieradzki
I believe that what's happen in your case. Could you confirm?
Root cause:
TelemetryConfiguration being a singleton (static instance) is reused by host2.
When the host1 is disposed, it causes TelemetyConfiguration instance to be disposed.
This results in TelemetryChannel underneath to be disposed and set to null
So when app insights in host2 attempt to track dependency, it leads to the call to null telemetry channel.

@lmolkova thanks for doing all the analysis for this case. I confirm that this is the bigger part of the case, however I am still observing (after doing a workaround and reusing a single instance of DependencyTrackingTelemetryModule In multiple WebHosts) that some HttpRequests originating from OUTSIDE of the WebHost are throwing the aforementioned exception (I don’t know whether it is a lifetime or CallContext related issue).
It would be important to make sure requests coming from outside of the WebHosts are not broken as well.
Example from the ServiceFabric world: a service is exposing some listeners (each one is a WebHost) and a backup loop which is creating backups using AzureStorageSDK under some conditions backups are failing. One should not make any assumptions about the lifetime of listeners when compared to the backup loop...

@mkosieradzki
dependency tracking happens beyond WebHosts and relies purely on .NET - it is not aware of AspNet DI and web hosts and does not respect it's boundaries.
On our side we should make sure:

dependency tracking never ever throws no matter what
we do not reuse telemetry configuration between web hosts
we DO reuse dependency tracking listeners between hosts or make sure they only track their dependencies

BTW, we create a dependency tracking module per web host. you could easily check it with this test
            var host1 = BuildWebHost(new string[0], 5003);
            var host2 = BuildWebHost(new string[0], 5004);

            var depTracking1= host1.Services.GetServices(typeof(ITelemetryModule)).First();
            var depTracking2= host2.Services.GetServices(typeof(ITelemetryModule)).First();
            depTracking1 != depTracking2 at this point

Perhaps your workaround causes telemetry configuration to be re-created and this is the real workaround.

@lmolkova Yup, sounds reasonable. It also explains why it works only partially ;).

@mkosieradzki
BTW, would you mind sharing your workaround?

@lmolkova I am sorry for the delay.
I am not very proud of this workaround:
        private static readonly Lazy<DependencyTrackingTelemetryModule> dependencyTrackingModule = new Lazy<DependencyTrackingTelemetryModule>(() =>
        {
            var module = new DependencyTrackingTelemetryModule();

            var excludedDomains = module.ExcludeComponentCorrelationHttpHeadersOnDomains;
            excludedDomains.Add("core.windows.net");
            excludedDomains.Add("core.chinacloudapi.cn");
            excludedDomains.Add("core.cloudapi.de");
            excludedDomains.Add("core.usgovcloudapi.net");
            excludedDomains.Add("localhost");
            excludedDomains.Add("<IP_ADDRESS>");

            var includedActivities = module.IncludeDiagnosticSourceActivities;
            includedActivities.Add("Microsoft.Azure.EventHubs");
            includedActivities.Add("Microsoft.Azure.ServiceBus");

            return module;
        });

        private static IServiceCollection AddApplicationInsightsTelemetryProper(this IServiceCollection services, string instrumentationKey)
        {
            services.AddApplicationInsightsTelemetry(instrumentationKey);

            var invalidDependencyTrackingModule = services.FirstOrDefault(x => x.ImplementationType == typeof(DependencyTrackingTelemetryModule));
            services.Remove(invalidDependencyTrackingModule);
            services.AddSingleton<ITelemetryModule>(dependencyTrackingModule.Value);

            return services;
        }

        public static void Cleanup()
        {
            if (dependencyTrackingModule.IsValueCreated)
                dependencyTrackingModule.Value.Dispose();
        }

Cleanup is called after the hosting application exists...
It strongly improves application stability, however still causes failures.
It was basing on the, probably wrong, assumption that the cause for the exception below, was that ApplicationInsightsUrlFilter was accessing the already disposed DependencyTrackingTelemetryModule.
So I tried to prevent the disposal using AddSingleton override with the manual lifetime contol over the one with automatic disposal.
It basically worked (or seemed to work) for all the requests originating from the WebHost (that's a huge improvement), but as I wrote before it was failing for the requests originating from the backup loop outside of the WebHost.
Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.ApplicationInsightsUrlFilter.get_EndpointLeftPart()
   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.ApplicationInsightsUrlFilter.IsApplicationInsightsUrlImpl(String url)
   at Microsoft.ApplicationInsights.DependencyCollector.Implementation.HttpDesktopDiagnosticSourceSubscriber.b__7_0(String evnt, Object r, Object _)
   at System.Diagnostics.DiagnosticListener.IsEnabled(String name, Object arg1, Object arg2)
   at System.Diagnostics.HttpHandlerDiagnosticListener.RaiseRequestEvent(HttpWebRequest request)
   at System.Diagnostics.HttpHandlerDiagnosticListener.HttpWebRequestArrayList.Add(Object value)
   at System.Net.Connection.StartRequest(HttpWebRequest request, Boolean canPollRead)
   at System.Net.Connection.SubmitRequest(HttpWebRequest request, Boolean forcedsubmit)
   at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connName)


policy: Account for non Git signing keys in verify

This PR adds a check for cases where a key is technically trusted to sign for some namespace but it can only be used with gittuf policies, not Git objects. We have this check elsewhere.
Long term, this check can be pulled out because the signing method aligns with SSH keys. So I suggest we add this check in now and strip them out when we have support for Git <-> SSH keys.

Thanks @wlynch!
Also, cc @bradcush.


[Snyk] Security upgrade python from 3.9 to 3.14-rc-slim

Snyk has created this PR to fix 4 vulnerabilities in the dockerfile dependencies of this project.
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Snyk changed the following file(s):

Dockerfile

We recommend upgrading to python:3.14-rc-slim, as this image has only 37 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Vulnerabilities that will be fixed with an upgrade:

Issue
Score

CVE-2023-49462 SNYK-DEBIAN12-LIBHEIF-6105360
  614

Divide By Zero SNYK-DEBIAN12-LIBHEIF-5498469
  514

Integer Overflow or Wraparound SNYK-DEBIAN12-ZLIB-6008963
  500

Integer Overflow or Wraparound SNYK-DEBIAN12-ZLIB-6008963
  500

CVE-2024-41311 SNYK-DEBIAN12-LIBHEIF-8224949
  436

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 96.25%. Comparing base (8e4779d) to head (3496b43).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #64   +/-   ##
=======================================
  Coverage   96.25%   96.25%
=======================================
  Files          17       17
  Lines         374      374
=======================================
  Hits          360      360
  Misses         14       14

Components
Coverage Δ

core
100.00% <ø> (ø)

api
90.00% <ø> (ø)

tests
100.00% <ø> (ø)


Crash in multiplayer - character prediction issue?

We were testing DynamicCities with a few people online, also learned that for some reason in multiplayer the multijump functionality ends up enabled for infinite jumps. So it may somehow have to do with spamming multi-jump messing with the prediction?
Log snippet from server:
16:56:11.137 [main] WARN  o.t.l.c.ServerCharacterPredictionSystem - Received too much input from EntityRef{id = 89329, netId = 1955, prefab = 'engine:player'}, dropping input.
16:56:11.157 [main] ERROR o.terasology.engine.TerasologyEngine - Uncaught exception, attempting clean game shutdown
java.util.ConcurrentModificationException: null
        at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
        at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
        at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
        at org.terasology.logic.characters.ServerCharacterPredictionSystem.update(ServerCharacterPredictionSystem.java:161)
        at org.terasology.engine.modes.StateIngame.update(StateIngame.java:176)
        at org.terasology.engine.TerasologyEngine.mainLoop(TerasologyEngine.java:414)
        at org.terasology.engine.TerasologyEngine.run(TerasologyEngine.java:369)
        at org.terasology.engine.Terasology.main(Terasology.java:139)
16:56:11.158 [main] INFO  o.terasology.engine.TerasologyEngine - Shutting down Terasology...
16:56:14.667 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Save game finished
16:56:14.670 [main] INFO  o.t.p.i.ReadWriteStorageManager - Saving - Creating game snapshot
16:56:14.684 [main] INFO  o.t.p.i.ReadWriteStorageManager - Saving - Snapshot created: Writing phase starts
16:56:14.721 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: Xt
16:56:14.724 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: rzats
16:56:14.724 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: rzats
16:56:14.724 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: tdgunes
16:56:14.725 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: Cervator
16:56:14.725 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: manu
16:56:14.727 [main] INFO  o.t.n.internal.NetworkSystemImpl - Network shutdown
16:56:19.714 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Save game finished
16:56:19.986 [main] WARN  o.t.rendering.nui.skin.UISkinFormat - Failed to resolve UIWidget class UIInputBindButton, skipping style information
java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
        at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
        at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
        at org.terasology.logic.characters.ServerCharacterPredictionSystem.update(ServerCharacterPredictionSystem.java:161)
        at org.terasology.engine.modes.StateIngame.update(StateIngame.java:176)
        at org.terasology.engine.TerasologyEngine.mainLoop(TerasologyEngine.java:414)
        at org.terasology.engine.TerasologyEngine.run(TerasologyEngine.java:369)
        at org.terasology.engine.Terasology.main(Terasology.java:139)
For more details, see the log files in /opt/terasology/logs/2016-06-18_16-25-23

Edit: Immortius suggested the area simply hasn't been prepared well enough for thread-safety, which would make this a pretty likely outcome.

Crash also happened during today's test. Coincidentally also while testing DynamicCities - but that's probably not related.

Hmmm, I wasn't there. But, I think it may be something to do with thread safety. It doesn't have to do with multijumping, as I fixed that issue.

Yep it just seems to happen from time to time with enough player movement. That area just needs work :-)

Closing this as likely obsolete


Resource Landing Page: fix layout of publication agreement

This iframe brings in unnecessary content (like an arbitrarily cropped up version of the footer), does not wrap the text correctly and does not set appropriate padding. This is specially obvious in mobile devices.
Possible fixes:

Retrieve the content that matters from the iframe and remove everything else with javascript.
Move the agreement text to an HTML file in the project instead of keeping it in Mezzanine. This way we can easily include the text content in the agreement modal window.

No mobile phone support officially that will be a larger design effort in the code

No mobile phone support officially that will be a larger design effort in the code

This is an issue in non-mobile browsers, too


nix fails to record a dependency when home-manager is built this way

I noticed something weird about the way this installs home-manager. Starting with a fresh checkout of the  repo:
$ git reset --hard
HEAD is now at 5725905 update nixpkgs-unstable, stop using nixpkgs-channels
$ nix-shell

[nix-shell:~/gitprojects/public/home-manager-template]$ type home-manager

home-manager is /nix/store/kh0l3n7aba7gqz1asspbb55pkgx2w1p1-home-manager/bin/home-manager

Let's look at what nix thinks the dependencies of home-manager are:
[nix-shell:~/gitprojects/public/home-manager-template]$ nix-store --query --references /nix/store/kh0l3n7aba7gqz1asspbb55pkgx2w1p1-home-manager/bin/home-manager

/nix/store/7fdm1ij6f7xwjmhpici1bh3xh08n0sp7-coreutils-8.31
/nix/store/98an38bw0fwk6a5mickblc0h07jf00vz-gnused-4.8
/nix/store/chs6kqcqk2vmzmrckaa11dvm3qljcdni-less-551
/nix/store/k8p54jg8ipvnfz435mayf5bnqhw4qqap-bash-4.4-p23
/nix/store/ny7vshs45d5k193ql3b51dak0hjwwqc2-findutils-4.7.0

But look, there's a store path in there that was not picked up:
[nix-shell:~/gitprojects/public/home-manager-template]$ grep home-manager-src /nix/store/kh0l3n7aba7gqz1asspbb55pkgx2w1p1-home-manager/bin/home-manager

    for path in "/nix/store/qxdxvvbcfh2b39hd52zj1n2f6s24vnsc-home-manager-src" \

[nix-shell:~/gitprojects/public/home-manager-template]$

I can reproduce this basic thing just by running nix-build https://github.com/nix-community/home-manager/archive/refs/heads/master.zip -A home-manager so it's not necessarily a problem with this repository.  And it probably won't cause a problem with the way this repository is structured. But it means that if you set up a gcroot for home-manager and collect garbage, you'll be left with a broken home-manager.  Or if you try to use nix-copy-closure to copy home-manager somewhere, it will fail to copy the "home-manager-src" path.
What's going on?

Here is my workaround for this strange issue:
https://github.com/pdg137/config/commit/8c3de2f5d475760e7dd224365feb5f59ae765cb4
This allows me to build a home-manager switch script that runs fine after a garbage collection and that I can copy to another machine with nix-copy-closure.

See https://github.com/nix-community/home-manager/pull/5174


Migrate code generator .proto -> .ts from protobufjs to Telescope

In https://github.com/cosmos/ics23/pull/217 it was brought up the that current TypeScript code generator is not working properly anymore. My suggestion is to replace it instead of try to fix it.
There are basically 3 generations of TypeScript code generation we (Confio) used over the years

protobufjs (pretty old school, TS types are generated out of JavaScript doc comments)
ts-proto (quite nice, but a bit slow as it is a protoc plugin and does not allow interpreting Cosmos specific options)
Telescope (most advanced solution)

It would be great to get the the TypeScript code generation migrated from 1. (used now) to 3. In addition to being the most powerful and convenient solution, it also means we can remove protobufjs runtime dependency, which is often reported as having security issues.
This affects the code in the generated folder, not the rest: https://github.com/cosmos/ics23/tree/master/js/src/generated. The generated types will be organized slightly differently but it should not be hard to adapt the JS codebase to it.

I will close the issue, since we removed the Typescript implementation.


List of URLs and status codes?

Is it possible to see a list of retrieved URLs and status codes, like you can with siege?
This assumes that goad parses retrieved web pages and follows them - if that's not the case, please let me know.

No, goad does not parse the pages and follows links.
We are working on a plugin architecture though. So it might soon be possible to run goad in the way you want.
The first plugin will be inspired by siege to allow the user to pass a file of URLs to goad to put load on different routes of a web application.


Linode Volumes Tests

Purpose
Add Linode Volumes tests and mock data
Misc
Had these tests leftover after reviewing the Context API PR, and felt it didn't belong in that PR

Coverage increased (+0.9%) to 29.317% when pulling 2e8576d03a578ff90a3295fcdadd789d07b156aa on martinmckenna:volumes-tests into 2cf7a2876be56ab5f9968406c964e1bf4beff329 on linode:develop.


cmd/mount: add an option to enable ioctl, and disable it by default

Codecov Report
Base: 59.23% // Head: 59.23% // Decreases project coverage by -0.00% :warning:

Coverage data is based on head (0472a31) compared to base (2f34221).
Patch coverage: 83.33% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2859      +/-   ##
==========================================
- Coverage   59.23%   59.23%   -0.01%
==========================================
  Files         140      140
  Lines       24325    24332       +7
==========================================
+ Hits        14410    14414       +4
  Misses       8058     8058
- Partials     1857     1860       +3

Impacted Files
Coverage Δ

pkg/fuse/fuse.go
79.21% <0.00%> (-0.58%)
:arrow_down:

pkg/vfs/vfs.go
76.76% <ø> (ø)

cmd/mount.go
60.61% <100.00%> (+0.12%)
:arrow_up:

cmd/mount_unix.go
49.10% <100.00%> (+1.88%)
:arrow_up:

pkg/chunk/disk_cache.go
74.25% <0.00%> (-1.00%)
:arrow_down:

pkg/meta/sql.go
64.30% <0.00%> (-0.10%)
:arrow_down:

pkg/fs/fs.go
76.57% <0.00%> (+0.84%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Netcore 2.0.0, Type Providers, FSC Stack Overflow - Assembly Load Infinite Loop

When compiling using dotnet, I am running into a stack overflow exception. It appears to be related to the FSharp.Data.SqlProvider package, but regardless of what a 3rd party library does, I see this as a bug in fsc.exe
I understand that fsharp does not quite yet support type providers in netcore, I hope that this issue can help to work towards it working. Side note, could someone provide instructions to getting netcore fsc working on 2.0.0? I would attempt to troubleshoot further and PR this if I could compile the compiler.
Repro steps

Download FSharp.Data.SqlProvider.dll from nuget, and place it in an accessible location.
Create a file named test.fs that just has module x\rlet x = "" or some such.

Run the following command:
dotnet /usr/share/dotnet/sdk/2.0.0/FSharp/fsc.exe --noframework -r:/usr/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.0/build/netstandard2.0/ref/mscorlib.dll -r:/usr/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.0/build/netstandard2.0/ref/System.Runtime.dll -r:/usr/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.0/build/netstandard2.0/ref/netstandard.dll -r:/home/xenocons/.nuget/packages/sqlprovider/1.1.11/lib/FSharp.Data.SqlProvider.dll test.fs
Related information
Provide any related information

Debian 9
.NET Core 2.0.0

Stacktrace
FSC doesnt seem to produce a stacktrace, so I used lldb to get the below, this seems to repeat constantly and chew up the stack, it looks like perhaps assembly resolution is occurring in an endless loop.
00007FFFFFFEAC58 00007ffff6344f07 [HelperMethodFrame_PROTECTOBJ: 00007ffffffeac58] System.Reflection.RuntimeAssembly._nLoad(System.Reflection.AssemblyName, System.String, System.Security.Policy.Evidence, System.Reflection.RuntimeAssembly, System.Threading.StackCrawlMark ByRef, IntPtr, Boolean, Boolean, Boolean, IntPtr)
00007FFFFFFEAD90 00007FFF7CB2606E System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(System.Reflection.AssemblyName, System.Security.Policy.Evidence, System.Reflection.RuntimeAssembly, System.Threading.StackCrawlMark ByRef, IntPtr, Boolean, Boolean, IntPtr)
00007FFFFFFEADF0 00007FFF7CB52B48 System.Reflection.Assembly.Load(System.Reflection.AssemblyName, IntPtr)
00007FFFFFFEAE30 00007FFF7CBB14A4 System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyName(System.Reflection.AssemblyName)
00007FFFFFFEAE40 00007FFF7E6C0091 Microsoft.FSharp.Core.ReflectionAdapters+CustomAssemblyResolver.Load(System.Reflection.AssemblyName)
00007FFFFFFEAE50 00007FFF7CBB1400 System.Runtime.Loader.AssemblyLoadContext.ResolveUsingLoad(System.Reflection.AssemblyName)
00007FFFFFFEAE80 00007FFF7CBB123C System.Runtime.Loader.AssemblyLoadContext.Resolve(IntPtr, System.Reflection.AssemblyName)
00007FFFFFFEB3C0 00007ffff6344f07 [GCFrame: 00007ffffffeb3c0]

Before the endless loop we see this:
00007FFFFFFFBD78 00007ffff6344f07 [InlinedCallFrame: 00007fffffffbd78] System.Reflection.RuntimeAssembly.GetExportedTypes(System.Reflection.RuntimeAssembly, System.Runtime.CompilerServices.ObjectHandleOnStack)
00007FFFFFFFBD78 00007fff7cc54b22 [InlinedCallFrame: 00007fffffffbd78] System.Reflection.RuntimeAssembly.GetExportedTypes(System.Reflection.RuntimeAssembly, System.Runtime.CompilerServices.ObjectHandleOnStack)
00007FFFFFFFBD70 00007FFF7CC54B22 DomainNeutralILStubClass.IL_STUB_PInvoke(System.Reflection.RuntimeModule, System.Runtime.CompilerServices.ObjectHandleOnStack)
00007FFFFFFFBE00 00007FFF7CB25BDA System.Reflection.RuntimeAssembly.GetExportedTypes()
00007FFFFFFFBE20 00007FFF7E4E874E Microsoft.FSharp.Compiler.ExtensionTyping.GetTypeProviderImplementationTypes(System.String, System.String, range)
00007FFFFFFFBE80 00007FFF7E4EED87 Microsoft.FSharp.Compiler.ExtensionTyping+GetTypeProvidersOfAssembly@160.GenerateNext(System.Collections.Generic.IEnumerable`1<System.Tuple`2<Microsoft.FSharp.Core.CompilerServices.ITypeProvider,ILScopeRef>> ByRef)
00007FFFFFFFBEF0 00007FFF7EE86025 Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1[[System.__Canon, System.Private.CoreLib]].MoveNextImpl()
00007FFFFFFFBF40 00007FFF7EE861AD Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1[[System.__Canon, System.Private.CoreLib]].System-Collections-IEnumerator-MoveNext()
00007FFFFFFFBF50 00007FFF7EF2807C Microsoft.FSharp.Collections.SeqModule.ToList[[System.__Canon, System.Private.CoreLib]](System.Collections.Generic.IEnumerable`1<System.__Canon>)
00007FFFFFFFBFA0 00007FFF7E4E8C9F Microsoft.FSharp.Compiler.ExtensionTyping.GetTypeProvidersOfAssembly(System.String, ILScopeRef, System.String, ResolutionEnvironment, Boolean, Boolean, Microsoft.FSharp.Core.FSharpFunc`2<System.String,Boolean>, System.Version, range)
00007FFFFFFFC050 00007FFF7E1F83FF Microsoft.FSharp.Compiler.CompileOps+providers@4153.GenerateNext(System.Collections.Generic.IEnumerable`1<Microsoft.FSharp.Collections.FSharpList`1<Microsoft.FSharp.Compiler.Tainted`1<Microsoft.FSharp.Core.CompilerServices.ITypeProvider>>> ByRef)

Which the plot thickens.. GetTypeProviderImplementationTypes, which leads us to https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/ExtensionTyping.fs#L41 however I cannot actually see any source for an infinite recursion.

Pretty sure I know where this bug is and I know the fix.  I didn't know it also impacts TP's but now that I think about, I can see that it would.

@KevinRansom Yes, this has been blocking loading any TPs into .NET Core.  If you know the fix please let us know what to do...

I will submit a PR .... it's the same reason that FSI doesn't work on coreclr 2.0.  I have to figure out how to do mutually exclusive things.  But It shouldn't be hard.

fixed trying to figure out why we cant get the TypeInfo()

@xenocons
That type provider is compiled with a desktop FSharp.Core.dll.  To run cross platform on the coreclr it needs to be built with the coreclr version of FSharp.Core.dll.
If you have a repro of the failure using a correctly built tp I will take another look.
Note that the FSharp.Core has a reference to mscorlib, system.core, system and system.numerics.  While the TP references netstandard etc.
The correct FSharp.core.dll has references to system.runtime etc ...

thanks
Kevin


fix: changes necessary for semantic-release publishing

Summary
c3252c9 chore: reorder entries in package.json
4466df1 fix: add "publishConfig": { "access": "public" }
7305be1 fix: "plugins" section with @semantic-release/git specified in order
df839f8 fix: yarn add --dev @semantic-release/git
f62f5ff chore: use ${POLYMATH_NPM_TOKEN} in .npmrc

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Machine Learning with Sea Floor Sampling Video

Project Description:
Traditionally, surverys of the sea floor are conducted via vessel-mounted cameras which record video as the vessel moves in the water. Hundreds of hours of video are recorded and are often manually processed to determine which species are present on the locations in the video. This project seeks to automate the process using image processing. The intern will help prepare machine learning models, such as artificial neural networks, using available video footage from benthic surverying missions. The intern will partner with biology staff and software staff to train the model and perform data validation.
Expected Outcomes: A capable machine learning model which can be used to identify species from video transect data of the sea floor.
Skills required:
Familiarity with a programming language (Python, R) and a general understanding of how machine learning models operate. Experience with image processing is a bonus.
Difficulty:
Moderately difficult
Mentor(s):
@daltonkell Dalton Kell (Software Engineer), Matt Iannucci (Software Engineer), Tara Franey (GIS Specialist), Stephanie Berkman (Biologist), Joe Zottoli (Biologist)

@daltonkell are we looking at coral reefs specifically or are we going for all the species that are present ?

Hi @lohithmunakala, we're mostly looking at the very numerous species that were found, like sea stars and sand dollars.

Hey @daltonkell, I had looked at VIAME as suggested by @/jermy on the mailing list. I had also gone through this dataset titled National Coral Reef Monitoring Program: Benthic Cover Derived from Analysis of Benthic Images Collected during Stratified Random Surveys (StRS) across the Main Hawaiian Islands
I was going through how the dataset was developed by manual segmentation of the different species found in the video timestamps.
Can you let me know if this is what we are exactly targeting? The creation of databases by automating species recognition. Since VIAME has a GUI and is a full-blown object detector, would we be focused on only marine life and a simple application that generates the database of all the marine life found in the videos?
Lastly, would you let me know the formal communication process through which we could communicate further?
Thanks!

@daltonkell , I am Aditya Agarwal completing my graduation and specialization(Artificial Intelligence and Machine Learning) from Jaypee University Of Engineering And Technology, Guna.
I wanted to work on this project because it aligns with my field of specialization and interest.
I promise you that i will be able to complete the Machine Learning Model  to identify species from video provided.
I also have interest in knowing more about the underwater aquatic species as it has been one of the favorite shows on discovery and Sony BBC Earth channel.

Hi @mwengren @daltonkell,Hope this message finds you in good health , I am Nitish Singhal, a 3rd Year Undergrad from The Northcap University , Gurgaon ,India with a specialization in Data Science . Since past 2.5 years working with python , machine learning , Deep learning and recently image processing also .
What drives me to this project is the innovative idea and its huge real life applications . It aligns with my research as well as project interests too.
It would be very helpful if you can further guide us about the dataset .
As mentioned that we have to identify species from the video footage so do we first have to do the manual segmentation to get the images or we could first try the shape identification on the video part and then get the images from footage through segmentation and then identify the species.
Please provide your valuable guidance and brief the project a little more .

Hey @mwengren, @daltonkell!
I would like to know how to make a commit/PR for this particular project as there is no existing repo for this. Would it be advisable to create a repo on my GitHub profile and proceed further?
Please help me out with this.
Thanks!

hi @mwengren @daltonkell
I'm harsh shaw from SRM chennai a second year undegraduate student , i will like to contribute to this project as it seems very interesting topic to work on even for the DSC google challenge we took a topic on marine life poaching solution ... i will like to discuss further on this topic along with the mentors below i have attached my linkedin and github account
Thank you!
Linkedin:- https://www.linkedin.com/in/harsh-shaw-070105174
Github :-https://github.com/harshshaw

Hello @daltonkell @mwengren ,
I am Simardeep Singh Sethi. I am a third-year undergraduate student pursuing Bachelors in Technology (Computer Science) from Guru Gobind Singh Indraprastha University (GGSIPU), New Delhi, India. I have a great interest in Machine Learning/Deep Learning and really passionate in learning new topics introduced in the industry. I have a good command over Python language and understand how different models and CNN architecture’s function. I have a good experience in image pre-processing, which I think would be a benefit for this project. I have worked as a deep learning intern previously at MAVOIX solution private limited and created two models, an object recognition and text recognition, for real-time detection, this feature was to be used in an Android Application for the health-tech company. Hence, I am looking forward to work on this project.
Guidance Required:

Doubt regarding the data that would be provided, will it be annotated or raw data?
 How many different species will need to be identified using the model?

@daltonkell @mwengren
I am a Pre-final year computer Engineering student studying at Pune Institute of Computer Technology. My CGPA is 9.5/10 for the first four semesters. I am excited about advances in Machine Learning and have been exploring this field lately.
I am interested in this project. I read the abstract provided on the projects page of GSOC'21.
I would like to get more insights on this topic as to what is expected for the project and would be glad to get your guidance regarding how to proceed further.
Regards,
Kunal Shah.

Hi all,
Thank you for your interest in this project! Here is some more information:
Traditionally, seafloor habitat visual characterization surveys are conducted via towed or vessel-mounted cameras which record video and still images as the vessel travels along a transect. Hundreds of hours of video are recorded and are often manually processed to determine which species and habitat are present in the video and stills. This project seeks to automate the process using image processing. Specifically, we aim to:
A.	Successfully identify and enumerate (at least very abundant) organisms larger than a minimum threshold (e.g. 4 cm).
B.	Identify the habitat down to its specific components (e.g., sand, gravel, shells, biotic) in still images and calculate percent composition.
C.	Incorporate color correction as necessary.
D.	Calculate the total area surveyed (or in each image) using parallel-mounted calibration lasers
We currently review entire videos manually for species and calculate percent bottom composition by randomly assigning 50-100 points and manually identifying the bottom type under each point to classify individual images.
An effort to make this process more efficient included training a deep learning model and then performing object detection and tracking to count unique individuals for a couple highly numerous species. Interns would be asked to help improve and expand this effort, optimally in multiple of the areas listed below:

Identification and enumeration of species:
a.	Improving object detection: by identification and use of a more suitable model, image processing, alternative training method, or other method
b.	Improve tracking: by use or generation of a more suitable tracking algorithm
c.	Expand detection to additional species
Identification of habitat down to it specific components
a.	Develop or apply a model to classify regions of the video or still frames of the video based on the specific components (e.g. sand, gravel, shells).
Detect additional information from the video, such as calibration laser points or camera-added text, in order to support the above analysis or assist in managing the resulting data.
Define and generate metrics to benchmark and assess the effectiveness of different approaches.
Optimize output tables for easy integration with reports (standardized information and format).

The process will include working with biologists and other staff to create a workflow that is easy to use, repeatable, and produces output that allows the overall aim (seafloor habitat visual characterization) to be conducted more efficiently.

Hi all,
Thank you for your interest in this project! Here is some more information:
Traditionally, seafloor habitat visual characterization surveys are conducted via towed or vessel-mounted cameras which record video and still images as the vessel travels along a transect. Hundreds of hours of video are recorded and are often manually processed to determine which species and habitat are present in the video and stills. This project seeks to automate the process using image processing. Specifically, we aim to:
A. Successfully identify and enumerate (at least very abundant) organisms larger than a minimum threshold (e.g. 4 cm).
B. Identify the habitat down to its specific components (e.g., sand, gravel, shells, biotic) in still images and calculate percent composition.
C. Incorporate color correction as necessary.
D. Calculate the total area surveyed (or in each image) using parallel-mounted calibration lasers
We currently review entire videos manually for species and calculate percent bottom composition by randomly assigning 50-100 points and manually identifying the bottom type under each point to classify individual images.
An effort to make this process more efficient included training a deep learning model and then performing object detection and tracking to count unique individuals for a couple highly numerous species. Interns would be asked to help improve and expand this effort, optimally in multiple of the areas listed below:
1. Identification and enumeration of species:
   a.	Improving object detection: by identification and use of a more suitable model, image processing, alternative training method, or other method
   b.	Improve tracking: by use or generation of a more suitable tracking algorithm
   c.	Expand detection to additional species

2. Identification of habitat down to it specific components
   a.	Develop or apply a model to classify regions of the video or still frames of the video based on the specific components (e.g. sand, gravel, shells).

3. Detect additional information from the video, such as calibration laser points or camera-added text, in order to support the above analysis or assist in managing the resulting data.

4. Define and generate metrics to benchmark and assess the effectiveness of different approaches.

5. Optimize output tables for easy integration with reports (standardized information and format).

The process will include working with biologists and other staff to create a workflow that is easy to use, repeatable, and produces output that allows the overall aim (seafloor habitat visual characterization) to be conducted more efficiently.

@daltonkell , thanks for all the information! Is there any headstart with regards to the data that will be provided for training or will it be provided only after the program starts?
A sample dataset would help get an overall idea about how to write the proposal.
Thanks!

Hi all,
Thank you for your interest in this project! Here is some more information:
Traditionally, seafloor habitat visual characterization surveys are conducted via towed or vessel-mounted cameras which record video and still images as the vessel travels along a transect. Hundreds of hours of video are recorded and are often manually processed to determine which species and habitat are present in the video and stills. This project seeks to automate the process using image processing. Specifically, we aim to:
A. Successfully identify and enumerate (at least very abundant) organisms larger than a minimum threshold (e.g. 4 cm).
B. Identify the habitat down to its specific components (e.g., sand, gravel, shells, biotic) in still images and calculate percent composition.
C. Incorporate color correction as necessary.
D. Calculate the total area surveyed (or in each image) using parallel-mounted calibration lasers
We currently review entire videos manually for species and calculate percent bottom composition by randomly assigning 50-100 points and manually identifying the bottom type under each point to classify individual images.
An effort to make this process more efficient included training a deep learning model and then performing object detection and tracking to count unique individuals for a couple highly numerous species. Interns would be asked to help improve and expand this effort, optimally in multiple of the areas listed below:

Identification and enumeration of species:
a.	Improving object detection: by identification and use of a more suitable model, image processing, alternative training method, or other method
b.	Improve tracking: by use or generation of a more suitable tracking algorithm
c.	Expand detection to additional species
Identification of habitat down to it specific components
a.	Develop or apply a model to classify regions of the video or still frames of the video based on the specific components (e.g. sand, gravel, shells).
Detect additional information from the video, such as calibration laser points or camera-added text, in order to support the above analysis or assist in managing the resulting data.
Define and generate metrics to benchmark and assess the effectiveness of different approaches.
Optimize output tables for easy integration with reports (standardized information and format).

The process will include working with biologists and other staff to create a workflow that is easy to use, repeatable, and produces output that allows the overall aim (seafloor habitat visual characterization) to be conducted more efficiently.

@daltonkell Thank you for detailed description. I have some questions on the same:

How do pictures and videos differ? Do they have different data or just two formats? Are we going to use video for species identification  and pictures for habitat components?
Do we have any existing repository with the data or any models, on top of which we want to build the improved ones?
Also, how do I share the different approach that I plan to achieve the above tasks? Should I directly send the proposal or is there a platform to discuss the same?

Thanks,
Geetika

Hey @daltonkell!
I have submitted the proposal. Please let me know any changes that you would like me to implement to make the proposal more effective.
Thanks a ton!

@daltonkell
I have submitted my proposal for the project. If you could please review the same and let me know in case any changes are required.
Thanks & Regards,
Geetika

Thanks all to have sent in proposals - we will be reviewing shortly and providing feedback. Thanks!

Does anyone have thoughts on biigle? A collegue is exploring it for her similar use case.

Closing all past GSoC issues. Please open a new issue if you want to participate in GSoC23.


BetterBurnTime v1.6.1 - Please set minimum KSP version as KSP v1.3.1

Firstly many thanks for an amazing tool. I've only recently entered the Kerbal world and already CKAN has become invaluable to me along with many mods from the modding community. THANK YOU!
Background
CKAN Version:
1.25.1
KSP Version:
1.4.2
Operating System:
Windows 10
Have you made any manual changes to your GameData folder (i.e., not via CKAN)?
No
Affected mods and mod versions:
BetterBurnTime v1.6.1 by @KSPSnark
Problem
What steps did you take in CKAN?
N/A
What did you expect to happen?
To be able to install BetterBurnTime v1.6.1 in my KSP v1.4.2 instance (without fiddling with the compatible KSP versions).
According the module changelog and mod forum, BetterBurnTime v1.6.1 was originally released for KSP v1.3.1 and has worked unchanged with all KSP releases since then.
What happened instead?
BetterBurnTime v1.6.1 is listed in CKAN as only compatible with only KSP v1.4.4 and not KSP v1.4.2 which I am running (for compatibility with specific other mods) and so is unavailable to install without overriding the compatible KSP versions.
Can you please update the minimum version for this mod in the NetKAN metadata to KSP v1.3.1?
Looking at the changelog, I suspect it would also be backwards compatible with even earlier versions too but this is pure speculation on my part.
Thank you again.
Screenshots:
N/A
CKAN error codes (if applicable):
N/A

Sorry, I have no clue how CKAN does what it does (have never used it, myself).  The sum total of my involvement with CKAN is that when I host a new mod on SpaceDock, I click the little check-box that asks me whether I want to publish on CKAN.  That's it.  Whatever happens after that point is handled by automatic machinery that's invisible to me.
I can see how this issue would be vexing to someone who mainly uses CKAN, and I'm sorry about that, but since I don't maintain CKAN myself (and don't really have the bandwidth to get involved with it), I'm afraid this one's not something I expect to fix.  Thank you for the kind words, and thank you for taking the time to report the matter (and for making the effort to be detailed and precise about it)-- I'm sorry I can't be of more help.
FWIW, BetterBurnTime doesn't have any dependencies on any other mods and is super easy to install manually.  Just follow the instructions on it and unzip it into your GameData folder.

Hi, @KSPSnark! There are ways of solving this that involve you and ways that don't. If you were interested, a KSP-AVC file for this mod could effectively announce the game version range that you officially support (currently drawn solely from the single version on SpaceDock). The advantage of this is that CKAN would always reflect your exact intentions, for this version and all future versions, but obviously it's more maintenance for you.
Alternatively, with no changes on your side we can override the versions supported to something like this ("1.4" matches all 1.4.x game versions):
    "ksp_version_min": "1.3",
    "ksp_version_max": "1.4",

This might be worth doing, but it presents two problems. First, it might not reflect your intentions (you might want to test on future 1.4.x versions before announcing compatibility). Second, if you someday update the game version on SpaceDock to be 1.5.0 or later, we would have to come back and update the override to reflect that, and there wouldn't be an automated notification to alert us.
The third option, of course, is to say "too bad" and leave it as-is, since this dilemma is not at all unique to this mod. Users have the option to configure additional game versions to consider compatible in the client app, which would allow this and other mods to be installed:
https://github.com/KSP-CKAN/CKAN/wiki/User-guide#choosing-compatible-game-versions
I hesitate to recommend that, though, since we're talking about a 1.x level difference rather than 1.4.x level; installing 1.4-compatible mods on 1.3 may not be as safe in all instances as it is for BBT.
Just some thoughts to fill in the background here.

@KSPSnark I would be happy to help  you with the KSP-AVC file.  Once it is set up, all you would have to do is to update the .version file when you release an update.
Let me know

A giant thank you to all who responded, I'm really impressed with what I have seen in the KSP modding community. I may even join in myself once I've played the game more (I'm on my first career mode playthrough right now).
@KSPSnark, I mainly linked you as a polite FYI that there is something happening regarding your mod rather than expecting you to do anything - and why I logged the issue against NetKAN rather than your BetterBurnTime. I totally understand mod authors with limited time who won't want to get involved with a mod management tool they don't use themselves. That said if you did have the time and inclination (and with no expectation on my part), it seems that adding a KSP-AVC file to your mod with the KSP versions supported would be relatively simple and well appreciated by CKAN users as well as users of the KSP Addon Version Checker plugin.
Thanks @HebaruSan and especially @linuxgurugamer for jumping in so promptly and for all you do to support the KSP modding community. I've already noticed this in my limited time getting up to speed with KSP and the various mods out there.


How i can set cropping grid points mannually

@lykhonis Please help

Check this https://www.youtube.com/watch?v=1aRwxnX5nms


#|exports directive does not show source code in the docs

nbdev version 2.1.3 from pypi
Issue:
Using #|exports directive doesn't include cell source code in the generated docs. It does export cell code to python module.
Expected behavior:
#|exports directive exports cell code to .py file and includes the cell code in the generated docs.

This issue also applies to the conda version.


python 2.7.9+ and SSL context

URLLib2 breaks if you have SSL without a legitimate cert in Python 2.7.9. To disable SSL verification do the following:
import ssl
then in the get_data section define:
sslcontext = ssl._create_unverified_context()
I put this before the "tries" section. Then change the urllib request to:
response = json.loads(urllib2.urlopen(request, context=sslcontext).read())
I'm guessing maybe add a flag for this?
Thanks for your code.

I use the following validation:
if sys.version.split(' ')[0] == '2.7.9':
    ssl._create_default_https_context = ssl._create_unverified_context


fix(kibbeh/public/locales/fr/translation.json): fix old translations …

…and add new
I fixed some older french translations and added some new ones.

please follow the proper pattern for commit messages (💡TIP: you can use yarn commit or yarn commit:signed and answer the questions asked to generate a properly formatted one for you!)
to fix your most recent commit you can run the command git commit --amend -m "properly formatted commit message here"
to fix your older/multiple commits please take a look at this article


Consider removal of dependency quarkus-container-image-jib

The artifact quarkus-operator-sdk currently has a compile-time dependency on io.quarkus:quarkus-container-image-jib. Is this necessary or could it at most be marked optional?

I've been thinking about trimming dependencies, yes. The initial idea was to have a "batteries-included, opinionated" extension but that might be causing more issues than really helping :)
Do you have a particular issue with jib being there at compile-time?

Do you have a particular issue with jib being there at compile-time?

No technical issue. It's mostly dealing with limiting dependencies on our side as well. Before adding an exclude in our POM, I figured started here first :-)


Create comb_sort.c

Description
This is a simple program of comb sort in C language for hacktoberfest 2020.
Checklist:

My code follows the style guidelines of this project.
I have added my code in its respective C language folder.
[x] I have mentioned a short description about the code at the beginning of the program.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generate no new warnings.
[x] I have not deleted any files that already exist in the repository.

this is a simple program of comb sort in c language.


Update dependencies and prepare for exactOptionalPropertyTypes option

This updates our dependencies and prepares our codebase for the case
when users want to use the new exactOptionalPropertyTypes compiler
option.
Surprisingly there was just one place where I needed to change something.

we don't use a specific action, I think you'll just need to update to typescript 4.4 in the package.json to make this work.


Turing Machine uses off-beat arrow, not in most fonts.

Students trying to build Turing Machines may find that the transition labels don't look right as the state arrows don't render correctly.  The arrow is U+1F826 ”Rightwards triangle-headed arrow with medium shaft”.  It renders for me only because I have the font Julia Mono installed.
This patch fixes the issue by using a similar-looking but broadly supported arrow:
diff --git a/javascript/automata/turing/turingTransition.js b/javascript/automata/turing/turingTransition.js
index 3fffd39..6904961 100644
--- a/javascript/automata/turing/turingTransition.js
+++ b/javascript/automata/turing/turingTransition.js
@@ -56,7 +56,7 @@ class TuringTransition extends PushdownTransition {
                const element = $(`

                 <input type="text" spellcheck="false" maxlength="1" class="label-input read-input tuple-input">
-
🠦

+
→

                 <input type="text" spellcheck="false" maxlength="1" class="label-input write-input tuple-input">

,

                 <input type="text" spellcheck="false" maxlength="1" class="label-input move-input tuple-input">

Thank you for catching this. This is a big oversight on my part, it actually effects Pushdown Automata as well. I will put out a fix shortly.

Fixed in this commit: https://github.com/ElijahCirioli/jsflap/commit/d41b5c6e1699020771460f70c06469b75f62e86a
I don't really like the look of those arrow characters (which may be why I used this uncommon character in the first place), so I used the font awesome arrows that are used elsewhere in the UI.


[Automation] Generate Fluent Lite from Swagger hybridkubernetes#package-2021-10-01

[Automation] Generate Fluent Lite from Swagger hybridkubernetes#package-2021-10-01

API change check
API changes are not detected in this pull request.


Change failed backup list

If triggeredBackupCount - backupSuccessCount is less than zero (ie. on demand backups are present) then set backupFailureCount = backupResults.length - backupSuccessCount

Pull Request Test Coverage Report for Build 924

4 of 4 (100.0%) changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.01%) to 87.418%

Files with Coverage Reduction
New Missed Lines
%

lib/fabrik/DBManager.js
1
98.84%

Totals

Change from base Build 921:
-0.01%

Covered Lines:
6120

Relevant Lines:
6763

💛  - Coveralls


Will you plan to release fast RNN (LSTM), along bindings for torch?

Really thanks for sharing. But the warp-ctc is a small part of the algorithm ( I mean end-to-end asr system), without the RNN part, it is really hard to use it. To really ease the pain, will you plan to release the RNN implementation?

We are not planning on releasing a full end-to-end ASR system as part of this project.


Review and fix use of heading elements

Review and fix use of heading elements, particularly adding an H1 to all pages (likely as the “LegalNavigator” text at the top of the page).

@KellyeDC Rough Estimation 12-24 hours: we need a list of places to change, because it is difficult to remember all the places, and not everywhere they come from the Data Base

@yevhenobertas Please make all updates for both static and state content where applicable.  Once completed, the PO will review and we will identify any places that may have been missed.

@KellyeDC @grawdon As we previously discussed could you please define the particular places where we should review and fix use of heading elements? And what should be added?

@ozholob
Glenn Rawdon  6:31 PM
@Yevhen Obertas @Kellye As I understand it, the h1 tag is the most important heading because it’s the highest level tag that shows what your specific page is about. Search engines generally give this tag more weight over other headings, so it usually improves your search engine ranking when you use it correctly. Every page in LN should have an h1 tag and it should be the “LegalNavigator” text at the top of the page. You can search for these in the source code. I checked a couple of our pages and searched for "h1" and it didn't occur on the page. It needs to be added to each page of the site. I found a video on this if it helps. Please review this response and the video and let me know if you still have questions on what is needed. https://www.verticalmeasures.com/resources/seo/seo-101-tutorial-videos/using-h1-tag-improves-search-engine-ranking/#:~:text= To do so%2C follow these steps%3A ,h2%2C h3%2C h4%2C h5 and h6 More

@KellyeDC if we are talking about this elment:

we could not change it by h1 tag - since this element is an icon while h1 is applicable only to text.


Surround xpath expressions with backticks

The mix of or and | looks very confusing, until you understand that | is part of an xpath expression and or is part of a foxpath expression. Surrounding all xpath expressions with backticks might make these expressions a bit easier for the reader to parse.

Closing since this is now part of #14.


Fix AAT for scoringStrepla

In its present version, opensoar is not able to properly handle AAT task from scoringStrepla as the 'LSCSA0' lines are not read. Additionally, there was a little bug in task_info['time_window'] (datetime.time instead of datetime.timedelta). This pull request should fix the issues with the AAT for scoringStrepla. Maybe you could also add a linebreak in line 20 that I forgot.

Thanks for your contribution 👍
I don't have the time to look at it right now, but will do so soon.
Can you add an entry to the changelog?

also: do you have an example competition we can use to test this with?

somehow, the previous commit did not show the CI.
there was some errors with old tests. fixed those in the last commit, together with an added test running from a file

@jkretz: thanks for your contribution 👍


Introduction to API monitoring and performance

Topic Suggestion
Introduction to API monitoring and performance
Proposed article introduction
APIs are a powerful tool that helps software engineers to build web and mobile applications by allowing them to communicate with one another. APIs are used to get, send and exchange data from different web services.
API monitoring is the practice of tracking an API's availability, operability,  downtime, and its performance to verify that its available, functional, and within performance limits.
Key takeaways

Why API monitoring is important
What is an API monitoring service
How to minimize API downtime
How does API monitoring work
How to choose the right API monitoring service

Article quality
This article will give an in-depth lesson on API performance and monitoring, as well as best practices and technologies that may help developers, companies, and teams monitor their mission-critical APIs quickly and affordably.
References
Please list links to any published content/research that you intend to use to support/guide this article.
Conclusion
Finally, remove the Pre-Submission advice section and all our blockquoted notes as you fill in the form before you submit.
We look forwarding to reviewing your topic suggestion.

Good afternoon and thank you for submitting your topic to the EngEd program.
After some careful consideration, it struck us that this topic may be a bit over-saturated throughout other blog sites and official documentations.
We typically refrain from publishing content that is covered widely on the net or other blogs. We're more interested in original, practitioner-focused content that takes a deeper dive into programming-centric concepts.
But in order to approve the topic, it has to serve value to the larger developer community at large.
An option and a great way to write this as an in-depth article and make it more add value to the greater developer community at large would be to walk the reader through the USE of methods and functions by building a unique, different, useful project.

We would love to learn your thought process behind the solution you arrived at using these concepts and topics.

That way a developer could see them in action.
As mentioned above - we believe this topic is widely covered on other blog sites.
The best way for students to build a great portfolio is by building what does not exist and what can provide the most value.


Fetch from config object instead of process.env

dotenv.config returns a parsed object containing all env variables. While testing, fetching from process.env seemed to not work as the Insomnia environment changed.
See demo workspace here to test with.

Note you will need to direct the envFilePath variable at env files on your file system. These are the .env files I have on my file system for testing.
Using process.env, note values do not change:

Using config object:

@develohpanda Thank you. This is amazing!!!


Crash - Socket2Me._openNetworkSocket() EXC_BAD_ACCESS (KERN_INVALID_ADDRESS)

Socket2Me._openNetworkSocket()
EXC_BAD_ACCESS (KERN_INVALID_ADDRESS)
Crashed: Thread
0  CoreFoundation                 0x81968 _CFGetNonObjCTypeID + 92
1  CoreFoundation                 0xd44ac CFSocketCreateRunLoopSource + 76
2  Saverone                       0x13505c Socket2Me._openNetworkSocket() + 148 (Socket2Me.swift:148)
3  Saverone                       0x134dc4 @objc Socket2Me.createSocketDetached() + 75 (Socket2Me.swift:75)
4  Foundation                     0x5b808 NSThread__start + 716
5  libsystem_pthread.dylib        0x16cc _pthread_start + 148
6  libsystem_pthread.dylib        0xba4 thread_start + 8

Hi, it's still crashing sometimes in my app. What more information can I provide to help you understand how to fix it?

I still see a lot of threads being created in the background but I'm not reproducing it anymore when I test it. How are you calling start? Is it on the main thread or via another context?

Can you provide the output of the debugger command 'bt all' ?

you can see in this video the crash flow.
https://youtu.be/6meD0aMVjVw

why used with "try?" and not "try catch" ?

Using try catch will not stop the crash from happening because there is a pointer being dereferenced to an invalid location in memory. Somehow there is a fundamental misuse of the pinger because there should not be more than one background thread running at any one time. Can you guarantee that you are calling stopPinging before calling start again?

I always call 'stop' before I made a 'start'.

That looks great but still doesn't explain the multiple threads being started. How is your Pinger.start() getting called? Is it from a system callback like application delegate or notification?

it called from :
public func peripheralManager(_ peripheral: CBPeripheralManager, didReceiveWrite requests: [CBATTRequest]) {
}
Not all the time, only if I get something that tells me to start to ping.

I'm still not 100% sure what is happening but I've made a change that will now force the background thread to exit on ping stop. Give it a try and let me know if you are still seeing the crash in your app.

Hi, it's crashing again but in another place

Try setting the pinglet property to nil in the Pinger.stop() function and see if the crash goes away. If it does this points to a hidden retain cycle somewhere.

Okay, I set the 'pinglet' property to nil in the Pinger.stop() method and will let you know if something changes. Thanks."

its still crash
Crashed: Socket2Me internal utility
0  libobjc.A.dylib                0x2e5c objc_retain_x0 + 16
1  libobjc.A.dylib                0x2e5c objc_retain + 16
2  Saverone                       0x18640c closure #1 in Socket2Me.send(data:) + 216 (Socket2Me.swift:216)
3  Saverone                       0x186588 thunk for @escaping @callee_guaranteed @Sendable () -> () +<PHONE_NUMBER> (:4380370312)
4  libdispatch.dylib              0x24b4 _dispatch_call_block_and_release + 32
5  libdispatch.dylib              0x3fdc _dispatch_client_callout + 20
6  libdispatch.dylib              0xb694 _dispatch_lane_serial_drain + 672
7  libdispatch.dylib              0xc1e0 _dispatch_lane_invoke + 384
8  libdispatch.dylib              0x16e10 _dispatch_workloop_worker_thread + 652
9  libsystem_pthread.dylib        0xdf8 _pthread_wqthread + 288
10 libsystem_pthread.dylib        0xb98 start_wqthread + 8

Can you describe how you are using (initing/object lifecycle) the pinglet object? Are you using any of the callbacks or publishers to get notified when pings come in? I cannot tell from the code that you sent but it looks like they're not being used? I've tried testing with/without callbacks in various scenarios and I'm not able to reproduce this issue yet and I've not gotten any crash reports for my app PingPoint. I want to make sure that it an work for your use case.
I'd consider going back to using the serial queue instead of the global() queue from your example code and see if that has any effect on the crashing.
let serial queue = DispatchQueue (label: "Pinger serial queue", qos: .background)

If it does then I'll need to put some code in to ensure that we're only using the user designated queue in a serialized way.

I went ahead and added that change to 0add1ae2. Try using the latest commits and let me know if that fixes your crash.

Hi Kevin,
According to the issue in GitHub, I have added this crash. If you need more details, let me know.

If you need any further assistance, feel free to ask!
<EMAIL_ADDRESS>From: Kevin Ross @.>
Date: Monday, 3 June 2024 at 19:53
To: CocoaBeans/Pinglet @.>
Cc: Roee Nadam @.>, Comment @.>
Subject: Re: [CocoaBeans/Pinglet] Crash - Socket2Me._openNetworkSocket() EXC_BAD_ACCESS (KERN_INVALID_ADDRESS) (Issue #2)
I went ahead and added that change to 0add1ae2https://github.com/CocoaBeans/Pinglet/commit/0add1ae2e745db74496df34d9a5e47a0aa23ff6e. Try using the latest commits and let me know if that fixes your crash.
—
Reply to this email directly, view it on GitHubhttps://github.com/CocoaBeans/Pinglet/issues/2#issuecomment-2145694310, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGYKJ5DXKSKGPW3GAPYCJBTZFSNRZAVCNFSM6AAAAAA5BMAI3KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBVGY4TIMZRGA.
You are receiving this because you commented.Message ID: @.***>
Crashlytics - Stack trace
Application: com.Saverone.saverOne
Platform: apple
Version: 4.6 (10)
Issue: 64a13c21e816845ae46c0bb9537aee23
Session: 8a0b35b3a9a440d995868a1372edde60_DNE_0_v2
Date: Tue Jun 11 2024 13:09:57 GMT+0300 (Israel Daylight Time)
com.apple.main-thread
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  GraphicsServices               0x11a8 GSEventRunModal + 164
8  UIKitCore                      0x40a90c -[UIApplication _run] + 888
9  UIKitCore                      0x4be9d0 UIApplicationMain + 340
10 Saverone                       0x115974 main + 23 (AppDelegate.swift:23)
11 ???                            0x1beca1e4c (Missing)
Crashed: Pinglet internal
0  libobjc.A.dylib                0x1b60 objc_release_x0 + 16
1  libobjc.A.dylib                0x1b60 objc_release + 16
2  Saverone                       0x1807b4 Pinglet.sendPing(request:) + 359 (Pinglet.swift:359)
3  Saverone                       0x180b14 closure #1 in Pinglet.sendPing() + 370 (Pinglet.swift:370)
4  Saverone                       0x17cf3c thunk for @escaping @callee_guaranteed @Sendable () -> () +<PHONE_NUMBER> (:4373892924)
5  libdispatch.dylib              0x213c _dispatch_call_block_and_release + 32
6  libdispatch.dylib              0x3dd4 _dispatch_client_callout + 20
7  libdispatch.dylib              0xb400 _dispatch_lane_serial_drain + 748
8  libdispatch.dylib              0xbf30 _dispatch_lane_invoke + 380
9  libdispatch.dylib              0x16cb4 _dispatch_root_queue_drain_deferred_wlh + 288
10 libdispatch.dylib              0x16528 _dispatch_workloop_worker_thread + 404
11 libsystem_pthread.dylib        0x4934 _pthread_wqthread + 288
12 libsystem_pthread.dylib        0x10cc start_wqthread + 8
com.apple.uikit.eventfetch-thread
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  Foundation                     0xc7e4c -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
8  Foundation                     0xc7c9c -[NSRunLoop(NSRunLoop) runUntilDate:] + 64
9  UIKitCore                      0x41e640 -[UIEventFetcher threadMain] + 420
10 Foundation                     0xde718 NSThread__start + 732
11 libsystem_pthread.dylib        0x606c _pthread_start + 136
12 libsystem_pthread.dylib        0x10d8 thread_start + 8
Thread
0  libsystem_kernel.dylib         0x1590 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0x4980 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0x10cc start_wqthread + 8
com.google.firebase.crashlytics.MachExceptionServer
0  Saverone                       0x1c2f14 FIRCLSProcessRecordAllThreads + 393 (FIRCLSProcess.c:393)
1  Saverone                       0x1c32f4 FIRCLSProcessRecordAllThreads + 424 (FIRCLSProcess.c:424)
2  Saverone                       0x1d14c8 FIRCLSHandler + 34 (FIRCLSHandler.m:34)
3  Saverone                       0x1d1ccc FIRCLSMachExceptionServer + 521 (FIRCLSMachException.c:521)
4  libsystem_pthread.dylib        0x606c _pthread_start + 136
5  libsystem_pthread.dylib        0x10d8 thread_start + 8
com.apple.NSURLConnectionLoader
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  CFNetwork                      0xfdc90 _CFHostIsDomainTopLevel + 108176
8  Foundation                     0xde718 NSThread__start + 732
9  libsystem_pthread.dylib        0x606c _pthread_start + 136
10 libsystem_pthread.dylib        0x10d8 thread_start + 8
com.apple.CoreMotion.MotionThread
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  CoreFoundation                 0xc0f04 CFRunLoopRun + 64
8  CoreMotion                     0x10210 (Missing UUID e4a6f107b302327ca121c9bebacca8f4)
9  libsystem_pthread.dylib        0x606c _pthread_start + 136
10 libsystem_pthread.dylib        0x10d8 thread_start + 8
com.apple.CFSocket.private
0  libsystem_kernel.dylib         0x9474 select$DARWIN_EXTSN + 8
1  libsystem_kernel.dylib         0x9474 __select + 8
2  CoreFoundation                 0xbeb7c __CFSocketManager + 640
3  libsystem_pthread.dylib        0x606c _pthread_start + 136
4  libsystem_pthread.dylib        0x10d8 thread_start + 8
Thread
0  libsystem_kernel.dylib         0x1590 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0x4980 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0x10cc start_wqthread + 8
Socket2Me internal utility
0  libobjc.A.dylib                0x1b60 objc_release_x0 + 16
1  libobjc.A.dylib                0x1b60 objc_release + 16
2  Saverone                       0x186b9c type metadata accessor for Socket2Me +<PHONE_NUMBER> (:4373932956)
3  libswiftCore.dylib             0x3ab36c _swift_release_dealloc + 56
4  libswiftCore.dylib             0x3ac488 bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1> >::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 136
5  libsystem_blocks.dylib         0x21b0 _call_dispose_helpers_excp + 48
6  libsystem_blocks.dylib         0x2164 _Block_release + 252
7  libdispatch.dylib              0x3dd4 _dispatch_client_callout + 20
8  libdispatch.dylib              0xb400 _dispatch_lane_serial_drain + 748
9  libdispatch.dylib              0xbf30 _dispatch_lane_invoke + 380
10 libdispatch.dylib              0x16cb4 _dispatch_root_queue_drain_deferred_wlh + 288
11 libdispatch.dylib              0x16528 _dispatch_workloop_worker_thread + 404
12 libsystem_pthread.dylib        0x4934 _pthread_wqthread + 288
13 libsystem_pthread.dylib        0x10cc start_wqthread + 8
Crashlytics - Stack trace
Application: com.Saverone.saverOne
Platform: apple
Version: 4.6 (10)
Issue: 84f5c9b8299520a843cdf77551b21072
Session: aa174245d55144e38022490c5f0e0f1e_DNE_0_v2
Date: Fri Jun 07 2024 14:16:07 GMT+0300 (Israel Daylight Time)
com.apple.main-thread
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  GraphicsServices               0x11a8 GSEventRunModal + 164
8  UIKitCore                      0x40a90c -[UIApplication _run] + 888
9  UIKitCore                      0x4be9d0 UIApplicationMain + 340
10 Saverone                       0x115974 main + 23 (AppDelegate.swift:23)
11 ???                            0x1bcb09e4c (Missing)
com.apple.uikit.eventfetch-thread
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  Foundation                     0xc7e4c -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
8  Foundation                     0xc7c9c -[NSRunLoop(NSRunLoop) runUntilDate:] + 64
9  UIKitCore                      0x41e640 -[UIEventFetcher threadMain] + 420
10 Foundation                     0xde718 NSThread__start + 732
11 libsystem_pthread.dylib        0x606c _pthread_start + 136
12 libsystem_pthread.dylib        0x10d8 thread_start + 8
com.google.firebase.crashlytics.MachExceptionServer
0  Saverone                       0x1c2f14 FIRCLSProcessRecordAllThreads + 393 (FIRCLSProcess.c:393)
1  Saverone                       0x1c32f4 FIRCLSProcessRecordAllThreads + 424 (FIRCLSProcess.c:424)
2  Saverone                       0x1d14c8 FIRCLSHandler + 34 (FIRCLSHandler.m:34)
3  Saverone                       0x1d1ccc FIRCLSMachExceptionServer + 521 (FIRCLSMachException.c:521)
4  libsystem_pthread.dylib        0x606c _pthread_start + 136
5  libsystem_pthread.dylib        0x10d8 thread_start + 8
com.apple.NSURLConnectionLoader
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  CFNetwork                      0xfdc90 _CFHostIsDomainTopLevel + 108176
8  Foundation                     0xde718 NSThread__start + 732
9  libsystem_pthread.dylib        0x606c _pthread_start + 136
10 libsystem_pthread.dylib        0x10d8 thread_start + 8
com.apple.CoreMotion.MotionThread
0  libsystem_kernel.dylib         0x1808 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x5008 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x4f20 mach_msg_overwrite + 436
3  libsystem_kernel.dylib         0x4d60 mach_msg + 24
4  CoreFoundation                 0x53f5c __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x53600 __CFRunLoopRun + 1208
6  CoreFoundation                 0x52cd8 CFRunLoopRunSpecific + 608
7  CoreFoundation                 0xc0f04 CFRunLoopRun + 64
8  CoreMotion                     0x10210 (Missing UUID e4a6f107b302327ca121c9bebacca8f4)
9  libsystem_pthread.dylib        0x606c _pthread_start + 136
10 libsystem_pthread.dylib        0x10d8 thread_start + 8
com.apple.CFSocket.private
0  libsystem_kernel.dylib         0x9474 select$DARWIN_EXTSN + 8
1  libsystem_kernel.dylib         0x9474 __select + 8
2  CoreFoundation                 0xbeb7c __CFSocketManager + 640
3  libsystem_pthread.dylib        0x606c _pthread_start + 136
4  libsystem_pthread.dylib        0x10d8 thread_start + 8
Thread
0  libsystem_kernel.dylib         0x1590 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0x4980 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0x10cc start_wqthread + 8
Thread
0  libsystem_kernel.dylib         0x1590 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0x4980 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0x10cc start_wqthread + 8
Crashed: Socket2Me internal utility
0  libobjc.A.dylib                0x3020 objc_msgSend + 32
1  Saverone                       0x18640c closure #1 in Socket2Me.send(data:) + 216 (Socket2Me.swift:216)
2  Saverone                       0x186588 thunk for @escaping @callee_guaranteed @Sendable () -> () +<PHONE_NUMBER> (:4341507464)
3  libdispatch.dylib              0x213c _dispatch_call_block_and_release + 32
4  libdispatch.dylib              0x3dd4 _dispatch_client_callout + 20
5  libdispatch.dylib              0xb400 _dispatch_lane_serial_drain + 748
6  libdispatch.dylib              0xbf30 _dispatch_lane_invoke + 380
7  libdispatch.dylib              0x16cb4 _dispatch_root_queue_drain_deferred_wlh + 288
8  libdispatch.dylib              0x16528 _dispatch_workloop_worker_thread + 404
9  libsystem_pthread.dylib        0x4934 _pthread_wqthread + 288
10 libsystem_pthread.dylib        0x10cc start_wqthread + 8
Thread
0  libsystem_kernel.dylib         0x1590 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0x4980 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0x10cc start_wqthread + 8
Thread
0  libsystem_kernel.dylib         0x1590 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0x4980 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0x10cc start_wqthread + 8

Thank you for the detailed crash report! I think I might have fixed it by forcing a copy of the data read from the socket before passing it to the callback. Let me know if this change appears to fix the crash.

OK, great.
But why do I get v1.0.3 when updating with Xcode SPM?

I hadn't made a new release for this code change until I had verification that it fixed your issue but I went ahead and tagged the release so now it should display in Xcode as v1.0.5 when updated.

or is you can give me new branch for this fix and I tagged Xcode spm to this branch and check this for you.

on 1.0.5 the app crash.

Is it the same crash and stack trace as before?

Hi I've got a few questions to help debug and reproduce what is going on:

Are you calling your Pinger.start() method on the main thread?
What ping interval are you using in your configuration?
Do any of the other threads in the rest of the stack traces contain anything else that has any Pinglet or Socket2Me symbols in it?
Is your app in the background when the crash happens?

I have a few more defensive changes on the way but I want to see if I can reproduce it to guarantee that the problem is fixed.

its still happend for me, in the foreground after 1 hour of pinging.

still have a crash.

com.Saverone.saverOne_issue_0c10daf429585ad1326ebc5c77cf39af_crash_session_4649cf5bcb84433da3a7772e974b167b_DNE_0_v2_stacktrace.txt


Optimization tmp fix

This solves the model optimization problems for now. Although I'd like to solve this more elegantly later.

Codecov Report

:exclamation: No coverage uploaded for pull request base (demo_init@5cbf0ea). Click here to learn what that means.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             demo_init       #5   +/-   ##
============================================
  Coverage             ?   99.17%
============================================
  Files                ?        6
  Lines                ?      362
  Branches             ?        0
============================================
  Hits                 ?      359
  Misses               ?        3
  Partials             ?        0

Impacted Files
Coverage Δ

GPflowOpt/acquisition.py
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5cbf0ea...58a41d3. Read the comment docs.


add permalink to entity->schema/details, sample data, lineage

Users would like to share the schema or lineage or sample data , profilet etc.. links with others
https://sandbox-beta.open-metadata.org/pipeline/sample_airflow.dim_product_etl we are not showing which tab they are on in the URL. we should add /schema, /profile, /lineage at the end such they are sharable and bookmarkable.

assign this to me

@ShaileshParmar11 assigned to you.


Add grpc proxy backend

This adds a proxy backend that uses the grpc protocol. When the size of the blobs are unknown (i.e. when using the bazel-remote is used with the REST api), the proxy relies on the asset api to get the blob size, so in order to use another bazel-remote instance as backend --experimental_remote_asset_api must be provided to the proxy.
Tested mostly with the added unit test and some local testing done building bazel-remote itself. These are the step used to test locally (each command executed in a different terminal)
$ rm -rf /tmp/proxy && bazel run //:bazel-remote -- --dir /tmp/proxy -max_size 5 -port 8080 -grpc_port 9090 --storage_mode uncompressed --experimental_remote_asset_api --tls_cert_file=/tmp/certs/server/cert.crt --tls_key_file=/tmp/certs/server/key.key --tls_ca_file=/tmp/certs/client/cert.crt
$ rm -rf /tmp/cache1 && bazel run //:bazel-remote -- --dir /tmp/cache1 --max_size 5 -port 8081 -grpc_port 9091 --storage_mode uncompressed --grpc_proxy.url localhost:9090 --grpc_proxy.cert_file=/tmp/certs/client/cert.crt --grpc_proxy.key_file=/tmp/certs/client/key.key
$ rm -rf /tmp/cache2 && bazel run //:bazel-remote -- --dir /tmp/cache2 --max_size 5 -port 8082 -grpc_port 9092 --storage_mode uncompressed --grpc_proxy.url localhost:9090 --grpc_proxy.cert_file=/tmp/certs/client/cert.crt --grpc_proxy.key_file=/tmp/certs/client/key.key
$ bazel clean && bazel build //:bazel-remote --disk_cache= --remote_cache=grpc://localhost:9091 --remote_upload_local_results && bazel clean && bazel build //:bazel-remote --disk_cache= --remote_cache=grpc://localhost:9092 --remote_upload_local_results --remote_accept_cached

Hi @mostynb, any blocker on merging this?

HI, sorry this is taking me a while. I wanted to make a release before landing this, because I will probably want to make a few changes afterwards. That is done now (after a small hiccup in 2.4.2 and a rushed out 2.4.3 release).
I will try to find time to look at this again tomorrow- thanks for persevering.

@mostynb any update on this one?

I finally got back to reviewing this, sorry for the delay. I will continue tomorrow.

I landed a squashed version of this- I think it will be easier to iterate on this on master with a few smaller PRs than to drag this one on any longer. Thanks again for being so patient.


The gem url should honour the path as well as the host.

I've setup geminabox to respond on a subpath, http://host/gems/, I needed an extra Rack layer to do this and my reverse proxy is mapping onto the service.
This works fine with gem, but the gem plugin for uploading breaks since it is uploading to http://host/upload, throwing away the path element.
I believe it should be ok to keep the path element since the gem command can handle it for downloading.

I believe I am experiencing this issue with the latest version (0.5.2).

Is this reproducible in 0.6.0?
The HTTP pusher has been more or less completely rewritten and should be more stable.

I have not had a chance to try 0.6.0 but this might not be considered an issue anymore.  If you use the thin web server (http://code.macournoyer.com/thin/) with a --prefix switch.  If it is still useful to you, I will can try to reproduce the issue with 0.6.0 when I have time.

I got the wrong end of the stick. It sounds like the fail was inside the web server. So the requests were having their paths stripped before they hit the app.
Setting this up with the --prefix option or similar is the right way.
Cheers.