errors on exporting CityGML in 3D Database

Hi,
I use the 3DCityDB Importer Exporter tool to export my own CityGML data in the 3DCityDB database, but when the process finished, the FME Inspector show there is no geometry about the export file. I open the file with Notepad++(see figure 1), the right one should be figure 2. It looks like I miss some part during the process. I don't know which part is wrong, is the import part, the export part or the setup database part? Thanks for help

Kind regards,
Wang Benshuo

Please provide (an excerpt of) your CityGML dataset, if possible, so that we can try and reproduce the issue.

strijp-s.zip
thanks

Ok, I did the following:

Set up a new 3DCityDB 4.0 using EPSG 28992
Imported your dataset strijp-s.xml
Exported the database content into a new file

I cannot reproduce your issue. I get all data exported correctly: test.zip
Maybe you were using some filter settings on the CityGML export tab due to which no buildings were exported? Or did you receive any error messages during export?
Nevertheless, note that you CityGML dataset does not validate against the schema. Simply click the "Just validate" button on the CityGML import tab of the Importer/Exporter to get a list of the issues.

thanks, here is another problem, I want to turn the CityGML format to glTF, based on the document, I need to import CithGML and Export CityGML first. But when I export the glTF and open it with Google Earth, the location is nor right. When i import the test file and validate, it shows some errors. Therefore, are these errors cause the problem?  Thanks

Well, did a KML/COLLADA export and loaded it into Google Earth. The position of the buildings looks fine.

@FxKu is right, I used EPSG:28992 to set up the database, simply because this reference system is used in your original dataset strijp-s.xml. Make sure to not mix up reference systems.
The validation errors are mostly due to incorrect gml:id values (in your case, a gml:id must not start with a number). The 3DCityDB can correctly handle the data. However, the error should be fixed in your data.
And there are a lot of falsely oriented polygons in your dataset. If you visualize it in Google Earth, it will look like these polygons are missing, which leaves holes in the buildings. You can force their visualization by checking Ignore surface orientation in the "Preferences -> KML/COLLADA/glTF Export -> Rendering -> Building" settings (which is what I did for the above screenshot). But again, this issue should be fixed in your data in the first place.


asyncData: Recursion and promises. What's the right way?

First of all: I don't exactly know where to place this question. If wrongly placed, a hint on where to post would be nice :)
The problem:
In my API-enabled backend I have a series of post categorized by year. Uitlizing asyncData and using axios I want to fetch these posts based on the various years; thus return all posts from 2017, posts from 2016, etc. as separate items in a collected array.
I can make a call to return all available years (http://api.template-studio.nl/wp-json/wp/v2/categories?parent=19). In my theory I should then loop over the years and return the post-content year by year (http://api.template-studio.nl/wp-json/wp/v2/werkitem_nl?categories=+yearId).
 PUSH TO EMPTY ARRAY.

        var recursiveData = []

        for (var i = 0, len = res.data.length; i < len; i++) {
          var yearId = res.data[i].id
          axios.get('http://api.template-studio.nl/wp-json/wp/v2/werkitem_nl?categories=' + yearId).then((resNested) => {
            recursiveData.push([resNested.data[0].categories[0], resNested.data])
          })
        }

        // RETURN THE COLLECTED data

        return recursiveData

      })
    ])

    return {
      recursiveCategoryData: recursiveCategoryDataRes,
    }

  }

This method seems to somewhat work, but is not always returning the recursively-gathered content fully. As if the script ends before all recursive calls have been made.
Any suggestions on how to handle nested/recursive calls within asyncData?
All the best
Lasse

Use axios.all:
async asyncData () {
  const res = await axios.get('/api/categories')
  const categories = res.data

  const results = (await axios.all(
    categories.map(category => axios.get(`/api/some-resource/${category.id}`))
  ))
  .map(result => result.data)
  .reduce((acc, curr) => {
    acc.push(...curr)
    return acc
  }, [])

  console.log(results)
}

Hey! Nice. Thanks. Highly chained js! Thanks for clearing up. :+1:


Restricting visibility of specific CRDs to some namespaces

We have our operator working nicely on OCP 4. Our operator(EnMasse, AMQ Online) is a "global" operator, but not all of the CRDs makes sense to expose to all namespaces. This makes the console view a bit confusing because you can create resources that are not being acted on by the operator.
Is it possible, or will it be possible, to put restrictions in which namespaces CRDs can be created and/or be visible? For instance, we have 4 CRDs that only makes sense to create in the operator namespace, and 3 api-service backed resources that should be possible to create across the entire cluster.

@lulf

Is it possible, or will it be possible, to put restrictions in which namespaces CRDs can be created and/or be visible?

From OLMs perspective, the roles to create, edit, and view CRs for a provided API (a resource provided by a CRD or APIService) are generated implicitly as ClusterRoles which are aggregated to kube's default admin, edit, and view ClusteRoles. This means that CRUD capabilities on CRs are not tied to its OperatorGroup's namespace selection, but are instead available to any user with a RoleBinding to kube's default admin, edit, or view ClusterRoles. A view ClusterRole is also generated for the CRD which is also aggregated to the default view ClusterRole (see our docs for more details on OperatorGroup RBAC). Although we generate a view ClusterRole for the CRD, we can't actually prevent users from viewing CRDs because they are surfaced in kube's discovery regardless of any RBAC.
Currently, OLM will generate copied CSV's in the target namespaces of an OperatorGroup for each operator in its operator namespace. This is how OLM  provides information about what operators, and consequently, CRs are "available" in a target namespace.

For instance, we have 4 CRDs that only makes sense to create in the operator namespace, and 3 api-service backed resources that should be possible to create across the entire cluster.

Could you explain the use case for those 4 CRDs in more detail? It will help us determine if we are missing a valid mode of operator deployment, or if we can fit your operator into our current model in some other way.

Could you explain the use case for those 4 CRDs in more detail? It will help us determine if we are missing a valid mode of operator deployment, or if we can fit your operator into our current model in some other way.

Our operator is a global operator, i.e. we support only 1 of it across the cluster. The model is that a service admin is responsible for installing and managing this operator on the cluster. Which namespace the operator is installed to does not matter, but we have the following in the CSV:
  installModes:
  - type: OwnNamespace
    supported: true
  - type: SingleNamespace
    supported: true
  - type: MultiNamespace
    supported: false
  - type: AllNamespaces
    supported: true

We then have 4 CRDs which are only relevant to the service admin: AddressPlan, AddressSpacePlan, StandardInfraConfig and BrokeredInfraConfig
The service admin is responsible for creating instances of those CRs, which affect the operator behavior.

Then there is the developer, who is using the 3 api-service backed resources: AddressSpace, Address and MessagingUser. This user does not have access to the namespace where the operator is running, but is creating these 3 resources in order to 'self-service' without caring about what the underlying operator is doing (which is is managed and visiable only by the service-admin).
My concern then, is that in the UI, the CRDs that the service admin (or anyone with sufficient privileges) is using to manage the service, are also visible to the developer. From the developer POV, this is confusing, because those resources makes no sense for the developer to create.
So for the CLI, I understand that standard RBAC rules will allow the developer to create these resources. However, it would be nice if we could give a hint in the CSV as to which namespaces the resource should be visiblej in the UI, or alternatively give a hint on how to restrict the RBAC rules for creating/viewing the resource to a specific namespace (i.e. namespaceVisibility: installed | all)

Here is our CSV for reference: https://github.com/EnMasseProject/enmasse/blob/master/templates/csv/enmasse.clusterserviceversion.yaml

@njhale
Perhaps an even simpler explanation of their use is that the CRDs are used for configuring the operator itself, i.e their creation is not causing the operator to take action and create other resources.
The instances of these CRDs are reade when the other 3 api-service backed resources are created. So you can think of them as similar to ConfigMaps that the operator uses to configure itself, but with schema validation, custom syntax etc.

@lulf Sorry it took so long for me to get back. I think that both of your explanations are super clear and describe a totally legitimate mode of operation for an operator. We don't currently have a way to restrict access to subsets of CRs that a CSV provides. @alecmerdler may have some insight about how we might alter the UI to support hiding, but I think there should be a feature in OLM that supports your use-case.
I'm going to confer with the rest of the team about adding this as a story to our board. I will keep you posted with updates.

Closing in favor of #975 as I think it more concisely describes the requested feature. Feel free to reopen this if there's some distinction I missed.


Inter - Capital 'A' not showing correctly

Duplicate of Glitch with Exo 2 uppercase A, B, D, Å, Ä letters since version 4.4.0+, works in 4.3.0 but for Inter font:
https://github.com/fontsource/fontsource/issues/243
Inter is mentioned in the comments there but it is never addressed
Related to: https://github.com/google/fonts/issues/2602
Tried importing<EMAIL_ADDRESS>and<EMAIL_ADDRESS>without any behaviour change
Latest version "@fontsource/inter": "^4.5.1"

Jost is another font that has this problem:

I'm pretty unsure since it seems that even the variable font version of Inter is also affected by this. It's something that the upstream issue in Google has to resolve over anything I can do.

@DecliningLotus from what you know, this there anything that I can do as a developer to make sure that my site can render this letter properly?
How is this such a big global issue that is not being resolved?
Is there somewhere I can monitor the progress on the issue?

@tayloraucoin, keeping track of google/fonts#2602 is where you can monitor progress on this matter. I believe the true root of the issue is google/woff2#123.
There is a solution for this. It's embarrassing to say considering Fontsource is all about making self-hosting easy, but the official release for Inter on rsms/inter does not have the bug. You could still self-host if you download it from there. rsms/inter#302 (comment)
This is where it becomes awkward for me. I leave Google Font bugs up to Google to fix themselves and let Fontsource mirror it all. It's not in my interest to add more maintenance to the Fontsource repo by writing new code to support exceptions like these and manually patching fonts due to their sometimes incompatible build process. I just don't have the time to spare for that.
In the end, it's up to Google to sort this out.

I don't know if this helpful in any way but in case of "Montserrat" the problem does not occur when using the font directly from Google. Maybe that means it's already fixed?
I created a StackBlitz project which uses the font from Google Font.
https://web-platform-8h8bxp.stackblitz.io
https://stackblitz.com/edit/web-platform-8h8bxp
And another one which uses the font from the fontsource package. (I had to create an Angular project to be able to include the files from the node_modules directory.)
https://angular-ivy-urpsfp.stackblitz.io
https://stackblitz.com/edit/angular-ivy-urpsfp

Thanks for the explanation. I can confirm that at least for me https://fonts.googleapis.com/css2?display=swap&family=Montserrat redirects to https://fonts.gstatic.com/s/montserrat/v23/JTUHjIg1_i6t8kCHKm4532VJOt5-QNFgpCtr6Hw5aXx-p7K4KLg.woff right now.


Move openshift-oauth-proxy into main workspace deployment

What does this PR do?
At a high level, this PR modifies the controller to create artifacts related to the openshift-oauth-proxy inside the main workspace deployment instead of as free-standing deployment. This requires significant changes, however:

Previously, the entirety of the oauth-proxy was provisioned by the workspacerouting controller. This meant that no coordination between routing objects and deployment objects was required. Additionally, the workspacerouting could avoid creating routes for services that will be exposed through the proxy. With these changes, some coordination is necessary between the two.
We now have to create a serviceaccount for each workspace, since this service account requires annotations to support the proxy service. The relevant rolebindings also have to be updated to be more generic (matching any serviceaccount in their namespace).
There are more changes that I've forgotten since I wrote them :)

This is a draft PR as I'd like to clean up the code and smooth out some rough edges, but I wanted some feedback before continuing to plug away at it. The commit history is a bit of a winding road, so may not be very useful to review (as some commits introduce issues that are later resolved). I've tried to give descriptive commit messages to ease reasoning through the history.
Commits 966bf67..3d72f37 refactor the oauth-proxy solver to separate out the provisioning of routing objects from deployment objects in order to make it possible to move generating the proxy into the main deployment. Commit 3d72f37 should represent a refactor with no real functional changes (i.e the workspace should be similar). Commit f2ca475 moves proxy creation into the workspace controller, and the following commits are cleanup and fixups. Commit 5848259 adds basic authentication to the oauth proxy, limiting workspace access to users who can exec into pods.
Background
The requirements for setting up the oauth-proxy correctly, and having it work as expected, are:

The ServiceAccount used for the pod containing the proxy container must be annotated with serviceaccounts.openshift.io/oauth-redirectreference.<value>. This means that the common che-workspace SA cannot be used, and a new SA must be created for each workspace. This reference must refer to the proxy route by name (linking the generation of the two)
The proxy container has to be started with parameters for correct listen and proxy ports. This links container creation to service creation.
The proxy route must point at the proxy service on the correct port.
The secret used for the service.alpha.openshift.io/serving-cert-secret-name annotation on the proxy service must match a secret mounted into the proxy container.
Any misconfiguration in any of these objects means connecting to the workspace will fail with absolutely no description of what is wrong.
Additionally, on the workspace controller side, the internal ComponentInstanceStatus must be updated to accomodate the proxy.

Known issues/TODOs

Coalesce name generation and common metadata into one place; there are a few places where names have to be hard-coded to ensure everything stays in sync (e.g. that the ServiceAccount and Route and Service all have the correct names)
On minishift, I am unable to open a terminal due to errors connecting from the Theia side (self-signed cert):Extension-Host:Unable to create ws connection for task plugin. Cause: Error: self signed certificate in certificate chain

The route generated for the proxy can be too long for the URL field in the workspace status, resulting in part being cut off (e.g. URL=s://server95b4730759e311ea-oauth-proxy-theia-4400-che-workspace-con.<IP_ADDRESS>.nip.io) Fixed
The openshift-oauth-proxy has some limitations that have ugly workarounds:

Each proxy container can only listen on one port; this means we have to add another container for each secured port in the workspace
The single port limitation means that each secured endpoint will require a separate login; this is a significant issue we need to figure out
This limitation also means that adding the proxy to the che-machine-exec route breaks terminal access

General code cleanup; it's written quickly and messily. Good enough for now.

What issues does this PR fix or reference?
https://github.com/eclipse/che/issues/15786
Is it tested? How?
Tested workspaces java-mysql.yaml (no oauth), java-mysql-oauth.yaml (with oauth), and cloud-shell.yaml on minishift.
TODO: Test on minikube

@sleshchenko I'm marking this as "ready to review", mostly because I've stared at it for too long at this point so I have trouble seeing what's a real issue and what's not. There are a few TODOs in the additions that I don't have a good answer for right now.

I've rebased this PR onto the current master to avoid conflicts.
I've also added a commit https://github.com/che-incubator/che-workspace-operator/pull/22/commits/238bb09b0e51a38365ec5ec65d4982268ffaa617 which adds an option to disable webhooks added in #19 to enable the operator to be run locally, let me know what you think @sleshchenko

Is closed in the favor of https://github.com/che-incubator/che-workspace-operator/pull/36


⚠ Bofrost.es has degraded performance

In 4e370c5, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1630 ms

Resolved: Bofrost.es performance has improved in 52fb4ee after 2 hours, 1 minute.


osvrRenderManager related crash  in UE4Editor_OSVR!FCurrentCustomPresent::GetProjectionMatrix()

I've seen this crash a few times:
Unknown exception - code c06d007f (first/second chance not available)

KERNELBASE
UE4Editor_OSVR!__delayLoadHelper2() [f:\dd\vctools\delayimp\delayhlp.cpp:386]
UE4Editor_OSVR!_tailMerge_osvrRenderManager_dll()
UE4Editor_OSVR!FCurrentCustomPresent::GetProjectionMatrix() [d:\dev\descentunderground\gameproject\plugins\osvr2\source\osvr\private\osvrcustompresentd3d11.h:180]
UE4Editor_OSVR!FOSVRHMD::GetStereoProjectionMatrix() [d:\dev\descentunderground\gameproject\plugins\osvr2\source\osvr\private\osvrhmd.cpp:650]
UE4Editor_Engine!ULocalPlayer::GetProjectionData() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\engine\private\localplayer.cpp:976]
UE4Editor_Engine!ULocalPlayer::CalcSceneViewInitOptions() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\engine\private\localplayer.cpp:655]
UE4Editor_Engine!ULocalPlayer::CalcSceneView() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\engine\private\localplayer.cpp:716]
UE4Editor_Engine!UGameViewportClient::Draw() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\engine\private\gameviewportclient.cpp:927]
UE4Editor_Engine!FViewport::Draw() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\engine\private\unrealclient.cpp:1149]
UE4Editor_UnrealEd!UEditorEngine::Tick() [d:\build\++ue4+release-4.12+compile\sync\engine\source\editor\unrealed\private\editorengine.cpp:1531]
UE4Editor_UnrealEd!UUnrealEdEngine::Tick() [d:\build\++ue4+release-4.12+compile\sync\engine\source\editor\unrealed\private\unrealedengine.cpp:368]
UE4Editor!FEngineLoop::Tick() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\launch\private\launchengineloop.cpp:2775]
UE4Editor!GuardedMain() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\launch\private\launch.cpp:148]
UE4Editor!GuardedMainWrapper() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\launch\private\windows\launchwindows.cpp:126]
UE4Editor!WinMain() [d:\build\++ue4+release-4.12+compile\sync\engine\source\runtime\launch\private\windows\launchwindows.cpp:200]
UE4Editor!__scrt_common_main_seh() [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:264]
kernel32
ntdll

Appears to be crashing when loading or accessing osvrRenderManager.dll

It is possible the following code in OSVRClientKit.Build.cs possibly leading to a path for the dll that isn't valid at runtime.
            PublicLibraryPaths.Add(String.Format("{0}/lib/{1}", ModuleDirectory, PlatformAbbrev));
            PublicAdditionalLibraries.Add("osvrClientKit.lib");
            PublicAdditionalLibraries.Add("osvrRenderManager.lib");

            var osvrDlls = new string[] {
                "osvrClientKit.dll",
                "osvrClient.dll",
                "osvrCommon.dll",
                "osvrUtil.dll",
                "osvrRenderManager.dll",
                "d3dcompiler_47.dll",
                "glew32.dll",
                "SDL2.dll"
              };

            PublicDelayLoadDLLs.AddRange(osvrDlls);

            string baseBinaryDirectory = ModuleDirectory + "/bin";
            if (!System.IO.Directory.Exists(baseBinaryDirectory))
            {
                baseBinaryDirectory = "$(EngineDir)/Binaries/ThirdParty/OSVRClientKit/bin";
            }

            string DllFormat = "{0}/{1}/{2}";
            foreach (var dll in osvrDlls)
            {
                var src = String.Format(DllFormat, baseBinaryDirectory, PlatformAbbrev, dll);
                RuntimeDependencies.Add(new RuntimeDependency(src));
            }

Not sure if you can point a debugger at the build process, but you can use Console.WriteLine in OSVRClientKit.Build.cs to see where these paths are ending up. Console.WriteLine output ends up in the build output window. Try outputing baseBinaryDirectory before and after the Directory.Exists if statement, and each of the src in the osvrDlls foreach loop. That should give us a good idea what's going on. I'm not able to reproduce this on my end, so it's possible my environment is different from yours somehow.

This is very odd - the crash eventually went away after multiple rebuilds and changing version number of newer plugin added to the project itself to 0.2.


Configurable log message filter

After upgrading to phoenix 1.4 bugsnag was filled with 404 because they are reported to logger as errors. This change adds a hook that will allow end users to filter whatever logs they don't want to be reported.

Thanks for the contribution!


CoC.nvim (RuntimeError) no shell process input given for yes?/2

          👋🏼 @mhanberg After upgrading to `0.1.1` and forth, I'm not able to run the server using CoC again. Do I need to change anything in the custom LS configuration? Here's the error:

** (RuntimeError) no shell process input given for yes?/2
    (mix 1.14.5) lib/mix/shell/process.ex:148: Mix.Shell.Process.yes?/2
    (mix 1.14.5) lib/mix/tasks/deps.compile.ex:256: Mix.Tasks.Deps.Compile.handle_rebar_not_found/1
    (mix 1.14.5) lib/mix/tasks/deps.compile.ex:218: Mix.Tasks.Deps.Compile.do_rebar3/2
    (mix 1.14.5) lib/mix/tasks/deps.compile.ex:96: anonymous fn/4 in Mix.Tasks.Deps.Compile.compile/2
    (elixir 1.14.5) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
    (mix 1.14.5) lib/mix/tasks/deps.compile.ex:80: Mix.Tasks.Deps.Compile.compile/2
    (mix 1.14.5) lib/mix/tasks/deps.loadpaths.ex:84: Mix.Tasks.Deps.Loadpaths.deps_check/2
    (mix 1.14.5) lib/mix/tasks/deps.loadpaths.ex:43: Mix.Tasks.Deps.Loadpaths.run/1
[Error - 07:48:22.845] ~/projects/elixir/credo-ls/bin/credo-language-server exited with code: 1
[Error - 07:48:22.845] Connection to server got closed. Server will not be restarted.
[Error - 07:48:22.845] Server initialization failed.
  Message: Pending response rejected since connection got disposed
  Code: -32097

And this is the config working until 0.1.1:
    "credoLS": {
      "command": "~/projects/elixir/credo-ls/bin/credo-language-server",
      "args": [
        "--stdio"
      ],
      "filetypes": [
        "elixir",
        "eelixir",
        "heex"
      ],
      "rootPatterns": [
        "mix.exs"
      ],
    },

Originally posted by @bigardone in https://github.com/elixir-tools/credo-language-server/issues/7#issuecomment-1606696713

how do you have Elixir and Erlang installed?
newer versions start an erlang node to run your code in, and I'm wondering if it's using a different version of Elixir than you are meaning to, so rebar is not installed yet.

I'm using asdf:
❯ asdf current
elixir          1.14.5-otp-25   /Users/user/projects/project/.tool-versions
erlang          <IP_ADDRESS>        /Users/user/projects/project/.tool-versions

what happens if you delete the .elixir-tools/_build directory and restart?
I'm very confused at this, so just want to make sure something didn't get goofed up during an update.

@bigardone https://github.com/elixir-tools/elixir-tools.vscode/issues/27#issuecomment-1608118594

@mhanberg I already had the same configuration. However, I have upgraded to 0.2.0, and everything is working like a charm again 🎉
Thank you so much 🙌🏼


database/sql: ExecContext leaks goroutine on panic.

What version of Go are you using (go version)?
go version devel +04c446f Mon Jul 3 15:39:44 2017 -0700 linux/amd64
What operating system and processor architecture are you using (go env)?
~
What did you do?
While playing around with the sql package, I tried putting a panic in namedValueToValue to see how it was called (I was investigating an unrelated bug).   Instead of the panic unwinding the stack, the code hung and never returned.  Looking at a stacktrace shows that sql.Tx.Rollback() tries to get write access to the closemu lock.  It does this as part of a defer in my application code while the panic is propagating up.
The lock is never acquired because RUnlock has not yet been called on closemu.  Investigating where this should have happened reveals Stmt.ExecContext.  Unlike pretty much every other cleanup idiom used in Go, this one leavea a brief window before properly doing cleanup:
func (s *Stmt) ExecContext(ctx context.Context, args ...interface{}) (Result, error) {
	s.closemu.RLock()
	defer s.closemu.RUnlock()

	var res Result
	strategy := cachedOrNewConn
	for i := 0; i < maxBadConnRetries+1; i++ {
		if i == maxBadConnRetries {
			strategy = alwaysNewConn
		}
		dc, releaseConn, ds, err := s.connStmt(ctx, strategy)
		if err != nil {
			if err == driver.ErrBadConn {
				continue
			}
			return nil, err
		}

		res, err = resultFromStatement(ctx, dc.ci, ds, args...)
		releaseConn(err)
		if err != driver.ErrBadConn {
			return res, err
		}
	}
	return nil, driver.ErrBadConn
}

When there is a panic in resultFromStatement, releaseConn is never called.  I put the panic there in my testing, but it's still pretty bad practice to not immediately schedule the cleanup after construction.  Deferring cleanup immediately after checking the returned error from s.connStmt lets the panic propagate properly.

@carl-mastrangelo Thank you for the report. I apologize for the delay in getting to this issue.
I'd like to back up a bit. Why do you consider the current behavior a bug? Or more exactly, can you describe how this behavior would be triggered outside of manually adding a panic?

@kardianos in the event of a panic, the connection is not properly released.  I manually added one, but they can occur naturally.  Better to program defensively.

I'm willing to listen to another opinion on this subject.
Unless I can see how a panic can happen in this situation I'm inclined to close this issue.

@kardianos what?  The code is already written.  It brings the sql code in the style of the rest of Go, and fixes the possibility of a panic.  What's your hesitation?  Perhaps someone else can review it?

I'm not seeing an error to fix here.
I'm not seeing the "defense" her as something needing to be defended against.
The proposed CL adds another closure which is another allocation and another function call. I would also argue it is slightly more complicated then the original code (although I like the switch statement).
Let me explain my reasoning. The following is common Go code:
f, err := os.Open("myfile.txt")
if err != nil {
    return err
}
_, err = io.Copy(os.Stdout, f) // This will not panic.
f.Close() // So this is safe.
if err != nil {
   return err
}

Or this example:
mutex.Lock()
sharedMap["foo"] = 42 // This will not panic.
mutex.Unlock() // So this is safe.

Yes, we commonly use defer to ensure resources are closed. However a deeper style/convention is to ensure functions don't panic in the first place. Go makes this easy to check and ensure. Panics don't just "happen". One way the sql package ensures locks are properly used is to use defer when unlocking locks around driver (user package) code. So in principle I agree with you. But defer is cheep and not free. Allocation is cheep, but not free. I have no issue making an allocation when needed, but unlike other languages panics / exceptions are tightly controlled.
On another note, you are not responding to my questions, but just saying "Why? Just merge it already, it is already there and it is better." This is not a productive way forward in any conversation. I've already told you my hesitation. I've now expanded on my hesitation. You are free to refer to someone else to this issue, but I would prefer to first direct your attention and reply to my questions and statements. If we discover they are false or without merit I will happily reverse my decision.

I'm not seeing an error to fix here.
There is no error, just a sharp edge.  That is what needs to be fixed.

I'm not seeing the "defense" her as something needing to be defended against.
Moving cleanup code as close to the creation point is good style, in any language.

However a deeper style/convention is to ensure functions don't panic in the first place.

I think you are intentionally missing the intent of this change.  Unless you can statically prove there are no panics possible, why take the chance?  Furthermore, it is closer stylistically to the rest of the sql package.  Why be a cowboy?

The proposed CL adds another closure which is another allocation and another function call.
That is the worst possible reason.  If you look at my other contributions to the Go project, they have all been performance related.  The change as written will not have a measurable impact.

On another note, you are not responding to my questions, but just saying "Why? Just merge it already, it is already there and it is better."

Because you don't see the problem you act on behalf of the whole project.  Did you look at the code at all?  Even if it didn't match up with anything I said prior, you can see the code is structured better:

The retry logic is separate from the execution logic.
The code is now resilient to unforseen panics.
Even if panics are provably impossible today, code changes; they could be possible in the future.
The maxBadConnRetries + 1 wart is gone.

You are free to refer to someone else to this issue
I would actually.  Who is the next most appropriate reviewer for the database/sql package?


unable to compile with Spark 2.3.1

I'm unable to compile to current master using Spark 2.3.1 using sbt 0.13.15 and jdk1.8.0_181
 sbt  -Dwith.hive=true  -Dwith.parquet=true  -Dwith.mesos=true  -D"scala.version"=2.11.8 -D"spark.version"=2.3.1 clean compile
gives

[info] Compiling 43 Scala sources to /XX/spark-notebook/modules/common/target/scala-2.11/classes...
[error] missing or invalid dependency detected while loading class file 'DefaultWrites.class'.
[error] Could not access type LocalTime in package org.joda.time,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)
[error] A full rebuild may help if 'DefaultWrites.class' was compiled against an incompatible version of org.joda.time.
[error] missing or invalid dependency detected while loading class file 'DefaultWrites.class'.
[error] Could not access type LocalDate in package org.joda.time,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)
[error] A full rebuild may help if 'DefaultWrites.class' was compiled against an incompatible version of org.joda.time.
[error] missing or invalid dependency detected while loading class file 'DefaultWrites.class'.
[error] Could not access type DateTime in package org.joda.time,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)
[error] A full rebuild may help if 'DefaultWrites.class' was compiled against an incompatible version of org.joda.time.
[error] missing or invalid dependency detected while loading class file 'DefaultReads.class'.
[error] Could not access type LocalTime in package org.joda.time,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)
[error] A full rebuild may help if 'DefaultReads.class' was compiled against an incompatible version of org.joda.time.
[error] missing or invalid dependency detected while loading class file 'DefaultReads.class'.
[error] Could not access type LocalDate in package org.joda.time,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)
[error] A full rebuild may help if 'DefaultReads.class' was compiled against an incompatible version of org.joda.time.
[error] missing or invalid dependency detected while loading class file 'DefaultReads.class'.
[error] Could not access type DateTime in package org.joda.time,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)
[error] A full rebuild may help if 'DefaultReads.class' was compiled against an incompatible version of org.joda.time.
[error] 6 errors found
[info] Compiling 3 Scala sources to /XX/spark-notebook/modules/git-notebook-provider/target/scala-2.11/classes...
[warn] there were three deprecation warnings; re-run with -deprecation for details
[warn] there was one feature warning; re-run with -feature for details
[warn] two warnings found
[info] Compiling 16 Scala sources to/XX/spark-notebook/modules/sbt-project-generator/target/scala-2.11/classes...
[warn] there was one deprecation warning; re-run with -deprecation for details
[warn] one warning found
[error] (common/compile:compileIncremental) Compilation failed
[error] Total time: 21 s, completed Sep 5, 2018 12:06:34 PM

Looks like a Java/scala-Version issue, but I connot figure out how to fix it.
The build works well if I use Spark 2.1.1 or Spark 2.2.1:
sbt  -Dwith.hive=true  -Dwith.parquet=true  -Dwith.mesos=true  -D"scala.version"=2.11.8 -D"spark.version"=2.1.1 clean compile

Some more infos: The problem occurs during the compilation of the "common" module. I've used sbt -Dwith.hive=true -Dwith.parquet=true -Dwith.mesos=true -D"scala.version"=2.11.8 -D"spark.version"=2.3.1 --debug
in sbt:
> project common
> compile

gives (/X/ denotes my home directory):

[info] Compiling 43 Scala sources to /X/spark-notebook/modules/common/target/scala-2.11/classes...
[debug] Getting org.scala-sbt:compiler-interface:0.13.15:component from component compiler for Scala 2.11.8
[debug] Getting org.scala-sbt:compiler-interface:0.13.15:component from component compiler for Scala 2.11.8
[debug] Running cached compiler 10c4dc75, interfacing (CompilerInterface) with Scala compiler version 2.11.8
[debug] Calling Scala compiler with arguments  (CompilerInterface):
[debug]         -bootclasspath
[debug]         /usr/java/jdk1.8.0_60/jre/lib/resources.jar:/usr/java/jdk1.8.0_60/jre/lib/rt.jar:/usr/java/jdk1.8.0_60/jre/lib/sunrsasign.jar:/usr/java/jdk1.8.0_60/jre/lib/jsse.jar:/usr/java/jdk1.8.0_60/jre/lib/jce.jar:/usr/java/jdk1.8.0_60/jre/lib/charsets.jar:/usr/java/jdk1.8.0_60/jre/lib/jfr.jar:/usr/java/jdk1.8.0_60/jre/classes:/X/.m2/repository/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar
[debug]         -classpath
[debug]         /X/spark-notebook/modules/common/target/scala-2.11/classes:/X/spark-notebook/modules/observable/target/scala-2.11/classes:/X/spark-notebook/modules/subprocess/target/scala-2.11/classes:/X/spark-notebook/modules/core/target/scala-2.11/classes:/X/spark-notebook/modules/sbt-dependency-manager/target/scala-2.11/classes:/X/.m2/repository/org/codehaus/janino/janino/3.0.8/janino-3.0.8.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-plugin-api/3.0.5/maven-plugin-api-3.0.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/io/reactivex/rxjava/1.0.0-rc.5/rxjava-1.0.0-rc.5.jar:/X/.m2/repository/org/spire-math/spire-macros_2.11/0.13.0/spire-macros_2.11-0.13.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/mesos/mesos/1.4.0/mesos-1.4.0-shaded-protobuf.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-repl_2.11/2.3.1/spark-repl_2.11-2.3.1.jar:/X/.m2/repository/org/apache/hadoop/hadoop-yarn-server-nodemanager/2.7.3/hadoop-yarn-server-nodemanager-2.7.3.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/io/aeron/aeron-driver/1.2.5/aeron-driver-1.2.5.jar:/X/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/kuali/common/kuali-s3/1.0.1/kuali-s3-1.0.1.jar:/X/.m2/repository/org/glassfish/hk2/hk2-api/2.4.0-b34/hk2-api-2.4.0-b34.jar:/X/.m2/repository/org/apache/arrow/arrow-memory/0.8.0/arrow-memory-0.8.0.jar:/X/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/2.7.3/hadoop-mapreduce-client-app-2.7.3.jar:/X/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.7.3/hadoop-mapreduce-client-core-2.7.3.jar:/X/.m2/repository/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.jar:/X/.m2/repository/org/apache/xbean/xbean-asm5-shaded/4.4/xbean-asm5-shaded-4.4.jar:/X/.m2/repository/com/github/rwl/jtransforms/2.4.0/jtransforms-2.4.0.jar:/X/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/X/.m2/repository/javolution/javolution/5.5.1/javolution-5.5.1.jar:/X/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/X/.m2/repository/com/thoughtworks/paranamer/paranamer/2.6/paranamer-2.6.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-artifact/3.0.5/maven-artifact-3.0.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-streaming_2.11/2.3.1/spark-streaming_2.11-2.3.1.jar:/X/.m2/repository/com/carrotsearch/hppc/0.7.2/hppc-0.7.2.jar:/X/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.7.3/hadoop-yarn-client-2.7.3.jar:/X/.m2/repository/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar:/X/.m2/repository/com/typesafe/play/play-datacommons_2.11/2.5.15/play-datacommons_2.11-2.5.15.jar:/X/.m2/repository/org/sonatype/aether/aether-util/1.13.1/aether-util-1.13.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-sql_2.11/2.3.1/spark-sql_2.11-2.3.1.jar:/X/.m2/repository/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar:/X/.m2/repository/com/typesafe/play/build-link/2.5.15/build-link-2.5.15.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-servlet/8.1.14.v20131031/jetty-servlet-8.1.14.v20131031.jar:/X/.m2/repository/com/googlecode/javaewah/JavaEWAH/0.3.2/JavaEWAH-0.3.2.jar:/X/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/X/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.5/jackson-core-2.6.5.jar:/X/.m2/repository/org/xerial/snappy/snappy-java/<IP_ADDRESS>/snappy-java-<IP_ADDRESS>.jar:/X/.m2/repository/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar:/X/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/jcabi/jcabi-log/0.14.3/jcabi-log-0.14.3.jar:/X/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/X/.m2/repository/org/apache/curator/curator-framework/2.7.1/curator-framework-2.7.1.jar:/X/.m2/repository/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-joda/2.6.5/jackson-datatype-joda-2.6.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-unsafe_2.11/2.3.1/spark-unsafe_2.11-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/orbit/javax.transaction/1.1.1.v201105210645/javax.transaction-1.1.1.v201105210645.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/hibernate/hibernate-validator/5.1.2.Final/hibernate-validator-5.1.2.Final.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.3.1/spark-core_2.11-2.3.1.jar:/X/.m2/repository/org/sonatype/sisu/sisu-inject-bean/2.3.0/sisu-inject-bean-2.3.0.jar:/X/.m2/repository/io/dropwizard/metrics/metrics-graphite/3.1.5/metrics-graphite-3.1.5.jar:/X/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/X/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/X/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.7.3/hadoop-mapreduce-client-common-2.7.3.jar:/X/.m2/repository/org/glassfish/hk2/hk2-utils/2.4.0-b34/hk2-utils-2.4.0-b34.jar:/X/.m2/repository/log4j/apache-log4j-extras/1.2.17/apache-log4j-extras-1.2.17.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/vividsolutions/jts-core/1.14.0/jts-core-1.14.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/wagon/wagon-provider-api/2.4/wagon-provider-api-2.4.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/parquet/parquet-hadoop/1.8.3/parquet-hadoop-1.8.3.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-io/8.1.14.v20131031/jetty-io-8.1.14.v20131031.jar:/X/.m2/repository/org/spark-project/hive/hive-exec/1.2.1.spark2/hive-exec-1.2.1.spark2.jar:/X/.m2/repository/com/vlkan/flatbuffers/1.2.0-3f79e055/flatbuffers-1.2.0-3f79e055.jar:/X/.m2/repository/org/spark-project/hive/hive-metastore/1.2.1.spark2/hive-metastore-1.2.1.spark2.jar:/X/.m2/repository/com/typesafe/config/1.3.1/config-1.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.11/1.0.6/scala-xml_2.11-1.0.6.jar:/X/.m2/repository/net/iharder/base64/2.3.8/base64-2.3.8.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-model/3.0.5/maven-model-3.0.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.6.5/jackson-datatype-jdk8-2.6.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-jndi/8.1.14.v20131031/jetty-jndi-8.1.14.v20131031.jar:/X/.m2/repository/com/google/inject/extensions/guice-servlet/3.0/guice-servlet-3.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-kvstore_2.11/2.3.1/spark-kvstore_2.11-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/parquet/parquet-column/1.8.3/parquet-column-1.8.3.jar:/X/.m2/repository/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar:/X/.m2/repository/com/google/guava/guava/16.0.1/guava-16.0.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-util/8.1.14.v20131031/jetty-util-8.1.14.v20131031.jar:/X/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.5/metrics-core-3.1.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/jcabi/jcabi-aether/0.10.1/jcabi-aether-0.10.1.jar:/X/.m2/repository/org/apache/curator/curator-client/2.7.1/curator-client-2.7.1.jar:/X/.m2/repository/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar:/X/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar:/X/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.14/plexus-interpolation-1.14.jar:/X/.m2/repository/com/github/luben/zstd-jni/1.3.2-2/zstd-jni-1.3.2-2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/agrona/agrona/0.9.5/agrona-0.9.5.jar:/X/.m2/repository/org/glassfish/jersey/bundles/repackaged/jersey-guava/2.22.2/jersey-guava-2.22.2.jar:/X/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/X/.m2/repository/com/google/inject/guice/4.0/guice-4.0.jar:/X/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/X/.m2/repository/com/typesafe/play/play-exceptions/2.5.15/play-exceptions-2.5.15.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-sketch_2.11/2.3.1/spark-sketch_2.11-2.3.1.jar:/X/.m2/repository/org/datanucleus/datanucleus-rdbms/3.2.9/datanucleus-rdbms-3.2.9.jar:/X/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/slf4j/jul-to-slf4j/1.7.21/jul-to-slf4j-1.7.21.jar:/X/.m2/repository/org/apache/directory/api/api-asn1-api/1.0.0-M20/api-asn1-api-1.0.0-M20.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/slf4j/jcl-over-slf4j/1.7.21/jcl-over-slf4j-1.7.21.jar:/X/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/X/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/sonatype/aether/aether-connector-file/1.13.1/aether-connector-file-1.13.1.jar:/X/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar:/X/.m2/repository/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar:/X/.m2/repository/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.jar:/X/.m2/repository/com/sun/jersey/jersey-server/1.9/jersey-server-1.9.jar:/X/.m2/repository/com/fasterxml/jackson/module/jackson-module-paranamer/2.6.5/jackson-module-paranamer-2.6.5.jar:/X/.m2/repository/org/apache/calcite/calcite-linq4j/1.2.0-incubating/calcite-linq4j-1.2.0-incubating.jar:/X/.m2/repository/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar:/X/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.7.3/hadoop-yarn-api-2.7.3.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.5/jackson-databind-2.6.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/orbit/javax.mail.glassfish/1.4.1.v201005082020/javax.mail.glassfish-1.4.1.v201005082020.jar:/X/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/kuali/maven/wagons/maven-s3-wagon/1.1.20/maven-s3-wagon-1.1.20.jar:/X/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/sonatype/aether/aether-connector-wagon/1.13.1/aether-connector-wagon-1.13.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-http/8.1.14.v20131031/jetty-http-8.1.14.v20131031.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/typesafe/akka/akka-protobuf_2.11/2.4.18/akka-protobuf_2.11-2.4.18.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-network-common_2.11/2.3.1/spark-network-common_2.11-2.3.1.jar:/X/.m2/repository/com/typesafe/play/play-json_2.11/2.5.15/play-json_2.11-2.5.15.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scalaz/scalaz-core_2.11/7.2.13/scalaz-core_2.11-7.2.13.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-network-shuffle_2.11/2.3.1/spark-network-shuffle_2.11-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scalaz/scalaz-concurrent_2.11/7.2.13/scalaz-concurrent_2.11-7.2.13.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-module-scala_2.11/2.6.5/jackson-module-scala_2.11-2.6.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/parquet/parquet-jackson/1.8.3/parquet-jackson-1.8.3.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/orc/orc-core/1.4.4/orc-core-1.4.4-nohive.jar:/X/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/X/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.4.0-b34/aopalliance-repackaged-2.4.0-b34.jar:/X/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/X/.m2/repository/com/typesafe/akka/akka-stream_2.11/2.4.18/akka-stream_2.11-2.4.18.jar:/X/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.7.3/hadoop-mapreduce-client-jobclient-2.7.3.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.6.5/jackson-datatype-jsr310-2.6.5.jar:/X/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-catalyst_2.11/2.3.1/spark-catalyst_2.11-2.3.1.jar:/X/.m2/repository/org/apache/calcite/calcite-avatica/1.2.0-incubating/calcite-avatica-1.2.0-incubating.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/net/sf/py4j/py4j/0.10.7/py4j-0.10.7.jar:/X/.m2/repository/org/codehaus/janino/commons-compiler/3.0.8/commons-compiler-3.0.8.jar:/X/.m2/repository/asm/asm/3.1/asm-3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/parquet/parquet-common/1.8.3/parquet-common-1.8.3.jar:/X/.m2/repository/org/glassfish/hk2/hk2-locator/2.4.0-b34/hk2-locator-2.4.0-b34.jar:/X/.m2/repository/org/apache/directory/server/apacheds-i18n/2.0.0-M15/apacheds-i18n-2.0.0-M15.jar:/X/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/2.7.3/hadoop-yarn-server-common-2.7.3.jar:/X/.m2/repository/org/antlr/antlr4-runtime/4.7/antlr4-runtime-4.7.jar:/X/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/X/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.4.18/akka-actor_2.11-2.4.18.jar:/X/.m2/repository/org/reactivestreams/reactive-streams/1.0.0/reactive-streams-1.0.0.jar:/X/.m2/repository/com/univocity/univocity-parsers/2.5.9/univocity-parsers-2.5.9.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-mllib-local_2.11/2.3.1/spark-mllib-local_2.11-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-mesos_2.11/2.3.1/spark-mesos_2.11-2.3.1.jar:/X/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar:/X/.m2/repository/org/typelevel/machinist_2.11/0.6.1/machinist_2.11-0.6.1.jar:/X/.m2/repository/org/jodd/jodd-core/3.5.2/jodd-core-3.5.2.jar:/X/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-module-jsonSchema/2.6.5/jackson-module-jsonSchema-2.6.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/io/reactivex/rxscala_2.11/0.22.0/rxscala_2.11-0.22.0.jar:/X/.m2/repository/com/typesafe/play/play-iteratees_2.11/2.5.15/play-iteratees_2.11-2.5.15.jar:/X/.m2/repository/io/netty/netty/3.10.6.Final/netty-3.10.6.Final.jar:/X/.m2/repository/org/glassfish/jersey/core/jersey-client/2.22.2/jersey-client-2.22.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-tags_2.11/2.3.1/spark-tags_2.11-2.3.1.jar:/X/.m2/repository/org/apache/maven/shared/maven-dependency-tree/2.1/maven-dependency-tree-2.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/sonatype/aether/aether-connector-asynchttpclient/1.13.1/aether-connector-asynchttpclient-1.13.1.jar:/X/.m2/repository/javax/activation/activation/1.1.1/activation-1.1.1.jar:/X/.m2/repository/com/google/inject/extensions/guice-assistedinject/4.0/guice-assistedinject-4.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-continuation/8.1.14.v20131031/jetty-continuation-8.1.14.v20131031.jar:/X/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-xml/8.1.14.v20131031/jetty-xml-8.1.14.v20131031.jar:/X/.m2/repository/net/sourceforge/f2j/arpack_combined_all/0.1/arpack_combined_all-0.1.jar:/X/.m2/repository/org/apache/parquet/parquet-format/2.3.1/parquet-format-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/springframework/spring-core/3.1.2.RELEASE/spring-core-3.1.2.RELEASE.jar:/X/.m2/repository/org/apache/calcite/calcite-core/1.2.0-incubating/calcite-core-1.2.0-incubating.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier_2.11/1.0.0-RC4/coursier_2.11-1.0.0-RC4.jar:/X/.m2/repository/org/sonatype/aether/aether-impl/1.13.1/aether-impl-1.13.1.jar:/X/.m2/repository/io/airlift/aircompressor/0.8/aircompressor-0.8.jar:/X/.m2/repository/com/twitter/chill-java/0.8.4/chill-java-0.8.4.jar:/X/.m2/repository/org/apache/commons/commons-exec/1.3/commons-exec-1.3.jar:/X/.m2/repository/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-aether-provider/3.0.5/maven-aether-provider-3.0.5.jar:/X/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-core/3.0.5/maven-core-3.0.5.jar:/X/.m2/repository/org/apache/hadoop/hadoop-auth/2.7.3/hadoop-auth-2.7.3.jar:/X/.m2/repository/com/typesafe/play/twirl-api_2.11/1.1.1/twirl-api_2.11-1.1.1.jar:/X/.m2/repository/org/iq80/snappy/snappy/0.2/snappy-0.2.jar:/X/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/X/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar:/X/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/X/.m2/repository/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar:/X/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.4/httpclient-4.5.4.jar:/X/.m2/repository/org/apache/hadoop/hadoop-yarn-server-web-proxy/2.7.3/hadoop-yarn-server-web-proxy-2.7.3.jar:/X/.m2/repository/org/apache/commons/commons-lang3/3.5/commons-lang3-3.5.jar:/X/.m2/repository/org/apache/hadoop/hadoop-common/2.7.3/hadoop-common-2.7.3.jar:/X/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.5/metrics-jvm-3.1.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/wololo/jts2geojson/0.8.0/jts2geojson-0.8.0.jar:/X/.m2/repository/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar:/X/.m2/repository/org/apache/curator/curator-recipes/2.7.1/curator-recipes-2.7.1.jar:/X/.m2/repository/org/glassfish/jersey/media/jersey-media-jaxb/2.22.2/jersey-media-jaxb-2.22.2.jar:/X/.m2/repository/org/antlr/ST4/4.0.4/ST4-4.0.4.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/jboss/logging/jboss-logging/3.1.3.GA/jboss-logging-3.1.3.GA.jar:/X/.m2/repository/com/twitter/chill_2.11/0.8.4/chill_2.11-0.8.4.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-plus/8.1.14.v20131031/jetty-plus-8.1.14.v20131031.jar:/X/.m2/repository/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar:/X/.m2/repository/org/datanucleus/datanucleus-api-jdo/3.2.6/datanucleus-api-jdo-3.2.6.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.5/jackson-annotations-2.6.5.jar:/X/.m2/repository/com/esotericsoftware/kryo-shaded/3.0.3/kryo-shaded-3.0.3.jar:/X/.m2/repository/org/json4s/json4s-jackson_2.11/3.2.11/json4s-jackson_2.11-3.2.11.jar:/X/.m2/repository/org/json4s/json4s-ast_2.11/3.2.11/json4s-ast_2.11-3.2.11.jar:/X/.m2/repository/org/apache/commons/commons-crypto/1.0.0/commons-crypto-1.0.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-yarn_2.11/2.3.1/spark-yarn_2.11-2.3.1.jar:/X/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/X/.m2/repository/org/apache/directory/api/api-util/1.0.0-M20/api-util-1.0.0-M20.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/orbit/javax.activation/1.1.0.v201105071233/javax.activation-1.1.0.v201105071233.jar:/X/.m2/repository/com/sun/jersey/jersey-json/1.9/jersey-json-1.9.jar:/X/.m2/repository/com/twitter/parquet-hadoop-bundle/1.6.0/parquet-hadoop-bundle-1.6.0.jar:/X/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7.jar:/X/.m2/repository/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-security/8.1.14.v20131031/jetty-security-8.1.14.v20131031.jar:/X/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/X/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/X/.m2/repository/org/codehaus/plexus/plexus-classworlds/2.4/plexus-classworlds-2.4.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-mllib_2.11/2.3.1/spark-mllib_2.11-2.3.1.jar:/X/.m2/repository/com/typesafe/play/play-functional_2.11/2.5.15/play-functional_2.11-2.5.15.jar:/X/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/X/.m2/repository/org/scalactic/scalactic_2.11/3.0.1/scalactic_2.11-3.0.1.jar:/X/.m2/repository/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar:/X/.m2/repository/org/scala-stm/scala-stm_2.11/0.7/scala-stm_2.11-0.7.jar:/X/.m2/repository/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar:/X/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/X/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-webapp/8.1.14.v20131031/jetty-webapp-8.1.14.v20131031.jar:/X/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/amazonaws/aws-java-sdk/1.4.2/aws-java-sdk-1.4.2.jar:/X/.m2/repository/org/spire-math/spire_2.11/0.13.0/spire_2.11-0.13.0.jar:/X/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.7.3/hadoop-yarn-common-2.7.3.jar:/X/.m2/repository/org/antlr/stringtemplate/3.2.1/stringtemplate-3.2.1.jar:/X/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar:/X/.m2/repository/org/glassfish/jersey/core/jersey-common/2.22.2/jersey-common-2.22.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.7/httpcore-4.4.7.jar:/X/.m2/repository/com/typesafe/play/play-netty-utils/2.5.15/play-netty-utils-2.5.15.jar:/X/.m2/repository/org/datanucleus/datanucleus-core/3.2.10/datanucleus-core-3.2.10.jar:/X/.m2/repository/io/netty/netty-all/4.1.17.Final/netty-all-4.1.17.Final.jar:/X/.m2/repository/com/github/fommil/netlib/core/1.1.2/core-1.1.2.jar:/X/.m2/repository/org/sonatype/aether/aether-api/1.13.1/aether-api-1.13.1.jar:/X/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/parquet/parquet-encoding/1.8.3/parquet-encoding-1.8.3.jar:/X/.m2/repository/org/apache/hadoop/hadoop-annotations/2.7.3/hadoop-annotations-2.7.3.jar:/X/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/X/.m2/repository/org/lz4/lz4-java/1.4.0/lz4-java-1.4.0.jar:/X/.m2/repository/net/razorvine/pyrolite/4.13/pyrolite-4.13.jar:/X/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.7.16/slf4j-log4j12-1.7.16.jar:/X/.m2/repository/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-settings-builder/3.0.5/maven-settings-builder-3.0.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/X/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/X/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:/X/.m2/repository/org/glassfish/jersey/core/jersey-server/2.22.2/jersey-server-2.22.2.jar:/X/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/X/.m2/repository/org/antlr/antlr-runtime/3.4/antlr-runtime-3.4.jar:/X/.m2/repository/net/hydromatic/eigenbase-properties/1.1.5/eigenbase-properties-1.1.5.jar:/X/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.7.3/hadoop-mapreduce-client-shuffle-2.7.3.jar:/X/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/X/.m2/repository/org/joda/joda-convert/1.8.1/joda-convert-1.8.1.jar:/X/.m2/repository/org/roaringbitmap/RoaringBitmap/0.5.11/RoaringBitmap-0.5.11.jar:/X/.m2/repository/org/sonatype/sisu/sisu-guava/0.9.9/sisu-guava-0.9.9.jar:/X/.m2/repository/org/scalanlp/breeze-macros_2.11/0.13.2/breeze-macros_2.11-0.13.2.jar:/X/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.5/metrics-json-3.1.5.jar:/X/.m2/repository/org/apache/derby/derby/<IP_ADDRESS>/derby-<IP_ADDRESS>.jar:/X/.m2/repository/com/typesafe/play/play-streams_2.11/2.5.15/play-streams_2.11-2.5.15.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/orc/orc-mapreduce/1.4.4/orc-mapreduce-1.4.4-nohive.jar:/X/.m2/repository/org/apache/directory/server/apacheds-kerberos-codec/2.0.0-M15/apacheds-kerberos-codec-2.0.0-M15.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/eclipse/jetty/jetty-server/8.1.14.v20131031/jetty-server-8.1.14.v20131031.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/aspectj/aspectjrt/1.8.2/aspectjrt-1.8.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/jcabi/jcabi-aspects/0.18/jcabi-aspects-0.18.jar:/X/.m2/repository/com/typesafe/play/play_2.11/2.5.15/play_2.11-2.5.15.jar:/X/.m2/repository/org/sonatype/aether/aether-spi/1.13.1/aether-spi-1.13.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scalaz/scalaz-effect_2.11/7.2.13/scalaz-effect_2.11-7.2.13.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-settings/3.0.5/maven-settings-3.0.5.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/typesafe/akka/akka-remote_2.11/2.4.18/akka-remote_2.11-2.4.18.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/fasterxml/classmate/1.0.0/classmate-1.0.0.jar:/X/.m2/repository/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-launcher_2.11/2.3.1/spark-launcher_2.11-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cache_2.11/1.0.0-RC4/coursier-cache_2.11-1.0.0-RC4.jar:/X/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.11/0.7.0/scala-java8-compat_2.11-0.7.0.jar:/X/.m2/repository/org/apache/arrow/arrow-format/0.8.0/arrow-format-0.8.0.jar:/X/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/com/ning/async-http-client/1.6.5/async-http-client-1.6.5.jar:/X/.m2/repository/org/sonatype/sisu/sisu-guice/3.1.0/sisu-guice-3.1.0-no_aop.jar:/X/.m2/repository/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/kuali/common/kuali-threads/1.0.9/kuali-threads-1.0.9.jar:/X/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/X/.m2/repository/org/scalanlp/breeze_2.11/0.13.2/breeze_2.11-0.13.2.jar:/X/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.8/plexus-utils-3.0.8.jar:/X/.m2/repository/com/sun/jersey/jersey-client/1.19/jersey-client-1.19.jar:/X/.m2/repository/org/typelevel/macro-compat_2.11/1.1.1/macro-compat_2.11-1.1.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-model-builder/3.0.5/maven-model-builder-3.0.5.jar:/X/.m2/repository/org/glassfish/hk2/osgi-resource-locator/1.0.1/osgi-resource-locator-1.0.1.jar:/X/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/X/.m2/repository/org/sonatype/sisu/sisu-inject-plexus/2.3.0/sisu-inject-plexus-2.3.0.jar:/X/.m2/repository/org/apache/commons/commons-math3/3.4.1/commons-math3-3.4.1.jar:/X/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/io/aeron/aeron-client/1.2.5/aeron-client-1.2.5.jar:/X/.m2/repository/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar:/X/.m2/repository/org/apache/arrow/arrow-vector/0.8.0/arrow-vector-0.8.0.jar:/X/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.22.2/jersey-container-servlet-core-2.22.2.jar:/X/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/X/.m2/repository/com/esotericsoftware/minlog/1.3.0/minlog-1.3.0.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-graphx_2.11/2.3.1/spark-graphx_2.11-2.3.1.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-repository-metadata/3.0.5/maven-repository-metadata-3.0.5.jar:/X/.m2/repository/org/glassfish/hk2/external/javax.inject/2.4.0-b34/javax.inject-2.4.0-b34.jar:/X/.m2/repository/com/chuusai/shapeless_2.11/2.3.2/shapeless_2.11-2.3.2.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scalap/2.11.8/scalap-2.11.8.jar:/X/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/X/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.4.18/akka-slf4j_2.11-2.4.18.jar:/X/.coursier/cache/v1/https/repo1.maven.org/maven2/org/apache/spark/spark-hive_2.11/2.3.1/spark-hive_2.11-2.3.1.jar:/X/.m2/repository/org/apache/avro/avro-mapred/1.7.7/avro-mapred-1.7.7-hadoop2.jar:/X/.m2/repository/org/scalatest/scalatest_2.11/3.0.1/scalatest_2.11-3.0.1.jar:/X/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.9/jersey-guice-1.9.jar:/X/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet/2.22.2/jersey-container-servlet-2.22.2.jar:/X/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.7.3/hadoop-hdfs-2.7.3.jar:/X/.m2/repository/org/json4s/json4s-core_2.11/3.2.11/json4s-core_2.11-3.2.11.jar:/X/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar

I've now completed wiped local artefact caches (~.m2/ and ~.coursier/ ) and now the compilation runs fine!
Interestingly I had jars from ~.m2/ in the classpath (see above), after deleting the two caches and re-compiling spark-notebook, ~.m2/ remains empty and only jars from ~.coursier/ appear in the classpath. How can that be?

Interestingly I had jars from ~.m2/ in the classpath (see above), after deleting the two caches and re-compiling spark-notebook, ~.m2/ remains empty and only jars from ~.coursier/ appear in the classpath. How can that be?

It's a good question to sbt project ;)


Response of write and delete endpoint

I've been working on writing some integration tests using CircleCI 2.0 and the quay.io/cayleygraph/cayley image. I ran into some strange behavior while looking at the responses of the /api/v1/write and /api/v1/delete endpoint.

When deleting one non-existing quad, the delete endpoint always returns Successfully deleted 1 quads.. It also does not delete the quad.
When writing the same quad multiple times with the bare quay.io/cayleygraph/cayley, the write endpoint always returns Successfully wrote 1 quads. It does seems to write the quad, as returning the first 100 quads via the GraphQL API contains the written quad.

I'm guessing this strange behavior has something to do with the fact that I haven't passed a data volume (seeing as this is running in CircleCI 2.0), however it seems to me that either it should break more explicitly or it should work the way it's saying it does based on the responses I'm getting. Also, the delete endpoint seems to behave this way in normal circumstances as well, as I tested it on one of my production servers.
Reproducing the error:

docker run -d quay.io/cayleygraph/cayley
docker exec -i -t ${contained_id} /bin/bash
Write quad: curl 'http://localhost:64210/api/v1/write' --data '[{"subject":"<subject>","predicate":"<predicate>","object":"object","label":""}]' --compressed
Delete quad: curl 'http://localhost:64210/api/v1/delete' --data '[{"subject":"<subject>","predicate":"<predicate>","object":"object","label":""}]' --compressed
List quads: curl 'http://localhost:64210/api/v1/query/graphql' --data $'{\n  nodes(first: 100){\n    id\n  }\n}' --compressed

Update: Passing a data volume doesn't seem to fix the strange write behavior. The production server I tested on used a Postgres backend, so it might just be BoltDB that is behaving strangely with respect to writes. The strange delete behavior also occurs on the Postgres-backed Cayley instance.

There are multiple issues here:

Quad delete API does not return any indication of quad being actually removed (https://github.com/cayleygraph/cayley/issues/608)
Quad write has no indication of how many unique quads were written
Some backends will not garbage-collect nodes (https://github.com/cayleygraph/cayley/issues/617). Your last query shows nodes, not quads. Try curl http://localhost:64210/api/v2/read instead.

Issues 1 and 3 are tracked separately. And do you have a use case where issue 2 matters? Is it related to transactions?
Also I would suggest to switch to v2 api, if possible.

My usecase for issue 2 is that I'm trying to deduplicate a stream of quads by writing them and filtering the result stream based on the response from Cayley. As for the v2 api, it seems to suffer from issue 2 as well, but it fixes issue 1:

curl 'http://localhost:64210/api/v2/write' --data '<subject> <predicate> "object" .' --compressed results in {"result": "Successfully wrote 1 quads.", "count": 1}, even after already writing the quad.
curl 'http://localhost:64210/api/v2/delete' --data '<subject> <predicate> "object" .' --compressed deletes the quad correctly! Resulting in {"result": "Successfully deleted 1 quads.", "count": 1} and {"error": "delete \u003csubject\u003e -- \u003cpredicate\u003e -\u003e \"object\": quad does not exist"} consequently, which is what I would expect.

OK, I see the cause of the problem now. In v0.6 the default behavior was to report if duplicated quad was inserted. But it was a non-intuitive, since most large datasets contains duplicates. So we switched behavior, and now Cayley server ignores duplicates by default.
The fix would be to set ignore_duplicates to false in config:
load:
  ignore_duplicates: false

Aah, I see! I have to say, I didn't know this was a config option or that there was a v2 API. Neither seem to be documented in the docs folder. However, I think this will solve my issues!

Yeah, our docs is a bit outdated. We will address it at least partially for 0.7 release.
I'm closing this, but feel free to reopen.


Can I use this in Android application?

Yes, it's generic C/CPP code that can run on any platform. What did you have in mind?
I created an Android app with this code in the Google app store (Fast Image Viewer). It's got some GUI issues due to recent Android changes, but it should still work.


DateMessage is never user

Hi,
I was trying to change the placeholder text, and saw that there is dateMessage attribute. I suppose this is the attribute to change that placeholder text? However, it seems You didn't use it. Are You keeping it for something else or just forgot? :)

Keeping it for future functionality, error handeling. Haven't gotten that far yet, work and such


feat: New group notification[fix]

Fixes #3383
Correction of #3389
Describe the changes you have made in this PR -
Fixed dependent destroy error in PR#3389
Screenshots of the changes (If any) -
https://user-images.githubusercontent.com/85568177/227726278-cc475956-8bbf-4bfd-af7b-1dba909825f4.mp4
Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

GroupMember model was missing the params for which it had to look for before deleting notifications hence was giving error, added here
Video after change
https://user-images.githubusercontent.com/85568177/227726278-cc475956-8bbf-4bfd-af7b-1dba909825f4.mp4


gpio_set_level for GPIO_NUM_16 & GPIO_NUM_25 is not working for me. (IDFGH-5235)

Environment

Module or chip used:  ESP32-WROOM-32
IDF version : v4.2.1
Build System: idf.py
Compiler version : 8.4.0
Operating System:  Windows
(Windows only) environment type: ESP Command Prompt
Using an IDE?: Yes, VSCode IDE
Power Supply:        USB

Problem Description
//Detailed problem description goes here.
I have on going project which has binary size of 1457 KB.
When I try to gpio_set_level(GPIO_NUM_16,1) after initializing everything properly, that GPIO 16 it is not set to High.
Expected Behavior
When I call for gpio_set_level(GPIO_NUM_16,1). It has to set to High.
Actual Behavior
It is not Set to High. But when I call it in While loop without any delay or any other function it is set to high.
Steps to reproduce
I have tried to modify the gpio example code. it works perfectly. but when I tried to add it to my project it is not working.
I even tried to disable entire program logic. and just used the same code as like as example. But result is same.
And I tried to call the gpio_set_level in a loop which makes it work
This is happening only for the GPIO_NUM_16 and GPIO_NUM_25. other GPIOs are working perfectly.
Code to reproduce this issue
// the code should be wrapped in the ```cpp tag so that it will be displayed better.
#include "esp_log.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/queue.h"
#include "driver/gpio.h"

#define GPIO_OUTPUT_IO_0    16
#define GPIO_OUTPUT_IO_1    25
#define GPIO_OUTPUT_PIN_SEL  ((1ULL<<GPIO_OUTPUT_IO_0) | (1ULL<<GPIO_OUTPUT_IO_1))
#define GPIO_INPUT_IO_0     4
#define GPIO_INPUT_IO_1     5
#define GPIO_INPUT_PIN_SEL  ((1ULL<<GPIO_INPUT_IO_0) | (1ULL<<GPIO_INPUT_IO_1))
#define ESP_INTR_FLAG_DEFAULT 0

#define HIGH 1
#define LOW 0

void app_main(void)
{
    gpio_config_t io_conf;
    //disable interrupt
    io_conf.intr_type = GPIO_PIN_INTR_DISABLE;
    //set as output mode
    io_conf.mode = GPIO_MODE_OUTPUT;
    //bit mask of the pins that you want to set,e.g.GPIO18/19
    io_conf.pin_bit_mask = GPIO_OUTPUT_PIN_SEL;
    //disable pull-down mode
    io_conf.pull_down_en = 0;
    //disable pull-up mode
    io_conf.pull_up_en = 0;
    //configure GPIO with the given settings
    gpio_config(&io_conf);

    //interrupt of rising edge
    io_conf.intr_type = GPIO_PIN_INTR_POSEDGE;
    //bit mask of the pins, use GPIO4/5 here
    io_conf.pin_bit_mask = GPIO_INPUT_PIN_SEL;
    //set as input mode
    io_conf.mode = GPIO_MODE_INPUT;
    //enable pull-up mode
    io_conf.pull_up_en = 1;
    gpio_config(&io_conf);

    //change gpio intrrupt type for one pin
    gpio_set_intr_type(GPIO_INPUT_IO_0, GPIO_INTR_ANYEDGE);

    //create a queue to handle gpio event from isr
    // gpio_evt_queue = xQueueCreate(10, sizeof(uint32_t));
    // //start gpio task
    // xTaskCreate(gpio_task_example, "gpio_task_example", 2048, NULL, 10, NULL);

    //install gpio isr service
    gpio_install_isr_service(ESP_INTR_FLAG_DEFAULT);
    //hook isr handler for specific gpio pin
    // gpio_isr_handler_add(GPIO_INPUT_IO_0, gpio_isr_handler, (void*) GPIO_INPUT_IO_0);
    // //hook isr handler for specific gpio pin
    // gpio_isr_handler_add(GPIO_INPUT_IO_1, gpio_isr_handler, (void*) GPIO_INPUT_IO_1);

    //remove isr handler for gpio number.
    // gpio_isr_handler_remove(GPIO_INPUT_IO_0);
    // //hook isr handler for specific gpio pin again
    // gpio_isr_handler_add(GPIO_INPUT_IO_0, gpio_isr_handler, (void*) GPIO_INPUT_IO_0);

    int cnt = 0;
    gpio_set_level(GPIO_OUTPUT_IO_0,HIGH);
    gpio_set_level(GPIO_OUTPUT_IO_1,HIGH);
    // while(1) {
    //     printf("cnt: %d\n", cnt++);
    //     vTaskDelay(1000 / portTICK_RATE_MS);
    //     gpio_set_level(GPIO_OUTPUT_IO_0, cnt % 2);
    //     gpio_set_level(GPIO_OUTPUT_IO_1, cnt % 2);
    // }
}

// If your code is longer than 30 lines, GIST is preferred.
Debug Logs
entry 0x40080710
I (29) boot: ESP-IDF v4.2 2nd stage bootloader
I (29) boot: compile time 17:10:29
I (29) boot: chip revision: 1
I (31) qio_mode: Enabling default flash chip QIO
I (36) boot.esp32: SPI Speed      : 40MHz
I (41) boot.esp32: SPI Mode       : QIO
I (46) boot.esp32: SPI Flash Size : 4MB
I (50) boot: Enabling RNG early entropy source...
I (56) boot: Partition Table:
I (59) boot: ## Label            Usage          Type ST Offset   Length
I (66) boot:  0 nvs              WiFi data        01 02 00009000 00004000
I (74) boot:  1 otadata          OTA data         01 00 0000d000 00002000
I (81) boot:  2 phy_init         RF data          01 01 0000f000 00001000
I (89) boot:  3 ota_0            OTA app          00 10 00010000 001c2000
I (96) boot:  4 ota_1            OTA app          00 11 001e0000 001c2000
I (104) boot:  5 storage          Unknown data     01 82 003a2000 00040000
I (111) boot:  6 nvs_key          NVS keys         01 04 003e2000 00001000
I (119) boot: End of partition table
I (123) boot: No factory image, trying OTA 0
I (128) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x46c80 (289920) map
I (232) esp_image: segment 1: paddr=0x00056ca8 vaddr=0x3ffbdb60 size=0x04140 ( 16704) load
I (238) esp_image: segment 2: paddr=0x0005adf0 vaddr=0x40080000 size=0x00404 (  1028) load

I (239) esp_image: segment 3: paddr=0x0005b1fc vaddr=0x40080404 size=0x04e1c ( 19996) load
I (256) esp_image: segment 4: paddr=0x00060020 vaddr=0x400d0020 size=0xf9804 (1021956) map
0x400d0020: _stext at ??:?

I (592) esp_image: segment 5: paddr=0x0015982c vaddr=0x40085220 size=0x19314 (103188) load
0x40085220: coex_classic_bt_release$part$0 at ld_fm.c:?

I (632) esp_image: segment 6: paddr=0x00172b48 vaddr=0x400c0000 size=0x00064 (   100) load
I (648) boot: Loaded app from partition at offset 0x10000
I (682) boot: Set actual ota_seq=1 in otadata[0]
I (682) boot: Disabling RNG early entropy source...
I (682) cpu_start: Pro cpu up.
I (685) cpu_start: Application information:
I (690) cpu_start: Project name:     ABCD
I (695) cpu_start: App version:      40569bf-dirty
I (700) cpu_start: Compile time:     May 10 2021 17:10:17
I (706) cpu_start: ELF file SHA256:  471e1d12ceecb37a...
I (712) cpu_start: ESP-IDF:          v4.2
I (717) cpu_start: Starting app cpu, entry point is 0x40081890

I (0) cpu_start: App cpu up.
D (727) memory_layout: Checking 11 reserved memory ranges:
D (733) memory_layout: Reserved memory range 0x3ffae000 - 0x3ffae6e0
D (739) memory_layout: Reserved memory range 0x3ffae6e0 - 0x3ffaff10
D (746) memory_layout: Reserved memory range 0x3ffb0000 - 0x3ffb6388
D (752) memory_layout: Reserved memory range 0x3ffb8000 - 0x3ffb9a20
D (758) memory_layout: Reserved memory range 0x3ffbdb28 - 0x3ffbdb5c
D (765) memory_layout: Reserved memory range 0x3ffbdb60 - 0x3ffcd7b0
D (771) memory_layout: Reserved memory range 0x3ffe0000 - 0x3ffe0440
D (778) memory_layout: Reserved memory range 0x3ffe3f20 - 0x3ffe4350
D (784) memory_layout: Reserved memory range 0x40070000 - 0x40078000
D (791) memory_layout: Reserved memory range 0x40078000 - 0x40080000

D (797) memory_layout: Reserved memory range 0x40080000 - 0x4009e534

D (803) memory_layout: Building list of available memory regions:
D (810) memory_layout: Available memory region 0x3ffaff10 - 0x3ffb0000
D (816) memory_layout: Available memory region 0x3ffb6388 - 0x3ffb8000
D (823) memory_layout: Available memory region 0x3ffb9a20 - 0x3ffbdb28
D (829) memory_layout: Available memory region 0x3ffcd7b0 - 0x3ffce000
D (836) memory_layout: Available memory region 0x3ffce000 - 0x3ffd0000
D (842) memory_layout: Available memory region 0x3ffd0000 - 0x3ffd2000
D (849) memory_layout: Available memory region 0x3ffd2000 - 0x3ffd4000
D (856) memory_layout: Available memory region 0x3ffd4000 - 0x3ffd6000
D (862) memory_layout: Available memory region 0x3ffd6000 - 0x3ffd8000
D (869) memory_layout: Available memory region 0x3ffd8000 - 0x3ffda000
D (875) memory_layout: Available memory region 0x3ffda000 - 0x3ffdc000
D (882) memory_layout: Available memory region 0x3ffdc000 - 0x3ffde000
D (889) memory_layout: Available memory region 0x3ffde000 - 0x3ffe0000
D (895) memory_layout: Available memory region 0x3ffe0440 - 0x3ffe3f20
D (902) memory_layout: Available memory region 0x3ffe4350 - 0x3ffe8000
D (908) memory_layout: Available memory region 0x3ffe8000 - 0x3fff0000
D (915) memory_layout: Available memory region 0x3fff0000 - 0x3fff8000
D (922) memory_layout: Available memory region 0x3fff8000 - 0x3fffc000
D (928) memory_layout: Available memory region 0x3fffc000 - 0x40000000
D (935) memory_layout: Available memory region 0x4009e534 - 0x400a0000
I (941) heap_init: Initializing. RAM available for dynamic allocation:
D (949) heap_init: New heap initialised at 0x3ffaff10
I (954) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
D (960) heap_init: New heap initialised at 0x3ffb6388
I (965) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
D (971) heap_init: New heap initialised at 0x3ffb9a20
I (976) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
D (982) heap_init: New heap initialised at 0x3ffcd7b0
I (987) heap_init: At 3FFCD7B0 len 00012850 (74 KiB): DRAM
I (994) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (1000) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
D (1006) heap_init: New heap initialised at 0x4009e534
I (1012) heap_init: At 4009E534 len 00001ACC (6 KiB): IRAM
I (1018) cpu_start: Pro cpu start user code
D (1023) clk: waiting for 32k oscillator to start up
D (1059) clk: RTC_SLOW_CLK calibration value: 11414464
D (1065) intr_alloc: Connected src 46 to int 2 (cpu 0)
D (1065) intr_alloc: Connected src 17 to int 3 (cpu 0)
D (1066) intr_alloc: Connected src 24 to int 9 (cpu 0)
D (1071) FLASH_HAL: extra_dummy: 1
D (1074) spi_flash: trying chip: issi
D (1078) spi_flash: trying chip: gd
D (1082) spi_flash: trying chip: mxic
D (1085) spi_flash: trying chip: generic
I (1089) spi_flash: detected chip: generic
I (1094) spi_flash: flash io: qio
D (1098) chip_generic: set_io_mode: status before 0x200
I (1103) cpu_start: Starting scheduler on PRO CPU.
D (0) intr_alloc: Connected src 25 to int 2 (cpu 1)
I (0) cpu_start: Starting scheduler on APP CPU.
D (1119) heap_init: New heap initialised at 0x3ffe0440
D (1129) heap_init: New heap initialised at 0x3ffe4350
D (1139) intr_alloc: Connected src 16 to int 12 (cpu 0)
D (1139) APP: [APP] StartuP..
D (1139) APP: [APP] Free memory: 206936 bytes
D (1149) APP: [APP] IDF version: v4.2
D (1149) partition: Loading the partition table
D (1169) event: running task for loop 0x3ffbd778
D (1169) event: created task for loop 0x3ffbd778
D (1169) event: created event loop 0x3ffbd778
D (1169) APP: Heaps Are Valid
D (1179) SLEEP: Not a deep sleep reset

D (1179) history: Initializing SPIFFS
I (1209) history: Partition size: total: 233681, used: 502
D (1209) ADC: ADC Calibrartion Type 0

I (1209) gpio: GPIO[15]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1219) gpio: GPIO[16]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1229) gpio: GPIO[17]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1239) gpio: GPIO[19]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1249) gpio: GPIO[21]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1259) gpio: GPIO[22]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1269) gpio: GPIO[23]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1269) gpio: GPIO[12]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (1279) gpio: GPIO[18]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
D (1289) intr_alloc: Connected src 22 to int 13 (cpu 0)
D (1299) intr_alloc: Connected src 35 to int 17 (cpu 0)
D (1299) intr_alloc: Connected src 49 to int 18 (cpu 0)

Try something more rudimentary like the below.
const int io_16 = 16;
gpio_pad_select_gpio(io_16);
ESP_ERROR_CHECK(gpio_set_direction((gpio_num_t)io_16 , (gpio_mode_t)GPIO_MODE_OUTPUT));
ESP_ERROR_CHECK(gpio_set_level((gpio_num_t)io_16 , 1));

Try something more rudimentary like the below.
const int io_25 = 25;
gpio_pad_select_gpio(io_25);
ESP_ERROR_CHECK(gpio_set_direction((gpio_num_t)io_25 , (gpio_mode_t)GPIO_MODE_OUTPUT));
ESP_ERROR_CHECK(gpio_set_level((gpio_num_t)io_25 , 1));

Note the following IOs are used for flash/ram and SHOULD not be used for general ios:

IO11
IO7
IO16
IO17
IO6
IO8

I tried this. But it didn't show any progress. The issue is still there.
Also, As PSRAM is Optional one, I didn't use that in my design.

D (0) intr_alloc: Connected src 25 to int 2 (cpu 1)
This has nothing to do with gpios, interrupt sources have their own numbering system

But I can't able use those pins as output. Particularly , GPIO16, 17 and 25. Which is connected to the intr_alloc. Can you please help me with this.?

@negativekelvin Can I get any suggestion regard this. I am still getting this issue. even after I update esp-idf to v4.3.

Hi @Dineshkumar-J ,
Your issue looks like a concurrency issue, I wonder if there are too many controller for the same GPIO. Are you using gpio_set_level in different threads for GPIO16 and GPIO25 respectively? btw, using interrupt will not reduce your problem, unless you are moving all the code into the interrupt, otherwise you are just adding one possible controller for the GPIO to make it more uncontrollable.
Another thought, as you say, if your code is simpled enough (only one task), and looks very like the example but still can't work. The only difference between your code and the example is the sdkconfig. Maybe you can compare the sdkconfig of the example, and your project.

Hi @mkeyno ,
I don't know too much about 3rd party module/devkits. But basically which pins are occupied by Flash and PSRAM is determined by:

the efuse inside your module. (decides CLK, CS, MOSI, MISO, HD, package)
bootloader sdkconfig (decides WP, will also check the package above)
PSRAM sdkconfig (decides CLK and CS of PSRAM, will also check the package above)

But before continue debugging, I suggest check if those pins are break out on your devkit first. If you are sure your devkit vendor supports that, please come up with:

The sdkconfig you are using
The efuse inside your chip (by calling espefuse.py summary)

btw, I think this issue is case by case, next time please open a new issue, instead of replying under the same topic. This topic will be closed as soon as this discussion is resolved.

Hi @mkeyno ,
I don't know too much about 3rd party module/devkits. But basically which pins are occupied by Flash and PSRAM is determined by:

the efuse inside your module. (decides CLK, CS, MOSI, MISO, HD, package)
bootloader sdkconfig (decides WP, will also check the package above)
PSRAM sdkconfig (decides CLK and CS of PSRAM, will also check the package above)

But before continue debugging, I suggest check if those pins are break out on your devkit first. If you are sure your devkit vendor supports that, please come up with:

The sdkconfig you are using
The efuse inside your chip (by calling espefuse.py summary)

Thanks for reporting, will close due to short of feedback, feel free to reopen.

For now, there is a function to check whether a pin on the current SoC is reserved for other purpose, for example:
#include "esp_gpio_reserve.h"
...
uint32_t pin = 16;
bool is_reserved = esp_gpio_is_pin_reserved(pin);

Hello from 2025! They did change location of the include file as well as the function name.


Add links to the new "Automation Examples" and "App Central" articles

Purpose of this pull request
This pull request (PR) adds links to direct readers to these two new articles:

https://help.sumologic.com/docs/cse/automation-service/automation-service-examples/
https://help.sumologic.com/docs/cse/automation-service/automation-service-app-central/

Issue number: None

Select the type of change:

[ ] Minor Changes - Typos, formatting, slight revisions
[x] Update Content - Revisions and updating sections
[ ] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - Updates, maintenance, and new packages for the site, Gatsby, React, etc

Thanks for fixing those oopsies, @kimsauce.


how to use the model

Holle，sir, thank your great job!
I have train the model, but I want to see the effect. I want to take a video ( or use a video from the datasets) , then input the model. At last I hope get a video annotated by the model. Please teach me how to use , thank.

Hi,
Thanks for using our code. Regarding the code for generating demo, unfortunately we cannot provide you with customized code. For any modifications or new code please either follow the code documentation or refer to online forums for samples.

I donot need customized code. When I have trained the model, I need to transfor a clip of vedio ( not include annotations). but your paper said the model alse needs to input speed of car and other information.
So i want to know how to use the model. In your code, not provide the ways.

Hi,
if I understood your first question correctly, you would like to visualize the results of the model. Unfortunately, current release of the code does not provide any visualizations. If you would like to run the model on an arbitrary video clip (not from the dataset) the video should have the same annotations as the videos in the PIE dataset and pie_data interface should be modified to parse the new annotations.

yes，you are right.
Using the CVAT tool can annotate the image.
however in annotations_vehicle folder and annotations_attributes folder, what tool can be used.


terminitor plugin

These commits add a new plugin for terminitor (https://github.com/achiu/terminitor), a tool that "automates your development workflow setup".

Thanks, would you mind adding a description and some example usage to the Plugins page on the wiki?
Cheers,
Robby

There you go:
https://github.com/robbyrussell/oh-my-zsh/wiki/Plugins/_compare/909dd0ff298932a72ef2520c910f41c0bf0d01cc^...909dd0ff298932a72ef2520c910f41c0bf0d01cc
Cheers,
Philipp


Fix infra-scan-pipeline missing image

@maxwolfs something to have a look at..

We stuck at the same problem which seems to be fixed with this PR

This is more of a quickfix, we should look at it again with @maxwolfs:

The best option is to not limit source to *.md, but copy the whole directory. But this fails, if one of the Markdown files has issues in rendering
Another option is to then give a list of specific files to copy, "source": ["docs/*.md", "docs/images/*.png"]. But this will break paths, because the images will be in the same directory level as the .md, breaking relative paths.
Third option is to use option 1 but enforcing that included documents are all compatible with Docusaurus. This will always fall back on the people from the upstream repos


Running SCIB

Hello,
I have a couple of questions regarding running the SCIB pipeline,
1- i downloaded the pancreas dataset from: https://figshare.com/articles/dataset/Benchmarking_atlas-level_data_integration_in_single-cell_genomics_-integration_task_datasets_Immune_and_pancreas/12420968, i believe that the dataset is normalized and we don't need to preprocess it once again?
I wanted to use this dataset to integrate batch-to-batch, in other words, i wanted to generate pairs from the pancreas dataset (9 batches -> 36 pairs) and run the other methods on those pairs.
My question is, how did you run the other integration methods, with or without HVG information? I mean do I need to generate HVG and pass them as input to those methods? Or I will pass the whole dataset?
There are two HVG selection methods in your pipeline, which one do you recommend to use?
Thank you very much.

Hi @hezarfenfly,

Correct, there is log-scran-normalized data in there
We did both with and without HVG selection. Check www.github.com/theislab/scib-pipeline for the implementation details in our snakemake pipeline. For most methods we subsetted the data, but for some we just passed HVGs as a list (everything based on seurat objects).

Good luck with your analysis!

Thank you very much for your reply,
Another question, regarding other methods such as scanorama and BBKNN, how did you evaluate them, i could not see much details about this step in your publication and supplementary,
for example, BBKN outputs the following:
AnnData object with n_obs × n_vars = 3289 × 19093
obs: 'tech', 'celltype'
uns: 'neighbors', 'pca'
obsm: 'X_pca'
varm: 'PCs'
obsp: 'connectivities', 'distances'
which data from those above do you consider the integrated matrix?
and for scanorama, it outputs the following:
AnnData object with n_obs × n_vars = 3289 × 19093
obs: 'tech', 'celltype'
obsm: 'X_emb', 'X_scanorama'
Can you please provide us with more details?
Thank you very much.


Does not working with django-jet

Please help me.
It works very well without django-jet, but it doesn't with django-jet.

I'm sorry but third party support is out of scope for this project.
Feel free to fork this and make it work with "django-jet".


Embedded document links are brokens in NuGet README

Version
v1.0.3
Platform
N/A
Description
See in title. All internal markdown links are broken: CONTRIBUTING, LICENSE and so on.

New approach does not work.


transfer_dust()  for multiple assets API Error 5001

Hello!
This works:
dust = 'BTT,XLM'
client.transfer_dust(asset=dust)

However, the following didn't because of an unsupported asset:
dust = 'BTT,XLM,BZRX'
client.transfer_dust(asset=dust)

I was trying to convert multiple assets a while back and received an API 5001 error (Asset not Supported).
Do we have a list of assets that are not supported by transfer_dust()? Just so we can exclude them from the argument first, and be able to convert the rest.
Or, might the transfer_dust() function get an enhancement to throw an exception stating which assets in the supplied argument are not supported. This would be very helpful if in case we would be converting tens or hundreds of assets in one run, specially that we may only do so every 6 hours.
Thank you!

BZRX to OOKI
https://bzx.network/blog/bzrx-to-ooki-migration