ncinfo gives ERROR: key not found: 2  when times is Datatype:   char

Hi!
I'm trying to port a Matlab code that reads a ncfile with climatic data. and it's throwing this error: (at the terminal)
julia> ncinfo("wrfout_d02_2012-08-08_00_UVW.nc")

##### NetCDF File #####

/TallerModelacionNumericaAyudantia/wrfout_d02_2012-08-08_00_UVW.nc

##### Dimensions #####

           Name   Length
-------------------------------
     DateStrLen       19
      west_east      204
 west_east_stag      205
     bottom_top       27
south_north_stag      124
bottom_top_stag       28
           Time      121
    south_north      123

##### Variables #####

                Name                Type          Dimensions
---------------------------------------------------------------
                   W               FLOAT          west_east, south_north, bottom_top_stag, Time,
                 U10               FLOAT          west_east, south_north, Time,
                   V               FLOAT          west_east, south_north_stag, bottom_top, Time,
                PSFC               FLOAT          west_east, south_north, Time,
             XLONG_U               FLOAT          west_east_stag, south_north, Time,
              XLAT_V               FLOAT          west_east, south_north_stag, Time,
             XLONG_V               FLOAT          west_east, south_north_stag, Time,
                XLAT               FLOAT          west_east, south_north, Time,
                 ZNU               FLOAT          bottom_top, Time,
                 ZNW               FLOAT          bottom_top_stag, Time,
                   U               FLOAT          west_east_stag, south_north, bottom_top, Time,
                  T2               FLOAT          west_east, south_north, Time,
               TimesError showing value of type NcFile:
ERROR: key not found: 2
 in getindex at dict.jl:617
 in show at /Users/argelramirez/.julia/v0.3/NetCDF/src/NetCDF.jl:505
 in anonymous at show.jl:1159
 in with_output_limit at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in showlimited at show.jl:1158
 in writemime at replutil.jl:2
 in display at REPL.jl:117
 in display at REPL.jl:120
 in display at multimedia.jl:149
 in print_response at REPL.jl:139
 in print_response at REPL.jl:124
 in anonymous at REPL.jl:586
 in run_interface at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in run_frontend at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in run_repl at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in _start at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib

In Matlab the same file gives, for the variable times:
Times
           Size:       19x121
           Dimensions: DateStrLen,Time
           Datatype:   char

When i try to do the same in the Jupyter notebook it only throws (Don't know if it's relevant):
-------------------------------
---------------------------------------------------------------

key not found: 2

 in getindex at dict.jl:617
 in show at /Users/argelramirez/.julia/v0.3/NetCDF/src/NetCDF.jl:505
 in anonymous at show.jl:1159
 in with_output_limit at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in showlimited at show.jl:1158
 in writemime at replutil.jl:2
 in writemime at multimedia.jl:41
 in sprint at iostream.jl:229
 in display_dict at /Users/argelramirez/.julia/v0.3/IJulia/src/execute_request.jl:25

I believe it's a bug.
Cheers!

Thanks for reporting this. I am in the process of moving the package to the JuliaGeo organization https://github.com/JuliaGeo/NetCDF.jl and have done major changes so that the package is compatible to julia v0.4.
Currently I really lack the time to still do changes on the old version of the package, I am sorry for this. In order to circumvent the error you can try to edit your ~/.julia/v0.3/NetCDF/src/NetCDF.jl
Look for this Dict definition
global const nctype2string=@Compat.Dict(NC_BYTE=>"BYTE",
               NC_SHORT=>"SHORT",
               NC_INT=>"INT",
               NC_FLOAT=>"FLOAT",
               NC_DOUBLE=>"DOUBLE",
               NC_INT64=>"INT64")

and add a line like
NC_CHAR=>"CHAR"

This should at least solve the bug when showing the result. I will also fix this in the new refactored version of the package, but probably will not touch the old one anymore.

                                                                                I thank you for all the efforts!Will make those changes to my local files.Cheers!                                                                                Argel Ramírez Reyes                                                                                De: Fabian GansEnviado: miércoles, 23 de septiembre de 2015 09:56 a.m.Para: meggart/NetCDF.jlResponder a: meggart/NetCDF.jlCC: argelouskiAsunto: Re: [NetCDF.jl] ncinfo gives ERROR: key not found: 2  when times is Datatype:   char (#25)Thanks for reporting this. I am in the process of moving the package to the JuliaGeo organization https://github.com/JuliaGeo/NetCDF.jl and have done major changes so that the package is compatible to julia v0.4.

Currently I really lack the time to still do changes on the old version of the package, I am sorry for this. In order to circumvent the error you can try to edit your ~/.julia/v0.3/NetCDF/src/NetCDF.jl
Look for this Dict definition
global const nctype2string=@Compat.Dict(NC_BYTE=>"BYTE",
NC_SHORT=>"SHORT",
NC_INT=>"INT",
NC_FLOAT=>"FLOAT",
NC_DOUBLE=>"DOUBLE",
NC_INT64=>"INT64")
and add a line like
NC_CHAR=>"CHAR"
This should at least solve the bug when showing the result. I will also fix this in the new refactored version of the package, but probably will not touch the old one anymore.
—Reply to this email directly or view it on GitHub.


PhantomJs: ??? in onError + JSCom has been closed

Reproducies:
git clone https://github.com/japgolly/scalacss.git
cd scalacss
git checkout aed8b40
sbt core-js/test

It's been failing ever since I switched jsEnv to phantom js.
I get (abbreviated):
  /tmp/phantomjs-launcher5284409069744574786.js:9 in onError
  /tmp/phantomjs-launcher5284409069744574786.js:11 in onError
  file:///home/golly/projects/scalacss/core/target/js/scala-2.11/core-test-fastopt.js:79391 (in function "doWriteLine__p4__T__V")
  /tmp/phantomjs-launcher5284409069744574786.js:13
[trace] Stack trace suppressed: run last core-js/test:test for the full output.
[error] Could not run test japgolly.scalacss.DefaultsTest: org.scalajs.jsenv.ComJSEnv$ComClosedException: JSCom has been closed

When I run last core-js/test:test I get:
[debug] Running TaskDef(japgolly.scalacss.mutable.RegisterTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@6cd53313, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.UsageTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@11aa802b, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.full.StandaloneTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@6d90e4, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.DefaultsTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@1eb0487b, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.CondTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@206986cc, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.AttrTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@389590e0, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.ValueTTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@aa44995, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.StyleTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@531db443, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.full.InlineTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@51993e4a, false, [SuiteSelector])
[debug] Running TaskDef(japgolly.scalacss.ComposeTest, org.scalajs.testadapter.FingerprintSerializers$DeserializedSubclassFingerprint@16430dbf, false, [SuiteSelector])
org.scalajs.jsenv.ComJSEnv$ComClosedException: JSCom has been closed
	at org.scalajs.jsenv.phantomjs.PhantomJSEnv$ComPhantomRunner.receiveFrag(PhantomJSEnv.scala:287)
	at org.scalajs.jsenv.phantomjs.PhantomJSEnv$ComPhantomRunner.loop$1(PhantomJSEnv.scala:258)
	at org.scalajs.jsenv.phantomjs.PhantomJSEnv$ComPhantomRunner.receive(PhantomJSEnv.scala:273)
	at org.scalajs.testadapter.ComUtils$.receiveResponse(ComUtils.scala:42)
	at org.scalajs.testadapter.ComUtils$.receiveLoop(ComUtils.scala:21)
	at org.scalajs.testadapter.ComUtils$.receiveLoop(ComUtils.scala:16)
	at org.scalajs.testadapter.ScalaJSTask.execute(ScalaJSTask.scala:62)
	at sbt.TestRunner.runTest$1(TestFramework.scala:76)
	at sbt.TestRunner.run(TestFramework.scala:85)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
	at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
	at sbt.TestFunction.apply(TestFramework.scala:207)
	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:235)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Thanks for the report. I'll probably not be able to look into it before 26th, though.

np :)
I'll lock in a commit in the reproduction instructions above and switch off phantomjs for now.

Sweet!! Finally we can reproduce this.

I can't reproduce this :( It seems this is really just a race somewhere...

Oh no really? You definately checked it out to aed8b40 (cos I switched to Node JS after that commit). For me it happens 100% of the time. Maybe there's an environmental factor...
> uname -a
Linux golly-desktop 3.18.6-1-ARCH #1 SMP PREEMPT Sat Feb 7 08:44:05 CET 2015 x86_64 GNU/Linux

> phantomjs -v
2.0.0

> java -version
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b25)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)

$ phantomjs -v
1.9.0

That might be it... :S

I've just started getting this in a completely different (private) repo. :(

This has something to do with object initialisation order I believe.
Changing val to def or lazy val in one of my objects  has made this go away. But really, I can't see anything even close to a cyclic dependency or anything. In fact, the object I mentioned is a new one I created an hour ago that only 2 tests reference.

Is there anyway we can prioritise this? This issue follows me around from project to project.

It always mentions doWriteLine__p4__T__V. Is there any chance this doWriteLine fn isn't available in PhantomJS 2.0 or something like that?
  /tmp/phantomjs-launcher3933850603413980206.js:11 in onError
  file:///.../webapp-client-test-fastopt.js:244379 (in function "doWriteLine__p4__T__V")

This makes me want to swear. A lot.
https://github.com/ariya/phantomjs/issues/13112
PhantomJS releases are very few and far between... Could we pretty-please have a workaround in SJS for this? Like a configuration setting that determines which console.xxx method to use?

If you can come up with a work-around in doWriteLine that still uses console.error on other environments, I'm happy to take it. A configuration setting is unlikely to work, since we can't change the code of doWriteLine from a setting.
In any case I'll schedule a least a work-around for 0.6.3.

I wonder what we could do. Thinking slightly bigger, it might be handy if developers were able to control the behaviour of System.{err,out} in their Scala-JS apps... In addition to allowing devs to solve this specific problem themselves (still a burden though), it will also allow them to do things like mute noisy libraries, and replace stderr (in prod) with an ajax call that sends error messages back to the server.
What do you think about System.{err,out} either delegating to some globals that are initialised with defaults? (eg. The value System.out is in a JS global ScalaJS.System.out distinguishable from java.lang.System.out)

You can mutate java.lang.System.out if you want.

How? System.setErr(System.out) causes Referring to non-existent method jl_System$.setErr__Ljava_io_PrintStream__V.

Ah, right. Well I guess we need to add those. :-s

Ah, I thought I was missing something again. :) Raised #1598
So the above will facilitate users being able to workaround it themselves (everywhere in source). What should we do to prevent such a need? Maybe redirect System.err to  out for all PhantomJS envs? Is there a setting similar to initialCommands in Console but for JSEnvs? If so, users could add the redirection once in SBT to have it executed on env startup before client code is run.

You can subclass JSEnv classes. I think they all have an overridable method that corresponds to this. For PhantomJS that would be https://github.com/scala-js/scala-js/blob/master/js-envs/src/main/scala/org/scalajs/jsenv/ExternalJSEnv.scala#L30

Indeed. Actually, I think we should rather solve this particular problem on a JS level (so the Scala code remains as VM agnostic as possibled). Stderr will automatically be redirected to console.log, if console.error does not exist. Therefore, it should be sufficient to do this:
class PhantomJS2Env(
    phantomjsPath: String = "phantomjs",
    addArgs: Seq[String] = Seq.empty,
    addEnv: Map[String, String] = Map.empty,
    val autoExit: Boolean = true,
    jettyClassLoader: ClassLoader = null
) extends PhantomJSEnv(phantomjsPath, addArgs, addEnv, autoExit, jettyClassLoader) {

  override protected def vmName: String = "PhantomJS2"

  private val consoleNuker = new MemVirtualJSFile("consoleNuker.js")
      .withContent("console.error = undefined;")

  override def jsRunner(classpath: CompleteClasspath, code: VirtualJSFile,
      logger: Logger, console: JSConsole): JSRunner = {
    new PhantomRunner(classpath, code, logger, console) {
      override protected def initFiles(): Seq[VirtualJSFile] = super.initFiles :+ consoleNuker
    }
  }

  override def asyncRunner(classpath: CompleteClasspath, code: VirtualJSFile,
      logger: Logger, console: JSConsole): AsyncJSRunner = {
    new AsyncPhantomRunner(classpath, code, logger, console) {
      override protected def initFiles(): Seq[VirtualJSFile] = super.initFiles :+ consoleNuker
    }
  }

  override def comRunner(classpath: CompleteClasspath, code: VirtualJSFile,
      logger: Logger, console: JSConsole): ComJSRunner = {
    new ComPhantomRunner(classpath, code, logger, console) {
      override protected def initFiles(): Seq[VirtualJSFile] = super.initFiles :+ consoleNuker
    }
  }
}

And then use this, instead of the standard PhantomJS:
// Set other stuff as you require
postLinkJSEnv := new PhantomJS2Env(jettyClassLoader = scalaJSPhantomJSClassLoader.value)

That looks like it'd do the job. Is this something you envision being part of Scala.JS? It's quite a large snippet :)
Also that's a lot of boilerplate just to add some initial-JS. It might be worthwhile to add an initFiles: Seq[VirtualJSFile] arg to constructors of all JSenv classes, that appends it to each runner.

I don't know if this should be part of Scala.js. Are you working with PhantomJS 2.0 or with a pre-release version? Maybe we should upgrade to PhantomJS 2.0 anyways internally...
@sjrd what do you think? That would also give us proper websockets, meaning we could upgrade to Jetty9.

Upgrading to PhantomJS 2.0 would be nice, but isn't it a bit too early to drop support for PhantomJS 1.x?
I'd sooner drop support for JDK 6 ...

It's PhantomJS 2.0 final. The issue is marked as a regression so it must be a problem in certain version of PhantomJS 1.x as well.

Is the error we get from PhantomJS catchable? If yes, we can easily monkey patch console.error to handle the failure gracefully. Otherwise we'll need to:

Distinguish different PhantomJS versions
Never user console.error when in PhantomJS

In any case, this will be a testing nightmare for us, since we'll have to test with multiple versions of PhantomJS :(

What do we want to do about this? It's definitely a crasher in PhantomJS, so it must eventually be fixed.
Maybe we could add an overrideable method customInitFiles() in ExternalJSEnv, and have AbstractExtRunner.initFiles() call its enclosing JSEnv's customInitFiles() instead of returning an empty seq. That would reduce significantly the ceremony of the above workaround:
class PhantomJS2Env(
    phantomjsPath: String = "phantomjs",
    addArgs: Seq[String] = Seq.empty,
    addEnv: Map[String, String] = Map.empty,
    val autoExit: Boolean = true,
    jettyClassLoader: ClassLoader = null
) extends PhantomJSEnv(phantomjsPath, addArgs, addEnv, autoExit, jettyClassLoader) {

  override protected def vmName: String = "PhantomJS2"

  private val consoleNuker = new MemVirtualJSFile("consoleNuker.js")
      .withContent("console.error = undefined;")

  override protected def customInitFiles(): Seq[VirtualJSFile] =
    super.customInitFiles() :+ consoleNuker
}

@japgolly Would that be acceptable until they fix the issue in PhantomJS?
Should we provide PhantomJS2EnvWithConsoleErrorWorkaround defined above instead?

Should we provide PhantomJS2EnvWithConsoleErrorWorkaround defined above instead?

I don't think we should. In the long run, we should provide proper support for PhantomJS 2.0, with RFC 6455 websockets and Jetty9. So we might will have to envision to create a new separate environment. In the meantime, adding customInitFiles is probably the way to go.

@sjrd after #1630 is in, I think we should close this and open the a more general PhantomJS 2.0 support ticket.

I would leave this one open. After all it's not solved. We can cross-reference this ticket from the more general PhantomJS 2.0 one.

OK customInitFiles() is in. I'll keep this open, as it's not actually solved, but remove it from v0.6.3.

@japgolly Would that be acceptable until they fix the issue in PhantomJS?
Should we provide PhantomJS2EnvWithConsoleErrorWorkaround defined above instead?

Anything that allows me to workaround this is acceptable to me, but that's my selfish view :) It might be better to have something provided because anyone who uses PhantomJS 2 is going to have this problem, and it could be a very long time before it's fixed upstream.

Using the PhantomJS2Env @gzm0 provided above solves this 98% of the time. (example).
Even with this fix applied however, this issue still occurs in rare occasions (like when there is an error  initialising an object in a test). #1702 will address this.
If we apply #1702 and make PhantomJS2Env part of scala-js proper (preferably replacing PhantomJSEnv as it affects various versions of PhantomJS 1.x too), then this issue could (hopefully) be closed for good and for all users :)

Closing this as wontfix, since we'll be dropping PhantomJS.


[RFE] Pagination UI: Clusters List

Is your feature request related to a problem? Please describe.
Currently we have ~40 (edge-)clusters in 1 Rancher-instance, will probably be more in the future. This becomes quite a long list.
Describe the solution you'd like
Add pagination to the rancher ui - "Clusters"
Describe alternatives you've considered
Additional context

To confirm, you mean the clusters shown on the /home page?
We're hoping to implement this via the new server-side paginated process in 2.10.0 - https://github.com/rancher/dashboard/issues/10283

@richard-cox Yes  on "/home" it sprung into my eyes, but pagination is probably useful in all screens when too many resources are shown.
Thx for the infinfo/link somehow missed that one.

@strowi No problem. Most UI issues are over at https://github.com/rancher/dashboard
Almost all lists in the new ui are paginated, however it's done locally in the browser which does not scale well. We're slowly changing these to paginate in the backend, hopefully we're have more news soon


OAK-D PRO don't have depth topic

Hello, I'm using Turtlebot4 and OAK-D PRO, OS is Ubuntu22.04 and ROS2-humble, when I call ros2 topic list, the camera only have these topic:
/oakd/rgb/preview/camera_info
/oakd/rgb/preview/image_raw
/oakd/rgb/preview/image_raw/compressed
/oakd/rgb/preview/image_raw/compressedDepth
/oakd/rgb/preview/image_raw/theora
Is there any NOT compressedDepth topic? Because I wanna use RTAB-map, so is there any settings can change to depth(not compressed)? Thank you.

Hi, this can be the result of your config file, more specifically pipeline type. You can find more on pipeline types here

@Serafadam  Hello, thanks for replying fast. When I run camera.launch.py, it shows:
[component_container-1] [ERROR] [1722860415.251114816] [oak]: No available devices
[component_container-1] [INFO] [1722860415.251352386] [oak]: No ip/mxid specified, connecting to the next available device.
Is there any way can fix this?

Hi, I think it would be best if you go through the steps described in our bug report template. You can also check this page for general information regarding connectivity issues

Hello, sorry for the late reply, so after I connect to laptop the camera works find(USB3.0),but why it's wrong with rtabmap, all of settings are default, and I run: ros2 launch depthai_ros_driver rtabmap.launch .py, but it feels so lagging, and it shows very weird, do you know where might be the problem?

Hi, based on this image I think it is related to odometry losing track of features, this can be cause by fast movements or exposure changes, you can retrigger the odometry via ROS service or RTABMap's GUI. Regarding latency, it is probably related to ROS2 DDS, you can find some more information on improving the latency here

Thanks for replying fast, but I run visualized_pointcloud.py( default setting), which is in depthai_ros_driver, it shows like this, I am wondering if it's camera problems?
This is far:

This is close:

These are actually similar to rtabmap

@Serafadam Did you ever try rtabmap before? If you did it do you mind let me see your parameters? Thank you.


Missed synchronization

While investigating other issues I’ve found probably few hard to reproduce issue.
I haven’t seen them happening, but I’ve seen several issues which are claimed as hard to reproduce/happens under heavy load, so may be it could help there. In original PR (where code has been introduced) that was mentioned also.
This block accessing sockets array https://github.com/dotnet/SqlClient/blob/8ad8da6bc1f7314ec37b5fda1b4e991a878a6072/src/Microsoft.Data.SqlClient/netcore/src/Microsoft/Data/SqlClient/SNI/SNITcpHandle.cs#L341
is not synchronized with this one https://github.com/dotnet/SqlClient/blob/8ad8da6bc1f7314ec37b5fda1b4e991a878a6072/src/Microsoft.Data.SqlClient/netcore/src/Microsoft/Data/SqlClient/SNI/SNITcpHandle.cs#L368
The following order seems is possible:

if (sockets[i] != null && !sockets[i].Connected) is true

if (sockets[i] != null) gives true
if (sockets[i].Connected) gives true
sockets[i].Dispose();
availableSocket = sockets[i];
break; return availableSocket;

In this case socket is first returned and then at random time disposed while being used (user does not receive any exceptions, connection just disposes).
But there are also many hard investigate synchronization/exceptions things there. I think may be that entire file and related should be checked/reviewed again.

the Clear local function is only invoked from the cts, the cts is disposed in the finally block before the socket is returned to the user. So yes there is a small window between the break and finally where the socket could be disposed.

In the scenario I'm offering, there are already two threads (cancellation has been already started while Connect is still in progress). So first cancellation thread passes all "fail" conditions, then main thread passes all "success" conditions and returnes the socket. Dispose does not affect this scenario anyhow.
What's left: socket is returned and being used, same time cancellation thread is scheduled to dispose that socket and I believe timing of that can not be anyhow estimated. For example, if cancellation thread came from pool, it may be has just finished heavy job and is now planned after all other N threads to finish same heavy N-jobs. As you can imagine, this case is nearly impossible to reproduce on purpose. I've just searched issues by "ObjectDisposed" keyword and there are at least 2 from top 20 which describes similar behavior:
https://github.com/dotnet/SqlClient/issues/449
https://github.com/dotnet/efcore/issues/15069

How would you fix it though? If you ref the socket and then check for null/disposed before you return after the finally how do you recover from it being null/disposed?

I don't see any other option but to do some synchronization. Probably fastest would be to introduce "interlocked exchange". But this code also relies on try/catch{} and number of non-reproducable issues should grow too fast to maintain.
The best in my opinion would be to introduce Socket.Connect(timeout) (timeout of IO operation seems logical to me).
Or if we can not expect changes to Socket itself, we still can use Socket.ConnectAsync which returns Task and thus we can use simple Task synchronization methods.

If the need were acute the method could be added to the runtime but that's an issue for /runtime and even if it were added the first shipping product that contained it would be .net6 and we couldn't use it in this library on downlevel runtimes so it wouldn't be very useful.
I understand what you're saying and agree that there is the possibility of returning a disposed socket. However there is no causation proof and I've debugged my way through too many incorrect assumptions to think that we can assert that this causes an observable problem. Is there a way that we can either 1) fix the issue you've highlighted without externally observable effect? or 2) enhance the tracability of the ObjectDisposedExceptions allowing us to see if this problem is manifesting?

However there is no causation proof and I've debugged my way through too many incorrect assumptions

I have found this issue to be reported and reproduced, repro is here: https://github.com/dotnet/SqlClient/issues/449#issuecomment-591552032 It describes exactly same behavior as I do.
Randomness of this issues are predicted by the theory.
In 449 we were lucky the customer to report and repro the issue.

Some other cases I see got closed as non-reproducable.
But the impossibility to reproduce them (or difficult repro) is supposed by current SqlClient solution.
The correctness of assumptions in this tickets can be proven by static code-analysis tools. This is my next point: code-anlysis is proposed specifially to avoid hard to observe issues and I believe this should be used in SqlClient.

fix the issue you've highlighted without externally observable effect?

I think last option which I offered should do this: ConnectAsync should be used for easy synchronization of 2 tasks. Or again any synchronization eventually will do the thing.

enhance the tracability of the ObjectDisposedExceptions allowing us to see if this problem is manifesting?

As I said I see it's much more easier to fix the thing relying on code-analysis than to trace it.

The best in my opinion would be to introduce Socket.Connect(timeout) (timeout of IO operation seems logical to me).

Am not following this issue in details, but if you're looking for a socket timeout without doing async, then there's already a way for doing this. You can set the socket to non-blocking, call Connect (at which point you get a WouldBlock exception), and then use Socket.Select to wait for the connection to complete, but with a timeout. It's not incredibly pretty but it works.

The best in my opinion would be to introduce Socket.Connect(timeout) (timeout of IO operation seems logical to me).

Am not following this issue in details, but if you're looking for a socket timeout without doing async, then there's already a way for doing this. You can set the socket to non-blocking, call Connect (at which point you get a WouldBlock exception), and then use Socket.Select to wait for the connection to complete, but with a timeout. It's not incredibly pretty but it works.

Thank you, this would work for us. But the code which we are discussing has been introduced under the sign of performance. Probably, we should avoid exceptions at least on success execution path

@jinek as this is code which establishes a physical network connection (i.e. TCP handshake), I really doubt this extra exception would have any impact at all. I'd at least set up a quick BenchmarkDotNet benchmark to verify this rather than assuming it's true.

Perhaps we're overthinking this. The caller handles null and not-connected cases by returning an sql error so why not just check the connected state of the return candidate after the cts has been disposed but before returning the value. At that point it can no longer be touched by the timer so it it's alive it'll stay alive and if it's dead we clean it up and return null allowing the caller to interpret that as a standard failure to connect, which is what it is.

why not just check the connected state of the return candidate after the cts has been disposed but before returning the value.

The socket passes checks (as it got connected) while same time Cancel method is already running.
CTS disposal does not abort Cancel thread.

At the point when the cts is disposed either the cancel function has been invoked or it will not be invoked, as such either cancellation is done/pending or won't happen. But yes you're right due to the vagaries of threading we can't assume anything about orering. So the socket array needs to change to a (Socket, int) class array and interlocks on the int used to guard socket access, 0-1 on cancel 0-2 on take. Have I ever mentioned that threading and async give me headaches? I also disklike allocations. I don't like there being an array allocated for just two things, especially when most of the time it's never going to hit the second one, and that second socket never gets disposed.
Fancy working on this?

either cancellation is done/pending or won't happen

By the way, anything in the world is either happened/gonna happen or won't happen 🤣

Fancy working on this?

I would love sqlclient to work with multithreading. I'm still not sure about implementation details, but something like you mention should happen.

@jinek thanks for bringing this up. While I was working on issue #422 I came a cross an error message on debug mode. (@Wraith2 you can reproduce the issue by attaching your project to the driver and run it under netcoreapp3.1-Debug. Do not forget to add the switch to use ManagedSNI). The error message was stating that we were trying to write to a disposed socket.

I think last option which I offered should do this: ConnectAsync should be used for easy synchronization of 2 tasks. Or again any synchronization eventually will do the thing.

There has been anther issue in system.Data.SqlClient regarding the socket in connectAsync that the team had to change the socket connection to synch from async. I will find the exact issue number and will update here.

I've tried to run benchmark for implementations which we have discussed here.
Solution offered by @roji almost equal in time to async solution:

Method
CurrentSNITcpSolution
Mean
Error
StdDev
Median

Connect
Original
9.396 us
0.7643 us
1.473 us
8.800 us

Connect
ViaException
9.427 us
0.6319 us
1.247 us
8.700 us

Connect
ViaAsync
8.800 us
0.7240 us
1.324 us
8.350 us

Code is here: https://github.com/jinek/SqlClient/commit/e9e73099e7afb5448447af210874309c8b66e7ca
Here are implementations with async and exception (I've created delegate to substitute implementations at runtime):
Async
        public static Socket GetAvailableSocketViaAsync(int port, TimeSpan timeout, bool isInfiniteTimeout,
            IPAddress[] ipAddresses)
        {
            if (isInfiniteTimeout) //todo: for performance tests not considering infinite timeouts
                throw new NotSupportedException();

            Task timeoutTask = Task.Delay(timeout);

            foreach (IPAddress ipAddress in ipAddresses)
            {
                var socket = new Socket(ipAddress.AddressFamily, SocketType.Stream, ProtocolType.Tcp);

                Task connectTask = socket.ConnectAsync(ipAddress, port);

                if (Task.WaitAny(timeoutTask, connectTask) == 1)
                {
                    if(connectTask.Status==TaskStatus.RanToCompletion)
                        return socket;

                    Exception theOnlyException = connectTask.Exception.InnerExceptions.Single();
                    if (theOnlyException is SocketException)
                    {
                        socket.Dispose();
                    }
                    else
                    {
                        throw new NotSupportedException("Not expected socket error", theOnlyException);
                    }
                }
                else
                {
                    connectTask.ContinueWith(task =>
                    {
                        Exception theOnlyException = task.Exception.InnerExceptions.Single();
                        switch (theOnlyException)
                        {
                            case ObjectDisposedException _:
                            case SocketException _:
                                break;
                            default: // throwing to TaskScheduler.UnobservedException
                                throw new NotSupportedException("Not expected socket error", theOnlyException);
                        }
                    }, TaskContinuationOptions.OnlyOnFaulted);
                    socket.Dispose();
                }
            }

            return null;
        }

Exception
        public static Socket GetAvailableSocketViaException(int port, TimeSpan timeout, bool isInfiniteTimeout,
            IPAddress[] ipAddresses)
        {
            if (isInfiniteTimeout) //todo: for performance tests not considering infinite timeouts
                throw new NotSupportedException();

            Stopwatch stopwatch = Stopwatch.StartNew();

            foreach (IPAddress ipAddress in ipAddresses)
            {
                var socket = new Socket(ipAddress.AddressFamily, SocketType.Stream, ProtocolType.Tcp)
                {
                    Blocking = false
                };

                bool socketSelected = false;

                try
                {
                    socket.Connect(ipAddress, port);
                    throw new NotSupportedException($"Call to {nameof(Socket.Connect)} must throw {nameof(SocketException)} with {SocketError.WouldBlock.ToString()} error code");
                }
                catch (SocketException socketException) when (socketException.SocketErrorCode ==
                                                              SocketError.WouldBlock)
                {
                    // https://github.com/dotnet/SqlClient/issues/826#issuecomment-736224118

                        var timeLeft = timeout - stopwatch.Elapsed;

                        if (timeLeft <= TimeSpan.Zero)
                            return null;
                        try
                        {
                        Socket.Select(null, new List<Socket> {socket}, null,
                            (int)(timeLeft.TotalMilliseconds * 1000));
                        }
                        catch (SocketException) { }

                        if (socket.Connected)
                        {
                            socketSelected = true;
                            socket.Blocking = true;
                            return socket;
                        }

                }
                finally
                {
                    if (!socketSelected)
                        socket.Dispose();
                }
            }

            return null;
        }

Original was
        public static Socket GetAvailableSocketOriginal(int port, TimeSpan timeout, bool isInfiniteTimeout, IPAddress[] ipAddresses)
        {
            Socket[] sockets = new Socket[2];
            CancellationTokenSource cts = null;

            void Cancel()
            {
                for (int i = 0; i < sockets.Length; ++i)
                {
                    try
                    {
                        if (sockets[i] != null && !sockets[i].Connected)
                        {
                            sockets[i].Dispose();
                            sockets[i] = null;
                        }
                    }
                    catch
                    {
                    }
                }
            }

            if (!isInfiniteTimeout)
            {
                cts = new CancellationTokenSource(timeout);
                cts.Token.Register(Cancel);
            }

            Socket availableSocket = null;
            try
            {
                for (int i = 0; i < sockets.Length; ++i)
                {
                    try
                    {
                        if (ipAddresses[i] != null)
                        {
                            sockets[i] = new Socket(ipAddresses[i].AddressFamily, SocketType.Stream, ProtocolType.Tcp);
                            sockets[i].Connect(ipAddresses[i], port);
                            if (sockets[i] != null) // sockets[i] can be null if cancel callback is executed during connect()
                            {
                                if (sockets[i].Connected)
                                {
                                    availableSocket = sockets[i];
                                    break;
                                }
                                else
                                {
                                    sockets[i].Dispose();
                                    sockets[i] = null;
                                }
                            }
                        }
                    }
                    catch
                    {
                    }
                }
            }
            finally
            {
                cts?.Dispose();
            }

            return availableSocket;
        }

As for be both solution looks good, but solution by @roji does not have async code, which surely looks more attractive for sync code. Additionally, it allows to exclude the path of exception for infinite timeout case.
@JRahnama Has been that "async" issue logged in this repository?

@jinek the issue and its related materials could be followed at issue #583 and it also can be tracked back to a change in 2018 in corefx at this issue.

@jinek the issue and its related materials could be followed at issue #583 and it also can be tracked back to a change in 2018 in corefx at this issue.

I see, thank you. Can we try Socket.Select(null, new List<Socket> {socket}, null, (int)(timeLeft.TotalMilliseconds * 1000)); as in code above?


Detecting faces in an image read out of a unix socket

face_recognition version: 1.0.0
Python version: 3.5.2
Operating System: Ubuntu 16.04.3

Description
I'm trying to detect faces in an image received from a unix socket
What I Did
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.connect('/path/to/socket')
data = False

while True:
	buf = sock.recv(4096)

	if not buf:
		break

	if not data:
		data = buf
	else:
		data = data + buf

face_locations = face_recognition.face_locations(data)
face_encodings = face_recognition.face_encodings(data, face_locations)

But I get the error Expected writable numpy.ndarray with shape set.
What do I need to do to get it to work?

I forgot to load the image first. I tried it like so:
image = face_recognition.load_image_file(data)

But that gave an embedded null byte error. I then wrapped the data variable in a BytesIO call, and that did it
from io import BytesIO

sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.connect('/path/to/socket')
data = False

while True:
	buf = sock.recv(4096)

	if not buf:
		break

	if not data:
		data = buf
	else:
		data = data + buf

image = face_recognition.load_image_file(BytesIO(data))
face_locations = face_recognition.face_locations(image)
face_encodings = face_recognition.face_encodings(image, face_locations)

Great, thanks for sharing the answer. This should be helpful to anyone trying to figure this out later.


KDE/Kinoite Night Color Broken

Hey folks, I recently rebased from regular Kinoite to ublue-Kinoite upon switching to a Nvidia GPU. Everything seems to work so far, except KDE's Night Color feature doesn't appear to change the color temperature as it should. Is this a known issue and is there a fix? Thanks.

Known issue nvidia driver doesn't support GAMMA_LUT. Nvidia needs to fix it

ref: https://github.com/NVIDIA/open-gpu-kernel-modules/issues/162


chore(main): release 0.6.2

:robot: I have created a release beep boop
0.6.2 (2024-04-08)
Bug Fixes

resolve block sizing issues and paragraph margin issues (436a2fb)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/commonhorizoncommun/design-system/releases/tag/v0.6.2 :sunflower:


Add GhostRadio to allow ghosts to hear radio messages

About the PR
Resolves #3708
Changelog

:cl:

add: Ghost have now attuned themselves to radio waves allowing them to listen in on the station chatter.

this is really sus since its duplicated a but but its fine for now i think, until i finish my radio ecs thing


Bugfix proportion aggregator

In numpy 1.15.2, if you construct a masked array without explicitly setting the mask to False, the array mask is a boolean value rather than an array:
>> import numpy.ma as ma
>> array = ma.array([1, 2, 3], mask=False)
>> array.mask
array([False, False, False])

>> array = ma.array([1, 2, 3])
>> array.mask
False

This is a catch for that scenario which calculates the number of unmasked points using the array itself, not the mask.
The assumption here is that if numpy returns a boolean as an array.mask, it will only ever be False.

@bjlittle Tests are failing, but looks like it could be a cartopy issue causing it rather than this.

See #3230 regarding test fails.

Looks like you have changes that don't belong here now. I guess your branch is based on master but is now targeted for merge into v2.2.x. This needs to be resolved before this can be merged.

@corinnebosley Can you cherry-pick your commit onto a branch based on 2.2.x instead.
As @ajdawson mentions, otherwise you'll be pulling in master branch changes onto the 2.2.x branch, instead of the other way around.
Thanks

Replaced by #3233


use npm version to tag releases

@sgress454 @mikermcneil can you guys use npm version to tag releases, and then just git push --tags? It looks like 0.12-rc2 is published to npm, but there's no corresponding tag on github. It makes it difficult to search and debug issues in untagged releases.

Yup, sorry about that.  Pushed tags for v0.12.0-rc1 and v0.12.0-rc2.


Do not touch the original file if the result is identical to the original

Fixes #357

Thanks, this is great!
I added a couple tests and merged.


Remove anonymous sharing

As outlined in #802, anonymous sharing is being removed from github gists.
This CL:

Links users who wish to share to the sharing guide.
Changes the export button to open the current gist id instead.

Testing URL: https://dartpad-test.appspot.com/embeddings_demo.html

@mit-mit, fyi, new sharing instructions are here: https://github.com/dart-lang/dart-pad/wiki/Sharing-Guide

And:

just re-directs to that page

This looks good!
A small iteration on this would instead open a model dialog when the user hits Share.... I think this would be slightly less surprising. The dialog could contain some brief verbiage about sharing + gists, and offer a link to the wiki page on sharing for more details.
Landing this PR is a P0 however, and any further tweaks to the UI are lower priority.
Thanks!

It looks like the build step on travis in unhappy: https://travis-ci.org/dart-lang/dart-pad/builds/354077210#L530, but it's not immediately clear why.

Landing so we can push to prod.

It looks like the latest dev version of dart2js hits compile issues wit several packages that are transitive dependencies of ours. I'm going to roll us back to depend on a dart2js version w/o these issues.

(the travis issue was fixed in https://github.com/dart-lang/dart-pad/pull/804)


Add demo image classification models and datasets

Enable Skin Cancer detection with single label.
It takes around 4 minutes to predict 1285 images in test split of marmal88/skin_cancer datasets:

oh and @bmalezieux, make sure you use the pre-commit hook from this repo since the lint is failing

I aligned all datasets and models to use string of class labels now.
There are 2 new notebooks to try all models and datasets on.

I would just add tests to the core stuff (no need to test the demo dataloaders or models) rather the general wrappers


Add dart docs to clients index

Depends on https://github.com/edgedb/edgedb.com/pull/526

This was merged rather prematurely and broke the site build

(Context: I'm trying to update the Prod but couldn't do that because the Dart PR in edgedb.com isn't yet merged and I'm not sure it's there)


ENH: Add DWIWorkflow python file to AutoWorkup directory

Usage:
DWIWorkFlow.py
--inputDWIScan <path/to/DWI/image>
--baseDIR <path/to/the/output/directory>
This pull request also includes two patches already submitted for fixing 'gtractResampleDWIInPlace' bugs.

@hjmjohnson
Hans,
Please review the python file for DWI processing pipeline.
This workflow is run an tested on a sample input DWI image. I have created a presentation of  all intermediate files of this processing pipeline:
http://prezi.com/1noeohp9p7hk/?utm_campaign=share&utm_medium=copy&rc=ex0share
Thank you,
Ali


Use BeamDeprecationWarning in BQ client code.

Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

retest this please

Run Python PostCommit

R: @pabloem


Integrate mongodb-crypt module

Description
This PR relocates the mongocrypt Java bindings from the libmongocrypt repository to the mongo-java-driver repository to streamline maintenance and the release process.
Key changes to the Gradle build script, originally copied from the libmongocrypt repository, include:

Removal of Maven publication configuration; now managed by the publish.gradle script.
Removal of signing configurations; managed by the the publish.gradle script.
Removal of explicit Slf4j dependency, as it is inherited from the root build script located here.

Review Process
To aid the review process, I have taken several steps:

Downloaded the mongocrypt version 1.11 from Maven Central and unzipped the source, Javadoc, runtime jar, and pom into the git folder.
Run a Maven local publication specifically for the mongocrypt module using the command gradle mongocrypt:clean mongocrypt:publishToMavenLocal -x signMavenJavaPublication to produce a publication. Subsequently, I unzipped and applied a git diff over the ones from version 1.11. The diff can be viewed here.

Notable differences include many Javadoc variations due to the mongocrypt module inheriting configurations from javadoc.gradle. Key areas to focus on are the pom and manifest files, along with some source files modified by the Spotless plugin, which corrected redundant imports.
I have built the jar files using JDK 11.0.24 (Homebrew), which is the same version used to build 1.11 currently on Maven Central.
Note for Reviewers:
I would appreciate it if reviewers could be particularly vigilant when reviewing the build configurations and produced artifacts in this PR. If you have any doubts or cannot be certain that the changes do not impact previously guaranteed functionalities, please do not hesitate to raise your concerns.
NOTE: Publishing snapshot to Sonatype has not been tested yet.
JAVA-5582
JAVA-5583

I'm holding off on merging this as I noticed an issue after running the publish snapshot. The driver-core POM in the snapshot no longer marks mongodb-crypt as an optional dependency, despite it being specified as such in the build configuration: implementation project(path: ':mongocrypt', configuration: 'default'), optional. The setting isn’t being applied correctly. Here's the POM for reference.

Hoping to have fixed the 'optional' support - will review the build in the morning

@rozza do you expect/desire that the optional mongodb-crypt dependency appears in driver-core, driver-sync, and driver-reactive-streams? Currently it's only in driver-core (even though driver-core doesn't actually use it!).

I think the latest changes might have broken the build.  I see a lot of
FAILURE: java.lang.NoClassDefFoundError: com/mongodb/crypt/capi/MongoCryptOptions (java.lang.NoClassDefFoundError)

Thanks, @rozza for your help on this issue!
I identified the root cause of the issue with the existing nebula plugin for optional dependencies. The mismatch between the mongocrypt submodule name and the artifactId mongodb-crypt used during publishing caused the plugin to skip appending the optional flag as detailed in this plugin code.
To address this, I renamed the module name to mongodb-crypt while keeping the directory as mongocrypt. All tests pass, and the artifact now correctly includes the optional flag.
I've pushed these changes to a separate branch to avoid overriding current changes and I've run the full set of Evergreen tests, and all tests have passed successfully. You can review the results here.
The commit with the fix is here: 6e3e7068f1b0f2648a2fa0610e2c13a3f1dd2e3f
@jyemin @rozza please review it, and if everything looks good, feel free to cherry-pick it into this PR.

@vbabanin your change looks good, great catch with the naming.  I'm thinking renaming the directory to match the archive name also makes sense. Then we don't have to worry about changing the projects name in settings.gradle

@rozza do you expect/desire that the optional mongodb-crypt dependency appears in driver-core, driver-sync, and driver-reactive-streams? Currently it's only in driver-core (even though driver-core doesn't actually use it!).

I was trying to workaround the feature support limitations but seems it failed anyway. driver-core has to take a dependency as some of the options classes shared between sync and reactivestreams use the library.  Thankfully, @vbabanin discovered the cause of the plugin failing 👍

evergreen retry


IPv6 support

Hello!
Can this library be used with IPv6? Looking at Elixir's Postgrex I believe I need to add inet6 as a socket option but I could not spot a way to do this.
Thanks,
Louis

I don't recall. Should be simple to fix if not.

Fab, thanks. I've been looking at the source code and it looks fairly straightforward.
Do you have a preference for what the API should be?

-type pool_config() :: #{host => string(),
                         port => integer(),

                         %.... etc

                         %% TCP socket options
                         ipv => inet | inet6.

I had a crack at this by allowing the user to specify the inet6 value to pass to gen_tcp:connect (as you can see here https://github.com/erleans/pgo/pull/55) but I was unable to get it to work.
I can't see any differences between this and how Postgrex works when given socket_options: [:inet6], but in my test application on fly.io's ipv6 network I can connect with Postgres but not with PGO.
Can you see anything I've done wrong? I'm afraid I'm not very familiar with gen_tcp.

I don't see postgrex doing anything special for ipv6 and checked the fly.io docs and there is no mention of passing an option to ecto to setup ipv6, https://fly.io/docs/getting-started/elixir/#generate-the-app-and-deploy-with-postgres

Probably easiest if we just ping @ericmj :)

I've spoken to the folks at fly and they say you need to add socket_options: [:inet6] for it to work, which seems to be the case in my testing with Postgrex. Their documentation is a little patchy at the moment.
As far as I can tell adding the socket_options: [:inet6] option just adds the inet6 option to gen_tcp:connect call, so I expected that to be sufficient for PGO, but it still failed in my test.

Hm, ok. I assume there is no useful error message or logs? I guess I'll just need to give this a try so I can play around with it.

Not that I can see, but I am a bit unwell so I may be messing something obvious up.
I'm starting the connections like so
    {:ok, pid} =
      Postgrex.start_link(
        hostname: hostname,
        username: database,
        password: password,
        database: database,
        port: port,
        socket_options: [:inet6]
      )

    :pgo.start_pool(:default, %{
      host: String.to_charlist(hostname),
      port: port,
      user: String.to_charlist(database),
      password: String.to_charlist(password),
      database: String.to_charlist(database),
      pool_size: 5,
      ipv: :inet6
    })

    Process.register(pid, :postgrex_conn)

And using them like this
    :pgo.query("select 1", [], %{pool: :default})
    |> IO.inspect(label: :pgo)

    Postgrex.query(:postgrex_conn, "select 1", [])
    |> IO.inspect(label: :postgrex)

Nothing is printed by starting the connections, and PGO returns {:error, :none_available} after a few seconds when making a query.

I just tried @lpil 's ipv6 branch from #55 and it allowed me to connect to a local postgresql instance with listen_addresses = '::1' in /etc/postgresql/13/main/postgresql.conf, using this config:
[
 {pgo, [{pools, [{default, #{pool_size => 5,
                             host => "0::1",
                             database => "mydb",
                             user => "postgres",
                             password => "postgres",
                             ipv => inet6}}]}]}
].

(I cloned the repo, modified config/example.config, and successfully executed a query from rebar3 shell)

Ah fab, thanks. I've updated my PR to accept a list of socket options and I've asked a question on the Fly forums to see if there's something else they can think of that would prevent connections.

I added printing of this error here
https://github.com/erleans/pgo/blob/e356380d0ab71ae2139a4c84763a2ac02c1e40fa/src/pgo_connection.erl#L95
And I found this:
{error,{unknown_message,authentication,<<0,0,0,10,83,67,82,65,77,45,83,72,65,45,50,53,54,0,0>>}}

Where the binary decodes to
^@^@^@
SCRAM-SHA-256^@^@

So I think the problem is that PostgreSQL is using a newer authentication method that PGO does not yet support. I'll open a new issue.

 That has been on my list of things to do... I since postgrex supports it that it'll be easy to add since I can just copy the code :)

Support has been merged in, can close this after the next release

v0.12.0 was released


node version requirement, env var cleanup

Issue: some of the syntax like optional chaining used in this app requires at least node version 14 and higher source This app currently doesnt check for the minimum required version in the package.js.
Issue: the package-lock.json includes the dotenv module but doesnt load it anywhere in the app, making a .env file useless. would be nice to have it required and loaded somewhere or add in the readme that you need TERRA_LCD, TERRA_MANTLE, TERRA_CHAIN_ID, optional START_BLOCK_HEIGHT passed in the command line.
Issue: unknown -d option

><EMAIL_ADDRESS>migration-run
> node --require ts-node/register -r tsconfig-paths/register ./node_modules/typeorm/cli.js migration:run -c migration -d src/orm/migrations

cli.js migration:run

Runs all pending migrations.

Options:
  -h, --help         Show help                                         [boolean]
  -c, --connection   Name of the connection on which run a query.
                                                            [default: "default"]
  -t, --transaction  Indicates if transaction should be used or not for
                     migration run. Enabled by default.     [default: "default"]
  -f, --config       Name of the file with connection configuration.
                                                          [default: "ormconfig"]
  -v, --version      Show version number                               [boolean]

Unknown argument: d

from my understanding this d option is to specify the directory for generating a subscriber. It isn't an option for running or generating a migration. Also, the default directory is already defined in the ormconfig file so passing into the npm script is redundant. Removing the -d flag fixes the migration command.
Besides the errors, some refactoring could also be done to also remove the redundancy in the npm script.

Thanks for comment

I'll update requirement after the graphql stuff is finish.

just added about .envrc

dotenv is requirement of typeorm so it is needed

migration-run option fixed

Thanks. 😀


Download URL Scraping is Broken

>>> pytinytex.download_tinytex()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/envs/pandoc/lib/python3.9/site-packages/pytinytex/tinytex_download.py", line 24, in download_tinytex
    tinytex_urls, _ = _get_tinytex_urls(version, variation)
  File "/opt/anaconda3/envs/pandoc/lib/python3.9/site-packages/pytinytex/tinytex_download.py", line 80, in _get_tinytex_urls
    version = tinytex_urls_list[0].split('/')[5]
IndexError: list index out of range

ohh yes @jmsmdy
It's broken because of the recent change in how GitHub populate theirs release page. I'll write a fix for it within the next couple of days


fix(task-kubernetes:apply-ks): --dry-run was appended to the path, ma…

Hello @onedr0p,
the task kubernetes:apply-ks  was causing the PATH to be invalid it appended the --dry-run to the path.

thanks!


CKEDITOR is not defined

Error: CKEDITOR is not defined
at CKEditor.ckeditorInit (http://localhost:3000/vendor.bundle.js:47779:13)
at CKEditor.ngAfterViewInit (http://localhost:3000/vendor.bundle.js:47748:13)
line 47779:
if (!CKEDITOR) {
console.error('Please include CKEditor in your page');
return;
}
implementation:
directives: [
XLarge, CKEditor
],
in the template:
<ckeditor
[(ngModel)]="ckeditorContent"
[config]="{uiColor: '#99000'}"
debounce="500">

Did you include ckeditor library (https://github.com/chymz/ng2-cli-ckeditor-sample/blob/master/src/index.html#L17) ?


access: restore go-approvers and security GitHub memberships

It's me again.
I think the Google bot unceremoniously dropped me from the whole Go organization.
It'd probably make sense for me to be in

https://github.com/orgs/golang/teams/go-approvers
https://github.com/orgs/golang/teams/security
(to get @golang/security cc's, even if I'm not going to be active on them for a bit)

See https://go.dev/wiki/GithubAccess

@dmitshur @rsc PTAL

Yep, googlebot tends to do this. Re-invited. Welcome back!


Backtest

How do you backtest your strategy?
a backtest tool will be very useful.

It does not provide and no plan to provide backtest. Sorry.


BottomBar behind navBar

Updated 29/08/2016
- layout code
Hi, thanks for your work roughike your update is awesome.
I just have a little issue, the bottomBar stay hidden behind the navBar even if i'm not using the "underNavBar" behavior, I would like to put it at the top of the navBar.
I tried to use fitsSystemWindows with no success.
Update :
The BottomBar is inside a Fragment
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    xmlns:app="http://schemas.android.com/apk/res-auto"
>
        <com.roughike.bottombar.BottomBar
            android:id="@+id/bottomNavigation"
            android:layout_width="match_parent"
            android:layout_height="60dp"
            android:layout_alignParentBottom="true"
            app:bb_tabXmlResource="@xml/bottom_navigation_tabs"
            app:bb_activeTabColor="@color/colorAccent"
        >
    </com.roughike.bottombar.BottomBar>
</LinearLayout>

What does your layout containing the BottomBar look like?

Thanks for your answer, i updated my post with the bottomBar layout file

@NekoNX you are supposed to use the underNavBar behavior — that way it will add additional space at the bottom of the layout to accomodate for the system nav bar.


Error message when there is no error

gulp: 3.9.1
gulp-sass: 3.0.0
After I tried to run my gulp-sass task with a .scss file that a already used in other projects without problems, I got a strange error message that didn´t make any sense.
the original code in my .scss file is this:
.notice {
	visibility: hidden;
	position: fixed;
	z-index: 9999;
	right: 0;
	left: 0;
	bottom: 0;
	background-color: #000000;
	color: #fff;
}

the error message i got was this:
Invalid CSS after "...tice {":
expected "}", was "{"
    relativePath:
src/core/scss/components/notice.scss

I first thought something might be wrong with my file since there was no code error in it. So i decided to delete that file and created a testfile. In this completely new testfile that I placed in the same directory the other file was in before I wrote in the following:
body { background-color:#f00; } and I started the sass task again.
This error message is what I got:
Invalid CSS after "...nd-color:#f00;}":
expected 1 selector or at-rule, was "{}"
        on line 1 of src/core/scss/components/temp.scss
>> body{background-color:#f00;} {}
   -----------------------------^

this file is completely new, I didn´t copy and paste something in it so there are no hidden characters
or something like that in it . What can it be? And what can I do?
The strange thing about it is that I changed the src  path in the sass task to another scss directory and ran the task and everything was good as expected.

I found out after a lot of testing what the problem was. Somehow the option indentedSyntax:true caused this error. When I removed it everything worked ok again

Indented syntax is Sass' old syntax
body
  background-color: #f00

It uses an entirely different parser so the SCSS syntax isn't valid.


URL Template escapes port numbers

If I set up a URL Template node with http://{base}/api, and use localhost:11111 as the input, the node will output http://localhost%3A11111.

Believe it or not, there's a whole extra syntax hidden in there:
https://breadboard-ai.github.io/breadboard/docs/kits/template/#the-urltemplate-component
It supports the https://datatracker.ietf.org/doc/html/rfc6570, I believe in your case, you need to do http://{+base}/api. Definitely need to document this better.


Restore Lasso

Write proposal for lasso.

[ ] Add vr teleport / smooth walk
[ ] Add two touch based picking

Lasso for non-VR (raycast / dot in the screen to select something. maybe outline highlight)

Lasso is restored, but can't pickup objects https://github.com/V-Sekai/v-sekai-game/issues/40.


Preview mode apply text color

Seems like there is no text color customization for preview mode apply text via attrs
bottomToolbar.apply.textColor
That one isn't applied for suggested property.
Thank you.

@DummyCo
I thought it was not necessary to customize Back and Apply button text, because the page background is black. But you can contribute if you want this feature. And if you do contribute, differ buttons on different UI cause they have different background colors.


Reconsider package structure

For quite a long time QM has followed Yarn's package structure and came up with its own rules for the package structure.
Lately I've been looking a bit more closely at the Mojang mappings, and given the recent refactors on Mojang's part to increase the data-drivability of Minecraft, I'm finally starting to understand better the Mojang package structure.
So now we probably should ask ourselves, is the QM package structure still making sense given the refactors?
Advantages of using Mojang's package structure

Ability to map the already existing package-info.class files;
Not have to wonder where to put classes when mapping;
Mappings are easier to juggle with mojmap, making it easier to understand for modders if accustomed with mojmap;
Package structure might make more sense given the direction taken by Mojang for the future of the game.

Disadvantages of using Mojang's package structure

We constrain ourselves to the Mojang package structure.

Unresolved Questions

Do we use the exact same package names or do we adapt to our naming convention?
Do we reconsider some of our naming convention in the process?
What will be the public reception?

I agree that using the Yarn's package structure is less better than Mojang's one because of the recent... Codecify? of minecraft resources. But I do not think that following exactly the same package structure as Mojang is a good idea. IMO, Quilt Mappings should be able to change some parts of the Mojang package structure to use it's own conventions.
As the the public reception, I do think that this change should be applied for Quilt Mappings 2.0. People currently knows that this version will bring a lot of changes to these mappings, so I think that it's the best moment to consider a package structure refactor.

To be perfectly honest, as a modder I do not look at package names much. That said, I believe following mojang's package structure would have 2 main benefits :

tracking mojang's intent more closely
eliminating concerns about package-private elements

For consistency, I would say that packages should be renamed when they are about a concept that has been remapped (e.g. world).

eliminating concerns about package-private elements

Having the same package structure as Mojang also heavily simplifies the usage of QM on NeoForge for those who dare want to do that as NeoForge is using Mojmap as the runtime mapping meaning there's no visibility remap step.

I support this move, just wanted to chime in with a few points:

if we choose not to copy mojang's names (imo we should continue using our own) what do we do about net/minecraft/unmapped? we would need to establish some sort of manual map from moj package names to QM package names, defaulting to moj names for new, unmapped, packages
whether we reconsider our conventions can be considered once we start refactoring. it would be fun to have the whole team on an enigma multiplayer doing the work together!
as pyro said, I don't think any mappings users will be opposed

I mostly like the idea, i would need to see the mappings with the new structure to really get a feel for it.

so. plan time
this will require a few things on the tech side to assure validity. in terms of setting up packages, my proposal is that:

we establish a JSON file (like simple_type_field_names) that acts as a map of mojang package names to QM versions. all classes stay organised the same way as mojang, but we get to rename packages.
we build an enigma plugin that disallows changing package names from enigma, redirecting mappers to said file. this plugin also automatically fills out the JSON file as new mojang packages are added, simply adding an entry of "x/y/z": "x/y/z" whenever a package is missing. this plugin also handles automatically moving new classes to the proper package as dictated by mojmap.

this will require some new API in enigma: plugins that add checks for the validity of a rename. should be fairly simple to implement

we automatically download a mojmap JAR as part of QM setup. analysing this this will allow the plugin to get all those fancy features that were just described.

We will definitely want a plugin that auto-moves classes to the correct package. As for already named classes, it should be fairly simple and probably not need a full plugin, but definitely a bunch of code to read the files and move them to the correct directories. Since it's such a one-off thing I don't think it'll be that useful to keep around.

update since I'm incredibly wise. instead of building new enigma APIs, we simply automatically correct packages on everything using the power of dynamic name proposal.


Synthesis failed for java-translate

Hello! Autosynth couldn't regenerate java-translate. :broken_heart:
Here's the output from running synth.py:
 [DEBUG] > Running: git log -1 --pretty=%at 254325ed3c9caa6bf6d9c8b11ce2674544ba6e7c
2021-02-08 14:33:00,340 autosynth [DEBUG] > Running: git log -1 --pretty=%at 35ce99fec72979f6f9b2a5adae835a49648a3231
2021-02-08 14:33:00,344 autosynth [DEBUG] > Running: git log -1 --pretty=%at 5e3dacee19405529b841b53797df799c2383536c
2021-02-08 14:33:00,347 autosynth [DEBUG] > Running: git log -1 --pretty=%at 78e0057d81c6969507bf1195b5aad8ac3e7feafd
2021-02-08 14:33:00,350 autosynth [DEBUG] > Running: git log -1 --pretty=%at c1e59709c1d28795fe9b70eb479579556056bfad
2021-02-08 14:33:00,354 autosynth [DEBUG] > Running: git log -1 --pretty=%at 8d8c008e56f1af31d57f75561e0f1848ffb29eeb
2021-02-08 14:33:00,357 autosynth [DEBUG] > Running: git log -1 --pretty=%at 692715c0f23a7bb3bfbbaa300f7620ddfa8c47e5
2021-02-08 14:33:00,361 autosynth [DEBUG] > Running: git log -1 --pretty=%at 33366574ffb9e11737b3547eb6f020ecae0536e8
2021-02-08 14:33:00,365 autosynth [DEBUG] > Running: git log -1 --pretty=%at 4679e7e415221f03ff2a71e3ffad75b9ec41d87e
2021-02-08 14:33:00,370 autosynth [DEBUG] > Running: git checkout 8a40a6ec80b3a23d10f9fbac620e7e3454c934af
Note: checking out '8a40a6ec80b3a23d10f9fbac620e7e3454c934af'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 8a40a6e deps: update dependency com.google.cloud:google-cloud-core to v1.94.1 (#409)
2021-02-08 14:33:00,392 autosynth [DEBUG] > Running: git checkout 8d8c008e56f1af31d57f75561e0f1848ffb29eeb
Note: checking out '8d8c008e56f1af31d57f75561e0f1848ffb29eeb'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 8d8c008e feat: generate sample code in the Java microgenerator
2021-02-08 14:33:00,409 autosynth [DEBUG] > Running: git checkout 4679e7e415221f03ff2a71e3ffad75b9ec41d87e
Note: checking out '4679e7e415221f03ff2a71e3ffad75b9ec41d87e'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 4679e7e docs: update python contributing guide (#943)
2021-02-08 14:33:00,415 autosynth [DEBUG] > Running: git branch -f autosynth-97
2021-02-08 14:33:00,419 autosynth [DEBUG] > Running: git checkout autosynth-97
Switched to branch 'autosynth-97'
2021-02-08 14:33:00,425 autosynth [INFO] > Running synthtool
2021-02-08 14:33:00,425 autosynth [INFO] > ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2021-02-08 14:33:00,425 autosynth [DEBUG] > log_file_path: /tmpfs/src/logs/java-translate/97/sponge_log.log
2021-02-08 14:33:00,427 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --
2021-02-08 14:33:00,658 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/java-translate/synth.py.
On branch autosynth-97
nothing to commit, working tree clean
2021-02-08 14:33:00,806 synthtool [DEBUG] > Ensuring dependencies.
2021-02-08 14:33:00,818 synthtool [DEBUG] > Cloning googleapis.
2021-02-08 14:33:00,819 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis
2021-02-08 14:33:00,823 synthtool [DEBUG] > Which version of bazel will I run?
bazel 3.0.0
2021-02-08 14:33:00,828 synthtool [DEBUG] > Generating code for: //google/cloud/translate/v3beta1:google-cloud-translation-v3beta1-java.
2021-02-08 14:33:01,029 synthtool [ERROR] > Failed executing bazel --max_idle_secs=240 build //google/cloud/translate/v3beta1:google-cloud-translation-v3beta1-java:

Loading:
Loading: 0 packages loaded
INFO: Call stack for the definition of repository 'com_github_grpc_grpc' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:113:1
WARNING: Download from https://github.com/grpc/grpc/archive/v1.33.1.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 502 Bad Gateway
ERROR: An error occurred during the fetch of repository 'com_github_grpc_grpc':
   java.io.IOException: Error downloading [https://github.com/grpc/grpc/archive/v1.33.1.zip] to /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/com_github_grpc_grpc/v1.33.1.zip: GET returned 502 Bad Gateway
ERROR: no such package '@com_github_grpc_grpc//bazel': java.io.IOException: Error downloading [https://github.com/grpc/grpc/archive/v1.33.1.zip] to /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/com_github_grpc_grpc/v1.33.1.zip: GET returned 502 Bad Gateway
INFO: Elapsed time: 0.185s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)

Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/java-translate/synth.py", line 30, in <module>
    destination_name='translate',
  File "/tmpfs/src/github/synthtool/synthtool/languages/java.py", line 306, in bazel_library
    service=service, version=version, diregapic=diregapic, **kwargs
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 68, in java_library
    service, version, "java", tar_strip_components=0, **kwargs
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 197, in _generate_code
    shell.run(bazel_run_args)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', '--max_idle_secs=240', 'build', '//google/cloud/translate/v3beta1:google-cloud-translation-v3beta1-java']' returned non-zero exit status 1.
2021-02-08 14:33:01,095 autosynth [ERROR] > Synthesis failed
2021-02-08 14:33:01,096 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 8a40a6e deps: update dependency com.google.cloud:google-cloud-core to v1.94.1 (#409)
2021-02-08 14:33:01,102 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2021-02-08 14:33:01,107 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 354, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 189, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 334, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 65, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth_toolbox.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Autosynth passed, closing! :green_heart:


Upload VCF Variant Set Fails

I took a SS of the error, but all the logs in /var/log/supervisor don’t show it.
Attached is the VCF file and screenshot of the error.
tenaillon_ancestor.vcf.txt

I was trying to upload a 6.9kb deletion I found in the Tenaillon ancestor strain to make a reference genome that is closer to the samples (and also matches the loci positions used in the paper, which are relative to the ancestor).

The error was caused because the input is bad. In the GT:DP entries for each sample you'll notice this entry which merges two entries: ./.:01/1:27.


Auto discover routes created as LambdaEndpointRouteBuilder

We added support for this for Spring Boot
https://issues.apache.org/jira/browse/CAMEL-18166
See this example
https://github.com/apache/camel-spring-boot-examples/blob/main/endpointdsl/src/main/java/sample/camel/MyCamelApplication.java
I think camel-quarkus need a build step that can detect routes created in this style ^^^^ and automatic register them. See how we did this with spring-boot in the commit below
https://github.com/apache/camel-spring-boot/commit/c8f7d977cd11707c1dbcf4e7cf8feefb7a1bddad

This can maybe be done in the endpointdsl extension


search: Not seeing "SQL tuning with EXPLAIN" when I search for "performance"

If I search for 'performance' I do not see the (new?) SQL tuning with EXPLAIN in the first page of results.
(Note: today is Wednesday, November 13, 2019)

cc @lhirata for appropriate labelling etc.

Should we retitle this to SQL Performance Tuning with EXPLAIN, use "performance" a few times on the page, and include it as a meta keyword?

Should we retitle this to SQL Performance Tuning with EXPLAIN, use "performance" a few times on the page, and include it as a meta keyword?

IDK - I assume you and Lauren know more than I do about whether those actions would help Algolia out (presumably yes?).  I defer to your knowledge. FWIW the term "performance" appears in the text of the page a few times.

Yeah, you're right it's there, but those few instances aren't enough given the other results pretty much all have "Performance" in their titles. We'll definitely be able to tweak this once we upgrade algolia. For now, we should make other general changes to influence this, also on Google.

Closing in favor of https://github.com/cockroachdb/docs/issues/5852, which is broader.


[Twig][Live] Require Symfony 5.4+

Q
A

Bug fix?
no

New feature?
no

Tickets
n/a

License
MIT

Both @weaverryan and I agree that bumping the min requirement of these library's would be beneficial. The primary reason being, the usability pre-5.3 is sub-par. Component's cannot be autoconfigured with the AsTwigComponent/AsLiveComponent attributes - they must be tagged manually. Requiring 5.4 makes documentation simpler, cleans up the test suite, and fixes some other compatibility issues. We figured, as we require PHP 8.0 already, requiring Symfony 5.4+ would be palatable.

Pinging some community members who are clearly using these library's: @Lustmored, @seb-jean, @norkunas, @allejo, @mdoutreluingne. Any thoughts/objections?

None from me, using 6.0 for the project

I think its a good idea. My projects are all >=5.4 , so lets go :-)

Same here - 6.0 and PHP8.1 only. In fact I was just thinking about that when making changes in test Kernel :smiley:

Keeping it up to date with last active Symphony's LTS version is perfectly fine for me.

Agreed - 5.4 as the minimum is fine by me and I also think it's sensible to make this change sooner rather than later.

It's totally understandable and acceptable to demand Symfony 5.4+, as it's an LTS release. Anyway, active projects have every interest in moving to Symfony 5.4 or more for security reasons and more... So let's go 👍

No concerns here! Using PHP8.1 & Symfony 5.4 here 🥳


Ominpay is big and complex - could it be replaced with a lighter library?

This works fine with Omnipay, but it's a big library - surely there are lighter Stripe libraries out there?

Hi Ryan,
In #130 I said that maybe Payum would be a better alternative to Omnipay.
To my total shame I have to admit that after having tried for several days, I was absolutely unable to make it work.
I tested with Stripe and nothing... An order is recorded but no money is paid. I have no idea why.
Anyway. Concerning a lighter Stripe library, there are several libraries you can find on GitHub but they are more like one-time things that are not necessarily updated. Stripe, on the other hand, updates often.
Having said so, Stripe API is really easy to follow and use so maybe there is no need for another library.
There are really only 3 steps to the process:
1- set the API key
2- After javascript took care of card data and returned a token, charge the client using that token
3- check for success or failure and act accordingly.
Adding another library just for that might not be necessary?

Yes, I reckon this just need to use the main Stripe library.
I released the Stripe add-on for eCommerce for 5.6.x, and that's all it used - wasn't too hard!

Hey Ryan,
I meant to branch and then send a pull request and instead, I modified it directly. Sorry for that.
But it works...
Anyway, Omnipay is gone and it's using Stripe PHP library directly now. I tested with stripe.js (custom form) and with their own form and it's all good.
Also, I show a descriptive error message when it's a card problem but for everything else, I show a generic error (which you should probably make nicer) and log the details for admins to check.
Oh and you probably should update the language file as I'm not sure how to do that

Fantastic work, I'll review and push up a release file when I have the chance!

Just a quick question on this update. My understanding is that Omnipay is a general purpose payment layer between Community Store and the payment processor that is desired. It abstracts the vendor specific payment APIs into a common payment API.  What are the implications of no longer having Omnipay as part of Community Store? How are the various payment gateway APIs wired into Community Store?
I'm not questioning the decision to do this as I understand that Omnipay is not necessarily simple and falling behind. Good work mnakalay.

@dbuonomo Omnipay is not actually part of Community store. Out of 2 payment add-ons (Paypal and Stripe) only Stripe was using it and the Stripe add-on was providing Omnipay.
Having removed Omnipay from the Stripe add-ons doesn't mean others can't develop their own payment solutions using Omnipay. there's nothing in the code that would prevent it

@Mesuva sorry I don't help more.

Just gave this a test, all seemed to work well. I've pushed up a release


migrate miette!/Report errors to properly typed ones

When I started cargo-dist I wanted to mess around with using miette!/Report more aggressively for "totally untyped errors" but it's hitting a scaling limit and really fighting against what miette wants you to do. We should abandon this approach in favour of the more fully typed approach that miette hungers for.
Examples:

axoasset
oranda

This also would entail reckoning with my totally unprincipled use of eprintln! in some places and warn! in others. We should be using eprintln everywhere. Here's an example of printing a miette-based warning that just needs warn! changed to eprintln!:
https://github.com/axodotdev/cargo-dist/blob/4daf3da315032d73c2dcc263412d55c27cc1525d/cargo-dist/src/tasks.rs#L2288-L2293

done


Remove aws provider config from module

Hi!
I came across this module and while I was trying it out I found a problem with our current Terraform configuration because the aws provider is specified inside the module, so it prevents from configuring the provider in some other way in the parent module (like assuming a role).
I believe it would be a best practice to delegate the provider configuration to the parent module that calls this module and removing also the region and default_tags variables, let me know what you think!
Example usage that wasn't possible before:
provider "aws" {
  region = "eu-west-1"

  assume_role {
    role_arn = "arn:aws:iam::123456789:role/some-role"
  }

  default_tags {
    tags = {
	  project = "terraform-aws-organization-and-sso"
      management = "terraform"
    }
  }
}

module "aws_organizations_and_sso" {
	source  = "chris-qa-org/organzation-and-sso/aws"

	.
	.
}

Hi @Roberdvs,
Yep, you're completely right - I shouldn't have put the provider config in there 🤦♂ ...
I tested it out, and fortunately it has no bad side effects when removing the provider block which it claims in the docs (https://www.terraform.io/language/modules/develop/providers):

As a consequence, you must ensure that all resources that belong to a particular provider configuration are destroyed before you can remove that provider configuration's block from your configuration. If Terraform finds a resource instance tracked in the state whose provider configuration block is no longer available then it will return an error during planning, prompting you to reintroduce the provider configuration.

I'll run the tests, merge, and version bump
Thanks for noticing and opening the PR 👍

Thank you for accepting the PR!


Theme "flickering" on reload if dark reader is installed (even if it's turned off)

from Quizfreely issue 1: (see https://github.com/ehanahamed/quizfreely/issues/1)

Every page reload and every time someone navigates to a different page, the whole page flashes black before the correct theme colors are applied. This "flickering" happens really quickly, it flickers so fast that it's almost not noticeable in dark theme, but it is super annoying with light theme.

This only happens if dark reader is installed (even if it's turned off 💀)
from Quizfreely issue 1:

This "flickering" only happens if dark reader is installed AND if prefers-color-scheme is dark (browser/device dark mode setting). Even if dark reader is turned off, the flickering still happens as long as it's installed. Setting browser/device colorscheme to light makes it not flicker even with dark reader installed.

this from darkreader's contributor docs disables darkreader, might fix it
<head>
    <meta name="darkreader-lock">
</head>

idk ngl


Let production build environment assertion allow using Java 8 to run the build

use Java 8 to run the build
and Java 7 for compilation
and Java 8 for running tests
Context
As part of dogfooding the Gradle Kotlin DSL, we need to run the gradle/gradle build with Java >=8. This change is a follow up for #3735.

Can we merge this or will the distribution build now fail?

Can we merge this or will the distribution build now fail?

Right, we need to get https://github.com/gradle/gradle/pull/3749 and https://github.com/gradle/gradle/pull/3750 in first.

I guess https://github.com/gradle/gradle/pull/3750 doesn't matter so much.
We basically need to merge this PR together with https://github.com/gradle/gradle/pull/3749, if not the distribution build will fail.


Make DelegatingPositionOutputStream a concrete class

I fail to understand why this is an abstract class. In my example I want to write the Parquet file to a java.io.FileOutputStream, hence have to extend the DelegatingPositionOutputStream and store the pos information, increase it in all write(..) methods and return its value in getPos().
Doable of course, but useful? Previously yes but now with the OutputFile changes to decouple it from Hadoop more, I believe no.
related to: https://issues.apache.org/jira/browse/PARQUET-1142
Reporter: Werner Daehn / @wernerdaehn
Note: This issue was originally created as PARQUET-1184. Please see the migration documentation for further details.

Ryan Blue / @rdblue:
The reason why this is an abstract class is so that you can use it to wrap implementations that provide a position, like Hadoop's FsOutputStream. It would not be correct to assume that the position is at the current number of bytes written to the underlying stream. An implementation could wrap RandomAccessFile and expose its seek method, which would invalidate the delegating stream's position.
The delegating class is present for convenience only. You don't have to use it and can implement your own logic as long as you implement PositionOutputStream.


always pick __resolveReference on extended types if federation is enabled and type has a key directive

First off, thanks for making this package! It's real nice having a schema-first approach in TypeScript so very similar to gqlgen.
This PR attempts to address an issue where combining a federated type + locally extending that type removes __resolveReference from the generated type stubs.
This naively picks this field in addition to other fields being picked. I couldn't think of a great way to determine the exact definition to pick, but since the field is optional I figure there's not much harm.
Please let me know if there's anything you'd like changed and I'll be happy to address. (or feel free to address yourself if so inclined)

Hi @wilhelmeek , thanks for this PR! I'll take a look in a couple of weeks after I'm back from holidays 🙏

/release-snapshot

/release-snapshot

Hi @wilhelmeek , I made some changes directly to run and fix CI. Looks great!
Could you please try the alpha version for this feature please?
yarn add -DE @eddeee888/gcg-typescript-resolver-files@pr224-run400-1

Thanks @eddeee888! I'll give it a crack this week and let you know

@eddeee888 this seems to work as expected, thank you!

This is now released as<EMAIL_ADDRESS>Thank you for driving it forward!


fix: Use correct compiler for Win64 GCC in Makefile

Fix: Use correct compiler for Win64 GCC in Makefile
Description:
This PR fixes the Makefile to use the appropriate compiler for Win64 GCC. The original line used 'x86_64-w64-mingw32-gcc-win32', which is incorrect when using MSYS2. The correct executable name for MSYS2 is 'x86_64-w64-mingw32-gcc', which is now updated in the Makefile.
Changes Made:

Modified the Makefile to use 'x86_64-w64-mingw32-gcc' for building on Win64 GCC.

Testing:
I tested the changes locally on my system to ensure that the Makefile builds the 'run.exe' executable correctly using the correct compiler.
Thanks for considering this PR.

@richinseattle can you take a look?

Maybe setup one job on the GitHub Actions workflow to test this, using the https://github.com/marketplace/actions/setup-msys2

This is fine to pull, ubuntu distributions will indirectly call to the path I hardcoded but can be updated via deb-alternatives style schemes
(base) rjohnson@codex:~/llama2$ ll /usr/bin/x86_64-w64-mingw32-gcc
lrwxrwxrwx 1 root root 40 Jul 26 02:16 /usr/bin/x86_64-w64-mingw32-gcc -> /etc/alternatives/x86_64-w64-mingw32-gcc*
(base) rjohnson@codex:~/llama2$ ll /etc/alternatives/x86_64-w64-mingw32-gcc
lrwxrwxrwx 1 root root 37 Jul 26 02:16 /etc/alternatives/x86_64-w64-mingw32-gcc -> /usr/bin/x86_64-w64-mingw32-gcc-win32*

@kroggen I can take this. Just didn't get which OS msys2 based on? windows or other?

It is for Windows. I just think it does not support arm, as is in the workflow, so maybe a separate job may be needed. Just x86_64 may be sufficient

mingw is a gcc compiler with windows header/library support. It can run on linux or windows.
MSYS2 is a unix-like shell environment built around mingw on windows. MSYS2 is out of scope here.
The anticipation in my mind was for cross-compiles in a github release eventually as an add-on convenience when adding MSVC support. Currently, the MSVC build will be most performant because it supports openmp. I tested -fopenmp on the mingw compiler but it crashes when executed in the Windows environment.
This PR should be merged and the ARM topic should be opened seperately (I don't know anything about that)

@richinseattle What do you think about add support for mingw, MSYS2 and MSVC?
The goal eith this is to give options to Windows developers choose.
ARM is something to research more here too.
Thanks

I can confirm the correct executable name is x86_64-w64-mingw32-gcc, pushed a PR that implements CI step to verify compilations on msys2/mingw #175
That PR also fixes executable name in Makefile


feat: Add custom browse command support

The changes introduced in this commit allow users to specify a custom command for browsing commits in a browser.

@aaronhallaert this should work now incase you looked at it before.


Object literal may only specify known properties, and 'theme' does not exist in type 'Partial & Omit<AppTheme, "theme">'

When adding a custom theme to App.tsx as documented here: https://backstage.io/docs/getting-started/app-custom-theme i should be able to make use of my custom theme -
const app = createApp({
  apis,
  plugins: [badgesPlugin],
  themes: [{
    id: 'customTheme',
    title: 'Custom',
    variant: 'dark',
    theme: customTheme,
 }],
  components: {
    SignInPage:
      props => {
        return (
          <SignInPage {...props}
           Guest to be used only for development purposes
           providers={[...providers]}
            align="center" />
        );
      },
  },

I am getting the following error once i upgraded to release-2021-12-23
 error TS2322: Type '{ id: string; title: string; variant: "dark"; theme: BackstageTheme; }' is not assignable to type 'Partial<AppTheme> & Omit<AppTheme, "theme">'.
  Object literal may only specify known properties, and 'theme' does not exist in type 'Partial<AppTheme> & Omit<AppTheme, "theme">'.

Hi @regicsolutions , there were a few deprecations in the latest release, one of which impacted the use of the theme property. It appears that I may have missed updating the documentation in this instance.
I've confirmed that it is possible to set a custom theme by defining a theme provider. See the example code below:
import { createTheme, lightTheme } from '@backstage/theme';
import { ThemeProvider } from '@material-ui/core/styles';
import CssBaseline from '@material-ui/core/CssBaseline';

const customTheme = createTheme({
  palette: lightTheme.palette,
  fontFamily: 'Comic Sans MS',
  defaultPageTheme: 'home',
});

const app = createApp({
  apis,
  bindRoutes({ bind }) {
    bind(catalogPlugin.externalRoutes, {
      createComponent: scaffolderPlugin.routes.root,
      viewTechDoc: techdocsPlugin.routes.docRoot,
    });
    bind(apiDocsPlugin.externalRoutes, {
      createComponent: scaffolderPlugin.routes.root,
    });
    bind(scaffolderPlugin.externalRoutes, {
      registerComponent: catalogImportPlugin.routes.importPage,
    });
  },
  themes: [
    {
      id: 'my-theme',
      title: 'My Custom Theme',
      variant: 'light',
      Provider: ({ children }) => (
        <ThemeProvider theme={customTheme}>
          <CssBaseline>{children}</CssBaseline>
        </ThemeProvider>
      ),
    },
  ],
});

@Rugvip, I can update the documentation to instruct users to define a theme provider if you could confirm that this is, in fact, the desired approach. Thanks!

@cmpadden yep, docs are outdated :p

Thanks @cmpadden for the heads up, working now!


Revert "Revert "Improve URL preview to recognize messages with links and text & rework splitting of links when sending a new message""

Reverts BlueBubblesApp/BlueBubbles-Android-App#1401

Force merging this because the other PR will fix the errors