HDFS-15937. Reduce memory used during datanode layout upgrade

When the datanode block layout is upgrade from -56 (256x256) to -57 (32x32), we have found the datanode uses a lot more memory than usual.
For each volume, the blocks are scanned and a list is created holding a series of LinkArgs objects. This object contains a File object for the block source and destination. The file object stores the path as a string, eg:
/data01/dfs/dn/current/BP-586623041-<IP_ADDRESS>-1617017575175/current/finalized/subdir0/subdir0/blk_1073741825_1001.meta
/data01/dfs/dn/current/BP-586623041-<IP_ADDRESS>-1617017575175/current/finalized/subdir0/subdir0/blk_1073741825
This is string is repeated for every block and meta file on the DN, and much of the string is the same each time, leading to a large amount of memory.
If we change the linkArgs to store:

Src Path without the block, eg /data01/dfs/dn/previous.tmp/BP-586623041-<IP_ADDRESS>-1617017575175/current/finalized/subdir0/subdir0
Dest Path without the block eg /data01/dfs/dn/current/BP-586623041-<IP_ADDRESS>-1617017575175/current/finalized/subdir0/subdir10
Block / Meta file name, eg blk_12345678_1001 or blk_12345678_1001.meta

Then ensure were reuse the same file object for repeated src and dest paths, we can save most of the memory without reworking the logic of the code.
The current logic works along the source paths recursively, so you can easily re-use the src path object.
For the destination path, there are only 32x32 (1024) distinct paths, so we can simply cache them in a hashMap and lookup the re-useable object each time.
I tested locally by generating 100k block files and attempting the layout upgrade. A heap dump showed the 100k blocks using about 140MB of heap. That is close to 1.5GB per 1M blocks.
After the change outlined above the same 100K blocks used about 20MB of heap, so 200MB per million blocks.
A general DN sizing recommendation is 1GB of heap per 1M blocks, so the upgrade should be able to happen within the pre-upgrade heap.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 49s

trunk passed

+1 :green_heart:
compile
1m 20s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 13s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  3s

trunk passed

+1 :green_heart:
mvnsite
1m 22s

trunk passed

+1 :green_heart:
javadoc
0m 56s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 28s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
3m  8s

trunk passed

+1 :green_heart:
shadedclient
16m  2s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  7s

the patch passed

+1 :green_heart:
compile
1m 10s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 10s

the patch passed

+1 :green_heart:
compile
1m  8s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
1m  8s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 52s

hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 30 unchanged - 3 fixed = 30 total (was 33)

+1 :green_heart:
mvnsite
1m 13s

the patch passed

+1 :green_heart:
javadoc
0m 45s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 16s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
3m  4s

the patch passed

+1 :green_heart:
shadedclient
15m 45s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
227m 53s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 45s

The patch does not generate ASF License warnings.

312m 54s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.TestRollingUpgrade

hadoop.hdfs.server.datanode.TestBlockRecovery

hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2838/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2838

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 0359a8b54474 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 6c30ed2108877a3e628a4f52b4e4578f769da7f7

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2838/2/testReport/

Max. process+thread count
2963 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2838/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 32s

trunk passed

+1 :green_heart:
compile
1m 19s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 14s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  2s

trunk passed

+1 :green_heart:
mvnsite
1m 21s

trunk passed

+1 :green_heart:
javadoc
0m 54s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 26s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
3m  2s

trunk passed

+1 :green_heart:
shadedclient
16m  0s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 12s

the patch passed

+1 :green_heart:
compile
1m 14s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 14s

the patch passed

+1 :green_heart:
compile
1m  3s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
1m  3s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 54s

hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 30 unchanged - 3 fixed = 30 total (was 33)

+1 :green_heart:
mvnsite
1m 14s

the patch passed

+1 :green_heart:
javadoc
0m 46s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 21s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
3m  5s

the patch passed

+1 :green_heart:
shadedclient
15m 49s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
230m  1s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 44s

The patch does not generate ASF License warnings.

315m  2s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestBlockScanner

hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized

hadoop.hdfs.server.datanode.TestBlockRecovery

hadoop.hdfs.server.balancer.TestBalancer

hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots

hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2838/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2838

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux e7d77a5eb290 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 559216c798b02b2b2803be64e978d263368e55ef

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2838/3/testReport/

Max. process+thread count
3222 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2838/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Thanks @Hexiaoqiao ! @jojochuang have you got any more comments or do you want to take another look before I commit this?


EventHandlers in EJBs/JAXB EndPoints are only registered once per class/pool

A JEE container typically creates several instances of an EJB or JAXB endpoint in a pool, to handle the incoming requests. The CDI code currently forces instantiation of one Bean and registers that, but the other instances will not receive any events.

EJB SLSB (Pooled) corresponds to a RequestScope of CDI. What is your proposal? Currently, the CDI scope is explicitly set to singleton (ApplicationScoped.class is used) for any beans, which are constructed.

Bert is referring to a side-effect of our early instantiation of beans I believe. The end result is that we wind up with our own copy of an application scoped bean while once the CDI container actually starts up, it does not see this and creates another instance of the same application scoped bean (in fact this might be Weld specific behavior only). The solution I believe is to try to see if we can further defer the instantiation of beans (event listeners specifically). That may or may not be possible and we may need to live with this being an idiosyncrasy of Axon CDI integration.
One solution I am reluctant to pursue right now is abstract all bean access behind a byte code generated proxy so that references are only obtained once the application actually starts up. A PR that does this would certainly be welcome. I am happy to provide further guidance to anyone interested.

Please be careful: @ApplicationScoped does not guarantee that all beans are singletons. Only @Singleton does this. All @XxxScoped annotations are meant to indicate lifetime/statefulness, not number of instances. A @RequestScoped bean has no state beyond the lifetime of the corresponding (Web) request. @ApplicationScoped signifies the bean can be depended on to maintain state during the lifetime of the entire application.
If you look at the logging of a run of a simple application with a REST endpoint in e.g. Payara micro, you will see there are 5 threads handling requests. Give objects a unique id, and you will see 5 unique IDs in the logging. Definitely NOT singletons!
This is NOT a result, nor an unintended side-effect, of any code in the CDI module. This is how JEE works. The Application Server is free to use a pool of beans to service requests, and Payara apparently will use 5 instances by default.
As long as the CDI module forces the pre-instantiation of a single bean, and then only registers that one as an EventHandler, any subsequently instantiated beans will be ignored. If you try to prevent this behaviour by putting a @Singleton on the endpoint class, you will get an error, because the Application Server wants to be able to create a pool of handlers.
BTW there is no need to go for byte-coding or runtime generation of classes. You can hook into the CDI ProcessInjectionTarget event and replace the InjectionTarget with an anonymous class to add registration of the Bean on PostConstruct, and deregister again at PreDestroy. This does not affect the actual class that has the @Inject on a field, only the BeanManager. However, is does mean we need an event processor that can add/remove handlers after start() has been called.

OK. I'll definitely make sure to discuss this with the CDI experts and see what they think. Thanks.

Just had a small discussion with Allard on this. I'm back with my feet on the ground.
We agreed that it is fundamentally silly to put an EventHandler in a SessionScoped or even RequestScoped Bean. When an event comes in, what is the relevant scope to process it in? Which of the Beans do we select? Events only make sense in ApplicationScoped beans, unless you have recorded the original request or session where the command was sent from, and correlate the two, AND the scope is still open. The costs are however high, possibly unreasonably so. (diplomatically phrased) The obvious exception is an EventHandler in an Aggregate, but that is already taken care of.
EventHandlers put in a REST endpoint, which is the admittedly fundamental issue in my example, expose a problem, but should not be the example to validate the extension against. To get it working correctly, I moved the EventHandler to a separate Singleton Bean, and made the endpoint Inject that. My apologies that I kept yakking about the errors the code caused, while the example wasn't actually a valid use-case. The underlying discussion about the difference between bean scope and instance quantity remains valid however.
Perhaps the first iteration should simply generate a logged WARNING that an EventHandler is defined in a class which is not a Singleton. If you really want to do that, a solution exists, but it is expensive and probably not what you want anyway. The part of the problem that we SHOULD fix however, is when to instantiate beans. The extension should do that in the AfterDeploymentValidation CDI Event handler, not in the AfterBeanDiscovery handler where currently practically all work is done.


Roll Flutter Engine from c2ef01f6f1ab to ceec2504bfab (4 revisions)

https://github.com/flutter/engine/compare/c2ef01f6f1ab...ceec2504bfab
2024-05-20<EMAIL_ADDRESS>Roll Skia from 0b8d8ce44d1f to 977a43773f7c (1 revision) (flutter/engine#52938)
2024-05-20<EMAIL_ADDRESS>[Impeller] write glyphs to malloc buffer. (flutter/engine#52937)
2024-05-20<EMAIL_ADDRESS>Roll Dart SDK from 08f4324c988b to 01a77883e9e9 (1 revision) (flutter/engine#52936)
2024-05-20<EMAIL_ADDRESS>[Impeller] Fix use-after-move in SwapchainVK. (flutter/engine#52933)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC bdero@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Version bump cibuildwheel

update cibuildwheel to newer version. One of the benefits is 3.10 support and musslinux wheels.

Superseded by #1087. Thanks for bringing this to my attention.


Add a few widgets

add Image/Icon widgets
add mainAxis, crossAxis to Row/Column
Card is ongoing

Closing since this was merged in #35


OKTA SLO Integration

Hi
Im working on SAML integration with AZURE and SAML
Azure is working fine but i have problem with OKTA SLO
Added:
config.saml_configure do |settings|
settings.security[:logout_requests_signed]  = true
settings.certificate = "..."
settings.private_key = "..."
end
and certificate used in settings is added as Signature Certificate on OKTA  Edit SAML Integration form
but when im trying to log out i get: "User single sign out from app failure: Invalid Signature" Error. on LogoutRequest
My question: Has anyone implemented OCTA SLO? What is wrong?
Im uploading good certificate?
should i fill something in SP Issuer input on Edit SAML Integration form on OKTA page?

Invalid Signature means that the entity was not able to validate the signature related to the SAML Message.
Verify that the entity that verifies the signature has registered the same public cert of the entity that generated the SAMLResponse.

With Okta as an IDP, for SLO to work, then in the Okta App for your SP, you
need to upload the SP's signing certificate into Okta's "Signature
Certificate" section near the other SLO fields in the "Advanced settings"
section. Note that Okta has a different configuration for the token
encryption certificate, which should be your SP's encryption certificate
(in most cases, the SP uses the same certificate for encryption and signing
but technically they can be different). Also, I have observed that Okta
supports SP-initiated SLO but not IDP-initiated SLO.
On Wed, Jun 2, 2021 at 4:30 PM Sixto Martin @.***>
wrote:

Invalid Signature means that the entity was not able to validate the
signature related to the SAML Message.
Verify that the entity that verifies the signature has registered the same
public cert of the entity that generated the SAMLResponse.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/onelogin/ruby-saml/issues/592#issuecomment-853362201,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACXOM7TSMXHGMXCIKQK7BP3TQ2IF5ANCNFSM45HSIEFQ
.

--
Darryl Hopkins
Director, Business Analysis
2300 N Street NW, Suite 500
Washington, DC 20037
*C *571.296.3359
@.***
everfi.com http://www.everfi.com/

Okta supports both SP-initiated SLO and IDP-initiated SLO, but does not support HTTP-Redirect binding and only HTTP-POST for the generated LogoutRequest and LogoutResponses.
SLO at Okta requires LogoutRequest and LogoutResponses to be signed, so you need to register the SP cert at Okta that will allow the Signature validation.

I was able to get OKTA SLO working (SP-initiated) by adding:
settings.idp_slo_service_binding = OneLogin::RubySaml::Utils::BINDINGS[:redirect]

to the SAML settings before creating the OneLogin::RubySaml::Logoutrequest
The metadata returned by my OKTA app includes both bindings:
<md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="OKTA_APP_URL/slo/saml"/>
<md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="OKTA_APP_URL/slo/saml"/>

However, this library picks the first one. So when I actually tried to redirect to the generated SLO URL, OKTA was looking for the certificate in the query params. However, the signature was actually embedded in the SAML document.

@pitbulk please close this issue.


KSampler Config is no longer working for sampler_name since Oct 05

Since ComfyUI update from Oct 5, the KSampler Config Module is no longer able to connect sampler_name or scheduler with any other module. Tested with default KSamplers and a few other like Impact FaceDetailer and EfficientNode.

Yea, looks like there's a lot of churn there in the type.. not sure what to do, they've broken it twice in 6fc7314 and then changed it again in b9b178b.
I think the place they'll end up is way better than what they had, but I'm not sure the current design takes anything into consideration that isn't the specific PrimitiveNode, which doesn't help this node (or any other node with list outputs)

Found a workaround, if you use a reroute (rgthree) and connect it first to the KSampler for instance, and then to KSampler Config, you can use for the session. However, you lose the connection when you load a workflow.

Interesting, maybe I can fix the disconnecting from my end while they figure out what they want to do with list types more broadly.
FWIW, I created an issue here: https://github.com/comfyanonymous/ComfyUI/issues/1674

That would be awesome for now! Thank you!

FaceDetailerPipe:
    - Return type mismatch between linked nodes: scheduler, ['normal', 'karras', 'exponential', 'sgm_uniform', 'simple', 'ddim_uniform'] != ['normal', 'karras', 'exponential', 'sgm_uniform', 'simple', 'ddim_uniform', 'AYS SDXL', 'AYS SD1', 'AYS SVD']

FaceDetailerPipe:
    - Return type mismatch between linked nodes: scheduler, ['normal', 'karras', 'exponential', 'sgm_uniform', 'simple', 'ddim_uniform'] != ['normal', 'karras', 'exponential', 'sgm_uniform', 'simple', 'ddim_uniform', 'AYS SDXL', 'AYS SD1', 'AYS SVD']

Source of the issue: https://github.com/ltdrdata/ComfyUI-Impact-Pack/blob/Main/modules/impact/core.py#L44 Commit: ltdrdata/ComfyUI-Impact-Pack@1b4b830

I've been having the same issue with schedulers, but coming from the efficiency nodes.
Commit: https://github.com/jags111/efficiency-nodes-comfyui/commit/0ee8860d6b65cd3b297d5fb23378bd970617fa13
Not proficient at all with python, and have no idea how this could be made compatible with efficiency and impact nodes at the same time, specially since it seems to be a comfyui problem, as said previously.
But, since I mostly prefer the efficiency KSampler, I fixed on my end by adding this to ksampler_config.py, and then changing the scheduler types on it to point to SCHEDULERS:
try:
    from efficiency_nodes import SCHEDULERS as EFFICIENCY_SCHEDULERS
    SCHEDULERS = EFFICIENCY_SCHEDULERS
except ImportError:
    # Handle the case where the module or variable doesn't exist
    SCHEDULERS = comfy.samplers.KSampler.SCHEDULERS

Of course, trying to use any of the non default schedulers on regular KSampler, will throw an error. Possibly importing from impact pack can fix it for both samplers, since it has more options on impact, but the same error would show up, if choosing a non supported scheduler for the sampler to which it's connected.

Unfortunately, the ComfyUI-Inspire-Pack is also affected. The scheduler cannot be connected directly there either.
The workaround with a reroute node looked promising, but something seems to get lost on the way to the image preview and no new image arrives there.

At least I stumbled upon a workaround for the Inspire Pack and Impact Pack found by doomndoom on Reddit. You can connect the scheduler via the Impact Scheduler Adapter (from the Impact Pack).

However, this workaround does not work for the Efficiency Nodes nor for ComfyUI_tinyterraNodes.


shared_preferences: Failed to handle method call

From the trace it looks like the new Threadpool implementation is the likely cause.
Steps to reproduce -

Flutter run
Enter url to login
In the main screen click on the switch mode button

Log
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): Failed to handle method call
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): java.util.concurrent.RejectedExecutionException: Task io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl$1@c5cdb16 rejected from java.util.concurrent.ThreadPoolExecutor@bf1c89f[Running, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 1]
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2086)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1394)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl.commitAsync(MethodCallHandlerImpl.java:137)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl.onMethodCall(MethodCallHandlerImpl.java:66)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:233)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at io.flutter.embedding.engine.dart.DartMessenger.handleMessageFromDart(DartMessenger.java:85)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at io.flutter.embedding.engine.FlutterJNI.handlePlatformMessage(FlutterJNI.java:818)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at android.os.MessageQueue.nativePollOnce(Native Method)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at android.os.MessageQueue.next(MessageQueue.java:336)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at android.os.Looper.loop(Looper.java:174)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at android.app.ActivityThread.main(ActivityThread.java:7710)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at java.lang.reflect.Method.invoke(Native Method)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:516)
E/MethodChannel#plugins.flutter.io/shared_preferences(31423): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950)
E/flutter (31423): [ERROR:flutter/lib/ui/ui_dart_state.cc(186)] Unhandled Exception: PlatformException(error, Task io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl$1@2ecc23e rejected from java.util.concurrent.ThreadPoolExecutor@bf1c89f[Running, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1], null, java.util.concurrent.RejectedExecutionException: Task io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl$1@2ecc23e rejected from java.util.concurrent.ThreadPoolExecutor@bf1c89f[Running, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1]
E/flutter (31423): 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2086)
E/flutter (31423): 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
E/flutter (31423): 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1394)
E/flutter (31423): 	at io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl.commitAsync(MethodCallHandlerImpl.java:137)
E/flutter (31423): 	at io.flutter.plugins.sharedpreferences.MethodCallHandlerImpl.onMethodCall(MethodCallHandlerImpl.java:66)
E/flutter (31423): 	at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:233)
E/flutter (31423): 	at io.flutter.embedding.engine.dart.DartMessenger.handleMessageFromDart(DartMessenger.java:85)
E/flutter (31423): 	at io.flutter.embedding.engine.FlutterJNI.handlePlatformMessage(FlutterJNI.java:818)
E/flutter (31423): 	at android.os.MessageQueue.nativePollOnce(Native Method)
E/flutter (31423): 	at android.os.MessageQueue.next(MessageQueue.java:336)
E/flutter (31423): 	at android.os.Looper.loop(Looper.java:174)
E/flutter (31423): 	at android.app.ActivityThread.main(ActivityThread.java:7710)
E/flutter (31423): 	at java.lang.reflect.Method.invoke(Native Method)
E/flutter (31423): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:516)
E/flutter (31423): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950)
E/flutter (31423): )
E/flutter (31423): #0      StandardMethodCodec.decodeEnvelope
package:flutter/…/services/message_codecs.dart:581
E/flutter (31423): #1      MethodChannel._invokeMethod
package:flutter/…/services/platform_channel.dart:158
E/flutter (31423): <asynchronous suspension>
E/flutter (31423): #2      MethodChannelSharedPreferencesStore.setValue
package:shared_preferences_platform_interface/method_channel_shared_preferences.dart:30
E/flutter (31423): <asynchronous suspension>

I solved this issue by upgrading the shared_preferences plugin from ^2.0.3 to ^2.0.4
@harchani-ritik I propose to apply this change.

I'm facing the same issue and using latest package :(

@pranav1230 could you elaborate and also send the error log

[VERBOSE-2:ui_dart_state.cc(186)] Unhandled Exception: 'dart:async/future_impl.dart': Failed assertion: line 543 pos 12: '': is not true.
#0      _AssertionError._doThrowNew (dart:core-patch/errors_patch.dart:46:39)
#1      _AssertionError._throwNew (dart:core-patch/errors_patch.dart:36:5)
#2      Future._completeError (dart:async/future_impl.dart:543:12)
#3      _completeOnAsyncError (dart:async-patch/async_patch.dart:264:13)
#4      MethodChannelSharedPreferencesStore.setValue (package:shared_preferences_platform_interface/method_channel_shared_preferences.dart)

I'm trying to increment Quotation number(Using shared prefs to store the data.) after generating the Pdf on button click.

I'm using await and .then() together in _incrementCounter method.

I'm facing the same issue and using latest package :(

This issue was resolved using ^2.0.4 version of the plugin.

@pranav1230 I am unable to recreate this issue in the current build. As suggested by @rj-since-2000, using the latest version of the shared preferences package resolved the issue

I'm trying to increment Quotation number(Using shared prefs to store the data.) after generating the Pdf on button click.

Are you referring to functionality in this app. Because I don't think there is a "pdf generation" function in this app

@rj-since-2000 I am closing this issue since this already seems to be resolved.

@rj-since-2000 I am closing this issue since this already seems to be resolved.


Manual Review Maana Token

MAN :  Correcting error in syntax.

Please note: This repository is being rebuilt to accept the new volume of token additions and modifications. PR merges will be delayed.

I agree to not ping anybody on Discord/Twitter/email about this pull request. Instead I will inquire by posting a new comment in the pull request if needed.

PRs are reviewed in bulk and and can take up to two weeks to be merged.
This repository is managed using an auto merge action. Please ensure your PR has no deleted lines, and it will be merged.
Please provide the following information for your token.
Please include change to the src/tokens/solana.tokenlist.json file in the PR.
DON'T modify any other token on the list.
At minimum each entry should have

Token Address: 6DTw9cra3RdWEJEnu7mCVE7rjxnXdV8qDa4StzuUcXcq
Token Name: Maana
Token Symbol: MAN
Logo: (logo should be uploaded under assets/mainnet//*.<png/svg>)
Link to the official homepage of token:
Coingecko ID if available (https://www.coingecko.com/api/documentations/v3#/coins/get_coins__id_):

Auto merge requirements
Your pull request will be automatically merged if the following conditions are met:

Your pull request only adds new tokens to the list. Any modification to existing
tokens will require manual review to prevent unwanted modifications.

Your pull request does not touch unrelated code. In particular, reformatting changes to unrelated
code will cause the auto merge to reject your PR.

Any asset files added correspond to the token address you are adding. Asset files
must be PNG, JPG or SVG files.

Your change is valid JSON and conforms to the schema. If your change failed validation,
read the error message carefully and update your PR accordingly.

No other tokens shares the same name, symbol or address.

For example, this change would be rejected due to unrelated changes:

The bot runs every 60 minutes and bulk-merges all open pull requests to prevent conflicts.
This means that you need to wait up to 60 minutes for your pull request to be merged or reprocessed.

this is incorrect, in json the last element cannot have a comma (super annoying json property)


Added status code to callback when the server closes the connection

The callback will now include the status code received by the server when the connection was closed.
local function websocket_callback(self, conn, data)
	if data.event == websocket.EVENT_DISCONNECTED then
		print(data.code)
		print(data.message)
	end
end

Fixes #44

The PR unifies emscripten and wslay implementations when it comes to how the connection is closed.


Add file parameter to output XML from input test url

Currently, the json input to this tool includes a url with the filepath of a ran test, something like this:
{"test":{"id":3,"name":"package test","suiteID":0,"groupIDs":[2],"metadata":{"skip":false,"skipReason":null},"line":4,"column":3,"url":"file:///root/project/flutter/package/test/package_test.dart"},"type":"testStart","time":8218}

However the output XML will only include the classname currently. Something like this:
<testcase classname="flutter.package.test.package" name="package test" time="0.042">
  <failure message="1 failure, see stacktrace for details">Failure:\n\nExpected: false Actual: <true> </failure>
</testcase>

I'd expect the output XML to look something like this instead:
<testcase file="/root/project/flutter/package/test/package_test.dart" classname="flutter.package.test.package" name="package test" time="0.042">
  <failure message="1 failure, see stacktrace for details">Failure:\n\nExpected: false Actual: <true> </failure>
</testcase>

I'm happy to throw up a PR to work on this as well.

Can you point to a JUnit xml specification where the file attribute is defined?


MyCHATAI

Hi,

What is the value of the "temperature" parameter you have set? I'm asking because I found an application (I don't remember the name now) where I can set these parameters when entering the prompt, as mentioned in the OPEN AI API documentation. This is particularly important since the GPT-4 API does not have a "standard" value for this parameter, and it must be set programmatically for any application."?
If you have set the default value yourself, please write down what it is and let me know if I can smoothly change it like in other native available applications.

Unlimited mode is cost-effective, It will only select the most relevant messages as context. - What did you mean by writing that - could you please explain in more detail?

What is the value of the "temperature" parameter you have set now?

ok, Top_p will  be supported as well in the next update.
Maybe in 3-5 workdays.

next update ?

I have one more question: what is the set maximum value for the input and output tokens in your native application, taking into account the GPT-4 API? Have you considered adjusting these values like you've done with the "temperature" parameter, as shown in the image below?


Can anyone help me with disabling back button on certain scenes?

Version
Tell us which versions you are using: ^4.0.0-beta.17
I used to nest Scenes into buckets to stop users going to the previous screen. This is not working
now on latest build. Can anyone please help me out? Below is the code.
Logic is that user should be shown splash screen only once. Later on user cannot go back to splash screen.
<Router>
      <Scene key="root">
          <Scene key="splash" component={SplashScreen} initial  />
          <Scene key="Screen1" component={Screen1} />
          <Scene key="Screen2" component={Screen2} />
      </Scene>
 </Router>

You can use type="replace" in Screen1 scene to stop users going back to splash screen.
Try this for Scene1
<Scene key="Screen1" type="replace" component={Screen1} />

Awesome. Finally something that worked. @zainmustafa i wonder why this prop is nowhere in v4 DOCS.

Yeah, it has to be there in docs. Glad it helped.

Added to docs, closing.

type={ActionConst.RESET} in Scene

Replace did not work for. This is what worked in my case:
`import {BackHandler} from 'react-native';
UNSAFE_componentWillMount() {
BackHandler.addEventListener("hardwareBackPress", this.handleBackButton);
}
handleBackButton = () => {
// Do nothing
return true;
};`


Fix wildcard import statement

import { * as lodash } from 'es-toolkit/compat';

In order to be valid code, we have to remove the brackets.

🤦 Good catch, thank you!

Released in v0.3.1 😃


Incomplete sentence in a page.

Section Adding tools and customizing the container > Bullet point 2

Adding tools and customizing the container
In this walkthrough, you created a basic build agent. The suggested base Dockerfiles are just that: suggestions. You can extend the Dockerfile to include additional tools and their dependencies, or build your own container using this one as a base layer. Just make sure that the following things are left untouched:

The start.sh script is called by the Dockerfile
The start.sh script is the last command that the Dockerfile
Ensure that derivative containers do not remove any of the dependencies stated by the Dockerfile

The sentence seems incomplete

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b31d4f6c-d16d-2ba1-1294-6aa94e328046
Version Independent ID: 88aec599-4410-697d-1002-1c41b1c17bb6
Content: Running a self-hosted agent in Docker - Azure Pipelines
Content Source: docs/pipelines/agents/docker.md
Product: devops
Technology: devops-cicd-agents
GitHub Login: @steved0x
Microsoft Alias: sdanie

@aishik-pyne Thank you for the feedback, please consider becoming a contributor.
The documentation has been updated and is pending review.


cocoapods: define updateScript

Motivation for this change
This means nix-shell maintainers/scripts/update.nix --argstr maintainer lilyball will update cocoapods.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[X] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

This pull request has been mentioned on Nix community. There might be relevant details there:
https://discourse.nixos.org/t/automating-updates-of-ruby-packages/2996/3

I've updated this PR to use an alternative approach that avoids the need for updateScript.

Nice, didn't know about maintainers/scripts/update.nix


must-gather: skip helper pod creation and ceph log collection

skip helper pod creation and ceph log collection for MCG only cluster
Signed-off-by: Mudit Agarwal<EMAIL_ADDRESS>

/hold
tested on MCG only cluster, need to test on the normal one.

/cc @yati1998

/hold cancel
Tested on both type of clusters, helper pod is not being created on the standalone cluster

/cc @umangachapagain

/cherry-pick release-4.10

@yati1998 please review


Create a parallel Document that's actor protected, with async methods for updating content

The idea being ultimately replacing the DispatchQueue protection, and shifting to fully embracing swift's async/await structure. With Swift 6 (fully embracing this safety marker) coming this June (2024), we'll want to make sure at a minimum we pass all Sendable checking.
In addition, the more recent updates to UniFFI (2.6.1) have started to embrace coordinating async calls between Rust and Swift. We're not taking advantage of these yet, but it may prove useful down the road.
Since I think we'll want to maintain backward compatibility for a while, a parallel type to Document would make a lot of sense, and we can slowly shift/deprecate the current version as the experiments vet themselves out in practical (app) use.

I guess by using an internal DispatchQueue and conforming to Sendable (maybe unsafe) your are already ready for Swift 6. Not sure if it is worth rewriting everything using actors

@lightsprint09 You're correct - I've verified Automerge-swift is completely good for Swift6 mode already, but in learning more about it, it seems like it may be a good way to also accomplish it, so I wanted to try it out


rename path from eve/api to eve-api

It still used the old paths, which made the import fail.

I suspect the CI may not clear until #2 is in.

Rebased.


Proposal: version control for BREAD configuration (WIP)

Laravel Version: Any compatible
Voyager Version: 1.x.x
PHP Version: Any compatible

Also mentioned in:

https://github.com/the-control-group/voyager/issues/1030
https://github.com/the-control-group/voyager/issues/1995
https://github.com/the-control-group/voyager/issues/1921
https://github.com/the-control-group/voyager/pull/2014

And probably a few others
Problem:
When creating a BREAD configuration for a table, the settings are stored in the database. This makes it hard for teams to work together on a project without exporting/importing these tables settings or maybe sharing full database back-ups.
Presented solutions in the issues above used migrations and seeds to share this data. While this indeed is a valid solution to share the data, I don't think this is the best solution for people working in teams. Why? As long as you are developing locally or maybe even on the staging/testing environment it's perfectly fine to drop a table or truncate a table and re-create or re-seed these. But when the project is in production we can't drop a table and rerun a migration holding the full table since we could lose the data this table is holding.
Development
I'm developing the solution presented below, you can find it at https://github.com/PendoNL/voyager. As long as I haven't said it's ready for testing here, use it at your own risk and until it's properly tested: don't use it in any of your live projects.
Solution:
I've started working out an idea, roughly what I'm going to try to do is:

(1) Change the BreadAdded and BreadEditted events to receive $request->all() besides the $dataType and $data
(2) Use the BreadChanged event to dispatch a SaveBreadConfigurationToFilesystem listener
(3) The json_encode($event) (I guess I also need serialization here) is then saved to {$event->dataType->slug}.json in a new directory (database/bread/) which is published when installing voyager

The above is working so far. Next steps are to create a new refresh method on the VoyagerDatabaseController which deletes the current data (safely) and then adds the data from within the configuration files. This is all wrapped within Database Transactions to prevent false data when something goes wrong along the process. Also I created a setting enabled_bread_configuration_version_control which can be disabled if there's no need to save the configuration to the file system.
There still are a few things I need to be aware of:

I need a way to compare the filemtime of the newest saved configuration to the updated_at timestamp of the last editted record in data_types. This can be used to determine wether a user needs to refresh the BREAD configurations to stay in sync with his/her team. Also, I have to take timezones into account I guess.

To prevent duplicate configurations being saved when the slug is changed for a BREAD configuration. I need to find a better solution which can keep track of this change and remove the old file from the database/bread directory. Solution: the $dataType->name field holds the database name (and is not changeable). There still is a situation where a BREAD configuration can be duplicate, but I suppose we can check the existance of a table before adding the configuration.

When refreshing the configuration I need to use the delete method of VoyagerDatabaseController (also need to use storeBread and updateBread of that same controller), some of the logic of these methods need to be extracted to a function to be used in the refreshBread method. I found a way to check the existence of a DataType and then add or update accordingly, this way there is no need (yet or for ever?) for deleting a DataType from the database.

In order to prevent updates of the BREAD in production, the setting enabled_bread_configuration_version_control can be used as wel. Being true, this would disable the ability to EAD the BREAD configurations in production.

A new commando voyager:refresh-bread-config can be run from within the console in production in order to have the latest configuration loaded.

A separate, but in similar, problem for version control exists for the Admin menu.

In order to have valid migrations in version control, I propose to disable the database add/edit methods when enabled_bread_configuration_version_control is set to true. This forces the developers to use the native make:migration which can hold changes to tables as well.

I can see that this might add a few steps to development since you can't always use the graphical interface of Voyager. But I don't think there is another way to keep these settings in version control and shareable between teams. Besides, having the enabled_bread_configuration_version_control setting you can still unleash full control using the GUI if set to false.

Just want to provide big support for this.
It's exactly what every project that needs to be shared between developers or has a proper CI setup needs.

Me too, massive support for this.
The lack of version control of project config/settings is why I moved away from Expressionegine, CraftCMS and a few others. This feature is critical in my opinion.

Fair warning with developing this (sorry, not sure how I missed it last month...).  We're in the process of rebuilding the BREAD builder (and a number of other related features) from the ground up, so it's entirely possible this won't port over well.  I just don't want the work to be in vain.

Your rebuilding it? I'd hope it'd be file based by default in that case - saving this stuff in the DB makes CI/CD incredibly hard. I've been doing it with Seeders, but a clunky hack. Flat file would be MUCH better.
</two-cents>

I have to agree here, having an export/import possibility for bread configuration is essential to get things running properly on CI and making it realistic to use in teams.

I've stopped developing this since you already told me that on Slack @fletch3555, feel free to take anything I made for your benefit if it's usable. I've stopped using Voyager because it simply wasn't what I was looking for after all and we're going to build something custom. Thumbs up for all your efforts tho!

@fletch3555 will this new BREAD builder use file-based configuration, usable for version control and proper deploys?


Fix styling in 404 RTD page + potential smart redirect

The redirect pages on readthedocs are missing styling.
E.g.: https://cirq.readthedocs.io/en/stable/generated/cirq.PauliString.expectation_from_state_vector.html

This should be easy to fix. The other issue is that for reference docs the user now have to find that same page in the new structure. Maybe we could add some Javascript magic to extract the target page from the current URL and do a JS based redirect instead of the HTML based one.

I think I can repro.
It seems the index page has the right path, but not the 404 one:
/google/.../github/Cirq [03:45 PM]$ cat rtd_docs/_build/html/index.html  | grep style
      <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
      <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
      <link rel="stylesheet" href="_static/tweak-style.css" type="text/css" />

versus:
/google/.../github/Cirq [03:45 PM]$ cat rtd_docs/_build/html/404.html | grep style
      <link rel="stylesheet" href="/en/latest/_static/pygments.css" type="text/css" />
      <link rel="stylesheet" href="/en/latest/_static/css/theme.css" type="text/css" />
      <link rel="stylesheet" href="/en/latest/_static/tweak-style.css" type="text/css" />

I am not sure why.

I'd like to see if we can get a redirect working as part of the launch.

@dabacon It looks like the redirect is working to the quantum AI homepage (there's just a five second delay).  RTD has been deprecated for over a year now, so I don't think it's worth fixing styling or making a smart redirect.  I vote we close this issue.
@dabacon If you agree, can you close this?

yep


Support flutter attach on iOS devices on non-macOS devices

It would be nice to be able to flutter attach to iOS devices on non-macOS devices.
This is related to #22771, but specifically for attaching.
I used to have a patch (older, shorter) (newer version, but longer) to be able to do this, but #50252 has made listing devices Mac-only by using an XCode-specific tool.
Even if it's not possible to build iOS apps on Linux or Windows, it would still be very useful to people who don't have a Mac, since we can use tools like Codemagic to build the app.
So, with this new feature:

flutter devices would detect iOS devices on any platform, maybe with a disclaimer that flutter run is not supported
flutter run would warn the user that it's not possible without a Mac, and would suggest using a build service like Codemagic
flutter attach would work the same as with a Mac or with Android devices
IDE/Daemon would work the same way

If a pull request to add this feature was made, considering the code adheres to the quality standards of the project, would it be merged?

It would be really great if we could support iOS debugging on non macOS platforms. Unfortunately the libimobile device functionality which we used to use had a number of stability and maintenance issues, so we ultimately decided to move to xcode supported tooling.
If we could find a way to structure the re-inclusion of this dependency so that it was both optional and did not affect the macOS development experience then maybe.


feature: add support raw ja3 data

I want you to add support for specifying raw data ja3 in header per request and global config
I know that utls supports generating hello client from raw ja3.
const headers = {
  'x-tlsproxy-upstream': 'socks5://localhost:1234',
  'x-tlsproxy-ja3':  '771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0'
}

What do you think about this?
May be and add return socket with upstream proxy onConnect so that you can write in one open connection with proxy and one tls session?

Openning connection with upstream proxy may be time consuming.
Therefore, I suggest opening a connection on onConnect using the tls configuration from headers onConnect

I want you to add support for specifying raw data ja3 in header per request and global config I know that utls supports generating hello client from raw ja3.
const headers = {
  'x-tlsproxy-upstream': 'socks5://localhost:1234',
  'x-tlsproxy-ja3':  '771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0'
}

What do you think about this?
May be and add return socket with upstream proxy onConnect so that you can write in one open connection with proxy and one tls session?

This project is currently using snowflake's NewUTLSHTTPRoundTripperWithProxy function to implement a custom RoundTripper function. In order to support uTLS client composition based on a given ja3 fingerprint, it would be necessary to reimplement the function. This should be doable, but I currently don't have much time to work on that.

Openning connection with upstream proxy per request may be time consuming. Therefore, I suggest opening a connection on onConnect using the tls configuration from headers onConnect

Setting the upstream proxy and TLS fingerprint per request is the main objective of this project. In order to open a connection to the desired upstream proxy based on request headers, the request needs to be intercepted and parsed first. If you do not need to change options on each request, you'd be better off using LyleMi/ja3proxy.

I want to change the fingerprint for each connection (socket) with tlsproxy, and all requests in this connection will be with the same fingerprint and one ip.
I want to use a proxy balancer and have one fingerprint and one proxy session (socket) used per tlsproxy connection.

I want to change the fingerprint for each connection (socket) with tlsproxy, and all requests in this connection will be with the same fingerprint and one ip. I want to use a proxy balancer and have one fingerprint and one proxy session (socket) used per tlsproxy connection.

Such functionality, while certainly useful in some cases, would require a lot of effort to implement. It would require rewriting tlsproxy from scratch, without relying on goproxy or snowflake, in order to handle CONNECT requests directly. That's beyond the scope of tlsproxy and not a priority for me. It might be easier to implement this in ja3proxy, but this would also require significant changes to that project.

Hi @PandaWorker. I have seen you on many issues on which we are currently interested in, looks like you have a deep understanding of proxies, http/https requests, etc.
Do you provide any consulting services? If yes, how we could get in touch with you? Would be really happy to get some help from you on solving our issue.


Update Java plugin: Omit extra 'Input' suffix from an input's class name if the input type's name ends in Input

The crux of the problem I'm trying to solve is the following: consider a schema that includes a type like
input MyEntityInput {

In this case, the current codegen will generate a Java class called MyEntityInputInput.java. I'd like to update this to simply generate MyEntityInput.java. I've updated the code to check whether the input type from the schema ends with "Input" and omit adding the "Input" suffix if it does.
The big use case here is when we're using graphql for CRUD operations. If that's the case, then it's very likely that the read operations will necessitate a schema that includes
type MyEntity {

but that means that we'd want to include the following for writes:
input MyEntityInput {

The one thing I'm hesitant about with my change is backwards compatibility. For anyone that's been using this tool and has FooInputInput.java classes in their codebase, this would start generating different class names. Depending on the philosophy of this open source project, this change could either be considered a breaking change for that use case, or there could be something introduced to the config file to maintain backwards compatibility, such as allowing users to specify a suffix or omit the default suffix. Let me know what your thoughts are!

Thank you @ataylormays !
Maybe in order to prevent the breaking change, this could be a a optional configuration?
Also, what about a use case like this?
type User {

}

type UserInput {

}

this will generate two types with the same name, no?

@ataylormays ?

Hey @dotansimha! Sorry for the delay. I can look into making this an optional configuration. I was trying to look back at the docs for the configs but the website isn't rendering them: https://graphql-code-generator.com/docs/plugins/java#configuration Any idea when those will be back up? It'd be helpful for working on config changes.

Regarding your use case above, this change wouldn't affect it because the change only applies to input types in your schema. The only time you'd have conflicting classnames is if you had something like the following:
type UserInput {

}

input User {

}

Ok sounds good. And the website was fixed ;)
Thanks!!!


Use GraphQL v15

@kamilkisiela Wow 😮
You are super fast 👍
Please report any issues with 15.0.0 so I can fix them ASAP.

@IvanGoncharov we're right on the edge with pretty much each commit checked by our internal tool in @the-guild-org :) We bumped GraphQL to 15.0.0 already in:

GraphQL Code Generator
GraphQL Toolkit
here in GraphQL Tools
GraphQL Inspector
GraphQL Mesh
GraphQL Config
GraphQL Modules
Sofa
and few more

So far so good 👍


VFD Definition

Currently: An alarm that indicates a loss of water e.g. during transport
BEDES: https://bedes.lbl.gov/bedes-online/variable-frequency-drive
ASHRAE: https://xp20.ashrae.org/terminology/index.php?term=vfd&submit=Search

I didn't realize - you actually use the ASHRAE definition already for Variable_Frequency_Drive, so just update to match the two.

Good catch! Fix is in https://github.com/BrickSchema/Brick/pull/164

Fixed in https://github.com/BrickSchema/Brick/pull/164


module can be VK_NULL_HANDLE

vkspec: If the graphicsPipelineLibrary feature is enabled and an instance of VkShaderModuleCreateInfo is included in the pNext chain, module can be VK_NULL_HANDLE.
Even though module could also be VK_NULL_HANDLE when using shaderModuleIdentifier, but gfxr doesn't support shaderModuleIdentifier extensions, so we don't need to care about it.
closed: https://github.com/LunarG/gfxreconstruct/issues/1120

CI gfxreconstruct build queued with queue ID 527.

CI gfxreconstruct build # 2885 running.

CI gfxreconstruct build queued with queue ID 543.

CI gfxreconstruct build # 2886 running.

CI gfxreconstruct build # 2886 passed.

CI gfxreconstruct build queued with queue ID 1965.

CI gfxreconstruct build # 2934 running.

CI gfxreconstruct build queued with queue ID 1993.

CI gfxreconstruct build # 2935 running.

CI gfxreconstruct build queued with queue ID 2014.

CI gfxreconstruct build # 2937 running.

CI gfxreconstruct build # 2937 failed.

CI gfxreconstruct build queued with queue ID 2366.

CI gfxreconstruct build # 2939 running.

CI gfxreconstruct build # 2939 failed.

Probably needs to be rebased on dev in order to pass CI on tcubumes4 because of the memory types PR merged with dev.

CI gfxreconstruct build queued with queue ID 2685.

CI gfxreconstruct build # 2944 running.

CI gfxreconstruct build # 2944 passed.


Remove NewBlockHashPacket/GetNewBlockPacket packets

Is your feature request related to a problem? Please describe.
I think that in general we could get rid of whole "NewBlockHashPacket" packet. When I checked it is only send inside "onNewBlockVerified" to "in some way randomized" set of peers that do not have this block already(or at least we did not mark it as known fro them) and I am not sure what is the purpose of it ?
If such peer received only block hash but does not have it locally it will instantly request our node to send him full block. If we dont send "NewBlockHashPacket"(why not send full block straight away ?), we could get rid also of "GetNewBlockPacket", which  is sent only if we received "NewBlockHashPacket", but we have not received full block yet.
Seems to be like a lot of unnecessary code and complexity.... @justinsnapp @mfrankovi you might know ?
Describe the solution you'd like.

I think this should be closed as it is saving quite a lot of network bandwidth


Use ==/!= to compare str, bytes, and int literals

Python 3.8 will probably raise SyntaxWarnings on the flake8 F632 issue raised below.
flake8 testing of https://github.com/spulec/moto on Python 3.7.1
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
./tests/test_awslambda/test_lambda.py:1137:24: F632 use ==/!= to compare str, bytes, and int literals
                assert len(messages) is 3
                       ^
1     F632 use ==/!= to compare str, bytes, and int literals
1

Coverage decreased (-0.03%) to 93.516% when pulling c3c75c12d956071650ba7f6ce45cf383e41572d7 on cclauss:patch-1 into 36a8a4ad00a6983ec6afc8bb6f77afa2a035389b on spulec:master.

Coverage decreased (-0.03%) to 93.516% when pulling c3c75c12d956071650ba7f6ce45cf383e41572d7 on cclauss:patch-1 into 36a8a4ad00a6983ec6afc8bb6f77afa2a035389b on spulec:master.

Coverage decreased (-0.03%) to 93.516% when pulling c3c75c12d956071650ba7f6ce45cf383e41572d7 on cclauss:patch-1 into 36a8a4ad00a6983ec6afc8bb6f77afa2a035389b on spulec:master.

LGTM


Yaml linting

Start that conversation linting in the repo, as well as YAML coding standard. This PR is strictly focused on YAML files, but I think getting this set. The next logical linting step would be for kube based YML files ...using something like kudeconform.

I wouldnt merge this yet as some of the in place yml files need to be updated

Hey @KingBain excellent idea! Currently we're using both nomo and the kubeval plugin for kpt but any additional linting improvement we can make is more than welcome :)

Since the code base in basically in place I changed the line length error to just a warning.
Its not a deal breaker, but some lines are 300 characters long and a lot of the license text is over 80 characters

reviewing over the next 24-48h

Resolved merge conflicts
Updated files for linting

👍


Not able to connect  clickhouse ODBC from Tableau Server

Hi We are trying to connect clickhouse with the Tableau server
To create a connection in Tableau  we establishing odbc connection
Following are deployment type

Clickhouse server (IP: <IP_ADDRESS>)
Tableau server (IP <IP_ADDRESS>)
Now we installed clickhouse ODBC in the Tableau server (IP <IP_ADDRESS>)
Following are the steps
Clickhouse-odbc building from Debian/Ubuntu
Build-time dependencies: UnixODBC
Execute the following in the terminal:
sudo apt install build-essential git cmake libpoco-dev libssl-dev libicu-dev unixodbc-dev
Build-time dependencies: iODBC
Execute the following in the terminal:
sudo apt install build-essential git cmake libpoco-dev libssl-dev libicu-dev libiodbc2-dev
Build steps
Install latest cmake
apt purge cmake
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates gnupg \ software-properties-common wget
wget -qO - https://apt.kitware.com/keys/kitware-archive-latest.asc | sudo apt-key add
sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
sudo apt-get update
sudo apt-get install cmake
Clone the repo with sub modules:
git clone --recursive https://github.com/ClickHouse/clickhouse-odbc.git
Enter the cloned source tree, create a temporary build folder, and generate a Makefile for the project in it:
cd clickhouse-odbc
mkdir build
cd build
Configuration options for the project can be specified in the next command in a form of '-Dopt=val'
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
Build the generated solution in-place:
cmake --build . --config RelWithDebInfo
cmake --build . --config RelWithDebInfo --target package
...and, optionally, run tests (note, that for non-unit tests, preconfigured driver and DSN entries must exist, that point to the binaries generated in this build folder):
cmake --build . --config RelWithDebInfo --target test

odbc.ini file
[ODBC Data Sources]
clickHouse = clickHouse
[clickHouse]
Driver    = /opt/clickhouse-odbc-1.1.9-Linux/lib/libclickhouseodbc.so
server    = <IP_ADDRESS>
username  = default
password  = default
port      = 8123
database  = ananth
uid       = default
trace     = 1
tracefile = /opt/clickhouse-odbc.log
When i try with isql -v clickhouse
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> help
[S1000]Connection refused
[ISQL]ERROR: Could not SQLTables**

Check clickHouse/server IP address in odbc.ini

Check clickHouse/server IP address in odbc.ini

The IP address is configured correctly

The IP address is configured correctly


Create Studio step form container

Part of  #902.
TASK: Create container, store, route etc for studio step form. This should enable developing further components for studio step form.

This has become irrelevant due to change in the UX.


Unable to use nestjs and admin-bro in the same app due to body-parser and formidable.

Backend api built with nestjs serves call at /, endpoints looks like /users/me  or /auth/login etc.
Before adding AdminBro, call to /auth/login with post body of {"email": "<some-email>", "password": "<some-password>" } works fine.
But when adding a module named Backoffice to my app.module.ts post body is not found when calling /auth/login and throws an exception in the controller.
All calls to AdminBro works fine under /admin
@Module({
    imports: [
      AdminModule.createAdminAsync({
        imports: [
          TypeOrmModule.forRootAsync({
            imports: [SharedModule],
            inject: [ConfigService],
            useFactory: (configService: ConfigService) => configService.typeOrmConfig,
          }),
          UsersModule,
        ],
        inject: [UsersService, ConfigService],
        useFactory: (usersService: UsersService, configService: ConfigService) => {
          return {
            adminBroOptions: {
              rootPath: '/admin',
              resources: AdminBroResources,
            },
            auth: {
              authenticate: async (email, password) => {
                const user = await usersService.findOne({
                  email: email,
                  password: ConfigService.getPasswordsHash(password),
                });
                if (user) {
                  return { email: `${user.firstName}`, title: user.email.split('@')[0] };
                }
                return false;
              },
              cookiePassword: configService.get('COOKIE_SECRET'),
              cookieName: 'backoffice',
            },
          } as AdminModuleOptions;
        },
      }),
    ],
    // providers: [UsersService],
  })
  export class BackofficeModule {}

package.json dependencies used:
"@admin-bro/express": "^3.0.1",  "@admin-bro/nestjs": "^1.0.0",  "@admin-bro/typeorm": "^1.3.0",  "express-formidable": "^1.2.0",
I've narrowed it down to ExpressLoader in this repo which moves jsonParser and urlencodeparser in layer stack below AdminBro router. Commenting this out breaks AdminBro but nestjs api works again.

should this issue belong to admin-bro-nestjs repository?

yep, not sure how I ended up posting it here. Will move it.


What Python versions are supported?

I see that the source code only supports Python 3.6-3.12, so are these the only versions supported currently? The following is a source code snippet：
        minVer := pythonVer(3, 6)
	maxVer := pythonVer(3, 12)
	if version < minVer || version > maxVer {
		return nil, fmt.Errorf("unsupported Python %d.%d (need >= %d.%d and <= %d.%d)",
			major, minor,
			(minVer>>8)&0xff, minVer&0xff,
			(maxVer>>8)&0xff, maxVer&0xff)
	}

Thanks!

Yes, this is correct.

With https://github.com/open-telemetry/opentelemetry-ebpf-profiler/pull/190 support for Python 3.13 was added.


[Test][e2e] Jdbc test checking data consistency.

Purpose of this pull request
Base on PR #5560  add e2e test case.
Issues #5623 .
Does this PR introduce any user-facing change?

How was this patch tested?

Check list

[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

[ ] Update the release-note.

Please resolve conflict

Please resolve conflict

@Hisoka-X It's done.
Originally posted by @mosence in https://github.com/apache/seatunnel/issues/5560#issuecomment-1778965487
I guess that@TestContainerswill be mutil thread for test. so more task running on sametime.  The result will be all task result. So incompare` mehtod had unpredictable error occurred.
So the e2e test I will add in Issue #5623.
Or someone can solve this problem?

`

I did some updated. Let's waiting CI.


CrossPlatform video stream not working

Describe the bug
Sometimes ios to Android video stream is not working
To Reproduce
Steps to reproduce the behavior:

IOS start a stream
Android try to join not able to see video but audio and rtm is working fine
IOS try to join working fine

Expected behavior
IOS-Android streaming should work
Screenshots
Smartphone (please complete the following information):

iphone x, Red mi note 8 pro
OS: Iphone , Android
Version: agora_rtc_engine: "3.3.2"

Additional context
IOS-IOS working fine Android-Android working fine it produces sometimes but when it produces it is happening with all the devices like i am testing with ios and android devices started stream with one ios device and jointed that stream in several devices of android none of them are showing video stream but iphone device jointed showing the stream.

could you pls provide a sdklog for when the issue happens? or at least provide a channel name you used and corresponding timestamp

Having the exact same issue. It worked perfectly until yesterday. Now video streams between iOS and Android aren't working anymore. Only one of the devices is showing the video streams (local and remote). On the other device local and remote streams are not working. I'm using SDK version 4.0.2. Here the logs from my iPhone 12 Pro Max where the local and remote streams weren't working: agorasdk.log
This seems to be related: #281


z.preprocess ignores .optional() modifier

zod version: 3.17.3
code:
const schema = z.preprocess(data => data === '' ? undefined : data, z.number()).optional();

schema.parse(undefined) // works, returns undefined
schema.parse('') // throws

However, this works:
const schema = z.preprocess(data => data === '' ? undefined : data, z.number().optional());
schema.parse(undefined) // works, returns undefined
schema.parse('') // works, returns undefined

I would expect the first schema to behave the same as the second schema.

This is behaving as intended, and I think it's actually important that these two behaviors can both be represented. Think of .optional() as wrapping the existing schema. It checks if the input is undefined upfront (before being parsed by the inner schema) and returns undefined if so. In the first example, the "optional" check happens even before the "preprocess".

It does make sense, thank you!


OSError: open failed

I googled this issue and I believe I got an usb device permission issue. I hope someone can help and thanks in advance.
I ran the 'cm-rgb-cli add-udev-rule' in the installation section but I am not sure if it was effective or not. Here is what I got:
'SUBSYSTEM=="usb", ATTR{idVendor}=="2516", ATTR{idProduct}=="0051", TAG+="uaccess", TAG+="udev-acl"' > /etc/udev/rules.d/60-cm-rgb.rules
Created /etc/udev/rules.d/60-cm-rgb.rules
udev should apply new rule automatically. If not, try running 'sudo udevadm trigger'

when I ran any cm-rgb-cli command, I got the OSError: open failed issue:
self.device.open_path(device_list[0]["path"])
File "hid.pyx", line 72, in hid.device.open_path
OSError: open failed

my system is Ubuntu 20.04 system. Again, thanks for any helps.

Here is how I resolved it: --refer to https://github.com/gfduszynski/cm-rgb/issues/7 (I had a very similar with that link, I neither cannot simply use sudo to call the CLI command.)

reinstall hid in https://github.com/trezor/cython-hidapi
reinstalled it with sudo pip install cm-rgb
then sudo pip install --upgrade cm-rgb

Now I can use sudo cm-rgb-cli set logo --mode=off save to turn off the rgb!
Question: I don't know what did this command do, why did it help me with this issue? Could you please explain a little bit? Thanks!

Hi, I'm not absolutely sure whats the reason behind this.
Did you reboot your PC in between attempts ?
If so, then perhaps udev rule has only started working after reboot.


Locate all lessons in one folder with a landing page with links to each lesson

Based on a group discussion at our monthly meeting there is a desire to change how lessons from multiple sources are stored. Current lessons are downloaded separately into /ods_dir/datacarpentry.org and /ods_dir/swcarpentry.github.io. This will clutter the root directory as we add more lesson sources and also makes displaying the links to lessons slightly more difficult.
Based on my understanding of the discussion I propose the following changes:

Store all lessons in a single lessons subdirectory with each lesson source in it's own subdirectory within lessons. E.g., /ods_dir/lessons/datacarpentry/R-ecology-lesson and /ods_dir/lessons/softwarecarpentry/git-novice. Alternatively we could abandon the lesson source entirely in our file structure, but my initial take is that this will actually cause more confusion and provide less flexibility for displaying links to lessons on a landing page. That said, it's no more work technically, so if the consensus is to drop the source that's certainly easy.
Provide a landing page for lessons (/ods_dir/lessons/index.html) that automatically presents links to all available lessons with headings differentiating the sources. Alternatively this could be generated in https://github.com/carpentriesoffline/carpentriesoffline-installer/ if that is preferred.

@jsteyn & @colinsauze how does this sound for addressing the concerns you raised yesterday?

Design note - we cannot simply move the lessons around, we need the final destination directory to be generated directly by wget. Not letting wget handle everything  looses the ability to only download files if they've been updated.


add support for PWA

The reactjs docs website does not have support for PWA, It would be better if is added as it can be then installed on any device

can I work on this feature

Feel free to send PRs. We can take it from there 😄

please check this out #4530


upgrade server hardware

Step 1: https://www.galaxus.ch/en/shoplist/show/CfDJ8HX6w6VPuHtCgkQ86Rt4COqSpzv6IaqMdmf-RCH4MD4ZQW1P3WMngsS8wTaz-IRIKKR5MmxVR1Y2TaYVCXHOGKfKldrA8WxAJKhudQk1xbHHLbp-GHZR3glN-VMs1cX46A
Step 2: https://www.galaxus.ch/en/shoplist/show/CfDJ8HX6w6VPuHtCgkQ86Rt4COo8eNs1P1YbGHtKrnayBGKk2s-TPsV2GTdD40k7-O-NTPTV6EQ9J0fHtrfeq_Wd1jbiS8IpAlpPtxpO4BSXRpftV6Hdy5OpYDgb7k5KmTkUrA

When installing zfs, read arch wiki
look for ashift and disk id

Also add automatic scruba

test-remove one m.2 drive to check if still bootable (if so i'm very happy :))

Use nixos?
Could actually be easier. And makes it more reproducible/easier to fix.

don't store server config on the server itself (would not be very smart lol)


[staging-next] Update GuPNP

Description of changes
Due to breakage of gupnp 1.4.z on staging-next, this causes quite a rebuild so targeting staging-next.
Targetting: https://github.com/NixOS/nixpkgs/pull/248496
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Due to breakage of gupnp 1.4.z

Will #246931 help? Switching between libsoup 2/3 [^1] is not something I can do bravely and I am honestly more confident to review #246931 instead if the breakage is caused by meson 1.2 :bow:
[^1]: If I read the changes right bbcd089a930dfa9f0f8797b474a400b9a5ecf4a2 and a40f54fbc2596ce8f52f621648847a68253277cd is doing this. See the #182618 "Main pain points", in my experience some of the breakage can only be observe in run-time, though I did not check what reverse dependencies are evolved here.

Will be testing soon, about to finish compilation.

Checked https://github.com/NixOS/nixpkgs/pull/246931, that would indeed help.

Booted succesfully into a system with these changes on top of staging-next. gnome-shell seems fine, anything I should test in particular?


How to get randomly generated pfx password from command line?

I need to get pfx password to use it in node.js app at IIS server, When trying to redirect wacs.exe console input it crashes:
c:\www64\ispdevenv\ispdev\socket.io>nodemon wacs.js
[nodemon] 1.8.1
[nodemon] to restart at any time, enter `rs`
[nodemon] watching: *.*
[nodemon] starting `node wacs.js`
stdout:

stdout:  [INFO] A simple Windows ACMEv2 client (WACS)

stdout:  [INFO] Software version 200.0.6965.40688 (RELEASE)

stdout:  [INFO] IIS version 8.5
 [INFO] Please report issues at https://github.com/PKISharp/win-acme

stdout:

stdout:  1: crm.host.ru - renewed 6 times, due after 2020.1.12 9:00:09
 C: Cancel

stdout:  Show details for renewal?:
stdout:

stdout:  Renewal
stdout:  [EROR] Unable to list details for target
System.IO.IOException: Неверный дескриптор.

   в System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   в System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
   в System.Console.get_CursorTop()
   в PKISharp.WACS.Services.InputService.Show(String label, String value, Boolea
n first, Int32 level)
   в PKISharp.WACS.Wacs.ShowRenewals()

child process exited with code 0
[nodemon] clean exit - waiting for changes before restart

There is --list option but no specific option to see the last renewal information for the specific host, including generated pfx password.

By the way there is something wrong with application console input / output handling, as even the simple redirection of --help to file crashes:
c:\www64\ssl\win-acme>wacs.exe --help > help.txt

Необработанное исключение: System.IO.IOException: Неверный дескриптор.

   в System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   в System.Console.SetCursorPosition(Int32 left, Int32 top)
   в PKISharp.WACS.Configuration.ArgumentsParser.ShowArguments()
   в PKISharp.WACS.Services.OptionsService.ShowHelp()
   в PKISharp.WACS.Wacs.Start()
   в PKISharp.WACS.Program.Main(String[] args)

c:\www64\ssl\win-acme>

Can you explain your use case? Because it sounds like what you should do is to use the script installation plugin, which you can configure to provide the password to the script as a parameter.
I wouldn't recommend parsing console output in any case because it's subject to changes from version to version.
If you need the password outside of a renewal then you can parse it from the renewal.json files on disk, but you have to turn off encryption first.

Redirected output cannot handle manipulation of the cursor position used to prettify the output in some places. In some places it's handled as a special case but you've found some new ones. I'll add some extra checks when I get around to it.

I want to use pfx certificate with password in node.js application, running along with IIS.
Specifying --pfxpassword somepass option causes the following error:
c:\www64\ssl\win-acme>wacs.exe --renew --pfxpassword somepass --baseuri "https:/
/acme-v02.api.letsencrypt.org/"

 [INFO] A simple Windows ACMEv2 client (WACS)
 [INFO] Software version 200.0.6965.40688 (RELEASE)
 [INFO] IIS version 8.5
 [INFO] Please report issues at https://github.com/PKISharp/win-acme

 [EROR] Store parameters cannot be changed during a renewal. Recreate/overwrite
the renewal or edit the .json file if you want to make changes.

How do I turn off the encryption in renewal.json ?

Hi @Dmitri-Sintsov, you cannot turn off encryption in renewal.json, you have to do that in settings.json, which is in the program folder (next to the .exe). After changing the setting, you can use an option in the main menu (More options... > Encrypt/decrypt) to decrypt all existing data.

Hi! There was no JSON config but XML settings.config, where I changed True to False :
<setting name="EncryptConfig" serializeAs="String">
    <value>False</value>
</setting>

But there is no More options > Encrypt / decrypt menu in the program:
c:\www64\ssl\win-acme>wacs
[INFO] A simple Windows ACMEv2 client (WACS)
[INFO] Software version 200.0.6965.40688 (RELEASE)
[INFO] IIS version 8.5
[INFO] Please report issues at https://github.com/PKISharp/win-acme
N: Create new certificate
M: Create new certificate with advanced options
L: List scheduled renewals
R: Renew scheduled
S: Renew specific
A: Renew all
C: Cancel scheduled renewal
X: Cancel all scheduled renewals
T: (Re)create scheduled task
I: Import scheduled renewals from WACS/LEWS 1.9.x
Q: Quit
Please choose from the menu:

Hi @Dmitri-Sintsov, that option is there in more recent versions of the program, though I can't remember exactly when it was introduced.


Wish: sh not bash

Please consider if you can't manage this by using only sh rather than bash, so that people who do not use bash can still use ocamlbrew.

Unfortunately I won't have time for this until I decide to use this system (for now I use both GODI and custom makefile based system).
Regarding how to test for bashism, on Debian there is dash which purpose IIRC is to implement nothing that's not POSIX. Maybe you have this for ubuntu as well.

I am going to close this issue for now.  If someone decides to create a set of patches to convert ocamlbrew into a sh script rather than a bash script I will consider them as long as they do not make the code more difficult to maintain.  Until that time, this will be a wontfix.


Plugin setCookie sets only the last cookie

Expected Behavior
In a plugin, calling context.request.setCookie would set multiple cookies:
module.exports.requestHooks = [(context) => {
  context.request.setCookie('first', 'first-value');
  context.request.setCookie('last', 'last-value');
}];

Expecting to see first=first-value; last=last-value in the request cookie header.
Actual Behavior
Instead I see last=last-value in the request cookie header.
Reproduction Steps
Add the plugin shown above. Make a request. Look at the cookie header value in Timeline tab.
Is there an existing issue for this?

[X] I have searched the issue tracker for this problem.

Additional Information
I believe the error is here:
https://github.com/Kong/insomnia/blob/bbceb6097c855f93f93b69c85af680d85ae0ac57/packages/insomnia/src/main/network/libcurl-promise.ts#L195-L197
Unfortunately curl won't add multiple cookies like this. It wants the entire cookie string. From the man docs on cookie.

The data should be in the format "NAME1=VALUE1; NAME2=VALUE2

So I can probably hack around this issue by carefully constructing the "name" here.
I think it's worth mentioning that my particular use case would be much easier to handle if I could actually pass an arbitrary string value to the curl command for --cookie. I'd like to take advantage of setting an file name here:

If no '=' symbol is used in the argument, it is instead treated as a filename

However this may be a can of curl worms you don't want to open.
Insomnia Version
2022.4.2
What operating system are you using?
macOS
Operating System Version
macOS Monterey Version 12.4
Installation method
download from insomnia
Last Known Working Insomnia version
No response

+1


Fix crash on first error in formatPlugin without stack trace

I came across an Unexpected token } in JSON at position error using the built-in JSON formatter, without context as to which tsconfig.json was the culprit. Turns out quite a few tsconfigs were badly formatted (having trailing commas in lists).
This PR makes sure that if an error happens, I know where to look, and avoids crashing on the first error it encounters.

@zkochan Could this be merged? Any blockers?


Add TF 2 support

Hello,
Thank you so much for the great plugin. I think if you advertise it correctly, it will be helping many developers!
I wonder if you could add support for TF2 tensor arrays too. Currently, I have to manually convert tf tensors to numpy tensors with .numpy() to be able to visualize them.
Many thanks

Hi,
Thank you for your suggestion.
Due to I do not have any experience in TF2, I just made a little changes in the python eval expression:
(${vn}.numpy() * 255.0 if (hasattr(${vn}, 'dtype')) and (${vn}.dtype == np.float64 or ${vn}.dtype == np.float32) else ${vn}.numpy()) if callable(getattr(${vn}, 'numpy', None)) else (${vn} * 255.0 if (isinstance(${vn}, (np.ndarray)) and (${vn}.dtype == np.float64 or ${vn}.dtype == np.float32)) else ${vn})
As you noticed that I used "callable(getattr(${vn}, 'numpy', None))" to test if the variable is a tensor. So you must ensure Eager Execution is enabled in TF2 to use this extension. Then I found it's very lucky that tf.dtype is same as np.dtype, so I can directly use np.float32 instead of tf.float32 to test the type avoiding forced "import tensorflow".
Now this change bring a tiny trick to customize if your object cannot be a numpy array directly, just add a "numpy()" method and an additional "dtype" property supporting the normalized image to your object, even you can make a metaclass or decorator something else to take effect for convenience.
for example:
class DebugViewImageWrapper:

    def __init__(self, a):
        self.a = a

    def numpy(self):
        return self.a

    @property
    def dtype(self):
        return self.a.dtype

img = cv2.imread("test.jpg")
normalized_img = img.astype(np.float32) / 255.
wrapped_img = DebugViewImageWrapper(img)
wrapped_img = DebugViewImageWrapper(normalized_img)


TCXO control not working with B-L072Z-LRWAN1

I'm making a custom board /node based on B-L072Z-LRWAN1 dev kit (just CMWX1ZZABZ module, no other MCU).
By default in B-L072Z-LRWAN1 TCXO_VCC (pin48) is on all the time (JP9, pins 2-3 shorted). LMIC-node works in this setting.
In order to save current in a battery powered node TCXO_VCC (pin48) must be shut down when not transmitting. In B-L072Z-LRWAN1 this is done by controlling TCXO_VCC (pin48) with PA12 (pin 1) by shorting pins 1-2 in JP9. LMIC-node does not work in this setting: LMIC-node doesn't control PA12 at all.
Is my finding correct? How can I fix it?

Thanks,
Tipo

Hi,
LMIC-node does not support / take into account power saving features like deep sleep or in this case powering down TXCO.
LMIC-node does not support these because deep sleep and any other power saving features are different per MCU platform and also depends on what other hardware features are implemented on any board (or not).
Being able to enable via (solder) jumper, the controlling of TXCO VCC via a GPIO pin is a board specific implementation.
LMIC-node supports the B-L072Z-LORAWAN1 in its default configuration. For any board specific options you will have to customize LMIC-node yourself. This also explains why LMIC-node "does not control PA12 at all".
I'm not sure how you could implement this. My first assumption is that this may require some hooks in the LMIC library to make it possible. Maybe you can ask your question (related to the LMIC library) on MCCI's GitHub repository.


Server Internal Error when acessing an event to which I send a proposal

I send a proposal to the event Python Brasil[12] - Apresentações, but when I'm logged in I can't access the event page. If I access the page without being logged, everything is normal, but when I log in to vote I get 'Server Internal Error' page. I tried in several browsers, and the error appears everywhere.

We are sorry for this. :(
I will investigate this problem. Ping me if you need more help.

Is this issue solved?

@vmesel I don't know. But we have a huge problem here: Speakerfight lacks of a simple alarm system.
We don't know what happened because it is inside a log, a big text file.
We could use something like Sentry to intercept and log tracebacks.

@mauricioabreu can you join me on gitter so we can try to solve this problem? We will need to contact @julianaklulo to see what she was doing that made this problem appear.
What Sentry does?

@vmesel it monitors and log tracebacks. It is easier than asking users "what happened" or "reading a huge log file". :)

Hello folks, i will lot into that right now. o/
@mauricioabreu in fact we have a Sentry setup and it is working https://sentry.io/speakerfight/speakerfight/
But as i was busy for a while we have a lots of events not resolved in there.


Remove empty bullet point

Hi @aharpervc, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

@aharpervc : Thanks for your contribution to the Azure documentation! The author, @arramac, has been notified to review your proposed change.

#sign-off

thanks for the fix, @aharpervc


Consider using localEndpoint instead of peer.* tags

peer.service, peer.ipv4, peer.ipv6 and peer.port are opentracing tags whereas zipkin uses the localEndpoint attribute:

This certainly is the case for https://github.com/Kong/kong-plugin-zipkin/issues/39 and https://github.com/Kong/kong-plugin-zipkin/issues/53

Hi, thanks for opening this issue as well. The current (after #52) implementation might already be doing some part of what you suggest. But I need your help filling up the gaps.
First, there's localEndpoint and remoteEndpoint. I am not sure about what they mean, or what they should mean in the context of Kong.
Assuming #57 is also merged, the each HTTP request proxied generates the following spans:

A request span called GET for the whole thing.
A proxy span called GET (proxy) for Kong's internal processing.
Zero or more spans called GET (balancer try n) for each balancer try.

Of these, the behavior of localEndpoint is:

null on the request span
null on all the balancer try spans
If a Kong service has been identified while proxying, the proxy's localEndpoint is { serviceName = "My-Service-Name" }.

For reference, here's how the remoteEndpoint field works, currently:

{ ipv4 = <remote address of the client making the request>, port = <client port> } on the request span
{ ipv4 = <balancer try ip>, port = <balancer try port> } on each balancer port.
{ ipv4 = <balancer success ip>, port = <balancer success port> } on proxy span.

Comments / questions:

I read that the serviceName must be lowercase. That restriction does not apply to server names. Shall I lowercase them before sending them to Zipkin? Are there other restrictions (i.e. no spaces)?
Should I set the localEndpoint for the request span and the balancer try spans, in addition to the proxy span?
I take I should add ipv4/port to localEndpoint. But which ips/should should I use there?  Perhaps the <remote address of the client making the request> (currently being used on the proxy span's remoteEndpoint)? That way the remoteEndpoint would point "upstream", to the services, while the localEndpoint would point "downstream", to the "client" making the request.
It that's the correct answer, then we should move the serviceName field to the remoteEndpoint.

But perhaps I am not understanding the roles of localEndpoint and remoteEndpoint correctly.

The local localEndpoint is basically the service itself, in this case it will be kong service. The remoteEndpoint is the endpoint to be called i.e. it involves a remote operation. If you have that information then it will be good to include, otherwise it is not mandatory to do it.

Of these, the behavior of localEndpoint is:

null on the request span
null on all the balancer try spans
If a Kong service has been identified while proxying, the proxy's localEndpoint is { serviceName = "My-Service-Name" }.

localEndpoint in all these cases should be kong (with its name, IP and port) I believe.

For reference, here's how the remoteEndpoint field works, currently:
{ ipv4 = , port =  } on the request span

This is correct

{ ipv4 = , port =  } on each balancer port.
{ ipv4 = , port =  } on proxy span.

Are these operations that involve a remote node? I guess yes, hence it is correct.

I read that the serviceName must be lowercase. That restriction does not apply to server names. Shall I lowercase them before sending them to Zipkin? Are there other restrictions (i.e. no spaces)?

I am not entirely sure about this (where did you read it?). what is true is that zipkin displays the service names in uppercase now:

Should I set the localEndpoint for the request span and the balancer try spans, in addition to the proxy span?

Yes please. That should be part of all spans.

I take I should add ipv4/port to localEndpoint. But which ips/should should I use there? Perhaps the  (currently being used on the proxy span's remoteEndpoint)? That way the remoteEndpoint would point "upstream", to the services, while the localEndpoint would point "downstream", to the "client" making the request.

localEndpoint should include the service's IP. In this case, kong's IP.

It that's the correct answer, then we should move the serviceName field to the remoteEndpoint.

remoteEndpoint can have a serviceName but that should be name of the downstream service which usually we don't know (maybe in kong case is different).
In sum, if kong calls service s1 then localEndpoint.serviceName = "kong" and localEndpoint.ip4 = $(kongs' ip) and remoteEndpoint.serviceName = s1 (only if I have access to that information) and remoteEndpoint.ip4 = $(s1 IP).

I read that the serviceName must be lowercase [...]

I am not entirely sure about this (where did you read it?)

In https://zipkin.io/zipkin-api/, models -> Endpoint -> serviceName. It currently says (emphasis mine):

serviceName: string. Lower-case label of this node in the service graph, such as "favstar". Leave absent if unknown. This is a primary label for trace lookup and aggregation, so it should be intuitive and consistent. Many use a name from service discovery.

Hi, I have given this a try in #63.
Here's the "before and after" traces:
https://gist.github.com/kikito/4cca7fb37498fb65830a4f1b8ba2236c
I've got some concerns:

For all the spans created by the plugin, the localEndpoint is now { serviceName = "kong" }, and nothing else (kong doesn't know its own IP address). It seems a bit ... unuseful?
When a span is tagged error: false it appears red on the UI. Should I remove the error tag and only use it when there's an error?
On the usual "API gateway" scenario, kong proxies traffic between a "downstream consumer" and an "upstream service", generating a root "request" span, with one proxy span and 1 or more balancer tries. These (now) all have the same serviceName, when Kong has identified a service to proxy to. But the ip4/port values can be very different: on the root span they point to the consumer, while on the other spans they point to the upstream service ips/ports. I think it is weird that they have the same serviceName but such different ips/ports.

Hi @kikito, this is great stuff.

For all the spans created by the plugin, the localEndpoint is now { serviceName = "kong" }, and nothing else (kong doesn't know its own IP address). It seems a bit ... unuseful?

It is not because now you can search spans based on that serviceName.

When a span is tagged error: false it appears red on the UI. Should I remove the error tag and only use it when there's an error?

Usually we don't pass zero values, error:false is a zero value because it means that there is not error.  It would be great if you can remove it.

On the usual "API gateway" scenario, kong proxies traffic between a "downstream consumer" and an "upstream service", generating a root "request" span, with one proxy span and 1 or more balancer tries. These (now) all have the same serviceName, when Kong has identified a service to proxy to. But the ip4/port values can be very different: on the root span they point to the consumer, while on the other spans they point to the upstream service ips/ports. I think it is weird that they have the same serviceName but such different ips/ports.

If kong is calling service A on behalf of an upstream service then that span should keep the localEndpoint.serviceName=kong and the IP could be kong's IP whereas the IP of service A should be the IP in the remoteEndpoint of the span (see https://zipkin.io/zipkin-api/#/default/post_spans).
Does that make sense?


Can't execute EnterFullScreenCommand or ExitFullScreenCommand

Hi,
Just to say this code is amazing, but when i try to execute one of the commands above everything freezes, I tried with async and non async commands but the result is same, video freezes.

@GoranBunjevac Did you manage to get a this working, as I'm stuck in the same part.


Invalid username / password for demo

I'm trying to login to the demo site using<EMAIL_ADDRESS>/ password, but I'm getting wrong credential errors. Credentials have been changed?

yep, someone remove adminuser ...

@cle61 Demo not working again, sorry. Can you may be make sure to remove :destroy, :update from Admin so that nobody changes it again. Thank you.

done, thanks