[SUPPORT] Job hanging on toRdd at HoodieSparkUtils

Describe the problem you faced
My job hangs during the toRdd task here is some screenshots of the task size

My spark-submit
spark-submit --deploy-mode cluster --conf spark.executor.cores=5 --conf spark.executor.memoryOverhead=6g --conf spark.executor.memory=43g --conf spark.dynamicAllocation.maxExecutors=50 --conf spark.sql.hive.convertMetastoreParquet=false --conf spark.rdd.compress=true --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --conf spark.kryoserializer.buffer.max=512m --packages org.apache.hudi:hudi-spark-bundle_2.12:0.8.0,org.apache.spark:spark-avro_2.12:3.0.1,com.audienceproject:spark-dynamodb_2.12:1.1.2 --py-files s3://bucket/modules.zip --files s3://bucket/config.yml s3://bucket/main.py
My cluster configuration

I also set shuffle.parallelism=2000
Expected behavior
It should run without hang
Environment Description

Hudi version : 0.8

Spark version : 3.0

Hive version : 3.1.2

Hadoop version : Amazon 3.2.1

Storage (HDFS/S3/GCS..) : s3

Running on Docker? (yes/no) : no

Need more details to understand what's going on:

Which line of code from HoodieSparkUtils was ran here?
What Hudi actions are you trying to perform?
What is the total input data size are you reading?
How many executors were actually created during the run?

Hi, thanks for you reply
Which line of code from HoodieSparkUtils was ran here?
The jobs hangs before even start, it hangs when it start to list files and tries to read s3 files.
the hanged task that the spark history shows me is this one

What Hudi actions are you trying to perform?
This job was suppose to join some tables and save the output to s3, the code line where it hangs ia an create table operation, here is code line
`        hudi_options = {
'hoodie.table.name': self.table_name,
'hoodie.datasource.write.recordkey.field': self.primary_key,
'hoodie.datasource.write.table.name': self.table_name,
'hoodie.datasource.write.operation': 'bulk_insert',
'hoodie.bulkinsert.shuffle.parallelism': self.bulk_insert_shuffle_parallelism,
'hoodie.datasource.hive_sync.enable': self.hive_sync_enabled,
'hoodie.datasource.hive_sync.database': self.hive_database_name,
'hoodie.datasource.hive_sync.jdbcurl': f'jdbc:hive2://{self.hive_jdbc_url}:10000',
'hoodie.datasource.hive_sync.table': self.table_name,
'hoodie.datasource.hive_sync.partition_extractor_class': 'org.apache.hudi.hive.NonPartitionedExtractor',
'hoodie.datasource.hive_sync.support_timestamp': 'true',
'hoodie.datasource.write.keygenerator.class': 'org.apache.hudi.keygen.NonpartitionedKeyGenerator',
'hoodie.datasource.write.row.writer.enable': 'false',
'hoodie.parquet.small.file.limit': 536870912,
'hoodie.parquet.max.file.size':<PHONE_NUMBER>,
'hoodie.parquet.block.size': 536870912
}
spark_df.write.format("hudi").options(**hudi_options).mode("overwrite").save(self.table_path)`
What is the total input data size are you reading?
1,6TB
How many executors were actually created during the run?
37

Ok @ZeMirella so basically it's about bulk insert a chunk of data. Was bulk_insert_shuffle_parallelism set to 2000 in this case? given the data size, you may want to try partitioning the output dataset. Say you choose a field foo from the schema as the partition field, and roughly it gives 200 partitions. A few configs you may try changing accordingly

hoodie.bulkinsert.shuffle.parallelism = 200
hoodie.datasource.write.row.writer.enable = true // why disabled it explicitly?
Use a proper class for hoodie.datasource.write.keygenerator.class and hoodie.datasource.hive_sync.partition_extractor_class. It should fit the partitioning strategy you chose. Check out this blog.
the code for spark write should be sth likespark_df.repartition(200, spark_df.col("foo")).write.format("hudi").options(**hudi_options).mode("overwrite").partitionBy("foo").save(self.table_path)

also, I see you are setting max parquet size to 1GB. is that intentional?

@ZeMirella : Can we get any updates on this end. Did the proposed suggestions work for you. please let us know.

Closing due to inactivity. Please reopen if you see this behaviour recurring.


Off-by-one buffer overflow in load_instrument

timidity/instrum.c load_instrument line 672 (and likewise 825 I think) only allocates an extra +1 byte, but line 788 copies the last sample (2 bytes) to the position one past the size. This causes a write one byte past the end of the buffer, which is caught by musl libc's mallocng at free time and presumably caught by instrumentation like ASan, valgrind, etc. Changing the +1's to +2 should fix it.

Should be fixed by commit 8367bc3ed2, thanks.
(It took me a while that the issue was in SDL-1.2 branch :)

Thanks! Sorry for the branch mixup - I forgot this was a nondefault one. Was the bug already fixed in the current one?

Was the bug already fixed in the current one?

Yep.


wc: streaming --files0-from and other improvements

My original focus was on --files0-from, which should be handled as a stream if the file specified (or stdin) is not a regular file. I've accomplished this by making a new Inputs type to represent the desired list of files to process. Its try_iter method does most of the heavy lifting to figure out how to actually run.
Secondarily, I have attempted to reduce the number of String allocations that occur while printing the results. Now, unless a file name needs escaping, or an error occurs, no additional allocations should be necessary to print results. An Input now borrows file names from the command line, unless they're read from --files0-from. print_stats was the biggest abuser, allocating a String for each column before before being join'd into a String for the whole line.  The TitledWordCount type is not necessary at all, which was another source of String allocation.
Finally, I've made some effort to make more cases match GNU wc's output. Errors encountered processing --files0-from as well as any encountered processing files listed on the command line will be escaped more consistently like GNU wc.  File names printed to the right of their stats will be less aggressively quoted, to match GNU wc, now only if they are not UTF-8 or if they contain a newline.

Can you please run cargo clippy? Thanks!

Yes. Apologies, I thought I had!
On Fri, Mar 31, 2023, 01:59 Daniel Hofstetter @.***>
wrote:

Can you please run cargo clippy? Thanks!
—
Reply to this email directly, view it on GitHub
https://github.com/uutils/coreutils/pull/4696#issuecomment-1491571856,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOVRSXXI6M6D5LVCYTOQ5LW62MGVANCNFSM6AAAAAAWOJ2MOM
.
You are receiving this because you authored the thread.Message ID:
@.***>

Yes. Apologies, I thought I had!

could you please split the PR into smaller commits ? This is a big patch ...

I've broken up the change into 4 different commits.  The lines between some concerns were a little blurry, as I ended up touching a lot of stuff, but I've tried my best to keep them each cleanly readable on their own.
I'm still working on getting the GNU coreutils tests running completely, but I've at least got the one complaining to run.  And, it turns out wc will read the entirety of a --files0-from file before processing so long as it is no more than 10MiB, so I need to do a little refactoring on the last commit...

please remove the "draft" when ready to be reviewed :)

Alright, I think I'm ready for some feedback to iterate on here.  On one point in particular, I didn't realize that when I introduced use of AsRawFd/FromRawFd that the import paths were new as of 1.66.  Also, the handy ilog10 method for determining how many digits an int will need for printing was only added in 1.67!  I see the current minimum is 1.64, which is pretty recent compared to some other repos' minimum version.  What's your policy on updating the minimum version of Rust?

What's your policy on updating the minimum version of Rust?

We do it only after cutting a release and check how important the new features are. Essentially we bump it by need. But always in a separate PR.

See this failure:

     Running `sccache rustc --crate-name uu_relpath --edition=2021 src/uu/relpath/src/relpath.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=9e8dee9d6aa6280e -C extra-filename=-9e8dee9d6aa6280e --out-dir /home/runner/work/coreutils/coreutils/target/debug/deps -L dependency=/home/runner/work/coreutils/coreutils/target/debug/deps --extern clap=/home/runner/work/coreutils/coreutils/target/debug/deps/libclap-e00c16ba6075349e.rmeta --extern uucore=/home/runner/work/coreutils/coreutils/target/debug/deps/libuucore-1bf9a7e8458cc252.rmeta -Awarnings -L native=/home/runner/work/coreutils/coreutils/target/debug/build/blake3-ca32ffe4cb3ec7ed/out -L native=/home/runner/work/coreutils/coreutils/target/debug/build/blake3-ca32ffe4cb3ec7ed/out`
error[E0599]: no method named `ilog10` found for type `u64` in the current scope
   --> src/uu/wc/src/wc.rs:719:46
    |
719 |                 let total_width = (1 + total.ilog10())
    |                                              ^^^^^^ help: there is an associated function with a similar name: `log`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `uu_wc` due to previous error

Caused by:

Hey Sylvestere.  Yeah, that's what prompted my question about your policy around minimum versions.... If you anticipate moving to 1.67 soon, I'd leave that in place.  Otherwise, we'll need to do something different.  It's unfortunate Rust hasn't stabilized its version check, https://doc.rust-lang.org/beta/unstable-book/language-features/cfg-version.html

If you anticipate moving to 1.67 soon

I think not just for this.

It's unfortunate Rust hasn't stabilized its version check

Honestly, I'm glad 😄 At least for this project, keeping a consistent MSRV would always be the best option. For libraries, it might be different of course.

If you anticipate moving to 1.67 soon

I think not just for this.

Oh sure, this is not a killer feature worth it on its own.  I'll see about copying it in verbatim if it's short, or writing a facsimile...

It's unfortunate Rust hasn't stabilized its version check

Honestly, I'm glad 😄 At least for this project, keeping a consistent MSRV would always be the best option. For libraries, it might be different of course.

Those jerks just keep adding useful stuff to the standard library!  This is certainly not the first time I stumbled onto something useful that just happened to be very new and caused this problem.

@jeddenlea on android:
 error: pathspec './tests/fixtures/wc/alice' did not match any file(s) known to git
error: pathspec 'in' did not match any file(s) known to git

rings a bell ?

@jeddenlea on android:
 error: pathspec './tests/fixtures/wc/alice' did not match any file(s) known to git
error: pathspec 'in' did not match any file(s) known to git

rings a bell ?

I had made a symlink named alice in wonderland.txt that pointed to alice_in_wonderland.txt, hoping to emphasize that their contents were the same.  But, that Android script is not friendly to symlinks with whitespace in their name.  So, to keep things simple, I'll just make alice in wonderland.txt a file too.

The GnuTest failures look like a hiccup with apt.  Hopefully the Windows tests were also just bad luck...

nope, I think the test doesn't work on windows. see:
  TRY 1 SLOW [>4800.000s] coreutils::tests test_wc::test_files0_progressive_stream
   Canceling due to interrupt: 1 tests still running
       ABORT [4856.604s] coreutils::tests test_wc::test_files0_progressive_stream
     Message [         ] code 0xc000013a: {Application Exit by CTRL+C}
The application terminated as a result of a CTRL+C. (os error 572)
------------
     Summary [5131.687s] 2147 tests run: 2140 passed (1 slow, 4 leaky), 7 failed, 24 skipped
error: test run failed
Error: The operation was canceled.

it times out

nope, I think the test doesn't work on windows. see:
  TRY 1 SLOW [>4800.000s] coreutils::tests test_wc::test_files0_progressive_stream
   Canceling due to interrupt: 1 tests still running
       ABORT [4856.604s] coreutils::tests test_wc::test_files0_progressive_stream
     Message [         ] code 0xc000013a: {Application Exit by CTRL+C}
The application terminated as a result of a CTRL+C. (os error 572)
------------
     Summary [5131.687s] 2147 tests run: 2140 passed (1 slow, 4 leaky), 7 failed, 24 skipped
error: test run failed
Error: The operation was canceled.

it times out

:facepalm: I guess I could have looked myself!  Sorry about that.  I think I understand what the failure is, and it shouldn't be too hard to fix in theory.  I'm currently in the middle of getting VirtualBox running so I can actually work on the thing in Windows....

Woo! Looks like I fixed the Windows stuff.  That was a.... pain.  VirtualBox didn't work as it is wont to do.
Anyway, in the interest of preventing any individual commits entering the tree which fail CI on their own, I patched the last two here and rebased them all again.  You can see the diffs here: https://github.com/jeddenlea/coreutils/compare/old_wc..wc

impressive, well done :)


Software license?

Hi Jasper,
I've found your analysis script (visualize_alphafold_results.py) particularly useful and would like to ensure that I respect its intended use. I notice you haven't stated a particular software license in your repo, which defaults to a restrictive licence. Did you intend for this code to be Open Source, and would you consider a permissive license such at MIT?
Cheers,
Thomas

Thanks for pointing this out, Thomas. I added the MIT license to the repository. Best of luck with your AlphaFold adventures :-)
Warm regards,
Jasper


feat: weak_partial

playing around a bit here.  this replace all of the various SlotCaller subtypes with a single weak_partial object that I can imagine wanting to use in other libraries (might make it a public export eventually).

ha ... turns it it's very hard to touch anything here without decreasing performance :joy:

see #178 instead


HTML markup appearing in bank transfer instructions

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to any Collective.
Click on Donation.
Donate... (any amount).
See error:

Expected behavior
Should not appear   or  whatever... nor  and  (see screenshot above).
Desktop (please complete the following information):

OS: RPi OS 32-bit.
Browser: Chromium
Version  88.0.4324.187 (Build oficial) Built on Raspbian , running on Raspbian 10 (32 bits)

Thank you for reporting this @fabianmendes.
I thought we migrated all of them but apparently, we still have 41 hosts with HTML markup in their instructions. We should sanitize that to remove all markup.
SELECT * FROM "Collectives" c WHERE (settings -> 'paymentMethods' -> 'manual' -> 'instructions')::varchar LIKE '%<%'

Actually, this is a duplicate of https://github.com/opencollective/opencollective/issues/4002


New topic: Web API +  authentication and authorization

We've had several customer requests for this.
Related: Topic: how to set up authentication for ASP.NET Core Web API with ADFS #3845
From Web API tutorial

Everything works great and I must say amazing job in step by step explanation. Hope this sample with authentication and authorization will be up soon. Btw, it would be great if on one example we make full circle, create API (this tutorial), add authentication second, then deploy to Azure or other platform and then consume from other application.E

@danroth27  can you assign someone to write a sample and outline the topic?
cc @scottaddie

is this only available with Identity Server 4? or is there something built-in now?

@danroth27 can you assign someone to write a sample and outline the topic?

I can do it and have talked to @scottaddie about it

@danroth27 Do you have an outline in mind, or should @cwoodruff take a shot at that?

We certainly write topics about securing Web APIs using Azure AD, Azure AD B2C and ADFS. We should look first to see if there are existing Azure topics that we can just reference. For ADFS I suspect we'll need a new topic.
We don't currently have a story for issuing tokens from an ASP.NET Core app. For that you need to use something like IdentityServer4. Folks wanting to use IdentityServer4 with ASP.NET Core should refer to the IdentityServer4 docs, which we already link to.

See forum post
@scottaddie  can you follow up with @cwoodruff  on this?

@cwoodruff Please let us know if you still have interest and time to write this doc.

@CamSoper  would you be able to work on any of securing Web APIs using Azure AD, Azure AD B2C and ADFS?

@Rick-Anderson Yes, I have recent experience there!  I can help with that.

@CamSoper  can you give us an ETA for starting this work?

@Rick-Anderson I'm loaded up for the rest of this sprint, but if you want to assign it for 140, I can get to it then.

@CamSoper  scheduled for 141

Thanks, @Rick-Anderson .  Sorry it's been taking me so long.  I've had a lot of irons in a lot of fires these past couple sprints.

@CamSoper 	moved to 142

@isaac2004 This has been assigned to me in the backlog for a while, it's time we got it done. You still up for helping out?  It'd be fun to collaborate!  HMU on Teams.

@danroth27 - Hey Dan, @isaac2004 and I were just taking a look at this and we have questions.  Can you clarify the ask here?
Are we looking specifically for a topic wiring up an ASP.NET Core Web API and ADFS specifically?  Vittorio has that scenario on his blog, albeit it's a bit older and doesn't work for ASP.NET Core as written.
Basically I'm asking if the object of the doc is to wire up a Web API with a generic OAuth authentication provider (like Azure AD, ADFS, etc.) or if the doc is to wire up a Web API with ADFS and only ADFS.
And how much of a client do we want?  SPA?

BTW, the extension that seems to possibly do this is
  services.AddAuthentication(AzureADDefaults.AuthenticationScheme)
            .AddWsFederation(options => new WsFederationOptions
            {
                Wtrealm = Configuration["ida:Wtrealm"],
                MetadataAddress = Configuration["ida:ADFSMetadata"]
            });

Would need to test by spinning up a DC however

@CamSoper Please work with @blowdart on this. There is also relevant work happening in ASP.NET Core 2.2 around Web API authentication that you'll probably want to take into account.

FYI everybody (esp. @danroth27 and @rick-anderson) - @blowdart and I decided that this topic should wait until 2.2 is near release because 2.2 will have some critical new functionality in this regard.  Barry says we should focus on AAD and AAD B2C scenarios first -- Which happens to be the other issues assigned to me, so that's what I'll be doing in the short term.

@Rick-Anderson are there any updates?
I need to authenticate and authorize the user inside ASP.NET Core 2.2 Web API which is deployed to GKE (Google Kubernetes Engine) as docker container and is being called by UI app as well as by other other APIs (microservices). In request header a bearer token passed to my API. So I need to identify the user the request sent on behalf of, and figure out the user permissions. I also will need to pass the token to other APIs when I call them via HttpClient. The users and their permissions come from ADFS and login process is SSO.
Any pointers to any helpful materials are nuch appreciated. Thanks!

This topic seems to be focusing on guidance using heavyweight directories such as AD and Azure AD, but I would like to put my vote in for including guidance for securing APIs with authentication/authorisation that don't have these services supporting them. Maybe using a custom user repository like the SQL one that comes by default with Identity (that may or may not be appropriate, but then that is why I am asking for guidance...).
My personal scenario is for securing web api's that are internet facing that are being consumed by client systems not users/web apps (I guess this would be a B2B integration scenario).

Per @HaoK
There's a bit of conflation here, and there are two parts of identity, users/roles and that stuff is applicable everywhere, identity by default uses cookies to store that info, so its that piece that is not applicable to WebAPI, so they would have to do the work to use identity to login/etc, and then typically generate a bearer token (which is why IdentityServer/OpenIdict is used there).
We are also integrating identity server with identity:
https://github.com/aspnet/AspNetCore/issues/5833

@Rick-Anderson so is there guidance anywhere for the first part you mentioned i.e. use identity to login to a web api?
i think it should be reasonably straightforward to wire in the non-ui bits of identity to a web api project to get the sql backend in place, but i am wondering if there is inbuilt framework stuff or guidance for the logging in bit or whether i need write code for example for grabbing an http authorisation header (e.g. client using basic auth over https) and running some sort of validateuser method on the extracted/decoded creds (which feels a bit old skool). i have seen a few different articles on the web, just not sure what is the best way for my scenario.

I've watched Dan Roth and Chris Woodruff's you tube video's to get us going with Core's Web API's but we have to authenticate via .net Identity (which our current .net 4.6.1 Azure website uses).   Rick Anderson seemed to say to wait until the beginning of 2019.  Can you point me to how to do this?
BTW - I also commented on this issue at:
https://docs.microsoft.com/en-us/aspnet/core/security/authentication/identity?view=aspnetcore-2.2&tabs=visual-studio#feedback

@wewald thanks for watching the videos. I am planning on getting some blog posts and videos developed covering AuthN and AuthZ for ASP.NET Core Web API. If you can use a third-party solution, I would recommend looking at Auth0 https://auth0.com/.

@javiercn are there docs covering this already?

@cwoodruff  or @CamSoper  - any chance one of you could work on this?

@mkArtakMSFT This is for on premise active directory (not Azure Active Directory) which is not something we have out of the box support. I've googled it and found the following docs:
https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/development/single-page-application-with-ad-fs
I recommend @danroth27 or @Rick-Anderson follow up with the Identity group doc folks to either update their docs for asp.net core or put together a separate sample. Ultimately it would mean to change the code to use ASP.NET Core.
@mkArtakMSFT I don't think we have to take any action here. So we should either close it or move it to the appropriate docs repo. There is a github repo for something like it https://github.com/microsoft/adfs-sample-msal-dotnet-webapp-to-webapi and I'm not sure if there is a more appropriate repo or who to reach out.
@danroth27 @Rick-Anderson Do any of you know the right folks to reach out to?

@mkArtakMSFT I consider my part in this handled, as this scenario is not something our team directly owns.

@blowdart this seems to be a highly requested topic to document. Do you want to do that? Or maybe @anurse can suggest somebody to to handle this?

No. If it's ADFS then it should probably fall to the identity / ADAL team

I created https://github.com/MicrosoftDocs/windowsserverdocs/issues/3312 and reached out to the team.

In seeing the possibility of getting some traction on this item, I wanted to clarify that the need we  see is to extend website authentication currently done via Identity (2019 link below) to authenticate RestAPIs users.  My understanding is that OAuth is needed to do this, which would not seem to have any need to  involve Active Directory Federation Services (which to my understanding is to use Azure's Active Directory database as opposed to the SQL tables that are built as part of Identity).
https://docs.microsoft.com/en-us/aspnet/core/security/authentication/identity

the need we see is to extend website authentication currently done via Identity (2019 link below) to authenticate RestAPIs users.

That's what the issue was opened on. I think the issue got hijacked by ADFS.  @blowdart  @javiercn  please respond.

Well now you're changing what the original request was for. The original request was ADFS. Using identity is something different and involves identity server, again, a third party product and not something we would document.

"Identity" is not a 3rd party product.
"The ASP.NET membership system was introduced with ASP.NET 2.0 back in 2005, and since then there have been many changes in the ways web applications typically handle authentication and authorization. ASP.NET Identity is a fresh look at what the membership system should be when you are building modern applications for the web, phone, or tablet." https://docs.microsoft.com/en-us/aspnet/identity/overview/getting-started/introduction-to-aspnet-identity

@wewald  see https://identityserver.io/ which is not a Microsoft product.

chiming in here again as i agree with wewald that the request for guidance is nothing to do with adfs. there is guidance in the docs for how to do this with aspnet mvc but there is no comparable guidance for web api (which is why the request materialised in the first place i think).
@Rick-Anderson you pointed us to identityserver, are you saying then that there is no solution in aspnet core web api and identity for authenticating web api's and that our only recourse is a third party product?

I am very surprised to learn about this product ambiguity as we got started with Membership in  2006 and are of the perspective that we need to move to Identity to align what we want to be an  Azure based strategy and we now need to take orders via RestAPIs to our offerings.
The link below has a list of .net Framework support from 2.0-4.8.   We need guidance from Microsoft on how we extend the path Microsoft guided us to, to that that allows us to use RestAPIs.
https://docs.microsoft.com/en-us/dotnet/api/system.web.security.membership

Fixed in #15288


fix: extension class conflicts due to mui makestyles classNames conflicts

Signed-off-by: Abhishek-kumar09<EMAIL_ADDRESS>Description

Some compilations are creating conflicts due to overuse of root classnames in the mui.
Different Classes are hashed to same string causing ui issues after webpack compilation.

This PR fixes #
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Codecov Report

Merging #5383 (efd2d05) into master (0bc68d1) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #5383   +/-   ##
=======================================
  Coverage   17.47%   17.47%
=======================================
  Files         107      107
  Lines        9882     9882
=======================================
  Hits         1727     1727
  Misses       7733     7733
  Partials      422      422

Flag
Coverage Δ

e2etests
∅ <ø> (∅)

gointegrationtests
6.00% <ø> (ø)

integrationtests
∅ <ø> (∅)

unittests
13.16% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bc68d1...efd2d05. Read the comment docs.


Picard error in ema_wrapper.sh

Getting an error from Picard in ema_wrapper.sh:
Exception in thread "main" htsjdk.samtools.SAMFormatException: Error parsing SAM header. Problem parsing @PG key:value pair ID:NA12878_WGS_v2:LibraryNotSpecified:1:unknown_fc:0 clashes with ID:ema. Line:
@PG     ID:ema  PN:ema  VN:0.1.0        CL:/home/vlad/bin/ema align -1 bucket000/subset_L001_R1.preproc.fastq -2 bucket000/subset_L001_R2.preproc.fastq -r /data/cephfs/punim0010/extras/10x/refdata-GRCh37/fasta/genome.fa -o bucket000/ema_out.sam -R @RG     ID:NA12878_WGS_v2:LibraryNotSpecified:1:unknown_fc:0    SM:NA12878_WGS_v2; File /data/cephfs/punim0010/extras/vlad/synced/umccr/10x/ema/
10k/bucket000/ema_out.sorted.bam; Line number 87

The @RG line in bucket000/ema_out.sorted.bam is indeed having two ID tags, the first one ID:NA12878_WGS_v2:LibraryNotSpecified:1:unknown_fc:0
is the one that I provided with the -R option to ema_wrapper.sh:
EMAPATH=/home/vlad/bin/ema \
PICARDPATH=/data/cephfs/punim0010/extras/10x/miniconda/envs/10x/share/picard-2.17.11-0/picard.jar \
/data/cephfs/punim0010/extras/vlad/synced/umccr/10x/ema/ema_wrapper.sh \
-r /data/cephfs/punim0010/extras/10x/refdata-GRCh37/fasta/genome.fa \
-R $'@RG\tID:NA12878_WGS_v2:LibraryNotSpecified:1:unknown_fc:0\tSM:NA12878_WGS_v2' \
-t 2

And another one is ID:ema, meaning it was added by the pipeline as well?
I'm wondering if it's me doing something wrong?

Ah, It's not a duplicated flag, it's unescaped ema align command line. Taking it back.


[REVIEW] VardaanAggarwal

GitHub Profile Link
https://github.com/VardaanAggarwal

Profile Review
TLDR;
Your profile looks awesome! See the full details with my suggestions below.

Full details
Your profile looks awesome! Here some suggestions from my side:

remove the profile view counter due its inaccurate
consider removing the hg follower count due its duplicate information
consider adding some more words about yourself

looks good
I suggest pinning an even number of repos
you duplicated the Twitter link, I suggest you remove one


Polish traceloop code

Hi.
In this PR, I added several commits to polish actual traceloop code.
Best regards.

LGTM after the change annotation->label.
Since you are adding a field isPointer to describe syscall arguments, you could add another field isString to describe arguments of type char *. That would address the following TODO:
// TODO Find all syscalls which take a char * as argument and add them there.
var syscallDefs = map[string][6]uint64{

But that can be done in a future PR.


In eosjsV21.0.3, eosioV2.0.7 when trying to create an account it is accepting "0287cha66yyr" and directly creating an account in its normalized form i.e. ".2..cha..yyr",created without any error.

Version of EOSJS: 21.0.3
EOSIO version 2.0.7
Clear guidelines of eosio account naming states that "Each account is identified by a human readable name between 1 and 12 characters in length. The characters can include a-z, 1-5, and optional dots (.) except the last character."
But when trying to create an account with characters other than this, error should be thrown. But right now, it is automatically normalizing it and created an account with the normalized name without any warning or error thrown. This can be misleading as user might think the account has been created by the name he has passed.
This issue is inconsistent i.e. It throws error sometimes and sometimes it doesn't.
Expected behavior
Consistently error should be thrown for all unacceptable usernames

I have just tested the name you mentioned 0287cha66yyr and I received the error: Name should be less than 13 characters, or less than 14 if last character is between 1-5 or a-j, and only contain the following symbols .12345abcdefghijklmnopqrstuvwxyz which was added during #726 and #748.
Could you give more details on how you are arriving at this issue with eosjs? The regex added during those PRs should stop the code before it attempts any serialization.


Sybase Support

How about sybase support?
If you are interested, here are the file for Sybase IAnywhere.

Originally reported on Google Code with ID 125
Reported by @cbrandsma-cs on 2009-06-29 21:10:15

 * *Attachment: [IAnywhere.zip](https://storage.googleapis.com/google-code-attachments/elmah/issue-125/comment-0/IAnywhere.zip)*

Now there is something I have not thought about in a very long time.

@cbrandsma-cs See, you were not forgotten. :wink: I've always left this issue open so anyone wanting to add Sybase support would be able to search, find and start with the attachment. However since no one's picked up the lead on this, I'm considering it stale and closing the issue. In any event, the simpler way of going about this today would be to share via a separate GitHub repo and publish on NuGet.


Update Markdown and Django Version

Markdown was outdated and preventing to compile. Also Django 3.2.20 has a security flaw so suggesting to upgrade.
Fixed error

Thanks for the contribution


Fix compile error under newer toolchain and arrow 10

Fix several build failures

Fix build gae with grpc 1.30 (which is the default version of ubuntu:latest)
Fix build gie and gle with arrow10 and gcc11

Codecov Report

Merging #2355 (b92238a) into main (8fec826) will decrease coverage by 3.56%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2355      +/-   ##
==========================================
- Coverage   43.59%   40.03%   -3.57%
==========================================
  Files          88       88
  Lines        9714     9714
==========================================
- Hits         4235     3889     -346
- Misses       5479     5825     +346

Impacted Files
Coverage Δ

python/graphscope/tests/unittest/test_java_app.py
0.00% <0.00%> (-100.00%)
:arrow_down:

python/graphscope/deploy/hosts/cluster.py
30.48% <0.00%> (-57.32%)
:arrow_down:

python/graphscope/analytical/app/java_app.py
24.36% <0.00%> (-27.92%)
:arrow_down:

python/graphscope/framework/utils.py
43.38% <0.00%> (-14.47%)
:arrow_down:

python/graphscope/framework/app.py
70.50% <0.00%> (-9.68%)
:arrow_down:

python/graphscope/client/rpc.py
73.88% <0.00%> (-6.72%)
:arrow_down:

python/graphscope/framework/graph_schema.py
52.73% <0.00%> (-6.63%)
:arrow_down:

python/graphscope/tests/conftest.py
48.20% <0.00%> (-4.24%)
:arrow_down:

python/graphscope/analytical/udf/utils.py
36.17% <0.00%> (-3.20%)
:arrow_down:

python/graphscope/client/session.py
65.15% <0.00%> (-3.04%)
:arrow_down:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee5185e...b92238a. Read the comment docs.


Mask support for heatmap layer

Will heatmap.js have a mask support?
Thanks

could you elaborate a bit? do you mean the leaflet layer? any of the maps layers? or the heatmap core library? also what kind of mask support are you looking for? (applying a svg mask onto the exported image could already work)

We want to dynamically hide parts of heat layer by a leaflet polygon mask. For example, we define a ROI with a polygon layer , leaflet will only show parts of heatmap within ROI. If we edit shape of ROI, the leaflet will update the heatmap layer with the mask.

Is it possible to do this now?
I would like to show the heatmap inside a mask, but the example doesn't work for me


ngFormBuilder translation

This looks good. We will need to so some testing on our end. I am rolling out a low-risk build right now and will include these PR's in the next build.
Thanks you for your contribution.

This looks really good. The only thing is we should use the formioTranslate from ngFormio (https://github.com/formio/ngFormio/blob/163aeaa8af2d61a4b260d9ae4fa7184b4a56e5fa/src/formio.js#L63).
ngFormio is already a dependency so it should already be available. No need to recreate it.

@AbdallaMohamed I want to pull in this pull request, but see you did not respond to @randallknutson comments. Do you think you can modify the formioTranslate filter in the renderer to include this functionality?  If so, then I will move this into the platform.

@travist I just pushed a commit and did the things @randallknutson pointed out in his comment. I don't get what you mean by modifying the formioTranslate filter in the renderer though? Could you explain what you exactly need me to do? I'll be happy to make a PR there as well!

@AbdallaMohamed what he is saying is that in our Renderer @ https://github.com/formio/ngFormio we already define a filter called formioTranslate that includes our old way of doing it @ https://github.com/formio/ngFormio/blob/163aeaa8af2d61a4b260d9ae4fa7184b4a56e5fa/src/formio.js#L63. It seems you are also creating a filter in this pull request with the same name, so I think they will collide with each other if we pull in this pull request since this repo depends on the Renderer library.
What we propose is to move your filter changes into the Renderer @ https://github.com/formio/ngFormio and then this library will just pull in those changes since it depends on the renderer. This will also keep the collision from occurring.
Thanks for your contribution.

Basically, we would like for you to make the changes you made here inside this file so that it works with either translation strategies. https://github.com/formio/ngFormio/blob/163aeaa8af2d61a4b260d9ae4fa7184b4a56e5fa/src/filters/translate.js

@travist I made a pull request in the renderer. https://github.com/formio/ngFormio/pull/316
Let me know if that coding style makes sense, or if I should code that in a different way.

@AbdallaMohamed You should remove src/filters/formioTranslator.js as well.

Looks perfect. Thanks for the changes!


Pushing the same screen with pushReplacement invokes "Duplicate GlobalKeys" error

I have a screen with list of widgets which is scrolling to selected the element on tap. To achieve this I use Scrollable.ensureVisible and GlobalObjectKeys. It usually works well except one case. If I am at my scroll screen and pushing this screen again by Navigator.pushReplacement occurs "Duplicate GlobalKeys detected in widget tree" error (if I push some other screen and then my scroll screen - everything is OK). But everything works well - app isn't crashing, scroll works as expected. Can't find a way to resolve this problem.
Full description of the exception:
======== Exception caught by widgets library =======================================================
The following assertion was thrown while finalizing the widget tree:
Duplicate GlobalKeys detected in widget tree.

The following GlobalKeys were specified multiple times in the widget tree. This will lead to parts of the widget tree being truncated unexpectedly, because the second time a key is seen, the previous instance is moved to the new location. The keys were:
- [GlobalObjectKey String#bd95f]
  [GlobalObjectKey String#392a1]
  [GlobalObjectKey String#b1172]
  [GlobalObjectKey String#65986]
  [GlobalObjectKey String#47dd0]
  [GlobalObjectKey String#5a612]
  [GlobalObjectKey String#4cad0]
  [GlobalObjectKey String#9344c]
This was determined by noticing that after widgets with the above global keys were moved out of their respective previous parents, those previous parents never updated during this frame, meaning that they either did not update at all or updated before the widgets were moved, in either case implying that they still think that they should have a child with those global keys.
The specific parents that did not update after having one or more children forcibly removed due to GlobalKey reparenting are:
- RepaintBoundary(renderObject: RenderRepaintBoundary#c9ecc relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#6838c relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#a8b6b relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#0ed26 relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#6b21e relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#96952 relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#59325 relayoutBoundary=up4)
  RepaintBoundary(renderObject: RenderRepaintBoundary#bf00e relayoutBoundary=up4)
A GlobalKey can only be specified on one widget at a time in the widget tree.
When the exception was thrown, this was the stack:
#0      BuildOwner.finalizeTree.<anonymous closure> (package:flutter/src/widgets/framework.dart:3083:15)
#1      BuildOwner.finalizeTree (package:flutter/src/widgets/framework.dart:3108:8)
#2      WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:866:19)
#3      RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:381:5)
#4      SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1289:15)
#5      SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1218:9)
#6      SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:1076:5)
#7      _invoke (dart:ui/hooks.dart:145:13)
#8      PlatformDispatcher._drawFrame (dart:ui/platform_dispatcher.dart:338:5)
#9      _drawFrame (dart:ui/hooks.dart:112:31)
====================================================================================================

Minimal reproducible example (just tap FAB to reproduce the problem):
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  final ScrollController _chipsScrollController = ScrollController();
  String _currentChip = 'Chip 1';
  static const _chips = [
    'Chip 1',
    'Chip 2',
    'Chip 3',
    'Chip 4',
    'Chip 5',
    'Chip 6',
    'Chip 7',
    'Chip 8',
  ];

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: SizedBox(
          height: 70,
          child: ListView.separated(
            controller: _chipsScrollController,
            physics: const ClampingScrollPhysics(),
            padding: const EdgeInsets.fromLTRB(16, 12, 16, 12),
            scrollDirection: Axis.horizontal,
            itemCount: _chips.length,
            itemBuilder: (final BuildContext context, final int index) {
              return ChoiceChip(
                  key: GlobalObjectKey(_chips[index]),
                  label: Text(_chips[index]),
                  selected: _chips[index] == _currentChip,
                  onSelected: (final bool value) {
                    _currentChip = _chips[index];
                    Scrollable.ensureVisible(
                      GlobalObjectKey(_chips[index]).currentContext!,
                      alignment: 0.5,
                    );
                  });
            },
            separatorBuilder: (final BuildContext context, final int index) =>
                const SizedBox(width: 12),
          ),
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () => Navigator.pushReplacement(
          context,
          MaterialPageRoute(
            builder: (_) => const MyHomePage(
              title: 'Flutter Demo Home Page',
            ),
          ),
        ),
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ),
    );
  }
}

Flutter doctor summary:
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.7.10, on macOS 13.2.1 22D68 darwin-arm64, locale ru-RU)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
[✓] Xcode - develop for iOS and macOS (Xcode 14.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.1)
[✓] IntelliJ IDEA Ultimate Edition (version 2023.1)
[✓] IntelliJ IDEA Community Edition (version 2022.2)
[✓] VS Code (version 1.74.3)
[✓] Connected device (3 available)
[✓] HTTP Host Availability

• No issues found!

@abigotado
does it occur w/o GlobalObjectKey?

@abigotado I think this is caused by using the same _chips object array. You can try using ObjectKey instead (I tried and see it works).
It seems your question is best suited for support channels like StackOverflow, so please ask the same there for solutions if any.
Closing for now. If you disagree, please write in the comments and I will reopen it.

Well, the solution is to use GlobalKey for the screen itself too.
https://stackoverflow.com/a/75990230/17209451


Support ESP32 as Bluetooth controller

Describe the bug
I want to build the samples/bluetooth/peripheral_hr as a host only with an extern esp32 as a controller.
For this I use the nucleo_f411re board.
*** Booting Zephyr OS build zephyr-v2.4.0  ***
ASSERTION FAIL [err == 0] @ WEST_TOPDIR/zephyr/subsys/bluetooth/host/hci_core.c:375
        k_sem_take failed with err -11
[00:00:10.014,000] <err> os: r0/a1:  0x00000003  r1/a2:  0x0000000a  r2/a3:  0xffffffbf
[00:00:10.014,000] <err> os: r3/a4:  0x08001761 r12/ip:  0x00000000 r14/lr:  0x080037d9
[00:00:10.014,000] <err> os:  xpsr:  0x61000000
[00:00:10.014,000] <err> os: Faulting instruction address (r15/pc): 0x080037e4
[00:00:10.014,000] <err> os: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
[00:00:10.014,000] <err> os: Current thread: 0x20000da8 (unknown)
[00:00:10.059,000] <err> os: Halting system

To Reproduce
I add hw-flow-control to the dts, the PINMUX functions for RTS/CTS pins of the UART_1 and set the configuration BT_UART_ON_DEV_NAME="UART_1".
&usart1 {
	current-speed = <115200>;
	status = "okay";
	hw-flow-control;
};

static const struct pin_config pinconf[] = {
#if DT_NODE_HAS_STATUS(DT_NODELABEL(usart1), okay) && CONFIG_SERIAL
	{STM32_PIN_PB6, STM32F4_PINMUX_FUNC_PB6_USART1_TX},
	{STM32_PIN_PB7, STM32F4_PINMUX_FUNC_PB7_USART1_RX},
	{STM32_PIN_PA11, STM32F4_PINMUX_FUNC_PA11_USART1_CTS},
	{STM32_PIN_PA12, STM32F4_PINMUX_FUNC_PA12_USART1_RTS},
#endif

Steps to reproduce the behavior:

cd samples/bluetooth/peripheral_hr
west build -b nucle_f411re
west flash
See error

Expected behavior
I expected that I can use Bluetooth specific APIs in my application if I connect an external Bluetooth controller, like the esp32.
Impact
I can't implement my application, because I want to use Zephyr, instead of the esp-idf for the hardware independent.
Logs and console output
Here is logic analyzer trace:

I see that there is a communication, but I don't know why the RTS/CTS pins don't change the state.
Environment (please complete the following information):

OS: linux
zephyr 2.4.0
arm-none-eabi-gcc v10.2
board: nucleo_f411re
bluetooth controller: esp32 with a hci_uart interface

Additional context
On the ESP32 run this sample which make a uart_hci interface available. This works fine with a raspberry pi 2b+.
I used here the BlueZ and attach the device via: btattach -B /dev/ttyAMA0 -P h4 -S 115200

@Cimex97 Hard to reproduce w/o the matching HW. Can you enable CONFIG_BT_DEBUG_HCI_DRIVER ?

Here is the output with CONFIG_BT_DEBUG_HCI_DRIVER=y
*** Booting Zephyr OS build zephyr-v2.4.0  ***
[00:00:00.004,000] <dbg> bt_driver.h4_open:
[00:00:00.004,000] <dbg> bt_driver.rx_thread: started
[00:00:00.004,000] <dbg> bt_driver.rx_thread: rx.buf 0x00000000
[00:00:00.004,000] <dbg> bt_driver.h4_send: buf 0x20003370 type 0 len 3
ASSERTION FAIL [err == 0] @ WEST_TOPDIR/zephyr/subsys/bluetooth/host/hci_core.c:375
        k_sem_take failed with err -11
[00:00:10.014,000] <err> os: r0/a1:  0x00000003  r1/a2:  0x0000000a  r2/a3:  0xffffffbf
[00:00:10.014,000] <err> os: r3/a4:  0x08001761 r12/ip:  0x00000000 r14/lr:  0x080039d1
[00:00:10.014,000] <err> os:  xpsr:  0x61000000
[00:00:10.014,000] <err> os: Faulting instruction address (r15/pc): 0x080039dc
[00:00:10.014,000] <err> os: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
[00:00:10.014,000] <err> os: Current thread: 0x20000da8 (unknown)
[00:00:10.059,000] <err> os: Halting system

@Cimex97 genuine question: ESP32 is supported on Zephyr as a WIFI controller, but is that supposed to work in BLE mode ?

@Cimex97 genuine question: ESP32 is supported on Zephyr as a WIFI controller, but is that supposed to work in BLE mode ?

@jhedberg maybe you can answer this question?

Zephyr don't support the ESP32 with BLE mode. This wrote @carlescufi in this issue.
But in my case, zephyr doesn't run on the ESP32.

@Cimex97 ok, so we can close this issue then, right ?

Maybe not. I want to use a nucleof411re board as a Bluetooth host and the ESP32 as a Bluetooth controller.
The ESP32 is connected over UART to the nucleof411re and I want to use the hci_uart interface to have Bluetooth support on this board.

I want to use a nucleof411re board as a Bluetooth host and the ESP32 as a Bluetooth controller.

So maybe it could work, but in that case, since there is no official support for that, this is not a bug but a feature request and might require some development.

Zephyr don't support the ESP32 with BLE mode. This wrote @carlescufi in this issue.
But in my case, zephyr doesn't run on the ESP32.

Is there a build that acts as a controller over UART for the ESP32? Or can you make one? If you can come up with one, then using Zephyr as the Host is trivial and requires only configuration.

I have running this sample on the ESP32 and can it use as a Bluetoothcontroler. This is come from the ESP-IDF.
This works fine when I build a Zephyr sample for the native_posix_64 board. But this doesn't work if I build it for the nucleo_f411re.

@Cimex97, would you mind providing mode information on the sample configurations you're using (native_posix_64 vs nucleo_f411re).
Right now I don't have ESP32, so I can't test and see what does (/not) happen.
Though, I'm comparing uart drivers, and IIC, native_posix_64 doesn't handle "hw-flow-control".
Are you sure hw-flow-control is expected on ESP32 side ? Have you tried running the sample w/o this option ?
I'm sorry for asking these questions, but w/o available HW to reproduce, my ability to support is limited.

I'm sure that the ESP32 need hw-flow-control. This is required for the ESP32 as a UART HCI controller.
I tested it with the samples/bluetooth/peripheral_hr (same as above). I only deactivate BT_HCI_ACL_FLOW_CONTROL=n. Maybe this isn't supported by the ESP32 HCI stack. If it is enabled I get this warning: [00:00:00.300,000] <wrn> bt_hci_core: opcode 0x0c33 status 0x11 and this error Bluetooth init failed (err -5).
This is the configuration of the sample:
CONFIG_BT=y
CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_SMP=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DIS=y
CONFIG_BT_DIS_PNP=n
CONFIG_BT_BAS=y
CONFIG_BT_HRS=y
CONFIG_BT_DEVICE_NAME="Zephyr Heartrate Sensor"
CONFIG_BT_DEVICE_APPEARANCE=833

BT_HCI_ACL_FLOW_CONTROL=n

I build and flash it onto the nucleo_f411re and I got the error message above. If I build it for native_posix_64, then I can start the application with sudo build/zephyr/zephyr.exe --bt-dev=hci0. Here the ESP32 is with a USB <-> UART converter attached to my Linux system.

@Cimex97, please note that BT_HCI_ACL_FLOW_CONTROL, is HCI level flow control, which is different from the RTS/CTS uart level flow control.
What I'm proposing is to have a try w/o the UART level flow control (&usart1 {hw-flow-control;};), as these pins don"t seem to be handled by native_posix driver. This being said, this is a naive try, as native_posix is not a basic uart driver.
Anyway, I should hopefully be able to get an ESP32 this week.

Ok I tried to disable the hw-flow-controll.
This works for the natvie_posix_64. I attached the device with btattach -B /dev/ttyUSB1 -S 115200 -P h4 -N, where -N means no hw-flow-contoll.
On the nucle_f411re I get now a lot of this error messages:
 [00:00:00.009,000] <err> bt_driver: TX interrupt but no pending buffer!
Note: I have to connect the CTS pin from the ESP32 to the RTS-pin. I don't know why.

I have to connect the CTS pin from the ESP32 to the RTS-pin

That's how CTS/RTS works.

On the nucle_f411re I get now a lot of this error messages:
[00:00:00.009,000] <err> bt_driver: TX interrupt but no pending buffer!

Can you try to increase BT_RX_STACK_SIZE, say CONFIG_BT_RX_STACK_SIZE=2048 ?

Than I get this message again:
*** Booting Zephyr OS build zephyr-v2.4.0-586-g4d0750920b52  ***
ASSERTION FAIL [err == 0] @ WEST_TOPDIR/zephyr/subsys/bluetooth/host/hci_core.c:376
        k_sem_take failed with err -11
[00:00:10.015,000] <err> os: r0/a1:  0x00000003  r1/a2:  0x0000000a  r2/a3:  0xffffffbf
[00:00:10.015,000] <err> os: r3/a4:  0x080016c5 r12/ip:  0x00000000 r14/lr:  0x08003839
[00:00:10.015,000] <err> os:  xpsr:  0x61000000
[00:00:10.015,000] <err> os: Faulting instruction address (r15/pc): 0x08003844
[00:00:10.015,000] <err> os: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
[00:00:10.015,000] <err> os: Current thread: 0x20000d80 (unknown)
[00:00:10.060,000] <err> os: Halting system

ok, txs.
I'll really need to get a way to test this myself. I'll get an ESP32 and get back to you.

Hey @erwango,
do you have some new information about this issue?
If I could help you to just ask me :).

Hi @Cimex97, sorry finally I haven't got a chance to find a ESP32 (was a ESP8266, so no bt...). I need to order one.
So I'm actually sorry, but w/o way to reproduce for now, it will be difficult to help more, at  least for my part.

Okay, thank you for your answer.

@Cimex97 not sure it could help, but since it could have an impact on the API behavior, maybe you could have a try to https://github.com/zephyrproject-rtos/zephyr/pull/31192

@Cimex97 not sure it could help, but since it could have an impact on the API behavior, maybe you could have a try to https://github.com/zephyrproject-rtos/zephyr/pull/31192

@Cimex97 would you be able to try this again with the latest master now that #31192 is merged?

@Cimex97 would you be able to try this again with the latest master now that #31192 is merged?

Hey @carlescufi,
I will do it in a few weeks.

Hey @carlescufi,
I will do it in a few weeks.

Hey, I update now to master. I got the same error.

@Cimex97 that failing k_sem_take you are getting is due to the controller not replying to a command. As to why it is not responding, the most likely reason is a UART driver problem. Could you post a full logic analyzer trace of the whole exchange, and, if possible, a transcription of the trace as bytes going to the controller and coming from the controller?

I traced it with the logic analyzer and realized that the ESP32 does not answer. After I enable the hw-flow-control and set the CTS/RTS pins in the device tree it just works. Many thanks for your support.
pinctrl-0 = <&usart1_tx_pb6 &usart1_rx_pb7 &usart1_rts_pa12 &usart1_cts_pa11>;

Note: I have to disable BT_HCI_ACL_FLOW_CONTROL. Maybe the ESP32 does not support that.

@Cimex97 Thanks for this news! Let me know if I can close the issue.

@Cimex97 I think flow control is supported by ESP32. However there are 2 issues that prevents it from working:

The ACL MTU without header must be >= 1017 (i.e. ACL packet length must >= 1021). This is a known quirk as noted by IDF devs here and easily worked around by just increase the L2CAP MTU in Zephyr.
The SCO MTU and count probably needs to be non-zero. I wasn't able to verify this because for some very weird reason my ESP32 fails to even respond to Reset command (returns error code 0x80) when I set dummy values for hbs->sco_mtu and hbs->sco_pkts in set_flow_control in Zephyr (remind you that I run Zephyr on a separate processor and all requests are flowing through UART so it shouldn't affect HCI in any way :confused: ).

With these 2 solved we probably could get the flow control working but I still have no idea what's wrong with case 2.

Ah, I see. Thank you. ESP's implementation has a bug which bothers me, so I'm gonna search further. I thought Zephyr can also work as a HCI controller firmware :)

Can it (by now)? I'm sorta lost about whether I can use an ESP32 on linux using basic HCI or not by today.

You don't need zephyr, you can just use esp's hci_uart_sample (or similar,
its in their repo). I could fine-tune it and proved to work for quite a
long time already.
On Sun, 11 Dec 2022, 06:58 Mirko Vogt, @.***> wrote:

Ah, I see. Thank you. ESP's implementation has a bug which bothers me, so
I'm gonna search further. I thought Zephyr can also work as a HCI
controller firmware :)
Can it (by now)? I'm sorta lost about whether I can use an ESP32 on linux
using basic HCI or not by today.
—
Reply to this email directly, view it on GitHub
https://github.com/zephyrproject-rtos/zephyr/issues/28872#issuecomment-1345442271,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACZKPOHFHM24DWFEONPSRDDWMU7OJANCNFSM4SBYB6SA
.
You are receiving this because you commented.Message ID:
@.***>

I don't know, I'm using it with full featured UART to SPI chips, as HCI
needs proper CTS, RTS, and of course RX and TX and GND.
On Sun, 11 Dec 2022, 14:53 Mirko Vogt, @.***> wrote:

You don't need zephyr, you can just use esp's hci_uart_sample (or similar,
its in their repo). I could fine-tune it and proved to work for quite a
long time already.
Interesting, thanks!
Does it work with a normal cheap node-mcu pcd and their CH9102X UART<->USB
bridges? Asking, because I read that RTS/CTS is a necessity for attaching
it as UART driven BLE-HCI peripheral.
—
Reply to this email directly, view it on GitHub
https://github.com/zephyrproject-rtos/zephyr/issues/28872#issuecomment-1345520511,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACZKPOF5Z4KRY3TFAMYKQ4DWMWXCJANCNFSM4SBYB6SA
.
You are receiving this because you commented.Message ID:
@.***>


Best practice on many HOC

Hello! I have a question: what about many HOC`s? How i need use it: with stateful component or lifecycle and etc.?
for example:
const enhance = compose(
  defaultProps(listDefaultProps),
  setPropTypes(ListPropTypes),
  mapProps(({ listKey, ...props }) =>
    ({ listKey: props.isAlbums ? ALBUMS_LIST_KEY : listKey, ...props })),
  connect(getPropsNow, toActions, toMerge),
  onlyUpdateForKeys(['listKey', 'store', 'resource', 'exist']),
  lifecycle({
    componentDidMount() {
      this.props.init();
    },
    componentWillReceiveProps(nextProps) {
      const { listKey, exist, init } = nextProps;
      if (this.props.listKey !== listKey && !exist) init();
    },
    componentWillUnmount() {
      const { clear, removeState } = this.props;
      if (clear) removeState();
    },
  }),
  branch(({ exist }) => !exist, renderNothing),
);

Any advice? Mistakes in that code?
Thank you.

@Dictory you can override how connect() compare props and state. Please see react-redux doc for how to do it.
@JunyuanZheng I'm not familiar with TypeScript, but I think you can define the type for each HOC like how we do with Flow in this file.

I'm going to close this issue now because there is not too much we can do here. Please feel free to discuss here or re-open it if you have further thought.


Observable.EveryUpdate fails to capture GetKeyDown events from keyboard

The observable below will not emit anything when the space bar is pressed.
Observable.EveryUpdate().Where(_ => Input.GetKeyDown(KeyCode.Space))
Meanwhile using an Input.GetKey() predicate does capture the key being held down as expected.
Input.anyKeyDown predicate captures mouse events, but again fails to capture keyboard key presses.
Any idea what's going on here?

Observable
                .EveryUpdate()
                .Where(_ => Input.GetKeyDown(KeyCode.Space))
                .Subscribe(_ => Debug.Log("Aaaa"));

This code is working for me in UniRx 7.1.0

I realized we were on 6.2.2. Will test later with the newest version.

Seeing this on 7.1.0

For me disabling enter play mode options worked

For me disabling enter play mode options worked

Can confirm the bug and the fix, that would sadly mean that UniRx is not supporting play mode options atm


⚠ Bofrost.de has degraded performance

In fbfc1bc, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1417 ms

Resolved: Bofrost.de performance has improved in a2aab8f after 7 minutes.


Explicitly document Python overhead in JAX computations.

Often newcomers get confused by JAX's relative performance to NumPy on CPU when using smaller arrays even after using jit.
AFAIK most of this can be explained away by the overhead of dispatching JAX computations from Python which does non-trivial amount of work even when jitted. Currently, description of this extra work is scattered across the docs and some is only apparent after exploring the source code or xprof traces. Would be great if it were succinctly listed somewhere, perhaps in the sharp bits.

I made a simple comparison (on light workload) with PyTorch, it is 2 order of magnitude slower.
https://colab.research.google.com/drive/1nwjiNsKDBWsbLskEMnB7lMRpUky3UjQs?usp=sharing

@cloudhan Note that if you apply jax.jit to your train_step function, jax is faster than PyTorch (on CPU in Colab).
In general JAX is much more optimized for the case where you jit larger blocks of code.

@hawkinsp that improvement is surprising. I updated that notebook.

There's some better documentation on this these days: https://jax.readthedocs.io/en/latest/faq.html#is-jax-faster-than-numpy


Incorrect parentage shown in AmiGO

Hello,
GO:0034061 DNA polymerase activity  and GO:0003964 RNA-directed DNA polymerase activity look like direct children of 'catalytic activity' in AmiGO, but they are not. Why the discrepancy ?

Thanks, Pascale

We have a bug, only spotted today, it might be related?
https://www.pombase.org/browse-curation/fission-yeast-go-slim-terms
Kim hasn't changed our code locally and we are sure that it was working previously. I wondered if it is anything to do with relationship changes...

When I look at the editors' version of the ontology in Protege, I see that DNA polymerase activity is asserted to be a catalytic activity, acting on  DNA and RNA-directed DNA polymerase activity is an asserted child of DNA polymerase activity.
This is also reflected in the snapshot files:
go-basic.obo
[Term]
id: GO:0003964
name: RNA-directed DNA polymerase activity
namespace: molecular_function
def: "Catalysis of the reaction: deoxynucleoside triphosphate + DNA(n) = diphosphate + DNA(n+1). Catalyzes RNA-template-directed extension of the 3'- end of a DNA strand by one deoxynucleotide at a time." [EC:<IP_ADDRESS>]
synonym: "deoxynucleoside-triphosphate:DNA deoxynucleotidyltransferase (RNA-directed) activity" EXACT [EC:<IP_ADDRESS>]
synonym: "DNA nucleotidyltransferase (RNA-directed) activity" EXACT [EC:<IP_ADDRESS>]
synonym: "reverse transcriptase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "revertase activity" RELATED [EC:<IP_ADDRESS>]
synonym: "RNA revertase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RNA-dependent deoxyribonucleate nucleotidyltransferase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RNA-dependent DNA polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RNA-directed DNA polymerase, group II intron encoded" NARROW []
synonym: "RNA-directed DNA polymerase, transposon encoded" NARROW []
synonym: "RNA-instructed DNA polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RT" RELATED [EC:<IP_ADDRESS>]
xref: EC:<IP_ADDRESS>
xref: MetaCyc:RNA-DIRECTED-DNA-POLYMERASE-RXN
xref: reactome:R-HSA-164504 "Synthesis of minus strand strong stop DNA (-sssDNA)"
xref: reactome:R-HSA-164520 "Minus strand DNA synthesis resumes"
is_a: GO:0034061 ! DNA polymerase activity
[Term]
id: GO:0034061
name: DNA polymerase activity
namespace: molecular_function
def: "Catalysis of the reaction: deoxynucleoside triphosphate + DNA(n) = diphosphate + DNA(n+1); the synthesis of DNA from deoxyribonucleotide triphosphates in the presence of a nucleic acid template and a 3'hydroxyl group." [EC:<IP_ADDRESS>, GOC:mah]
synonym: "deoxynucleate polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "deoxyribonucleate nucleotidyltransferase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "deoxyribonucleic acid polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "deoxyribonucleic polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "DNA nucleotidyltransferase activity" EXACT [EC:<IP_ADDRESS>]
is_a: GO:0016779 ! nucleotidyltransferase activity
is_a: GO:0140097 ! catalytic activity, acting on DNA
This is in today's snapshot of go.obo
[Term]
id: GO:0003964
name: RNA-directed DNA polymerase activity
namespace: molecular_function
def: "Catalysis of the reaction: deoxynucleoside triphosphate + DNA(n) = diphosphate + DNA(n+1). Catalyzes RNA-template-directed extension of the 3'- end of a DNA strand by one deoxynucleotide at a time." [EC:<IP_ADDRESS>]
synonym: "deoxynucleoside-triphosphate:DNA deoxynucleotidyltransferase (RNA-directed) activity" EXACT [EC:<IP_ADDRESS>]
synonym: "DNA nucleotidyltransferase (RNA-directed) activity" EXACT [EC:<IP_ADDRESS>]
synonym: "reverse transcriptase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "revertase activity" RELATED [EC:<IP_ADDRESS>]
synonym: "RNA revertase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RNA-dependent deoxyribonucleate nucleotidyltransferase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RNA-dependent DNA polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RNA-directed DNA polymerase, group II intron encoded" NARROW []
synonym: "RNA-directed DNA polymerase, transposon encoded" NARROW []
synonym: "RNA-instructed DNA polymerase activity" EXACT [EC:<IP_ADDRESS>]
synonym: "RT" RELATED [EC:<IP_ADDRESS>]
xref: EC:<IP_ADDRESS>
xref: MetaCyc:RNA-DIRECTED-DNA-POLYMERASE-RXN
xref: reactome:R-HSA-164504 "Synthesis of minus strand strong stop DNA (-sssDNA)"
xref: reactome:R-HSA-164520 "Minus strand DNA synthesis resumes"
is_a: GO:0034061 ! DNA polymerase activity
relationship: part_of GO:0006278 ! RNA-dependent DNA biosynthetic process
[Term]
id: GO:0034061
name: DNA polymerase activity
namespace: molecular_function
def: "Catalysis of the reaction: deoxynucleoside triphosphate + DNA(n) = diphosphate + DNA(n+1); the synthesis of DNA from deoxyribonucleotide triphosphates in the presence of a nucleic acid template and a 3'hydroxyl group."  DNA biosynthetic process
So at the source file level, this all looks ok to me.

OK our problem was a different problem...

Other possible expressions:
https://github.com/geneontology/amigo/issues/525

Is #525 the same thing ?
In #525 you talk about recent changes in the ontology. I dont think the two terms here were edited recently.


Compatible Version Numbers

What content needs to be created or modified?
A table or listing of compatible versions of the component services
Describe the solution you'd like
A table with references to the DAPR runtime version and the min/max versions of the related services that are compatible or not compatible (ie. Redis Cache username supported in 6 and above, and Redis streams not supported before 5.0)
Where should the new material be placed?
At the bottom of the page. It's not super pertinent for every case, but it would be good to know which versions of the relevant software is compatible for upgrading/downgrading.
The associated pull request from dapr/dapr, dapr/components-contrib, or other Dapr code repos
N/A
Additional context
This came about with my company looking to upgrade Redis from version 5 to 6. There's an assumption that it should be compatible, but I don't know that for certain without digging into specific tabs on the page to know what is and isn't compatible at certain versions. The Redis state store page doesn't give me a minimum version unless I look at the Kubernetes tab where it says to install a minimum version of 5, but there's no mention of max version or if the latest versions are or are not compatible. If it were a table on the bottom of the page, it would be a link in the page navigation to jump or link directly to and be easier to present to infrastructure/architecture.

@mplover - I see. So you are asking the SDK version supported in each of the components. Seems fair, but is hard to update for each component and this only applies to certain components, for example public cloud services always remain current. Did you have another example other than Redis?

Exactly. I assume this would just be a task of going through each sdk used for each component across the various versions of dapr and interpreting that information into a table.


Add Netwide Assembler

NASM
The Netwide Assembler (NASM) is an assembler and disassembler for the Intel x86 architecture. It can be used to write 16-bit, 32-bit (IA-32) and 64-bit (x86-64) programs. It is considered one of the most popular assemblers for Linux.
Wikipedia Article
Logo

Added in 9ef3264351d651401a679e2041980dff3ae7b128
Thanks


Fix segfault in example code

This change fixes a segfault which occurs in the validation example code. When the validated struct contains no errors cv.validator.Struct(i) returns nil. Calling Error() on this nil return value was causing the segfault.

Closing this as duplicate of #182


Support username patterns as allowed users

Fixes #100
It is possible to define a regular expression (RE) to configure the allowed users. The idea is that RE to be the most strict as possible.
A new environment variable is expected to be set ALLOWED_USER_PATTERN which is a regular expression. The new variable is only valid if ALLOWED_USERS is not set.

Coverage increased (+0.07%) to 92.857% when pulling e3ac2077b3e07a38795a98089fc63cd16eb37ce8 on rafaelcaricio:support-username-pattern into c4f43932c2d568637ffd8e89b0e317e40f238acd on zalando:release1.9.

Coverage increased (+0.07%) to 92.857% when pulling 9978a17e25154a3180180023d1407b2d55c65c0f on rafaelcaricio:support-username-pattern into c4f43932c2d568637ffd8e89b0e317e40f238acd on zalando:release1.9.

Just talked to @dryewo and the username does not change so this PR is optional.

It can still be hard for the users to find the right username to put in the configuration so I would say this is still an useful feature. :+1:


How to solve this problem?

INFO:root:preprocess 12720 2425 (12720, 2425)
INFO:root:done preprocessing...
True
INFO:root:getting spot network...
INFO:root:initializing labels...
Traceback (most recent call last):
File "run_STARCH.py", line 34, in
operator = STARCH(i,n_clusters=n_clusters,num_states=3,normal_spots=normal_spots,beta_spots = beta_spot,nthreads=nthreads,gene_mapping_file_name=gene_mapping_file_name)
File "/beegfs/home/msq/St/CNV/STARCH-master/STARCH.py", line 133, in init
self.initialize_labels()
File "/beegfs/home/msq/St/CNV/STARCH-master/STARCH.py", line 478, in initialize_labels
km = KMeans(n_clusters=self.n_clusters).fit(dat.T)
File "/beegfs/home/msq/miniconda3/envs/python2/lib/python2.7/site-packages/sklearn/cluster/k_means_.py", line 974, in fit
return_n_iter=True)
File "/beegfs/home/msq/miniconda3/envs/python2/lib/python2.7/site-packages/sklearn/cluster/k_means_.py", line 314, in k_means
order=order, copy=copy_x)
File "/beegfs/home/msq/miniconda3/envs/python2/lib/python2.7/site-packages/sklearn/utils/validation.py", line 573, in check_array
allow_nan=force_all_finite == 'allow-nan')
File "/beegfs/home/msq/miniconda3/envs/python2/lib/python2.7/site-packages/sklearn/utils/validation.py", line 56, in _assert_all_finite
raise ValueError(msg_err.format(type_err, X.dtype))
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

I use the 10x spatial result
The coed : runSTARCH.py -i ../cnv/ -t 20 --output yy --n_clusters 3 --outdir ./test

Also could you provide the more details explanation of results


Consolidate job handling in a new module, parsl.jobs

Description
Prior to this, a lot of job handling was tangled into the parsl.dataflow module.
There's a growing desire in Globus Compute to use the job handling parts of Parsl as a user-exposed feature, distinct from the data flow parts of Parsl.
This PR supports that new use case by creating a new module, parsl.jobs, for job-related code to live in, to help with ongoing work relaxing the previous requirement that a Dataflow Kernel would always exist to manage parsl components.
This PR only moves code around - it should not cause any change in functionality.
This is a breaking change for any externally implemented providers - they would need the same (hopefully superficial) import fixups that this PR does to providers in parsl/providers/
(this is probably a breaking change for Globus Compute making out-of-scope use of Parsl internals, but ... that's out of scope: this PR is part of making that sort of stuff exposed to users properly)
Type of change

Code maintentance/cleanup

re if TYPE_CHECKING: that conditional import breaks with any runtime typechecking (see typeguard, especially version 4 onwards which is much more aggressive about what it manages to typecheck than previous versions). So although it might not break in this case right now, it's an idiom I'd like to avoid. https://peps.python.org/pep-0649/ discusses that more.

most of the other comments seem reasonable for fiddling with in another PR - I'll probably not get round to that though


sys.stderr does not always have an attribute isatty()

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.06.04. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x ] I've verified and I assure that I'm running youtube-dl 2018.06.04

Before submitting an issue make sure you have:

[ x] At least skimmed through the README, most notably the FAQ and BUGS sections
[ x] Searched the bugtracker for similar issues including closed ones
[x ] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[ x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Description of your issue, suggested solution and other information
Traceback (most recent call last):
  File "/home/mary/code/little-mary/src/puker.py", line 133, in basic_res
    d2 = ydl.extract_info(url)
  File "/home/mary/miniconda3/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/home/mary/miniconda3/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 500, in extract
    ie_result = self._real_extract(url)
  File "/home/mary/miniconda3/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1545, in _real_extract
    ytplayer_config = self._get_ytplayer_config(video_id, video_webpage)
  File "/home/mary/miniconda3/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1256, in _get_ytplayer_config
    patterns, webpage, 'ytplayer.config', default=None)
  File "/home/mary/miniconda3/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 866, in _search_regex
    if not self._downloader.params.get('no_color') and compat_os_name != 'nt' and sys.stderr.isatty():
AttributeError: '_StdoutProxy' object has no attribute 'isatty'

I even have the quiet mode activated. You should add something like hasattr(sys.stdout, "isatty") there.
BTW, this problem arises only when I run the script via iPython in an SSH session. I guess the SSH terminal is 'weak'? If so, can't I just somehow make it fake a more powerful terminal?

I tried the following workaround but it didn't work:
class PseudoTTY(object):
    def __init__(self, underlying):
        self.__underlying = underlying
    def __getattr__(self, name):
        return getattr(self.__underlying, name)
    def isatty(self):
        return True

sys.stdout = PseudoTTY(sys.stdout)
sys.stderr = PseudoTTY(sys.stderr)

When I call sys.stderr.isatty(), it returns True, but it seems youtube-dl is somehow bypassing my intervention.

Something in your setup patches sys.stderr with nonconforming implementation lacking isatty. Googling for _StdoutProxy brings to prompt toolkit.

Yeah, I know something does that. The problem is youtube-dl should work
with it. It’s not like it’s necessary when I am invoking youtube-dl from
within Python in the quiet mode. 😐
And it only takes a hasattr check anyways.
On Fri, Jun 8, 2018 at 11:26 PM Sergey M<EMAIL_ADDRESS>wrote:

Closed #16659 https://github.com/rg3/youtube-dl/issues/16659.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/16659#event-1671577690, or mute
the thread
https://github.com/notifications/unsubscribe-auth/Aii--qYiUgCKwluQ7qWuxiv29kuFuPbNks5t6sjXgaJpZM4UdIXr
.

Quoted from Python official documentation:

stdout and stderr needn’t be built-in file objects: any object is acceptable as long as it has a write() method that takes a string argument.

This buf effects some advanced GUI programs that redirect sys.stdout to somewhere else. The code really should check hasattr. Or, at least, have a try~except statement.


event logs not saved if reboot occurs immediately

Need to flush the write after the fclose() for each log when created

we are not doing this anymore.


DEV-890: Add cheqd cli identity docs

I will take a closer look at this for readability and clarity, but please make sure that Markdown linter passes for the edited files (you can ignore the other Markdown errors)

Converting to draft while I change menu structure

@ankurdotb  - just trying to understand the status of this PR. I can see 3 Lint checks are failing however what else is pending before closing this?

@Tweeddalex Has this been accounted for in identity-docs repo?

Yes, since this was a completely new identity document added in the pull request, I've added it directly into the identity docs repo


Uses GitHub Actions to test pinot-servicemanager on PR and merge

This uses GitHub Actions to test the pinot-servicemanager image so that
maintainers have a harder time breaking it by accident. For example, a
change that crashes the image will fail this test.

You can see the status here now, as it is integrated directly into GitHub

This is great, @adriancole ! :)

Have we tried implementing the in CircleCI? At a glance, I'd think we'd only need to add a the actual compose/verify commands to the existing build. Separate from the broader discussion of CircleCI vs GH Actions as a CI provider, I think using multiple providers concurrently adds complexity unless there's a good justification.

Have we tried implementing the in CircleCI? At a glance, I'd think we'd only need to add a the actual compose/verify commands to the existing build. Separate from the broader discussion of CircleCI vs GH Actions as a CI provider, I think using multiple providers concurrently adds complexity unless there's a good justification.

I gave it a go - it drops in almost identically (the only difference being that our plugins are built for circleci, so couldn't discover the branch name in GH actions and defaulted to test):
https://github.com/hypertrace/pinot/pull/35/files

Either way seems fine to me as long as both plugins have the same support (like both are official or both have a big community).

Either way seems fine to me as long as both plugins have the same support (like both are official or both have a big community).

There's no additional plugins being used in the circleci verify command (compared to our existing build which is still running in either PR) - just vanilla docker-compose and CCI's built in run command.
On the other hand, the GH version is bringing in multiple actions (not sure if gradle or docker ones were well supported, didn't verify). My bigger hesitation with it though is both maintaining two sets of CI scaffolding and introducing extra tools for developers to understand unless there's a clear benefit.

I agree that it's better to stick to one CI provider. If we eventually make the cut to GH for valid reasons, we should move all CCI stuff to GH actions as well.

I think there is enough interest on people working on hypertrace ecosystem on trying github actions. This check is easy and small enough (and also @aaron-steinfeld showed it is very easy to port to circle CI) that would be a perfect candidate to try github actions in open source projects and hypertrace ecosystem in general. I believe we should give it a try.

I agree with @jcchavezs ! This is an investment but this will help us in long term. So we should switch to GitHub actions for Hypertrace atleast!

I think there is enough interest on people working on hypertrace ecosystem on trying github actions. This check is easy and small enough (and also @aaron-steinfeld showed it is very easy to port to circle CI) that would be a perfect candidate to try github actions in open source projects and hypertrace ecosystem in general. I believe we should give it a try.

I do generally want to try out GH actions, my concern is that we should treat that as a first class effort. That is, decide if it's something worth investing time in over other priorities, do any research/comparison effort, and then work out how we're going to migrate all workflows. As part of that, piloting a full workflow (rather than a single shell invocation of an existing workflow) certainly makes sense.
The state I don't want to be in is that we piecemeal change things here or there, but don't invest the time to do a proper migration and just end up with a patchwork of CI providers and workflows with serious duplication between them that needs to be kept in sync, and generally increased complexity.

2p on this one: Most things have glitches you'll never know unless you try. There's no huge investment to trying, unless the build itself is strange and needs a lot of custom things to work. Trying removes a lot of hypothetical from technical decisions, and makes us more informed.
Now that this is merged, we can watch this (and migrate this to the ingester repo once other things do). This can happen alongside circleci and we can see how both behave. It is usually the case that CI providers trade off being bad at something. We can learn.


Applicants universities mapping

Describe the task
We need to link applicants with universities so that we can add filters to the application listing based on university. These filters will be helpful for evaluation and campaigning.
Expected behavior

Seed the current distinct colleges we have in the production database into universities table.
Add university_id into applicants table
Write a script that matches university name from applicants table and assigns university_id
We will keep the current college field as it is. For new applications, there should be an option in the UI for admins to select university from dropdown.

Closing this issue, since it is completed and deployed on the production


settings in getting-started documentation should have descriptions for sub-settings

String-based settings with multiple options should have their details expanded in the settings documentation

This is complete, they show up as:


forward text from text nodes as prop

Currently there's no easy way to get the text of a text node from the renderers:
<PayloadLexicalReactRenderer
key={node.id}
content={node}
elementRenderers={{
    ...defaultElementRenderers,
    heading: (props) => {
        // props.children is nested with fragments and there's no easy way to get text from it
        return (
            <CustomHeading
                id={`#${props.text}`} // intended usecase
                as={props.tag}
             >
                 {props.children}
            </CustomHeading>
        );
    },
}}
/>

The intended usecase is for adding the text as ids to the heading element.

@schardev there is no easy fix for this, as headings can always contain various marks. You would need to iterate through the children and create a slug from the texts, but I don't think this is vary practical. I would consider using a custom block with a slug and title field to accomplish this.


Program icon would be a lot better with white background

This is cosmetic
On my black  windows 10 taskbar  the icon is almost invisible ?

I'll take a look. Thanks for your suggestions.


Fix issues caused by updating to Factorio 2.0

I've updated all the errors I could find.

Fixed differences in Factorio 2.0 modding API
Updated to a 2.0 version of stdlib and flib
Copy pasted in an old version of flib.gui so I didn't have to rewrite all the gui logic as I didn't understand it.
Other fixes too, happy to elaborate. Works for what I'm doing.

Thanks! I swear I lost all my Factorio modding/lua skills after 3 years of not working with it. So if you can test it out in various situations without crashing then I will believe you and publish it

I just need to fix a little bit more, but working on it!

Fixed a few more issues but now discovered it's totally broken when you load a game. The HUD is out of sync/frozen, clicking a combinator on the ground crashes the game, etc.
I don't know Factorio's event chain/mod system so unsure how to fix. I've hit a brick wall :(

I have a full working version (I use it locally) in #7. Feel free to pick that up.


Added prevent default to formfield price onWheel

PR Checklist
Please check all that apply to this PR using "x":

[X] I have checked that this PR is not a duplicate of an existing PR (open, closed or merged)
[X] I have checked that this PR does not introduce a breaking change
[ ] This PR introduces breaking changes and I have provided a detailed explanation below

PR Type
What kind of change does this PR introduce?

[X] Bugfix
[ ] Feature
[ ] Code style update (formatting)
[ ] Refactoring (no functional changes)
[ ] Documentation changes
[ ] Other - Please describe:

Fixes
Issue Number: https://github.com/lbryio/lbry-desktop/issues/1674
What is the current behavior?
form-field-price number input changes value on scroll when focused
What is the new behavior?
form-field-price no longer changes number value by scroll wheel when focused
Other information
I didnt take either of the suggestions in the PR to use a tooltip with key bind ctrl and mouse wheel or confirmation alert because I found the user experience to be a little odd with that.  However, if it is desired let me know and I will implement that instead.

@DustinRobison, thanks for the PR! We'll get a review on this soon.
Can we show you some appreciation for the contribution?
Also, we're giving away some Hacktoberfest bonuses and goodies for this month. We'll get it all figured out after you shoot us an email after this is reviewed/merged.


govendor list +o returns non-empty

When running govendor list +o on current master, it returns some missing libraries:
  m github.com/jtolds/gls
  m github.com/smartystreets/assertions
  m golang.org/x/image/font
  m golang.org/x/image/font/basicfont
  m golang.org/x/image/math/fixed
  m roughtime.googlesource.com/go/client/monotime
  m roughtime.googlesource.com/go/config
  m roughtime.googlesource.com/go/protocol

Some of them come from our roughtime app, but I don't know if all of them do.
Find out why we have these missing libraries and modify consequently our vendor.json

I can speak for the golang.org/x/image libs. They are for test image servers for webapp. I can supply the proper versions

Patch attached.
add-imgserver-test-vendor-libs.patch.txt

I don't really trust this govendor tool on that. For example, govendor list +u shows a couple packages which are apparently unused. However, removing them with govendor remove +u and then trying to compile the apps fails because of missing dependencies.

The entries regarding roughtime were real. I have created PR #60 fixing roughtime (this vendor plus other small things). After that PR we only have this govendor list +o :
  m github.com/jtolds/gls
  m github.com/smartystreets/assertions

We have to decide if we ignore them and close this PR, or do something.

The latest commits in PR #60 solve this issue. Now govendor list +o is empty.


Various Cron Job bugs

[x] The cron schedule cannot be empty, but is allowed to be it. This results in an error when you deploy the Cron job
[x] The cron schedule * * * * * is displayed as triggers constantly. It should be Triggers every minute instead.
[x] Deleting the last cron job doesn't remove it from the list until you refresh the page
[x] Deleting a cron job should be a queued change as well
[x] Ordering of job execution should be newest execution first

@MatiasFrank - This is now fixed, will be out in next release


Rewrite AutoGen Studio Database Layer to Use SQLModel ORM

Why are these changes needed?
This PR rewrites the database module for autogenstudio using   use SQLModel (Pydantic + SQLAlchemy). This provides

[x] Entity linking (skills, models, agents and workflows are linked via association tables). This addresses multiple existing issues where changes to skills, models or agents do not propagate to workflows.
[x]  Supports for multiple database backend dialects supported in SQLAlchemy (SQLite, PostgreSQL, MySQL, Oracle, Microsoft SQL Server). The backend database can be specified a --database-uri argument when running the application. For example, autogenstudio ui --database-uri sqlite:///database.sqlite for SQLite and autogenstudio ui --database-uri postgresql://user:password@localhost/dbname for PostgreSQL.
[x] Adds support for LocalCommandLineCodeExecutor and DockerCommandLineCodeExecutor introduced by @jackgerrits and @ekzhu

Note: This PR introduces significant changes to the database schema, incompatible with v< 0.0.56.

TODO

[ ] Add initial tests for the database manager
[ ] Add some database migration logic (e.g., detect incompatibilities and take some action)

Testing
Install autogenstudio from autogenstudio-orm branch. Run autogenstudio ui

Related issue number
#1694  #2128

Checks

[x] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ ] I've made sure all auto checks have passed.

Wow, this is fantastic! Happy to see support for more DBs!

@ma-armenta I have invited you to join as a collaborator to review this issue.

When I start AutoGen studio built from this branch, it shows multiple "Connection error 404 Not Found" in a banner error message.

@victordibia The field "admin_name" is declared twice with the exact same specifications and default value from AgentConfig under datamodel.py. version: autogenstudio 0.0.56rc1

@victordibia The field "admin_name" is declared twice with the exact same specifications and default value from AgentConfig under datamodel.py. version: autogenstudio 0.0.56rc1

Thanks. Fixed.

@victordibia Unable to delete sessions, got
"The Session is linked to another entity and cannot be deleted".
version: 0.0.56rc3

@victordibia Unable to delete sessions, got "The Session is linked to another entity and cannot be deleted". version: 0.0.56rc3

FYI: It violates foreign key constraint "message_session_id_fkey" on table "message"

@victordibia I am building from source to test things out. Using postgresql and got ModuleNotFoundError: No module named 'psycopg2'. Lib is installed, however I have been using psycopg2-binary instead to avoid any compiler or external library requirements, if any, just some thoughts.


Small typo

"Ror example, an Azure public IP resource with"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e38786a5-e931-e614-55a1-2980b8a28087
Version Independent ID: a72b6f97-dd0f-252c-f6d9-f1c517aa9a2f
Content: Management group and subscription organization - Cloud Adoption Framework
Content Source: docs/ready/enterprise-scale/management-group-and-subscription-organization.md
Service: cloud-adoption-framework
Sub-service: ready
GitHub Login: @JefferyMitchell
Microsoft Alias: brblanch

Thanks for reporting - this issue is under review. This is a Microsoft Internal DevOps Bug ID # 494398

This issue has been addressed.  Thank you!


Possibility to edit/use own UI

Hi,
Awesome setup. I just wanted to check if there's a possibility of using this library with my own VC which has a few custom changes instead of the default one that's provided?

Hi @Shoshin23,
It's intended to do this by using AVDocumentScanner directly, but I just checked and that class isn't public.
I'll look into it and expose it to users of this pod.
The idea here is to use the CALayer that is exposed by the scanner and use it in a VC of your own to display a preview.

Thanks! That would make it extremely useful for me. I make apps for people with a visual impairment and WeScan can help my users take better photos because the pre-processing is 👌

This is now available with 2.1.0.
If you have any further questions, feel free to ask 👍


Do we need Developer Mode enabled to execute tests?

Hi,
One of the core concern in adopting WinAppDriver is a pre-requisite on Enabling Developer mode.

With YWinAppDriver, do we still need to enable Developer Mode?
How easy to migrate frameworks and Test Scripts from WinAppDriver to YWinAppDriver?

Thanks,
John

For your first question, It depends on what kind of app you are testing. If you are test app is UWP, likely yes because you have to enable developer mode to install the test app. I don't think YWinAppDriver requires you do this （of course I may have wrong statement here because I always have developer mode enabled in my machine)
It's hard to tell how easy. If you are using a feature YWinAppDriver doesn't have but WinAppDriver have, then it's hard because you have to modify YWinAppDriver and fix it by yourself. If you are just have clicking event, very likely it's an easy task.


Update content for add external key modal

Describe the bug
Update content for add external key modal
Instead of saying “Mobile Key” should say “key”

@ASN-BitHyve
cc: @ben-kaufman

Heading: "Importing External Key"
Subheading: "Keep the other Keeper App ready"
Line beneath the illustration: "To import an external key here, go to the Settings of the key in the other Keeper app, click on Key Details, and scan the QR code presented there."

@AreejKhalid21 please update the text in the image in the description, with the text I have given in the comment above.

Text has been updated on design screen modal
screen link
https://www.figma.com/design/m5eMFTKD391AiJAXaEzgxB/Bitcoin-Keeper?node-id=209-558&t=4VetbErPbaJSPckM-1

Verified this issue on keeper dev app v 1.2.18(408)


GitHub/Lab OAUTH breaking dashboard and now showing as button on login page

Checks

[X] I have checked for existing issues.
[X] I have looked through the FAQ.

Question
I have overwrite for webserver_config as show on docs to use OAUTH with github or gitlab and when I log into the dashboard it directly tries to authenticate with the provider, I don't even get to see the dashboard and logic options?

@hkvia the issue will be that you have AUTH_USER_REGISTRATION_ROLE set to "Public" (which gives no access), but you have not provided any AUTH_ROLES_MAPPING to give the "Viewer", "User" or "Admin" role from a group.
If simply being logged into the OAUTH should give access to the Airflow UI (for your security situation), then update AUTH_USER_REGISTRATION_ROLE to "Viewer", "User" or "Admin".
We should update the  "How to authenticate airflow users with LDAP/OAUTH?" docs to explain AUTH_USER_REGISTRATION_ROLE  needs to be set to "Admin" if AUTH_ROLES_MAPPING is not used.

Is there another way to do this authentication, seems like flask-appbuilder has bugs with open issues that fixes them but no one bothers merging them. Is this literally the only way this can be done?

@thesuperzapper  Thanks for that I've updated my settings. However I think what really fixed it is updating flask-appbuilder to 3.4.3 now Airflow dashboard is showing with button for a given auth method


Redesign Toast component

Description
Redesign Toast component.

@QuentinSaubadu TODO designs with all the possible status (error, info etc..).

Technical question:

Are we changing the lib we use for the toasts or do we keep this one?

Notes:

23/01/2022 - Just saw that the toast component is shown as "coming soon" in the radix docs, might be better for us to wait for the release of the component before starting this task.

Description
Redesign Toast component.

@QuentinSaubadu TODO designs with all the possible status (error, info etc..).

Technical question:

Are we changing the lib we use for the toasts or do we keep this one?

Can I get somewhere a list of all different possible status?

So far in the app we are just using toast for success and error messages

@QuentinSaubadu radix released their toast component so you can create the specs for this one https://www.radix-ui.com/docs/primitives/components/toast


upgrade engine version to 2.2.2, asset.resource.instantiateRenderEntity render issue when app.scene.ambientLight is WHITE

app.scene.ambientLight.set(1, 1, 1);
engine version 1.74.0:

engine version 2.2.2:

would you have a repro for this?
I tested it in this example: https://playcanvas.vercel.app/#/graphics/render-asset
by modifying code - see two changed lines, all it works as expected for me

try https://skfb.ly/p8uRq
or try model without material.diffuseMap?

I tried the listed model, and that still works

By the way, what platform / browser do you see it on? Have you tried different browsers just to limit this being a browser specific issue.

I'm using Google Chrome 130.0.6723.117 (arm64)
on MacBook Pro

model - glb - Original format:

projectmodel-viewer-main branch :
src/viewer.ts -> add this.app.scene.ambientLight.set(1, 1, 1);

in chrome:

in firefox:

please try the project model-viewer, thank you


Update custom layer docs

Change List

Update the description of event system & prop forwarding to match the latest API
Replace outdated examples

Coverage remained the same at 62.388% when pulling 18ecce12549265e0837c43c79c7233ac1fb3a613 on x/custom-layer-docs into be3e4e20dadcddff978ff32f02210aff8c52fcab on master.


[API][Phoning] Fix config-survey

Closed because it's the same fix as https://github.com/EnMarche/en-marche.fr/pull/6782


Remove --insecure-experimental-approve-all-kubelet-csrs-for-group flag

Flag prevents kube-controller-manager to start with k8s master ( 1.14+ ) builds as it was removed.

Fixes:
https://github.com/openvswitch/ovn-kubernetes/issues/534

cc: @alinbalutoiu @aserdean

Thanks a lot for the fix @adelina-t .
Apparently that flag is no op since Kubernetes v1.7.
I also did a sanity check deploying a cluster and everything was fine.
Just a small nit can you update the title to adhere to:
https://github.com/openvswitch/ovn-kubernetes/blob/master/CONTRIBUTING.md#commit-summary
Oh, also can you provide a Signed-off-by for this patch?
https://github.com/openvswitch/ovn-kubernetes/blob/master/CONTRIBUTING.md#developers-certificate-of-origin

/LGTM