Fix issues with object importing and tolerate unpadded empty strings in encrypted documents

Description of the changes

Make object importer more resilient

Correctly copy references to the root object
Deal with deep object graphs in a more stack-safe manner (we now only
recurse within top-level objects, those usually are not that deep)

When decrypting strings and streams using one of PDF's AES-CBC-based crypt filters, if the plaintext is empty after decryption, do not reject it for lack of padding.

See #412.
Checklist
Please go over this checklist to increase the chances of your PR being worked on in a timely manner. Deviations are allowed with proper justification (see previous section).

[x] I have read the project's CoC and contribution guidelines.
[x] I understand and agree to the terms in the Developer Certificate of Origin as applied to this contribution.
[x] All new code in this PR has full test coverage.

For bug fixes (delete if not applicable)

[x] My changes do not affect any public API or CLI semantics.
[x] My PR adds regression tests (i.e. tests that fail if the bug fix is not applied).
[x] All new code in this PR has full test coverage.

@redfast00, you might want to throw your files at this one.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.83%. Comparing base (6177c1f) to head (5928c5b).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #414   +/-   ##
=======================================
  Coverage   98.83%   98.83%
=======================================
  Files         113      113
  Lines       16265    16284   +19
=======================================
+ Hits        16075    16094   +19
  Misses        190      190

Flag
Coverage Δ

unittests
98.83% <100.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


feat(tutorial): complete step 1

Closes #
{{short description}}
Changelog
New

{{new thing}}

Changed

{{change thing}}

Removed

{{removed thing}}

Already moved feature-flags declaration on top of the imports

👍 Looks good - 🎉 You are one step closer to completion of the Carbon React tutorial and earning the digital credential.


find MSRV

i'd like to use this tool to find the minimum supported rust version for my crate. Is that a supported use-case?
This will take a long time to run, as i don't know at what point features i'm relying on were added to the compiler.
cargo bisect seems to default to 2 days at a time, which is fine if you know the regression was fairly recent.
Is this configurable? is there some configuration i can use to make the bisection time delta grow (exponentially) with every iteration?

update: i'm quite a few iterations in now, and it looks like the delta is slowly growing.

I think that this may be the block that you want:
https://github.com/rust-lang/cargo-bisect-rustc/blob/a5f50340105f2983bccef97a933f9d21a1d1d278/src/main.rs#L733-L745

@chrissimpkins cheers! that clears that up.
even at 14 days it's going to be a slow process to go back potentially several years

Good luck!

This is something that I'm generally interested in, so I'll chime in with some thoughts.
When I've done this, I've done the much simpler quickly-hacked script (this version is from memory and may not be exactly right:
for minor in $(seq 42 1); do
  version="1.${minor}.0"
  rustup install "${version}"
  cargo "+${version}" test
end

I then scan the list for the first failure.
However, that version is only partially useful. The problem is that it completely ignores dependencies. They come into play in a few ways:

Many crates incorrectly state their minimum dependency version. For example, they say thing = "1.0" when they actually need 1.4, or they say thing = "1.4" when they would have compiled with 1.0.
Many crates don't document their MSRV or how they change their version number to indicate a MSRV change.
If a dependency does have their own MSRV, then your MSRV probably has to be equal to or newer than that version. Since the MSRV isn't (yet) accessible in a machine-readable format, this means you have to figure this out manually for each dependency (including transitive ones)

In practice, I've taken to finding "milestone releases" and targeting those. For example, Rust 1.15 (proc macros), 1.24 (impl trait), 1.31 (edition 2018), etc.
I then set up CI jobs to ensure that my minimum dependency versions are correct and that I can compile on my MSRV.

@shepmaster thanks for the input.
I'm all for formalising the MSRV as part of a crate's metadata, but then of course as you've pointed out you have to hope that author's aren't abusing it.
The process you've described above is about where I'm at- i've found a milestone release that my project builds against, and then set up CI to ensure that version continues to build my project. If i ever decide i want some shiny new feature (or one of my dependencies does!) i'll make a decision about bumping the MSRV and release at that point.
For a brand new crate finding the minimum (stable) compiler version the first time round wasn't too hard, but it was a bit mandraulic. Would be really nice if a tool like cargo-bisect-rustc could support finding the minimum compiler version, and it doesn't strike me as too wildly far outside what its doing already. Allowing the step size to continue growing in some adaptive fashion would be a big help, for example. An option to target stable compiler versions could be another.

There's cargo-msrv, which - as far as I can tell - covers this specific use case.

There's cargo-msrv, which - as far as I can tell - covers this specific use case.

It sure does! I've been making use of it for the last two years since I posted that comment


use Genome Name (instead of Organism Name) as col header in "new workspace" and "old PATRIC"

After consulting with Maulik and Dustin and Harry, there is agreement that in genome lists throughout the site, the column header should say "Genome Name" and not "Organism Name"  (that is already what is being queried to fill the data).   This needs changes on both the current PATRIC website code and the new Workspace code.   (So this should probably be 2 tickets instead of 1?   I assume the 1st fix can go in as a comment and the 2nd fix can close this issue?)

I just committed the rename of that column in the workspace's genome list.
Dustin
On Mar 19, 2015, at 1:54 PM, rwill1 wrote:

After consulting with Maulik and Dustin and Harry, there is agreement that in genome lists throughout the site, the column header should say "Genome Name" and not "Organism Name" (that is already what is being queried to fill the data). This needs changes on both the current PATRIC website code and the new Workspace code. (So this should probably be 2 tickets instead of 1? I assume the 1st fix can go in as a comment and the 2nd fix can close this issue?)
�
Reply to this email directly or view it on GitHub.

I have change genome list/genome finder as well.


useHaptic doesn’t seems to be working even on bare rn projects.

I tested on my real device, it works fine. 😥


Use standardized CC, AR, NM env vars

Companion to https://github.com/WebAssembly/wasi-libc/pull/275

OK https://github.com/WebAssembly/wasi-libc/pull/275 is merged, so I updated the submodule here as promised. If @sunfishcode you want to approve CI, it should hopefully work.

It looks like the ubuntu and macos builds succeeded, but the windows build is getting an error:

CMake Error at C:/Program Files/CMake/share/cmake-3.23/Modules/CMakeDetermineCCompiler.cmake:49 (message):
Could not find compiler set in environment variable CC:
cl.

@sunfishcode That looks like an existing issue, see the top of main's CI results: https://github.com/WebAssembly/wasi-sdk/runs/5727959507?check_suite_focus=true
It does look like that error is analogously caused by an inconvenience CC set in the ambient environment, but actually CC=cl is explicit in the YAML.
I was going to take a stab at fixing that in a separate PR under the assumption it was the the same sort of thing, but now that I see this is different I am not quite sure what the intent is nor how how to fix it.

@lygstate Do you know if the current CI errors on main are related to your recent PRs? If so, could you take a look?

@lygstate Do you know if the current CI errors on main are related to your recent PRs? If so, could you take a look?

I have create the pr to fix IT， please merge

The CI issues are fixed in other PRs, and this looks right to me and fixes an issue I'm seeing, so I'm going ahead and merging this.


Unable to get Data through Web Api

Please help.
I can get the data with the same code while doing it with WinForm but unable to get any data through Web API.

Thanks in advance.

Sorry, but this is not a very good description of a problem. If you need help with Web API, you should refer to their documentation.


Unable to init, S3 usage and Version Constraints

Hi,
I am wondering if this cannot work without the S3?
What am I missing here, what is it S3 used for?
terraform init

Initializing the backend...

Warning: Version constraints inside provider configuration blocks are deprecated

  on main.tf line 26, in provider "tls":
  26:   version = "~> 2.1"

Terraform 0.13 and earlier allowed provider version constraints inside the
provider configuration block, but that is now deprecated and will be removed
in a future version of Terraform. To silence this warning, move the provider
version constraint into the required_providers block.

(and 2 more similar warnings elsewhere)

Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.

Please see https://www.terraform.io/docs/language/settings/backends/s3.html
for more information about providing credentials.

Error: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Another thing it seems that this repository was deprecated, so thus will be his terraform?
[DEPRECATED] This provider was temporar fork of Telmate/proxmox, use that instead.

This provider is a fork of Telmate/proxmox, with some needed changes to make it available on Terraform Registry. Temporarily this was put on registry as well.

Please use Telmate/proxmox instead.
https://github.com/ondrejsika/terraform-provider-proxmox

@uhlhosting sorry for the late reply. S3 is used for storing the terraform state, which is a one of the ways to do remote backends. You may delete the S3 backend in this block or change to your AWS profile if you would like to have the terraform state in S3 instead of local (by default). Kindly note that I didnt use object locking in which you may find it useful too. One thing I wanna mention as well is that the naming of some files and keys (e.g. inventory file name, key name) are accustomed to my environment. Please feel free to change it to your own favorite naming or enhance it to use more general ones.
Terraform 0.13 has brought about big changes in the provider hierarchical structure and Terraform Registry. As far as I remember I was using 0.12 at that moment (my active development in May 2020). You may try to download and use older Terraform CLI version for it to work. Pull request to make it compatible to newer Terraform verison is also welcomed. I may upgrade this repo to use newer Terraform Registry and Terraform CLI some time later when I am free. (Sorry that this is just my experimental project to learn about Proxmox, Kubernetes, Terraform, Ansible back in 2020, and didn't expect people would really see this or use it for production)

@dy2k there is an increase in automation @ proxmox, and I am sure people would had used your repo if they knew about it. I have already a working deployment automated via Terraform for Proxmox, yet it is based on QEMU Images, will like to build one based on LXC containers. Would be greatly appreciated if you would take some of your time to update the repository. I could even add it into my next blog post about the automation of K8s on Proxmox. So you say I should just try an older terraform for the above errors, will give it a try and give feedback.

The warnings can be safely ignored, I've gotten it running with the same provider versions just this week. I did upgrade to Telmate/proxmox - will put a PR for that soon. The issue here though appears to be that you simply don't have AWS credentials configured for your local. You can either aws configure see here
Or adjust the backend block to use a local state (not recommended) https://www.terraform.io/language/settings/backends/local

Also, to clarify, I am running Terraform v1.1.5 and I don't believe any adjustments were needed.

And @dy2k this project is beautiful don't sell yourself short. I have a few notes (as I said, PR coming soon) but this got me so far with exactly what I was trying to do 😗 👌

I can't wait to dive into trying this one out... Also wanted to mention that if you don't want to use S3 for your remote backend there are other options, including Terraform Cloud (which is free to a certain extent) :)


Package request: makima

Project description

Makima is a daemon for Linux to remap keyboards, mice, controllers and tablets.
Metadata

homepage URL: N/A
source URL: https://github.com/cyber-sushi/makima
license: GPLv3
platforms: linux

Add a :+1: reaction to issues you find important.

I created a pull request following this packaging request https://github.com/NixOS/nixpkgs/pull/297604


Mutli-Tenancy considers port as part of domain name

I found that domain name used to get Site instance includes port number as a result, when port is used in a request (http://some.domain.com:8000/), the line 44 in mezzanine/utils/sites.py:
domain = request.get_host().lower()
gives "some.domain.com:8000" as a result, so your code will use default site instead of "some.domain.com" if your Mezzanine serves several sites.
Next will fix the issue:
domain = request.get_host().lower()
domain = domain.split(':')[0] if ':' in domain else domain

Alternative way is to explicitly consider the port part as a part of a domain name and add this notice to Mezzanine docs.

You could argue either way being correct, but since it's been this way for some time, people may be depending on it, so it's probably not worth changing the behaviour at this point.


[Bug] Controller events do not reset shutdown timer anymore

the system shuts down while playing

Fixed with 3.3.0


Typos in chap 16

p541, figure 16.1: tesselation -> tessellation
p545, line 6: succesful -> successful
p550, section <IP_ADDRESS>: uniformative -> uninformative
p550, equation 16.12. It seems the brackets should be removed, or place the right bracket at the end before the big parenthesis, otherwise the parentheses in the exp() do not match.
Thanks for these great books.

fixed, thanks


Handle importing CSS files

We can implement importing CSS files in JavaScript by rewriting it to a JavaScript script that injects the link HTML tag when the module is imported.

I think it's cool how Symfony handles this, but I don't have the energy to chase this right now.


Feature not visible in APIM

Hi, I do not see the GraphQL option in 'Create New API'. Is this feature released? My resources are in 'West Europe' but i am working out of the Middle East.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3e56a0c4-ddb8-4979-9458-13545a5cc69f
Version Independent ID: fbd73f31-124f-5d54-0497-f507d1461c7d
Content: Import a GraphQL API using the Azure portal
Content Source: articles/api-management/graphql-api.md
Service: api-management
GitHub Login: @dlepow
Microsoft Alias: danlep

@falrnacn can you please share the doc you are refencing so we can assign the issue to the appropriate team? Since this is Azure Docs repo, we need a doc reference link to assign the issue. Thanks.

https://docs.microsoft.com/en-us/azure/api-management/graphql-api

@falrnacn we'll review your feedback and get back to you shortly. Thanks.

@falrnacn - The rollout to different Azure regions is in progress and expected to complete within approximately 1 week. The feature is currently available in some regions, for example East US. Thanks for contacting us.

This feature should be visible in all regions now, so I'll proceed to close this issue. Thanks for contacting us.
#please-close


MethodError when trying to autodiff through a complex matrix

When trying to take the derivative of the qliouville function from the following gist: https://gist.github.com/golanor/05a4ca2ad90a257625325a43da3a789a
The following error appears:
MethodError: no method matching similar(::Tuple{Matrix{ComplexF64}, Matrix{ComplexF64}, Vector{Float64}, Float64})

Can you simplify your code to use Enzyme.jl directly?

Closing for lack of info, feel free to reopen


Consider adding documentation on select[multiple]

The <select> HTML element has an alternative mode that allows for selecting multiple values, also known as a multi-select listbox. This is enabled using the multiple flag. We don't explicitly support using multiple with our Nunjucks macro, however service teams can still enable it through the attributes parameter.
We do not support multiple in our CSS and discourage using it for a few reasons:

Using multi-select is complicated, requiring the user to hold down one or more keyboard keys (⌘ Cmd/⌃ Ctrl or ⇧ Shift) whilst clicking on options. This is difficult to convey and usually not well understood by users.
Multi-select cannot be easily used with keyboard navigation alone, practically requiring a pointer device to use.
Multi-select is not (to my knowledge) well supported by assistive technologies.

We may also want to include information on the size attribute, which we also do not support in Frontend's code, although it doesn't have the same pitfalls as multiple.
Related documentation
Select
Suggestion
A user has suggested adding documentation about select[multiple]; specifically, adding information on why we discourage using it and don't support it: https://github.com/alphagov/govuk-frontend/issues/3317
In most situations, we would probably recommend using a list of checkboxes instead.
Evidence (where applicable)

<select> your poison on 24 Accessibility

👋 I'm that user.
Note that, due to Nunjucks templates only working on Node, we're having to implement our own templates to generate compatible html for this design system. So we can't treat the values that macros accept as the 'spec' for what's supported by the design system, instead we have to work off the html equivalents and reverse-engineer nunjucks templates to infer the behaviour we need to re-implement on our side. And, as you say, attributes can be passed in via the macro anyway, so even that's not a useful interface to us to know what's supported and what is not.
So the html is the 'spec', and there's nothing to indicate that either the rows or multiple attributes are not supported. This could have saved me considerable time trying to understand why they didn't work when I was poking about trying to understand the reason behind #3317.

Note: The default mechanisms for <select multiple> are not accessible to people looking to use input methods that don't include a keyboard.
@calvin-lau-sig7 found a 2019 article by Sarah Higley which includes a section on how multi-select for select elements works.
The article describes the options for how to interact with a <select> element when selecting multiple options. If the article is still accurate today, then a key detail is that that you need to use a keyboard to select multiple options. It cannot be usably done via a mouse-only or single non-dragging pointer touch-only method.
If the only way to make multi-select usable is to involve a keyboard input device, then that means that  fails a WCAG 2.1 AAA criterion: 2.5.6 Concurrent Input Mechanisms
While were not legally obligated to meet WCAG AAA success criteria, this one is pretty important for real-world accessibility. People should be able to interact with government services without requiring a keyboard as the input device.
@CharlotteDowns suggested we might want to add some guidance on why we don't use <select multiple>, and referenced this GitHub issue as a place to have that conversation.


Clean up error when passing in incompatible options

When passing in incompatible options, the logging is messy and contains webpack junk. Clean this up.

This work has been done here.


JMS AMQP 1.0 Source/Sink Kamelets: Replace QPID client with camel-amqp

Motivation
The client qpid dependency does not works in native mode (stacktrace log Caused by: java.lang.ClassNotFoundException: org.apache.qpid.jms.JmsConnectionFactory).
Description
The camel-amqp component is designed for the kamelets use case : supporting the AMQP 1.0 protocol through the JMS QPID Client. So replacing the client dependency by the component makes sense.
Note
Will probably need some backports

Additional note : I don't need the camel-jms component since it is a dependency from camel-amqp. When removing it the validate script fails because it is not present anymore in dependencies.
Should I remove it and make the change to adapt the kamelets (I am guessing a change of names and the apache group in metadatas at least) ?

I think the validation must be reviewed at all also I do think we don't think the qpid-jms-client in the bom, since it's not managed anymore now.

I think the validation must be reviewed at all also I do think we don't think the qpid-jms-client in the bom, since it's not managed anymore now.

Ha you are right, I will remove the QPID client everywhere it needs to be.

@gansheer I'm just curious that

The client qpid dependency does not works in native mode

Is it Quarkus native mode? If it is, I think we need to use org.amqphub.quarkus:quarkus-qpid-jms. see https://github.com/amqphub/quarkus-qpid-jms

@gansheer I'm just curious that

The client qpid dependency does not works in native mode

Is it Quarkus native mode? If it is, I think we need to use org.amqphub.quarkus:quarkus-qpid-jms. see https://github.com/amqphub/quarkus-qpid-jms

Kamelets are not designed to work only with quarkus as runtime, it was like that in the beginning, so I don't think it's the case to add a quarkus dependency directly in the yaml.

@zhfeng I am talking about the library "mvn:org.apache.qpid:qpid-jms-client:1.0.0".
I tested the modified kamelets with camel-k, which means camel-quarkus, and it works well. I don't think I can add a quarkus dependency in the Kamelets :thinking: .

I get it and thanks for your clarifying.


Migrate hosting to GCP from AWS

From @dongbohu:

Daniel, I created a new database and a new virtual machine on Google Cloud Platform to host the connectivity-search API. Please take a look at it:
http://<IP_ADDRESS>/v1/
If it looks okay with you, please register the IP address "<IP_ADDRESS>" with "search-api.het.io". When the registration gets effective, I will install an HTTPS certificate and enable HTTPS. The migration will be done.

Okay I updated the DNS record to be:
search-api 1000 IN A <IP_ADDRESS>

So now you can generate the SSL certifacte?
Does this migration only affect search-api.het.io and not the postures database at search-db.het.io? Are you also planning to migrate the database?

Okay I updated the DNS record to be:
search-api 1000 IN A <IP_ADDRESS>

So now you can generate the SSL certifacte?
Does this migration only affect search-api.het.io and not the postures database at search-db.het.io? Are you also planning to migrate the database?

Thanks @dhimmel. I enabled HTTPS and the new https://search-api.het.io seems to be  alive now.
The new API server actually is using a new database, whose IP address is <IP_ADDRESS>. Please feel free to update seaarch-db.het.io too.

Thanks @dhimmel. I enabled HTTPS and the new https://search-api.het.io seems to be  alive now.
The new API server actually is using a new database, whose IP address is <IP_ADDRESS>. Please feel free to update seaarch-db.het.io too.

Okay updated that DNS record as well, so the records are now:
search-api 1000 IN A <IP_ADDRESS>
search-db 1800 IN A <IP_ADDRESS>

So the old database and instance can go now?

Okay updated that DNS record as well, so the records are now:
search-api 1000 IN A <IP_ADDRESS>
search-db 1800 IN A <IP_ADDRESS>

So the old database and instance can go now?

I have stopped the original EC2 and RDS instances on AWS. Will keep it for a few more days. If everything is okay, I will delete them permanently.

I have stopped the original EC2 and RDS instances on AWS. Will keep it for a few more days. If everything is okay, I will delete them permanently.


adding 5.15 to AmazonLinux2 kernel urls

Signed-off-by: Logan Bond<EMAIL_ADDRESS>
What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind documentation

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area crawler

/area ci

/area utils

What this PR does / why we need it:
This PR adds support for new AmazonLinux2 5.15.x kernels, which are stored under a different path name. We had an instance today of a customer running one of these new kernels that kernel-crawler was unable to find. In a local test, adding the path to the repo list was the simple fix 😄
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: EXONER4TED
Once this PR has been reviewed and has the lgtm label, please assign maxgio92 for approval by writing /assign @maxgio92 in a comment. For more information see:The Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: cc504cba7a97f8147013ffd9423db88b463d4220

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: EXONER4TED, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


can i read matlab cell array by using matlabreader?

My data source just like this

and my code is

But it throws an exception System.NotSupportedException.
What should i do?

If I remember correctly only scalars, matrices and vectors are supported, and then in an old format. The new Matlab file formats are more complicated to read.


'HGRU4Rec' object has no attribute 'predict_next'

Hi, I am trying to experience your work on my dataset, but I find that when I use evaluate_sessions() in evalutaion.py, something is wrong with "preds = pr.predict_next(sid, prev_iid, prev_uid, items_to_predict)", do you notice that?

Hi @henglily , can you please paste here the error log so I can try to trace down the error?

Hi @henglily , can you please paste here the error log so I can try to trace down the error?

hi, I feel so sorry that I did not keep the error log, but I encountered the error during I use the function called evaluate_sessions() in evaluation.py. In fact that I can't find the method, "predict_next()"


Extracting a rectified image for android device

Hello,
I am using D435 depth camera. I could compile the sample application in the repo for an android device using android studio, but I would like to extract rectified frames. How can I do it? Or is there any detailed documentation for the realsense sdk for android?
I appreciate your answer in advance.

Hi @Tikitaka02  Almost all RealSense stream formats except Y16 are already rectified by default so you do not need to write a special routine to rectify them.  The rectification is applied to the image inside the camera hardware before the image is sent along the USB cable.
There are example programs for the SDK's Android wrapper at the link below.
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/android/examples

Thank you for your message. I meant rectified images as not only the undistorted images, but also the combined images from 2 cameras.

For an undistorted image the easiest method would be to use Y16 format (available as infrared and RGB), as it does not have a distortion model applied to it.  It is a monochrome image though.
There is not an unrectified format available for depth.
Unrectified RGB can also be obtained by accessing the camera's raw RGB frames and converting them to a usable RGB format with OpenCV's cvtColor instruction.
When you mention two cameras, do you mean two D435 cameras, or aligning two images (depth and colour) on the same camera, please

I meant the 2 views in the same camera. I will explain the background. I noticed that there 4 sensors on the camera. 1 for infrared, 2 for depth, and 1 for normal rgb camera. I could place the normal rgb screen and depth screen side by side like the example named capture. However, the 2 screens have different perspective since I think the depth screen is the combination of 2 sensors. I would like to have the combined view as rgb image.

The stereo depth image is constructed from images produced by the left and right infrared sensors.  These are raw infrared frames in the camera hardware.  Depth is not generated from the Infrared and Infrared 2 streams, which is why depth can be enabled even if the Infrared and Infrared 2 streams are inactive.
You can align the depth and RGB screeens into a single combined image.  There are few references though for how to perform depth-color alignment in Android.  One of the best available references is https://github.com/IntelRealSense/librealsense/issues/5520
As well as depth to color alignment (where the depth image is mapped onto the RGB image), you can also perform color to depth alignment, where the RGB image is mapped onto the depth image.

Thank you so much. I could align the view. Would it be possible to render some points and lines on the aligned view?

Thank you for the answer. Now I can draw on the display, but I would like to only display rgb stream and put the depth stream background so that I can still get depth information. How can I do it?

You could perhaps create two separate pipelines (pipeline1 and pipeline2), each with their own configuration (config1 and config2).  Put the color stream definition in the config1 definition and the depth stream in the config2 definition.  Start both pipeline1 and pipeline2 so that both are streaming data, but only visually render pipeline1 (the color one).

But I would like to align the color and depth also.

Possibly you could put the depth code within a true / false bool condition so that when 'depth = true' the depth information is allowed to be produced, and if 'depth = false' then the depth information is not allowed to be produced.

So what I have done is like this
`    private void configAndStart() throws Exception {
try(Config config  = new Config())
{
// Set the format corresponding to bitmap
config.enableStream(StreamType.COLOR, 640, 480, StreamFormat.RGBA8);
config.enableStream(StreamType.DEPTH, 640, 480);
// try statement needed here to release resources allocated by the Pipeline:start() method
try(PipelineProfile pp = mPipeline.start(config)){
        }

    }
}`

`try(FrameReleaser fr = new FrameReleaser()) {
try (FrameSet frames = mPipeline.waitForFrames(1000).releaseWith(fr)) {
try (VideoFrame videoFrame = frames.first(StreamType.COLOR).releaseWith(fr).as(Extension.VIDEO_FRAME);) {
try (Frame f = frames.first(StreamType.DEPTH)) {
f.releaseWith(fr);
DepthFrame depthFrame = f.as(Extension.DEPTH_FRAME);
extrinsic = videoFrame.getProfile().getExtrinsicTo(depthFrame.getProfile());
float[] rot = extrinsic.getRotation();
float[] trans = extrinsic.getTranslation();
for (int i =0; i<rot.length; i++) {
Log.e("CamParams", "intRot "+String.valueOf(i)+" : "+String.valueOf(rot[i]));
}
for (int i =0; i<trans.length; i++) {
Log.e("CamParams", "intTrans "+String.valueOf(i)+" : "+String.valueOf(trans[0]));
}
                            intrinsicDepth = depthFrame.getProfile().getIntrinsic();
                            float fxD = intrinsicDepth.getFx();
                            float fyD = intrinsicDepth.getFy();
                            float PpxD = intrinsicDepth.getPpx();
                            float PpyD = intrinsicDepth.getPpy();

                            Log.e("CamParamsDepth", "intFx: "+String.valueOf(fxD));
                            Log.e("CamParamsDepth", "intFy: "+String.valueOf(fyD));
                            Log.e("CamParamsDepth", "intPpx: "+String.valueOf(PpxD));
                            Log.e("CamParamsDepth", "intPpy: "+String.valueOf(PpyD));
                        }

                        intrinsic = videoFrame.getProfile().getIntrinsic();
                        float fx = intrinsic.getFx();
                        float fy = intrinsic.getFy();
                        float Ppx = intrinsic.getPpx();
                        float Ppy = intrinsic.getPpy();

                        Log.e("CamParamsColor", "intFx: "+String.valueOf(fx));
                        Log.e("CamParamsColor", "intFy: "+String.valueOf(fy));
                        Log.e("CamParamsColor", "intPpx: "+String.valueOf(Ppx));
                        Log.e("CamParamsColor", "intPpy: "+String.valueOf(Ppy));

                    }

                    mGLSurfaceView.upload(frames);
                }

            }
            mHandler.post(mStreaming);
        }
        catch (Exception e) {
            Log.e(TAG, "streaming, error: " + e.getMessage());
        }`

I could extract camera extrinsics and intrinsics from both depth an color views, but how can I find the pixcel correspondance to know the specific pixel in color view is corresponding to that pixel in depth view?

So what I have done is like this
` private void configAndStart() throws Exception {
try(Config config = new Config())
{
// Set the format corresponding to bitmap
config.enableStream(StreamType.COLOR, 640, 480, StreamFormat.RGBA8);
config.enableStream(StreamType.DEPTH, 640, 480);
// try statement needed here to release resources allocated by the Pipeline:start() method
try(PipelineProfile pp = mPipeline.start(config)){
    }

}

}`
and
`try(FrameReleaser fr = new FrameReleaser()) {
try (FrameSet frames = mPipeline.waitForFrames(1000).releaseWith(fr)) {
try (VideoFrame videoFrame = frames.first(StreamType.COLOR).releaseWith(fr).as(Extension.VIDEO_FRAME);) {
try (Frame f = frames.first(StreamType.DEPTH)) {
f.releaseWith(fr);
DepthFrame depthFrame = f.as(Extension.DEPTH_FRAME);
extrinsic = videoFrame.getProfile().getExtrinsicTo(depthFrame.getProfile());
float[] rot = extrinsic.getRotation();
float[] trans = extrinsic.getTranslation();
for (int i =0; i<rot.length; i++) {
Log.e("CamParams", "intRot "+String.valueOf(i)+" : "+String.valueOf(rot[i]));
}
for (int i =0; i<trans.length; i++) {
Log.e("CamParams", "intTrans "+String.valueOf(i)+" : "+String.valueOf(trans[0]));
}
                        intrinsicDepth = depthFrame.getProfile().getIntrinsic();
                        float fxD = intrinsicDepth.getFx();
                        float fyD = intrinsicDepth.getFy();
                        float PpxD = intrinsicDepth.getPpx();
                        float PpyD = intrinsicDepth.getPpy();

                        Log.e("CamParamsDepth", "intFx: "+String.valueOf(fxD));
                        Log.e("CamParamsDepth", "intFy: "+String.valueOf(fyD));
                        Log.e("CamParamsDepth", "intPpx: "+String.valueOf(PpxD));
                        Log.e("CamParamsDepth", "intPpy: "+String.valueOf(PpyD));
                    }

                    intrinsic = videoFrame.getProfile().getIntrinsic();
                    float fx = intrinsic.getFx();
                    float fy = intrinsic.getFy();
                    float Ppx = intrinsic.getPpx();
                    float Ppy = intrinsic.getPpy();

                    Log.e("CamParamsColor", "intFx: "+String.valueOf(fx));
                    Log.e("CamParamsColor", "intFy: "+String.valueOf(fy));
                    Log.e("CamParamsColor", "intPpx: "+String.valueOf(Ppx));
                    Log.e("CamParamsColor", "intPpy: "+String.valueOf(Ppy));

                }

                mGLSurfaceView.upload(frames);
            }

        }
        mHandler.post(mStreaming);
    }
    catch (Exception e) {
        Log.e(TAG, "streaming, error: " + e.getMessage());
    }`

I could extract camera extrinsics and intrinsics from both depth an color views, but how can I find the pixcel correspondance to know the specific pixel in color view is corresponding to that pixel in depth view?

So what I have done is like this
`    private void configAndStart() throws Exception {
try(Config config  = new Config())
{
// Set the format corresponding to bitmap
config.enableStream(StreamType.COLOR, 640, 480, StreamFormat.RGBA8);
config.enableStream(StreamType.DEPTH, 640, 480);
// try statement needed here to release resources allocated by the Pipeline:start() method
try(PipelineProfile pp = mPipeline.start(config)){
        }

    }
}`

and
`Runnable mStreaming = new Runnable() {
@Override
public void run() {
try {
try(FrameReleaser fr = new FrameReleaser()) {
try (FrameSet frames = mPipeline.waitForFrames(1000).releaseWith(fr)) {
//                        FrameSet aligned_frames = align.process(frames).releaseWith(fr);
                    try (VideoFrame videoFrame = frames.first(StreamType.COLOR).releaseWith(fr).as(Extension.VIDEO_FRAME);) {

                        try (Frame f = frames.first(StreamType.DEPTH)) {
                            f.releaseWith(fr);
                            DepthFrame depthFrame = f.as(Extension.DEPTH_FRAME);

                            extrinsic = videoFrame.getProfile().getExtrinsicTo(depthFrame.getProfile());
                            float[] rot = extrinsic.getRotation();
                            float[] trans = extrinsic.getTranslation();
                            for (int i =0; i<rot.length; i++) {
                                Log.e("CamParams", "intRot "+String.valueOf(i)+" : "+String.valueOf(rot[i]));
                            }
                            for (int i =0; i<trans.length; i++) {
                                Log.e("CamParams", "intTrans "+String.valueOf(i)+" : "+String.valueOf(trans[0]));
                            }

                            intrinsicDepth = depthFrame.getProfile().getIntrinsic();
                            float fxD = intrinsicDepth.getFx();
                            float fyD = intrinsicDepth.getFy();
                            float PpxD = intrinsicDepth.getPpx();
                            float PpyD = intrinsicDepth.getPpy();

                            Log.e("CamParamsDepth", "intFx: "+String.valueOf(fxD));
                            Log.e("CamParamsDepth", "intFy: "+String.valueOf(fyD));
                            Log.e("CamParamsDepth", "intPpx: "+String.valueOf(PpxD));
                            Log.e("CamParamsDepth", "intPpy: "+String.valueOf(PpyD));
                        }

                        intrinsic = videoFrame.getProfile().getIntrinsic();
                        float fx = intrinsic.getFx();
                        float fy = intrinsic.getFy();
                        float Ppx = intrinsic.getPpx();
                        float Ppy = intrinsic.getPpy();

                        Log.e("CamParamsColor", "intFx: "+String.valueOf(fx));
                        Log.e("CamParamsColor", "intFy: "+String.valueOf(fy));
                        Log.e("CamParamsColor", "intPpx: "+String.valueOf(Ppx));
                        Log.e("CamParamsColor", "intPpy: "+String.valueOf(Ppy));

                        Log.e("bitmapSize","Height"+ String.valueOf(image.getHeight()));
                        Log.e("bitmapSize: ","Width" + String.valueOf(image.getWidth()));
                    }
                    mGLSurfaceView.upload(frames);
                }
            }
            mHandler.post(mStreaming);
        }
        catch (Exception e) {
            Log.e(TAG, "streaming, error: " + e.getMessage());
        }
    }
};`

I could extract camera extrinsics and intrinsics from both depth an color views, but how can I find the pixcel correspondance to know the specific pixel in color view is corresponding to that pixel in depth view?

So what I have done is like this
    private void configAndStart() throws Exception {         try(Config config  = new Config())         {             config.enableStream(StreamType.COLOR, 640, 480, StreamFormat.RGBA8);             config.enableStream(StreamType.DEPTH, 640, 480);             try(PipelineProfile pp = mPipeline.start(config)){             }         }     }
and
`Runnable mStreaming = new Runnable() {
@Override
public void run() {
try {
try(FrameReleaser fr = new FrameReleaser()) {
try (FrameSet frames = mPipeline.waitForFrames(1000).releaseWith(fr)) {
                    try (VideoFrame videoFrame = frames.first(StreamType.COLOR).releaseWith(fr).as(Extension.VIDEO_FRAME);) {

                        try (Frame f = frames.first(StreamType.DEPTH)) {
                            f.releaseWith(fr);
                            DepthFrame depthFrame = f.as(Extension.DEPTH_FRAME);

                            extrinsic = videoFrame.getProfile().getExtrinsicTo(depthFrame.getProfile());
                            float[] rot = extrinsic.getRotation();
                            float[] trans = extrinsic.getTranslation();
                            for (int i =0; i<rot.length; i++) {
                                Log.e("CamParams", "intRot "+String.valueOf(i)+" : "+String.valueOf(rot[i]));
                            }
                            for (int i =0; i<trans.length; i++) {
                                Log.e("CamParams", "intTrans "+String.valueOf(i)+" : "+String.valueOf(trans[0]));
                            }

                            intrinsicDepth = depthFrame.getProfile().getIntrinsic();
                            float fxD = intrinsicDepth.getFx();
                            float fyD = intrinsicDepth.getFy();
                            float PpxD = intrinsicDepth.getPpx();
                            float PpyD = intrinsicDepth.getPpy();

                            Log.e("CamParamsDepth", "intFx: "+String.valueOf(fxD));
                            Log.e("CamParamsDepth", "intFy: "+String.valueOf(fyD));
                            Log.e("CamParamsDepth", "intPpx: "+String.valueOf(PpxD));
                            Log.e("CamParamsDepth", "intPpy: "+String.valueOf(PpyD));
                        }

                        intrinsic = videoFrame.getProfile().getIntrinsic();
                        float fx = intrinsic.getFx();
                        float fy = intrinsic.getFy();
                        float Ppx = intrinsic.getPpx();
                        float Ppy = intrinsic.getPpy();

                        Log.e("CamParamsColor", "intFx: "+String.valueOf(fx));
                        Log.e("CamParamsColor", "intFy: "+String.valueOf(fy));
                        Log.e("CamParamsColor", "intPpx: "+String.valueOf(Ppx));
                        Log.e("CamParamsColor", "intPpy: "+String.valueOf(Ppy));

                        Log.e("bitmapSize","Height"+ String.valueOf(image.getHeight()));
                        Log.e("bitmapSize: ","Width" + String.valueOf(image.getWidth()));
                    }
                    mGLSurfaceView.upload(frames);
                }
            }
            mHandler.post(mStreaming);
        }
        catch (Exception e) {
            Log.e(TAG, "streaming, error: " + e.getMessage());
        }
    }
};`

I could extract camera extrinsics and intrinsics from both depth an color views, but how can I find the pixcel correspondance to know the specific pixel in color view is corresponding to that pixel in depth view?

So what I have done is like this
private void configAndStart() throws Exception {
    try(Config config  = new Config())
    {
        // Set the format corresponding to bitmap
        config.enableStream(StreamType.COLOR, 640, 480, StreamFormat.RGBA8);
        config.enableStream(StreamType.DEPTH, 640, 480);
        // try statement needed here to release resources allocated by the Pipeline:start() method
        try(PipelineProfile pp = mPipeline.start(config)){

        }

    }
}

and
Runnable mStreaming = new Runnable() {
        @Override
        public void run() {
            try {
                try(FrameReleaser fr = new FrameReleaser()) {
                    try (FrameSet frames = mPipeline.waitForFrames(1000).releaseWith(fr)) {

                        try (VideoFrame videoFrame = frames.first(StreamType.COLOR).releaseWith(fr).as(Extension.VIDEO_FRAME);) {

                            try (Frame f = frames.first(StreamType.DEPTH)) {
                                f.releaseWith(fr);
                                DepthFrame depthFrame = f.as(Extension.DEPTH_FRAME);

                                extrinsic = videoFrame.getProfile().getExtrinsicTo(depthFrame.getProfile());
                                float[] rot = extrinsic.getRotation();
                                float[] trans = extrinsic.getTranslation();
                                for (int i =0; i<rot.length; i++) {
                                    Log.e("CamParams", "intRot "+String.valueOf(i)+" : "+String.valueOf(rot[i]));
                                }
                                for (int i =0; i<trans.length; i++) {
                                    Log.e("CamParams", "intTrans "+String.valueOf(i)+" : "+String.valueOf(trans[0]));
                                }

                                intrinsicDepth = depthFrame.getProfile().getIntrinsic();
                                float fxD = intrinsicDepth.getFx();
                                float fyD = intrinsicDepth.getFy();
                                float PpxD = intrinsicDepth.getPpx();
                                float PpyD = intrinsicDepth.getPpy();

                                Log.e("CamParamsDepth", "intFx: "+String.valueOf(fxD));
                                Log.e("CamParamsDepth", "intFy: "+String.valueOf(fyD));
                                Log.e("CamParamsDepth", "intPpx: "+String.valueOf(PpxD));
                                Log.e("CamParamsDepth", "intPpy: "+String.valueOf(PpyD));
                            }

                            intrinsic = videoFrame.getProfile().getIntrinsic();
                            float fx = intrinsic.getFx();
                            float fy = intrinsic.getFy();
                            float Ppx = intrinsic.getPpx();
                            float Ppy = intrinsic.getPpy();

                            Log.e("CamParamsColor", "intFx: "+String.valueOf(fx));
                            Log.e("CamParamsColor", "intFy: "+String.valueOf(fy));
                            Log.e("CamParamsColor", "intPpx: "+String.valueOf(Ppx));
                            Log.e("CamParamsColor", "intPpy: "+String.valueOf(Ppy));

                            Log.e("bitmapSize","Height"+ String.valueOf(image.getHeight()));
                            Log.e("bitmapSize: ","Width" + String.valueOf(image.getWidth()));
                        }
                        mGLSurfaceView.upload(frames);
                    }
                }
                mHandler.post(mStreaming);
            }
            catch (Exception e) {
                Log.e(TAG, "streaming, error: " + e.getMessage());
            }
        }
    };

I could extract camera extrinsics and intrinsics from both depth an color views, but how can I find the pixcel correspondance to know the specific pixel in color view is corresponding to that pixel in depth view?

The SDK has an instruction called rs2_project_color_pixel_to_depth_pixel that can convert a single specific color pixel coordinate to a depth pixel coordinate in order to obtain 3D XYZ without using alignment or pointcloud.

Would it be possible to paste the instruction page here? I cannot find it...

There are not Android references about using rs2_project_color_pixel_to_depth_pixel with Android, unfortunately.  kafan1986 at https://github.com/IntelRealSense/librealsense/issues/5766 where you just commented will likely have Android programming knowledge about using this instruction.

For almost all 400 Series depth cameras except D405, the default depth scale value is 0.001 and does not change unless deliberately changed by the program or a user input.  So this value can be hard-coded into a script instead of retrieving the scale value from the camera in real-time.

Got it thanks! Also, what do the depth max and min range  mean? When I try to obtain them without hard code, there are so many options to choose from.

I have previously looked at depth_min and depth_max at https://github.com/IntelRealSense/librealsense/issues/10484#issuecomment-1137028484

Thank you so much. I have one additional question. How can I change the orientation of view? Now I have 640px width and 480px height, but I would like to make it vertucal view like 360px width and 480px height without chaging the context meaning if a person is stands upright on a frame, the person should stand upright on a frame after changing the orientation.

You could crop the image horizontally so that the width of the image is reduced but the height remains the same.  It is not a simple thing to do though.
A RealSense user at https://github.com/IntelRealSense/librealsense/issues/8512#issuecomment-791742675 cropped a colorized depth image and shared the C++ code of their solution.
A different approach that another RealSense user described at https://github.com/IntelRealSense/librealsense/issues/2016#issuecomment-403804157 was to define a bounding box on their depth image and exclude from the depth image all of the coordinates outside of the defined box.

Sorry for this repetitive question. I was also wondering if there are any concerns about using the d435 camera under a glass screen.

It's no trouble at all.  :)
It should be no problem to use the D435 behind a glass screen so long as both sides of the screen have an anti-reflective coating applied to them.  This is to avoid back-reflections from the camera's laser.  The image below is of a D435 in a custom housing for underwater use.

Hello, Thank you for the information. Another question. Would it be possible to access only an RGB camera without using sdk in android?

The two main examples of RealSense Android applications that are able to render RGB onscreen are Capture (which you mentioned earlier in this case) and the RsCamera demo app on the Google Play store.
https://play.google.com/store/apps/details?id=com.intel.realsense.camera
The project source code for the RsCamera app is available at the link below as the Camera tool.
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/android/tools/camera

Hello, I have 2 additional questions.
If I want to place the intel camera behind a glass (thickness 2.81mm), what is the exact type of anti-reflection treatment I have to place? There are a lot of types available in the market. Can I apply the treatment only in the area around the camera?

There are not recommendations for a particular type of anti-reflective coating or for a minimum / maximum distance of the glass from the camera lens.  Yes, you could apply the coating only in the area around the camera.
If you would prefer not to coat the glass, you could alternatively try applying a thin-film linear polarization filter over the camera lenses on the outside of the camera.  This filter dampens glare from reflections in the scene such as window glass, like the 'without / with filter' image below.

Section 4.4  When to use polarizers and waveplates of Intel's white-paper guide about use of optical filters with 400 Series cameras has more information about linear polarization filters.
https://dev.intelrealsense.com/docs/optical-filters-for-intel-realsense-depth-cameras-d400#4-the-use-of-optical-filters
Any polarization filter that is linear will work, though the circular filters in 3D glasses will not.  This makes the filters inexpensive to purchase.  You can search stores such as Amazon for the term linear polarizing filter sheet

Hi @Tikitaka02  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.


WiP: Add TypeScript definitions

I was able to add most of the definitions but React components definitions are still missing. I still don't know to use the React.Component as parent. I still need to research but if anybody knows how to do that, let me know.
Also I need to test this properly in some real project which should be today when I get to work.
#95

Seems like
import * as React from "react";

interface LinkProps {
    children: React.ReactElement<any>;
    className: string;
    href: Href;
    persistQuery: boolean;
    replaceState: boolean;
    target: string;
    onClick: React.EventHandler<React.MouseEvent<HTMLAnchorElement>>;
    style: Object;
    location: Location;
    push: Function;
    replace: Function;
}

declare class Link extends React.Component<LinkProps, any> {
}

should work

Is there any reason not to merge this?

What's happening with this?  I really want to use this package on a new project but no typescript is a deal breaker

@tptee can we get this one merged?

Sorry for the wait y'all! Will be getting this ready for merge this week!

any progress?

Do you want any help with this?  Would be good to get it in

Hey, is this PR dead? Would really love to get this merged, as @hisuwh mentioned no TS declaration is a deal breaker. Let me know if I could be of help.

Got bored of waiting for this so added the types to DefinitelyTyped
If this ever gets merged I will I remove it there

https://github.com/FormidableLabs/redux-little-router/pull/269 was merged, enjoy your TS defs 🎉

This issue can be closed, the typings are included in the package.

Thanks @renchap! We're not doing much active maintenance on redux-little-router anymore so I hadn't seen this yet.


Collapse menu via sliding finger

This is an amazing app. I'm using it on my phone instead of mouse of tablet. One suggestion i have is to slide the app menu in the direction of the arrow to hide it. Its difficult to click on the arrow sometimes since its far out of reach. Thanks.

Thanks for the suggestion.  This is something I will look into implementing in a future version.

Maybe also show by sliding in?


TypeError: Cannot read properties of undefined (reading 'value')

Hi there,
My tests in CI are failing with the error:
_Test suite failed to run
TypeError: Cannot read properties of undefined (reading 'value')
at Object.<anonymous> (node_modules/react-native-launch-arguments/dist/index.js:2:43)_

Please, could you add optional chaining to the second line of the code below:
import { NativeModules } from "react-native";
const raw = NativeModules.LaunchArguments.value;
const parsed = {};
for (const k in raw) {
    const rawValue = raw[k];
    try {
        parsed[k] = JSON.parse(rawValue);
    }
    catch {
        parsed[k] = rawValue;
    }
}
export const LaunchArguments = {
    value() {
        return parsed;
    },
};

const raw = NativeModules.LaunchArguments.value; will be
const raw = NativeModules.LaunchArguments?.value; This will remove the TypeError that I get in CI.

If the native module isn't set up correctly then you could experience this error.
LaunchArguments should always be present in your build, so using ? will surpress any potential configuration errors.

Had the same error in my jest tests.
I added this to my jest.setup.ts
jest.mock('react-native-launch-arguments', () => {
  return {
    LaunchArguments: {
      value: jest.fn().mockReturnValue({}),
    },
  };
});


Containerize your microservice using Docker

As a developer
I need docker to deploy containers for the microservices
So that the e-commerce development can be tested
Details and Assumptions
* e-commerce products and services, workflow

Acceptance Criteria
gherkin
Given completion of microservices units
When i need unit test
Then i can deploy to containers

tecnical debt required


CUDA out of Memory Error

Hi Team,
Thanks for giving us the features like K2 , Lhotse. They will be a big role in coming times.
I am currently running the CTC based training framework using CSJ corpus and as soon as the training proceeds to batch number 3 or 4 in first epoch, cuda memory runs out and training process stops. Currently, the training framework is running on single cuda core. I have 11 GB of memory per cuda core.
Is it possible to run the training on multiple cuda cores ?
Regards,
Mohit

Great!
If you could help us create an example for CSJ corpus in Lhotse, it would
be easier for us to help you debug.
Right now we haven't fully debugged multiple CUDA cores (there is a hang).
Dan
On Mon, Feb 22, 2021 at 4:48 PM manu51188<EMAIL_ADDRESS>wrote:

Hi Team,
Thanks for giving us the features like K2 , Lhotse. They will be a big
role in coming times.
I am currently running the CTC based training framework using CSJ corpus
and as soon as the training proceeds to batch number 3 or 4 in first epoch,
cuda memory runs out and training process stops. Currently, the training
framework is running on single cuda core. I have 11 GB of memory per cuda
core.
Is it possible to run the training on multiple cuda cores ?
Regards,
Mohit
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/snowfall/issues/107, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO6C6PXTC7XUPTB2DBTTAIK4HANCNFSM4YAC3FZA
.

Hi,
Thanks for the reply. Yes surely I can create recipe for CSJ corpus in Lhotse similar to other receipes. I can see that duration in CSJ .wav files ranges from 600 sec to 5245 sec (quite long) and preparing the cutset from them. I think that will be a limiting factor for me.
Any idea how I can handle such varies length of recordings for training?
Regards,
Mohit

I would assume that csj comes  with some kind of segmentation information
so you don't have to train on entire wav files?
On Mon, Feb 22, 2021 at 6:28 PM manu51188<EMAIL_ADDRESS>wrote:

Hi,
Thanks for the reply. Yes surely I can create recipe for CSJ corpus in
Lhotse similar to other receipes. I can see that duration in CSJ .wav files
ranges from 600 sec to 5245 sec (quite long) and preparing the cutset from
them. I think that will be a limiting factor for me.
Any idea how I can handle such varies length of recordings for training?
Regards,
Mohit
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/snowfall/issues/107#issuecomment-783270246,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO3IQU2A2EL3WNWL7Q3TAIWVTANCNFSM4YAC3FZA
.

As Dan said, there will typically be segmentation information (in Lhotse represented as SupervisionSet). After you create the CutSet, call .trim_to_supervisions() on it to have each cut represent a single segment.

Hi @danpovey , @pzelasko ,
Thanks very much for the response. I will try trim_to_supervisions() to check the results.
Thanks and Regards,
Mohit

Hi @pzelasko ,
I have added .train_to_supervison() to represent each cut a single segment. But unfortunately there is an assertion error in the validation error from the following lines : https://github.com/lhotsespeech/lhotse/blob/master/lhotse/dataset/speech_recognition.py#L122
assert (cut.start - 1e-5) <= supervision.start <= supervision.end <= (cut.end + 1e-5),
AssertionError: Cutting in the middle of a supervision is currently not supported for the ASR task. Cut ID violating the pre-condition: '218b7cc8-594f-4114-b2e7-67863db7f0ce'.

Can you find and print the cut with that ID (cut_set['218b7cc8-594f-4114-b2e7-67863db7f0ce'])? I wonder if this is a rounding error or something bigger.

Hi @pzelasko,
Yes I printed the information for CutID ['218b7cc8-594f-4114-b2e7-67863db7f0ce']. Here is what I get,
"id": "218b7cc8-594f-4114-b2e7-67863db7f0ce",
"start": 0.32775,
"duration": 0.4866875,
"channel": 0,
"supervisions": [
  {
    "id": "R01M0278_0000295_0000733_sp0.9",
    "recording_id": "R01M0278_sp0.9",
    "start": 0.0,
    "duration": 0.4866875,
    "channel": 0,
    "text": " \u5b87\u5b99+\u540d\u8a5e",
    "speaker": "R01M0278"
  }
],
"features": {
  "type": "fbank",
  "num_frames": 9180,
  "num_features": 40,
  "frame_shift": 0.01,
  "sampling_rate": 16000,
  "start": 0.0,
  "duration": 91.80225,
  "storage_type": "lilcom_hdf5",
  "storage_path": "exp/data/fbank/train_all/feats-0.h5",
  "storage_key": "47fb1d58-d402-4925-a06a-eec4f460041b",
  "channels": 0
},
"recording": {
  "id": "R01M0278_sp0.9",
  "sources": [
    {
      "type": "file",
      "channels": [
        0
      ],
      "source": "/home/sysadmin/CSJ_RAW/WAV/noncore/R01M0278.wav"
    }
  ],
  "sampling_rate": 16000,
  "num_samples": 1468836,
  "duration": 91.80225,
  "transforms": [
    {
      "name": "Speed",
      "kwargs": {
        "factor": 0.9
      }
    }
  ]
},
"type": "Cut"

},
I am consistently finding the cutset is starting in the middle of supervision for other cutsets as well (most of them I say).
For example this one,
{
"id": "6f320bf1-2132-421b-9f7d-a8218acb7066",
"start": 79.54275,
"duration": 3.1881875,
"channel": 0,
"supervisions": [
{
"id": "S05M0469_0087497_0091004_sp1.1",
"recording_id": "S05M0469_sp1.1",
"start": 0.0,
"duration": 3.1881875,
"channel": 0,
"text": " \u79c1+\u4ee3\u540d\u8a5e \u304c+\u52a9\u8a5e/\u683c\u52a9\u8a5e \u5352\u696d+\u540d\u8a5e \u3059\u308b+\u52d5\u8a5e/\u30b5\u884c\u5909\u683c/\u9023\u4f53\u5f62 \u9803+\u540d\u8a5e \u304a\u30fc+\u611f\u52d5\u8a5e",
"speaker": "S05M0469"
}
],
"features": {
"type": "fbank",
"num_frames": 73800,
"num_features": 40,
"frame_shift": 0.01,
"sampling_rate": 16000,
"start": 0.0,
"duration":<PHONE_NUMBER>,
"storage_type": "lilcom_hdf5",
"storage_path": "exp/data/fbank/train_all/feats-5.h5",
"storage_key": "b7cecc64-68d3-4c9a-83ca-f3a6ba720e64",
"channels": 0
},
"recording": {
"id": "S05M0469_sp1.1",
"sources": [
{
"type": "file",
"channels": [
0
],
"source": "/home/sysadmin/CSJ_RAW/WAV/noncore/S05M0469.wav"
}
],
"sampling_rate": 16000,
"num_samples": 11807994,
"duration": 737.999625,
"transforms": [
{
"name": "Speed",
"kwargs": {
"factor": 1.1
}
}
]
},
"type": "Cut"
},

The cuts look correct to me. The „start” field has a different semantics in cut and in supervision: for cut, it is relative to the start of the recording; for the supervision, it is relative to the start of the cut. It seems the assertion in the K2SpeechRecognitionDataset is incorrect; I will commit a fix later today.

Ok. Thanks.
I will try the training again after your fix.
Regards,
Mohit

I merged it, please check out the latest Lhotse and try again:
pip uninstall lhotse
pip install git+https://github.com/lhotse-speech/lhotse

Hi,
I have re-run the training and it still fails at the same validation check with following error :
Traceback (most recent call last):
File "./ctc_train.py", line 408, in
main()
File "./ctc_train.py", line 275, in main
train = K2SpeechRecognitionDataset(
File "/home/sysadmin/miniconda3/lib/python3.8/site-packages/lhotse/dataset/speech_recognition.py", line 69, in init
self._validate()
File "/home/sysadmin/miniconda3/lib/python3.8/site-packages/lhotse/dataset/speech_recognition.py", line 123, in _validate
assert supervision.start >= -tol, f"Supervisions starting before the cut are not supported for ASR"
AssertionError: Supervisions starting before the cut are not supported for ASR (sup id: A01M6710_0604926_0615368_sp1.1, cut id: 4fb131fa-272c-437b-a9bb-005671c25cb6)
and here is the cutset for id 4fb131fa-272c-437b-a9bb-005671c25cb6 :
"id": "4fb131fa-272c-437b-a9bb-005671c25cb6",
"start":<PHONE_NUMBER>,
"duration": 5.3045625,
"channel": 0,
"supervisions": [
  {
    "id": "A01M6710_0604926_0615368_sp1.1",
    "recording_id": "A01M6710_sp1.1",
    "start": -9.4926875,
    "duration": 9.49275,
    "channel": 0,
    "text": " \u3042\u306e\u30fc+\u611f\u52d5\u8a5e \u5f0a\u793e+\u540d\u8a5e/\u4e00\u822c \u306e+\u52a9\u8a5e/\u683c\u52a9\u8a5e \u3088\u3046+\u540d\u8a5e/\u975e\u81ea\u7acb/\u52a9\u52d5\u8a5e\u8a9e\u5e79 \u306b+\u52a9\u8a5e/\u526f\u8a5e\u5316 \u3053\u3046\u3044\u3046+\u9023\u4f53\u8a5e \u3044\u308d\u3093\u306a+\u9023\u4f53\u8a5e \u8077\u7a2e+\u540d\u8a5e/\u4e00\u822c \u304c+\u52a9\u8a5e/\u4e00\u822c/\u683c\u52a9\u8a5e \u8f09\u3063+\u52d5\u8a5e/\u81ea\u7acb/\u9023\u7528\u30bf\u63a5\u7d9a/\u4e94\u6bb5\u30fb\u30e9\u884c \u3066\u308b+\u52d5\u8a5e/\u975e\u81ea\u7acb/\u57fa\u672c\u5f62/\u4e00\u6bb5 \u30b5\u30a4\u30c8+\u540d\u8a5e/\u4e00\u822c \u3067+\u52a9\u8a5e/\u4e00\u822c/\u683c\u52a9\u8a5e \u30fc+\u540d\u8a5e/\u4e00\u822c/\u30fc <sp> \u63a1\u7528+\u540d\u8a5e/\u30b5\u5909\u63a5\u7d9a \u6210\u529f+\u540d\u8a5e/\u30b5\u5909\u63a5\u7d9a \u3057+\u52d5\u8a5e/\u81ea\u7acb/\u9023\u7528\u5f62/\u30b5\u5909\u30fb\u30b9\u30eb \u3066+\u52a9\u8a5e/\u63a5\u7d9a\u52a9\u8a5e \u3044\u308b+\u52d5\u8a5e/\u975e\u81ea\u7acb/\u57fa\u672c\u5f62/\u4e00\u6bb5 \u4f01\u696d+\u540d\u8a5e/\u4e00\u822c \u69d8+\u540d\u8a5e/\u63a5\u5c3e\u8f9e/\u4eba\u540d \u306f+\u52a9\u8a5e/\u4fc2\u52a9\u8a5e \u3069\u3046+\u526f\u8a5e/\u52a9\u8a5e\u985e\u63a5\u7d9a \u3057+\u52d5\u8a5e/\u81ea\u7acb/\u9023\u7528\u5f62/\u30b5\u5909\u30fb\u30b9\u30eb \u3066+\u52a9\u8a5e/\u63a5\u7d9a\u52a9\u8a5e \u3044\u308b+\u52d5\u8a5e/\u975e\u81ea\u7acb/\u57fa\u672c\u5f62/\u4e00\u6bb5 \u304b+\u52a9\u8a5e/\u526f\u52a9\u8a5e\uff0f\u4e26\u7acb\u52a9\u8a5e\uff0f\u7d42\u52a9\u8a5e \u3063\u3066+\u52a9\u8a5e/\u9023\u4f53\u5f62/\u683c\u52a9\u8a5e \u3068\u3053\u308d+\u540d\u8a5e/\u975e\u81ea\u7acb/\u526f\u8a5e \u3067\u3059+\u52a9\u52d5\u8a5e/\u57fa\u672c\u5f62/\u7279\u6b8a\u30fb\u30c7\u30b9 \u306d+\u52a9\u8a5e/\u7d42\u52a9\u8a5e <sp> \u3067+\u63a5\u7d9a\u8a5e \u30fc+\u540d\u8a5e/\u4e00\u822c/\u30fc \u306a\u3093\u304b+\u52a9\u8a5e/\u526f\u52a9\u8a5e \u30fc+\u540d\u8a5e/\u4e00\u822c/\u30fc <sp> \u306e+\u52a9\u8a5e/\u683c\u52a9\u8a5e \u3054+\u63a5\u982d\u8f9e/\u540d\u8a5e\u63a5\u7d9a \u8aac\u660e+\u540d\u8a5e/\u30b5\u5909\u63a5\u7d9a \u304c+\u52a9\u8a5e/\u4e00\u822c/\u683c\u52a9\u8a5e \u3067\u304d\u308c+\u52d5\u8a5e/\u81ea\u7acb/\u4eee\u5b9a\u5f62/\u4e00\u6bb5 \u3070+\u52a9\u8a5e/\u63a5\u7d9a\u52a9\u8a5e \u306a\u3042+\u52a9\u8a5e/\u7d42\u52a9\u8a5e \u3068+\u52a9\u8a5e/\u5f15\u7528/\u683c\u52a9\u8a5e \u601d\u3063+\u52d5\u8a5e/\u81ea\u7acb/\u9023\u7528\u30bf\u63a5\u7d9a/\u4e94\u6bb5\u30fb\u30ef\u884c\u4fc3\u97f3\u4fbf \u3066+\u52a9\u8a5e/\u63a5\u7d9a\u52a9\u8a5e \u3044+\u52d5\u8a5e/\u975e\u81ea\u7acb/\u9023\u7528\u5f62/\u4e00\u6bb5 \u307e\u3057+\u52a9\u52d5\u8a5e/\u9023\u7528\u5f62/\u7279\u6b8a\u30fb\u30de\u30b9 \u3066+\u52a9\u8a5e/\u63a5\u7d9a\u52a9\u8a5e",
    "speaker": "A01M6710"
  },
  {
    "id": "A01M6710_0615368_0621203_sp1.1",
    "recording_id": "A01M6710_sp1.1",
    "start": 0.0,
    "duration": 5.3045625,
    "channel": 0,
    "text": " \u305f\u3057\u304b\u306b+\u526f\u8a5e/\u4e00\u822c \u3042\u306e+\u9023\u4f53\u8a5e \u9732\u51fa+\u540d\u8a5e/\u30b5\u5909\u63a5\u7d9a \u3092+\u52a9\u8a5e/\u4e00\u822c/\u683c\u52a9\u8a5e \u30fc+\u540d\u8a5e/\u30fc/\u56fa\u6709\u540d\u8a5e <sp> \u3042\u3063+\u611f\u52d5\u8a5e \u3054\u3081\u3093\u306a\u3055\u3044+\u611f\u52d5\u8a5e <sp> \u3042\u306e\u30fc+\u611f\u52d5\u8a5e \u3044\u308d\u3093\u306a+\u9023\u4f53\u8a5e \u3068\u3053\u308d+\u540d\u8a5e/\u975e\u81ea\u7acb/\u526f\u8a5e \u306b+\u52a9\u8a5e/\u4e00\u822c/\u683c\u52a9\u8a5e \u51fa\u3059+\u52d5\u8a5e/\u81ea\u7acb/\u57fa\u672c\u5f62/\u4e94\u6bb5\u30fb\u30b5\u884c \u3063\u3066+\u52a9\u8a5e/\u9023\u4f53\u5f62/\u683c\u52a9\u8a5e \u306e+\u540d\u8a5e/\u975e\u81ea\u7acb/\u4e00\u822c \u306f+\u52a9\u8a5e/\u4fc2\u52a9\u8a5e \u78ba\u7387+\u540d\u8a5e/\u4e00\u822c \u306f+\u52a9\u8a5e/\u4fc2\u52a9\u8a5e \u4e0a\u304c\u308b+\u52d5\u8a5e/\u81ea\u7acb/\u57fa\u672c\u5f62/\u4e94\u6bb5\u30fb\u30e9\u884c \u3093+\u540d\u8a5e/\u975e\u81ea\u7acb/\u4e00\u822c \u3067\u3059+\u52a9\u52d5\u8a5e/\u57fa\u672c\u5f62/\u7279\u6b8a\u30fb\u30c7\u30b9 \u3088+\u52a9\u8a5e/\u7d42\u52a9\u8a5e",
    "speaker": "A01M6710"
  }
],
"features": {
  "type": "fbank",
  "num_frames": 136848,
  "num_features": 40,
  "frame_shift": 0.01,
  "sampling_rate": 16000,
  "start": 0.0,
  "duration": 1368.4845625,
  "storage_type": "lilcom_hdf5",
  "storage_path": "exp/data/fbank/train_all/feats-39.h5",
  "storage_key": "6bc2e318-a256-4583-af13-3e83ac214858",
  "channels": 0
},
"recording": {
  "id": "A01M6710_sp1.1",
  "sources": [
    {
      "type": "file",
      "channels": [
        0
      ],
      "source": "/home/sysadmin/CSJ_RAW/WAV/core/A01M6710.wav"
    }
  ],
  "sampling_rate": 16000,
  "num_samples": 21895753,
  "duration": 1368.4845625,
  "transforms": [
    {
      "name": "Speed",
      "kwargs": {
        "factor": 1.1
      }
    }
  ]
},
"type": "Cut"

},

OK but this time it is different. You have a 5.3s cut with two supervisions - one of them spans the whole cut, the other one is much longer - begins 9.5 seconds before and ends about 4 seconds after. So they are overlapping and it seems that they are coming from the same speaker. Are you sure that this is expected and the issue is not in the creation of the SupervisionSet?
BTW I am not sure how well the current snowfall recipes will handle overlapped speech. In principle the training should not crash, but I don't think the model will learn something meaningful.
If you are completely sure that your data is correct, then you can mitigate the current problem by doing sth like:
for cut in cuts:
    cut.supervisions = cut.trimmed_supervisions

You can read what it does here: https://github.com/lhotse-speech/lhotse/blob/master/lhotse/cut.py#L52. You must do this before any mixing or padding though.

Hi,
Yes it is from the same speaker. The segments look fine and the issue is not in the creation of the SupervisionSet.
Let me try with above method to check.
Regards,
Mohit

Hi @pzelasko,
I am finally started with the training and it is running fine till now.
The changes in validation function are looking good.


Complicated URL after choosing „Personal question about Corona“

Which hotline
+49<PHONE_NUMBER>
Current Implementation
After choosing your language and then choosing "Personal question about Corona", the voice will give you an URL. This URL contains https://. This isn't necessary.
Suggested Enhancement
Say
"infektionsschutz.de"
instead of
"h t t p s : / / w w w . i n f e k t i o n s s c h u t z . d e". (have to write it like that so that GitHub doesn't transform it to the link above)
Expected Benefits
More users understanding and correctly entering the URL.

Internal Tracking ID:  EXPOSUREAPP-5148

@Ein-Tim Same here, is this still a valid issue? Thanks!

@dsarkar
Yes, just checked.

@Ein-Tim ok, thanks. Internal Tracking ID:  EXPOSUREAPP-5148

This issue is still valid, over one year after I reported it.

@Ein-Tim
This issue is no longer relevant and can be closed, since the TAN hotline closed down today. See README.

I re-checked and this issue is still valid...

As the CWA project went into ramp-down mode, I don't think any new recordings for the CWA hotline will be done anymore. I'm therefore closing this issue now.


Support registry repositories

Instructions in the Cocoapods repository PR:
https://github.com/yahavi/CocoaPods/pull/2


Add a filter for memo and receiver field size

Closes: #3766
Description

This PR adds two packet configurations:

ics20_max_memo_size which filters packets having a memo field bigger than the configured value
ics20_max_receiver_size which filters packets having a receiver field bigger than the configured value

These values default to the respective ones in ibc-go v8: https://github.com/cosmos/ibc-go/blob/04e14a7079e18323c64d4e00e12cf77708d24701/modules/apps/transfer/types/msgs.go#L15

PR author checklist:

[x] Added changelog entry, using unclog.
[ ] Added tests: integration (for Hermes) or unit/mock tests (for modules).
[x] Linked to GitHub issue.
[x] Updated code comments and documentation (e.g., docs/).
[x] Tagged one reviewer who will be the one responsible for shepherding this PR.

Reviewer checklist:

[ ] Reviewed Files changed in the GitHub PR explorer.
[ ] Manually tested (in case integration/unit/mock tests are absent).

Left a few comments. In general I am wondering why we don't apply the checks for all msgs that contain a packet, i.e. recv, timeout and ack.

Good catch, we should indeed do that! Completely slipped my mind…

I refactored the memo and receiver field configuration and validation in this commit 451b5781f809fd6bf13b1c66340a67a0e4bc6b27.
I added the option to disable the filter by having the configuration format as following: { enabled = true, size = "32KiB" }
The reason why I put the Ics20FieldSizeLimit struct in the relayer crate is because I feel like validation linked to the configuration shouldn't be in the relayer-types crate.
I only applied the verification https://github.com/informalsystems/hermes/pull/3765/commits/451b5781f809fd6bf13b1c66340a67a0e4bc6b27#diff-072b872f1fbf4ea17384cecde28b35ee339faa9ce969ff9f767647a071da1e36R1405 as I wanted to have your opinion on this solution @ancazamfir @romac. If this seems like a good solution we can think more in detail to where we want to validate these fields

The check is now performed directly in generate_operational_data for all events which pertain to an ICS-04 packet: https://github.com/informalsystems/hermes/commit/3d5687eb534dc72a94c08e95e7afdfb6492d3d22

The guide template checker is failing but that's unrelated to this PR. Will fix out of band.


Prevent lower level mobs from attacking the higher level players

Level 1 to 25 monsters do not attack level 36 players.
That is how it was in original game.

As discussed on Discord before this was submitted, the aggro range should grow or shrink according to the level difference, not abruptly disappear after a given threshold.
You could try implementing it that way if you want? If not, someone else will get around to it eventually.

As discussed on Discord before this was submitted, the aggro range should grow or shrink according to the level difference, not abruptly disappear after a given threshold.
You could try implementing it that way if you want? If not, someone else will get around to it eventually.


#110 improve code quality/mongoqueryengine/queryengine

Relating to issue #110
Refactored code to switch statements and smaller functions that manage each query result and handle errors.
This should be both easier to manage as well as add to in the future.
I tested all the function implementations after running 'wails dev' and using the linux gui and it worked the same as it did when I ran against the main branch.
Please let me know if you have any suggested changes. If you're okay with the changes I'll go ahead and work on the next issue on codefactor.

I added the missing code and rechecked all the functions and cases, now they all cases check for the config.CreateLogFn and run it.
I also noticed I misnamed the receiver variables as mqw vs mqe, so I updated that to remain consistent with the other functions.

Awesome! Looks good 👍


Ruptures-based BH cutoff detection

Feel that canny edge detection is perhaps a better idea. Will try and implement that first


Cant find SCSS file

I used
import SwiperCSS from 'react-id-swiper/src/styles/scss/swiper.scss';

and cant find Module react-id-swiper/src/styles/scss/swiper.scss
Can any one please share how should I use scss file if its changed?

Hi @deependhamecha , it was removed from 1.6.7 , but I put it back from 1.6.8


Test against multiple Postgres versions

coverage: 93.853%. remained the same when pulling ca86f30643ae44c76367655aa2f4c7b3f6df57c9 on pg-versions-test into 3bb765661ed9510eab90430d35df2a222980936d on main.

Don't worry about this:

Once the build name strings stabilize, we can update the repo settings for "expected checks".


Add Nexon Launcher

SteamDB app page links to a few games using this

Mabinogi
MapleStory
Vindictus
Dungeon Fighter Online
Hyper Universe
MapleStory 2

Brief explanation of the change
Find nxsteam.exe, the executable for Nexon Steam Game Launcher.

After running changed rules in this pull request (this bot only tests rules.ini changes):
Changed technologies: Launcher.Nexon
These apps now match: (15)<PHONE_NUMBER>00<PHONE_NUMBER>40<PHONE_NUMBER>00<PHONE_NUMBER>80<PHONE_NUMBER>30<PHONE_NUMBER>40<PHONE_NUMBER>420 1711430
Please confirm that all these changed apps are correct.


New Deployment Failed

Please provide us with the following information:

This issue is for a: (mark with an x)
- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

git clone
azd up
Any log messages given by the failure

2024-05-16T13:51:50.898060210Z Writing output script to '/opt/startup/startup.sh'
2024-05-16T13:51:51.016543144Z WARNING: Could not find virtual environment directory /home/site/wwwroot/antenv.
2024-05-16T13:51:51.016580045Z WARNING: Could not find package directory /home/site/wwwroot/oryx_packages.
2024-05-16T13:51:52.405061416Z [2024-05-16 13:51:52 +0000] [54] [INFO] Starting gunicorn 21.2.0
2024-05-16T13:51:52.405893042Z [2024-05-16 13:51:52 +0000] [54] [INFO] Listening at: http://<IP_ADDRESS>:8000 (54)
2024-05-16T13:51:52.433453512Z [2024-05-16 13:51:52 +0000] [54] [INFO] Using worker: sync
2024-05-16T13:51:52.436088395Z [2024-05-16 13:51:52 +0000] [61] [INFO] Booting worker with pid: 61
2024-05-16T13:51:52.524627086Z [2024-05-16 13:51:52 +0000] [61] [ERROR] Exception in worker process
2024-05-16T13:51:52.524683288Z Traceback (most recent call last):
2024-05-16T13:51:52.524688488Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
2024-05-16T13:51:52.524692289Z     worker.init_process()
2024-05-16T13:51:52.524695289Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/workers/base.py", line 134, in init_process
2024-05-16T13:51:52.524699089Z     self.load_wsgi()
2024-05-16T13:51:52.524702089Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
2024-05-16T13:51:52.524716689Z     self.wsgi = self.app.wsgi()
2024-05-16T13:51:52.524719889Z                 ^^^^^^^^^^^^^^^
2024-05-16T13:51:52.524722789Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
2024-05-16T13:51:52.524732190Z     self.callable = self.load()
2024-05-16T13:51:52.524735390Z                     ^^^^^^^^^^^
2024-05-16T13:51:52.524738490Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
2024-05-16T13:51:52.524741690Z     return self.load_wsgiapp()
2024-05-16T13:51:52.524744590Z            ^^^^^^^^^^^^^^^^^^^
2024-05-16T13:51:52.524747590Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
2024-05-16T13:51:52.524750590Z     return util.import_app(self.app_uri)
2024-05-16T13:51:52.524753690Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-16T13:51:52.524756791Z   File "/opt/python/3/lib/python3.11/site-packages/gunicorn/util.py", line 371, in import_app
2024-05-16T13:51:52.524760091Z     mod = importlib.import_module(module)
2024-05-16T13:51:52.524763191Z           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-16T13:51:52.524766191Z   File "/opt/python/3/lib/python3.11/importlib/init.py", line 126, in import_module
2024-05-16T13:51:52.524769291Z     return _bootstrap._gcd_import(name[level:], package, level)
2024-05-16T13:51:52.524772291Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-16T13:51:52.524775191Z   File "", line 1204, in _gcd_import
2024-05-16T13:51:52.524778691Z   File "", line 1176, in _find_and_load
2024-05-16T13:51:52.524781991Z   File "", line 1140, in _find_and_load_unlocked
2024-05-16T13:51:52.524785291Z ModuleNotFoundError: No module named 'main'
2024-05-16T13:51:52.525492614Z [2024-05-16 13:51:52 +0000] [61] [INFO] Worker exiting (pid: 61)
2024-05-16T13:51:52.987981797Z [2024-05-16 13:51:52 +0000] [54] [ERROR] Worker (pid:61) exited with code 3
2024-05-16T13:51:52.996905679Z [2024-05-16 13:51:52 +0000] [54] [ERROR] Shutting down: Master
2024-05-16T13:51:52.998807239Z [2024-05-16 13:51:52 +0000] [54] [ERROR] Reason: Worker failed to boot.
Expected/desired behavior

Full Deployment.
OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
Linux (Ubuntu)

azd version?

run azd version and copy paste here.
Updated to 1.9.2

Versions

Mention any other details that might be useful

Thanks! We'll be in touch soon.

It's back with the most recent update. Thanks.

Could be. I run into an issue from the existing employment following the github update and tried to make a fresh new one to figure out the source of error. The new one did failed, which was also confirmed by the log. So I didn't go further from there. In the meanwhile, I found the github update itself got 3 failed deployments and I waited until a successful one was out. Giving it another shot with azd up didn't solve the issue through. The azd down --> azd up did.
On the other hand, upon looking at the log info., the previous failed deployment error was gone, which was the reason why I thought what you mentioned could be the case.
BTW, a side question here. It was mentioned that the Entra External ID (b2c) authentication/authorization would be worked up a few months back. I'm wondering if there's a near term plan on this? And I did figure out how to do that code wise but certain parameters was threaded deep into the few files, which would require quite some time to replace them altogether. A simple yes or no reply would make life easier. Thanks.

Entra External ID is still planned. We can't commit to a specific date as to when that would be ready unfortunately.

@mattgotteiner This is a good enough answer, thanks.


Setting up a login and signup system

There is no backend system for login and signup. The users can create their own account and keep track of all their works they have assessed till now.
A Landing page demonstrating the services offered and a route to login/signup before commencing on utilizing the products.

Tech stack: Express, Mongodb, bcrypt, Tailwind

I want to grab up this issue, if approved

I am interested in taking up this issue.

Hello @StephenStolk, thanks for raising this issue. I want you to create two separate PRs.

For the Landing Page
For the Signup page. Use Firebase for authentication. Don't push the Firebase config in this. We will configure the configs. Don't change any files in the Backend.
Thanks!

The landing page should have these Sections:

Alright. I will pull up the design after building it up.

Signup to https://www.oxiton.live/
and follow https://github.com/oxiton-foundation


Gitter/Telegram

Hey @zhuhaow,
Where to connect for any Queries/Questions? can u please provide Gitter/Telegram?
Thanks

They are all available from the readme.


Fix broken link checker

Short description

Fix some problems in the broken link checker
Proposed changes

Fix change of pagination size in broken link checker
Fix pagination in broken link checker
Show same URLs only once in broken link checker
Delete outdated link objects
Translate link checker status
Fix uncaught TypeError in bulk action script
Add chunk size option of 500

Note: I just noticed I forgot to delete old links on the XLIFF import - this I will have to change as well, otherwise outdated links might be shown in the list.
Resolved issues

Fixes: #1405
Fixes: #1413
Fixes: #1422

However, I have some more general comments and I stumbled upon one stack trace while playing updating a broken link with 3 occurrences but I'm not sure how to reproduce

I think this might have been caused by a race condition - the post_save signal is called asynchronously in a separate thread - when old links are deleted while the post_save signal is running, the database integrity seems to be violated.
I think I solved this by acquiring linkcheck's internal lock for this - could you play around with it again and see whether you observe any errors?

I tested a little more and could not find issues.


chore(master): release 1.0.0

:robot: I have created a release beep boop
1.0.0 (2022-12-08)
Features

initial commit (238e700)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wayofdev/docker-node/releases/tag/v1.0.0 :sunflower:


Undefined name 'serializers'

I meet this error! And I don't know how to solve this! It's clear that the variable 'serializers' are not declared!

Hello @datvtwkm, thank you for commenting. I think I should link some example and link to documentation, take a look at this serializer file:
https://github.com/charafau/todo_list/blob/master/lib/model/serializers.dart
and read post about serialization in built_value:
https://medium.com/dartlang/darts-built-value-for-serialization-f5db9d0f4159

Thanks, I made it!


[SPARK-19539][SQL] Block duplicate temp table during creation

What changes were proposed in this pull request?
Current CREATE TEMPORARY TABLE ...  is deprecated and recommend users to use CREATE TEMPORARY VIEW ... And it does not support IF NOT EXISTS clause. However, if there is an existing temporary view defined, it is possible to unintentionally replace this existing view by issuing CREATE TEMPORARY TABLE ...  with the same table/view name.
This PR is to disallow CREATE TEMPORARY TABLE ... with an existing view name.
Under the cover, CREATE TEMPORARY TABLE ... will be changed to create temporary view, however, passing in a flag replace=false, instead of currently true. So when creating temporary view under the cover, if there is existing view with the same name, the operation will be blocked.
How was this patch tested?
New unit test case is added and updated some existing test cases to adapt the new behavior

@gatorsmile @cloud-fan Thanks!

Test build #72670 has started for PR 16878 at commit 133bd99.

Test build #72670 has finished for PR 16878 at commit 133bd99.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72670/
Test PASSed.

Merged build finished. Test PASSed.

Test build #72800 has started for PR 16878 at commit f7253c5.

Test build #72800 has finished for PR 16878 at commit f7253c5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72800/
Test PASSed.

Test build #72834 has started for PR 16878 at commit 6683d8c.

Test build #72837 has started for PR 16878 at commit 3a989cf.

LGTM, pending jenkins, thanks for working on it!

Test build #72834 has finished for PR 16878 at commit 6683d8c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72834/
Test PASSed.

Merged build finished. Test PASSed.

Test build #72837 has finished for PR 16878 at commit 3a989cf.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72837/
Test PASSed.

Thanks! Merging to master.


minor typo in ReadME

Hello, thank you for your great work!
I found a minor typo in install pip:
pip -r install requirements.txt
to
pip install -r requirements.txt

Hello @seongsukim-ml,
Thank you for pointing out the typo in the ReadME and for your kind words and I have corrected the typo.
Your feedback is greatly appreciated. If you notice any other issues or have further suggestions, please feel free to let us know.
Best regards!


So Microsoft is now in charge of what our users can do ???

SO MS opens the door for our users to buy product without any admin control then tells us you handle it..."Organizations can then rely on their own internal policies, procedures and communications to ensure that those individuals making self-service purchases are complying with company policies."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0beb3664-94e2-fc84-15ee-184d34f0d947
Version Independent ID: 64bbd0e0-8ded-c2e0-6688-5b6a51144a40
Content: Self-service purchase FAQ
Content Source: microsoft-365/commerce/subscriptions/self-service-purchase-faq.md
Service: o365-administration
GitHub Login: @cmcatee-MSFT
Microsoft Alias: cmcatee

Exactly, it's not on.  Plus we need this off by default, and not managed via PowerShell.

@marquer Based on your feedback, we’ve adjusted our approach to better address the needs of both IT admins and end users within organizations.  We are providing IT admins a way to turn off self-service purchasing on a per product basis via PowerShell.

Why can you just have that functionality at the admin console ??
Bob
Sent from my iPhone

On Nov 1, 2019, at 10:54, Cern McAtee<EMAIL_ADDRESS>wrote:

@marquer Based on your feedback, we’ve adjusted our approach to better
address the needs of both IT admins and end users within organizations.
We are providing IT admins a way to turn off self-service purchasing on
a per product basis via PowerShell.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

LinkedIn -
http://www.linkedin.com/company/16817?trk=tyah&trkInfo=clickedVertical%3Acompany%2Cidx%3A3-1-7%2CtarId%3A1432328924645%2Ctas%3Aedward don
Twitter - http://twitter.com/EdwardDonCo
Facebook -
http://www.facebook.com/pages/Edward-Don-Company/113824285620323
Instagram - http://instagram.com/edwarddonco
YouTube - http://www.youtube.com/user/EdwardDonCo


Support rhel8 by bumping the provider

Bump the provider version to have the updated repos from https://github.com/IBM-Cloud/terraform-provider-ibm/pull/4033

Waiting on the release of ibm terraform provider 1.47.0 (non-beta)


Unable to plan after storage integration from state was destroyed in environment

Provider Version
0.25.19
Terraform Version
1.0.7
Describe the bug
Can't plan after a storage integration in the state has been removed from the environment.
To reproduce:

Create a snowflake_storage_integration using the provider
Delete the resulting integration
Either:

Execute terraform refresh to update the state
Execute a plan to change other resources
Try to plan for destruction (terraform plan -destroy)

Expected behavior
A frefresh should allow the provider to notice the removal of the storage integration and update the state
Code samples and commands
resource "snowflake_storage_integration" "abc" {
  name    = "ABC"
  comment = "Storage integration for the alphabet"
  type    = "EXTERNAL_STAGE"

  enabled = true

  storage_allowed_locations = [
  ]

  storage_provider     = "S3"
  storage_aws_role_arn = data.aws_iam_role.snowflake.id
}

show integrations;
drop integration abc;

Additional context
I'm raising an issue because I thought I could work around this by creating a PR ( #692 ) and then just going for a destroy of my environment in the meantime, but a destroy does not work either. So I'm now going try to recreate the storage intregration by hand to allow the provider to delete it again, or if that fails, delete my state and manually remove all resources mentioned there (not something I really want to have to do).

I feel stupid, but there is a simple workaround:
terraform state list
terraform state rm snowflake_storage_integration.integration

https://github.com/chanzuckerberg/terraform-provider-snowflake/pull/692 is a great fix to avoid the workaround though! Thank you for the contribution :smile:


[UI - Dashboard/Cell Editor] Variable selector value dropdown not showing values consistently

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

Create variables of type CSV, MAP and Query
Create a Dashboard and then a cell
In the cell edit view open Script Editor
Open variables.
Hover over the different variables.  A popover loads with a dropdown for the variable values
Click on the variable values dropdown in the popover

Expected behavior:
Expected to see the values for the variables as they are displayed when inspecting variables in Settings > Variables
Actual behavior:
In the CSV case the dropdown contains 1 item "None Selected" and no values are shown.
In the Map case the currently selected item is "No Results", however clicking on the drop down reveals the values.
Screencast

Related
#17146
#17101
This is similar to the closed issue #17146, in which it is mentioned that this feature was in process and being tracked from #17101
Environment info:

System info: Run uname -srm and copy the output here

Linux 4.15.0-88-generic x86_64

InfluxDB version: Run influxd version and copy the output here
Detected in nightly builds

ts=2020-03-12T09:13:10.573725Z lvl=info msg="Welcome to InfluxDB" log_id=0LVW92gG000 version=nightly commit=88a6a693f build_date=2020-03-12T09:13:10Z

Other relevant environment details: Container runtime, disk info, etc
Browser: Chrome 80

Config:
Copy any non-default config values here or attach the full config as a gist or file.
Started with e2e testing enabled
--e2e-testing=true

closed by #17360


Android curl_init_global(CURL_GLOBAL_ALL) leads to pthread_rwlock_wrlock nullpointer?

I'm using a well tested third party library for Android that requires the use of CURL . The library works great on most devices, but on some (Samsung S7 -- Snapdragon arm64-v8a) I'm getting a nullpointer dereference crash that looks like this:
Revision: '15'
ABI: 'arm64'
pid: 8597, tid: 8627, name: Native Voice BG  >>> com.foo.foo <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
Cause: null pointer dereference
backtrace:
#00 pc<PHONE_NUMBER>0698bc  /system/lib64/libc.so (pthread_rwlock_wrlock)
#01 pc<PHONE_NUMBER>1e52cc  /system/lib64/libandroid_runtime.so (CRYPTO_STATIC_MUTEX_lock_write+8)
#02 pc<PHONE_NUMBER>24923c  /system/lib64/libandroid_runtime.so (CRYPTO_get_ex_new_index+76)
When running Android Studio it appears that the crash happens when calling curl_init_global(CURL_GLOBAL_ALL). I've read that this is most likely a race condition. Is this correct?
What compile options might I try for Android to remedy this?
Note, I'm using CURL v7.69.1 running with OpenSSL v1.1.1g
Thanks!

Marking as inactive, to be closed at the end of 3.4 dev, barring further input


chore: bdd-ghfjxy-import-1585704206 to 0.0.1

chore: Promote bdd-ghfjxy-import-1585704206 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


RazorEngine throw a NotSupportedException when compiling template in Azure Function

I must compile a razor view in an azure function to send an email, but something goes wrong. I obtain the a NotSupportedException error: The given path's format is not supported.
Here my code:
private IRazorEngineService _engine;

public MyCtor(bool isTestEnvironment)
{
    TemplateServiceConfiguration configuration = new TemplateServiceConfiguration();
    configuration.Debug = isTestEnvironment;

    this._engine = RazorEngineService.Create(configuration);
}

public string GetHtmlEmailBody(string templateFileName, object emailData, string layoutFileName)
{
    //Get data type of email data
    Type emailDataType = emailData.GetType();

    string layoutFullFileName = Path.Combine(this._layoutPath, layoutFileName);
    string layoutContentString = File.ReadAllText(layoutFullFileName);
    var layout = new LoadedTemplateSource(layoutContentString, layoutFullFileName);

    this._engine.AddTemplate("layoutName", layout);

    string templateFullFileName = Path.Combine(this._templatePath, templateFileName);
    string templateContentString = File.ReadAllText(templateFullFileName);
    var template = new LoadedTemplateSource(templateContentString, templateFullFileName);

    this._engine.AddTemplate("templateName", template);

    this._engine.Compile("templateName"); //<-- Here I get the exception
    string htmlEmailBody = this._engine.Run("templateName", emailDataType, emailData);

    return htmlEmailBody;
}

Paths are similar to D:\\...\\Emails\\Templates.. I am testing locally and it does not work... I have googled and it seems that Azure Functions have some limitations in caching and in file system management, but it is not clear how can I solve the problem. I think I have same problem this person has written here
Any idea how can I solve it? There is something wrong in what I am doing?
I am using RazorEngine 3.10.0
Thank you

@csimone86 thanks for this information, it fixed the same error for me as well. I've applied your fix to a fork of the repo (although directly within the UseCurrentAssembliesReferenceResolver class) to make this temporary workaround easier for others.
https://github.com/KevinMellott91/RazorEngine