Dataset.concatenate() agrees to concat dictionaries with different keys

Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS 10.13.5
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.8.0
Python version: 3.6.5
Bazel version (if compiling from source): NA
GCC/Compiler version (if compiling from source): NA
CUDA/cuDNN version: NA
GPU model and memory: NA
Exact command to reproduce: NA

Describe the problem
It seems like Dataset.concatenate will concatenate datasets of dictionaries with different keys (values from the second key will be concatenated to the first one).
(a small demo is attached)
I've looked at python/data/util/nest.py and in _recursive_assert_same_structure it seems like _yield_value only returns values for dictionaries.
Is that intended? I would expect it to either fail or put None in the missing fields.
Source code / logs
The following code:
ds1 = tf.data.Dataset.from_tensor_slices({'f1': list(range(20)), 'f2': [1]*20})
ds2 = tf.data.Dataset.from_tensor_slices({'f2': list(range(100,120)), 'f3': [2]*20})
dataset = ds1.concatenate(ds2).batch(5)
curr_batch = dataset.make_one_shot_iterator().get_next()

with tf.Session() as sess:
  for _ in range(8):
    data = sess.run(curr_batch)
    print(data)

Results in:
{'f1': array([0, 1, 2, 3, 4], dtype=int32), 'f2': array([1, 1, 1, 1, 1], dtype=int32)}
{'f1': array([5, 6, 7, 8, 9], dtype=int32), 'f2': array([1, 1, 1, 1, 1], dtype=int32)}
{'f1': array([10, 11, 12, 13, 14], dtype=int32), 'f2': array([1, 1, 1, 1, 1], dtype=int32)}
{'f1': array([15, 16, 17, 18, 19], dtype=int32), 'f2': array([1, 1, 1, 1, 1], dtype=int32)}
{'f1': array([100, 101, 102, 103, 104], dtype=int32), 'f2': array([2, 2, 2, 2, 2], dtype=int32)}
{'f1': array([105, 106, 107, 108, 109], dtype=int32), 'f2': array([2, 2, 2, 2, 2], dtype=int32)}
{'f1': array([110, 111, 112, 113, 114], dtype=int32), 'f2': array([2, 2, 2, 2, 2], dtype=int32)}
{'f1': array([115, 116, 117, 118, 119], dtype=int32), 'f2': array([2, 2, 2, 2, 2], dtype=int32)}

ping @mrry

Nagging Assignee @shivaniag: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Good catch, thanks for finding this bug. Fixed this :)

This would now fail, closing this issue.

I still face this issue , tensorflow 2.2.

From the snippet, this failed. This is expected behavior, the issue was that earlier it agreed to concat dictionaries with different keys.
Please let us know if this behavior is not what you expect/you have a use-case for the behavior different than failing trying to concatenate dataset with different keys.


Generating an elf file

I've been using Ghidra to help try and get close to generating an elf file and I have found the entry point that I am looking for, however, I am unable to figure out what I need to do to generate the elf file. Any help would be greatly appreciated!

I was able to generate an elf file with this bash script https://gist.github.com/JayFoxRox/5eaa45b374a44ce27143388bab1753b7
What you have to do in order to make this bash script work is you have to install n64chain in order for it to correctly generate the elf file https://github.com/tj90241/n64chain. However, I have hit a roadblock. While the entry point address needed to generate the elf file is correct, the N64Recomp program still spits out an error when feeding in the toml and elf files. The game that I am using to test this is Conker's Bad Fur Day. Hopefully this helps people get further along with figuring this out, but until now, this issue is still not resolved.

What error are you getting?

What error are you getting?

This is the error that I am receiving when I give the program the toml file

Same here

And my (much more smarter than I) pal is also getting the same error.

I've been trying to same thing.  Running N64Recomp in the debugger I see the ELF's segments array is empty, size 0, so when the .data section tries to reference segment data it fails.  It appears the ELF created by these steps lacks segment data.
My experience with disassembling n64 roms is limited to that one time I made USF which was just some minor MIPS machine code hackery.  I can't say if we're entirely on the wrong path here or close to the solution.

I've been trying to same thing. Running N64Recomp in the debugger I see the ELF's segments array is empty, size 0, so when the .data section tries to reference segment data it fails. It appears the ELF created by these steps lacks segment data.

Sounds about right because I placed the elf file in Ghidra and noticed some oddities with it that didn't seem right

I also tried the process but failed just like you. Cannot wait to try Mystical ninja starring goemon out on PC, sadly the decomp project got removed from github (Author was fluvian) :(

I don't know if ghidra-generated elf files will be compatible with this tool. The elf files I've used have all come from disassemblies, decomps, or romhacks which is the main supported use case. In the future I'll be adding a mechanism for providing a symbol list in a custom format which will let you use a ROM (provided you can generate the list) instead of an elf.

What symbols are we going to need to generate a working recomp? I'm interested in applying this to Smash 64 or Smash Remix - I don't know of any decomp projects for either of those, though I'm guessing that the existence of Smash Remix implies that a symbol table for both games can be generated.

disassemblies

So, what software do you recommend? How do we use disassemblies?

I comment it more in detail here https://github.com/Mr-Wiseguy/N64Recomp/issues/17, but in general it's not something I'd really be able to provide instructions for. There's a lot of foundational reverse engineering knowledge needed to set up a disassembly that would be very difficult for me to write a guide for, but in the future there may be tools to help make that easier.

@Mr-Wiseguy Am I correct to understand that the purpose of the ELF is to contain both the binary machine code, binary data, and metadata for sections and segments to allow the tool to know where the machine code content inside the ELF is vs the data segments and what virtual memory addresses to map the binary file offset to?  If I am correct on that it sounds like the tool to use here is something like splat but I have almost no experience with REing n64 rom images.
I understand that REing is a unique case by case situation as I've done it with x86 machine code a few times so I'd just like to get the details of what is needed as an end result here. I'd also suggest, if I am correct on this, that you add a small bit to the README explaining that the ELF is being used to contain these bits of information and that currently having such data requires some existing reverse engineering effort to at least organize and label the sections and segments.
Personally, I'd like to try to port Battlezone: Rise of the Black Dogs to PC just because it would be interesting. I've extracted the game's assets in the past and converted them to work with the original PC version the game is based and ported from, so I have a lot of experience with this oddball of a game.  Looking at its data though it appears to have a very aggressive mixing of machine code and binary data meaning manually mapping out the ranges between machine code and data, if I am correct as to the need to do so, is going to be quite a process.  This is why I'm asking for clarification on what exactly is needed from the ELF before I proceed.

That's exactly right, and splat is a great tool to use for this. If you can identify the regions of the ROM that are code and what vram address they're linked against, that's most of the way there. You'll have to also identify SDK symbols, which the https://github.com/decompals/flib tool can help with, so that they can be replaced with the native reimplementations I wrote (currently found in the Zelda64Recomp repo). Using Ghidra in conjunction with the other tools may help find code, though I don't have a lot of experience using it for N64 binaries.

Additionally, the symbol input file mechanism that I'm currently working on should clarify exactly what info is needed by the recompiler. When that's done I'll update the Zelda64Recomp to use one that you can use as a reference, though an elf file may still be easier to provide (since most of the info in that file will have come from the elf).

I think it's going to be up to the community to provide support for things like dumping the elf, that might fall outside the scope of this project.
With that said, ghidra and splat should be enough to get you there with minimum knowledge of the n64 architecture/ instruction set. When I get a good, consistent method going, I'll do a write up if nobody else has.

Could a low-level emulator also be used to generate the proper symbols? It appears that the primary purpose of the ELF file is to determine which sections of the ROM are code and which are data, so an emulator could record which cartridge load calls it ends up making while it runs, and where the MIPS instruction pointer goes, while you're playing the game.

This seems like a good idea to me.  Could use a n64 emulator with good plugin support to dump the instruction calls, maybe.

Could a low-level emulator also be used to generate the proper symbols?

This seems like a good idea to me.  Could use a n64 emulator with good plugin support to dump the instruction calls, maybe.

It probably would, which would likely mean it would take longer than Ghidra and Splat if those options are viable. Though after a good playthrough, you could probably have an algorithm highlight the portions of the ROM that never got executed or loaded, and then make educated guesses on what's left - you'd really just have to make sure that every entity in the game got used at least once.
There are also enough people who do 100% glitchless speedruns of games that you might be able to get the gameplay you need simply by recording their inputs and playing them back.

This is the current step I am stuck at.
I've tried using splat to split z64 down, and that at least gives me some information for entrypoint, but where to go from here is beyond me without further technical know-how to recompile into ELF.
I was working on trying this out on Gauntlet Legends, because that game deserves some love.


[FLINK-6685] Adjust scopes of SafetyNetCloseableRegistry usage

This PR addresses remaining issues with the SafetyNetCloseableRegistry:

reset the registry to null after usage in Task.
use the registry in the scope of the async checkpointing thread.

@tillrohrmann thanks for the review and the merge!


added note on --etcd-servers-overrides

Added a note on etcd-servers-overrides option, to clarify CRDs are not supported (yet).

/assign @rajakavitha1

Thank you, @droccom!
/lgtm

Moved to kubernetes/kubernetes.


fix: CSharp: Using dependency management / partOfProperty to render optional types with question mark

Description

Using ConstrainedObjectPropertyModel to get info on if the property is required
I wasn't (still) able to test changes since tests are falling with

13:48:33 lp-0372  csharp-generate-required-properties git:(next) ✗  $ npm run start

> start
> ../../node_modules/.bin/ts-node --cwd ../../ ./examples/$npm_package_config_example_name/index.ts

/home/andrey_zaytsev/code/zaytsevand/modelina/node_modules/ts-node/src/index.ts:744
    return new TSError(diagnosticText, diagnosticCodes);
           ^
TSError: ⨯ Unable to compile TypeScript:
src/generators/AbstractRenderer.ts:10:13 - error TS2314: Generic type 'AbstractGenerator<Options, RenderCompleteModelOptions>' requires 2 type argument(s).

10   G extends AbstractGenerator = AbstractGenerator,

Related issue(s)
Fixes #941

The error has changed, but I still cannot run code on the branch (next currently):

silly me :)


Old InternetExplorer.Application code no longer working, missing howto

https://docs.microsoft.com/en-us/previous-versions/windows/desktop/ms763742(v=vs.85)
Coming from VBA in MS Access 2016 or newer and have trouble replacing code that once worked
Set objIE = CreateObject("InternetExplorer.Application")
..
strHTML = objIE.Document.All("content").innerHTML
..
Now they're talking about
Dim xmlhttp As New MSXML2.xmlhttp
xmlhttp.Open "GET", strURL, False
xmlhttp.send
I try to find help docs and code examples and find a long list of libraries, lib versions, coding methods and no VBA code or no VBA code that works.  Please slow down the "new way" and try to do it right the first time and then spend the time properly documenting it (accessible from VBA editor).

@tygrus2 "We're no longer updating this content regularly."
#please-close


npx voxa-cli interaction causes an uncaught promise

Running npx voxa-cli interaction results in an uncaught promise:
 $ npx voxa-cli interaction
npx: installed 213 in 9.978s
timeframe: 2136.128ms
(node:5637) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'replace' of undefined
    at sanitizeView (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/Processor.ts:66:8)
    at _.chain.reduce (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/Processor.ts:96:21)
    at arrayReduce (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/node_modules/lodash/lodash.js:683:21)
    at Function.reduce (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/node_modules/lodash/lodash.js:9683:14)
    at /Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/node_modules/lodash/lodash.js:4374:28
    at arrayReduce (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/node_modules/lodash/lodash.js:683:21)
    at baseWrapperValue (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/node_modules/lodash/lodash.js:4373:14)
    at LodashWrapper.wrapperValue (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/node_modules/lodash/lodash.js:9052:14)
    at voxaSheetsViews.map (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/Processor.ts:107:8)
    at Array.map (<anonymous>)
    at Object.viewsProcessor (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/Processor.ts:77:26)
    at new Schema (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/Schema.ts:42:18)
    at new DialogflowSchema (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/DialogflowSchema.ts:56:5)
    at Object.<anonymous> (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/src/InteractionBuilder.ts:109:20)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/mat/.npm/_npx/5637/lib/node_modules/voxa-cli/lib/src/InteractionBuilder.js:4:58)
(node:5637) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:5637) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

It doesn't seem like IVoxaSheet.data is checking if it has a value property defined before running it through the sanitizeView function.

Turns out it's caused by empty cells in the sheet, but these could be skipped during processing


Data download and process

Thank you for your awesome work.
I tried using your script to download data, but many of the data indicated that the source could not be found. I tried to download the data from PATS again, but after decompression, it was all in H5 format. Can you tell me how to obtain the corresponding video and audio files? Or how to extract corresponding videos and audio from these H5 files?

hii @GruelMi
First of all, the h5 file provided by PATS only contains pose information, while the raw audio and video can only be downloaded via the url in the meta file cmu_intervals_df.csv in data-preparation folder.
Actually, the full meta file cmu_intervals_df.csv provided by PATS contains many videos that can't be downloaded due to permissions and other issues, and I've removed this part of the url when I preprocessed the data.
At least when I processed the data the url's corresponding to the interval id's in filtered_intervals.json were available for download. Maybe it's because some of the url's permissions have changed again after a while.
Can you provide me with some url that is not downloadable, and I'll try to see if it's a problem with the url itself? If there are still too many videos that can't be downloaded, please contact<EMAIL_ADDRESS>

Thanks for the detail response! As discussed in email, the key is to download the tool: yt-dlp.


Added lowest common ancestor for BST

Added lowest common ancestor algorithm and tests to the BST.

[x] Read the contribution guidelines
[x] Added unit tests (or unit tests have already been added)
[x] Updated the Contents

Closes #167, closes #154.

@camilne Thanks a lot for sending a PR! I'm currently working on implementing graphs, so I'll review your PR right after that's complete. Sorry for the delay.

I would like to contribute Lowest Common Ancestor(for any tree - O(lg N)).  If no, to which folder should I add the file?

Hi @chamow97. To answer your question above, please feel free to open an issue for lowest common ancestor. Just know that our current priority is #163, then we'll move onto graph stuff after that.


Automated package installation test in CI

Describe the feature
The project is currently tested on a separate machine to ensure that the package can be installed and used properly. It needs to be included in the CI so that third-party users of this library should not face any inconveniences.
Use Case
This will ensure that the package is always installable for users.
Proposed Solution
Add a sample project and a separate workflow file that runs this.
Other Information
No response
Acknowledgements

[X] I may be able to implement this feature request
[ ] This feature might incur a breaking change

Version used
Latest
Environment details (OS name and version, etc.)
All environments.

Implemented in CI for both GitHub and VPM.


Qunit tests for inverse selection of contenteditable elements fixed

see comment here

@testcafe-build-bot \retest

This PR is waiting for https://github.com/DevExpress/testcafe-hammerhead/pull/1307

@testcafe-build-bot \retest


Remote Cache Warning GRPC

DEADLINE_EXCEEDED: deadline exceeded after 59.999913100s. [closed=[], open=[[buffered_nanos=33620, ....]]

Hi 👋  We've been using this remote cache backed by s3 and have been recently seeing timeouts in grpc. (blob based s3 storage) with 3 instances of the service running with 4 vcpu. running on version 2.3.9.
Our CPU and memory usage are only 80 & 30% peaks respectively. I don't have a reliable repro for this, but was wondering if you had any insight as to what could be going wrong here?

Do you see any errors in bazel-remote's logs when the client shows these timeouts?
Do you have iowait monitoring on the bazel-remote machine and on the s3 storage (if it's something you're running locally)? If you see iowait spikes when then maybe your storage bandwidth is saturated.

@mostynb, yeah, we see a lot of GRPC BYTESTREAM READ FAILED
our requests increased when we added
coverage --experimental_fetch_all_coverage_outputs
coverage --experimental_split_coverage_postprocessing

Do you have iowait monitoring on the bazel-remote machine and on the s3 storage

I don't believe so, I can take a look here.
In general, is it better to scale vertically or horizontally? Currently we have 3 replicas on ECS

yeah, we see a lot of GRPC BYTESTREAM READ FAILED

Are there any more details provided in the logs besides bytestream read failed and the resource/blob name? If so, could you share a few of them here?

In general, is it better to scale vertically or horizontally? Currently we have 3 replicas on ECS

The REAPIv2 cache service has strong coherence requirements, and bazel doesn't behave nicley when those assumptions fail. eg bazel builds can fail if they make a request to one cache server, then make a request to another server (with a different set of blobs) during the same build. This makes horizontal scaling risky, unless you arrange things in such a way that a given client only talks to a single cache server during a single build.
Evicting items from bazel-remote's proxy backends can also break these assumptions. To avoid this we would need to figure out a way to imlpement some sort of LRU-like eviction for S3 (but I don't have an AWS account to do this work myself).


Crate duplication

Hey,
I'm one of the maintainer of https://github.com/rust-openvr/rust-openvr
I stumbled by chance to your repository.
I don't think it's necessary to have two different repo for one binding, our goal is probably the same.
Is there any chance we could merge together? Is there any functionality that you definitely need right now that is not available within rust-openvr?
I'm spending quite some time in to openvr and I just want the best, and I think splitting the community in two is not the way to go. Especially while valve is updating the specification quite often.
Have a nice day.

Your project looks good, I will poke around with it when I have some time, I think it makes sense to have only one crate.


README: fix typo in softether name

Someone linked project on hypeirc and when skimming readme that typo really bugged me ;)

Thanks.


F&O should support footers

F&O supports grouping with a header, but no footer.
A footer is needed to display summary information not known when writing the header, for example the size/count of the files, similar to:
 Volume in drive C has no label.
 Volume Serial Number is A61F-3821

 Directory of c:\windows\System32\WindowsPowerShell\v1.0

09/13/2016  09:46 AM           446,976 powershell.exe
09/13/2016  09:46 AM           212,992 powershell_ise.exe
               2 File(s)        659,968 bytes
               0 Dir(s)  23,720,255,488 bytes free

For clarification, what is F&O?

Format and output.


[refactor] Deprecated linker argument in compile function to be passed instead through a prelink function

[refactor]: deprecated linker argument in compile function to be passed instead through a prelink function.
description: per Angular docs, implementing a transclude/linker function passed to the compile function is deprecated and should instead be passed through the link function.

Interesting timing.. :)

That's just uncanny.


Wrong review date time for SDK and API Review Github Issue

User scheduled a SDK Review meeting for 9/21/2022 2PM . The calendar meeting was sent correctly, but the GH issue displayed 4PM.
The meeting in the Data Verse is correctly stored.
i.e email:

Github issue:

Same happens for API meetings. The error is also 2h

PS: I went and manually updated the issues so it has the correct information

We decided to remove the field Review date completely and follow up with our admin users how they want us to display the date

I've removed Review date from the GH tracking issue


[HOLD] Enable React Concurrent Mode

Problem
There are a number of performance issues that we have long-term aspirations to solve via the React concurrency features introduced in React 18+. This was a big portion of the motivation to enable the new React Native architecture.
While we are still working on the New Arch on iOS and Android, we already have the ability to enable React concurrent mode on web. However, during the RNW 0.19 upgrade, it was found that some aspects of our codebase, particularly withOnyx, are not compatible with concurrent mode, so we turned it off.
Solution
Let's investigate what it will take to re-enable React Concurrent Mode and start working on that in parallel to the New Arch upgrade, which is well over a year in the making and should land hopefully by the end of January.

Reassigning to @muttmuure for tracking as part of the NewDot Performance project

Put out a call for volunteers: https://expensify.slack.com/archives/C05LX9D6E07/p1703292300758289

Still looking for a volunteer, but I assume the people I tagged in slack may be partially OOO. This will remain on HOLD for this week, so I'll bump next week if I haven't heard anything

On HOLD for New Arch

On HOLD for New Arch

Still on HOLD

Though, we could try to get a volunteer to start looking into enabling this on web to get a headstart?

Ready for assignment :)

For now, the only platform that's ready for this is web, so you can focus your efforts there to start (while we're working on New Arch)

Hello @roryabraham !
I spent around 3h today on investigating it and wanted to share my thoughts.
I would split the work here into 2 parts.
The first one is making sure onyx is ready for the concurrent rendering and the second one is that all of our components in the app don't have any side effects that could break concurrent rendering.
When it comes to onyx one I see 2 potential issues:

The first one is that we have optimistic reads from cache in withOnyx component. Because concurrent rendering can (and most likely will) split the rendering into multiple parts it's possible that some of the components will read newer state than the others. We probably will need to use useSyncExternalStore here (Maybe by wrapping withOnyx in functional component). We already have allowStaleData option so maybe reading inconsistent state is fine in this case.

Second one is of course sending updates to withOnyx components. What can happen now is that we will call setState on the first withOnyx listener but the rest will be batched later so again we ends up inconsistent state as some components already know the newer value. I remember that someone was already working on adding Jotai to onyx. Jotai already has support for react concurrent rendering and would solve that problem. What's the current state of it? If we don't plan to use Jotai in onyx we will need to use useSyncExternalStore hook.

For the second part (so making sure all components don't have any dangerous side effect) we can use StrictMode component from React (https://react.dev/reference/react/StrictMode#strictmode) and just traverse the whole app too everything works properly and we don't have any warning on the console. I already added that locally and the report screen stopped to work.

The first one is that we have optimistic reads from cache in withOnyx component ... We probably will need to use useSyncExternalStore here

Fortunately we have an issue to implement a useOnyx hook and it's happening! The current proposal is to use the useSynxExternalStore hook 👍🏼

I remember that someone was already working on adding Jotai to onyx ... What's the current state of it? If we don't plan to use Jotai in onyx we will need to use useSyncExternalStore hook.

This is the first I've heard of adding Jotai to Onyx, but as I just mentioned we'll be setting up useSyncExternalStore soon. Maybe we'll have to update withOnyx to internally use useOnyx?

For the second part (so making sure all components don't have any dangerous side effect) we can use StrictMode component from React (https://react.dev/reference/react/StrictMode#strictmode) and just traverse the whole app too everything works properly and we don't have any warning on the console

👍🏼 to enabling strict mode on dev. Maybe with a config-level flag to easily enable/disable it so we can turn it off and on easily.

I already added that locally and the report screen stopped to work

Great 🙃
So it sounds like this is the best thing to investigate next? Or do you think you'll be blocked on Onyx needing to use useSyncExternalStore?

I spent another 2.5h to check what was the problem with report screen.
It turned out that react navigation is not updating the params because it things the component has been unmounted.

It was recommended in the docs (https://es.legacy.reactjs.org/blog/2022/03/29/react-v18.html) to turn on the StrictMode but looks like it checks cases that don't really occur on production with concurrent mode.
That's true that useMemo can be called multiple times with the same deps in concurrent mode but that shouldn't be true for useEffect. The StrictMode does checkes other important things. For instance Gesture Handler seems to use findNodeHandle that is deprecated in react 18. But for now I would wait for the new useOnyx hook that is based on useSyncExternalStore before we test the app further.
In the meantime I will double check if running useEffect twice (with empty dips array) is actually possible on production wit the concurrent mode on.

Thanks @Szymon20000!

For instance Gesture Handler seems to use findNodeHandle that is deprecated in react 18

Is there any upstream issue to clean this up? If not, can we create one?

@roryabraham Not yet I think but I will open one. Looks like we can open another one in react navigation as that library doesn't work with StrictMode.
I asked the question about StrictMode in react repo https://github.com/facebook/react/issues/28347 and they replied that calling useEffect twice can happen now with fast Refresh and will happen in the future on production with offscreen API. Some other hooks can be already be called twice on production with preserving the state.
I think for now we can just patch react-navigation and open them a pr with that simple change that will make their problematic useEffect symmetrical. Then use it in our app to catch side effects that won't work with concurrent rendering. If at certain point we will see that one of our deps doesn't work well with strictMode we can just remove it and continue testing the app with concurrent mode manually. The only issue is that a lot of concurrent cases are rare and hard to catch without the StrictMode component.

Thanks for plugging away at this. Would love to see upstream issues created with minimal reproductions in react-navigation and RNGH. That would be great next steps and either you can work on fixing them if you choose or we can ask the maintainers to get involved. Those sound like steps we'd want to take before we add new patches imo

any updates in the last week?

Hello @roryabraham !
We are waiting for useOnyx hook to avoid false-positive StrictMode issues. In the meantime @perunt will open issues in react-navigation as well as in RN-gesture-handler.

great, going to place this on HOLD for https://github.com/Expensify/react-native-onyx/pull/462

@roryabraham can we take this off hold now that the linked PR is merged?

Are we going to include this in the Onyx bump? https://github.com/Expensify/App/pull/38114

yeah, let's move this to be on HOLD for https://github.com/Expensify/App/pull/38114

useOnyx has landed in E/App - taking this off HOLD

Sounds like @chrispader is going to be taking this one over soon: https://expensify.slack.com/archives/C035J5C9FAP/p1713477420910129?thread_ts=1713455927.317579&cid=C035J5C9FAP

this is on the backlog - it does promise some good performance boosts but isn't particularly urgent for the roadmap

The feature I'm personally most excited for is useTransition - a hook that allows you to perform state updates like useState, without blocking the UI while those updates are happening.

Not overdue

@Szymon20000 Thanks for the exploration on this one so far.
Now as we are in a better place with the New Architecture, we would like to switch gears on this one and keep up the pressure so we can use React 18 and its relevant features. We would like to keep the limited resources of Margelo focused on the comment linking/ pagination which is high priority, so we are going to move this task over to different agency with goal of getting daily updates on next steps here.

Hey @mountiny @roryabraham !
That sounds good. We've been struggling to make time for this feature so this makes sense! We'd want to push this as fast as possible now that New Arch enabled! 🙌🏼
We're actually talking to @WoLewicki at App.js Conf about pushing this forward right now, so handing this over is gonna be smooth...

Thanks for understanding, we agree that it would be great to make a steady daily progress on this task now.

Linking also an issue to research the react compiler which is going to be related https://github.com/Expensify/App/issues/42211

hey I will be taking over and working on this :)

I started to look at this today (only 3-4hrs):
Good:

if we turn on the concurrent mode, web app builds and runs and there are no obvious problems (although might be smaller ones, will continue to test)

Bad

while app works on concurrent mode, adding <StrictMode> breaks part of the app, and according to what I see the culprit is react-navigation, which doesn't work with StrictMode. So basically consistent with this previous finding: https://github.com/Expensify/App/issues/33531#issuecomment-1951879159
Relevant issue in react-navigation about strict mode: https://github.com/react-navigation/react-navigation/issues/10988
I suspect just like described above the withOnyx hook might still be problematic but I need to investigate further - when I wrapped a new SearchPage screen in StrictMode it worked - this is a pretty fresh screen that uses only useOnyx hook - so this change is definitely promising

Technically we don't NEED StrictMode for the app to work. It's a dev mechanism to find potential bugs, but I think its unsafe to move forward without it.
This is all preliminary and I will provide more updates as I dig further into this.

Today a short update from me: most of the day I had to take care of conflicts on my PR related to Search which had a high priority as Search is on a deadline.
Next week I will be back with more updates.
That being said in my opinion the next biggest step to move this forward is to try and migrate the app onto useOnyx hook, as this hook is using useSyncExternalStore which is the only safe way to read data in concurrent React.
I will start research how easy it will be to do this, most likely via reimplementing withOnyx hoc using useOnyx hook.

I will start research how easy it will be to do this, most likely via reimplementing withOnyx hoc using useOnyx hook.

This seems doable in the short-term. withOnyx returns null if data is still loading from Onyx, but useOnyx returns an isLoading state. We could use useOnyx internally in withOnyx, and use that isLoading state and to return null early in withOnyx
I also might like to see us invest more in the "real" fix which would be to migrate our E/App components from withOnyx to useOnyx. That hasn't been a priority in the past because (outside of it being hard to use with TypeScript), there hasn't been a real problem with withOnyx. Maybe we could consider doing what we did for the light theme migration, and create a codemod / babel script that automatically converts components from using withOnyx to useOnyx? That could end up actually being pretty easy to do technically - the challenge is that we don't know when the render-blocking behavior of withOnyx matters or not...

withOnyx returns null if data is still loading from Onyx, but useOnyx returns an isLoading state. We could use useOnyx internally in withOnyx, and use that isLoading state and to return null early in withOnyx

@chrispader is now working on updating this, and Onyx will return undefined if the key is not available yet. This will definitely require some changes in the app https://github.com/Expensify/react-native-onyx/pull/554
I agree with rory though that we should maybe invest more effort into migrating over to useOnyx from withOnyx

Yeah I also agree that in the long run fully switching to useOnyx is the best way and definitely worth it. The problem with withOnyx is that it might affect how the app works and it might not - you'll never know. It might work right now but break in a month after some new code is merged.
However I assumed that it would take too long to rewrite everything to the hook so I thought that rewriting withOnyx might be faster. Hard for me to guess how difficult the codemod might be - but I think non trivial since types of all the props will change and those would have to be updated manually perhaps? - with HOC you wrap component so it has a combination of props, hooks you use inside so the values from onyx no longer enter via props.
Also I'm unsure about: https://github.com/Expensify/react-native-onyx/pull/554 - we would have to migrate web onto this right? since not returning null sounds like BC - do I understand correctly?
What do you guys think I should focus now? I think the main difficulty right now is discerning what app problems come from react-navigation and what come from usage of withOnyx.

small update from me:
For the last 2 days I didn't spend a lot of time on this task, because I was finishing up sorting in Search and I also tried to dig up my old PR here: https://github.com/Expensify/App/pull/39072 - which is way overdue.
But now I'm almost done and tomorrow I will focus much more on concurrent React.
I have one finding, which I already discussed with @blazejkustra and wanted to discuss here:

The new useOnyx hook allows us to connect to 1 onyx value/collection (see: https://github.com/Expensify/react-native-onyx/blob/main/lib/useOnyx.ts#L78)
However withOnyx hoc allowed us to subscribe to multiple keys which we often do (case in point: https://github.com/Expensify/App/blob/main/src/pages/home/ReportScreen.tsx#L734-L765)

Because React hooks cannot be added/removed dynamically, actually rewriting internal implementation of withOnyx to use useOnyx will be non trivial and flaky. You can't just call useOnyx in a loop within the component. I believe it technically can be done, but it's not code that we would want on production anyway. Perhaps only for local testing.
So the cleaner way forward would be to rewrite app fully to useOnyx.
Because each case of 1 call to withOnyx will map to multiple usages of useOnyx hook, I think it will be safer to do it in batches manually by developers. Codemod might help, but IMO this migration will require a lot of extra manual tweaking.
My plan for next 1-2 days is to dig into the App and try to discern how much of possible problems comes from react-navigation vs from withOnyx usage. The migration might take a while, so first I want to make sure it's worth it - perhaps concurrent features would still work even on our current code 🤔 🤷
Meanwhile please tell me if you want me to focus on something more specific or if you have any more questions.
@roryabraham @mountiny

@Kicu thanks for the summary, I think your next steps sound good and we would eventually want to migrate to useOnyx but it will take a bit indeed.

I spent some hours in the past 2 days digging into this.
new findings
I have found a root cause of a few issues and that is actually something new - animations from react-native-reanimated, which seems to misbehave when inside <StrictMode>. Or at least some of our use cases break. This is something that I will be focusing on next week and I will definitely ask for help OS devs from SWM how maintain this library.
This is important because some of the animations result in big problems for the app. For example we are animating ReportScreen when switching chats on desktop from opacity 0 to 1 and if this animation glitches, opacity stays as zero, thus the user is not able to see the report.
status
However once I understood this is the problem and disabled/commented out some of the animations the whole App seems to behave mostly correctly, if a bit glitchy on the visual end.
The good news is that I now believe that the usage of withOnyx is actually not a big problem right now - even though in future we should definitely migrate to useOnyx sooner rather than later.
Also react-navigation seems to behave correctly after applying a small patch, however that is one more thing to investigate next week. Not sure yet whether the patch is sufficient and whether it doesn't break other things inside navigation - because it's quick&dirty right now.
glitching animation of FAB
https://github.com/Expensify/App/assets/3929868/657a5d1c-2db0-4fd8-a2bd-e7a3eaf4187e
glitch then ReportScreen stays at opacity 0

More updates to come next week.

Yesterday I announced the official deprecation of withOnyx, so at least we should not allow any new uses.

Thanks for these updates!

Small update from me:
new findings
With the help of a developer from reanimated OS team I was able to patch parts of reanimated lib that solved a lot of our problems. They will push this fix soon to upstream, so we should be able to update.
There is 1 animation bug right now which I temp-solved by commenting out this animation. (It's the exit fade out animation for mentions list): https://github.com/Expensify/App/pull/42592/files#diff-82ae7ae5193fe8f0f5d3cc064489bd3fdb78b2fc9738a7649a510fbf5864fe12R89
Now I'm fighting a very complicated bug that is 90% related to react navigation, where if use enters on root url / then they are not redirected to last visited reportID - which they should be.
Because StrictMode renders every component twice and then runs useEffect twice - these kinds of small bugs are hard to understand.
I'm giving myself time till tomorrow to try and understand this, and if not I will somehow move forward.
I expect to come back with bigger update about next steps in 1st half of next week.

For the past 3 days I've been mostly trying to understand how react-navigation breaks on StrictMode, because that was our biggest blocker.
Right now in StrictMode entering initial screen from deeplinks does not work, and you always get redirected to Home.
I have now understood mostly what happens (with great help from @adamgrzybowski 🎉 ) and I created a pure reproduction of this error here: https://github.com/Kicu/react-navigation-strictmode
Next steps is to show it to maintainers of the library and ask them for help or fix.
For us temporarily I can do some quick-and-dirty patch that will make the app work.
Other than react-navigation I have 2 patches for reanimated that should also make the lib work with StrictMode.
I think I need 1-2 more days to have a version of this that is ready to be tested by more people.
As usual more updates will come in a few days 🙃

Will concurrent mode improve composer behavior? I've noticed if you type while heavy commands are pending (like OpenApp) the focus is erratic and jumps back to the start position.
If you compare with other apps like Slack, focus is much more stable

@muttmuure
Maybe it could help, however I don't know composer related code well enough to say that with certainty. Soon we will be able to test the app and we will see what happens.
However this task is mostly about web version of the App, since on native I believe the New Architecture should already be running in concurrent mode so if you notice composer slowness there we might need to dig deeper into this.
But we can keep exploring this in future

Yep I agree its best now to ask QA to test this, there are some other adhoc build testing going on now so it might betake a week to complete

Great, the PR has just been merged. Soon, we'll want to follow-up with an investigation in #newdot-quality about how features such as Suspense and useTransition can be used to improve the responsiveness of the app

Upwork job got mangled

$250 for @rayane-djouah

Offer sent

Paid


Surprising difference in ?: and ?~ inference

When using ?~  ... instead of ?:  =(~ ...), inference seems to lose the list-ness of the test target. I would expect these to be equivalent. This causes problems using std lib list functions on the result.
> =col |=  l=(list @)
  ?:  =(~ l)
  [| -:!>(l)]
  [& -:!>(l)]
> =sig |=  l=(list @)
  ?~  l
  [| -:!>(l)]
  [& -:!>(l)]

> (col ~)
[%.n #t/it(@)]
> (sig ~)
[%.n #t/$~]

> (col ~[1])
[%.y #t/it(@)]
> (sig ~[1])
[%.y #t/{i/@ t/it(@)}]

(urbit 0.7.0, zuse %309)

Closing this. I'll make a separate issue for the TMI problem that has seen good discussion here.


Add Code of Conduct

This PR adds a Code of Conduct to the project. Would love to hear everyone's input on this before merging. @jalcine, @mrmrs

LGTM


feat(controller): Store artifact repository in workflow status. Fixes #6255

Signed-off-by: Alex Collins<EMAIL_ADDRESS>Fixes #6255
Changes:

Store the artifact repository in the workflow status.
We don't need to to a Get configmap for each reconciliation (up to 50% reduction in requests).
Clients can use the field to find the resource.
If the underlying configuration changes, you can still look up the value.

Example:
  status:
    artifactRepositoryRef:
      artifactRepository:
        archiveLogs: true
        s3:
          accessKeySecret:
            key: accesskey
            name: my-minio-cred
          bucket: my-bucket
          endpoint: minio:9000
          insecure: true
          secretKeySecret:
            key: secretkey
            name: my-minio-cred
      configMap: artifact-repositories
      key: default-v1
      namespace: argo
    conditions:
    - status: "True"
      type: PodRunning
    finishedAt: null
    offloadNodeStatusVersion: fnv:1056331912
    phase: Running
    progress: 0/1
    startedAt: "2021-07-07T22:48:59Z"

Thank you! This is exactly what we wanted in #6255.
Several side effects that come with the change seems ideal for argo itself too (perf improvement, stable handling of artifact repository config change).

@jessesuen I need your review on the proto changes for this v3.1 fix. It is to enable Kubeflow Pipelines to use v3.1.


Optimize vit model and some layers

Signed-off-by: Jael Gu<EMAIL_ADDRESS>Refactor and optimize vit model;
Update some vit related layers to have better docstring and support TimeSformer model

/lgtm
/approve
we will check the unit tests after some cleanup @jaelgu @junjiejiangjjj


HPA scale down

it takes several minutes for HPA to scale back to one node
can we make it faster?

Description
What:
Why:
When:
Where:
Tasks

[ ]
[ ]
[ ]
[ ]
[ ]

Acceptance Criteria

Constraints

N/A

References:

References #

closing, reference https://github.com/retaildevcrews/wcnp/issues/674


add the double where trick to 'sharp bits'

It might be worth adding the double where trick to JAX - the sharp bits. (I looked there first, before eventually finding an explanation for my error here.)

Originally posted by @Sohl-Dickstein in https://github.com/google/jax/issues/1052#issuecomment-1295824996

And/or: we can suggest the use of vmap-of-cond, which naturally avoids the problem.

Do we have any action items here? Is the existing discussion insufficient or too hard to find?

I think this is addressed.


Remove-DbaBackup should probably be renamed to Find-DbaBackup

Then we can just do Find-DbaBackup | Remove-Item
Or perhaps we need a Remove-DbaItem that tries Remove using account then uses xp_delete_file.
Or perhaps Remove-DbaBackup should be a wrapper around Find-DbaBackup
1.0 milestone

That's something I really wanted but I find myself costantly bashing around the fact that deleting a single file is not possible in older versions....Anyway I can work on it with a copy of a copy of a copy of a copy of a backup of a copy of a backup of a copy of my backup files. ^_^

not sure where we're at with this 🤔 @cjsommer you have any suggestions?

I must have missed this issue when it was opened. I actually kind of like the thought of having a Find-DbBackup. We could still keep the Remove-DbaBackup, but as suggested just have it as a wrapper around it. Will give this some more thought.

I'll do this ASAP. Proposal is to do just a find- what does whatever is doing now without the remove part. The original function should be deprecated and just left as a placeholder


ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]

Checklist

[ ] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2021.01.08
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', '140', '-x', '--audio-format', 'mp3', 'https://youtu.be/Lnezh5wQQQ4']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.01.08
[debug] Python version 3.9.0 (CPython) - macOS-10.15.7-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.1-tessus
[debug] Proxy map: {}
[youtube] Lnezh5wQQQ4: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1122)> (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1122)')))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 632, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2248, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/youtube_dl/utils.py", line 2734, in https_open
    return self.do_open(functools.partial(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1345, in do_open
    raise URLError(err)

Description

Not sure when, but between 20200101 and 20200113 the SSL failed certificate error creeped in again. adding '--no-check-certificate' seems to work, but this a temporary workaround rather than a fix. the command used is
youtube-dl -f 140 -x --audio-format mp3 https://youtu.be/Lnezh5wQQQ4

I am unable to reproduce this issue. Are you sure the url is playable in your browser?

I am unable to reproduce this issue. Are you sure the url is playable in your browser?

I am also able to reproduce this on macOS Big Sur 11.1 (20C69). Python 3.9.1 was installed using MacPorts. I am able to play the same video in my browser with no issues.
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://youtube.com/watch?v=<redacted>']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.01.08
[debug] Python version 3.9.1 (CPython) - macOS-11.1-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] <redacted>: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)> (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))
  File "~/Library/Python/3.9/lib/python/site-packages/youtube_dl/extractor/common.py", line 632, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "~/Library/Python/3.9/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 2248, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "~/Library/Python/3.9/lib/python/site-packages/youtube_dl/utils.py", line 2734, in https_open
    return self.do_open(functools.partial(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1345, in do_open
    raise URLError(err)

I am also able to reproduce this on macOS Big Sur 11.1 (20C69). Python 3.9.1 was installed using MacPorts. I am able to play the same video in my browser with no issues.
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://youtube.com/watch?v=<redacted>']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.01.08
[debug] Python version 3.9.1 (CPython) - macOS-11.1-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] <redacted>: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)> (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))
  File "~/Library/Python/3.9/lib/python/site-packages/youtube_dl/extractor/common.py", line 632, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "~/Library/Python/3.9/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 2248, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "~/Library/Python/3.9/lib/python/site-packages/youtube_dl/utils.py", line 2734, in https_open
    return self.do_open(functools.partial(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1345, in do_open
    raise URLError(err)

I am unable to reproduce this issue. Are you sure the url is playable in your browser?

Yes... I just tried it and it plays just fine on a browser. The URL is not really relevant, it happens with every single URL that I try.

I am unable to reproduce this issue. Are you sure the url is playable in your browser?

Yes... I just tried it and it plays just fine on a browser. The URL is not really relevant, it happens with every single URL that I try.

Hmm, that is strange. I just checked again and mine worked just fine. I'm on Win10 running python 3.4.4. Here's my log for reference.
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', '140', '-x', '--audio-format', 'mp3', 'https://youtu.be/Lnezh5wQQQ4', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.01.08
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 4.3.1-2021-01-01-full_build-www.gyan.dev, ffprobe 4.3.1-2021-01-01-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube] Lnezh5wQQQ4: Downloading webpage
[youtube] {18} signature length 108, html5 player 9f996d3e
[youtube] {136} signature length 108, html5 player 9f996d3e
[youtube] {247} signature length 108, html5 player 9f996d3e
[youtube] {398} signature length 108, html5 player 9f996d3e
[youtube] {135} signature length 104, html5 player 9f996d3e
[youtube] {244} signature length 108, html5 player 9f996d3e
[youtube] {397} signature length 104, html5 player 9f996d3e
[youtube] {134} signature length 108, html5 player 9f996d3e
[youtube] {243} signature length 108, html5 player 9f996d3e
[youtube] {396} signature length 108, html5 player 9f996d3e
[youtube] {133} signature length 104, html5 player 9f996d3e
[youtube] {242} signature length 108, html5 player 9f996d3e
[youtube] {395} signature length 108, html5 player 9f996d3e
[youtube] {160} signature length 108, html5 player 9f996d3e
[youtube] {278} signature length 108, html5 player 9f996d3e
[youtube] {394} signature length 108, html5 player 9f996d3e
[youtube] {140} signature length 108, html5 player 9f996d3e
[youtube] {249} signature length 108, html5 player 9f996d3e
[youtube] {250} signature length 108, html5 player 9f996d3e
[youtube] {251} signature length 108, html5 player 9f996d3e
[debug] Invoking downloader on 'https://r4---sn-ipoxu-u2xd.googlevideo.com/videoplayback?expire=1610657716&ei=VFsAYNbYLsTOqQH9hrPoDQ&ip=<IP_ADDRESS>&id=o-AP33dmb7Q9Pll4Rk1ShHnVb1sn0Wj8UzTTLvaTz709-e&itag=140&source=youtube&requiressl=yes&mh=tg&mm=31%2C29&mn=sn-ipoxu-u2xd%2Csn-un57en7l&ms=au%2Crdu&mv=m&mvi=4&pcm2cms=yes&pl=21&gcr=tw&initcwndbps=1108750&vprv=1&mime=audio%2Fmp4&ns=080Xx8Lyk7_8iNaUMfb3utYF&gir=yes&clen=3513581&dur=217.060&lmt=1578240876526034&mt=1610635742&fvip=4&keepalive=yes&c=WEB&txp=5531432&n=fZZ91gbdKDFnXg&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cgcr%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgA5hsJjEEaQwHzEHYXpJa0IiGCnmmleqR_DHLUP-uIkcCIBd7SWLWq0Y4rPBzcSimcSqBexVZ8pk5Bu6qNFn-z1Uf&sig=AOq0QJ8wRQIhAOJXSP_70sGFkadxWlxLR7yAYeP2qik7GcxHavv8A1e2AiBkkUitqNfphWi08HH4gKfnLdbgfU8JN-9Ri0tpHwO4ZQ==&ratebypass=yes'
[download] Destination: Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a
[download] 100% of 3.35MiB in 00:00
[ffmpeg] Correcting container in "Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a" -c copy -f mp4 "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.temp.m4a"
[debug] ffmpeg command line: ffprobe -show_streams "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a"
[ffmpeg] Destination: Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.mp3
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a" -vn -acodec libmp3lame "-q:a" 5 "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.mp3"
Deleting original file Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a (pass -k to keep)

Hmm, that is strange. I just checked again and mine worked just fine. I'm on Win10 running python 3.4.4. Here's my log for reference.
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', '140', '-x', '--audio-format', 'mp3', 'https://youtu.be/Lnezh5wQQQ4', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.01.08
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 4.3.1-2021-01-01-full_build-www.gyan.dev, ffprobe 4.3.1-2021-01-01-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube] Lnezh5wQQQ4: Downloading webpage
[youtube] {18} signature length 108, html5 player 9f996d3e
[youtube] {136} signature length 108, html5 player 9f996d3e
[youtube] {247} signature length 108, html5 player 9f996d3e
[youtube] {398} signature length 108, html5 player 9f996d3e
[youtube] {135} signature length 104, html5 player 9f996d3e
[youtube] {244} signature length 108, html5 player 9f996d3e
[youtube] {397} signature length 104, html5 player 9f996d3e
[youtube] {134} signature length 108, html5 player 9f996d3e
[youtube] {243} signature length 108, html5 player 9f996d3e
[youtube] {396} signature length 108, html5 player 9f996d3e
[youtube] {133} signature length 104, html5 player 9f996d3e
[youtube] {242} signature length 108, html5 player 9f996d3e
[youtube] {395} signature length 108, html5 player 9f996d3e
[youtube] {160} signature length 108, html5 player 9f996d3e
[youtube] {278} signature length 108, html5 player 9f996d3e
[youtube] {394} signature length 108, html5 player 9f996d3e
[youtube] {140} signature length 108, html5 player 9f996d3e
[youtube] {249} signature length 108, html5 player 9f996d3e
[youtube] {250} signature length 108, html5 player 9f996d3e
[youtube] {251} signature length 108, html5 player 9f996d3e
[debug] Invoking downloader on 'https://r4---sn-ipoxu-u2xd.googlevideo.com/videoplayback?expire=1610657716&ei=VFsAYNbYLsTOqQH9hrPoDQ&ip=<IP_ADDRESS>&id=o-AP33dmb7Q9Pll4Rk1ShHnVb1sn0Wj8UzTTLvaTz709-e&itag=140&source=youtube&requiressl=yes&mh=tg&mm=31%2C29&mn=sn-ipoxu-u2xd%2Csn-un57en7l&ms=au%2Crdu&mv=m&mvi=4&pcm2cms=yes&pl=21&gcr=tw&initcwndbps=1108750&vprv=1&mime=audio%2Fmp4&ns=080Xx8Lyk7_8iNaUMfb3utYF&gir=yes&clen=3513581&dur=217.060&lmt=1578240876526034&mt=1610635742&fvip=4&keepalive=yes&c=WEB&txp=5531432&n=fZZ91gbdKDFnXg&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cgcr%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgA5hsJjEEaQwHzEHYXpJa0IiGCnmmleqR_DHLUP-uIkcCIBd7SWLWq0Y4rPBzcSimcSqBexVZ8pk5Bu6qNFn-z1Uf&sig=AOq0QJ8wRQIhAOJXSP_70sGFkadxWlxLR7yAYeP2qik7GcxHavv8A1e2AiBkkUitqNfphWi08HH4gKfnLdbgfU8JN-9Ri0tpHwO4ZQ==&ratebypass=yes'
[download] Destination: Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a
[download] 100% of 3.35MiB in 00:00
[ffmpeg] Correcting container in "Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a" -c copy -f mp4 "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.temp.m4a"
[debug] ffmpeg command line: ffprobe -show_streams "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a"
[ffmpeg] Destination: Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.mp3
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a" -vn -acodec libmp3lame "-q:a" 5 "file:Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.mp3"
Deleting original file Enya - Only Time (8D AUDIO)-Lnezh5wQQQ4.m4a (pass -k to keep)

I am facing the same issue.
youtube-dl https://www.youtube.com/watch\?v\=o_TYVGp9n0s
[youtube] o_TYVGp9n0s: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)> (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))

I am able to play the video with my Chrome. However, the URL doesn't really matter, it happens to any URL.
Other Info

macOS Catalina 10.15.7
Python 3.8.6

I just had this same issue on a recent, up-to-date, Windows 10 install with a brand new install of Python 3.9.2.  I found a fix that doesn't seem to be documented yet.
youtube's root CA is "GlobalSign Root CA - R2".  The Windows certificate store has "GlobalSign Root CA - R1" but not R2.  Hence the error.
youtube and google must work in Microsoft Edge, so I guessed that Edge must be causing the certificate store to be updated somehow.  So I loaded Edge and entered google.com in the address bar.  The "GlobalSign Root CA - R2" certificate immediately appeared in the Windows certificate store.  And youtube-dl started working.  Hope this helps.  Thanks for youtube-dl!

I have the same problem with python code but no problem with command.


Suggestion: event listener for route change

I think it would be great if there was an event listener for route change.
For example one issue I'm facing right now is that when I switch to a new route with the Navigator component. I have no way to pause any videos that are autoplaying.

Hello!
Same problem here.
Video is running after route changed. Impossible to stop it.

Hey folks,
Pretty low bandwidth at the moment, working on a lot of things, if anyone has some spare time to throw at solving this problem a PR would be very welcome!

I was able to resolve this issue using the 'willfocus' event from the [Navigator].
I posted a gist for anyone interested.
https://gist.github.com/yamill/9ff0fe84e5b820cf80eb#file-video-scene-component-L7-L14

Wanted to re-open the discussion here if needed. We are trying to clean up open issues in this repo, so if no further comments in the next couple weeks it will be closed.

I would like to b able to listen to route changes


The Me button is confusing

The Me button is supposed to open the player tab if not open, but that doesn't happen if there are none or only one player has been selected. The behavior is that the button appears to do nothing at all.
The button should be hidden until there are at least two players in the list.

fixed in v2.0.19


Add GraphNode 'selected' and 'unselected' signals, simplify GraphEdit

Backport of #64840 from master to 3.x

Thanks!


Google MedLM-Medium is missed out.

https://github.com/langchain-ai/langchain-google/blob/26342d181798e4e21cb3137cec5071d425540365/libs/vertexai/langchain_google_vertexai/_utils.py#L139
I tried invoke medlm-medium and it is using the wrong vertexai.generative_models. It refers to GenerativeModel instead of TextGenerationModel and failling
Here is the fix,
if "medlm-large" in value.lower():
         return GoogleModelFamily.PALM
elif "medlm" in value.lower():
          return GoogleModelFamily.PALM

Pull request for this issue - https://github.com/langchain-ai/langchain-google/pull/283


Resuming while offline

Any way around this issue?
https://github.com/wandb/wandb/issues/4624

Hmm, I don't think we can do anything simple from the viewpoint of this package. If I understand correctly, wandb cannot resume offline runs, so this is a clear limitation from the viewpoint of the wandb command line client (which we use "as-is" in this package). There could be some option to read the history from the offline run, and then have whatever ML framework you're using "fake" that history and give it to wandb. But this would be quite inelegant is really out of the scope for this package.
I think the best option is to make sure the wandb issue doesn't get forgotten (I'm surprised they still didn't implement this!)


Broken since v1.0.0?

This is my favourite theme at the moment, great work! Very pleasant to work with.
Since the updates yesterday however, it does not work anymore (the packaging is not complete?)
The themes-folder is missing in the newer releases, which vscode complains about. I reverted to the old version for now.
See the difference to the old version and the new:
ll ~/.vscode/extensions/felixbohlin.cozy-dark-plus-0.16.0

total 336
-rw-r--r--@  1 usr  staff   1.0K Nov 21 13:39 LICENSE.txt
-rw-r--r--@  1 usr  staff   355B Nov 21 13:39 README.md
drwxr-xr-x@ 32 usr  staff   1.0K Nov 21 13:39 demo
-rw-r--r--@  1 usr  staff   143K Nov 21 13:39 example.png
-rw-r--r--@  1 usr  staff   8.5K Nov 21 13:39 icon.png
-rw-r--r--@  1 usr  staff   1.3K Nov 21 13:39 package.json
drwxr-xr-x@  4 usr  staff   128B Nov 21 13:39 src
drwxr-xr-x@  3 usr  staff    96B Nov 21 13:39 themes

ll ~/.vscode/extensions/felixbohlin.cozy-dark-plus-1.0.0

total 336
-rw-r--r--@  1 usr  staff   1.1K Nov 21 13:38 LICENSE.txt
-rw-r--r--@  1 usr  staff   367B Nov 21 13:38 README.md
drwxr-xr-x@ 32 usr  staff   1.0K Nov 21 13:38 demo
-rw-r--r--@  1 usr  staff   143K Nov 21 13:38 example.png
-rw-r--r--@  1 usr  staff   8.5K Nov 21 13:38 icon.png
-rw-r--r--@  1 usr  staff   1.3K Nov 21 13:38 package.json
drwxr-xr-x@  4 usr  staff   128B Nov 21 13:38 src

Thank you my friend!
I was doing some fixes but something between the screen and the chair went wrong. I'll fix it shortly!

There, fixed it! I totally forgot to build the files after doing my changes 🤦 Thanks for letting me know @multicolorist!


Figure 1.16 Wrong

Figure 1.16 on page 36 is wrong.

Yes, it's too bad this was missed. I'm a math teacher, and mathematically the symbols have no intrinsic meaning as you can read it either way. But in programming they have a clear meaning and it has been flipped.


Infinite queue

The league client often times bugs out and cant start the queue anymore
If you could implement a client restart when this happens that would fix the problem

yea, will be working on something for this soon.


[RFC] [Breaking] Retriever Callbacks

This is one way we could implement it. I try to break as few external APIs as possible, but I'll call out a few (of many more) places that get broken.
I don't leap for joy at how much code each change requires when we add new callbacks and how daunting the function signature starts to look for these interfaces. And I don't love how strict things have to be in passing the callback to each component, though maintaining a top level stack that's both thread safe and async context aware probably has other problems.
I haven't updated all the experimental autogpt/ baby agi memory things. and since memory currently doestn't require callbacks, a retriever memory (in experimental) wouldn't get hit

can we just keep get_relevant_docs as the entrypoint, and then have subclasses implement a _get_relevant_docs? same logic just different names (to keep same entrypoint)

I can do that, then I can see if we can do some magic to avoid breaking previous implementations outside the lib


Failed to parse x 'time' to Date object

I am constantly facing a failed to parse time to date object issue. I want to generate a graph which on the left (Y-axis) shows a probability. Say the probability is 0, 0.2, 0.4 etc.. to 1. On the graph, I would like to plot the probability of something at a specific time. The below code is what I am using.
arr1 is the array containing the probabilities.
arr5 is the array containing the timestamp for each of the probability in arr1. The time format is in "12:03:55".
The issue now is if I were to hard code the timestamps into the chart columns, the chart would display perfectly fine. However I am getting the time from a user uploaded .csv file, hence I am concatenating an array which has the timestamps. And this gives me the error of Failed to parse x '12:03:55' to Date object.
I did console.log(dataTime), so I know my concat has been performed correctly in the intended format. This is what the console.log returns.
["times", "12:03:55 ", "12:03:56 ", "12:03:56 ", "12:03:56 ", "12:03:57 ", "12:03:57 ", "12:03:58 ", "12:03:58 ", "12:03:59 ", "12:03:59 ", "12:04:00 "]
This is what the console.log for dataMood returns.
["Mood", 0.3677342, 0.34968433, 0.32662648, 0.3163717, 0.78009516, 0.97079295, 0.97183245, 0.9724318, 0.9689829, 0.7293974, 0.12735543]
Below is the code I am using.
var labelMood = 'Mood';   var dataMood = [labelMood].concat(arr1);   var times = 'times';   var dataTime = [times].concat(arr5);
var chart = c3.generate({ bindto: '#line',     data: {         x: 'times', 			xFormat: '%H:%M:%S',         columns: [ 		dataTime, dataMood         ]     },     axis: {         x: {             type: 'timeseries',             tick: {                 format: '%H:%M:%S'             }         }     } });
I really can't see why this isn't working since the format of the concatenated object is the same as how I would've hard coded the times into columns. I'm at my wits end after having Googled for hours. Hopefully someone has came across this issued before and has solved it successfully.

Hey there. I figured out your problem here. You can't natively transform the format of '12:03:05' (hours, minutes, seconds) to a date. try new Date('12:03:05') in your javascript console and you'll see it doesn't work. It expects a year and month along with it basically.
http://jsfiddle.net/Lkazydtb/1/
Check out this JS fiddle I made. It's a little more complicated as you'll see I had to take your strings and coerce them into a date object. I then added the minified d3 library to use d3.time.format on that date object.
Hope this helps!

@JethroNg see above, forgot to tag you there

Indeed the timeseries chart cannot work with only the time part.
You could either use the time as simple categories or use the workaround provided.


Reschedule healtchecks when host goes to reconnecting state

When agent goes through Reconnecting process:

as a pre-handler, deregister all the healtchecks scheduled on the host
as a post handler, register all the instances with health check enabled, for health check again. This particular scenario is to cover the case when its the only host in the system going through Reconnecting process (as health check gets de-registered on step 1)
Cleanup health check info on the instance removal

https://github.com/rancher/rancher/issues/2622


Hover over register type in disassembly differs from register view

Currently when hovering over a register in the disassembly view they are displayed as an integer while the register view uses hex. I think it would make more sense to have them as hex for the disassembly hover as well (or have a setting for it)

Fixed in 6eb8e37d4d85c418682ebce9d01ab50348e217f4. This will be in the next release.


Error: Couldn't find a compatible version of VS Live Share with this version of Code.

Product and Version [VS/VSCode]:
Version: 1.27.0
Commit: https://github.com/Microsoft/vscode/commit/3d71869a0c4ba1df833b938a2534581250a31d90
Date: 15/08/2018 13:56
Electron: 2.0.7
Chrome: 61.0.3163.100
Node.js: 8.9.3
V8: 6.1.534.41
Architecture: x64

OS Version [macOS/Windows]: Microsoft Windows [Version 10.0.15063]
Live Share Extension Version: Latest as of today
export logs.zip
Steps to Reproduce / Scenario:

Install VSCode builnding it from source: https://github.com/Microsoft/vscode/wiki/How-to-Contribute#build-and-run-from-source
Open VSCode
Go to https://marketplace.visualstudio.com/items?itemName=MS-vsliveshare.vsliveshare and hit the Install button
When VSCode opens, hit install/confirm installing
Now you see Error: Couldn't find a compatible version of VS Live Share with this version of Code.

Workaround
I managed to successfully installing it by:

Downloading the .vsix file https://marketplace.visualstudio.com/_apis/public/gallery/publishers/MS-vsliveshare/vsextensions/vsliveshare/0.3.535/vspackage
Opening VSCode, hitting Ctrl+Shift+P -> Extensions: Install from VSIX...
Picking up the download .vsix file

Related to:

https://github.com/MicrosoftDocs/live-share/issues/85 "Couldn't find a compatible version of VS Live Share with this version of Code."

I am not able to repro this with either build from the VS Code repo, or the latest installed from insiders channel.
Can you try updating to the latest insiders / rebuilding from head, and try agian?


Peer Config files

Not really a problem but more a query and looking for ways to perform backups and to do this in the past i have backed up the .CONF files which got created in the Root folder as well as a the wg0.conf file in etc/wireguard
previously if i had created 10 peers for example i would have had 10 peers listed in Root but when using he dashboard to create the Peers this no longer seems to be the case. Where are the Peer Config files being stored does anyone know ?

I'll take a look and let you know.

also @theonlynexus any plans to pull V3.1 from Donald and promote it to prod has a few good improvements to V 3.0.6 with things like dark mode options as well better overall usage statistics

Thanks for the tip, I have had my own share of life happening lately, so I haven't been able to keep up with the project, but I should be able to pick up the development for at least a few hours a week. I'll definitely look into pulling the latest changes from Donald.


🐳 Github action to automatically push updated image to ECR

As a Cloud Ops team member .
I want the pushing of updated docker images to ECR to be automated.
So that the process takes less time and doesn't require manual intervention from me.

Closing due to decision to use docker hub going fwd.


picture not showing

Describe the bug
Picture not showing see attachment
Document
If applicable, add *.docx document to help explain your problem.
docxjs_issues.docx
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
docxjs

office

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Additional context
Add any other cont
docxjs_issues.docx
ext about the problem here.

Added support of VML kind images, changes commited master. Will close issue sometime later, after publishing next version of package.
P.S. VML is legacy format. I would recommend to update document editing software.

Fixed in version 0.1.10


Update documentation-government-csp-list.md

Correct spelling of NeoSystems LLC (from NeoSustems LLC) in Azure Government CSP List

Thanks, @cstaich!


how can we contribute for hacktoberfest

@Prajwal0225 what enhancements can we make

I can improve the button style and hovering effects

You are welcome to suggest any additions or improvements to the website. If necessary, we can assign you the issue, and you can commence work on it.

what work is left to be assigned?

background does not look good you can work on it. If you are interested.

Yes ok!

we can also add a transition that will zoom in the photo of the superhero on hover
please assign me this task

we can also add a transition that will zoom in the photo of the superhero on hover
please assign me this task

Create a new issue related to background change. I will assign that issue to you.

Sorry, but issue #36 has already been raised try to find the new issue to work on.

What else can you suggest me?


Train error

Hi,
I got the following error when running python main.py --mode=train
==============Start Feature Engineering===========
Traceback (most recent call last):
File "main.py", line 47, in
main()
File "main.py", line 36, in main
finrl.autotrain.training.train_one()
File "/mnt/c/Users/JOSE/Documents/PROJECT/STOCK_TRADING/FinRL-Library/finrl/autotrain/training.py", line 30, in train_one
df = FeatureEngineer(df,
TypeError: init() got multiple values for argument 'use_technical_indicator'
I also get this at the beginning of the run:
/mnt/c/Users/JOSE/Documents/PROJECT/STOCK_TRADING/FinRL-Library/venv/lib/python3.8/site-packages/pyfolio/pos.py:26: UserWarning: Module "zipline.assets" not found; mutltipliers will not be applied to position notionals.
I installed all dependencies and pre-requisites before running. Can you help me please?
Thanks

I fixed it, please try again. Thanks.

I fixed it, please try again. Thanks.

Thanks, I pulled again and after re-running it I got the error below:
Traceback (most recent call last):
File "main.py", line 53, in
main()
File "main.py", line 36, in main
import finrl.autotrain.training
File "/mnt/c/Users/XXX/Documents/PROJECT/STOCK_TRADING/FinRL-Library/finrl/autotrain/training.py", line 14, in
from finrl.env.env_multistock import MultistockEnv
ModuleNotFoundError: No module named 'finrl.env.env_multistock'

Thanks, I pulled again and after re-running it I got the error below:
Traceback (most recent call last):
File "main.py", line 53, in
main()
File "main.py", line 36, in main
import finrl.autotrain.training
File "/mnt/c/Users/XXX/Documents/PROJECT/STOCK_TRADING/FinRL-Library/finrl/autotrain/training.py", line 14, in
from finrl.env.env_multistock import MultistockEnv
ModuleNotFoundError: No module named 'finrl.env.env_multistock'

It's fixed. I made some improvements early today broke this part again.

It's fixed. I made some improvements early today broke this part again.

@BruceYanghy pls tell us how to fixed it?


i met a problem with d415 creating pointcloud

Required Info

Camera Model
{ D400 }

Firmware Version
(pyrealsense2)

Operating System & Version
{Win10}

Kernel Version (Linux Only)

Platform
PC

SDK Version
<IP_ADDRESS>84

Language
{python }

Segment
{others }

Issue Description
I used d415 to generate a depth map and a color map and aligned them.
my saving code like this:
########################################################
depth_image_1 = np.asanyarray(depth_frame_1.get_data())
color_image_1 = np.asanyarray(color_frame_1.get_data())
# Apply colormap on depth image (image must be converted to 8-bit per pixel first)
depth_image_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.5), cv2.COLORMAP_JET)
    # Camera 2
    # Wait for a coherent pair of frames: depth and color
    frames_2 = pipeline_2.wait_for_frames()
    depth_frame_2 = frames_2.get_depth_frame()
    color_frame_2 = frames_2.get_color_frame()
    if not depth_frame_2 or not color_frame_2:
        continue
    # Convert images to numpy arrays
    depth_image_2 = np.asanyarray(depth_frame_2.get_data())
    color_image_2 = np.asanyarray(color_frame_2.get_data())
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.5), cv2.COLORMAP_JET)

    # Stack all images horizontally
    images = np.hstack((color_image_1, depth_image_1,color_image_2, depth_image_2))

    # Show images from both cameras
    cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
    cv2.imshow('RealSense', images)
    # Save images and depth maps from both cameras by pressing 's'
    ch = cv2.waitKey(1)
    if ch==115:
        cv2.imwrite("my_image_1.jpg",color_image_1)
        cv2.imwrite("my_depth_1.jpg",depth_image_1)
        cv2.imwrite("my_image_2.jpg",color_image_2)
        cv2.imwrite("my_depth_2.jpg",depth_image_2)
        print ("Save")

########################################################
and then i want to use open3d to generate pointcloud from this two pictures.
such as
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color_raw, depth_raw, convert_rgb_to_intensity=False)
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd_image, Intrinsic)
the pointcloud i created is completely wrong.
its seems like this

i think the depth is wrong.
can anybody help me?

depth_image_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.25), cv2.COLORMAP_JET)
and when i change the alpha , i can get different pointcloud

嗨，请问更改 alpha 后您的点云是否正确？

No, it's still wrong .

There was a Python Open3D case at https://github.com/IntelRealSense/librealsense/issues/12090 where the pointcloud was incorrect.  The RealSense user in that particular case found at https://github.com/IntelRealSense/librealsense/issues/12090#issuecomment-1673844543 that the problem was with their intrinsics code.
Does testing the script at https://github.com/IntelRealSense/librealsense/issues/10894#issuecomment-1245803237 produce a more correct pointcloud, please?

#12090 有一个 Python Open3D 案例，其中点云不正确。在该特定情况下，实感用户在 #12090（评论）中发现问题出在他们的内部代码上。
请在 #10894（评论）处测试脚本是否会产生更正确的点云？

In fact, I want to save the depth map and RGB image at that time when I press the 's' key, and then convert the depth map and RGB image into a point cloud offline. I found that if you do not perform the operation "depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.35), cv2.COLORMAP_JET)", the saved depth map will only be in black and white. Also, changing the alpha allows me to get a different depth map, but the point cloud generated with this depth map is still incorrect, can you give me an example of how to save the depth map correctly.
Thanks in advance.

There was a Python Open3D case at #12090 where the pointcloud was incorrect. The RealSense user in that particular case found at #12090 (comment) that the problem was with their intrinsics code.
Does testing the script at #10894 (comment) produce a more correct pointcloud, please?

In fact, I want to save the depth map and RGB image at that time when I press the 's' key, and then convert the depth map and RGB image into a point cloud offline. I found that if you do not perform the operation "depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.35), cv2.COLORMAP_JET)", the saved depth map will only be in black and white. Also, changing the alpha allows me to get a different depth map, but the point cloud generated with this depth map is still incorrect, can you give me an example of how to save the depth map correctly.
Thanks in advance.

There was a Python Open3D case at #12090 where the pointcloud was incorrect. The RealSense user in that particular case found at #12090 (comment) that the problem was with their intrinsics code.
Does testing the script at #10894 (comment) produce a more correct pointcloud, please?

This is all my saving code:
import pyrealsense2 as rs
import numpy as np
import cv2
ctx = rs.context()
devices = ctx.query_devices()
dev0 = ctx.query_devices()[0]
dev1 = ctx.query_devices()[1]
device_model0 = str(dev0.get_info(rs.camera_info.name))
device_model1 = str(dev1.get_info(rs.camera_info.name))
print(f'device_model0: {device_model0}')
print(f'device_model1: {device_model1}')
print('Config ... ')
pipeline_1 = rs.pipeline()
config_1 = rs.config()
config_1.enable_device(dev0.get_info(rs.camera_info.serial_number))
config_1.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config_1.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
pipeline_2 = rs.pipeline()
config_2 = rs.config()
config_2.enable_device(dev1.get_info(rs.camera_info.serial_number))
config_2.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config_2.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
Start streaming from both cameras
profile1 = pipeline_1.start(config_1)
profile2 = pipeline_2.start(config_2)
intr = profile1.get_stream(rs.stream.color).as_video_stream_profile().get_intrinsics()
print(intr)
try:
while True:
    # Camera 1
    # Wait for a coherent pair of frames: depth and color
    frames_1 = pipeline_1.wait_for_frames()
    depth_frame_1 = frames_1.get_depth_frame()
    color_frame_1 = frames_1.get_color_frame()
    if not depth_frame_1 or not color_frame_1:
        continue
    # Convert images to numpy arrays
    depth_image_1 = np.asanyarray(depth_frame_1.get_data())
    depth_image_11 = np.asanyarray(depth_frame_1.get_data())
    color_image_1 = np.asanyarray(color_frame_1.get_data())
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_image_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.35), cv2.COLORMAP_JET)

    # Camera 2
    # Wait for a coherent pair of frames: depth and color
    frames_2 = pipeline_2.wait_for_frames()
    depth_frame_2 = frames_2.get_depth_frame()
    color_frame_2 = frames_2.get_color_frame()
    if not depth_frame_2 or not color_frame_2:
        continue
    # Convert images to numpy arrays
    depth_image_2 = np.asanyarray(depth_frame_2.get_data())
    depth_image_22 = np.asanyarray(depth_frame_2.get_data())
    color_image_2 = np.asanyarray(color_frame_2.get_data())
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.35), cv2.COLORMAP_JET)

    # Stack all images horizontally
    images = np.hstack((color_image_1, depth_image_1,color_image_2, depth_image_2))

    # Show images from both cameras
    cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
    cv2.imshow('RealSense', images)
    # Save images and depth maps from both cameras by pressing 's'
    ch = cv2.waitKey(1)
    if ch==115:
        cv2.imwrite("my_image_1.jpg",color_image_1)
        cv2.imwrite("my_depth_1.jpg",depth_image_1)
        cv2.imwrite("my_image_2.jpg",color_image_2)
        cv2.imwrite("my_depth_2.jpg",depth_image_2)
        print ("Save")
    if ch==27:
        cv2.destroyAllWindows()
        break

finally:
# Stop streaming
pipeline_1.stop()
pipeline_2.stop()

Oh, I have solved it, saving the depth map in png format is no problem, anyway, thank you very much

You are very welcome.  It's great to hear that you achieved a solution.  Thanks very much for the update!

Hi @MentalBaka1  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.


Import GraphQL fragments.

Summary
When it comes to things like posts, authors, pages, and other various pieces that might share a common structure, it would be nice to be able to import a set of GraphQL fragments that hold common queryable fields, that gets parsed into GraphQL early.
Basic example
src/fragments/post.graphql
fragment PostParts on post {
  title
  path

  author {
    ...AuthorParts
  }
}

Motivation
Because of the way posts are handled in Gridsome it becomes necessary to share infrastructure beyond the current component, and start making things more manageable.

If this isn't possible, or not wanted, can the GraphQL layer be exposed at least? I can do it myself.

Closed for #232


Fix #116: trusted-users allows running commands as root without password

Update README.md to rather suggest user to directly put the extra-substituters in /etc/nix/nix.conf and remove nixConfig attribute from the flake.nix.
Rendered

These lines should also be updated:
https://github.com/input-output-hk/devx/blob/b06c33669b7905362c4d8d5f0a3453b1804394a4/docs/direnv.md?plain=1#L25-L27


Update release script with publish option

Developer's Certificate of Origin 1.1
   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

BTW, the script was used to create/publish Elyra 1.5.0.

Documentation for the script would be great if you want to enable other people to go through the process 😄

@ptitzler it is documented.

Merging as this is just a cherry-pick from x1.x branch


Import parsers from WebOfTrust/keriox

Although the deserializers (implementation of FromStr trait) were imported from WebOfTrust/keriox in commit 07da651260f9, none of the parsers were.  The parsers are necessary for parsing data streams, and as I understand, they should be included in cesrox.
The parsers in WebOfTrust/keriox can be found in event_parsing module.  You can grep for "nom::IResult" to take a quick look at which parsers are implemented in keriox.

Yes, there is still a significant featureset to pull over from Keriox, the parsers being a major part of it. That is one of the next major components to bring in.


Automated cherry pick of #8757

Cherry pick of #8757 on release-6.0.
/cc  @pablovelezvidal

/release-note-none

/update-branch


Saving the code from editor

How to save code from editor to database? Or get it to variable

You can use callback "(textChanged)="onChange($event)" or use
[(text)]="text" // possible two way binding


Unable to Deploy to Org

When I try to deploy via button to my dev org it fails with the below error

Yeah i messed up adding the VF demo controller and page, apologies. If you download the code and remove the demo controller class and page you should be good in the meantime.

I'll wait for your fix.

Fixed


Npm start does not work

Describe the bug
Npm start does not work . As son as I open my project with Bolt...it does not run, so I cannot use the IA in my project
Link to the Bolt URL that caused the error
https://bolt.new/~/stackblitz-starters-mw5kih
Steps to reproduce
From my Stackblitz project a press "Open in Bolt.new" button.
In bolt.new , on the left side, the log info indicates that install dependencies and start application FAILS.
And that's it
Expected behavior
Run the application for using the  new AI .
Screen Recording / Screenshot

Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 129
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 45160230,
  "usedJSHeapSize": 42886166,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = Frankystan
Chat ID = f7a9b01dc9ed

Additional context
No response

Appreciate the feedback! We are aware of this issue and are tracking it here: #2021


Add Qodana action

https://github.com/jetbrains/qodana-action

@sdras @gep13 @spekulatius @politician sorry for tagging you all, any chance you can take a look at this PR approval?  (First-time contributors need a maintainer to approve running workflows.) Thanks a lot!


gRPC errors

This PR resolves https://github.com/stargate/stargate/issues/923.
My first attempt was to add a custom interceptor to like TransmitStatusRuntimeExceptionInterceptor (https://grpc.github.io/grpc-java/javadoc/io/grpc/util/TransmitStatusRuntimeExceptionInterceptor.html). However, I think this is more for debugging or trusted internal services. Also, the internal implementation has some complexity around threading and Server.Listeners.  Let me know if you think this is a better approach.
I think the preferred approach is to use StreamObserver#onError(Throwable t). So this PR just handles all the unhandled and known exception types from the Persistence layer. Otherwise, code inside the service, where we control error handling, should use StatusException or StatusRuntimeException.

Rebased on master.


Add installation instructions for Homebrew

Pending merge of Homebrew/homebrew#37852

Thanks for PR :octocat: Although, as I've added notice in README before (b10f90ed913e7f70f1677f3e317c86f192ae3bc4), starts from pyenv v20141211, it bundles built-in auto-rehash feature as same as pyenv-pip-rehash. Basically we don't need this plugin other than the purpose of backward compatibility.
https://github.com/yyuu/pyenv/blob/e9aa6cf649673e7de416ef7346be19501d096cb1/CHANGELOG.md#20141211

Ah, whoops! Sorry about that. Perhaps it's still useful for those using an older version of pyenv?

It would be useful for the people who are still using pyenv older than v20141211.

Cool. So it is worth merging this in since the deprecation is already mentioned elsewhere in the README?

I'll merge this once Homebrew/homebrew#37852 has been merged.

Whoops, seems I didn't do my homework before submitting. This is already available in homebrew-boneyard. I've updated the instructions accordingly.


xcaddy build fails with AppIfConfigured returns 2 values

FROM caddy:2-builder AS builder
RUN apk update && apk add gcc g++ make libpcap-dev libpcap
RUN CGO_ENABLED=1 xcaddy build \
    --with github.com/mholt/caddy-ratelimit \
    --with github.com/gaukas/clienthellod/modcaddy

33.89 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
34.04 2024/11/07 22:03:41 [INFO] exec (timeout=0s): /usr/local/go/bin/go build -o /usr/bin/caddy -ldflags -w -s -trimpath -tags nobadger,nomysql,nopgx
68.73 # github.com/gaukas/clienthellod/modcaddy/listener
68.73 /go/pkg/mod/github.com/gaukas/clienthellod@v0.4.2/modcaddy/listener/listener.go:65:10: assignment mismatch: 1 variable but ctx.AppIfConfigured returns 2 values
68.76 # github.com/gaukas/clienthellod/modcaddy/handler
68.76 /go/pkg/mod/github.com/gaukas/clienthellod@v0.4.2/modcaddy/handler/handler.go:53:10: assignment mismatch: 1 variable but ctx.AppIfConfigured returns 2 values
69.91 2024/11/07 22:04:17 [INFO] Skipping cleanup as requested; leaving folder intact: /tmp/buildenv_2024-11-07-2203.2121729552
69.91 2024/11/07 22:04:17 [FATAL] exit status 1

Any ideas how to fix this?

Ah, I can see this is a problem with 0.4.2 but is fixed in 0.5 alpha.
https://github.com/gaukas/clienthellod/blob/bb3ee89b377cc9308cfaa46b0215c74e1cbb2b8f/modcaddy/listener/listener.go#L65
vs.
https://github.com/gaukas/clienthellod/blob/560e27ce8432c6eae9a23f559ccdc5027d11b35e/modcaddy/listener/listener.go#L61


Bundle Python Interpreter in virtual environments

For system wide installations of applications build with uv, it would be nice to have an option to build an execution environment which is self contained and does not rely on the user's local files (~/.local/share/uv). My usecase for this would be an application that I want to install to a server for system-wide usage. At the moment, you can achieve this with the following commands (with some path adjustments):
uv venv --no-cache
uv pip install --no-cache -r requirements.lock
cp -r ~/.local/share/uv/python/cpython-3.12.5-linux-x86_64-gnu python
rm .venv/bin/python
ln -s python/bin/python3.12 .venv/bin/python

# To run the program
.venv/bin/python <your script.py>

I.e., you can replace the symlinked python interpreter by a bundled copy.
I am not entirely sure how this should be implemented. In a first simple step, one could add a --bundle-python flag to uv venv which copies the downloaded Python interpreter to, e.g., .venv/python and uses it instead of a path in ~/.local/share/uv.
However, you can take this idea further, up to a uv install command which would do something similar to make install.

Ok, you can kinda do this already by passing UV_PYTHON_INSTALL_DIR (docs) to uv venv.

I think we want to add this feature.

Related

https://github.com/astral-sh/uv/issues/6970

Ohh til UV_PYTHON_INSTALL_DIR exists.
I was looking for this parameter in CLI --help and it's not mentioned there.
Perhaps

The env var should be mentioned in uv python * docs
uv python install * commands should take an --install-dir argument?

Yes we should mention it in the help documentation. I'd also be down to accept a target directory in the CLI.

Ok, I started working on this locally. Does --install-dir name work?


Add Julia language

thank you, if you liked this project star this project and share this project for helping others