Jedi autocompletion not working in VS Code native notebooks editor preview

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur
1.Right click on a notebook file
2.Choose Open with preview Notebook editor
3.Import OS module
4. os. doesn't display the list of available methods
Actual behavior no method/attribute is displayed
Expected behavior displaying the list of available methods/attributes

Your Jupyter and/or Python environment
Please provide as much info as you readily know

Jupyter server running: Remote
Extension version: v2020.8.93699-dev
VS Code version: 1.47
Setting python.jediEnabled: false
Setting python.languageServer: Pylance
Python and/or Anaconda version: 3.7.7
OS: Ubuntu through WSL2
Virtual environment: conda

Developer Tools Console Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

Hi! This issue is occurring with both pylance and jedi, isn't it?

This issue should not be happening with pylance if you're using insiders. Jedi doesn't currently support notebooks.

This issue should not be happening with pylance if you're using VS Code insiders. Jedi doesn't currently support notebooks.

@rchiodo Little fuzzy on what you mean. Jupyter running the regular way in a web browser uses ipython which uses jedi for autocompletion, as far as I understand.
Just speaking for myself, until VS code has autocompletion parity with a regular jupyter server (i.e. for custom classes ,for file names when doing something pd.read_csv('my_very_long_file_name') etc), then it's just not going to be as good as a normal jupyter server, and I can't really use it.
I rely on the feature set of jupyter autocompletion every day. This isn't a nice to have, it's a vital part of the experience.

Sorry, @astrowonk this bug is out of date.
The python extension has multiple intellisense providers. One of those is Jedi. The jedi provider (at the time this bug was created) did not support intellisense in the new native notebook editor.
However Jupyter itself also has intellisense and is handled with a different code path. Internally there are two 'autocompletion' providers. The one from the python extension (which is statically parsing the notebook and all modules in your workspace) and one that the jupyter extension provides by talking to a jupyter kernel (which is dynamically returning autocomplete items from the active kernel).
The combination of these two completion providers is what provides intellisense in a VS code notebook. The first one (from the python extension) when it specifies Jedi as the source doesn't work with notebooks at the moment (although that's soon to change). The second one (from the jupyter extension, and ultimately from the kernel) which uses Jedi under the covers does work with notebooks.


Add request.prepare_payload() to http_client_sync example

Even though the string_body is empty it should call prepare_payload to avoid an easy mistake for anyone reusing the example,
This is identical in the other client examples as well. If acceptet, this PR could be amended, or I can make similar PRs.

Should't the get us an empty-body instead?

@klemens-morgenstern Well, it could (and perhaps should, since there is no body content), but then the documentation should provide an example with string_body as well, that does call prepare_payload, imho. :) It is unavoidable that users often start by copying the examples in the documentation, and providing an example with a body and prepare_payload will avoid a small roadblock.

I agree that we should do something, I'm not exactly sure what.

Please check #2598, which should be a better basis for copy-pasta. Although maybe we need to instead do a cook-book.

Closing this pull request because calling prepare_payload() is not required in those examples, and adding it might cause confusion for readers.


NCCL Installation will fail on the Headnode with Error 137.

The NCCL post-install script is documented to be run on Headnode and Slurm Queue but the script will successfully execute on GPU nodes only. Documentation needs to be updated.

can you provide a log of the issue?


GPU OOM error when using keras and distributions strategy

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): tf-nightly-gpu==1.14.1.dev20190524
Python version: Python 3.7.3
CUDA/cuDNN version: CUDA 10, CUDNN <IP_ADDRESS>
GPU model and memory: 4 x NVIDIA V100

Describe the current behavior
Using tf.distribute.MirroredStrategy() together with Keras to train models as described in https://www.tensorflow.org/alpha/tutorials/distribute/keras results in GPU out of memory errors appearing after several epochs of training. We excluded our custom written code as the source of the memory leaks and made sure that the model actually fits into memory with enough headroom. It seams that either tf.data or tf.keras.metrics have a memory leak that starts showing up after several epochs of training and evaluation.
Describe the expected behavior
Tensorflow doesn't throw OOM errors.
Code to reproduce the issue
Unfortunately I cannot give a concrete code example to reproduce this issue since memory leaks appear anytime in between 10min to 12h of training. Though I am happy to provide more information and would be eager to get suggestions on how to properly debug this problem.
Other info / logs
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py", line 644, in fit
use_multiprocessing=use_multiprocessing)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_distributed.py", line 899, in fit
validation_freq=validation_freq)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_distributed.py", line 149, in fit_distributed
steps_name='steps_per_epoch')
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 409, in model_iteration
steps_name='validation_steps')
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 274, in model_iteration
batch_outs = f(actual_inputs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/backend.py", line 3351, in __call__
run_metadata=self.run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1458, in __call__
run_metadata_ptr)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: Failed to allocate memory for the batch of component 0

[[{{node MultiDeviceIteratorGetNextFromShard}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[[RemoteCall]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[[IteratorGetNext_7]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
(1) Resource exhausted: Failed to allocate memory for the batch of component 0
[[{{node MultiDeviceIteratorGetNextFromShard}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[[RemoteCall]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[[IteratorGetNext_7]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[[metrics_4/categorical_accuracy/Identity_2/_3447]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
0 successful operations.
3 derived errors ignored.

Do you have anything in your code such as a callback that keep adding new ops to the graph?

Do you have anything in your code such as a callback that keep adding new ops to the graph?

As far as we can tell, no. We are using the TensorBoard and ModelCheckpoint callbacks to monitor training process and don't have any other callbacks.
@yuefengz Is there a way to check if TensorFlow or our code adds new ops to the graph when training the keras model?
We are now able to also reproduce this error on a single GPU without distributions strategy. One thing to note is that we are using tf.data.Dataset::prefetch(tf.data.experimental.AUTOTUNE) as the last operation in the tf.data pipeline, though if I recall correctly tf.data.Dataset::prefetch shouldn't prefetch onto the GPU memory, correct?
Once we are able to reliably reproduce this issue I am happy to share a repo/gist with the code. Any help or pointer for debugging would be greatly appreciated in the meantime.

To check whether there is any new op added you graph after training starts, you can add loggings to the create_op method: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/ops.py#L3221
@jsimsa for dataset questions.

@lgeiger tf.data without distribution strategy will not prefetch any data to GPU (unless you are using prefetch_to_device which has been deprecated in lieu of multi-device iterator used by the distribution strategy under the hoods)

To check whether there is any new op added you graph after training starts, you can add loggings to the create_op method

Thanks. I double checked, that we don't add any operations to the graph after training starts.

Same problem. I also double checked my code that no new op after training starts. Suspect that there's some memory leak somewhere in tf.

We where able to resolve this issue by not using .prefetch(tf.data.experimental.AUTOTUNE) as the last dataset transformation, which caused problems with TensorFlow running inside a Kubernetes cluster without the correct resource requests and limits and data streaming from GCS.

I encountered this issue today. For me it turned out that I accidentally batched my dataset multiple times, instead of once.

@Trezorro Could you share how you batch your dataset multiple times accidentally and why would that cause this ResourceExhaustedError?
Is it something like in a single method chain of data.cache().batch(32).prefetch(1).batch(32)?
Or 4 assignments separated some distance apart in code? eg. data = data.cache(), data=data.batch(32), data = data.prefetch(1), data=data.batch(32)

I used assigments, so that I could put some steps within a conditional
statement. I used prefetch only at the very end however.
While fiddling with my pipeline, I accidentally had two lines with:
 examples = examples.batch(batchsize)
This of course got me in some memory issues, even when every single example
is just a 224x224 image.
Op ma 29 jun. 2020 om 10:53 schreef gitgithan<EMAIL_ADDRESS>
@Trezorro https://github.com/Trezorro Could you share how you batch
your dataset multiple times accidentally and why would that cause this
ResourceExhaustedError?
Is it something like in a single method chain of
data.cache().batch(32).prefetch(1).batch(32)?
Or 4 assignments separated some distance apart in code? eg. data =
data.cache(), data=data.batch(32), data = data.prefetch(1),
data=data.batch(32)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/29213#issuecomment-651026270,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AESTRG2VUV2PNURLJ5OGD5DRZBI7DANCNFSM4HR2RBWA
.

--
Met vriendelijke groet,
Milan Tresoor

Initially, I did train_ds = train_ds.cache().prefetch(buffer_size=2), then i removed .prefetch, still errors, then i removed .cache(), and it worked.
Note: cache.prefetch works fine on 1/5 the number of images. I had this error while scaling up the data.
Also there are other symptoms it will fail. I see my RAM go from 5gb to 15gb (using up all of it) as the training steps begin and always exactly at step 308/330 i get the error Resource Exhausted: Failed to allocate memory for the batch of component 0 [[node IteratorGetNext. After removing cache and prefetch, my RAM goes up to 9gb and maintains. Anyone has an explanation for how cache and prefetch affects RAM? Also, how can we be sure AUTOTUNE will not prefetch too many batches and cause errors in the middle of training?
@Trezorro Could you share how you batch your dataset multiple times accidentally and why would that cause this ResourceExhaustedError?
Is it something like in a single method chain of data.cache().batch(32).prefetch(1).batch(32)? Or 4 assignments separated some distance apart in code? eg. data = data.cache(), data=data.batch(32), data = data.prefetch(1), data=data.batch(32)

it works for me


[Enum Request] TAB "Cash Flow Forecast"

Following #1391
Seems like 'Source Type / Source Type Filter' are not extendable. Extendable enums might help.
We have currently introduced a new custom field instead, but are forced to duplicate flowfields like 'Amount (LCY') since flowfield definitions are not extendable including new flowfilters.
What is the best way forward to extend CF with new source types (in screenshot below the values 'Services' and 'Costs')? Extendable enums (forcing us to do another refactoring ...)?

In case extensible enums are provided, please review ALL related cashflow objects (e.g. table "Cash Flow Account", "Cash Flow Forecast Entry", ...).

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Fix the readme file so that it has valuable information

I want to see a better title, description and maybe a road map

Working on this now


Add dracut cmd to generate initramfs with drivers for RHEL raw builds

Issue #2462
Description of changes:
Raw builds are run on KVM vms. RHEL 8.x ships with dracut that builds initramfs with default host-only mode set to true. This causes the initramfs to only include drivers that are required to run kvm infrastructure. When the disks are exported and booted on a metal server, the OS fails to boot up due to this missing drivers.
This change runs dracut with --no-hostonly to rebuild initramfs to include all drivers
This PR also fixes another issue that bubbled up during testing the dracut changes. The build_type was not set to raw in the rhel json file, causing providers role to run qemu tasks as opposed to raw tasks.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

/lgtm
/approve


new first line for zone API docs

I never understand what a "dynamic extent" is. I'd like to propose a more "developer friendly" definition of a Zone in the API docs.
Open to wordsmithing.

cc @floitschG

@floitschG is working on a larger rewriting of the Zone docs, including the first line.

Thanks for the review. Glad to hear there's new Zone API docs coming.


[DEFECT] HERON fails on PIP linux machine

Defect Description
Describe the defect
What did you expect to see happen?
HERON tests to pass in a PIP machine (Machine where dependencies are installed via PIP)
What did you see instead?
8/10 tests fail
FAILED:
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/cashflows/Cashflows
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/labels/Labels
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/multimarket_fix_price/MultimarketFixPrice
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/min_demand/MinDemand
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/production_flex/ProductionFlex
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/validator/ValidatorExample
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/var_demand_fix_price/VarDemandFixPrice
Failed /home/civet/civet/build_0/raven/plugins/HERON/tests/integration_tests/var_demand_var_price/VarDemandVarPrice

Do you have a suggested fix for the development team?
N/A
Describe how to Reproduce
Steps to reproduce the behavior:

Install dependecies via PIP (linux machine)
run_tests --plugins --re=HERON

Screenshots and Input Files
Please attach the input file(s) that generate this error.  The simpler the input, the faster we can find the issue.
Platform (please complete the following information):

OS: Linux
Version: DEVEL
Dependencies Installation: PIP

For Change Control Board: Issue Review
This review should occur before any development is performed as a response to this issue.

[ ] 1. Is it tagged with a type: defect or task?
[ ] 2. Is it tagged with a priority: critical, normal or minor?
[ ] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
[ ] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
[ ] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure
This review should occur when the issue is imminently going to be closed.

[ ] 1. If the issue is a defect, is the defect fixed?
[ ] 2. If the issue is a defect, is the defect tested for in the regression test system?  (If not explain why not.)
[ ] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
[ ] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
[ ] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

This is a duplicate of #33. Closing.

This is a duplicate of #33. Closing.

Fine. I change the label tot he other one.

Fine. I change the label tot he other one.


fix: improve API reference URLs take 2

See https://github.com/cdklabs/construct-hub/pull/720 for original context.
Over the original PR, this adds the following fixes:

reduces the size of the output of the transliterator handler by taking the created/deleted S3Object arrays and only returning the keys instead. This is needed to prevent failures caused by the output being too large.
fixes logic for the linkFormatter and updates it to give incorrect error messages

TODO: update jsii-docgen version once https://github.com/cdklabs/jsii-docgen/pull/571 is merged

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Ready for review, but adding do-not-merge since the fixes from https://github.com/cdklabs/jsii-docgen/pull/571 need to also be added.


[docs] Migrate onepirate premium template to emotion part2

[x] I have followed (at least) the PR section of the contributing guide.

This is a second part of the migration
One of #16947
Preview https://deploy-preview-26707--material-ui.netlify.app/premium-themes/onepirate/

@oliviertassinari For some reason the hover effect that a button created was broken. Is this intentional?
after: https://next.material-ui.com/premium-themes/onepirate/ (view section "FOR ALL TASTES AND ALL DESIRES".

@vicasas Good catch, to fix :)


Change --check_fuzzer_help to --no_check_fuzzer_help

Because --check_fuzzer_help is a positive flag (defaults to True), there is no way to change it to False, because specifying it on the command line sets it to True. Change the flag to a negative one instead, named --no_check_fuzzer_help.

Codecov Report

Merging #3063 (44de43c) into main (7f1257b) will decrease coverage by 2.50%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #3063      +/-   ##
==========================================
- Coverage   31.39%   28.89%   -2.50%
==========================================
  Files         306      126     -180
  Lines       36659    13783   -22876
==========================================
- Hits        11510     3983    -7527
+ Misses      25149     9800   -15349

see 180 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Fix broken links in repo

Fix broken link in the azure-sdk-for-net repo.

/check-enforcer override


Should raise an error when module-alias cannot be found

if getMappingFromBabel(), you perform the lookup with:
const pluginConfig = c.config.plugins.find(p => p[0] === 'module-alias');
if the key cannot be found (babel for example also allows babel-plugin-module-alias), you'll end up with cryptic errors from eslint:
error  Resolve error: Cannot read property '1' of undefined                    import/no-unresolved

which is the user now has to track down.

Good catch @tals! Want to make a PR to also support the full name? And add a check for the plugin to fix this potential error 1 of undefined?
No worries if you don't have time, I'll do it tomorrow in that case.

Fixed in 1.5.0. Thanks @tals and sorry for the long long delay :/


Milestone-completion.

Please make all the milstones completed if we are done.

did you complete everything

Good call


deflate_medium: avoid emitting a suboptimal literal in the restart case

Import external patch to deflate_medium, improving compression very slightly:

deflate_medium: avoid emitting a suboptimal literal in the restart case
When we load new data into the window, we invalidate the next match, in
case the match would improve. In this case, the hash has already been
updated with this data, so when we look for a new match it will point
it back at itself. As a result, a literal is generated even when a
better match is available.
This avoids that by catching this case and ensuring we're looking at the
past.

Any benchmarks for compression size comparison?

For some reason I thought we had merged all the changes from Intel's fork.

Maybe I'm tired, but assuming we need to rewind just one byte seems wrong... Hash sliding is what I have worked on a lot and it seems that instead of invalidating the next match unconditionally we should use "saturated" subtraction for the match index so only match that goes too far will get invalidated. This obviously means we need to know what the match distance was before the window slide and how many bytes the window was slid.

develop
   text    data     bss     dec     hex filename
 129486    1376      32  130894   1ff4e libz-ng.so.2

 Tool: minigzip   Levels: 1-9
 Runs: 70         Trim worst: 40

 Level   Comp   Comptime min/avg/max/stddev   Compressed size
 1     54.164%      1.384/1.394/1.400/0.005        85,234,591
 2     43.876%      1.502/1.518/1.523/0.004        41,427,218
 3     42.383%      1.472/1.481/1.488/0.004        33,347,341
 4     41.669%      1.377/1.386/1.391/0.004        26,228,657
 5     41.211%      1.592/1.601/1.606/0.004        25,940,617
 6     41.038%      1.446/1.453/1.457/0.003        19,373,674
 7     40.781%      1.309/1.314/1.317/0.002        12,834,980
 8     40.706%      1.675/1.680/1.683/0.002        12,811,166
 9     40.423%      1.938/1.944/1.947/0.002        12,722,161

 avg1  42.917%                        1.530
 tot                                413.094       269,920,405

medium fix 5420eb959b365fb34781b987a8187cda76db5ebe
   text    data     bss     dec     hex filename
 129550    1376      32  130958   1ff8e libz-ng.so.2

 Tool: minigzip   Levels: 1-9
 Runs: 70         Trim worst: 40

 Level   Comp   Comptime min/avg/max/stddev   Compressed size
 1     54.164%      1.381/1.394/1.398/0.004        85,234,591
 2     43.876%      1.476/1.489/1.494/0.004        41,427,218
 3     42.383%      1.530/1.539/1.545/0.004        33,347,341
 4     41.669%      1.421/1.433/1.438/0.004        26,228,657
 5     41.211%      1.616/1.623/1.628/0.003        25,940,615
 6     41.038%      1.460/1.466/1.470/0.002        19,373,673
 7     40.781%      1.312/1.316/1.318/0.001        12,834,980
 8     40.706%      1.678/1.682/1.685/0.002        12,811,166
 9     40.423%      1.939/1.945/1.947/0.002        12,722,161

 avg1  42.917%                        1.543
 tot                                416.573       269,920,402

Compression time is increased on all deflate medium levels.

I am considering dropping this, I did several more tests and I don't really think the performance drop justifies the compression improvement. In a few cases the compression is even a byte worse instead of better, probably due to missing a better opportunity due to the rewind causing an overlapping opportunity.
Perhaps @mtl1979 has a better implementation idea?

@Dead2 When I initially read about the logic, my concern was that after the slide, the first two matches are backwards... This is because after the slide, it uses insert_string() and inside that we check that the current and previous values are not identical, but not if those are in wrong order. Essentially instead of going back one byte, we would need to replace the "current" match with the "previous" match in chain as that is the closest usable match after the slide. IF that previous match is 0, that means there wasn't any match before the slide or it went outside the window.
The root cause is that sliding the window is faster than clearing all the hashes and reinserting all matches from start of new window up to the current position we are comparing.


Average policy in mccfr

This pull requests is related to the issue https://github.com/deepmind/open_spiel/issues/499.
I created the class AveragePolicy which extends the base class Policy and implements the method action_probabilities. Within the solvers, the method average_policy initializes each time this class. @lanctot I mirrored what is done on the C++ side and this allows to return the average policy just for already explored infostates and using the random policy in the case of not explored infostates (as @VitamintK suggested). Also, I adapted the rest of the code accordingly.
I finally compared the outputs of the tests of this new code version with the outputs of the tests of the previous one. The results are exactly the same. I report these here for future reference:
Outcome sampling:
Kuhn2P, conv = 0.010184780058560161
Kuhn3P, conv = 0.12295759857995159
Leduc2P, conv = 2.1751618625344777
External sampling:
Kuhn2P, conv = 0.7420193061644658
Kuhn2P, conv = 0.6532886929945988
Kuhn3P, conv = 1.2682464363671362
Kuhn3P, conv = 1.7098108280062134
Leduc2P, conv = 4.887844390143719
Leduc2P, conv = 4.813241731029391
As a side note: I tried to mirror the corresponding versions on the C++, but indeed between the Python and C++ versions there are some important differences. For example, I had to duplicate the class AveragePolicy for outcome and external sampling monte carlo, since there is no a single place to put these in Python (that would be cfr.h, cfr.cc in C++). I guess that these differences might be because the different Python counterfactual versions were developed in different moments. In my opinion a good issue might be to refactor the Python code for completely replicating the C++ side, even if this will require some effort.
@lanctot @VitamintK hope that this is what you had in mind! Please let me know if it fine to you :)

@colibri17, this looks great! Yes this is exactly what I was hoping for... really wonderful.
Right, I see the issue, not as easy to do here. I noticed cfr.py has some shared functions use by CFR-BR too but that doesn't seem like a good place either.
Looks like in this case AveragePolicy is indeed the exact same class in both cases... how about we create a mccfr_utils.py or mccfr_helpers.py module and just put it in there, then both algorithms can just import that? There might be opportunity to move more stuff in there too but no need to do that right now.

Yes I think this is a good alternative. On the C++ side, methods, classes and structs for both pure CFR (cfr and cfr-br) and MCFFR (outcome sampling cfr and external sampling cfr) algorithms are placed within cfr.h and cfr.cc. But it seems to me that both in Python and in C++ the CFR algorithms have no methods or structures in common with the MCCFR ones, so it makes sense to split unrelated methods in separated modules.
Ideally we could think to refactor the code to make a super base class which contains the common structure shared by CFR and MCCFR algorithms (at the end CFR and MCCFR are both part of counterfactual algorithms). However, this is quite a lot of work and I think we would end up with complicated code, which is against the philosophy of the library of keeping things easy.
These are my 2 cents! :)
Back to the pull requests, yes the AveragePolicy is exactly the same.
Comparing the external and outcome sampling versions, there are also other methods which are exactly the same and could be moved in the mccfr_utils.py (look for example at _add_avstrat, _add_regrets, _lookup_infostate_info). For these we could create a base class containing these methods.
If you agree, I can move AveragePolicy in a new module called mccfr_utils.py , and then open another pull request for moving _add_avstrat, _add_regrets, _lookup_infostate_info in mccfr_utils.py . Maybe, if this is the case, this new module could be just called mccfr.py?

Hi @colibri17 , yes that sounds perfect-- let's do that (common base class in mccfr.py to put AveragePolicy and the shared methods)!

Great, I am going to do the changes! Will move the shared methods in another pull request.
 :)

Seems like our testing is broken on MacOS. Will fix it tomorrow and then import this. Thanks!

Seems like our testing is broken on MacOS. Will fix it tomorrow and then import this. Thanks!

Nevermind, seems like you got unlucky and github actions CI failed. They are running properly now.


huggingface demo code

Is there demo code for using the model via the hugging face API?

Hello!
Thanks for asking! yes, the model is an HF model, so you can use the standard API used for CLIP (here's the HF Documentation).

from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel

model = CLIPModel.from_pretrained("patrickjohncyh/fashion-clip")
processor = CLIPProcessor.from_pretrained("patrickjohncyh/fashion-clip")

image = Image.open("images/image1.jpg")

inputs = processor(text=["a photo of a red shoe", "a photo of a black shoe"],
                   images=image, return_tensors="pt", padding=True)

outputs = model(**inputs)
logits_per_image = outputs.logits_per_image  # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1)
print(probs)
image.resize((224, 224))

Let me know if this makes sense!


Not working with Graylog 3?

Hi,
This plugin looks like exactly what I need. However after the plugin gets loaded, the list of Notification types doesn't seem to change.
Downloading https://github.com/GDATASoftwareAG/Graylog-Plugin-AlertManager-Callback/releases/download/1.2.1/graylog-plugin-alertmanagercallback-1.2.1.jar ...
2020-03-31 14:09:03,474 INFO    [CmdLineTool] - Loaded plugin: AlertManager Callback 1.2.1 [de.gdata.mobilelab.alertmanagercallback.AlertManagerAlarmCallback] - {}

Anything you can do to help is much appreciated
Thanks!

Hi,
If i'm not mistaking Graylog alerts were reworked from v.3.1, so sadly this plugin is not working since :/

I've got it working just fine. It's just the content pack part which seems to be broken.

It is planned to rework this plugin to work with Graylog 3.1+ ?

Currently this plugin is working on Graylog 3.0.2

@StefanHufschmidt @jlnshr do you plan to update this plugin to work with Graylog 3.1+

I use this plugin just fine even on graylog 3.3 latest.  It shows up as a “legacy callback plugin”

@cablespaghetti @matasbalcaitis @jozefbarcin @benoc617 thanks for showing interest in the plugin. Yes, we're planning to add support for Graylog 3.1+. We're expecting this to happen within the next couple of weeks. Feel free to create a pull-request if you have already a working solution and you'd like to share it.

@cablespaghetti @matasbalcaitis @jozefbarcin @benoc617 we've just released the version 1.2.2 of this plugin. It contains some dependency upgrades and adjustments. The vesion 1.2.2 works with Graylog 3.3.5. You can choose it via the "Legacy Alarm Callbacks" for now.
We investigated the necessary work which is needed for the plugin to move into the regular notification plugin section. Unfortunately it is much more work than expected because the plugin development in Graylog changed a lot and would take some more time. In between you can use it from the legacy section. I've also added a small screenshot to the README.MD which shows how to get to the legacy option.


Cleanup old PRs

When opening a new PR, remove all the older PRs (by deleting the branches)

Codecov Report

Merging #34 (a760c9c) into main (8c35a52) will not change coverage.
The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff            @@
##              main       #34   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines           76        83    +7
  Branches         8        10    +2
=========================================
+ Hits            76        83    +7

Impacted Files
Coverage Δ

src/upstream-to-pr.ts
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Uncaught TypeError: Cannot read property 'dispatchEvent' of null (Panel.js:56)

After downloading new SDK today on my local machine every setup is having no data because of this exception

Picklist is having no data because of this exception

Picklist is having no data because of this exception

I see Panels in Retail Inspection and PickList_Discrete are working.  If there is some other scenario were Panels are not working, provide explicit steps to reproduce.

This issue still occurs with new SDK on my local machine

On my local machine sdk is git repo and build doesn't failed.
On VDI whenever i git repo SDK and compile the royale-compiler build failed because it can't download
E:\apache-royale-0.9.3-bin-js-swf\royale-compiler\compiler-externc\target\downloads\es5.js
E:\apache-royale-0.9.3-bin-js-swf\royale-compiler\compiler-externc\target\downloads\es6_collections.js
E:\apache-royale-0.9.3-bin-js-swf\royale-compiler\compiler-externc\target\downloads\browser\html5.js and 2 3 more dependencies
So i have to copied these from old sdk and paste into latest sdk
and then compiler compiled successfully

now i will check this issue on VDI by copying this SDK on VDI and compile library and setups,and then if this issue occurs on VDI as well.then its a issue of SDK


confusing phrase in readme.md

By config/settings/development.py is ignored

This sentence may need a quick look-over:  I can't parse out its meaning.  I suspect it should be

By config, settings/development.py is ignored

.. and I'd love to just make that trivial change, but I'm not sure that's the intent.

man, thanks a lot for opening an issue, I appreciate it!
I wanted basically to say that file is listed in .gitignore and will be ignored by git. It's corrected now.
Note that Readme.md gives you only high level explanation; detailed instructions are here .


[sw/silicon_creator] Control SRAM scrambling and init with OTP

This PR adds an OTP item to control the SRAM scrambling and init performed in rom_start.S to derisk the ROM in case the entropy source fails to work correctly at boot. In order not to change the default behavior that we have been testing so far and to favor security, HARDENED_BOOL_FALSE disables and any other value enables.
@GregAC @msfschaffner @moidx I think this PR will also require you to allow the changes in OTP.

CHANGE AUTHORIZED: hw/ip/otp_ctrl/data/otp_ctrl.hjson
CHANGE AUTHORIZED: hw/ip/otp_ctrl/data/otp_ctrl_mmap.hjson
CHANGE AUTHORIZED: hw/ip/otp_ctrl/rtl/otp_ctrl_part_pkg.sv
CHANGE AUTHORIZED: hw/ip/otp_ctrl/rtl/otp_ctrl_reg_pkg.sv

CHANGE AUTHORIZED: hw/ip/otp_ctrl/data/otp_ctrl.hjson
CHANGE AUTHORIZED: hw/ip/otp_ctrl/data/otp_ctrl_mmap.hjson
CHANGE AUTHORIZED: hw/ip/otp_ctrl/rtl/otp_ctrl_part_pkg.sv
CHANGE AUTHORIZED: hw/ip/otp_ctrl/rtl/otp_ctrl_reg_pkg.sv


Make single checkboxes more user friendly

Currently it appears that when creating a single check box the hidden field that should be rendered automatically by the Rails #check_box helper we're wrapping isn't present in the output.
I'm not sure why this happens, if there's a good reason it's not (and not just a misconfiguration bug) then it's probably a good idea to add the hidden field in this library when multiple: false.
Currently this is kind of mitigated by a note in the guide, but that can definitely be improved :sweat_smile:
= f.govuk_check_box(:awesome, true, multiple: false, label: -> { 'Awesome?' })

  <input id="person-awesome-true-field" class="govuk-checkboxes__input" type="checkbox" value="true" checked="checked" name="person[awesome]">
  Awesome?

[ ] Work out why with a single checkbox the hidden field that should be generated by Rails isn't there
[ ] Fix it if we've done something wrong. If not, take care of rendering it ourselves
[ ] Update the documentation accordingly

So, it seems that Rails only renders the necessary hidden field when the false value is 0, when the helper's called with false the output is omitted :facepalm:
[3] pry(#<GOVUKDesignSystemFormBuilder::Elements::CheckBoxes::FieldsetCheckBox>)> @builder.check_box(@attribute_name, {}, @value, false)
=> "<input type=\"checkbox\" value=\"11\" name=\"person[projects]\" id=\"person_projects\" />"
[4] pry(#<GOVUKDesignSystemFormBuilder::Elements::CheckBoxes::FieldsetCheckBox>)> @builder.check_box(@attribute_name, {}, @value, 0)
=> "<input name=\"person[projects]\" type=\"hidden\" value=\"0\" /><input type=\"checkbox\" value=\"11\" name=\"person[projects]\" id=\"person_projects\" />"


Object Detection

Can the optimizer be used for Object Detection? I tested it myself and it seems that there will be an error

set loss.backward(creat_graph=True).


Glob eating up 1GB of RAM on a Windows 7 x64 machine?

Directory lastfm_test contains 104212 JSON files. When I try to glob all JSON files:
var glob = require('glob');

glob(__dirname + '/lastfm_test/**/*.json', {}, function (err, files) {
    console.log(files.length);
});

... the node.exe process start eating RAM, until it reaches 1GB and stopping because the scan finished. Sometimes I get a Windows error about no memory left. It seems that RAM consumption grows as the number of files in the folder. Node version is 0.10.5.
Is this considered "normal"?

I suspect that this is no longer an issue, as this module has been heavily refactored and slimmed down since then.  Closing for now, but please do pipe up if it's still a problem in the latest release.

Still an issue. Globbing a folder with a depth of 1 subfolder, and total of 1.5 million files - takes up 12GB of memory. While it should be about 100MB.


测试以前的checkpoint，就卡住了。

用最新的不修改是ok的，想测试以前的，修改文件latest对应的checkpoint，启动web_demo.py就卡着了：
python web_demo.py
[2023-10-29 08:23:41,107] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-10-29 08:23:41,539] [WARNING] Failed to load bitsandbytes:No module named 'bitsandbytes'
[2023-10-29 08:23:41,540] [INFO] building FineTuneVisualGLMModel model ...
[2023-10-29 08:23:41,570] [INFO] [RANK 0] > initializing model parallel with size 1
[2023-10-29 08:23:41,570] [INFO] [RANK 0] You didn't pass in LOCAL_WORLD_SIZE environment variable. We use the guessed LOCAL_WORLD_SIZE=1. If this is wrong, please pass the LOCAL_WORLD_SIZE manually.
[2023-10-29 08:23:41,570] [INFO] [RANK 0] You are using model-only mode.
For torch.distributed users or loading model parallel models, set environment variables RANK, WORLD_SIZE and LOCAL_RANK.
/home/aifont/anaconda3/envs/vglm/lib/python3.10/site-packages/torch/nn/init.py:412: UserWarning: Initializing zero-element tensors is a no-op
warnings.warn("Initializing zero-element tensors is a no-op")
[2023-10-29 08:23:46,454] [INFO] [RANK 0] replacing layer 0 attention with lora
[2023-10-29 08:23:46,679] [INFO] [RANK 0] replacing layer 6 attention with lora
[2023-10-29 08:23:46,897] [INFO] [RANK 0] replacing layer 12 attention with lora
[2023-10-29 08:23:47,115] [INFO] [RANK 0] replacing layer 16 attention with lora
[2023-10-29 08:23:47,333] [INFO] [RANK 0] replacing layer 24 attention with lora
[2023-10-29 08:23:47,553] [INFO] [RANK 0]  > number of parameters on model parallel rank 0:<PHONE_NUMBER>
[2023-10-29 08:23:49,328] [INFO] [RANK 0] global rank 0 is loading checkpoint checkpoints/finetune-visualglm-6b-10-27-10-22/100000/mp_rank_00_model_states.pt

是不是对应的：model_config.json 也要做修改？


Can you help fix and improve a Fix a UDF File System Driver?

subject

I've never worked on UDF driver. But I can try it.
What's the issue? BSoD?

I have a guy working on it now. So, I am going to close this issue.


Testbed isn't running - The Compose file './docker-compose.yml' is invalid

Hi everyone,
IDS-testbed# docker-compose up -d
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for volumes: 'broker-fuseki'
Unsupported config option for networks: 'local'
Unsupported config option for services: 'omejdn'
Any comment?
Hi prio, because of the demo at Hannover...

Hi @jfernandezsqs , thanks a lot for quick reply!
As you know Ubuntu's default is 1.25.
I've updated to 1.27, things seem to run okay, but I have this error at the end:
`Creating omejdn              ... error
Creating omejdn-server       ... done
Creating broker-core         ... done
Creating broker-reverseproxy ... done
Creating omejdn-ui           ... done
ERROR: for omejdn  Cannot start service omejdn: driver failed programming externCreating connectora          ... done
Creating broker-fuseki       ... done
Creating connectorb          ... done
ERROR: for omejdn  Cannot start service omejdn: driver failed programming external connectivity on endpoint omejdn (78771c5356b24d1ef84c5ab5533eb05295a5cedf816022857f3772032784c587): Error starting userland proxy: listen tcp4 <IP_ADDRESS>:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.
`


Conway’s Law

Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.`

The Wikipedia community describes Conway’s Law like this; I paraphrase it like this: if the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins. The organizational divides are going to drive the true seams in the system.
The architecture of the system gets cemented in the forms of the teams that develop it. The system decomposition drives team ownership. Then the organizational lines of communication become reflected in the interfaces, with cleaner, better preserved interfaces along the lines where organizational distance increases. In small, co-located teams, short-cuts can be taken to optimize within the team. But each short-cut that introduces a dependency is like rebar in concrete–structurally efficient, but rigid. If the environment changes, demanding new lines to be drawn, the cost becomes clear. The architecture is hard to adapt.
One could say this is part of the innovator’s dilemma. Sustaining innovations, that is, incremental improvements within the cast of the architecture, are what the organization is adapted to be good at. But when a breakthrough innovation demands a new architecture, a new organization (unencumbered by power trees that grew up around the old architecture) tends to be more fleet in bringing the innovation to market.
Another implication of Conway’s Law is that if we have managers deciding on teams (what they’ll do, who will be on them, and how they will relate), and deciding which services will be built, by which teams, we implicitly have managers deciding on the system architecture. They determine system chunks (services or components) and capabilities by deciding who will build what.
Conway’s Law also kicks in if we take an initial guess at the system decomposition (a first-cut conceptual architecture), allocate subsystems to teams, and sally forth–the team boundaries will tend to become boundaries within the system. Anything else will be a feat of architectural heroics; hard to accomplish, when architectural heroics have to compete with schedule heroics driven by the steady beat of integration clocks. Yet, architecture is where we address cross-cutting concerns, or at least those that needs-must be addressed with a system perspective so that when it comes time to compose the system it will have the properties stakeholders care about, rather than emergent properties that may or may not suffice.
Roles are defined by their responsibilities and associated decisions. Architect is a role. Any person may play one or more roles. That is, the architect role may be shared among a group of people (as in many agile project teams), or one person may hold more than one role (as in many small teams, especially in startups).  This may be overt and declared. And it may be the result of decisions that are actually effected. If management decisions determine the architecture of the system, they are in effect its architects. If developers determine the architectural decisions, they are in effect its architects.
“Duh!” you might well be saying. Yes, a lot of what is absolutely common sense when it is put plainly, is so obtuse in the face of perplexifying reality. Simply witness all the heated arguments and misunderstandings you get around the topic of the role of the architect.
But what does it mean? Architecture needs to happen across the interfaces, and this also means across the system/organization interfaces. It means that system architects (who we call architects) and business/organization architects (who we call managers) should not work as if one has no impact on the other.
Ruth Malan
http://traceinthesand.com/blog/2008/02/13/conways-law
https://web.archive.org/web/20160318185712/http://traceinthesand.com/blog/2008/02/13/conways-law


[ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Mul node.

(venv) D:\F5-TTS>python F5-TTS-ONNX-Inference.py
Building prefix dict from the default dictionary ...
Loading model from cache D:\idm-temp\temp\jieba.cache
Loading model cost 0.458 seconds.
Prefix dict has been built successfully.
Run F5-TTS by ONNX Runtime.
NFE_STEP: 0
2024-11-28 00:11:27.9287186 [E:onnxruntime:, sequential_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running Mul node. Name:'/f5_transformer/transformer_blocks.0/attn/Mul_15' Status Message: D:\a_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:560 onnxruntime::BroadcastIterator::Append axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 2048 by 2855
Traceback (most recent call last):
File "D:\F5-TTS\F5-TTS-ONNX-Inference.py", line 175, in
noise = ort_session_B.run(
File "D:\F5-TTS\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 266, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Mul node. Name:'/f5_transformer/transformer_blocks.0/attn/Mul_15' Status Message: D:\a_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:560 onnxruntime::BroadcastIterator::Append axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 2048 by 2855
I have an AMD rx 6600, using windows 10. Only changed the model directories etc. Encountering this error. It only worked one time and it was using cpu, but at least I was able to get correct generation. After that always this error.
Can you also provide correct modifications to the .py for my amd gpu using onnx ?

The main issue is that your audio input exceeds the maximum length defined in the exported ONNX model settings. Specifically, MAX_SIGNAL_LENGTH = 2048 (set at line 68 in Export_F5.py), while your audio, after the STFT process, has a length of 2855. Please re-export all ONNX models with an appropriately larger value for MAX_SIGNAL_LENGTH and try more once.

I am not familiar with AMD GPUs, but you might want to search for blogs or official guides on your own. These websites could be helpful. Good luck!

https://rocm.docs.amd.com/projects/radeon/en/latest/docs/install/native_linux/install-onnx.html
https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html
https://huggingface.co/docs/optimum/onnxruntime/usage_guides/amdgpu
https://onnxruntime.ai/docs/execution-providers/DirectML-ExecutionProvider.html  (Windows, use DirectML call the AMD GPU)
How can use onnx model for inference with AMD GPU in Windows? microsoft/onnxruntime#19061
https://pypi.org/project/onnxruntime-directml/

ok, was able to convert onnx models with different MAX_SIGNAL_LENGTH and now want to quantize them to fp16.
When I run python fp16.py inside the "D:\F5-TTS-ONNX\Do_Quantize\Dynamic_Quant" folder with these changes inside fp16.py
Path Setting
original_folder_path = r"D:/sd/models/f5-tts-onnx/s"
quanted_folder_path = r"D:/sd/models/f5-tts-onnx/s/q"
model_path = os.path.join(original_folder_path, "F5_Transformer-4096.onnx")
quanted_model_path = os.path.join(quanted_folder_path, "F5_Transformer-4096-q.onnx")
The process gives this error,
Traceback (most recent call last):
File "D:\F5-TTS-ONNX\Do_Quantize\Dynamic_Quant\fp16.py", line 48, in
model=onnx.load(quanted_model_path),
File "D:\F5-TTS\venv\lib\site-packages\onnx_init_.py", line 212, in load_model
model = _get_serializer(format, f).deserialize_proto(load_bytes(f), ModelProto())
File "D:\F5-TTS\venv\lib\site-packages\onnx_init.py", line 149, in _load_bytes
with open(f, "rb") as readable:
FileNotFoundError: [Errno 2] No such file or directory: 'D:/sd/models/f5-tts-onnx/s/q\F5_Transformer-4096-q.onnx'
which is understandable because we don't have a quantized model yet ? what should I change here ?

Try to using this script instead. I will update the F5-TTS-Do_Quantize later.
import os
import gc
import sys
import onnx
import torch
import onnx.version_converter
from onnxslim import slim
from onnxconverter_common import float16
from onnxruntime.transformers.optimizer import optimize_model
from transformers import AutoModelForCausalLM

# Path Setting
original_folder_path = r"C:\Users\Downloads\Model_ONNX"                          # The original folder.
quanted_folder_path = r"C:\Users\Downloads\Model_ONNX_Quanted"                   # The quanted folder.
model_path = os.path.join(original_folder_path, "Model.onnx")                    # The original fp32 model path.
quanted_model_path = os.path.join(quanted_folder_path, "Model_quanted.onnx")     # The quanted model stored path.
download_path = None                        # Set the folder path where the LLM whole project downloaded, otherwise set "NONE".
use_gpu = True                                                                   # If true, the transformers.optimizer will remain the FP16 processes.
provider = 'CPUExecutionProvider'                                                # ['CPUExecutionProvider', 'CUDAExecutionProvider', 'CoreMLExecutionProvider']

# Convert the fp32 to fp16
model = onnx.load(model_path)
model = float16.convert_float_to_float16(model,
                                         min_positive_val=1e-7,
                                         max_finite_val=65504,
                                         keep_io_types=False,        # True for keep original input format.
                                         disable_shape_infer=False, # False for more optimize.
                                         op_block_list=['DynamicQuantizeLinear', 'DequantizeLinear', 'Resize'],  # The op type list for skip the conversion. These are known unsupported op type for fp16.
                                         node_block_list=None)      # The node name list for skip the conversion.
onnx.save(model, quanted_model_path)
model_size_bytes = sys.getsizeof(model.SerializeToString())
model_size_gb = model_size_bytes * 9.31322575e-10  # 1 / (1024 * 1024 * 1024)
if model_size_gb > 2.0:
    is_large_model = True
else:
    is_large_model = False
del model
gc.collect()

# ONNX Model Optimizer
slim(
    model=quanted_model_path,
    output_model=quanted_model_path,
    no_shape_infer=False,   # True for more optimize but may get errors.
    skip_fusion_patterns=False,
    no_constant_folding=False,
    save_as_external_data=is_large_model,
    verbose=False
)

# Upgrade the Opset version. (optional process)
model = onnx.load(quanted_model_path)
model = onnx.version_converter.convert_version(model, 21)
onnx.save(model, quanted_model_path, save_as_external_data=is_large_model)

# It is not recommended to convert an FP16 ONNX model to the ORT format because this process adds a Cast operation to convert the FP16 process back to FP32.

I know this is another matter but, I found another model with my native language and wanted to use it but when I try to convert it to ONNX format using export.py as I did with the default model that came with f5-tts, I am getting these errors :
Start to Export the F5-TTS Preprocess Part.
Traceback (most recent call last):
  File "D:\F5-TTS-ONNX\Export_ONNX\F5_TTS\Export_F5.py", line 268, in <module>
    f5_model = load_model(F5_safetensors_path)
  File "D:\F5-TTS-ONNX\Export_ONNX\F5_TTS\Export_F5.py", line 197, in load_model
    return load_checkpoint(model, ckpt_path, 'cpu', use_ema=True)
  File "D:\F5-TTS\src\f5_tts\infer\utils_infer.py", line 168, in load_checkpoint
    model.load_state_dict(checkpoint["model_state_dict"])
  File "D:\F5-TTS\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CFM:
        Missing key(s) in state_dict: "mel_spec.mel_stft.spectrogram.window", "mel_spec.mel_stft.mel_scale.fb".
        size mismatch for transformer.text_embed.text_embed.weight: copying a param with shape torch.Size([2555, 512]) from checkpoint, the shape in current model is torch.Size([2546, 512]).

The model is from here https://huggingface.co/marduk-ra/F5-TTS-Turkish. I downloaded both .safetensors and vocab.txt and using them as inputs yet getting these errors.

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (tensor(float)) , expected: (tensor(float16))
Apologies for the delayed response; it's morning now in our timezone (+8).
This error arises from the audio input format. Both parts A and C, in addition to part B, need to be converted to fp16. The format check occurs during the loading of model A, and it will automatically convert the audio to fp16 if 'Float16' is detected.
Inference.py, line 154:

if "float16" in model_type:
   audio = audio.astype(np.float16)

To export the specific F5 version, please download the latest F5-TTS-ONNX release. I have made some adjustments. After downloading, replace the vocab.txt file with your own. The path to the vocab.txt file is:
/F5-TTS-main/data/Emilia_ZH_EN_pinyin/vocab.txt

It should works, enjoy : )

The latest version is changing something in "F5-TTS\src\f5_tts\infer\utils_infer.py" and it always gives this error :
(venv) D:\F5-TTS-ONNX\Export_ONNX\F5_TTS>python Export_F5.py
Traceback (most recent call last):
  File "D:\F5-TTS-ONNX\Export_ONNX\F5_TTS\Export_F5.py", line 58, in <module>
    from f5_tts.infer.utils_infer import load_checkpoint
  File "D:\F5-TTS\src\f5_tts\infer\utils_infer.py", line 168
    model.load_state_dict(checkpoint["model_state_dict"], , strict=False)

Even if I change that line or completely redownload that file from the source it always changes back to this state and gives the error.
The original file doesn't contain that, instead this whole section gets added when I run export
        # patch for backward compatibility, 305e3ea
        # for key in ["mel_spec.mel_stft.mel_scale.fb", "mel_spec.mel_stft.spectrogram.window"]:
        #     if key in checkpoint["model_state_dict"]:
        #         del checkpoint["model_state_dict"][key]

        model.load_state_dict(checkpoint["model_state_dict"], , strict=False)
    else:
        if ckpt_type == "safetensors":
            checkpoint = {"model_state_dict": checkpoint}
        model.load_state_dict(checkpoint["model_state_dict"])

Sorry, It seems too more "," here.
model.load_state_dict(checkpoint["model_state_dict"], , strict=False)
It should be "model.load_state_dict(checkpoint["model_state_dict"], strict=False)"

Yes , I found out your utils_infer include that and after I posted this, updated the file , tried again ... Unfortunately
Start to Export the F5-TTS Preprocess Part.
Traceback (most recent call last):
  File "D:\F5-TTS-ONNX\Export_ONNX\F5_TTS\Export_F5.py", line 270, in <module>
    f5_model = load_model(F5_safetensors_path)
  File "D:\F5-TTS-ONNX\Export_ONNX\F5_TTS\Export_F5.py", line 199, in load_model
    return load_checkpoint(model, ckpt_path, 'cpu', use_ema=True)
  File "D:\F5-TTS\src\f5_tts\infer\utils_infer.py", line 168, in load_checkpoint
    model.load_state_dict(checkpoint["model_state_dict"], strict=False)
  File "D:\F5-TTS\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CFM:
        size mismatch for transformer.text_embed.text_embed.weight: copying a param with shape torch.Size([2555, 512]) from checkpoint, the shape in current model is torch.Size([2546, 512]).

Use your custom vocab.txt (with 2555 words).
The path is: /F5-TTS-main/data/Emilia_ZH_EN_pinyin/vocab.txt, Go to that folder and replace it.

Finally did it, and the tests are successful as well. Thanks a lot ! I am not actually a developer but trying to share my knowledge with using amd gpu's with various ai applications. I really appreciate what you are doing here, keep up the good work ! And have a nice day.


Remove Skyhour as it appears to have ended

Company site is not working for a few days and internet presence stopped.

Crunchbase still states the company is open but seems like it's an empty shell https://www.crunchbase.com/organization
Company domain is still registered but does point to any website
No open positions https://landing.jobs/at/skyhour and according to linked only has 5 people now https://www.linkedin.com/company/skyhour/about/
Dead social media sites https://pt-pt.facebook.com/skyhour/ https://twitter.com/skyhour

Original added on #90

Hi @G4brym, anything to add?

Hi, yes, the company has gone under during the beginning of covid


Implementation of Mergeable and SetOperations for pilosa's lookups

! Should be merged after: https://github.com/src-d/go-mysql-server/pull/277 !
This PR. Implements 2 interfaces (Mergeable and SetOperations) for following lookups:

indexLookup
ascendLookup
descendLookup

It closes: https://github.com/src-d/go-mysql-server/issues/212
simplified example
       unionLookup := lookup0.Union(lookup1, lookup2)

       ascLookup := pilosaIdx.AscendLessThan(keys...)
       diffLookup := ascLookup.Difference(unionLookup)

Ok, right now is  squashed and rebased (+ some fixes)

Ok, now it looks it contains only my chnages.


Fix shell quoting that stopped the alias command from working in zsh.

As mentioned in #236, the existing command didn't work in zsh because it needs to be properly quoted there. Not really sure why it worked in bash to begin with.

As mentioned in #236, the existing command didn't work in zsh because it needs to be properly quoted there. Not really sure why it worked in bash to begin with.

I think $() is its own quoting in bash, apparently not in zsh?


Examples list refactored

Hardcoded indexes & switch parsing refactored in OOP-friendly style

@freemo, could you have a look?


feat(preset-mini): add svh, lvh and dvh values to default theme

this adds svh, lvh and dvh as default values to height, min-height, max-height rules to improve compatibility with TailwindCSS v3.4.0

vh units are already supported. Maybe we can adjust its usage in the size rules?

vh units are already supported. Maybe we can adjust its usage in the size rules?

This adds the default rules of dynamic viewport height, not the values (which as you mentioned already exist), meaning min-height-dvh will generate min-height: 100dvh, which Tailwind just added in v3.4.0.
I think it's more reasonable to leave it in static rules because it is a static rule (does not have any dynamic value in it). It is just for Tailwind compatibility. For pure values like min-h-100dvh, support already exists so there is no reason to add that in sizes.

Then I think it is better to add this to the preset-wind instead

Then I think it is better to add this to the preset-wind instead

Done. I think that makes more sense.


not really important, but gamesandthings is broken on steam deck

i know steam deck can run these games natively (probably) but still worth pointing out

i dont own a steam deck, so unfortunately cannot test. try using chrome browser on desktop mode maybe it will work but using the native one it probably wont work

yeah, also big picture mode basically emulates the steam deck ui

yeah but im not sure if that also emulates the browser. will test

on steam big picture the context menu doesnt work, is that what you mean by it being broken?

yeah kinda


Allow users to disable theming in UXL Launcher and stick with the same appearance as Version 3.0 AND keep theming off by DEFAULT.

Due to theming limitations in Version 3.1 introduced with the theming of the menubar via a custom Professional Renderer, the menubar's back color is white instead of the Control color in Version 3.0. Something to also disable if the user doesn't want theming enabled would be the feature that causes the highlighted menuitems to have black text.
Also, keep theming off by default to not confuse people who used Version 3.0.

This would be controlled by a checkbox on the Personalization tab in the Options window that says "Enable alternate themes/color schemes".

Actually, the checkbox might say "Enable UXL Launcher Theme Engine" and the tooltip will say, "This allows you to change the colors of the UXL Launcher interface via predefined or custom themes."

Make sure to use the tooltip posted earlier!
"This allows you to change the colors of the UXL Launcher interface via predefined or custom themes."

I'm just having the theme info textbox tell the user what the theme engine can do if it's disabled rather than adding another tooltip. This will be closed after version 3.1 Stable is released.


[BugFix] Fix split constant non-ASCII source and empty delimiter (#29818)

Fixes #issue
What type of PR is this:

[ ] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[x] Yes, this PR will result in a change in behavior.
[ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.1
[ ] 3.0
[ ] 2.5
[ ] 2.4

[FE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[BE Incremental Coverage Report]
:heart_eyes: pass : 8 / 9 (88.89%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exprs/split.cpp
8
9
88.89%
[45]

[BE Incremental Coverage Report]
:heart_eyes: pass : 9 / 10 (90.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exprs/split.cpp
9
10
90.00%
[45]


Timeout when connecting to AIserver

I don't really know if this is a bug in Double-Take but I'm hoping it is possible to get some more debugging info.
I run a RaspberryPi 4 with CodeServer.AI as docker image, plain and simple:
  double-take:
    container_name: double-take
    image: skrashevich/double-take
#    image: jakowenko/double-take
    restart: unless-stopped
    volumes:
      - /home/pi/.config/appdata/double-take:/.storage
    ports:
      - 3000:3000

I'm running the latest version: 1.35.5:
Piece of logging:
info: Double Take v1.13.5
info: MQTT: connected
info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
info: Double Take v1.13.5
info: MQTT: connected
info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

It times out when trying to connect to the AIserver. In the config:
  aiserver:
    url: http://<IP_ADDRESS>:32168
    # number of seconds before the request times out and is aborted
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
    # only process images from specific cameras, if omitted then all cameras will be processed
    # cameras:
    #   - front-door
    #   - garage

In the same config it does connect to comprefase.
Both double take and aiserver are running on the same docker host. It doesn't matter if I use the redirected port or the internal IP of the docker container.
On the docker host I see the port 32168 being open.
Config of the aiserver:
  codeproject.ai:
    container_name: codeproject.ai
    image: codeproject/ai-server:rpi64
    restart: unless-stopped
    volumes:
      - ${DOCKERCONFDIR}/codeproject.ai/data:/etc/codeproject/ai
      - ${DOCKERCONFDIR}/codeproject.ai/modules:/app/modules
    ports:
      - 32168:32168

Any thoughts of where to start troubleshooting? I tried running the ping command inside the doubletake container, but it the executable isn't there (docker exec -it double-take /bin/bash).
ps. in aiserver I installed the face processing module and the object-detection modules too.

you need to put double-take and ai.server containers to same network and use internal IP for communicate between them

yes, same issue. I only have one network, and using the internal <IP_ADDRESS> IP results in the exact same timeout.

print output of docker inspect <container> command for both ai.server and double-take containers. needs only "NetworkSettings" part

double-take:
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "2023ea5b237305cc274a168c7ab3ccca1ac2dd29b8bb874b7971651bc66de9fa",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "3000/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "3000"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "3000"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/2023ea5b2373",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "compose_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "double-take",
                        "double-take",
                        "d0ee27f67804"
                    ],
                    "NetworkID": "cd62ebf4a398ab23b85507774a35e300762e712cbc743e818c94a92dd197635b",
                    "EndpointID": "3ee77b58bbe9fc195ee92ecbcad7758e7fd0a6bac2904dd4a1e3b63785258bc4",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:12:00:11",
                    "DriverOpts": null
                }
            }
        }
    }
]

aiserver:
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "8971a265a380412c9f52ea3c3d3ada044b43a948e4cbde8765a77b1e19503c10",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "32168/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "32168"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "32168"
                    }
                ],
                "5000/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/8971a265a380",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "compose_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "codeproject.ai",
                        "codeproject.ai",
                        "6a3b94f1b222"
                    ],
                    "NetworkID": "cd62ebf4a398ab23b85507774a35e300762e712cbc743e818c94a92dd197635b",
                    "EndpointID": "98e4271be389b1f5eda0b8c783fdde5ef0eeb3abc0583bf8d752ac8fe79d00a6",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:12:00:0d",
                    "DriverOpts": null
                }
            }
        }
    }
]

  aiserver:
    url: http://codeproject.ai:32168
    # number of seconds before the request times out and is aborted
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
    # only process images from specific cameras, if omitted then all cameras will be processed
    # cameras:
    #   - front-door
    #   - garage

(if it's will not work, put ip <IP_ADDRESS> instead of "codeproject.ai". I'm not sure how docker works with hostnames with dots)

change your config to
  aiserver:
    url: http://codeproject.ai:32168
    # number of seconds before the request times out and is aborted
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
    # only process images from specific cameras, if omitted then all cameras will be processed
    # cameras:
    #   - front-door
    #   - garage

(if it's will not work, put ip <IP_ADDRESS> instead of "codeproject.ai". I'm not sure how docker works with hostnames with dots)

I tried this too, but I get the same timeout with the IP.
Do you know of any debug logging for double take?

logs:
  level: debug

I have build for you container with some useful tools for troubleshoot, such as ping,traceroute,host,netstat,curl
you can get it on skrashevich/double-take:latestdebug. it's the same as latest build, but with additional tools. use it to check connectivity between your docker containers

fuck. everything was simple. documentation on codeproject site is outdated.
just use port 5000 instead of 32168

fuck. It's a bug in AI.Server. in some cases, related to docker network type (macvlan of ipvlan) it may stops listen on 32168.
try to use port 5000 instead of 32168 in config, compose and docker run also, check CPAI_PORT environment variable (it must be not set) and DisableLegacyPort option in appsettings.json file in ai.server container

And by this you mean 5000:32168 at port settings?

And by this you mean 5000:32168 at port settings?

5000:5000, but it doesn't rely to communicate between containers

mmm.. weird.
It keeps timing out from the GUI of double-take. I now point to url: http://<IP_ADDRESS>:32168, but it doesn't matter what I do, if I use 5000 or any other port assignment. It doesn't matter if I point to the <IP_ADDRESS> IP either, everything times out.
When using your latestdebug build where you have build in curl (tnx for that btw!) it opens the port. When using the command curl -v http://<IP_ADDRESS>:32186 it responds immideately.
See:
</html>root@7d8522660379:/double-take# curl -v http://<IP_ADDRESS>:32168
* Expire in 0 ms for 6 (transfer 0x55a2a60d80)
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55a2a60d80)
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 32168 (#0)
> GET / HTTP/1.1
> Host: <IP_ADDRESS>:32168
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Length: 28475
< Content-Type: text/html
< Date: Wed, 03 May 2023 07:54:31 GMT
< Server: Kestrel
< Accept-Ranges: bytes
< ETag: "1d9749638bdefbb"
< Last-Modified: Fri, 21 Apr 2023 21:14:13 GMT

try the latest build.
I still can't reproduce this problem in my environment, but made some changes which potencially can solve

have you added the chages to latestdebug?
23-05-03 09:52:06 error: AxiosError: aiserver process error: timeout of 15000ms exceeded

downloading now.

downloading now.

is problem solved?

downloading now.

is problem solved?

My apologies. I had a draft reply but haven't hit the comment button, but unfortunately it doesn't help. It keeps timing out.
The weird thing is it can connect from within the container over the port (curl -v command), why it times out doesn't make sense to me at all.

@mhoogenbosch is face processing module enabled and running?
main page of ai.server must looks like as this:

"Face processing" row must exists and have green background

@mhoogenbosch you need to go to "Install Modules" tab, Uninstall and then Install again "Face Processing" module.
After that, it should work. Please note that the installation may take a long time (about half an hour)

@mhoogenbosch you need to go to "Install Modules" tab, Uninstall and then Install again "Face Processing" module. After that, it should work. Please note that the installation may take a long time (about half an hour)

After reinstalling i'm getting the exact same issue.

try to clean reinstall... don't forget to remove contents of volumes:
- ${DOCKERCONFDIR}/codeproject.ai/data:/etc/codeproject/ai
- ${DOCKERCONFDIR}/codeproject.ai/modules:/app/modules

Annoying there is no 2.1.8 for ARM64, latest version is 2.1.4.

do you testing oncodeproject/ai-server:arm64-2.1.4 or oncodeproject/ai-server:rpi64-2.1.4 ?

Annoying there is no 2.1.8 for ARM64, latest version is 2.1.4.

do you testing oncodeproject/ai-server:arm64-2.1.4 or oncodeproject/ai-server:rpi64-2.1.4 ?

arm64, but doesn't matter if I enter rpi64 or arm64, it doesn't pull a new image which means they are the same.

Right.. so
the tag rpi64 leads to 2.1.4 and arm64 to 2.1.3. So I'm back using rpi64.
the face detection module is started and online when double take is not connected.

as soon as i change the config to talk to the aiserver, it crashes. If I piont it to another aiserver instance running on an old HP TC620, based on AMD64 image it runs.
I don't think there is anything in my config which could explain this?
# Double Take
# Learn more at https://github.com/jakowenko/double-take/#configuration
mqtt:
  host: <IP_ADDRESS>
  username: mosquitto
  password: mosquitto

frigate:
  url: http://<IP_ADDRESS>:5000
  attempts:
    # number of times double take will request a frigate latest.jpg for facial recognition
    latest: 10
    # number of times double take will request a frigate snapshot.jpg for facial recognition
    snapshot: 10
    # process frigate images from frigate/+/person/snapshot topics
    mqtt: true
    # add a delay expressed in seconds between each detection loop
    delay: 0

  image:
    # height of frigate image passed for facial recognition
    height: 500

detectors:
#  deepstack:
#    url: http://<IP_ADDRESS>:5001
#    key:
#    opencv_face_required: false
      # only process images from specific cameras, if omitted then all cameras will be processed
      # cameras:
      #   - front-door
      #   - garage

  compreface:
    url: http://<IP_ADDRESS>:8000
    # recognition api key
    key: 971ec64d-b0c2-4f59-9e16-cb9a6e4b0f88
    # number of seconds before the request times out and is aborted
    timeout: 15
    # minimum required confidence that a recognized face is actually a face
    # value is between 0.0 and 1.0
    det_prob_threshold: 0.8
    # require opencv to find a face before processing with detector
    opencv_face_required: false

  aiserver:
    #url: http://<IP_ADDRESS>:32168 // internal docker url
    #url: http://<IP_ADDRESS>:5000 // hp tc620
    url: http://<IP_ADDRESS>:5000
    # number of seconds before the request times out and is aborted
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
    # only process images from specific cameras, if omitted then all cameras will be processed
    # cameras:
    #   - front-door
    #   - garage

Hello,
I have the same problem: double-take can't connect to aiserver (timeout of 15000ms exceeded). On aiserver I only have the "Face Processing" module which starts up fine. But when double-take tries to connect to aiserver, the Face Processing module shuts down with the error (face.py: Fusing layers... Module FaceProcessing has shutdown).
Here are my configurations
Double-take - docker-compose.yaml
`
version: '3.7'
volumes:
double-take:
services:
double-take:
container_name: double-take
image: skrashevich/double-take
restart: unless-stopped
volumes:
- /root/config/doubletake:/.storage
ports:
- 3000:3000
`
code-project - docker-compose.yaml
`
version: '3.7'
volumes:
code-project:
services:
code-project:
container_name: code-project
image: codeproject/ai-server:rpi64
restart: unless-stopped
volumes:
- /root/config/codeproject/data:/etc/codeproject/ai
- /root/config/codeproject/modules:/app/modules
ports:
- 32168:5000
`
Double-take - configuration
`
mqtt:
host: <IP_ADDRESS>
username: ****
password: ****
frigate:
url: http://<IP_ADDRESS>:5000
detectors:
aiserver:
url: http://<IP_ADDRESS>:32168
# number of seconds before the request times out and is aborted
timeout: 15
# require opencv to find a face before processing with detector
opencv_face_required: false
`
Screen error :

Just wanted to chime in on this issue and say that I'm not having problems using aiserver. Wondering if its hardware related? I'm using the GPU airserver docker image and it does seem to work. I haven't actually confirmed if its utilizing my GPU yet, but I am getting recognitions coming through. I'm running this on an amd 5600x with an nvidia 3070. curious what hardware others are using?

v1.13.11


CSE Machine: error when assigning "undefined" to a variable.

While running the below code:
const x = undefined;

The CSE machine will get stuck at the step where undefined is being assigned to x. All subsequent steps will remain stuck on the same screen, but steps before this are still fine.
If you attempt to skip this assignment step using the slider, the steps after still will not work, but instead show whatever previous screen you were on.
Can be reproduced on https://sourceacademy.org/playground but works correctly on https://sourceacademy.nus.edu.sg/playground.

https://sourceacademy.nus.edu.sg/ is frozen on commit e1a3267e5c4b1f63c067dd42d76e4533c70b8a55, so if it works on the latter, means the regression is likely to be somewhere between these commits: https://github.com/source-academy/frontend/compare/e1a3267e5c4b1f63c067dd42d76e4533c70b8a55...master

I've found the reason here. The value of this.environment.head[key] is undefined, so the simplest fix would be to just place a ? in front of the descriptor property.
However, what is more concerning is that any error that occurs are not thrown at all inside the browser console, as it did take me some time to debug this manually. I found out that all errors related to the CSE Machine are actually suppressed by the try-catch statements inside JsSlangHelper.ts and WorkspaceSaga.ts.
May I know if there is a good reason why these try-catch statements are necessary? As without them, this issue and many others could have been solved much quicker.


Different pot. Energy

Hello, I got different energy in my case than the paper, what could be the reason? Could it be the reason that may be our a,b, etc, values are different? What you think? Any help would be appreciated.

how much did you get ??
if -28 or near so you are fine because the value in the paper is for potential energy per ion pair

Yes exactly the value we get is total potential energy after optimization but the value given in paper is the value of binding energy per ion.

okay thank you


Code for BLOB Insert to DB2 v11.1 through SOCI

Hi,
Can anyone post a sample code cut to Insert and Fetch BLOB to a table in db2 with version 11.1 ?
The following Statement working in DB2 v10.5 but not in v11.1.
statement Stmt = (sql->prepare<<"INSERT INTO TABLE_BLOB (COL1) VALUES (BLOB(:host_variable)), use(host_variable)) ;
where host_variable is a local variable of unsigned char and COL1 is of blob datatype.
Error that I am getting in DB2 v11.1 is ;
Statement execution error SQLMESSAGE: [IBM][CLI Driver][DB2/LINUXX8664] SQL0440N  No authorized routine named "BLOB" of type "FUNCTION" having compatible arguments was found.  SQLSTATE=42884
SOCI Version : 3.2.2
Linux : RHEL 7.3 (64 Bit)

The same question was asked on SO as Code for BLOB Insert to DB2 v11.1 through SOCI and remains unanswered.

The issue is unclear or report incomplete or simply there is no DB2 wizard around who would be able to tell if, perhaps the issue/incompatibility lays around client library used with SOCI.
Closing as there is no point to keep it open for long time.


Please update to 1.19.3

It would be nice to see an update for Minecraft 1.19.3, as this is the last mod I need that is not updated for 1.19.3. What would be a rough estimate for this update?

yes, an update would be greatly appreciated :)
especially useful for monitoring and resolving conflicts :))

I'll go ahead and get an update to 1.19.3 out asap

Updated to 1.19.3

tysm

Wow that was so fast! Thank you very much! ✧\(>o<)ﾉ✧


Removed extra word

There is a spurious "either"

I don't understand what the conflict is. I only have 1 commit and it's on top on master :thinking:

Ah, GitHub now says there are no conflicts :man_shrugging:


NotImplementError

hi, thank you for your project!
may I ask if you modify the original RAFT Network in order to convert it to onnx ? because I found the input is   fmap1, fmap2 = self.fnet([image1, image2],True), but the origin RAFT only input two images without the bool element 'True'.
Following the Readme.md, I just get the error below:

could you give me some help?
if you change the original RAFT code ,could you share what you have changed?

I'm sorry, I missed a few notes. You should use my slightly modified RAFT project, that is https://github.com/chenjianqu/RAFT

Thank you so much for your quick reply! but unfortunately, I still get the same error with your modified code.
I have tried the following example , it works well ,so I think my environment is OK, so Is there anything I was missing ?

I just modify export_onnx.py like this：

and modify core/raft.py like this：

and I just use this command to convert the model:
python export_onnx.py

I'm sorry, I missed a few notes. You should use my slightly modified RAFT project, that is https://github.com/chenjianqu/RAFT

Thank you so much for your quick reply!

when compile the code , I get these two error:
first:

second:

my tensorRT version is tensorRT8, and my g++ version is g++-10. so why I get the errors when compile? thanks

The first issue seems to be with the TensorRT version, I'm using TensorRT8.0.1 and there is no problem.
The second problem is with the GCC version, which requires GCC to support at least C++17.

Or, alternatively, you can modify the code slightly to make the project work.

Or, alternatively, you can modify the code slightly to make the project work.

你好，感谢你的工作！你说你对原RAFT网络进行了一些小修改，能否说一下你具体修改了什么？修改了之后你是重新训练了网络，还是想办法用了原作者提供的权重呢？


get comments progress bar incorrectly uses dataset comment count as total, not source

Reproduce against staging with:
cargo run -- get comments org1/collateral --dataset org1/collateral --reviewed-only true -f tmp.jsonl

The total number shown in the bar matches the total number of comments shown in the org1/collateral dataset settings pane, even though at most we'd download comments from the single source.

Opened https://github.com/reinfer/cli/issues/38

Closed as moved to backlog


how to get product_name OrderLine class in orders class

Can anyone tell me where this data is coming from?

If I understood You correctly, You might be interested in templates/order/_ordered_items_table.html.
{% for line in order %}

          {{ line.product_name }}

x {{ line.quantity }}

              {% price line.get_total display_gross=order.display_gross_prices %}

{% endfor %}

@saqibkhan2523 You should expand your Order model, then some changes in forms might be required, not sure what's your experience with Django, but Tango with Django book is usually a great start

Sorry if i was not clear enough but here is what i am doing:
As you can see in the image above i have created "Downlaod" which contains a link to some file.Every product has its own link when they are created . Now i have updated Product class in models and added a link field. From there i have tried multiple logics to get link of the respective object in that row above but its not working.
If you can tell me how to get that download link linked to orders detail page(what change are required in orders, orderline etc) it will be a great help.

You should take a look at the following function https://github.com/mirumee/saleor/blob/master/saleor/order/views.py#L27-L48
From what you can see, in the context there's order object, then in the file linked by oldPadavan above we are iterating over Order getting OrderLine {% for line in order %}
Orderline contains ForgeinKey to the ProductVariant (variant field), from ProductVariant contains ForgeinKey to the Product via product field.
So for each line you could use something like {{ line.variant.product.your_custom_url_field }} for accessing your product link url via templates

It works, Thanks a lot for this and also for the book.


Improve type support for model-related methods

Description
Enhanced type support by adding docblock annotations with class-string<Model> and, in some cases, adding the native type Model & Authenticatable to ensure the object extends the Model class and implements the Authenticatable interface.
Addressed an edge case for SpatieTagInput by checking if the getTagClassName method exists before calling it. This was necessary because there isn't a docblock annotation to describe an instance of a class using a specific trait.
Visual changes
None.
Functional changes

[x] Code style has been fixed by running the composer cs command.
[x] Changes have been tested to not break existing functionality.
[ ] Documentation is up-to-date.

Hi, I'm willing to do this but please target it at Filament v4. Thanks

Alright! I'll do that!

Pull request targeted to v4: #14083


raftstore panics during merge

Bug Report

What version of TiKV are you using?

b2fad976da192006888e2fc80746f9196cf425b6
What operating system and CPU are you using?

x86-64 CentOS 7.6
Steps to reproduce

Running tests on TiCDC with a large write throughput and CDC scan throughput.
What did you expect?
That everything works.
What did happened?
[2021/04/07 16:23:44.597 +08:00] [FATAL] [lib.rs:455] ["assertion failed: min_matched >= min_committed"] [backtrace="stack backtrace:\n   0: tikv_util::set_panic_hook::{{closure}}\n   1: std::panicking::rust_panic_with_hook\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/panicking.rs:595\n   2: std::panicking::begin_panic_handler::{{closure}}\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/panicking.rs:495\n   3: std::sys_common::backtrace::__rust_end_short_backtrace\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/sys_common/backtrace.rs:141\n   4: rust_begin_unwind\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/panicking.rs:493\n   5: core::panicking::panic_fmt\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/core/src/panicking.rs:92\n   6: core::panicking::panic\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/core/src/panicking.rs:50\n   7: raftstore::store::peer::Peer<EK,ER>::propose_normal\n   8: raftstore::store::fsm::peer::PeerFsmDelegate<EK,ER,T>::propose_raft_command\n   9: raftstore::store::fsm::peer::PeerFsmDelegate<EK,ER,T>::handle_msgs\n  10: <raftstore::store::fsm::store::RaftPoller<EK,ER,T> as batch_system::batch::PollHandler<raftstore::store::fsm::peer::PeerFsm<EK,ER>,raftstore::store::fsm::store::StoreFsm<EK>>>::handle_normal\n  11: batch_system::batch::Poller<N,C,Handler>::poll\n  12: std::sys_common::backtrace::__rust_begin_short_backtrace\n  13: core::ops::function::FnOnce::call_once{{vtable.shim}}\n  14: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35/library/alloc/src/boxed.rs:1484\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35/library/alloc/src/boxed.rs:1484\n      std::sys::unix::thread::Thread::new::thread_start\n             at /rustc/bc39d4d9c514e5fdb40a5782e6ca08924f979c35//library/std/src/sys/unix/thread.rs:71\n  15: start_thread\n  16: __clone\n"] [location=/home/tidb/tikv/components/raftstore/src/store/peer.rs:2813] [thread_name=raftstore-1-0]

The progress.committed_index may not accurate and progress.matched >= progress.committed_index may not be true as https://github.com/tikv/raft-rs/issues/426 found.
This assert just for robustness and not a fatal error, I think we can remove it and print a warning log instead /cc @BusyJay @gengliqi

print a warning log

Sound like a good quick fix.

The async ready has not been used already. So the panic happens probably because of the case mentioned here https://github.com/tikv/raft-rs/issues/426#issuecomment-814970073


Learn more about FastAPI

Learn more about FastAPI:

by watching videos and coding

Add new routes, edit the requerments file

Youtube video about FastAPI


WIP: Move constant time RSA padding checks out of libssl

In TLSv1.2 a pre-master secret value is passed from the client to the
server encrypted using RSA PKCS1 type 2 padding in a ClientKeyExchange
message. As well as the normal formatting rules for RSA PKCA1 type 2
padding TLS imposes some additional rules about what constitutes a well
formed key. Specifically it must be exactly the right length and
encode the TLS version originally requested by the client (as opposed to
the actual negotiated version) in its first two bytes.
All of these checks need to be done in constant time and, if they fail,
then the TLS implementation is supposed to continue anyway with a random
key (and therefore the connection will fail later on). This avoids
padding oracle type attacks.
This PR implements this within the RSA padding code so that we keep
all the constant time padding logic in one place and moves it out of libssl.
Marked as WIP because this code depends on #10152. Only the last 3 commits of this PR are relevant.

You might want to read #8446

You might want to read #8446

Thanks that's interesting reading. This PR doesn't attempt to solve the problem of the public API so in a sense this PR is tackling a slightly smaller problem, i.e. it is motivated by the need to remove the constant time padding code out of libssl and into the providers - and in particular the FIPS provider. For FIPS we're not allowed to have this crypto code outside of the module boundary.
However there is clearly a big overlap with what you started in #8446. It seems our thinking is largely aligned judging by your last comment:

So I think the suggestion is to have something for use in TLS only. Either a new API, or maybe just a padding mode in the current API. I'm currently leaning towards just creating a padding mode and not check the TLS version.

This PR implements a new padding mode. Although I'm not clear what you are saying in the last bit about "not checking the TLS version". If you are referring to the client version check - I think that does need to be done in the constant time code.

What I would like to see is that the new padding check function
only accepts properly padded input. That is, the caller should
always use BN_bn2binpad(). The padding check function should also
know the expected output size, which is known for TLS. If you
combine those 2 properties, the code can be simplified a lot.

A rebase would be nice...

What I would like to see is that the new padding check function
only accepts properly padded input. That is, the caller should
always use BN_bn2binpad(). The padding check function should also
know the expected output size, which is known for TLS. If you
combine those 2 properties, the code can be simplified a lot.

I separated out the TLS padding check into its own function rather than mixing it with the existing one. This new function expects properly padded input and knows the output size as you suggested.
I've also rebased this PR and taken it out of WIP. This is now ready for review.

I've updated as per the feedback above. I also had to rebase due to conflicts with master.

I think this looks good, but I currently don't understand the code about the parameters, so I won't completely review this.

I think this looks good, but I currently don't understand the code about the parameters, so I won't completely review this.

Thanks. I've pushed some fixup commits to address your latest feedback.
Since @kroeckx won't be providing the sign off for this, can someone else please review?

Please take a look at the latest fixups!

Travis red cross is not relevant.

Pushed. Thanks.


Adding support for RTL languages

Requirements for Contributing a Bug Fix
Identify the Bug
https://github.com/atom/atom/issues/10132 and all their related RTL issues.
https://github.com/atom/atom/issues/17276
https://github.com/atom/atom/issues/15628
Description of the Change
This is an initial push of the changes required for Atom to support RTL languages.
Here's the existing behaviour of Atom with RTL languages:

Now here's it working with RTL after the submitted changes:

You can clearly tell that it's not fully done yet for merging into the master branch. There are many bugs/features to be added here to make it fully functioning.
So here's why I'm opening this pull request before code is ready:

RTL are either closed because they're too old or locked as too heated.
So there is no (productive) channel to provide a collaborative ground for developers interested in RTL.

I'm hoping this would be a good starting point for all RTL developers to show their skills and contribute to this feature.
To get this done:

Please clone this fork and run it on your machine.
Please fill in the list of current issues that should be solved. Add them here in a comment. A GIF would be great.
Please contribute to the code and fix this issue. Please clone this fork and submit PR to it.
Please write test cases to cover up the following, RTL only text or LTR and RTL text in the same line on LTR/RTL text editor.

Alternate Designs

Possible Drawbacks
The code changes enforces RTL on the text editor and won't work on TRL. As started above, this code is not ready for merge just yet and needs further collaboration.
Verification Process
TBD
Release Notes

Text editor can now handle RTL text - still few bugs to fix before merging it.

@lkashef , please have a look and invite other members to contribute to it, if possible.

Hi @mohataher , Can You fix the lint errors so the CI can be green?

@sadick254 Could you leave a comment about the information you need? A tag does not help the author.

@mohataher Could you run script/lint --fix?

Hey @mohataher. Thank you for putting a lot of effort into this PR. It would certainly be a good addition to Atom. However, we won't be accepting this new feature. The maintenance cost of this new feature is huge and Atom currently doesn't have the capacity to deal with issues and pull requests that come as a result of this addition.
Once again, thank you for all the hard work on this.

No problem @sadick254. I ran out of resources to keep pushing this feature. We ended up forking and using our own solution internally. Will be happy to give support for anyone interested to push this important feature. Please message me if needed.

Dear @mohataher, is it possible to help us using your solution? I tried to follow your instructions above, but then the LTR broke. Are there other changes that I have to get?
I appreciate any help.


chore(main): release 2.0.0

:robot: I have created a release beep boop
2.0.0 (2023-10-25)
⚠ BREAKING CHANGES

drop the node v16

Miscellaneous Chores

changed the node version (ada0a71)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/A7mooz/ts-gen/releases/tag/v2.0.0 :sunflower: