[MINOR] Parallelize CI tests by modules

Unit tests in total takes ~40 min to finish and unit tests in hudi-client along takes 20 min. This parallelizing setup is to reduce CI wait time.
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@xushiyan : I understand this is still in works but thank you for taking this up and trying to improve CI experience for everyone.

Codecov Report

Merging #1619 into master will decrease coverage by 3.73%.
The diff coverage is n/a.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 404c7e8...48e863b. Read the comment docs.

@bvaradar The jobs are split quite evenly and CI time is reduced by 10+ min.. however, it is unknown to me why codecov is affected though I saw reports were submitted from each job
https://travis-ci.org/github/apache/incubator-hudi/jobs/686266577
==> Reading reports
    + ./packaging/hudi-spark-bundle/target/site/jacoco-ut/jacoco.xml bytes=2543953
    + ./hudi-client/target/site/jacoco-ut/jacoco.xml bytes=736300

https://travis-ci.org/github/apache/incubator-hudi/jobs/686266578
==> Reading reports
    + ./packaging/hudi-spark-bundle/target/site/jacoco-ut/jacoco.xml bytes=1809177
    + ./hudi-hadoop-mr/target/site/jacoco-ut/jacoco.xml bytes=138889
    + ./hudi-timeline-service/target/site/jacoco-ut/jacoco.xml bytes=42892
    + ./hudi-hive-sync/target/site/jacoco-ut/jacoco.xml bytes=73842
    + ./hudi-spark/target/site/jacoco-ut/jacoco.xml bytes=215369
    + ./hudi-utilities/target/site/jacoco-ut/jacoco.xml bytes=300100
    + ./hudi-common/target/site/jacoco-ut/jacoco.xml bytes=1338299

https://travis-ci.org/github/apache/incubator-hudi/jobs/686266579
==> Reading reports
    + ./packaging/hudi-spark-bundle/target/site/jacoco-ut/jacoco.xml bytes=2544517
    + ./hudi-spark/target/site/jacoco-ut/jacoco.xml bytes=215285

@ramachandranms @prashantwason : Can you help with the question that @xushiyan  has.
Thanks,
Balaji.V

No idea why its not merging. Everything looks good.

@xushiyan here's my take.
our code coverage reporting works by running unit tests in all modules and then the report is aggregated to one under packaging/hudi-spark-bundle. this report supersedes all the other smaller reports since it has aggregates of all of them.
Now when you split the test into two modules across two jobs in travis, the following happens

job with hudi-client finished first and uploads report only for hudi-client module
job without hudi-client finished last and uploads report excluding hudi-client module

this can be seen from

since there can only be one report against a commit in codecov, whichever job finishes last will overwrite the other. codecov does not aggregate report across multiple uploads. this means we are losing all the coverage from hudi-client.
Few comments on the parallelization,

Travis has a queue size. Usually 10 per repo. Since we are in apache repo and apache has bought 50 capacity in ~2014, we are sharing the queue with lot of other projects. I am not sure if they upgraded the queue size beyond 50.
Since the queue is shared, adding more jobs to the queue will be worse in the bigger picture since we are going to take 45 mins across 2 jobs instead of 30 mins for one. This might create bigger backlogs in the CI queue.
Also each job is independent of each other and it will be very hard to collate the unit tests across each other to arrive at a global coverage number for the project. We will have to push the reports to some external storage, run another job just to aggregate and then publish that result.

@ramachandranms : Jobs in travis-ci.org has a hard timeout of 50 mins. Splitting the jobs serves as the only way to be under this time-limit. From @xushiyan comment, I see that both jobs uploads report from hudi-spark-bundle. If one of the job does not upload from hudi-spark-bundle, will it work ?

@bvaradar i think a better idea would be to see how we can optimize the tests than breaking up into multiple jenkins jobs.
regarding missing coverage: the problem is more about missing coverage due to splitting up than multiple uploads. since we have cross module tests, not running some of the modules for the uploading job will miss those tests and coverage will drop.

@bvaradar @ramachandranms thanks for chime in.. from the info @ramachandranms provides, hudi-spark-bundle report will be overwritten by the last submission and that submission should cover all modules. So previously when we have only 2 jobs (unit tests and integration tests), it is due to unit tests taking longer and covering all modules so that report was accurate. If integration test job takes longer and finishes later, coverage report will also be affected. We need to avoid this kind of dependency.
When we modularize projects, by design we should be able to and encouraged to run unit tests for each module independently. The integration test then serves the purpose of checking cross-module functionalities. As for the overall coverage report, I think it should be a pure appending of all kinds of test reports without chronological overwriting. Thus, I think the setup perhaps can be improved in such way that all jobs are free to submit the reports whenever they finish and codecov  sums up all submissions.

Optimizing/Redesigning tests should be definitely done from modularity, test-quality and maintenance perspective and having running-time reduction would be a good side-effect. But, that may not guarantee that we will be under the time-limit.
I agree with @xushiyan that this looks like a race condition with code coverage report generation which needs to be resolved.

Parallelize CI tests is an nice idea ! 👍

Hi all, thank you for the feedbacks. Seems like it is favorable that the we avoid code coverage report to be override across running tasks. @ramachandranms would you agree with generating the reports independently from each modules and aggregated at codecov side? I imagine this is a feasible way to enabling tests parallelization. Would like hear your feedback on making such change. Thanks.

Hi Raymond. I very much like the idea of speeding up the tests and spitting the tests into multiple modules is a great way to do it. Unfortunately, AFAIK codecov doesnt support merging at their server side. They merely just parse the xml we upload and show it in the UI. You can may be check with the codecov community and see if they support any such feature.

Thank you all for the feedbacks..will look into codecov.. closing this for now


Fixes for multiple pretrained model files

Close https://github.com/NVIDIA/DIGITS/issues/849.
See explanation at https://github.com/NVIDIA/DIGITS/pull/498#issuecomment-226550671.

@gheinrich can you take a look?

This looks good to me. On Linux this is working for me. It's difficult for me to test on Windows as I don't have the set-up. I just verified that os.pathsep returns ; on Windows.

I will test it here on Windows.

Did the new image classification model page remove pre-trained weights field?  I couldn't find it on 4.0.0-rc.2.

Did the new image classification model page remove pre-trained weights field?

Look under the "Custom Network" tab

Thanks.  My bad.  Still a lot to learn about DIGITS.
I tested on Windows.  Separating multiple weight files with ';' and path starts with /, both caffe.exe and pycaffe can load those weights.  However, there is a bug in _pycaffe_args, so I need to send a PR on loading_script portion of _pycaffe_args.

Great, so we're good? Please merge if so.

Looks great.  I believe it works good in Windows now.


Consider using nodata based on reading value from file

Description of the Feature to be Added

In UrbanCentre, consider reading the nodata variable from the raster file itself as opposed allowing the user to define it. This could help minimise mistakes when compared to users entering it manually. The nodata variable can be read in at the same time as the raster crs, and then stored as a Class attributed (mangled). Then re-use this throughout the class.
This request relates to the review of PR #37.

Proposed solution has some issues:

Input from population raster is type float. Depending on whether the source is the original or the merged raster, nodata can be -200.0 or float("nan").
Intermediate outputs from the urban_centres module are rasters that either are of type bool or int.
When preparing a DataArray to vectorise, because the data type is int it won't accept a float for nodata.

Since the intermediate outputs of the urban_centres function are always going to be int, I propose that an easier solution would be to fix the value of nodata, removing it as an argument. Note that this value is only required for an internal transformation and will have no impact in the output.


Hand Written digit prediction

What is the objective of the Script?

It is a simple web app integrated with small convolutional model which takes the input drawing on canvas and predicts the output and displays the confidence of that predicted number

What the features will your script have
It is functional on all devices by adjusting itself to the screen and provides the suitable space for canvas ( for user to draw ) and gives the accurate outputs in realtime.

Screenshot

kindly assign this issue to me under GSSoC label.

Go Ahead @Raghucharan16


Roll back buildkit version

Description
It seems like our docker builds have been failing a lot recently with the message:
#33 ERROR: failed to push ghcr.io/thepalaceproject/circ-webapp:sha-blahblah: failed to copy: io: read/write on closed pipe

Motivation and Context
It looks like there are two upstream issues where others are seeing this:
https://github.com/docker/build-push-action/issues/761
https://github.com/moby/buildkit/issues/3347
One suggestion was to roll back the version of buildkit we are using temporarily. It appears this issue is correlated with the release of buildkit v11, so this may fix the issue.
Once there is a fix for this in buildkit, we will have to revert this fix so we are getting the latest version again.
How Has This Been Tested?
Going to let CI run and see if it works.

Well this build worked. Not really proof this fixes the issue, since its intermittent anyway, but seems like this workaround couldn't hurt?

Added additional debug option suggested, so if it fails we get some logs that might be useful to send upstream to troubleshoot.


66 meta info files

close #66

:+1:

Coverage remained the same at 95.789% when pulling 4620ae60a0db04b5f9fd6d3f55dd9f219f59c6dc on 66-meta-info-files into 0c10844cece3688c0081e4cceb31aab8c1468308 on master.

:+1:

Coverage remained the same at 95.789% when pulling 86505cd298a0c52aa0257bca39dad32c7af3d968 on 66-meta-info-files into 0c10844cece3688c0081e4cceb31aab8c1468308 on master.

👍


Weechat > irssi

I know it's blasphemy, but I felt the same way about screen and tmux originally.

Hahahaha nah it's awesome

Sorry I thought I merged forever ago!


Error running robotjs on electron

I tried to rebuild robotjs then this error https://imgur.com/a/eVAB26i
after i tried to start then this error https://imgur.com/a/9zjHfNq
any help would be appreciated
Environment

RobotJS version:lastest
Node.js version:lastest
npm version:latest
Operating System:win 10 64Bit

Are you sure that you followed that part of the documentation? https://github.com/octalmage/robotjs#building
Also, please set the versions in your issue

Are you sure that you followed that part of the documentation? https://github.com/octalmage/robotjs#building
Also, please set the versions in your issue

Yep did everything

@Angshu1 This problem doesn't seem a robotjs problem. A related issue has been opened. I'm not a windows user and can't help you directly but this link would be able to resolve your problem.

Thank you @mznet, closing it


[backport] Try to use -I in hipRTC

Backport of #5486 by @leofang

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit e7cc66fc66a29bb45f3ed360ea6839c89e7f75a9, target branch v9) succeeded!

doctest memory limit error seems unrelated


Discord Canary completely incompatible and only for me

On the latest Canary version of Discord, every time I select install or repair on Canary and it runs through it's process, it has been completely unusable. I'm not sure what happened but it all started maybe during the last weekend if I remember correctly.
There's multiple problems that this causes that makes it practically useless. The first few obviously noticeable problems I'm having is that the zoom is incredibly high and can't be changed and it has no bar at the top to close, maximize, or minimize. I am permanently muted and can't unmute myself. Going into settings shows that discord doesn't pick up any of my microphones or my camera. Not to mention none of my settings are able to be changed and do anything in reference to my audio or size. In the bottom images, it also shows that my discord is acting like it's open in a browser tab instead of the actual app which is absolutely unbelievable.
When I try and join a call or server voice channel, I doesn't say I'm connected in the badge on my taskbar with the green circle. Not to mention I can hear them still, but still can't unmute or say anything. And on top of all that, in settings, it doesn't even show that BandadgedBD is even installed.
Now despite all of this, regular discord surprisingly works just fine. Nothing has changed at all for it. I even tried uninstalling, ridding of all of my plugins and themes, etc. Doing the same for both has done nothing to fix Canary and regular Discord still works just like how it and Canary is supposed to.

Correct, BD is incompatible with latest Canary.

Correct, BD is incompatible with latest Canary.

Not entirely, my girlfriend is actually using canary too and it works just fine.

Not entirely, my girlfriend is actually using canary too and it works just fine.

Not working for me on Canary, latest Host Version: 1.0.21

Not working for me on Canary, latest Host Version: 1.0.21

Not working for me either, latest Host version: 1.0.21
Kinda annoying because I don't really wanna switch to PTB/Stable

Not working for me either, latest Host version: 1.0.21
Kinda annoying because I don't really wanna switch to PTB/Stable

It will be some time before it works on Canary again, so if you want to use BD, switch to either stable or ptb.

It will be some time before it works on Canary again, so if you want to use BD, switch to either stable or ptb.

Same here, gonna have to wait i suppose

Same here, gonna have to wait i suppose

That's more or less what I just indicated.

That's more or less what I just indicated.

Relax. People are just confirming that they too have an issue and it doesn't work for them either and subscribing to the thread.

Relax. People are just confirming that they too have an issue and it doesn't work for them either and subscribing to the thread.

Users are much better off indicating they face a similar issue by reacting with 👍 to the original issue post. They can also click the button on the right hand side if they wish to subscribe to the issue.

Users are much better off indicating they face a similar issue by reacting with 👍 to the original issue post. They can also click the button on the right hand side if they wish to subscribe to the issue.

Some people want to be noticed. The more widespread a problem is the more important it seems to people. Let people feel important.

Some people want to be noticed. The more widespread a problem is the more important it seems to people. Let people feel important.

Sorry, that's not what Github issues are for.

Sorry, that's not what Github issues are for.

For those that want more information on what the underlying issues are, you can read some discussion in #442 and also in this issue in the powercord repo.

This has finally been fixed. You can install the new version using https://github.com/BetterDiscord/Installer


When mounting a theme folder, /var/www/html is deleted in the container (except for the theme folder)

Hi,
When I add this volume mount:
- ./themes/themename:/var/www/html/themes/themename
Where "themename" contains lang/en/*.php
Once the container starts, the entire /var/www/html is empty except for the themes folder.
Probably just doing something dumb in docker, but if someone could point me to a working docker-compose that leverages the bookstack theme system, I'd appreciate it!

Hi @bensulli,
Looking at the way this container is built, BookStack appears to be installed in the container at /var/www/bookstack.
So you could try instead mounting to /var/www/bookstack/themes/themename and see if that works.
Just to confirm though, have not tried this myself.


Issues During Evaluation

Dear TransFuser Team,
I was trying to run evaluation with customized routes but encountered some error messages like this:
> Registering the global statistics Exception ignored in: <function LeaderboardEvaluator.__del__ at 0x7f90db186560> Traceback (most recent call last):   File "leaderboard/leaderboard/leaderboard_evaluator.py", line 124, in __del__     self._cleanup()   File "leaderboard/leaderboard/leaderboard_evaluator.py", line 157, in _cleanup     self._agent_watchdog.stop()   File "/home/zizheng/Documents/transfuser/scenario_runner/srunner/scenariomanager/watchdog.py", line 72, in stop     self._timer.cancel() AttributeError: 'NoneType' object has no attribute 'cancel'
Do you know what causes this issue and how to fix it?
Thanks in advance for your time and help!

Try changing the results file path to something else.
https://github.com/autonomousvision/transfuser/blob/9d5ebf01f7b5f6e5c02ebe0cba423119c465b388/leaderboard/scripts/run_evaluation.sh#L21


AWS SDK v2 Deprecation

Hey all,
I recently updated to the latest version of aws-azure-login (3.6.1) and noticed when running it that there's a deprecation notice now in the CLI about AWS SDK for JavaScript (v2) going into maintenance mode.
> aws-azure-login
(node:23916) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy

Are there already plans to update this package to the latest versions of the AWS SDK?

any updates on this?

Since older versions work how can we install older version until this is fixed?

Any update on this?

Thanks for the fix @aochsner. Looks like this has been merged and the tags were updated, but I don't see the new version at npmjs.com yet. Is there a build that needs to be run to publish it?


[Backport for PR #237] fix snakeyaml vulnerability issue by disabling detekt (#237)

Signed-off-by: Subhobrata Dey<EMAIL_ADDRESS>Description
fix snakeyaml vulnerability issue by disabling detekt
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #240 (e24d151) into 2.x (dea1aed) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                2.x     #240   +/-   ##
=========================================
  Coverage     83.76%   83.76%
  Complexity      434      434
=========================================
  Files            66       66
  Lines          2205     2205
  Branches        254      254
=========================================
  Hits           1847     1847
  Misses          261      261
  Partials         97       97

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Can custom elements creation via programming be abstracted out?

https://developers.google.com/web/fundamentals/web-components/customelements
Page Affected:
https://developers.google.com/web/fundamentals/web-components/customelements
What needs to be done?*
This is all very nice. Supporting custom elements programmatically is wonderful .
What I'd be more interested is a way to create custom elements that extend HTML elements and each other via some kind of text format or tool that is not as complex as using an XML schema or that simplifies creating these schemas. Basically, some way to create elements that don't necessarily have to be programmed or the programming is abstracted out.

Closing.


Egui Window ordering

Hi, is it possible to change the default ordering behaviour of egui Windows ?
I have big X Window and small Y Window. And if I drag Y over X and click on X, then X will move before Y and hide it.
I would like to set that if I click on window X it won't be moved to forefront and won't hide other windows.
Is this possible ?

Hi! Sorry, I don't have the answer, and as this question is specific to Egui itself (not the integration plugin), I'd recommend to ask this question in their repo or Discord server.

Hi, thanks for quick answer !
After looking around I think it's related to this:
https://github.com/emilk/egui/issues/1516
Also thanks for this crate, I am enjoying it. 🙂

Oops

Sorry for making mess 😓

Hi, thanks for quick answer !
After looking around I think it's related to this:
https://github.com/emilk/egui/issues/1516
Also thanks for this crate, I am enjoying it. 🙂

No worries! Thanks for linking the issue


Installation from scratch fail with file examples

Hi,
I installed (today) ggcat following the instructions provided in the README section, on a Linux machine, with installation of rust / nightly toolchain, cloning the sources and so on.
I tried to build the colored graph on the examples provided with a file color_mapping.in :
A1	sal1.fa
A2	sal2.fa
A3	sal3.fa

And running the command ggcat build -k 21 -c -d color_mapping.in gave me the following error :
Allocator initialized: mem: 2 GiB chunks: 8192 log2: 18
Using m: 10 with k: 21
Add index with color A1 => 0
Add index with color A2 => 1
Add index with color A3 => 2
Thread panicked at location: crates/io/src/sequences_stream/fasta.rs:15:14
Backtrace:    0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: <unknown>
  15: <unknown>
  16: <unknown>
  17: __libc_start_call_main
             at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  18: __libc_start_main_impl
             at ./csu/../csu/libc-start.c:392:3
  19: <unknown>

I was able to replicate the error on separate Linux machines.

Hi, the problem is that the files are not found, since the paths inside the mapping file were (wrongly) intended as relative to the current working directory. I fixed the problem, and now the relative paths inside a mapping file are adjusted using the mapping file path.

Thanks, it worked perfectly.


Allow entries with no pageref to be included.

Fixes #23

Coverage decreased (-0.8%) to 97.695% when pulling 73491faee6da687e359b3e8143ed2b923da0017e on pageref_bugfix into 946ae6218c0d9a125b50aeed654f4d4ce541276f on master.

Coverage decreased (-0.8%) to 97.695% when pulling 73491faee6da687e359b3e8143ed2b923da0017e on pageref_bugfix into 946ae6218c0d9a125b50aeed654f4d4ce541276f on master.

Coverage decreased (-0.8%) to 97.695% when pulling 73491faee6da687e359b3e8143ed2b923da0017e on pageref_bugfix into 946ae6218c0d9a125b50aeed654f4d4ce541276f on master.


Revisit prisma.$queryRaw("...") vs. prisma.$queryRaw template literal

Problem

Right now it's really easy to write a raw query that's susceptible to SQL injections
// Raw query. Susceptible to SQL injections!
prisma.$queryRaw(`SELECT \* FROM User WHERE email = ${email}`);

// Prepared query. Not susceptible to SQL injections.
prisma.$queryRaw`SELECT \* FROM User WHERE email = ${email}`;

The difference is in the ( and ). This is far too subtle and there's no warning about which is which.
Suggested solution

[x] Investigate: Can we differentiate between prisma.$queryRaw("...") and the prisma.$queryRaw template literal?

If so,

[x] Rename prisma.$queryRaw("...") to something else. Perhaps $queryDangerouslyRaw.
I think there are since there are places where you can't use a prepared statement (e.g. CREATE TABLE "${table}"). If so, I don't think we want to disallow the capability entirely.

[ ] Consider writing a codemod for npx @prisma/codemods to rename function calls (not template literals) to the new method

If not,

Consider other alternatives for clearly separating these two behaviors

Other Ideas

Maybe it's possible for our VS Code extension to detect it and return a warning.
Or maybe it's possible at runtime to detect that it's used the wrong way and throw an error?

Related: https://github.com/prisma/prisma/issues/7095

Internal Note: we consider this breaking change before the next major release.

The design of this change is the following:
Narrowed $queryRaw (breaking change)

$queryRaw now only supports a template literal.
This is always a parameterized query and therefore not susceptible to SQL injections.
The signature is prisma.$queryRaw`query`.
For example, prisma.$queryRaw`select * from users where email = ${email}`

Added $queryRawUnsafe

$queryRawUnsafe is new function that can be a string or template string.
The signature is prisma.$queryRawUnsafe(query, args...).
This allows you to pass a raw template string in. For example, prisma.$queryRawUnsafe("create table ${table}").
You could still have a parameterized query. For example, prisma.$queryRawUnsafe("select * from users where email = $1", "alice@prisma.io"). This is mostly to make the upgrade path easier

$queryRaw to $queryRawUnsafe codemod
We plan to add a codemod to https://github.com/prisma/codemods that will turn all $queryRaw functions into $queryRawUnsafe functions. It will ignore $queryRaw tagged templates, since those work as is.

Discussed it with @millsp on slack, but basically I got into the code of a pretty large customer that uses prisma  and I found a lot of sql injections due to improper use of prisma raw queries. So I support that 100%.
Basically I want the following to fail:
let query = `SELECT * FROM table`
if (id) {
  query += `WHERE id = ${id}`
}
prisma.$queryRaw(query)

But this should still work:
let query = sql`SELECT * FROM table`
if (id) {
  query = sql`${query} WHERE id = ${id}`
}
prisma.$queryRaw(query)

Closing since we've learned this will be possible.
Remaining tasks are tracked in: https://github.com/prisma/prisma/issues/8628


release 0.51

Problem
Features for 0.51 are not yet deployed.
Solution
Manual release process:

[x] Create a release/0.51 branch off of main. Ensure github test actions run. Merge Against release/0.51 branch
[x]  Bump Snapshot to 0.52.0-SNAPSHOT branch off of main. Merge against Main
[x] Tag as v0.51.0-rc2

[x] git tag
[x] Confirm new docker image version is deployed
[x] Deploy Version
[x] Finished Testing (Manual, Newman, Dapp, Acceptance)

[x] Review and update dependencies

[x] @hashgraph/hedera-local
[x] @hashgraph/sdk

[x] Previewnet Testing

[x] Deploy tagged version
[x] Manual testing
[x] Run newman tests
[x] Run Dapp Example Bootstrap and manual tests
[x] Run acceptance tests
[x] Run performance tests

[x] Testnet Testing

[x] Deploy tagged version
[x] Manual testing
[x] Run newman tests
[x] Run Dapp Example Bootstrap and manual tests
[x] Run acceptance tests
[x] Run performance tests

[x] Let Bake

[x] git tag
[x] Confirm new docker image version is deployed
[x] Write up release notes and changelist

[ ] Mainnet Testing

[ ] Deploy tagged version
[ ] Manual testing

Alternatives
No response

Deployed to mainnet


Support for Swift 4.1 runtime

The openwhisk-swift template defaults to 3.1.1. Is there plans to add support for Swift 4.1?

@chrislonge :wave: Swift 4.1 is also supported in OpenWhisk. If you use the runtime: swift:4.1 configuration parameter in the YAML, this should change the runtime verison. Let me know if this doesn't work for you?

Hey @jthomas, I was able to change the runtime to 4.1, thanks! I did have a couple questions however.

Will the default swift template app work using 4.1, or do I need to make changes? Doesn't seem to be working when I invoke with 4.1 on the template:{
  "error": "The action failed to generate or locate a binary. See logs for details."
}

Swift 4.2 comes out in about a week or two, is there plans to update template to default to at least 4.1?

Thanks for your help!

@chrislonge

https://github.com/apache/incubator-openwhisk-runtime-swift/issues/83


exact sync of video and audio

Can we guarantee to record the video stream and audio stream in the exact sync mode? As I know, hardware and digital processing can always bring some delays for several milliseconds. Are their timestamps able to bring the latency to much less than 1 millisecond? Thanks.

you should create a timecode like smpte timecode and sync both video and audio on it.


move custom role usage and creation to new pages

create new page ROOT:roles.adoc to show using a built-in and custom role
create new page theme:custom-role.adoc to show how to define a custom role
move Define a role section from theme:role.adoc to theme:custom-role.adoc
add text-transform key to role category table

@mojavelinux If you could review this for technical errors at your convenience, that would be helpful. It's not my greatest work :stuck_out_tongue_winking_eye: but it ticks the boxes enough that we'll be able to link from the What's New page "paragraphs have roles!" and "lead is now a role!" items to these pages.

I don't think I put the theme:custom-role.adoc page in the right place in the navigation list. Maybe it should go right above Page Numbers since that page is about customizing the page numbering whereas Fonts is more about what fonts are built in to the themes (though its child pages are about customizing fonts in your theme). Maybe the real problem is I'm not satisfied with where Customize the Theme and Apply a Theme are located in the list :stuck_out_tongue_closed_eyes:

I've reviewed and made a few updates. I think this improvement covers the topic well.

I don't think I put the theme:custom-role.adoc page in the right place in the navigation list.

I like it just below "Customize a Theme". You have to know how to do that to make a custom role...and it's often the first thing you will want to do next. So, to me, it's a natural fit.

I like it just below "Customize a Theme".

That seems good to me too. Thanks for the updates @mojavelinux!

Thanks for all the improvements to the docs, @graphitefriction!


performance test for vocabulary len being independent of its size

Small follow-up to recent work by @palasso.
I'm still not sure how robust this test will be to executing on other machines to be honest.
It also takes 2ish seconds on my computer, which means it'll add some overhead to the test suite overall. Not a big deal for CI, but may impact local development. At the same time, I personally don't actually run all the NLTK tests when I'm working on the lm module but only its tests.
If we at any point switch to pytest for managing tests, I could mark it (and some others) with a label so that they can be ignored when speed of test execution matters.

Thanks @iliakur; sorry for the delay!


[feature request] hook script to restart radvd when prefix is changed

Hi friends,
can you do this for radvd.service when the prefix is changed?
I'm using the way below, but I don't know if it's the best way
/usr/lib/dhcpcd/dhcpcd-hooks/70-radvd.conf
case "$reason" in
(DELEGATED6)
/bin/systemctl restart radvd.service
exit 0
;;
esac
Thank you very much

Why do you need this? radvd is capable of listening to interface and address changes.
https://github.com/radvd-project/radvd/blob/master/netlink.c#L210

doesn't seem to work properly.
The provider provides dynamic prefix and changes it a few times a day.
when the prefix changes, it doesn't send the old prefix deprecation to the devices, even with this option enabled in radvd.conf , and the devices get too many addresses.
not even reloading radvd the prefix deprecation is sent. it is necessary to stop and start.
this problem is described here:
https://github.com/radvd-project/radvd/issues/91

Ouch, that sucks!
I do not want to put this into dhcpcd because restarting a RA service will cause it to announce that it's no longer a default router and will break existing connections to the outside.
One option, albiet a touch extreme, would be to put a decent RA server in to replace radvd - rtadvd that comes with the BSD platforms works well, but that does mean changing the OS and probably a touch overkill. But I can vouch for NetBSD being up to the task.
Another alternative would be something like this maybe?
https://mdlayher.com/blog/corerad-a-new-ipv6-router-advertisement-daemon/
It doesn't have the feature you need, but it has rudimentary netlink support and they might be able to put your feature request in.

Hi Roy,
I've been using debian for 15 years. I will try to corad. Thank you very much. Hugs


Adds a more bountiful yml file for an example.

Replaces the somewhat vague and cryptic yml example with one retrieved from om --env env.yml staged-director-config --include-placeholders

@voor Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@voor Thank you for signing the Contributor License Agreement!

Thanks @voor. We might wait on this at the moment. This is an example of just for AWS configuration.
It was intentionally left vague at the moment until we figured a more consistent set of examples. For IAASes.

Not only is this PR IaaS-specific, but also potentially version-specific. I don't think om is the proper place to maintain these bountiful example configuration files. However, if there is a place you know of that might be more suitable, we are more than happy to link to that place.

It might make more sense to put these examples, or even call them out, in the terraform repositories


Replaying a WebAudioSound right when it ends would sometimes not work (have workaround)

Version

Phaser Version: 3.60

Operating system: Windows 10

Browser: Chrome

Description

Calling play() on a WebAudioSound would sometimes not work if timed exactly to the end of the previous play().
It's not very easy to reproduce, I've only been able to reproduce it when invoked by a mouse click / mobile tap / keyboard, not sure if that's important. It does take about a minute or two of clicks to reproduce.
I've already figured out what is happening inside Phaser
When a WebAudioSound finishes playing (source.onended), it marks itself as _this.hasEnded = true.
In the next frame's update(), it checks if (this.hasEnded) and if so it stops the sound and changes resets this.hasEnded = false.
The problem is that it is possible to call play() while hasEnded is already true, before update() runs, thereby restarting the sound, but then when update() happens it immediately stops the newly played sound, because hasEnded is still true.
Workaround:
Now that we know the problem, it's easy to solve by checking if sound.hasEnded is true before calling sound.play(), and if it is true, changing it back to false.
Generic workaround fix for all play() calls:
(Add this after creating the game with new Phaser.Game(config))
const originalWebAudioPlay = Phaser.Sound.WebAudioSound.prototype.play;
Phaser.Sound.WebAudioSound.prototype.play = function (
	markerName: string | Phaser.Types.Sound.SoundConfig,
	config: Phaser.Types.Sound.SoundConfig
): boolean {
	const sound = this;
	if ((sound as Phaser.Sound.WebAudioSound).hasEnded) {
		console.warn('sound hasEnded issue (handled)', sound.key);
		(sound as any).hasEnded = false; // "as any" because hasEnded is supposed to be read-only
	}

	return originalWebAudioPlay.apply(this, [markerName, config]);
};

Example Test Code

To reproduce, click the image once to play the sound, and then try to click again right before the progress number reaches 1.0 and resets. (Try to click a little bit after 0.9)
Usually takes about a minute or two of attempts to reproduce.
The bug is that if you time it just right the sound wouldn't play even though you clicked.
class Example extends Phaser.Scene
{
    preload ()
    {
        this.load.audio('explosion', [ 'assets/audio/SoundEffects/explosion.mp3' ]);
        this.load.image('wizball', 'assets/sprites/wizball.png');
    }

    create ()
    {
        const music = this.sound.add('explosion', {volume:0.4});
        this.add.image(400, 300, 'wizball').setScale(4).setInteractive().addListener('pointerdown',() => { music.play() });

        this.progressText = this.add.text(50, 50, '1');
        this.events.addListener('update', () => {
            const progress = music.seek / music.duration;
            this.progressText.setText(progress);
        });

        this.sound.pauseOnBlur = true;
    }
}

const config = {
    type: Phaser.AUTO,
    parent: 'phaser-example',
    width: 800,
    height: 600,
    pixelArt: true,
    scene: Example
};

const game = new Phaser.Game(config);

Additional Information

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.


[feat] Add a stop milling button to milling user interface

Hello @patrickcleeve2! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file fibsem/microscope.py:

Line 276:15: W291 trailing whitespace
Line 277:1: W293 blank line contains whitespace
Line 1659:76: E261 at least two spaces before inline comment
Line 1659:80: E501 line too long (98 > 79 characters)
Line 1659:99: W291 trailing whitespace
Line 1662:80: E501 line too long (87 > 79 characters)
Line 1663:30: W291 trailing whitespace
Line 1740:1: W293 blank line contains whitespace
Line 1741:1: W293 blank line contains whitespace

In the file fibsem/ui/FibsemMillingWidget.py:

Line 185:1: W293 blank line contains whitespace
Line 188:80: E501 line too long (86 > 79 characters)
Line 752:1: W293 blank line contains whitespace
Line 839:1: W293 blank line contains whitespace

In the file fibsem/ui/qtdesigner_files/FibsemMillingWidget.py:

Line 262:80: E501 line too long (80 > 79 characters)


improve cicd mainrepo

This is the same as #223 but done from main repo (checking if it can cause a failure)

Had to use force-merge in order to fix the pipelines


Villager pickup bug

Hi
I can pick up the villagers, but I cant place them. And when I pick up another item to use, the egg dissapears.
Is it something on my side or is it a bug?
I deleted every other mod only left fabric api en the mod itself, and it still doesnt work.

What version and mod version are you on? This has never happened to me or any of my friends.

Also are you playing in singleplayer or on a server, and if it is on a server do you have the mod placed in the servers mods folder too? Thats the only probably explanation in my head (that the mod was not installed on the server)

I am having this same issue, mod installed correctly on client and serve (as far as I know) but unable to place the egg to spawn a villager, if I try anything else it disappears


Objective classes

This PR introduces the new objective classes SingleTargetObjective and DesirabilityObjective as replacements for the former Objective class, together with an appropriate deprecation mechanism.

Hi @Scienfitz and @AVHopp, the objective PR is finally ready. I know it seems like a lot of changes, but please be aware that in most files only the references were updated, so it's not as much as it appears at the first glance. Also, I've created one initial commit where I copied the plain objective source code to the new subclass files, so you can also see diffs if you compare against that commit. Other than that, there was really no better way to break this PR down to smaller changes...
But let me know what you think 🙃

Hi @AVHopp @Scienfitz. All open issues have been addressed and the PR is now ready for merge from my side (except for that one open issue where I would appreciate some suggestions, @Scienfitz).
BTW: decided to update via merge instead of rebase because this would have been really error-prone...


Fixed #362 Switch to the new Docker Github action in Meshery-maesh repo

Fixes https://github.com/layer5io/meshery/issues/362

Fixed #14

@girishranganathan would you like to review given your familiarity here?


Problems writing my first PX4 program

HI!
i'm new in PX4 development and i'm trying to write my first program, I've followed all the instructions from the PX4 dev guide but when i'm trying to build my program this comes out:
[2/538] px4io-v2: nuttx-patches/00002-...ECTED-fix-shadow-wanings.patch applied
FAILED: cd /home/gesem/Firmware/build_px4fmu-v2_default/src/modules/px4iofirmware && /usr/bin/patch --verbose -d /home/gesem/Firmware/build_px4fmu-v2_default/px4io-v2/NuttX -s -p1 -N < /home/gesem/Firmware/nuttx-patches/00002-REJECTED-fix-shadow-wanings.patch && cmake -E touch /home/gesem/Firmware/build_px4fmu-v2_default/px4io-v2/NuttX/px4io-v2-nuttx_patch_00002-REJECTED-fix-shadow-wanings.patch.stamp
The text leading up to this was:
|diff --git NuttX/nuttx/include/signal.h NuttX/nuttx/include/signal.h
|index bd8ee9f..7ee242a 100644
|--- NuttX/nuttx/include/signal.h

+++ NuttX/nuttx/include/signal.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

The text leading up to this was:

|diff --git NuttX/nuttx/include/stdlib.h NuttX/nuttx/include/stdlib.h
|index aa259c9..f2b0640 100644
|--- NuttX/nuttx/include/stdlib.h

+++ NuttX/nuttx/include/stdlib.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

The text leading up to this was:

|diff --git NuttX/nuttx/include/sys/stat.h NuttX/nuttx/include/sys/stat.h
|index 3f2658b..41c8604 100644
|--- NuttX/nuttx/include/sys/stat.h

+++ NuttX/nuttx/include/sys/stat.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

The text leading up to this was:

|diff --git NuttX/nuttx/include/sys/statfs.h NuttX/nuttx/include/sys/statfs.h
|index 1d1786a..12f802b 100644
|--- NuttX/nuttx/include/sys/statfs.h

+++ NuttX/nuttx/include/sys/statfs.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

[2/538] px4fmu-v2: nuttx-patches/00002...ECTED-fix-shadow-wanings.patch applied

FAILED: cd /home/gesem/Firmware/build_px4fmu-v2_default && /usr/bin/patch --verbose -d /home/gesem/Firmware/build_px4fmu-v2_default/px4fmu-v2/NuttX -s -p1 -N < /home/gesem/Firmware/nuttx-patches/00002-REJECTED-fix-shadow-wanings.patch && cmake -E touch /home/gesem/Firmware/build_px4fmu-v2_default/px4fmu-v2/NuttX/px4fmu-v2-nuttx_patch_00002-REJECTED-fix-shadow-wanings.patch.stamp

The text leading up to this was:

|diff --git NuttX/nuttx/include/signal.h NuttX/nuttx/include/signal.h
|index bd8ee9f..7ee242a 100644
|--- NuttX/nuttx/include/signal.h

+++ NuttX/nuttx/include/signal.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

The text leading up to this was:

|diff --git NuttX/nuttx/include/stdlib.h NuttX/nuttx/include/stdlib.h
|index aa259c9..f2b0640 100644
|--- NuttX/nuttx/include/stdlib.h

+++ NuttX/nuttx/include/stdlib.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

The text leading up to this was:

|diff --git NuttX/nuttx/include/sys/stat.h NuttX/nuttx/include/sys/stat.h
|index 3f2658b..41c8604 100644
|--- NuttX/nuttx/include/sys/stat.h

+++ NuttX/nuttx/include/sys/stat.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

The text leading up to this was:

|diff --git NuttX/nuttx/include/sys/statfs.h NuttX/nuttx/include/sys/statfs.h
|index 1d1786a..12f802b 100644
|--- NuttX/nuttx/include/sys/statfs.h

+++ NuttX/nuttx/include/sys/statfs.h

File to patch:

Skip this patch? [y]

1 out of 1 hunk ignored

ninja: build stopped: subcommand failed.

Makefile:161: recipe for target 'px4fmu-v2_default' failed

make: *** [px4fmu-v2_default] Error 1

Does anyone know what is wrong and how to solve it ?

Are you synced up with the latest version of master?
Try make distclean then make.

Thank you! it worked very well i think ! but can you explain me what is master ? and why do i needed to write make distclean and then make ? i want to understand a bit more of all of this haha
THANKS!

I don't know how it got like that, but your local checkout was in a bad state. That target forced all the submodules back into a good state and through away all the build artifacts restoring you to the state of a clean checkout. Master is the latest development version of PX4 that you get by default when you. If was a secondary suggestion in case you were on an old version.


Save initial options with preserveSelected

Without this, after selectbox loaded and user clicked on some of option, another options is removed.

@maxvyaznikov since verb is breaking and they haven't released an update, and travis is breaking due to this; I'm going to run the tests manually on my machine.  Once it passes everything and I confirm that it fixes the issues I'll probably merge it unless @markcarver has any issues with it.

@maxvyaznikov looks like the build is passing now (can do a full build not just testing).  Will test the fix later today and hopefully get this merged in soon.

Yeah, thanks.

@maxvyaznikov out of curiousity do you have to trigger a change event on the select for it to notice the selected option?  I had to do that in my testing.  This definitely fixes the issue of it storing the selected option correctly though and thanks for that.
@markcarver merging this in because it definitely fixes half the problem with initial options not being selected properly.

@truckingsim, in my case, options for select is loaded with page in html, and only after that JS load to start fun with options.


Move 'Set status' dropdown and 'Save' button together

Is your feature request related to a problem? Please describe.
The 'Set status' and 'Save' buttons are on opposite sides of the Header. This placement can be confusing for users becuase they expect all actions to be grouped in the same area.
Describe the solution you'd like
The 'Set status' dropdown should be moved to the left side of the Header so it is next to the 'Delete unpublished changes' button.
Describe alternatives you've considered
None
Additional context

Figure: Similar UI elements on opposite sides of the header

Figure: Mockup - Proposed button positions
See this YouTube video that goes through this as well as some related changes - https://youtu.be/8vhHKf0nCS4

Thanks @bradystroud,
I'm also open to putting the delete button to the left of the publish button as those seem to be more related.
Also what do you think about ditching the status dropdown and use 3 buttons instead?

Hi @erezrokah,
I think having 3 buttons would be a bit messy.
I think a dropdown would be better, as long as we implement #5557


AMO Support E-mail expired

Your Support E-mail<EMAIL_ADDRESS>on the AMO page has expired:

The error that the other server returned was:
554 5.7.1<EMAIL_ADDRESS>Recipient address rejected: Account expired

Can you please update or remove the Support E-mail?

There is openmailbox.org if you search for a new web mail provider.


small typo in cron example

Hi all, just picked up a small typo in the Cron Expressions section:
A range (- operator) "5-7 * * * * *" at hh:mm:05,hh:mm:06, and hh:mm:05
I think the end of that sentence should be hh:mm:07
Regards,
Mark.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b5527c3b-a5da-c73d-66c2-20d96003ff7c
Version Independent ID: 42154c7b-b0e0-26db-79bc-3f1673c48e40
Content: Timer trigger for Azure Functions
Content Source: articles/azure-functions/functions-bindings-timer.md
Service: functions
GitHub Login: @tdykstra
Microsoft Alias: tdykstra

@MarkAtAgilliance We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Fix domain events

Follow up to #1904

Coverage remained the same at 80.933% when pulling 85895ea7e239ed5e1828b26e9178fd82a24e2e21 on eatonphil:fix-domain-events into bec361e22d586fc396c66df3fe4699eb0d9ff0a2 on linode:master.


build: bump @angular/cli and @angular/core to 17.3.0

PR Checklist

[ ] Unit Tests have been added for new changes
[ ] API tests have been updated if applicable
[ ] All commented code has been removed
[ ] If you've added a dependency, you've ensured license is compatible with Apache 2.0 and clearly outlined the added dependency.

What are you changing?

Anything the reviewer should know when reviewing this PR?
If the there are associated PRs in other repositories, please link them here (i.e. open-amt-cloud-toolkit/repo#365 )

:tada: This PR is included in version 3.5.4 :tada:
The release is available on:

GitHub release
v3.5.4

Your semantic-release bot :package::rocket:


prim.c: add /dev/urandom fallback for old OS X

arc4random_buf is not always available. Also changed MAC_OS_X_VERSION_MAX_ALLOWED check to MAC_OS_X_VERSION_MIN_REQUIRED, because no runtime checking of either the OS version or the presence of weak-linked symbols is being done, so runtime behaviour would be incorrect if you built against the latest SDK and deployed to an older OS version.

@microsoft-github-policy-service agree

Thanks! I pushed a fix to dev (merge with dev-slice soon). Can you test it?

@daanx I haven't been able to try builds yet, but the changes mostly look good, with the exception of the OS version checks. I don't know how familiar you are with Apple's availability checking system, so I'll give some background.
MAC_OS_X_VERSION_MAX_ALLOWED corresponds to the SDK you are building against. That of course determines which interfaces are available at compile time. MAC_OS_X_VERSION_MIN_REQUIRED is the oldest OS version that the binary being built will be able to run on. That determines which interfaces can be known to be available at runtime without further checking, and can be changed by setting MACOSX_DEPLOYMENT_TARGET in the environment or by using the -mmacosx-version-min option with clang.
For interfaces that were introduced in an OS version newer than MAC_OS_X_VERSION_MIN_REQUIRED, the binary will be weak-linked with the corresponding symbols. That means that the dynamic linker will bind them at runtime if they exist, but ignore them if they don't. Using an unbound symbol will cause a crash. So, you have to check for potentially weak-linked symbols at runtime before using them. With recent clang versions, __builtin_available is the recommended way of doing this, while with older compilers you just check if the symbol is NULL.
What all this means for the code at hand is that if you don't do any runtime availability checking, it is safe to use interfaces iff both MAC_OS_X_VERSION_MAX_ALLOWED and MAC_OS_X_VERSION_MIN_REQUIRED are >= the version they were introduced in. For arc4random_buf that's 1070 (10.7) and for CCRandomGenerateBytes it's 101000 (10.10).

Thanks for the explanation!  I updated the checks -- can you see if it looks good to you?

Just for the sake of completeness,

and can be changed by setting MACOSX_DEPLOYMENT_TARGET in the environment or by using the -mmacosx-version-min option with clang.

-mmacosx-version-min works with GCC too, not only Clang.

With recent clang versions, __builtin_available is the recommended way of doing this, while with older compilers you just check if the symbol is NULL.

AFAIK, __builtin_available is a clangism, so cannot be used with modern compilers as well, if those are not Clang.

Thanks for the explanation! I updated the checks -- can you see if it looks good to you?

There is indeed the problem that MAC_OS_X_VERSION_10_7 is undefined on systems older than 10.7. Otherwise looks correct, assuming the intent is to only use CCRandomGenerateBytes on 10.15 and later due to the performance issue on older OS versions (not sure if that is considered worse than the potential error handling issue with arc4random_buf.)

Thanks for the quick follow up!  I yet pushed another fix -- hope it is correct now. I think we just stick with CCGetRandomBytes as it is the "best" source.


Usable on bare metal? (amd64)

Hiya!
Do you think this will run on bare metal? If yes, how can I install it?
(The metal in question is a ThinkPad E495)

@subnut , thanks for taking interest in glaucus!
glaucus most certainly will be running on bare metal, once development further advances.


HAVING clauses may not contain window functions

Rejects having clauses if they contain windowed expressions. (most likely it will be fixed via CALCITE-6473)
Also added a check to produce a more descriptive error if an
OVER expression reaches the filter translation layer.

thank you for updating the PR @asdf2014 ; I must have forgot to format those lines

@sreemanamala this came up in the review of the release notes for Druid 31. Can you please write a user-facing release note for this change?


Clarification regarding app hosting behavior & Support for local hosting

Hello, I recently tested textual-web within a corporate network and encountered this warning/error:
           INFO     connecting to Ganglion
           WARNING  Unable to connect to Ganglion server. Will reattempt connection soon.

This led me to realize that apps run through textual-web are not hosted locally but are instead served from a third-party server, generating a unique URL. I find this worth mentioning explicitly in the README for clarity and to set proper user expectations, especially because this behavior deviates from the original textual where the 'apps' were obviously run locally.
Could you consider updating the README to specify this information?
Additionally, are there any plans to support local hosting of Textual apps using textual-web in the future?

this behavior deviates from the original textual where the 'apps' were obviously run locally

Just to be clear too: your applications are still run "locally"; as in they're still run on the machine that you're running textual-web on; but the communication with the client browser goes via our servers.

Is there a way to test fully locally as of today? Or the SaaS part of it is an integral part of the solution and can’t be left aside?

It depends on a server component that we host, which you can't yet run locally.

It depends on a server component that we host, which you can't yet run locally.

Got it. Thanks.

[A web search led me to this thread, so I'll ask my follow-up question here:]
Is one of the goals of Textual Web to be able to package a CLI app so that it can run in a browser, in a way that I can upload some files to my server to run at mydomain.com/myapp/?

It depends on a server component that we host, which you can't yet run locally.

Is there any update on local hosting here? This seems like a handy tool for local development but I don't want to expose anything to the web.

It depends on a server component that we host, which you can't yet run locally.

Is there any update on local hosting here? This seems like a handy tool for local development but I don't want to expose anything to the web.

Note that mixing web technologies and a server's terminal in applications can have a lot of security risks, especially when they use third-party applications. If you are creating internal tooling that interacts with your local server's terminal and that same tooling is now exposed to web technologies like HTML/Javascript, it introduces additional attack vectors directed at your server's most sensitive access points.
This is such an important issue that xterm.js (used to make VSCode's terminal interface) has a page dedicated to the security risks: https://xtermjs.org/docs/guides/security/
Something to consider is that many who build terminal apps are likely building them for internal use, which makes discussing security all the more important.


Fix typo in readme

Description

This just fixes a tiny typo in the name of the imported CSS file in the readme (...Styles.css -> ...Style.css)
Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist:

My code follows the code style of this project.
If my change requires a change to the documentation I have updated the documentation accordingly.

Pull Request Test Coverage Report for Build 999

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 93.548%

Totals

Change from base Build 992:
0.0%

Covered Lines:
43

Relevant Lines:
44

💛  - Coveralls


Is the new gem thread safe?

If we are using a sidekiq with multiple threads and do
Bigcommerce.configure do |config|
  config.store_hash = ENV['BC_STORE_HASH']
  config.client_id = ENV['BC_CLIENT_ID']
  config.access_token = ENV['BC_ACCESS_TOKEN']
end

# List orders
@orders = Bigcommerce::Order.all

in each worker, and they are executing at the same time, will Bigcommerce be isolated to the correct client in each worker?

The current version is not thread safe. I would love to support this, thanks for the suggestion!

Hey @homanchou thanks for reporting this.
Being thread safety is definitely on our TODO, but our focus right now is more on adding features/cleaning some stuffs.
We are really open to contributions though so if this is a priority for you, feel free to open PR!
thanks man

Any update on this? Is @hasclass fork working?

I'm also interested in thread safety for this.

@gregory - Have there been any updates on thread safety?

I am going to be taking a look over the two proposed implementations and taking a fresh look at the problem -- hope to have some proposals for discussion in the next few days.

I have embarked on adding thread safety based on the implementation presented by @gregory. AFAIK this is NOT a breaking change and adds a new method to query resources by allow one to pass a new faraday connection into the resource.
Ex.
# Thread safe
conn = Bigcommerce::Connection.build(
  Bigcommerce::Config.new(
    store_hash: ENV['BC_STORE_HASH'],
    client_id: ENV['BC_CLIENT_ID'],
    access_token: ENV['BC_ACCESS_TOKEN']
  )
)

Bigcommerce::System.time(connenction: conn)

# Not thread safe
Bigcommerce.configure do |config|
  config.store_hash = ENV['BC_STORE_HASH']
  config.client_id = ENV['BC_CLIENT_ID']
  config.access_token = ENV['BC_ACCESS_TOKEN']
end

Bigcommerce::System.time

I have a WIP PR #123, if people could help test and give some review, that would be great. Cheers!
cc @gregory @homanchou @hasclass

Here is the hacked together script I used to test the thread safety of the code between two stores. The threading implementation is using sucker_punch which in turn uses ruby-concurrent.
require 'bigcommerce'
require 'sucker_punch'

SuckerPunch.shutdown_timeout = 120

MAPPING = {
  'store_hash'   => 'email',
  'store_hash2'  => 'email2'
}.freeze

class UnsafeStoreInfoJob
  include SuckerPunch::Job

  def perform(store_hash, access_token)
    Bigcommerce.configure do |config|
      config.store_hash = store_hash
      config.access_token = access_token
      config.client_id = ENV['BC_CLIENT_ID']
    end

    sleep 0.1
    puts "UNSAFE #{MAPPING[store_hash] == Bigcommerce::StoreInfo.info.admin_email}"
  end
end

class SafeStoreInfoJob
  include SuckerPunch::Job

  def perform(store_hash, access_token)
    conn = Bigcommerce::Connection.build(
      Bigcommerce::Config.new(
        store_hash: store_hash,
        client_id: ENV['BC_CLIENT_ID'],
        access_token: access_token
      )
    )

    sleep 0.1
    puts "SAFE   #{MAPPING[store_hash] == Bigcommerce::StoreInfo.info(connection: conn).admin_email}"
  end
end

100.times do
  UnsafeStoreInfoJob.perform_async('store_hash', 'REDACTED')
  UnsafeStoreInfoJob.perform_async('store_hash2', 'REDACTED')

  SafeStoreInfoJob.perform_async('store_hash', 'REDACTED')
  SafeStoreInfoJob.perform_async('store_hash2', 'REDACTED')

  sleep 1
end

This (when using valid values) gives me an output like:
SAFE   true
UNSAFE true
SAFE   true
UNSAFE false
UNSAFE false
SAFE   true
SAFE   true
UNSAFE true
UNSAFE true
SAFE   true
UNSAFE false
SAFE   true
UNSAFE false
SAFE   true
UNSAFE true
SAFE   true
UNSAFE false
SAFE   true
UNSAFE true
SAFE   true
UNSAFE true
SAFE   true
...

I have run this quite a few times and have yet to see SAFE come back false -- indicating that the faraday connection is being reused between threads.
With this, I have confidence in saying if you use the connection param as proposed in #123 and by @gregory, you can ensure thread safety.


Extended Task on input.upload

Hello,
I have read the well-written document regarding extended_task from this page:
entended_task
There is a ui.bind_task_button which binds the button to the extended task so other functions run as normal when the event is triggered.
However, I would like to achieve the same result after uploading a file so that when the system is processing the file, other functions like the timer continue to operates without having to wait for the uploaded file to be completely processed. Here is the sample code:
import asyncio
import datetime
from shiny import App, reactive, render, ui

app_ui = ui.page_fluid(
    ui.p("The time is ", ui.output_text("current_time", inline=True)),
    ui.hr(),
    ui.input_file("file", "Choose a file to upload"),
    ui.output_text("sum"),
)

def server(input, output, session):
    @render.text
    def current_time():
        reactive.invalidate_later(1)
        return datetime.datetime.now().strftime("%H:%M:%S %p")

    # @ui.bind_task_button(button_id="btn")
    @reactive.event(input.upload)
    @reactive.extended_task
    async def sum_values():
        print("GOGO")
        await asyncio.sleep(5)
        print("GOGOGOGO")
        file_info_list = input.file
        print(file_info_list)
        for file in file_info_list:
            print(str(file))
        return 7 + 5

    @render.text
    async def sum():
        return str(await sum_values())

app = App(app_ui, server)

Currently, the code does run the print function, so “GOGO” and “GOGOGOGO” are printed. However, the for loop doesn’t execute, and the result number 12 isn’t presented either. I might be using @reactive.event(input.upload) and @reactive.extended_task incorrectly. Any advice is appreciated.

There were a lot of errors in your implementation. Here's a functional example, however I am uncertain whether it will achieve what you intend if you upload a very large file as this only deals with the reading of the file, but while the upload is in progress, it may still block. I can't really test that locally though.
import asyncio
import datetime
from shiny import App, reactive, render, ui, req

app_ui = ui.page_fluid(
    ui.p("The time is ", ui.output_text("current_time", inline=True)),
    ui.hr(),
    ui.input_file("file", "Choose a file to upload"),
    ui.output_text("sum"),
)

def server(input, output, session):
    @render.text
    def current_time():
        reactive.invalidate_later(1)
        return datetime.datetime.now().strftime("%H:%M:%S %p")

    @reactive.extended_task
    async def sum_values(file):
        print("GOGO")
        await asyncio.sleep(5)
        print("GOGOGOGO")
        file_info_list = file
        print(file_info_list)
        for file in file_info_list:
            print(str(file))
        return 7 + 5

    @reactive.effect
    @reactive.event(input.file)
    def btn_click():
        sum_values(input.file())

    @render.text
    async def sum():
        return str(sum_values.result())

app = App(app_ui, server)


Move relevant constants to ProjectSettings

Hi, I really like this plugin. I want to use it in my game, but I also want to heavily customize it. Currently it seems in order to do that, I have to edit the scripts themselves. This is not ideal, since that means potential merge conflicts every time I update it.
I propose that some constants are moved to the ProjectSettings, such as the command file directory and the ui handler scene.
I will very likely end up making my own small modifications to the scripts actual behaviour-wise, but I still want to minimize them.

That would make sense as GDShell already uses input maps to toggle the UI and it could be still modifiable trough code.


Support arrow function

Default webpack config does not support arrow function of ES6. Do you intend to fix it or open for PR?

I'm open for PR :)

You meant arrow function is not working on crau.config.js right?

Also which Node version you're using?

Ah I see, that's not arrow function issue, but class properties proposal
https://babeljs.io/docs/en/babel-plugin-transform-class-properties
Since it's still on proposal stage, I'm not including it, you can add it to your server/.babelrc

Ok I got it. Thank you very much!

I tried with your suggestion and got an error.

Could you please to help me? Thanks! @antonybudianto

Since you're overriding .babelrc, you need to include the default:
{
  "presets": ["env", "react"],
  "plugins": [
    "dynamic-import-node",
    "transform-object-rest-spread",
    "transform-class-properties"
  ]
}

Thanks for explanation! What a enthusiastic guru!


Hosting Request: ECU-TEST Execution plugin

Repository URL
https://github.com/tracetronic/ecu-test-execution-plugin
New Repository Name
https://github.com/jenkinsci/ecu-test-execution-plugin
Description
This plugin enables a platform-independent test execution. Due to the reduced complexity and communication via REST API, it provides an easy and reliable entry into continuous testing. It integrates ECU-TEST with Jenkins and allows report generation and its upload of automated test execution.
GitHub users to have commit permission
TT-S3-D1
TT-CX-User
MartinGroscheTT
cwellmtt
thardtke-tt
cpoenisch
Jenkins project users to have release permission
tracetronic
Issue tracker
GitHub issues

Please pick one build system, don't use maven and gradle

/hosting re-check

Hi @timja
Can you please explain for me the last required issue? I can't understand why the build.gradle shouldn't be parseable. At https://github.com/tracetronic/ecu-test-execution-plugin/runs/6782691306?check_suite_focus=true is a successful gradle run for that.
Thanks a lot.
Best
Sebastian

Hey not sure from a quick look but this should be changed to log the exception:
https://github.com/jenkins-infra/repository-permissions-updater/blob/ab78d28ae2ebb66d989750619ae5b7d49c0bc9b3/src/main/java/io/jenkins/infra/repository_permissions_updater/hosting/GradleVerifier.java#L142
I don't have time right now but should be a small change

Hi @timja,
sorry for interrupting you again.
Can you or maybe someone else of the jenkins-infra team help at this point? I think the required change conflicts with the hint.
Thanks a lot!
Best Regards
Sebastian

CERT will have a deeper look. Tracked internally as JENSEC-1864.

@TT-S3-D1,
We've had a deeper look and haven't found any vulnerabilities in the code.
Dependency scan has identified issues with<EMAIL_ADDRESS>which is supplied by<EMAIL_ADDRESS>and<EMAIL_ADDRESS>and<EMAIL_ADDRESS>Please confirm that you're no affected by those, or otherwise consider updating to a fixed version.
If you have any further questions please feel free to ask.

@yaroslavafenkin
thanks for the hint - I've updated all dependency versions

@yaroslavafenkin did you want to re-check?

@yaroslavafenkin did you want to re-check?

Re-checked, looks alright to me.

/hosting re-check

/hosting host


syncyomi: 1.1.1 -> 1.1.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/SyncYomi/SyncYomi/releases.
meta.description for syncyomi is: Open-source project to synchronize Tachiyomi manga reading progress and library across multiple devices
meta.homepage for syncyomi is: https://github.com/SyncYomi/SyncYomi
meta.changelog for syncyomi is: https://github.com/SyncYomi/SyncYomi/releases/tag/v1.1.2
Updates performed

Golang update

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.1.2 with grep in /nix/store/70hpa1gxfxdcaqn43sm0ynh54p5cx12d-syncyomi-1.1.2
found 1.1.2 in filename of file in /nix/store/70hpa1gxfxdcaqn43sm0ynh54p5cx12d-syncyomi-1.1.2

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

syncyomi

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/70hpa1gxfxdcaqn43sm0ynh54p5cx12d-syncyomi-1.1.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A syncyomi https://github.com/r-ryantm/nixpkgs/archive/77f5f203cf0566e7ff5a273a7a16bb236bad51ee.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/77f5f203cf0566e7ff5a273a7a16bb236bad51ee#syncyomi

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/70hpa1gxfxdcaqn43sm0ynh54p5cx12d-syncyomi-1.1.2
ls -la /nix/store/70hpa1gxfxdcaqn43sm0ynh54p5cx12d-syncyomi-1.1.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review

x86_64-linux

:white_check_mark: 1 package built:

syncyomi

Maintainer pings
cc @eriedaberrie for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

@NixOS/nixpkgs-merge-bot merge


'create-react-app' is not recognized as an internal or external command, operable program or batch file.

Is this a bug report?
I am using windows 10, I tried to create a new react app with "Create-react-app" but it gives me this error.
My node version is: 6.11.0
NPM: 3.10.10
path of create-react-app
C:\Users\Admin\AppData\Roaming\npm-cache\create-react-app
Please suggest

Can you also reproduce the problem with npm 4.x?

(Write your answer here.)
Which terms did you search for in User Guide?

(Write your answer here if relevant.)
Environment

node -v:
npm -v:
yarn --version (if you use Yarn):
npm ls react-scripts (if you haven’t ejected):

Then, specify:

Operating system:
Browser and version (if relevant):

Steps to Reproduce

(Write your steps here:)

Expected Behavior

(Write what you thought would happen.)
Actual Behavior

(Write what happened. Please add screenshots!)
Reproducible Demo

(Paste the link to an example project and exact instructions to reproduce the issue.)

Sounds like an issue with the PATH environment variable. Check echo %path% to see if the folder your create-react-app commands are in there and update accordingly.

Kindly check this https://github.com/facebookincubator/create-react-app

It is working now, The issue was when I installed CLI, it was created a separate folder in C drive and my node path was in the different folder.
CLI install in
C:\usr\local
node path was
C:\Users\Admin\AppData\Roaming\npm
Solution applied
Copied 'create-react-app' folder and cmd and paste it into Roaming/npm folder


Get rid of pull_variables.yml?

I noticed that some of our settings weren't applied, in particular on compute nodes one of the ntp servers is set to <IP_ADDRESS> which is incorrect for us. Turns out that this comes from pull_variables.yml which overrides the settings from group_vars.
It seems to me that except for setting running_as_ansible_pull and slurm_munge_key_from_nfs to True, pull_variables.yml doesn't do anything useful. E.g. where to pull from is already hardcoded in ansible-pull-script which syncs the group_vars where everything else is defined.

A facts issue. Pull doesn't gather facts and can't figure out the IP of the admin node from hostvars.
We could update default variables so that ntp_config_server looks like:

ntp_config_server: [ "{{ kickstart_server_ip }}", "{{ central_log_host|strip('@') }}" ]

Testing that out.

We don't have the admin node's IP in a variable in the examples/group_vars.

TASK [ansible-role-chrony : update chrony.conf from template] ******************
Friday 27 January 2017  10:51:06 +0200 (0:00:04.993)       0:00:25.845 ********
fatal: [io1]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: {{ ntp_config_server }}: [u\"{
{ hostvars[groups['install'][0]]['ansible_hostname'] }}\", u\"{{ hostvars[groups['admin'][0]]['ansible_hostname'] }}\"]: 'di
ct object' has no attribute 'ansible_hostname'"}

Similar issue with slurm_service_node variable as that's also from facts by default.
https://github.com/CSC-IT-Center-for-Science/ansible-role-fgci-install/pull/32

Okay after changing fgci-install role a few times the above ansible-pull doesn't fail anymore.
The central_log_host|replace is a bit hacky but we don't have admin group's hostvars in the install.yml playbook to use,  nor do we have the admin node's IP manually entered into the group_vars.
Added some documentation and reasoning..
How does it look?

Less clean than I had hoped for, but I guess it's unavoidable?

Another way would be to have a central store of the facts that the nodes can query when they run ansible-pull.
I haven't looked into this or tested it but seems cleaner and could perhaps also provide some performance boost: http://docs.ansible.com/ansible/playbooks_variables.html#fact-caching


input and target size mismatch when running test.lua

Hi Dave!
Sorry to interrupt you. I wanted to run this impressive pipeline but was stuck in an error. I haven't tried to train a new network yet and all data were downloaded from your dropbox directly. But when I run basset_test.lua, it occured that:
[myb@localhost ~]$ cd ./Basset/src
[myb@localhost src]$ basset_test.lua ../data/models/pretrained_model.th ../data/encode_roadmap.h5 ../data
/home/myb/torch/install/bin/lua: /home/myb/torch/install/share/lua/5.2/nn/BCECriterion.lua:24: input and target size mismatch
stack traceback:
[C]: in function 'assert'
/home/myb/torch/install/share/lua/5.2/nn/BCECriterion.lua:24: in function </home/myb/torch/install/share/lua/5.2/nn/BCECriterion.lua:22>
(...tail calls...)
./convnet.lua:1156: in function 'test'
/home/myb/Basset/src/basset_test.lua:75: in main chunk
[C]: in function 'dofile'
.../myb/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: in ?
Would you mind telling me what may lead to it and what I should do? Thanks a lot

It looks like the sequence lengths are mismatched. What steps did you use to generate the encode_roadmap.h5 file? I just ran the steps here https://github.com/davek44/Basset/blob/master/install_data.py, and it worked for me.

Well the networks in China are isolated partly so the server can't reach dropbox, run install_data.py and download the data. So I  have to download the data manually and may be the error occured. Are there some ways to get the data avoiding the dropbox?

Can you access Google Cloud storage?
On Fri, May 17, 2019 at 7:32 PM ThuMYB<EMAIL_ADDRESS>wrote:

Well the networks in China are isolated partly so the server can't reach
dropbox, run install_data.py and download the data. So I have to download
the data manually and may be the error occured. Are there some ways to get
the data avoiding the dropbox?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/davek44/Basset/issues/46?email_source=notifications&email_token=AABKFEGICCXAGAHSYLZAI73PV5TEJA5CNFSM4HNPBCA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVWGAWY#issuecomment-493641819,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABKFEF2N56NYB7MJDCHTETPV5TEJANCNFSM4HNPBCAQ
.

--
David Roy Kelley, PhD
Principal Investigator
Calico Life Sciences


Using input_output_response(...), Simple System Gives Erroneous Output

This system should not have the bump in the output you see about halfway through the simulation.
import matplotlib.pyplot as plt
import numpy as np
import control as ctrl

R = 22e3
C = 470e-6

circuit = ctrl.ss(-1/(R*C), 1/(R*C), 1, 0)
t, resp = ctrl.step_response(circuit)
plt.plot(t, resp)
plt.title('Step Response of the RC Circuit')
plt.grid()
print("The eigenvalue is: ",-1/(R*C))
print("numpy gives the eigenvalue, and eigen vector respectively as: ")
np.linalg.eig(circuit.A)
t = np.arange(0, 120, 0.01)
u = 3.3*148/256-3.3*40/256*np.heaviside(t-60,1)
plt.plot(t,u)
x0 = 3.3*128/256
t, vc = ctrl.input_output_response(circuit, t, u, x0)
plt.plot(t,vc)
plt.grid()
plt.title("Response of RC Circuit")
plt.legend(["u","$v_c$"])
plt.xlabel("time (seconds)")
plt.ylabel("Voltage (Volts)")
plt.show()

Confirmed this behavior.  Using forced_response instead of input_output_response works correctly:

So something odd going on in input_output_response...

It appears to me that the input_output_response is using Runga-Kutta 45, and that the tolerance is set to 0.001 or so.  I get the same results with that if the rtol is set to low.   Here is my python script with the RK45 method.  Here is the plot.

`# %% Imports
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
%% Define independent function and derivative function
def x(t):
return 1-np.heaviside(t-60, 1)
def f(t, y, c):
R = 22e3
C = 470e-6
dydt = x(t)/R/C - y/R/C
return dydt
%% Define time spans, initial values, and constants
tspan = np.linspace(0, 120, 1000)
yinit = [0]
c = []
%% Solve differential equation
sol = solve_ivp(lambda t, y: f(t, y, c),
[tspan[0], tspan[-1]], yinit, t_eval=tspan,
rtol = 0.001)
%% Plot independent and dependent variable
Note that sol.y[0] is needed to extract a 1-D array
plt.figure(1)
plt.clf()
fig, ax = plt.subplots(num=1)
ax.plot(sol.t, x(sol.t), 'k-', label='Input')
ax.plot(sol.t, sol.y[0], 'k--', label='Output')
ax.legend(loc='best')
plt.show()`

Correct: the input_output_response functions uses scipy.integrate.solve_ivp using its default arguments, which use 'RK45' as the solver and default values for rtol and atol (Scipy default values are 1e-3 for rtol and 1e-6 for atol).
The problem you are having is that 'RK45' doesn't detect/handle the discontinuity at t = 60.  You can fix this by using 'LSODA' as the method to solve the ODE:
t, vc = ctrl.input_output_response(circuit, t, u, x0, solve_ivp_method='LSODA')

which gives the expected response:

It's not clear that the right "fix" is for python-control, but at the least we (I) should add something to the documentation cautioning about the need to be careful with the solver if you have discontinuous inputs.

I was asking myself another day -- why not use LSODA as the default?

It looks like LSODA uses the ratio of the maximum eigenvalue to the minimum eigenvalue to determine the stiffness, and in so this differential equation would not default to a stiff solver with SODA, at least the way I understand it.  This problem has difficulty because the forcing function has a sharp discontinuity.  Picking a different solver might still solve this though, because the RK45 algorithm is pretty poor for stiff ODE's.


API Consumption for Top News Headlines

Implement code to generate data for Top News Headlines from the News API.

Hi! I wish to work on this issue.


Standardize Formatting of Documentation

This PR attempts to make it easier to keep the documentation in the repo a tiny bit cleaner for anyone who uses the EditorConfig extension

Add EditorConfig
Lint YAML files
Lint Markdown files

[x] I have signed the Contributor License Agreement.
[ ] This pull request is related to an issue.

Microsoft Reviewers: Open in CodeFlow

Fingers crossed!


[FR]: use sha256 instead of sha386 for integrity?

What is the current behavior?
now jq/yq binary use sha386 integrity
But mainstream bazel remote cache server can only cache the assets with sha256/blake3 integrity
Describe the feature
use sha256 integrity

@alexeagle @gregmagolan
Will this be accepted? If I take a pr.

I don't think we care which hash algorithm is used, we just prefer to mirror whatever data is already available rather than download all the binaries and compute checksums ourselves.

Is there an issue for whatever system is failing to understand other hash algorithms? If we change this, it's a workaround for someone else's bug, so I'd at least like to know when they've fixed it.

see
https://github.com/buildbuddy-io/buildbuddy/pull/6911#pullrequestreview-2142054120
https://github.com/buchgr/bazel-remote/blob/fea980bd8f528ccfc38919fb709e9e47d27a1622/server/grpc_asset.go#L79

Oh I see, this is just for the remote asset API. I don't think it helps much to change integrity format on a couple binaries here - the problem is much more widespread (i.e. https://github.com/aspect-build/toolchains_protoc/blob/main/protoc/private/versions.bzl#L54 )

yeah this is for Remote Asset API.
Traditionally the API has always been SHA256 based. Meaning that if you use it with some exotic hasher like SHA512 (rules_js) or SHA386 (jq, protoc etc…), then you get no cache hit from the Remote Cache and your download would always reach upstream URL. That’s functional, but not ideal for folks who rely on Remote Asset API for some speed up.
We recently implemented the changes in the Remote Asset API + Bazel that enables to send over a separate “storage” hasher in addition to the previous “content verify” hasher. This allows us to receive a download request with SHA512 verification, but BLAKE3 storage and download. There would still be no cache hit on this flow, but at least now, you can change the hasher Bazel would use to download/store the blob.
The remaining question is: how do we facilitate for cache hits for cases where the integrity in rctx.download is either SHA256 or BLAKE3 (the 2 popular hash method Bazel supports)?


Type check in IIFE (for JavaScript)

VSCode Version: 1.0.34 (user setup)
OS Version: Windows (v1903)

Steps to Reproduce:

Use the passed code:

// @ts-check
// @ts-ignore
return (function() {

	var a = 0;
	a = "1";

})();

No error is reported for the assignment in line 6 (the script is executed in our customized javascript runtime where global returns are allowed. Thats the reason for the // @ts-ignore comment).

Remove the return statement. You will get an error in line 6 because a is a number and the expression results in a string. Thats the expected behaviour:

When you add the return statement again, the error disappears:

I first thought that it is caused by the @ts-ignore comment. But removing this just results in reporting TS[1108] but not the error shown in (2):

Does this issue occur when all extensions are disabled?: Yes

As you note, this is not related to // @ts-ignore but to the global return, which causes TS to not analyze the function body. This seems like reasonable behavior to me since the global return means the file is syntactically invalid

Hmm, thats understandable but in our use case its possible to do a global return. Its a server software which can be extended with  Javascript. The global return value signals e. g. the success or the failure of the script. Not the best solution but can't be changed for historical reasons.
Any chances to change this behaviour? I can not see any disadvantages supporting this. Also there are plenty of other peoples which runs Javascript in a controlled environment which supports global return values, the would probably benefit, too.

An environment which allows global returns is not a JavaScript environment, but some superset thereof.
In addition to separating the return so it can be ignored as you pointed out above, you could potentially create a babel plugin (or similar) as a workaround. For instance, the separation and addition of // @ts-ignore could be automated for global returns before running the code through TS.
Alternatively, you could do something like the following and transform globalReturn after TS runs:
declare function globalReturn(value: unknown): void;

globalReturn((function() {

	var a = 0;
	a = "1";

})());

We're not really set up to figure out what syntactically-invalid JS "should" mean. There are assumptions about what's legal and what's not baked in pretty deep and we don't have the budget to start carving them out to support these kinds of setups.

If you have the option to change things moving forward, you might consider migrating your global returns to export default, which is syntactically legal in modern javascript, and can easily be hooked up to do what you want in node.js:
// shared shim code used by your script runner
import(scriptfile).then(e => process.exit(e.default));

If you have the option to change things moving forward, you might consider migrating your global returns to export default, which is syntactically legal in modern javascript, and can easily be hooked up to do what you want in node.js:
// shared shim code used by your script runner
import(scriptfile).then(e => process.exit(e.default));

Yes, that would be a workaround for this, but requires a lot of code changes.
Thought that this could help other peoples, too, because there is no disadvantage if vscode / typescript would analyse this function.

The fact that this is syntactically not javascript makes it a much larger effort to support than some other extensions, and the disadvantage would be to take time away from development efforts that benefit the entire TypeScript community.

Just throwing in my two cents: Global return is syntactically valid in CommonJS modules because Node wraps the module code in an IIFE before executing it.  I think I even remember seeing some modules in the wild that rely on this.


Minor fixes

added get_item function which calls get_tool_for and get_prompt_for
modified github workflows to update docs on push to master

minor, but can you open a new PR with a more descriptive branch name (e.g., tj202/PR_minor_fixes_1)


Warnings prevent writing shrinkwrap file

I'm unclear as to the point in distinguishing errors from warnings if either of them prevent the shrinkwrap from being written.
Unless there is an undocumented flag that allows writing shrinkwrap despite warnings?
Specifically, I have three git dependencies that don't have tags. We are pointing at shas in package.json because the particular modules don't have tagged versions. (In one case, they do, but the tag is mislabeled and doesn't actually match the module's version)

Further research clarified that there was indeed an error, but it was not identified as an error in the output:
WARN: Expected the git dependency async to have a valid version tag;
 instead I found <redacted> for the dependency git://<redacted>#<redacted>
WARN: Expected the git dependency rest-client to have a valid version tag;
 instead I found <redacted> for the dependency git://<redacted>#<redacted>
Expected the git dependency sequelize to have a tag;
 instead I found git://github.com/<redacted>
something went wrong. Did not write npm-shrinkwrap.json

The third logged item was not prefixed with 'ERROR' and was thus overlooked as a warning.


Kimono show deps within monorepo on a specific contained module

Knowing where all the deps are to a pkg that is changed within a monorepo would nice.

@rwxrob This is handled with the kimono dependents subcommand


Sort files by Last Modified not working with large numbers of files (>10,000)

Storage Explorer Version: 1.0.0
Platform: Windows
Architecture: ia32
Build Number: 20180415.2
Commit: 5d603052
Regression From: unknown
Steps to Reproduce:

Have a blob container with a large number of files (10,007 in my case)
Sort by Last Modified descending

Expected Experience:
The most recent files show at the top of the list.
Actual Experience:
The most recent files aren't shown at the top of the list.
If I click all the way through to page 101, then go back to the first page and sort descending again, then they do show up.

Let's (Storage Explorer team) follow-up with the platform team about having sorting options when listing blobs/files.

To be clear, this bug is by design (albeit, a less than ideal design...).
The APIs available to list blobs and files don't have any options for sorting. So to provide column sorting we had/have two options:

Get ALL of your blobs/files and then sort
Sort on what blobs/files we've currently gotten (cached) and sort that

We ended up going with 2 as solution 1 would have performance implications for people with lots of blobs/files. We've been asking the platform team for a server side sort for a while now, but unfortunately nothing has been committed to us yet. Please continue to 👍 the original issue if my explanation annoys you! Being able to show the number of 👍's to the platform team is a great way to convince them that y'all want something. 🙂

Same issue with Table.  https://msazure.visualstudio.com/One/_workitems/edit/3011494

In the meantime, could this be made a bit clearer to the user? Maybe a solution would be to add an option to request the full file list including a waiting bar until everything is loaded.

hope there will be a resolution for this, have had to go with an alternative software to access files by last modified date, but the rest of the UI is better in ASE

Do we have any ETA for the fix.  It would be good, if you could share with us.

tbh you should disable sorting by columns or at least show a warning in this situation - it is incorrect behavior.

Is this issue getting any attention from the platform team or a dead issue? Would be REALLY helpful for us to be able to sort on date (and other columns too).

See also https://github.com/microsoft/AzureStorageExplorer/issues/4692
The API doesn't have to return all blob metadata to sort by date, the API just needs a way to fetch the most recent blob metadata first (regardless of which data is in the cache) and then paginate through it from recent to older.
We use Blob Containers to store logs so usually we are mainly interested in the most recent entries.
Now we end up clicking "Load more" many, many times until all entries are cached, just to be able to sort by date.
And we need to do this every time because we need the newest entries.

Three years on and no update?
ORDER BY Date DESC feels a core requirement of any API which can list a lot of data.

Same here. Suprising to see a feature most websites have is not supported by Azure Storage Explorer.
In my case, I have to click 'next' like 20 times, until everything is loaded, and then I can sort by date in the last page.

This isn't the right place to complain -- this is an issue with Azure Blob Storage's API not with StorageExplorer. As @MRayermannMSFT said in the other thread there are only two options:

Get ALL of your blobs/files and then sort
Sort on what blobs/files we've currently gotten (cached) and sort that

1 is obviously infeasible, with many files this could take an indefinite amount of time; so StorageExplore has opted for 2 instead.
Personally my suggestion is: 3. Disable sorting by date completely -- don't pretend to support a feature that isn't possible with the underlying technology.

Personally my suggestion is: 3. Disable sorting by date completely -- don't pretend to support a feature that isn't possible with the underlying technology.

Yes, we agree (now) too. In fact this issue is the reason we changed our minds a while back towards what you say. 🙃  We're now much more conscientious about implementing features, especially complex ones, that aren't really supported by the service APIs. Nine times out of ten, it just leads to situations like this where the solution is sub-optimal, and y'all get something which is either only sort of helpful or confusing.
That said, we also do not like doing feature take back, so for that reason we have not removed this feature.

It would be great if this could be resolved.

It would be very helpful to have the files list sorted by Last Modified right after the list is loaded.
For my use case, there is no point in exploring ancient files.

There's a very simple solution - give us a load all button.
Even better, the option to change how many files are initially loaded and how many extra files are loaded when the Load More button is pressed. That way those of us who need the sort function don't have to keep clicking load more dozens of times, and everyone else is unaffected.
I was very concerned by the message above suggesting that the sort function be removed rather than fix the problem. I'd always rather have a half functioning feature than no feature at all. Being able to do my job is always preferable to not being able to do it at all. At present
@MRayermannMSFT - Are you able to give us an update on whether there's been any progress on this issue. There may not be that many people commenting on this thread, but I'm certain that a lot more people have fallen foul of this issue.
This is my first time using Azure Storage Explorer, and I immediately run into a bug. That seems to happen to me with almost every Microsoft product, and it's always a bug that was reported years ago and never been addressed. Very disappointing.

This problem hurts so bad, it's hard to describe.
Probably not more people raise their voice here because everyone thinks "This must be clear to MS and they'll fix it in the next update for sure"...

This problem hurts so bad, it's hard to describe.
Probably not more people raise their voice here because everyone thinks "This must be clear to MS and they'll fix it in the next update for sure"...

I think more likely they’ve given up on it ever getting fixed. I asked for an update 3 months ago and got nothing back. No point shouting into the void, you’re just wasting your breath I’m afraid.

Not an explorer solution, but you can list all entries via Azure CLI (Only tested with a few dozen entries, not 1000s).
az storage blob list --container-name <container-name>  --account-name <account-name> --auth-mode login --query "sort_by([], &properties.lastModified)[*].{name: name, lastModified: properties.lastModified}"
Documentation of az storage blob list
Use https://jmespath.org/ for the query syntax. With --include v you can also use the same logic to sort/show different blob versions.

This is incredibly painful.
Very easy fixes that the Storage Explorer team can do to alleviate the pain:

display a clear message that only the first 1,000 items were fetched.
make that limit configurable, so that users could bump it to 10,000.
persist the column sorting options

However I'd prefer that we get together and just fix the issue properly, since, as someone mentioned above, the portal supports this. It's clearly causing a lot of pain and frustration for many people. The worst about this issue is that you don't notice when you hit the limit, and the symptoms are "blobs stopped being added to the storage". So you are sent on a wild goose chase debugging why blobs are no longer uploading, reset and redeploy your systems, only to get the bright idea many hours later to try the API or the portal, which will show that Storage Explorer has been lying to you all along.

Hey @MRayermannMSFT, it seems like your recent release August 2023 (Version 1.31.1) fixed this?

@zerweck, I think you are referring to a sorting feature for tables. This issue is related to sorting of blobs.

Starting with 1.32.0, we will support global sort in Blob explorer and File Share explorer as well. The new behavior is also under the "global sort" setting that used to be for Table only. By clicking on the column header to sort, Storage Explorer will retrieve all the items and then sort all the items retrieved.

Is the sort setting persisted? I'd love for it to remember the sort setting I last chose. Either global or individual per particular resource (even better)

It is. In other words, once you enabled "Global Sort", if you close Storage Explorer and re-open it, the sort will still be doing global sort.

@JasonYeMSFT thanks for your information, this is exciting news. May I know when will 1.32.0 be released?

We are targeting Oct. 27th.


AssetManager - jsString & jsVar

Q
A

Is bugfix?
❌

New feature?
✔

Breaks BC?
❌

Fixed issues
Requires https://github.com/yiisoft/assets/pull/33 and https://github.com/yiisoft/view/pull/114

Should be applied to app as well.

Will do!