Systematic testing enhancements

Update SYSTEMATIC_TESTING_PRINTF to take a level argument.
Add cli argument for controlling output level of systematic testing printf's.
Add SYSTEMATIC_TESTING_YIELD and SYSTEMATIC_TESTING_PRINTF in a lot more places.

rebased to fix merge conflicts

rebased to fix merge conflicts

Once @jemc's comment about max value is addressed, I'm ok with these changes especially given that this is a new feature that I haven't used and I trust that @dipinhora is doing the right thing.

@dipinhora interested in finishing this off?

@dipinhora interested in finishing this off?

it's on my list.. but it can't be in this version of things.. need to account for things like the scheduler mutex (and/or get it working without pthreads) and the cycle detector critical which i didn't properly account for in this PR..

@dipinhora can i close this then?

doh! yes.. closing..


HAWQ - 276. Bump ORCA and GPOS version

@xinzweb @gcaragea @armenatzoglou
Please have a look when you get chance

LGTM

LGTM :+1:

Changes has been merged to master


Clicking upon user name redirects to wrong URL

If someone clicks on the username on the status card it redirects to wrong URL. I have solved the issue and sent a pull request please review it.

solved by #71


DO NOT MERGE - Add a global mutex for key slot data protection

This PR presents a simplified approach to multithreaded safety in PSA - single, global lock.
This PR is created mainly to test the idea and catch any CI failures.
This work is continued here, with a global readers-writer lock: https://github.com/ARMmbed/mbedtls/pull/5226

The current problem that this PR is experiencing is with full config, test_suite_psa_crypto_se_hal - Import-sign-verify: sign in driver, ECDSA:

psa_sign_internal locks the key slot (so the mutex is locked before that), then the test implementation of psa_driver_wrapper_sign_hash calls psa_import_key which also tries to lock the mutex. Same case for psa_verify_internal, psa_export_public_key. We might need some additional logic here.

Please note that the slot-focused approach is described in more detail here: https://github.com/ARMmbed/mbedtls/pull/5084

This work is continued with a global readers-writer lock here: #5226


Migrate proof-protocol-decoder crates

Move both crates from https://github.com/0xPolygonZero/proof-protocol-decoder to here, with renaming on the go.

protocol_decoder -> trace_parser / trace_decoder?
plonky_block_proof_gen -> proof_generator?

Done. Forgot to link this with the PR merge.


get_unread example needed

Hi all,
Does anyone have an example on how to get live incoming messages? so whenever a message is sent to you, the wrapper/scraper will read the message and store it.
I only need the part where it receives a message.
Currently I've this but I'm not sure if this is the best method:
            # Scraping unread messages.
            incoming_messages = self.driver.get_unread(use_unread_count=True)
            # Checking if we got any unread messages.
            if len(incoming_messages) > 0:
                    # Store message data.
                    continue
            else:
                    continue

Bump, anyone?

Hi, did you try the echo.py in the example? It uses driver.get_unread()


ReplicationInternal.stop() never executed.

ReplicationInternal.stop() is called when StateMachine state enter STOPPING state.
https://github.com/couchbase/couchbase-lite-java-core/blob/master/src/main/java/com/couchbase/lite/replicator/ReplicationInternal.java#L1300
However, ReplicationInternal.stop() method does not execute if isRunning() is false. isRunning() => RUNNING, IDLE, or OFFLINE
https://github.com/couchbase/couchbase-lite-java-core/blob/master/src/main/java/com/couchbase/lite/replicator/ReplicationInternal.java#L1089-L1090

Does the current replicator stop ever work before? I thought there is a unit test to test replicator stop.

I believe it works. But it takes long time because it does not clear batcher or some other property.

I remembered this now. This is a duplicate of https://github.com/couchbase/couchbase-lite-java-core/issues/1115.

I'm closing #1115 as this ticket has more info and discussion.

I'm wondering if this is related to the multiple replicators issue.

I strongly believe this is related with multiple replicator issue.
If my memory is precise, We set single thread mode for RequestWorkExecutor. Single thread mode is much slower than 5 threads mode. Stopping replicator could take longer (more than 60sec) because of this bug. But we have set timeout 60 to wait stop process. If timeout exhausts, new replicator starts. When capturing thread dump, stopped replicator could be not stopped yet.


Bad info

This page references a nonexistent cmdlet in its one example: Get-MsmqMessage. Terrible

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 83c2ea23-dc65-80d3-7cc1-d1fc61e0954f
Version Independent ID: ba1410a2-4359-0ae7-32e1-e4dcdf255d5d
Content: Receive-MsmqQueue (msmq)
Content Source: docset/winserver2012-ps/msmq/Receive-MsmqQueue.md
Product: w10
Technology: powershell
GitHub Login: @andreabarr
Microsoft Alias: v-anbarr

@joel74 An independent contributor will soon be able to follow-up with a clarification regarding your question. Thank you for providing feedback.

Closing via #1821


Port pomodo to Banglejs2

Made the pomodo app work on Banglejs2 in addition to Banglejs1.

Nice - thanks!


import generete_tesseract_results

minor change

There is no functional difference.

isn't inference.py used for training purpose?

Yes, I can't see any difference between the two scripts apart you have removed called main() function. May I know what is the reason behind removing that ?

its not --from utils import gererate_tesseract_results
but-- import gererate_tesseract_results

Oh my bad


git not available

Hi! I Have this error:
'git not available. It needs to be installed on the server for this script to work.'
Im using windows and bitbucket.
I already installed git and rsync on server.

If you read the script it looks like it is designed for NIX-based system only (Unix, Linux, MacOS...)


[BUG] "Cannot read property 'includes' of undefined" when deploying dataextensions

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Trying to deploy data extensions from one BU to another and getting an error right away.
Expected Behavior
Data extensions deployed to new BU
Steps To Reproduce

Takes all data extensions from one BU to another
Delete all data extensions already in BU you are moving data to
Run deploy command
See "Cannot read property 'includes' of undefined" error within seconds

Version
3.1.2
Environment

OS: Windows 10
Node: 14.9
npm: 8.3.2

Participation

[ ] I am willing to submit a pull request for this issue.

Additional comments
No response

hi @eklopfstein could you please check your /logs/ folder for the corresponding log file when this error happened and upload it here? I'm hoping to find a stack trace in it
warning: please do check the content for any hints on client data before uploading it to avoid problems...

@JoernBerkefeld
11:23:22.154 debug: :: mcdev 3.1.2 ::
11:23:22.805 info: mcdev:: Deploy
11:23:22.824 info: ::Deploying [project]/[BU]
11:23:23.495 debug: Extra access found:dfu_configure, journeys_aspr, journeys_delete, package_manager_package, package_manager_deploy, deep_linking_asset_read, deep_linking_asset_write, deep_linking_asset_delete, deep_linking_settings_read, deep_linking_settings_write
11:23:23.543 debug: Creating relevant folders for dataExtension in deploy dir
11:23:23.544 error: mcdev.deploy failed: Cannot read property 'includes' of undefined
11:23:23.544 debug: TypeError: Cannot read property 'includes' of undefined
at C:\Program Files\nodejs\node_modules\mcdev\lib\Deployer.js:195:69
at Array.filter ()
at C:\Program Files\nodejs\node_modules\mcdev\lib\Deployer.js:193:66
at Array.forEach ()
at Function.createFolderDefinitions (C:\Program Files\nodejs\node_modules\mcdev\lib\Deployer.js:170:25)
at Deployer.deploy (C:\Program Files\nodejs\node_modules\mcdev\lib\Deployer.js:69:24)
at Function._deployBU (C:\Program Files\nodejs\node_modules\mcdev\lib\index.js:469:32)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async Function.deploy (C:\Program Files\nodejs\node_modules\mcdev\lib\index.js:560:17)
11:23:23.544 info:
:: Deployed 1 BUs

Hey @eklopfstein, finally got a chance to look at this. I could only replicate the same kind of error when i remove the "r__folder_Path" field on the data extension object. Can you provide the metadata which you are trying to deploy and/or confirm if the field r__folder_Path is included?

hi @eklopfstein sorry we left this open this long. covid ruined our response times...
As Doug's research showed -  and I checked as well - the code that's failing has been there since 3.0.0
Could the download be from a pre-public version of the tool? Back then we didn't use that particular custom attribute.
Could you try downloading the DataExtension again and then doing the deploy with that download?
We'll keep this open for your reply.

There are 2 of us working on this, we are both on mcdev version 3.1.2
After looking through the files we noticed that some of the data extensions are missing the "r__folder_Path" field but not all of them. We were going to reload the BUs and run a retrieve again to see if that would fix it but we are getting the following error whenever we try to do anything now. "unable to get local issuer certificate". We believe this might be an issue with our network and are working to resolve it.
I will get back to you with a status on that tomorrow.

Hey @eklopfstein  - So that issue with certificates is something i also saw another client have issues with when using the SF library which lies behind our code. For now that can be resolved "insecurely" by trying
npm config set strict-ssl false
see https://stackoverflow.com/questions/36494336/npm-install-error-unable-to-get-local-issuer-certificate for details
In our next release, we move to sfmc-sdk which is a lib I am working on, where this issue was no longer faced (it replaces request.js with axios).  This release is on the develop branch if you wanna try it out.

closing this one for now. if it happens again, please let us know!


[BUG] Ctrl+"Autoformat Table" not working on macOS

Just FYI, the option to "Hold Ctrl and click the Autoformat Table quick access button to adjust column widths and un-hide all hidden columns." does not work on macOS.
https://nfdi4plants.org/nfdi4plants.knowledgebase/docs/implementation/QuickStart_swate_walkthrough.html#showing-ontology-reference-columns

Does it show any error message? Can you show me console output during the operation?

Console stays blank.
Problem is likely different mapping of macOS function keys (e.g. copy = command + c)
command + left click = behaves just like left click (auto format hides columns)
option + left click = behaves just like left click (auto format hides columns)
option + left click = behaves like right-click (context menu)

Ah this is actually good to know, i will check for this.

Should be fixed on > Swate-alpha06. On mac you might be able to holdcontrol or command now, let me now if it works or not 😄

Cannot find the `auto format hides columns in 0.8.0-alpha06

Yeah not yet released, it will be live in 0.8.0-alpha07. Just adding more stuff currently 👍


NVM_HAS_COLORS unbound variable in shell scripts running in Github Actions Ubuntu images

Operating system and version:
Github actions CI runner: https://github.com/actions/runner-images/blob/ubuntu20/20221024.1/images/linux/Ubuntu2004-Readme.md
nvm debug output:

Run source $HOME/.nvm/nvm.sh
  source $HOME/.nvm/nvm.sh
  nvm debug
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
nvm --version: v0.39.[2](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:2)
$SHELL: /bin/bash
$SHLVL: 1
whoami: 'runner'
${HOME}: /home/runner
${NVM_DIR}: '${HOME}/.nvm'
${PATH}: ${HOME}/.local/bin:/opt/pipx_bin:${HOME}/.cargo/bin:${HOME}/.config/composer/vendor/bin:/usr/local/.ghcup/bin:${HOME}/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
$PREFIX: ''
${NPM_CONFIG_PREFIX}: ''
$NVM_NODEJS_ORG_MIRROR: ''
$NVM_IOJS_ORG_MIRROR: ''
shell version: 'GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)'
uname -a: 'Linux 5.15.0-1022-azure #27~20.04.1-Ubuntu SMP Mon Oct 17 02:0[3](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:3):50 UTC 2022 x86_6[4](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:4) x86_64 x86_64 GNU/Linux'
checksum binary: 'sha2[5](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:6)[6](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:7)sum'
OS version: Ubuntu 20.04.5 LTS
curl: /usr/bin/curl, curl [7](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:8).6[8](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:9).0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.[9](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:10).3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
wget: /usr/bin/wget, GNU Wget 1.[20](https://github.com/jobackman/nvm-unbound-variable/actions/runs/3329175279/jobs/5506057361#step:5:21).3 built on linux-gnu.
git: /usr/bin/git, git version 2.38.1
grep: /usr/bin/grep, grep (GNU grep) 3.4
awk: /usr/bin/awk, GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.2.0)
sed: /usr/bin/sed, sed (GNU sed) 4.7
cut: /usr/bin/cut, cut (GNU coreutils) 8.30
basename: /usr/bin/basename, basename (GNU coreutils) 8.30
rm: /usr/bin/rm, rm (GNU coreutils) 8.30
mkdir: /usr/bin/mkdir, mkdir (GNU coreutils) 8.30
xargs: /usr/bin/xargs, xargs (GNU findutils) 4.7.0
nvm current: system
which node: /usr/local/bin/node
which iojs:
which npm: /usr/local/bin/npm
npm config get prefix: /usr/local
npm root -g: /usr/local/lib/node_modules

nvm ls output:
Getting this output on the runner is the issue

How did you install nvm?
Preinstalled on the github actions runner image: https://github.com/actions/runner-images/blob/ubuntu20/20221024.1/images/linux/Ubuntu2004-Readme.md
What steps did you perform?

Make a workflow for a github actions ubuntu-latest/20.04 run
Have a shellscript with the set -u flag enabled
Source nvm
Run nvm list

This fails
with-u-opt.sh
#! /bin/bash
set -u pipefail
source $HOME/.nvm/nvm.sh
nvm list

 -run: ./with-u-opt.sh

This works
without-u-opt.sh
#! /bin/bash
source $HOME/.nvm/nvm.sh
nvm list

 -run: ./without-u-opt.sh

What happened?
Example run from reproduction repo
/home/runner/.nvm/nvm.sh: line 1695: NVM_HAS_COLORS: unbound variable
Error: Process completed with exit code 1.

What did you expect to happen?
In v0.39.1 this error did not happen on the runner but after the latest image, which upgraded nvm to v0.39.2
I believe this has been the case since #2827 was merged
Reproduction repository:
https://github.com/jobackman/nvm-unbound-variable
Example run
Is there anything in any of your profile files that modifies the PATH?
Nope

If you are having installation issues, or getting "N/A", what does curl -I --compressed -v https://nodejs.org/dist/ print out?

Thanks for the report.

I'll be releasing a v0.39.3 today, which will include this fix.


User config for SSH connect timeout

Because of a slow SSH tunnel connection of one of my servers, I made some modification to dbeaver interface in order to be able to configure SSH tunnel timeout.
If you like to have this feature in the final build, I can make a PR.

Yes, please make a PR.

PR #972 done.


fix(Input): spread input props on the input child

Fixes #909
This PR spreads props on the input child when using children in advanced input variations.  It also updates the docs, hoisting all <input /> props to the <Input /> parent.
I've spread the props so that the input child props win, for backward compatibility.  Though, the suggested use here is to use the input child as a placeholder, without props.  I've updated the docs to reflect this as well.

Current coverage is 100% (diff: 100%)

Merging #916 into master will not change coverage

@@           master   #916   diff @@
====================================
  Files         137    137
  Lines        2259   2262     +3
  Methods         0      0
  Messages        0      0
  Branches        0      0
====================================
+ Hits         2259   2262     +3
  Misses          0      0
  Partials        0      0

Powered by Codecov. Last update 37f4468...b99945b

Released in<EMAIL_ADDRESS>


The specified key is not present

Hi
I've runned into an error which has been raised before about the absence of a key.
My set up is
data recorded with oe6.+,
toolbox: open-ephys-matlab-tools-1.0.2
matlab version: 2020a
Error using containers.Map/subsref
The specified key is not present in this container.
Error in BinaryRecording/loadContinuous (line 93)
stream.metadata.startTimestamp = syncMessages(streamKey);
Error in BinaryRecording (line 40)
self = self.loadContinuous();
Error in BinaryRecording.detectRecordings (line 254)
recordings{end+1} = BinaryRecording(recordingDirectories{recIdx}, expIdx, recIdx);
Error in RecordNode/detectRecordings (line 95)
self.recordings = BinaryRecording.detectRecordings(self.directory);
Error in RecordNode (line 71)
self.detectRecordings();
Error in Session/detectRecordNodes (line 70)
self.recordNodes{end+1} = RecordNode(paths{i});
Error in Session (line 61)
self.detectRecordNodes();
Thanks
G

Hi
Can you print the value of the streamKey variable in BinaryRecording/loadContinuous (line 93) right before this fails?

Hello,
I received the same error when trying to load data.
streamKey = 'Acquisition_Board-100.Rhythm Data'
This used to work just fine for me on an older install so I rolled back to an older version and checked - it works just fine with the version committed 10/04/2022.

Fixed in v1.3.0


Pass environment variables to Popen

Previously, when testing nvcc, environment variables were not being passed along to the subprocess. This could cause issues as compiler flags would get stripped or the $PATH not preserved. This fixes that issue by passing os.environ to Popen's env argument.

Assuming this change is agreeable, I'd also like to backport this change to v6. Happy to submit that as well or defer to whatever process you have in place for this sort of thing. 🙂

subprocess.Popen inherits environment variables of the parent process when env is not specified, so I guess this fix is not needed.

Ah you're right. Sorry for the noise.


Interactions don't work until you render stage twice

Hi,
please take a look at this example:
http://jsfiddle.net/jL7Ce/
Looking at source and docs, I would expect that my mousedown handler should be executed when I click a circle. However unless you set renderTwice to true, it won't work.
Also I'm a bit unsure about these lines:
https://github.com/GoodBoyDigital/pixi.js/blob/master/src/pixi/InteractionManager.js#L145-L149
This assumption there we need at least 33ms between updates can work fine during the animation, however consider such scenario:

render stage
remove all sprites, create a new set of interactive sprites
render stage again

The Interaction Manager update should be performed, but if there is less than 33ms between 1. and 3., it won't happen. This seems to cause problems in our app, where rendering can be triggered by user actions, initialization code etc. and we don't always use animation.

Interactions and the stage (any particular DO) will be decoupled in v3.

this should work like a champ in v3 now. One render and you should be good to go :+1:


netcdf-4.4.1 tarball missing nc_test4/tst_fill_attr_vanish.c

File is mentioned in nc_test4/CMakelists.txt, but source file does not exist in the tar ball.

Super. Thanks for reporting this, now I need to figure out why make distcheck didn't catch this.  Most likely I neglected to wire tst_fill_attr_vanish.c into the autoconf tests.  Clearly there are still holes in our 'is everything ready for a release' script.
Thanks!

We will probably put out a new service release, <IP_ADDRESS>, tomorrow or next Tuesday, to address this.

The <IP_ADDRESS> release is delayed until we address some other things, and the issue is fixed so closing this out.


Per channel notification settings

Internal APIs now supports an optional param to disable notifications (default to false): NotificationsDisabled (https://github.com/lbryio/internal-apis/commit/85c0c9ba473d123c821a24dbcc15637000c6dc09#diff-0b74aa8ad1cbaacf3aa174a3ce04d627R31)
The same api can be used to edit an existing subscription.

[ ] Add bell icon on channel/content pages for notifications when you are subscribed, with the ability to turn on and off. Make it clear which state is turned on (some users are still confused on mobile mode with heart/broken heart).

[ ] store and sync notification preference in the wallet

(Possibly reduce notifications to one property, with @tzarebczan )


API Publisher: Add scope to multiple selected resources

Description:
For an API with a large number of resources, it's very tedious to click through and assign a scope to each resource. Please enhance the UI to allow a scope to be added to multiple selected resources. Refer attached screenshot.

Tagging https://github.com/wso2/product-apim/issues/7292

will keep this track with https://github.com/wso2/product-apim/issues/7292


[Question] Type URI as string in model

Please take the time to search the repository, if your question has already been asked or answered.

[ ] What version of the library are you using? 2.7.0 Is it the latest version? yes

What kind of issue is this?

[x] Question. Is this a question about how to do a certain thing? yes, for now

[ ] Bug report. no, probably not

[ ] Feature Request. no

Here's the question, I've created the following request object:
public class EnvironmentUpdateRequest {
    @NotNull
    @ApiModelProperty(dataType = "string") // an attempt to fix my issue
    private URI consulUri; // that's java.net.URI

    public URI getConsulUri() {
        return consulUri;
    }

    public void setConsulUri(URI consulUri) {
        this.consulUri = consulUri;
    }
}

It appears as follows in the emitted Swagger JSON:
"EnvironmentUpdateRequest": {
    "properties": {
        "consulUri": {
            "$ref": "#/definitions/URI"
        }
    },
    "type": "object"
},

Since it's "type": "object", and there's a comprehensive description of URI in definitions, it'll use that. But in reality, Jackson perfectly serializes a string into the desired java.net.URI that's needed. So, I'd say that my API works fine, but its documentation is flawed. How can I make Springfox type this field as string?
Thank you for an awesome library!

I think as a workaround you could create a rule to map URI to String but I agree that it should just work out of the box.

This seems to work, and I'll move forward with this as a workaround:
.alternateTypeRules(
    newRule(typeResolver.resolve(URI.class), typeResolver.resolve(String.class)))

In combination with:
@Autowired
private TypeResolver typeResolver;

I don't think it's necessarily a bug that URI isn't mapped as string, but it is strange that dataType = "string" isn't adhered to. (No matter Springfox "Philosophically" wanting "to discourage using (swagger-core) annotations".)
Thanks for the quick response.

Actually it will work if you specified java.lang.String. dataType is ignored when not fully qualified.

Please note that I'm spelling string, not String. I was under the impression that string would be considered the primitive, and thus valid in the form as given. I'll quote the following links, although I'm unsure whether either of those is all too relevant to Springfox: "This can be the class name or a primitive", and types from JSON Schema Specification. I do agree that dataType = "java.lang.String" works, I hadn't tried that, my bad entirely! Thanks again.

@sverhagen Yeah there are slight deviations when using the annotations from the mother ship :) Mostly to protect library users from fat-fingering errors.


Use deterministic GUIDs to create event IDs

Not sure if this is a good idea...

This is a very good idea and will add great value to EventFlow.
For example - I need to correlate aggregateroots based on a deterministic underlying element - an email address - to locate registration requests and map them to existing accounts and users.
I am considering adding a Identity<TIdentity, TComposer, TValidator> abstract class where TComposer is used to compose the payload of the identity and this will be one of the available composers.
A related issue I have is in the planned use of EventFlow in distributed solution of desktop, server, web and mobile clients where aggregates may be created anywhere but need to be deterministicely identifiable across the network for reconciliation purposes. The same goes for events in the use case mentioned e.g. Who Changed What Where When Why to track colliding edits to aggregates in an optimistic/late locking approach.
Another potential benefit is dropping the aggregate/class name  from the identity and making it a pure GUID as the aggregate object name is tracked in the meta data.

@JC008 Thanks for the feedback. The factory for creating the UUIDs will be added as a generic factory that merely returns a Guid, making it possible for everyone to use it. I'll add support for it in the base Identity<> class by adding a new New(Guid namespaceId, string name) making it easier to create aggregate root IDs if you are using that base class for creating the IDs.

@JC008 If you have a link to an article/book that discusses your use cases, I would like to read it.

http://book.mixu.net/distsys/ (chpt 3 and 4)
Support for Lamport clocks is worth considering too.

@JC008 Thanks


Cherry pick few commits to retry and debug the PR check pipeline

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[ ] The toolchain has been rebuilt successfully (or no changes were made to it)
[ ] The toolchain/worker package manifests are up-to-date
[ ] Any updated packages successfully build (or no packages were changed)
[ ] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
[ ] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
[ ] All package sources are available
[ ] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[ ] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[ ] All source files have up-to-date hashes in the *.signatures.json files
[ ] sudo make go-tidy-all and sudo make go-test-coverage pass
[ ] Documentation has been updated to match any changes to the build system
[ ] Ready to merge

Summary

What does the PR accomplish, why was it needed?
Currently, prcheck pipeline is intermittently failing.
i ) Add debugging to the PR-check pipeline
ii) Add retry to the failing step ie pr check's Python dep installation step
Change Log

Change: Added system.debug variable
Change: add retryCountOnTaskFailure on failed step

Does this affect the toolchain?

YES/NO
No

Auto cherry-pick results:

main :x:

Auto cherry-pick pipeline run -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=450429&view=results


Azure VM worked OK

This tutorial worked OK for me using an Azure VM. Was able to reuse IoT Hub setup from other related tutorials.
Only problem I had was pasting the connection string (use right-click context).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e2289383-6ce2-a036-c710-7d1fad6e1700
Version Independent ID: 3b902941-0876-17c3-3f0b-a4b7f7770531
Content: Quickstart create an Azure IoT Edge device on Linux
Content Source: articles/iot-edge/quickstart-linux.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Allow third party components to modify deployment file or configuration

Author Name: Ruben S. Montero (@rsmontero)
Original Redmine Issue: 4880, https://dev.opennebula.org/issues/4880
Original Date: 2016-10-26

This can be done either by:

Generating deployment file in the driver instead of core
Adding support for template the deployment file

Original Redmine Comment
Author Name: EOLE Team (EOLE Team)
Original Date: 2016-11-04T12:19:58Z

+1 for the template system but I don't know a lib for that task.
We can even be crazy and imaging defining several templates and defining a new ONE template attribute to select which one to use.
For example:

define some templates
** @/etc/one/templates/vm-default.xml.tmpl@
** @/etc/one/templates/vm-scsi.xml.tmpl@
define some resource templates
** VM template without any specific attribute → default to using @/etc/one/templates/vm-default.xml.tmpl@
** VM template with an attribute, lets call it @LIBVIRT_TEMPLATE@, equal to @vm-scsi@ → use @/etc/one/templates/vm-scsi.xml.tmpl@ as base to generate the libvirt domain configuration

Regards.

Original Redmine Comment
Author Name: Anton Todorov (Anton Todorov)
Original Date: 2017-01-11T16:31:25Z

+1 from me too.
I am thinking of an extra step that is called on the FE just before the deploy action on the Host. Something like:
for each file in remotes/vmm//predeploy.d/
exec the file with VM_ID and the domain XML (base64 encoded) as arguments.
if the exit status of the script is SUCCESS the stdout of the script is the base64 encoded altered domain XML.
This way anyone can drop a script that alter in some way the domain XML. and the script will have access to all needed information to complete (acess datastores, vm,s hosts info from opennebula)
And, in a future step it could be possible to move all domain XML definition from the core to a script(s) there too.
Kind Regards,
Anton Todorov

Original Redmine Comment
Author Name: Anton Todorov (Anton Todorov)
Original Date: 2017-01-11T16:42:39Z

Also, the previously proposed scheme should be implemented for the attach actions too - option to alter the XML that is passed on an attach action.
Or this should be a separate feature request?
Kind Regards,
Anton Todorov

Original Redmine Comment
Author Name: Ruben S. Montero (@rsmontero)
Original Date: 2017-05-25T14:00:01Z

We can keep both in this issue. I'm moving this to next release.


Add loosebazooka to chains approvers

Hey @loosebazooka
I am not sure what the tekton community protocol is here, but would it be better to have you made a contribution first?
p.s I think you would be great for the job, but I am guessing approvers should typically have made a PR to the project before.
Please do correct me if I am wrong (anyone)?

Oh yeah, that's okay. I was just working with @priyawadhwa on some stuff.

/lgtm

@lukehinds there should be a PR open soon!


add documentation for priority queue

Do the checklist before submitting the PR:

[X] Have you read the guidelines mentioned in CONTRIBUTING.md?

Q: What version of Node.js you've used in the project? (e.g. Node v10.11.0)
A: Node v10.15.3
Q: Mention the Issue Number! (e.g. Fixed #8)
A: Fixed #207

Give additional information regarding the PR below:

Codecov Report

Merging #213 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #213   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          80     80
  Lines        1451   1451
=====================================
  Hits         1451   1451

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c75b30...d5dda1d. Read the comment docs.


[Feature Request] relax openai version constraint to support 1.x

Is your feature request related to a problem? Please describe.

Describe the solution you'd like
relax open ai version constraint

Additional context
Add any other context or screenshots about the feature request here.

After updating the openai library to version 1.1.1,
invoke using the latest method.
from openai import OpenAI
client = OpenAI()

completion = client.chat.completions.create(
  model="gpt-3.5-turbo-1106",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

Running will result in an error.
 File "C:\ProgramData\miniconda3\Lib\site-packages\promptflow\_core\openai_injector.py", line 131, in inject_openai_api
    openai.api_key_path = os.environ.get("OPENAI_API_KEY_PATH", openai.api_key_path)
                                                                ^^^^^^^^^^^^^^^^^^^
AttributeError: module 'openai' has no attribute 'api_key_path'

The following two lines in the openai_injector.py annotation can work properly.
openai.api_key_path = os.environ.get("OPENAI_API_KEY_PATH", openai.api_key_path)
openai.api_base = os.environ.get("OPENAI_API_BASE", openai.api_base)

@happyyuki7 Thanks for reporting this, we opened a PR for this:
https://github.com/microsoft/promptflow/pull/1072

@thy09 we shall also relax the version contraint in setup.py

@wangchao1230 Will do that when most stuffs are ready.

Is it possible for now to loosen it to even allow 0.28.1, the one right before 1.0? https://pypi.org/project/openai/0.28.1/
Currently your setup.py requires "openai<0.28.0" and I'm not sure why 0.28.0/0.28.1 wouldn't be okay.

Hi @pamelafox we are going to remove the constraint in the next release, is it OK for you?

That'd be fine as well. I ended up removing promptflow as a dependency entirely, as it also has upper version constraints on Flask that are incompatible with our app. It'd be great if you could relax as many upper version constraints as possible. See reasoning in: https://iscinumpy.dev/post/bound-version-constraints/

@pamelafox The reason we pin some package versions is that we met some issue when some dependency updates.
If you have concern of the version of flask, you may create another issue for it so we can track this.

We have relaxed the OpenAI version constraint and supported the OpenAI V1 in promptflow 1.1.0 and promptflow-tools 1.0.0, closing this issue.


setup and teardown are not passing original arguments from frameworks

QUnit 2 passes assert as an argument to theses callbacks:
QUnit.module( "router", {
    beforeEach: function( assert ) {
        this.router = new Router();
    },
    afterEach: function( assert ) {
        this.router.destroy();
    }
});

However TestModule is not passing these arguments to the user's callbacks from setup, teardown to invokeSteps

Thanks for reporting, this is an ember-qunit specific issue. I opened https://github.com/rwjblue/ember-qunit/issues/151 to track over there...


Creating Posts in Factory

In my factory, I'm creating 10 posts and most everything is working, but the Trix "field" column is set ot 0 instead of "content" as it is when I create a post via the UI.
            'post-trixFields' => $this->faker->paragraphs(1),
I'm sure it's simple, but I haven't figured out how to set the column properly. Since it's a seed/factory I don't need it too often, but it'd be nice to understand.

This seemed to be the solution
['content' => $this->faker->paragraphs(1, true)]


Bug in Numeric Expression Post-processor

As reported by @vince62s there is a bug with the Numeric Expressions Post-processor, here's an example:
There were more then 150 000 new cases of cancer => Il y a eu plus de 000 000 nouveaux cas de cancer
There were more then 150000 new cases of cancer => Il y a eu plus de 150000 nouveaux cas de cancer
There were more then 180,000 new cases of cancer => Il y a eu plus de 180,000 180,000 nouveaux cas de cancer

NE Post-processor does not handle well multiple target placeholders for the same source word.

branch: master commit:3185b56b42162cfcc58bb1e2b786a772b1bbbfe0
mvn clean install fails, with this error:
Failed tests:   testBigNumber_NotMatching(eu.modernmt.processing.numbers.NumericWordTest): expected:<147[530.]50> but was:<147[.530,]50>
testMultiple_Matching(eu.modernmt.processing.numbers.NumericWordTest): expected:<147[ ]530> but was:<147[]530>
testCurrency_NotMatching(eu.modernmt.processing.numbers.NumericWordTest): expected:<[$1.76]> but was:<[1,76$]>
testMultiple_NotMatching(eu.modernmt.processing.numbers.NumericWordTest): expected:<147.530[ ]??> but was:<147.530[]??>

Just pushed the fix into master.
Thanks @nicolabertoldi, sorry for that!


chorse: add ignored warning for fasttext

When running the fasttext appear the warning so much, but not really significant, I put the ignored warning to have better experimence in preprocess data.

Ok


Feature/fixing validation on master

Change description
Fix to get Validation working on Master.
FormGroup Validation is now configurable from a Json object.
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x ] No

Looks good for me to merge


whin number was banned from whatsapp.

There's nothing we can do about it.
You should stop using this nodes.

reset the hourly max rate to 20 / hour.

reduced from 100 to 20 / hour


Upgrade libsass to 3.3.5

woot woot woot

Coverage remained the same at 94.464% when pulling 88a0cbd0c91baa0e6be274f8a4de4b301aea6d54 on asottile:3_3_5 into 17198c87e1aa104ee1fc25e9d75f824b0dc1b688 on dahlia:master.


Unable to use with Selenium

Issue:
I'm trying to use Bokeh to save a picture of a plot. Under the hood, Bokeh uses Selenium and the firefox/geckdriver to render the plot in a headless browser and take a screenshot of it. I'm unable to use the firefox and geckodriver packages on the conda-forge channel to drive the headless browser.
Do you have any advice on how I should go about this? Thanks in advance for taking a look at this! I've been trying to get this work on and off for a few weeks now. I know it's a thorny problem and maybe out of scope to get this work for the firefox feedstock.
Here's a Dockerfile I'm using for this:
FROM continuumio/miniconda3

RUN conda install -c conda-forge selenium "firefox==81.0" geckodriver ipython

RUN echo 'from selenium import webdriver \n\
driver = webdriver.Firefox()' > /test_simple_way.py

# https://github.com/bokeh/bokeh/blob/branch-2.1/bokeh/io/webdriver.py#L55-L60
RUN echo 'from selenium import webdriver \n\
from selenium.webdriver.remote.webdriver import WebDriver \n\
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary \n\
from os.path import devnull \n\

binary = FirefoxBinary("/opt/conda/bin/firefox") \n\
options = webdriver.firefox.options.Options() \n\
options.add_argument("--headless") \n\
wd = webdriver.Firefox(firefox_binary=binary, options=options, service_log_path=devnull) \n\
wd = webdriver.Firefox(firefox_binary="/opt/conda/bin/firefox", options=options, service_log_path=devnull) \n\
' > /test_bokeh_way.py

Test simple script:
# test_simple_way.py
from selenium import webdriver
driver = webdriver.Firefox()

$ docker run -it tb-selenium python test_simple_way.py
Traceback (most recent call last):
  File "test_simple_way.py", line 2, in <module>
    driver = webdriver.Firefox()
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

Use code from Bokeh repository:
# test_bokeh_way.py
from selenium import webdriver
from selenium.webdriver.remote.webdriver import WebDriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from os.path import devnull
binary = FirefoxBinary("/opt/conda/bin/firefox")
options = webdriver.firefox.options.Options()
options.add_argument("--headless")
wd = webdriver.Firefox(firefox_binary=binary, options=options, service_log_path=devnull)
wd = webdriver.Firefox(firefox_binary="/opt/conda/bin/firefox", options=options, service_log_path=devnull)

$ docker run -it tb-selenium python test_bokeh_way.py
Traceback (most recent call last):
  File "test_bokeh_way.py", line 8, in <module>
    wd = webdriver.Firefox(firefox_binary=binary, options=options, service_log_path=devnull)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: binary is not a Firefox executable

Environment (conda list):

(base) root@71fb2d57ee08:/# conda list
# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
asn1crypto                1.3.0                    py37_0
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.1                      py_0    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2020.6.20        py37hc8dfbb8_0    conda-forge
cffi                      1.14.0           py37h2e261b9_0
chardet                   3.0.4                 py37_1003
conda                     4.8.5            py37hc8dfbb8_1    conda-forge
conda-package-handling    1.6.0            py37h7b6447c_0
cryptography              2.8              py37h1ba5d50_0
decorator                 4.4.2                      py_0    conda-forge
firefox                   81.0                 he1b5a44_0    conda-forge
geckodriver               0.27.0               he1b5a44_0    conda-forge
idna                      2.8                      py37_0
ipython                   7.18.1           py37hc6149b9_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jedi                      0.15.2                   py37_0    conda-forge
ld_impl_linux-64          2.33.1               h53a641e_7
libedit                   3.1.20181209         hc058e9b_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 9.1.0                hdf63c60_0
libstdcxx-ng              9.1.0                hdf63c60_0
ncurses                   6.2                  he6710b0_0
openssl                   1.1.1h               h516909a_0    conda-forge
parso                     0.5.2                      py_0
pexpect                   4.8.0            py37hc8dfbb8_1    conda-forge
pickleshare               0.7.5           py37hc8dfbb8_1001    conda-forge
pip                       20.0.2                   py37_1
prompt-toolkit            3.0.7                      py_0    conda-forge
ptyprocess                0.6.0                   py_1001    conda-forge
pycosat                   0.6.3            py37h7b6447c_0
pycparser                 2.19                     py37_0
pygments                  2.7.1                      py_0    conda-forge
pyopenssl                 19.1.0                   py37_0
pysocks                   1.7.1                    py37_0
python                    3.7.6                h0371630_2
python_abi                3.7                     1_cp37m    conda-forge
readline                  7.0                  h7b6447c_5
requests                  2.22.0                   py37_1
ruamel_yaml               0.15.87          py37h7b6447c_0
selenium                  3.141.0         py37h8f50634_1001    conda-forge
setuptools                45.2.0                   py37_0
six                       1.14.0                   py37_0
sqlite                    3.31.1               h7b6447c_0
tk                        8.6.8                hbc83047_0
tqdm                      4.42.1                     py_0
traitlets                 5.0.4                      py_1    conda-forge
urllib3                   1.25.8                   py37_0
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
wheel                     0.34.2                   py37_0
xz                        5.2.4                h14c3975_4
yaml                      0.1.7                had09818_2
zlib                      1.2.11               h7b6447c_3

Details about  conda and system ( conda info ):

(base) root@71fb2d57ee08:/# conda info

     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files :
          conda version : 4.8.5
    conda-build version : not installed
         python version : 3.7.6.final.0
       virtual packages : __glibc=2.28
       base environment : /opt/conda  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.8.5 requests/2.22.0 CPython/3.7.6 Linux/5.4.0-48-generic debian/10 glibc/2.28
                UID:GID : 0:0
             netrc file : None
           offline mode : False

(base) root@71fb2d57ee08:/#

Sorry for the inconvenience, thanks for filling out the full report!
This is due to a change in how geckodriver 0.27.0 validates the binary. This feedstock makes a $PREFIX/bin/firefox wrapper bash script which invokes the real firefox.  Firefox ships a lot of files that want to be in particular places relative to each other.
Today, the options are:

easy: pin to geckodriver =0.26.0

it's not that broken, haven't played with the PDF output, which was one of the marquee features

less easy: add $PREFIX/bin/FirefoxApp to $PATH (depends on platform) when testing
harder: detect and change the arguments to selenium

e.g. https://github.com/conda-forge/firefox-feedstock/blob/master/recipe/test_selenium.py#L33
some helpful "wrappers" make it hard to pass this kind of information in

In the future, we should investigate using a symlink to avoid this headache...

Hey @bollwyvl, thanks for the quick response. I'm perfectly happy to go the easy way and pin to 0.26.0 for now. I updated the Dockerfile:
FROM continuumio/miniconda3

RUN conda install -c conda-forge selenium "firefox==81.0" "geckodriver==0.26.0" ipython

RUN echo 'from selenium import webdriver \n\
driver = webdriver.Firefox()' > /test_simple_way.py

# https://github.com/bokeh/bokeh/blob/branch-2.1/bokeh/io/webdriver.py#L55-L60
RUN echo 'from selenium import webdriver \n\
from selenium.webdriver.remote.webdriver import WebDriver \n\
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary \n\
from os.path import devnull \n\

binary = FirefoxBinary("/opt/conda/bin/firefox") \n\
options = webdriver.firefox.options.Options() \n\
options.add_argument("--headless") \n\
wd = webdriver.Firefox(firefox_binary=binary, options=options, service_log_path=devnull) \n\
wd = webdriver.Firefox(firefox_binary="/opt/conda/bin/firefox", options=options, service_log_path=devnull) \n\
' > /test_bokeh_way.py

And got these results:
(taxbrain-dev) ~/Tax-Brain
$ docker run -it tb-selenium python test_bokeh_way.py
Traceback (most recent call last):
  File "test_bokeh_way.py", line 8, in <module>
    wd = webdriver.Firefox(firefox_binary=binary, options=options, service_log_path=devnull)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

(taxbrain-dev) ~/Tax-Brain
$ docker run -it tb-selenium python test_simple_way.py
Traceback (most recent call last):
  File "test_simple_way.py", line 2, in <module>
    driver = webdriver.Firefox()
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/opt/conda/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

It looks like the bokeh script is now producing the same error message as the simple script. Previously:
selenium.common.exceptions.InvalidArgumentException: Message: binary is not a Firefox executable

Now:
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

Do you have any ideas what may be happening? Do I need to actually install Firefox in the Docker image?

Unable to find a matching set

This means firefox crashed. You can crank up the verbosity of geckodriver (--log-level TRACE), but here's my hunch:
Yeah, even though it's "headless" it still needs a fair amount of heavy-duty stuff.
Having "a" modern firefox on your machine will usually be enough, as it will bring along all of the gtk and x11 deps you need.  At that point, one might just use a system firefox, no?! However, being able to pin (or at least track) the versions of firefox & geckodriver & selenium together is useful, I find.

Ah, that makes sense. Thanks for all of the help and info @bollwyvl! I talked with my colleague and he's going to look into some other approaches that don't use a headless browser.

a fair amount of heavy-duty stuff.

Understood. I tried to install chrome in a docker image and oh my:

Closing since my colleague and I are going a different direction. Thanks again for your time @bollwyvl !

Much like a real computer (vs docker/VMs), sure: a real browser can be overkill unless you are testing whether something works in a real browser. If you just want pictures, just about anything else will be more maintainable over time. Good luck with whatever you're doing!
If you do end up needing it: the advantage of the firefox/geckodriver stack:

LTS exists (e.g. 68 and 78)

good luck with any version of chrome that isn't bleeding-edge-today (it might upgrade itself while you're testing)
what even is safari?

geckodriver vs firefox compatibility has a useful, published support window

chromedriver needs an exact version of chrome these days, down to the build level... not the same version string as itself, of course!

For any others that come along: geckodriver 0.28.0 is no longer so hard-core about only starting a Real Firefox Binary:

Firefox can be started again via a shell script that is located outside of the Firefox directory on Linux

https://github.com/mozilla/geckodriver/releases/tag/v0.28.0


Test data URL will go down soon

Currently, the test data for this and our other pipelines is hosted on the UPPMAX milou webexport resource. This is going to disappear pretty soon. UPPMAX have no plans to replace this service anywhere else, so we need to come up with an alternative solution.
Using a project on the SNIC Science Cloud could work (we have a project there). We could also set up a mini sandboxed web server on one of our local servers. Or we could try to use some other kind of public data hosting service (such as GitHub if the data isn't too big?).
Phil

This is fixed and done..


Question regarding ContainerCreateOptsBuilder::mounts

Hi @vv9k,
I just noticed that you added ContainerCreateOptsBuilder::mounts, which expects the following data.
https://github.com/vv9k/podman-api-rs/blob/f87d80d50d4a4b3846c0f7f899b3f16ecf5fcd44/podman-api-stubs/lib/src/models.rs#L4694
I have been using my own fork of podman-api-rs with a slightly different mounts option for a while now and would love to switch back to upstream.
Here you can see how it looks.
https://github.com/marhkb/pods/blob/95dee64fc5d2a79a3ddd7fe7d03f790d2ef986d1/src/view/container/creation_page.rs#L771
https://github.com/marhkb/pods/blob/95dee64fc5d2a79a3ddd7fe7d03f790d2ef986d1/src/view/container/creation_page.rs#L882
podman-cockpit was taken as an example for this.
https://github.com/cockpit-project/cockpit-podman/blob/c53c2c9c5c7d8dd834994a9c7db4d7d09391ccdc/src/ImageRunModal.jsx#L220
What is noticeable here is that the schema of the data that podman-cockpit uses does not match that of the official API and that in podman-rs.
https://docs.podman.io/en/latest/_static/api.html?version=v4.2#tag/containers/operation/ContainerCreateLibpod
However, when I switch back to podman-api-rs upstream, I have problems creating the volumes.
.mounts(
                imp.volumes
                    .borrow()
                    .to_owned()
                    .to_typed_list_model::<model::Volume>()
                    .into_iter()
                    .map(|volume| podman::models::Mount {
                        bind_options: None,
                        consistency: None,
                        read_only: Some(volume.writable()),
                        source: Some(volume.host_path()),
                        target: Some(volume.container_path()),
                        tmpfs_options: None,
                        type_: Some("bind".to_owned()),
                        volume_options: None,
                        // Some({
                        //     let selinux = volume.selinux().to_string();
                        //     if !selinux.is_empty() {
                        //         options.push(selinux)
                        //     }

                        //     options
                        // }),
                    }),
            )

I'm quite clueless. :-D
Would you have any idea how to correctly use mounts option? Is the documentation of the podman API wrong, or do I use it in the wrong way?

Hey,
It seems that the documentation here is wrong and the actual type currently doesn't get generated. https://github.com/containers/podman/issues/13717
We'll probably have to add the new type by hand (as you have added in pods) for now and override the parameter used in ContainerCreateOptsBuilder.
Here is the full type to be added: https://github.com/opencontainers/runtime-spec/blob/main/specs-go/config.go#L110-L120


I'd like to contribute a PDF icon

I'd like to contribute an icon for PDF file types. I found the code where you're switching based on file extension, but I don't know how the icon(...) function converts that to an actual icon graphic. Is there a VSTS-provided library that implements that?
If you can point me in the right direction I'd be happy to submit a PR.
Thanks!

I was getting my icons from https://developer.microsoft.com/en-us/fabric#/styles/icons
Adding their pdf icon.

Thank you!!


Generate all bool fields in structs as pointers to allow sending false

The currently generated code creates structs with value type fields and the omitempty directive in the JSON tag. There is discussion around this as it relates to fields that have a default of true https://github.com/google/google-api-go-client/issues/54, namely that they can never be changed. However, the issue is really more broad, since omitempty is omitting that field if the value is the field type's zero value. That means for any bool field you can set it to true, but you can never set it to false through the API, since the serialization will always omit that. This is really an issue for all fields, i.e. you could never set a string to "", but its especially poignant for bools, as you're prevented from half of the possible values to set. The specific example we currently have is wanting to unsuspend a user, which the API supports with suspend=false passed along, but there's no way for the Go API client to generate that body.
Long term, this generated code probably needs to behave more like gRPC where all fields are pointers so they can be zeroed or nillified, and some helper methods are provided like GetSuspended, that will default to zero value when nil.

Like Go, we don't use Github for code review. We use Gerrit instead. We only use Github as a mirror and for its issue tracker.
See https://golang.org/doc/contribute.html except our upstream is https://code.googlesource.com/google-api-go-client/ (so replace "go.googlesource" with "code.googlesource" (or "go-review" with "code-review") and "google-api-go-client" instead of "go").
We should probably have more specific docs for how to contribute to this project but for a long time it was only me, and then recently @gmlewis and @mcgreevy.
Glenn, you want to add a CONTRIBUTING.md to this repo with tips?
In the meantime, closing this PR because we don't use PRs. We have to review and submit this upstream. Could you move it there?
/cc @gmlewis @mcgreevy


feat(gw-tools)!: enhance several commands

Deploy scripts with chained txs, this is much faster than deploying them one-by-one.

Don't use ckb-cli. It often has breaking changes. Just use our TransactionSkeleton and Wallet. I have also added various helper functions to make this easier.

Enhance generate-config so that kicker doesn't need to modify or read generated config files with sed/awk. See https://github.com/godwokenrises/godwoken-kicker/pull/345.

Removed unused ckb-sdk code.

(For reviewers: review from the second commit, the first one is the same as #940)

#940 is merged, please rebase this PR


Additional questions which need support

how often does replication happen? is it every 15 mins to hit the stated RPO? or is it more frequent (e.g. as changes happen), but guaranteed to satisfy the 15 min RPO?
do customers set up the second FHIR instance themselves in advance of the support ticket call? or does support do some of that on the back-end? or does support just walk them through that?
do customers set up Traffic Manager themselves in advance of the support ticket call? or does support do some of that on the back-end? or does support just walk them through that?
does the support team initiate the failback on their own? or does the customer initiate that?
if we are using something like FHIR Proxy, do we have to manually or somehow auto-detect FHIR failures to initiate redirect?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 65f2ac8e-5de7-3206-9d69-434b6e65d247
Version Independent ID: edd3ff01-892b-a965-9a54-9377db51305f
Content: Disaster recovery for Azure API for FHIR
Content Source: articles/healthcare-apis/azure-api-for-fhir/disaster-recovery.md
Service: healthcare-apis
Sub-service: fhir
GitHub Login: @SteveWohl
Microsoft Alias: zxue

Hi @ladydanes Thanks for reaching out, We will Investigate and update you shortly.

Thanks @SteveWohl
@ladydanes Here is the response to your query.

Data replication happens behind the scenes continuously once the DR feature is enabled. As documented, our RPO is 15 minutes
As documented, to enable the DR feature, all the customers need to do is submit a support ticket. Our engineering support team then enables the DR feature. It is an internal operation for the managed service and there is no walk-through on how the DR is configured with the customers. Of course, customers will be notified when the service is recovered and resumed.
No further action is required from your side. When the service is resumed, then you can use it without any configuration change. Yes, the settings and configurations are managed by the support engineering team.
As documented, compute failback happens automatically, but database failback is manually triggered by the engineering support team once the customer confirms the failback request.
The DR doc does not address FHIR Proxy which is an OSS implementation. No redirect is required because the service endpoint remains the same after the service is restored in the DR region

Hi @ladydanes We will proceed to close this issue now.
If there are further questions regarding this, please comment and we will gladly continue the discussion.


Add Sniffnet to Networking Applications

I saw this awesome list in the suggested repositories and thought that Sniffnet is a nice addition to the collection of networking resources!

I've also fixed the alphabetic order of the Networking Application section

Thank you!


close #2636: added json schemas for Aspire.Hosting.AppHost and Aspire.Hosting.Azure

Description
fixed an issues with intellisense inside appsettings.json when using Aspire.Hosting.AppHost or Aspire.Hosting.Azure
Changes:

added AspireAppHostConfiguration.json file and included it into Aspire.Host.AppHosting nuget package
added src/Aspire.Hosting.Azure/AspireAzureConfigurationSchema.json and included it into Aspire.Host.Azure package

This will close #2636
Microsoft Reviewers: Open in CodeFlow

@eerhardt  thank you for the feedback. Did the changes you asked for


Pull data directly fromSRA

Given that most of the raw data for this pipeline will be available in publicly available databases such as SRA it would be nice if we can implement functionality to download and process this automatically if SRA ids are provided. I generally recommend that the files are downloaded beforehand to avoid the pipeline crashing during the download. However, I think there may be a good enough case to automate this here or to write a separate script that downloads and checks the data for users before they plug it into the pipeline. I am not fully aware of the complexities when using fastq-dump, however maybe we can discuss this here?
Nextflow also has a fromSRA() function that we could use?

Here is a hacky way to get the list of SRA run IDs from https://www.ncbi.nlm.nih.gov/genbank/sars-cov-2-seqs/
lynx \
  -dump \
  -nonumbers \
  -listonly \
  https://www.ncbi.nlm.nih.gov/genbank/sars-cov-2-seqs/ |\
  grep 'https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR' |\
  cut -d '=' -f 2

I have put a restriction in the samplesheet validation script that these ids have to begin with SRR:
https://github.com/nf-core/viralrecon/blob/c38ba7b8df16f5fd709d502fa38143f0f2e8f86d/bin/check_samplesheet.py#L70
Depending on how we decide to implement this feature this will have to be extended to download using SRP (SRA study accession) and SRX (SRA experiment accession) to get multiple fastq files too?
Is there a way we can check that the corresponding samples were actually sequenced on an Illumina machine because the pipeline is'nt currently setup to deal with other platforms.
Given the various ways in which data can be uploaded and extracted from the SRA, I have also been thinking whether we need to have a separate process to validate the fastq files after they have been downloaded. Better that the pipeline fails sooner rather than later if there are any issues. This tool looks promising and is available on Bioconda:
https://github.com/nunofonseca/fastq_utils#fastq_info---validates-and-collects-information-from-single-or-paired-fastq-files

By the moment we can just go for the SRR, in the case that we want to add the SRA and the SRX in the future we can always read the SRA IDs from the study or the experiment and return the SRR IDs in a way that the functionality we are implementing can deal with them.
As we discussed to create a separate process to download these files, I think it will make sense to validate the files within the same process.

From the line tagged on https://github.com/JoseEspinosa/viralrecon/blob/fba94cfd88625c7052afe5e275970ad5d33ca2be/main.nf#L328 there is a possible implementation of how to download files fastq files, it needs to be cleaned and merged with the current version of the pipeline. Also, some code could be nicely put in a function. In any case, the current approach logic is:

Check the files that are given just by ID in the samplesheet (i.e fastq files that should be downloaded).

Download these files using nextflow fromSRA when possible.

Validate files coming from fromSRA and check which files are missing since it seems that the EBI server it is not completely updated with covid-19 data.

Retry the download of missing or not valid files using parallel-fastq-dump

Validate downloaded files by parallel-fastq-dump

If some file is not valid or not reachable the pipeline stops and throws an error.

If all files are downloaded the channels ("*fromSRA*" and "*fastqdump*") are mixed with the channels generated with the files provided directly by the user.

*Note: The current version of parallel-fastq-dump command subsamples only some reads from the fastq files for testing purposes (-X option)

Code updated to separate prefetch  and parallel-fastq-dump, this way if the latter fails the downloaded data is not lost.
Current version can be found here:
https://github.com/JoseEspinosa/viralrecon/blob/62e438282b919f1e26854766d42e4bfa274f9785/main.nf#L426-L620

I discovered that fromSRA is not faulty as I reported, actually I open this issue on the nextflow repo.  The problem was that if the pipeline was manually interrupted while a file it is being staged by nextflow, as the file, although truncated, exists on $NXF_WORK/stage, nextflow does not try to obtain the file again. I found it thanks to the kraken2 processes since I interrupted the pipeline manually and then nxf was complaining about the DB tar.gz file being truncated.
In summary, fromSRA will not work only if the file is not yet in the EBI server, otherwise it works fine. Anyhow, I think is nice to validate staged files at the beginning of the pipeline.

xref: https://github.com/nf-core/viralrecon/pull/50


Skip test_explain_with_binds when PREPARED_STATEMENTS=false

This version of explain is for Arel < 9 and cannot work w/o prepared
statements. The new-style Arel with bind values included in the AST
works fine. So just skip the test.

@dr-itz our current strategy with per branch releases is if a test is not applicable for the branch due to age we should just delete it.  So you can change this to remove that test.

The test is OK for AR 5.2, but only with prepared statements...plain SQL string and array of binds. That can only work with prepared statements, otherwise the bind values are lost in exec_query...
(And I just fixed that test in #904 (almost that is), just never tested with PREPARED_STATEMENTS=false 😉 )

@dr-itz Thanks for the explanation!


Compilation of RPostgreSQL fails for heroku-16

Hi 👋
I'm currently trying to get deploy a shiny application to heroku. Unfortunately, when using the heroku-16 stack together with the corresponding R-buildpack-branch heroku-16 I was not able to compile RPostgreSQL. The compilation process always halts with the following error message:
In file included from RS-PQescape.c:7:0:
RS-PostgreSQL.h:23:26: fatal error: libpq-fe.h: No such file or directory
compilation terminated.
/usr/lib/R/etc/Makeconf:159: recipe for target 'RS-PQescape.o' failed
make: *** [RS-PQescape.o] Error 1
ERROR: compilation failed for package 'RPostgreSQL'

Some research pointed me towards (maybe) missing dependency libpq-dev, as discussed within this StackOverflow question. Another answer reported a different location of libpq-fe.h, namely in /usr/include/postgresql/.
I tried compiling versions 0.6-2, 0.5-1 and 0.4-1 of RPostgreSQL - all with the same error.
Do you have an idea what I could try? I'm not sure if it makes sense to open an issue here - please apologize if this is the wrong place for reporting this.

Fortunately, @mgcdanny provided proper solution for this problem - see #103. Thanks a lot!

Just a note on this; you don't need to add the https://github.com/heroku/heroku-buildpack-apt buildpack when using the heroku-16 branch, since this buildpack incorporates the Aptfile functionality to install additional binary dependencies.
This is documented in the readme.


fix(pdk.service): corrects the error message in pdk.service.set_retries

Summary
The message thrown from set_retries is an miswritten one in case that the argument is out of range.
Checklist

[ ] The Pull Request has tests
[x] A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
[ ] There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference
FTI-6222

A similar PR has been already created #13605, switch this to draft


Who is at Google I/O for a chat about j2objc?

Hello guys,
Are there anyone on Google I/O to meet in person who can share experience about building apps with j2objc?
Regards

I'm not around, since I work out of Google's office in Kitchener, Ontario.
In the future, please use our j2objc-discuss mailing list for this sort of discussion instead of filing an issue.

Sure, thanks


[$250] Chat - Error message is received when attaching files with extension in caps

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Access staging.new.expensify.com
Sign into a valid account
Open any chat
Attempt to add any attachment through "+" or "Drag & Drop" with the type in CAPS (For example: .PDF or .PNG)

Expected Result:
User expects the attachment to be added with no issues
Actual Result:
The user receives an error message if the attachment type is in CAPS
Workaround:
Unknown
Platform:

Where is this issue occurring?

Web
Desktop App

Version Number:  1.1.94-4
Reproducible in staging?: Y
Reproducible in production?: Y
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:
https://user-images.githubusercontent.com/43996225/187495523-b24a409f-2aa0-4aba-aedf-ea826b6a2136.mp4
Upwork URL https://www.upwork.com/jobs/~01d1500d5c8671f017
Issue reported by: Applause internal team
Slack conversation:
View all open jobs on GitHub

PROPOSAL
I was able to reproduce the issue and fix it.
It is a simple fix. Just convert the extension string to lowercase using .toLowerCase() method when the file extension is separated from the complete file string.
       const fileExtension = splitFileName.pop().toLowerCase();
Thankyou

Woof, nice catch. Job is on Upwork here: https://www.upwork.com/jobs/~01d1500d5c8671f017

Hi. Who should be applying to this since my solution is also relevant?

Hi @theTrozen77, We're going to go with @varshamb's proposal here (and they will be implementing it). While your proposal came first and they are similar, I think that @varshamb's is more accurate in this case which is typically how we judge the "best" solution.

Not overdue, payment due tomorrow.
@varshamb @parasharrajat - can you both accept the offer please?

@trjExpensify Accepted.

Settled up with you both. Thanks! 🎉


Added the launch_template_custom_user_data variable to the metaflow-computation

Added the launch_template_custom_user_data variable to the metaflow-computation submodule.
Addresses issue #99

I wonder if the variable should expect raw string instead? and we can then use terraform's base64encode. function: https://developer.hashicorp.com/terraform/language/functions/base64encode
so, something along the lines of:
variable "launch_template_custom_user_data" {
  type        = string
  description = "Custom user data script for the launch template provided as a string."
  nullable    = true
  default     = null
}

and
resource "aws_launch_template" "cpu" {
    # ... other configuration ...

    user_data = var.launch_template_custom_user_data != null ? base64encode(var.launch_template_custom_user_data) : null
}

One can then supply the variable as:
  launch_template_custom_user_data = <<-EOF
    #!/bin/bash
    echo "Hello, World!"
    yum update -y
  EOF

If that makes for a better user experience I think that would be a good change. Will implement and submit a new version after ReInvent.


Update broken link to Azure Marketplace

@jeanrobertjs : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


chore(deps): lock file maintenance

This PR contains the following updates:

Update
Change

lockFileMaintenance
All locks refreshed

🔧 This Pull Request updates lock files to use the latest dependency versions.

Renovate configuration
📅 Schedule: "before 1am on Monday" in timezone Europe/Vienna.
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

⚠ Artifact update problem
Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.
♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

any of the package files in this branch needs updating, or
the branch becomes conflicted, or
you check the rebase/retry checkbox if found above, or
you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:
File name: packages/cli/yarn.lock
undefined


Add Derek to two new organizations (vielmetti, tinkerbell)

PR to follow.

Thanks @vielmetti, I wasn't sure what was happening so I have added both in via https://github.com/alexellis/derek/commit/83eea02affc6b5f34c2676d9b93864f0c28e5702
Hope I haven't stepped on your toes and that the spelling is right.

Thanks!

Please see our OpenFaaS example as a starting-point for a DEREK file https://github.com/openfaas/faas/blob/master/.DEREK.yml
Or Inlets - https://github.com/inlets/inlets/blob/master/.DEREK.yml


[Migration] Move 'UM_HYBRID_GROUP_ROLE' table to migration client

When migrating to IS 5.11.0, 'UM_HYBRID_GROUP_ROLE' table should be added for the 'Groups and Roles Migration' before executing the migration client. As it affects the user experience, we need to get rid of this additional step.
The approach discussed:

Make addAdmin property 'false' and start the migration process.
From the migration client create and assign the relevant admin role to the admin user and the admin group (If it is not already assigned)

We need to update the documentation with following
Under 'Groups and Roles Migration' of [1]
remove 'To enable this feature, execute the relevant DB script given below before starting the migration process' and the DB scripts
Before the migration, we need to make 'create_admin_account = false' in deployement.toml
[1] https://is.docs.wso2.com/en/5.11.0/setup/migrating-preparing-for-migration/

Closing this as it has already been resolved.


Login with AZ failed: ValueError: Unable to get authority configuration for https://login.microsoftonline.com/94b69cc2-950a-42b3-8ccb-d42b78e2038b

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az login
Errors:
The command failed with an unexpected error. Here is the traceback:
Unable to get authority configuration for https://login.microsoftonline.com/94b69cc2-950a-42b3-8ccb-d42b78e2038b. Authority would typically be in a format of https://login.microsoftonline.com/your_tenant_name
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/authority.py", line 83, in __init__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/authority.py", line 151, in tenant_discovery
ValueError: OIDC Discovery endpoint rejects our request

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 231, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 658, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 721, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 692, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 328, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/profile/custom.py", line 149, in login
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_profile.py", line 177, in login
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_profile.py", line 765, in find_using_common_tenant
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/auth/identity.py", line 209, in get_user_credential
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/auth/msal_authentication.py", line 39, in __init__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/application.py", line 1466, in __init__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/application.py", line 438, in __init__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/authority.py", line 87, in __init__
ValueError: Unable to get authority configuration for https://login.microsoftonline.com/94b69cc2-950a-42b3-8ccb-d42b78e2038b. Authority would typically be in a format of https://login.microsoftonline.com/your_tenant_name

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az login

Expected Behavior
Environment Summary
Windows-10-10.0.22000-SP0
Python 3.8.9
Installer: MSI

azure-cli 2.31.0

Additional Context

Could please check #20507?


Remove usage of Decimal objects

The issue with the use of Decimal objects to avoid rounding errors is that most people (me included) were not very confident in its usage which made cantools (IMO) disproportionately harder to modify and it lead to lurking bugs as well as maintenance and performance issues if the decimal number objects were used for computations instead of the "normal"  ones.
The most difficult drawback of this patch is that DBC files cannot be compared byte-for-byte anymore because rounding errors can be introduced during a load-store round-trip. What we do instead is ensure that "round tripped" file is "close enough" to the original one. Besides getting around the issue that the two database files might not be 100% identical due to rounding errors, this approach also allows to compare databases stemming from different formats (with minor modifications to avoid comparing the file format specific information).
Unfortunately, with this patch the output of the C code generator changes slightly, causing some code churn...
Andreas Lauser<EMAIL_ADDRESS>on behalf of MBition GmbH.
Provider Information

@juleq: I would like you to have a look at this because I believe that you have some skin in that game...

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

52 of 52 (100.0%)  changed or added relevant lines in 4 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.03%) to 94.022%

Files with Coverage Reduction
New Missed Lines
%

cantools/database/utils.py
1
91.3%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
6999

Relevant Lines:
7444

💛  - Coveralls

@andlaus Fine for me. If it eases maintenance, we should go for it.

Maybe convert raw_inital to int here?
Like this:
...
if signal.initial:
    signals_init_body += INIT_SIGNAL_BODY_TEMPLATE_FMT.format(signal_initial=int(signal.raw_initial),
                                                              signal_name=signal.snake_name)
...

In this case there will be no floating point values in initialization functions

Maybe convert raw_inital to int here?
[...]
In this case there will be no floating point values in initialization functions

the problem with that is that .raw_initial is not always an integer. (only if signal.is_float is False.)

Maybe convert raw_inital to int here?
[...]
In this case there will be no floating point values in initialization functions

the problem with that is that .raw_initial is not always an integer. (only if signal.is_float is False.)

So maybe like this?
if signal.initial:
    signals_init_body += INIT_SIGNAL_BODY_TEMPLATE_FMT.format(signal_initial=signal.raw_initial if signal.is_float else int(signal.raw_initial),
                                                              signal_name=signal.snake_name)

So maybe like this?
[...]

If I understood this correctly, that code should be a no-op...

Also, old _format_decimal() method could be converted to smth like _format_value() and placed into the scope
of Signal class in c_source to produce accurate output

as long as nothing too fancy is needed to format the output (AFAICS this currently is round(value, 5) and round(value)), and as long as there are only a handful of places where this is needed, I prefer to do this in-place because it is easier to see what exactly happens...

The most difficult drawback of this patch is that DBC files cannot be compared byte-for-byte anymore because rounding errors can be introduced during a load-store round-trip. What we do instead is ensure that "round tripped" file is "close enough" to the original one. Besides getting around the issue that the two database files might not be 100% identical due to rounding errors, this approach also allows to compare databases stemming from different formats (with minor modifications to avoid comparing the file format specific information).

Hi,
Please can this be explained and maybe an example?
Surely if you load a DBC file and then write it back to disk it should be 100% identical value wise?
I don't see how the use of decimals for scaling etc can be allowed to vary between reading and writing? Those values are important and this could affect the encoded and decoded signal value.
Apologies if I have the wrong end of the stick.
Thank you
Guy

Surely if you load a DBC file and then write it back to disk it should be 100% identical value wise?

yes, modulo rounding errors (mainly stemming from the fact that you have to convert float to/from strings using decimal representation). i.e. you will "only" get about 16 decimal digits of precision...

alright, I've rebased this on top of the latest master version. @zariiii9003: I'd like to get this merged and do a 39.0 release directly thereafter...

alright, I've rebased this on top of the latest master version. @zariiii9003: I'd like to get this merged and do a 39.0 release directly thereafter...

i'm still not 100% convinced, that we should remove it.... 😬
The lossless roundtrip is nice and some users might rely on it, who knows? Maybe it's enough to make the decimal attribute of the Signal class private?

Coming from a less familiar contributor - I like the fact that cantools generally doesn't mess with the file. Most of my issues with Decimal came from there being no type hints or obvious naming to identify something like a signal's value as being anything other than a float.
Could the same goal be reached by type annotating Decimal where it appears?

I like the fact that cantools generally doesn't mess with the file.

Be aware that if the file in question was not dumped by cantools, it almost certainly does mess with it. Also, as far as I know, it currently only works well with DBC files, while for other file formats significantly fewer efforts where made in this regard; For ARXML (I currently have a private PoC branch for write support) this would even be next to impossible AFAICS...

Most of my issues with Decimal came from there being no type hints or obvious naming to identify something like a signal's value as being anything other than a float.

As far as I can see, this is a typical experience which leads to quite a few issues, even for seasoned contributors.
@zariiii9003

The lossless roundtrip is nice and some users might rely on it, who knows? Maybe it's enough to make the decimal attribute of the Signal class private?

Yeah. Maybe a lot of the confusion stems from the fact that this is not marked as private. Another issue that personally confuses me a lot is that the class for storing these objects in Signal is called Decimal (defined in signal.py). IMO, the least which we should do is to get rid of that class and store what it contains separately as standard decimal.Decimal objects  (a la signal._decimal_scale, signal._decimal_offset, etc.)
That said, do you think that roundtripping is a very useful feature outside of unit tests? I think it would be much more useful to have lossless rountrip across file formats like:
dbc_str = open('my_file.dbc').read()
orig_db = cantools.database.load_string(dbc_str, 'dbc')
sym_str = orig_db.as_sym_string()
sym_db = cantools.database.load_string(sym_str, 'sym')
dbc2_str = orig_db.as_dbc_string()
assert dbc_str == dbc2_str

(it currently might work in some cases, but I cannot see a way to guarantee this.)
for people who need to compare databases, we could promote the objects_similar() function of this patch to the Database class, so that databases can be compared using something like
db.is_similar(db2, tolerance=1e-8)

okay: I've rebased this on the current master and also moved the "fuzzy compare" function to the Database (db1.is_similar(db2)) class and added a parameter that allows it to not include format specific information, i.e., databases stemming from different file formats can now be compared.
@zariiii9003: Do you think this can be merged soon? (If not, I'd like to tag a release without it this week...)

@andlaus let's start with a pre-release (e.g. beta or release candidate) to reduce the amount of new issues and complaints

@andlaus let's start with a pre-release (e.g. beta or release candidate) to reduce the amount of new issues and complaints

I agree, but how to do this? as far as I know the release process currently works by adding a tag in git and the rest is done via github actions (including the upload to pip). IOW, how can we tell pip that the new release is not supposed to be the default one?
thanks for the approval, BTW...

I think it only depends on the version number. "39.0.0rc0" should work.

okay, let's try this...

done: https://github.com/cantools/cantools/releases/tag/39.0.0-rc1 . Let's see how it works out...

The only major change I'm seeing in serialisation is all my GenSigStartValue (which is a FLOAT) defs are now being written out as 0.0 instead of 0. I quite prefer this, as it makes it clear that the values are floating point. It's all looking good, thanks for your work!


Add deploy script to replace ansible playbook

Not yet tested in production.
To test locally override the parent directory. Optionally pass a version from https://github.com/ome/www.openmicroscopy.org/releases, default latest
./deploy.py --parent /tmp/my/test/directory --version 2020.10.01-1
./deploy.py --parent /tmp/my/test/directory
...

If this works as intended deploy.py could be run in a cronjob to automatically deploy the latest tag.

@manics : I pushed https://github.com/ome/www.openmicroscopy.org/pull/426/commits/d3c6e74360a923d96341ae8c3d2af51f7420d437 to allow testing.
For everyone else, cf:
Would download /var/www/www.openmicroscopy.org/2020.11.26
Would remove symlink: /var/www/www.openmicroscopy.org/html (target=/var/www/www.openmicroscopy.org/2020.10.07/)
Would symlink /var/www/www.openmicroscopy.org/2020.11.26 to /var/www/www.openmicroscopy.org/html

Then with --force,
[jamoore@ome-www www]$ sudo _deploy/deploy.py  -f
Extracted https://github.com/ome/www.openmicroscopy.org/releases/download/2020.11.26/www.openmicroscopy.org.tar.gz to /var/www/www.openmicroscopy.org/2020.11.26
/var/www/www.openmicroscopy.org/2020.10.07/
Symlinked /var/www/www.openmicroscopy.org/2020.11.26 to /var/www/www.openmicroscopy.org/html

https://www.openmicroscopy.org/events/ is updated :+1: cc: @pwalczysko
NB:
[jamoore@ome-www www]$ du -sh /var/www/www.openmicroscopy.org/
1.9G	/var/www/www.openmicroscopy.org/

A few additional comments:

if absolute minimal downtime was a requirement, the same symlink swap strategy as the one used in https://github.com/ome/scc/blob/master/scc/deploy.py could be implemented.
slightly confused about why this script is in this repository, it seems that you need to clone/download www, run it and then re-download another copy. Should this script and the optional cron configuration be managed via https://github.com/ome/prod-playbooks/blob/master/www/playbook.yml eventually?
is this aim to completely replacing the deploy_archive step in https://github.com/ome/prod-playbooks/blob/master/www/www-static.yml#L12. Main concern about the dual deployment strategy is that playbook could potentially override the website updated by this script?

slightly confused about why this script is in this repository,

I had a similar thought. I could certainly see this logic living somewhere general even at the cost of needing to specify:
(scc deploygithub-release-deployer) ome/www.openmicroscopy.org /var/www/www.openmicroscopy.org/html

Note: there are several "github-release-deployer"-style tools on GitHub.

There's no single best place for this script so I opened it here since this is what it's related to and it seems a good place for the discussion. If someone wanta to revive the scc deploy behaviour I can close this, or this script could be moved to prod-playbooks, or a standalone repo?

I understand that the target server (the production ome-www server in this case) is hard-coded in the script ?
Even with a --dry-run option added, I still would think we should clarify:
What is the reason for dropping the step of deploying first on ome-www-dev ?
When something goes wrong in the sense that the release artifact itself, although deploying nicely (and thus --dry-run would pass) is somehow not sound, the first place we would find that out (if the ome-www-dev step is skipped) would be the direct live production op...py.org site ?

What is the purpose of the ``dry-run ? What warning should it give me ? See below the output with and without dry-run
(env562<EMAIL_ADDRESS>python deploy.py --parent /tmp/my/test/directory --version 2020.10.01-1 --dry-run
Would download /tmp/my/test/directory/2020.10.01-1
Would symlink /tmp/my/test/directory/2020.10.01-1 to /tmp/my/test/directory/html
(env562<EMAIL_ADDRESS>python deploy.py --parent /tmp/my/test/directory --version 2020.10.01-1
Would download /tmp/my/test/directory/2020.10.01-1
Would symlink /tmp/my/test/directory/2020.10.01-1 to /tmp/my/test/directory/html
(env562<EMAIL_ADDRESS>

Considering the output in https://github.com/ome/www.openmicroscopy.org/pull/426#issuecomment-739857592 and the remark in https://github.com/ome/www.openmicroscopy.org/pull/426#issuecomment-739854299 I have the feeling that the script has not been tested and is not really testable (unless used in an earnest, wet run) as things stand...

What is the purpose of the dry-run ? What warning should it give me ? See below the output with and without dry-run

That's one of the reasons I suggested not having both --dry-run and --force in https://github.com/ome/www.openmicroscopy.org/pull/426#discussion_r531101533 since it causes confusion like this- the default is --dry-run.

Tested the script on ome-www-dev
[pwalczysko@ome-www-dev _deploy]$ sudo python ./deploy.py --version 2020.10.01-1 -f
/var/www/www.openmicroscopy.org/2020.10.01-1 already exists, not downloading
/var/www/www.openmicroscopy.org/2020.12.07/
Symlinked /var/www/www.openmicroscopy.org/2020.10.01-1 to /var/www/www.openmicroscopy.org/html
[pwalczysko@ome-www-dev _deploy]$ sudo python ./deploy.py -f
/var/www/www.openmicroscopy.org/2020.12.07 already exists, not downloading
/var/www/www.openmicroscopy.org/2020.10.01-1
Symlinked /var/www/www.openmicroscopy.org/2020.12.07 to /var/www/www.openmicroscopy.org/html
[pwalczysko@ome-www-dev _deploy]$

It worked.

:+1: Thanks, @pwalczysko. I hadn't thought about it, but @sbesson pointed out that the primary downside of using the script as it stands is that running the playbook will revert those changes. i.e. we will need to make some decisions and changes before moving forward with this.

Moved to https://github.com/ome/prod-playbooks/pull/299 (thanks @sbesson)