[BUG]: Error Thrown at Agent Install:  ERR  VisualStudioServices] POST request to https://On-Premise-Server/_apis/CustomerIntelligence/Events is not authorized. Details: TF400813: The user is not authorized to access this resource.

What happened?
Specific Details from Microsoft Ticket:   TrackingID#2310180060004183
Installing an agent with the following command here:
D:\A5\agent\config.cmd --url https://On-Premise-Server/ --unattended --auth Integrated --pool Default --agent 'A5 on BuildMachine as TfsBuildSvc' --work 'D:\A5\Work' --gituseschannel --replace --runasservice --windowslogonaccount 'AD Service Account Here' --windowslogonpassword '************'
When I run the command above with agent version 2.217.2 everything works as expected.
However, when I run the EXACT same command above with agent version 3.230.2 I receive the following error in the diagnostics log:
[2023-12-11 21:04:08Z ERR  VisualStudioServices] POST request to https://On-Premise-Server/_apis/CustomerIntelligence/Events is not authorized. Details: TF400813: The user is not authorized to access this resource.
[2023-12-11 21:04:08Z WARN TelemetryPublisher] Telemetry command failed: Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: TF400813: The user is not authorized to access this resource.
at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.CustomerIntelligence.WebApi.CustomerIntelligenceHttpClient.PublishEventsAsync(CustomerIntelligenceEvent[] events, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.Listener.Telemetry.TelemetryPublisher.PublishEventsAsync(IHostContext context, CustomerIntelligenceEvent ciEvent) in D:\a\1\s\src\Agent.Listener\Telemetry\TelemetryPublisher.cs:line 95
at Microsoft.VisualStudio.Services.Agent.Listener.Telemetry.TelemetryPublisher.PublishEvent(IHostContext context, Command command) in D:\a\1\s\src\Agent.Listener\Telemetry\TelemetryPublisher.cs:line 83
Versions
The build machine where we are trying to install the agent is running Windows 10 19045.3693.
Again, agent version 2.217.2 works FINE and does NOT throw the error, but version 3.230.2 DOES throw the error.
Environment type (Please select at least one enviroment where you face this issue)

[X] Self-Hosted
[ ] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type
Azure DevOps Server (Please specify exact version in the textbox below)
Azure DevOps Server Version (if applicable)
Azure DevOps Server 2022.0.1
Operation system
Windows Server 2022 Standard
Version controll system
No response
Relevant log output
[2023-12-11 21:04:08Z ERR  VisualStudioServices] POST request to https://On-Premise-Server/_apis/CustomerIntelligence/Events is not authorized. Details: TF400813: The user is not authorized to access this resource.
[2023-12-11 21:04:08Z WARN TelemetryPublisher] Telemetry command failed: Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: TF400813: The user is not authorized to access this resource.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.CustomerIntelligence.WebApi.CustomerIntelligenceHttpClient.PublishEventsAsync(CustomerIntelligenceEvent[] events, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Listener.Telemetry.TelemetryPublisher.PublishEventsAsync(IHostContext context, CustomerIntelligenceEvent ciEvent) in D:\a\1\s\src\Agent.Listener\Telemetry\TelemetryPublisher.cs:line 95
   at Microsoft.VisualStudio.Services.Agent.Listener.Telemetry.TelemetryPublisher.PublishEvent(IHostContext context, Command command) in D:\a\1\s\src\Agent.Listener\Telemetry\TelemetryPublisher.cs:line 83

Dmitri,
Are you saying that the ROOT CAUSE of my reported bug is the same as what is logged in bug 4562?     I read the bug report for 4562 and these issues seem completely different to me.    Yes, we were reporting an issue with the same Agent version, but that is the only difference I can see.
In my case the agent installs properly, and is able to successfully run a pipeline with no issues.   The only issue is the reported error when the agent is installed.    In bug #4562 it appears that his pipeline gets stuck, which is NOT my issue.
I don’t believe that these are the same thing.
Dean
From: Dmitrii Bobreshev @.>
Sent: Tuesday, December 12, 2023 2:55 AM
To: microsoft/azure-pipelines-agent @.>
Cc: Dean Haase @.>; Mention @.>
Subject: [EXTERNAL] Re: [microsoft/azure-pipelines-agent] [BUG]: Error Thrown at Agent Install: ERR VisualStudioServices] POST request to https://On-Premise-Server/_apis/CustomerIntelligence/Events is not authorized. Details: TF400813: The user is not authori...
[EXTERNAL]
Hi @dihaasehttps://github.com/dihaase, thank you for the feedback, the error is the same as in 4562https://github.com/microsoft/azure-pipelines-agent/issues/4562, closing this issue, you can track the progress in the 4562https://github.com/microsoft/azure-pipelines-agent/issues/4562
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/azure-pipelines-agent/issues/4564#issuecomment-1851561124, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVTU2VZYWYIHAFRIH5DDNETYJALWZAVCNFSM6AAAAABAQNV662VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJRGU3DCMJSGQ.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>======================================================================
The information contained in this e-mail, including all of its attachments, is
confidential and proprietary information of IPG Photonics and its affiliates and
intended only for the use of the individual or entity named above.  If the
reader of this message is not the intended recipient, or the employee or agent
responsible to deliver it to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is strictly
prohibited.
If you have received this communication in error, please immediately notify the
sender by telephone or e-mail, and destroy this communication and all copies.
Thank you.
This Email has been scanned for all viruses by IPG Photonics Email Scanning Services.

Hi @dihaase Yes, the root cause is similar, the problem was that the catch block was missing for such errors. The PR is on review now and will be delivered in a next version.
The fix for your case was handled here

@dihaase, We can reopen this issue for tracking changes if you want, in case you worry that the fix from the PR  won't apply to your case?

Dmitri,
No that is fine, I just was not connecting the dots because they seemed different, which is why I was asking about root cause.     If the fix in PR solves both cases that is fine.      Roughly, when will the fix in PR be released in a final production release to the public?
Dean
From: Dmitrii Bobreshev @.>
Sent: Tuesday, December 12, 2023 9:39 AM
To: microsoft/azure-pipelines-agent @.>
Cc: Dean Haase @.>; Mention @.>
Subject: [EXTERNAL] Re: [microsoft/azure-pipelines-agent] [BUG]: Error Thrown at Agent Install: ERR VisualStudioServices] POST request to https://On-Premise-Server/_apis/CustomerIntelligence/Events is not authorized. Details: TF400813: The user is not authori...
[EXTERNAL]
@dihaasehttps://github.com/dihaase, We can reopen this issue for tracking changes if you want, in case you worry that the fix from the PR https://github.com/microsoft/azure-pipelines-agent/pull/4568/files won't apply to your case?
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/azure-pipelines-agent/issues/4564#issuecomment-1852288013, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVTU2V2APTJEXJXUADBI44TYJB3B3AVCNFSM6AAAAABAQNV662VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSGI4DQMBRGM.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>======================================================================
The information contained in this e-mail, including all of its attachments, is
confidential and proprietary information of IPG Photonics and its affiliates and
intended only for the use of the individual or entity named above.  If the
reader of this message is not the intended recipient, or the employee or agent
responsible to deliver it to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is strictly
prohibited.
If you have received this communication in error, please immediately notify the
sender by telephone or e-mail, and destroy this communication and all copies.
Thank you.
This Email has been scanned for all viruses by IPG Photonics Email Scanning Services.

Hi @dihaase, We'll start a new version right after the PR.


chore(example): use full links in GitHub templates

Description
Use full PR links in the changelog that is generated from GitHub templates
Motivation and Context
closes #487
How Has This Been Tested?
-
Types of Changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (no code change)
[ ] Refactor (refactoring production code)
[x] Other

Checklist:

[ ] My code follows the code style of this project.
[ ] I have updated the documentation accordingly.
[ ] I have formatted the code with rustfmt.
[ ] I checked the lints with clippy.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (bd5e4d2) 41.30% compared to head (6126473) 41.30%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #503   +/-   ##
=======================================
  Coverage   41.30%   41.30%
=======================================
  Files          15       15
  Lines        1034     1034
=======================================
  Hits          427      427
  Misses        607      607

Flag
Coverage Δ

unit-tests
41.30% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


docs: fix circuit testing doc

This PR fixes typos, changes variable names, and updates the tracking_testing.md doc for example circuit testing.

https://github.com/kroma-network/tachyon/pull/487/commits/951dbd3a46567e6f14ec0bee39f5da79530a87d7 should be typed with test.


Improving modal injection method by injecting into the user panel's React component instead of using DOM

Once implemented, this will fix:

Modal disappearing after changing Discord's language (or any update to the entire React DOM in general)

Currently there is no way to get the user panel's React component to patch it on startup (the component itself is defined in a function)
The only option is to wait for Discord to fully load; then we can get the instance and patch its render function
This might end up more janky than the DOM method

Will reopen once I have time to deal with this


Split Tracking and Booking service

This is an idea to beat the control policy restriction set by Covin.
Track and book feature wouldn't be of much use since the OTP expires very soon due to control policy. I was thinking how about to do this way?
Split the features - OTP + booking and  searching slots(user validation not required) into two separate services. Search slot service would keep looking for slots every second and once it finds the slots a voice notification(with pincode) would let the user know that there is a slot found. OTP + booking service would be deployed in another server where user feeds the OTP and books using pincode. This way Covin won't be able to invalidate the OTP based on control policy.

Hi,
I have gone through the rate-limiting.
I have written a blog on it as well.
https://yash-tibrewal.medium.com/how-to-update-your-vaccination-slot-booking-app-with-the-latest-rate-limiter-70735d66c965
I have suggested some suggestions to @yashwanthm as well,
if the pollFreq = 3250 (Tested from morning today, worked flawlessly)
It will work flawlessly (not even 1 error you will face )
OTP sessions would be 15 minutes as standard.
Moreover, even the public APIs are restricted for 100 per 5 minutes per I.P.
hence changed anything other than pollFreq will do you now good.
Thanks

@NaveenKS  Manual input is where the delays are and the availability runs out.
Even if we'd do all the effort, I do not see it adding any value.

@yashtibrewal Oh okay got it.
@yashwanthm Okay.


Role base authorization behavior not work on integration test environment

I have 3 project.

IdentityServer (with aspnet identity which use the oidc feature)
WebAPI (just the api for identityserver to protect)
WebMVC (the client)

Then all authentication and authorization feature is work in debug and release.
But authorization not work(or the behavior not the same as debug and release) in integration test .
Please go on reading follow:
After I finished config the integration test project.
I start the test , Let MVC login to IdentityServer by oidc ,then mvc got the token,then let mvc serve to the api‘s action which is protect by the [Authorize(Roles = "CommonUser")] .Then It got the 403 forbidden.
This opera I test use the webbrowser is no issue.
Then I start to monitor what is different in the variable,Then I found that it is different between test and run (take the note at the field name):

So I think this is the reason I can not serve to the api use the  [Authorize(Roles = "CommonUser")] protected
Image left is the integration test ,and image right is the debug mode run in real iis express.
left's role field name is role
right's role field name is http://schemas.microsoft.com/ws/2008/06/identity/claims/role
Then I know http://schemas.microsoft.com/ws/2008/06/identity/claims/role is the ClaimTypes.Role
So I try modify the mvc project's configure in startup.cs here:
Originally the code is that
                options.TokenValidationParameters = new Microsoft.IdentityModel.Tokens.TokenValidationParameters
                {
                    NameClaimType = "name",
                    RoleClaimType = "role"
                };

and after modify is that
                options.TokenValidationParameters = new Microsoft.IdentityModel.Tokens.TokenValidationParameters
                {
                    NameClaimType=ClaimTypes.Name,
                    RoleClaimType=ClaimTypes.Role
                };

It's still no use.
Then I try modify the GetApiResources in config.cs in identityserver project
Originally is that
        public List<ApiResource> GetApiResources()
        {
            var result = new List<ApiResource>
            {
                new ApiResource ("api1","The Sample.API Project")
                {
                    UserClaims = {"role"}
                }
            };
            return result;
        }

after modify is that
        public List<ApiResource> GetApiResources()
        {
            var result = new List<ApiResource>
            {
                new ApiResource ("api1","The Sample.API Project")
                {
                    UserClaims = {"http://schemas.microsoft.com/ws/2008/06/identity/claims/role"}
                }
            };
            return result;
        }

It's still no use.

It's seem like that current version not need this code
                //JwtSecurityTokenHandler.DefaultInboundClaimTypeMap.Clear();
                //options.TokenValidationParameters = new Microsoft.IdentityModel.Tokens.TokenValidationParameters
                //{
                //    NameClaimType = "name",
                //    RoleClaimType = "role"
                //};


Flash messages should close when a new conversion starts

Expected/desired behavior
If a user converts a file and there is an issue (e.g. error, warning), a flash message appears on the screen. However, if the user converts another file, the message continues to display.
Current behavior
If the user starts to convert another file, the message should disappear.
Additional information
Because the message continued to display, some users were thinking that all of their files were showing errors.

Works! Thanks.


Question for outgoing call open dialer screen

Hi @MrHertal
I start outgoing call from my iPhone to other android or iOS device. it is not moving to native screen(dialer screen) in my phone.
so is it anyway to open dialer screen if i start outgoing call to someone?
please help me.

Hi, there seems to be no way to achieve that on iOS.
Related issue: https://github.com/MrHertal/react-native-twilio-phone/issues/20


fix: refactored data processing into its own file

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)

Closes #328

I am a member of CodeDay and this is my first contribution. I believe I followed the guidelines closely but didn't see a good label for scope in PR title so left it blank.
I refactored the dashBoard data processing into its own file in /util and then imported it into dashtable_v2.js
Co-authored-by: Komal Kaur<EMAIL_ADDRESS>

Hi Gary,
Thanks for your PR! We have seen this and are working to be able to review all of our new PRs, but typically this is done by hand (pulling your changes onto my dev environment and testing for myself) so this can take a while with so many PRs.
However, I'll be reviewing this PR on Monday (07/09/23), just wanted to give you a heads-up that we have seen it.
Thanks for your contribution and patience!

@GuillermoFloresV I was able to figure out how to write tests. It is a pretty simple one but wanted to follow best practices. In the docs it says to put component tests next to their components but not sure if my test should go in the tests dir or in the same file as my method in ./util/processDashBoardData.js.
Once I can get confirmation I'll add the test by pushing to this branch

@GuillermoFloresV I was able to figure out how to write tests. It is a pretty simple one but wanted to follow best practices. In the docs it says to put component tests next to their components but not sure if my test should go in the tests dir or in the same file as my method in ./util/processDashBoardData.js.
Once I can get confirmation I'll add the test by pushing to this branch

Hi Gary, I've left some comments on your code. Regarding your question, we could move the testing file over to the __test__ folder. Please create a sub-folder named __util__ inside of __test__ and paste your test in there, while there is no set naming/folder structure convention, we are currently using a convention where all our tests live inside of the __test__ folder. Thanks!

Hi @GuillermoFloresV I work as well have to frontload the beginning of my week with my CodeDay hours. I just got this merge conflict resolved and I believe its good to go.


Bridge will not publish accessories

Like stated in #19 the bridge is broken at the moment. It will start w/o issues but does not show up in the iOS HomeKit apps.

Is there a chance that Apple somehow blocked it? "MyTouchHome" is saying "Accessory if blocked".

Hi, my Bridge freezes at "open ws connection for url......"
No connected to Bridge possible with Eve app on iPhone.
Is that a issue of this problem? Or is there something different wrong?

Is that a issue of this problem?
yes, thats the last thing which is logged. after that the logging should show connection procedure to iOS. The bridge is not frozen, there is some strange connection problem.

Ok, thank you. Hope you are able to fix that. fingers crossed


Why are training and testing so slow

Why are training and testing so slow?

I also meet the issue. Have you found a solution? @InstantWindy

Why are training and testing so slow?

I also meet the issue. Have you found a solution?

I meet the issue too. Have you found a solution? @BlankOtis @hanquansanren


When installing under VMWare on MacBook Pro buttons are off screen bottom

While installing under VMWare on MacBook Pro - Retina, the buttons you need to click to proceed are off screen at the bottom. Makes it hard to continue.

I had this issue as well on VMware Workstation 15, but after talking to Ken, I went into advanced options for the VM and changed from BIOS to UEFI and it fixed the issue.

Same for me using Trident-18.12-RELEASE-x64-20190115.iso

Can you check that out as well?

Change to UEFI solved this for me in 19.07-20190710.iso

Closing out this issue due to the move to Void Linux: Completely different backend/installer.
Just for reference though:
Switching to EFI or UEFI boot within VMWare fixes this issue. It appears to be some kind of mismatch in the VMWare/FreeBSD resolution reporting for legacy-boot.


TF-1521: win32: address spurious exceptions during RPC

It was reported that this module was throwing spurious exceptions during network operations on certain Windows devices. After some investigation, it is revealed that this was because the KJ_NONBLOCKING_SYSCALL macro was being used to inspect the outcome of a Winsock call, which results in errno being consulted erroneously instead of WSAGetLastError(), and thus causing the "accept EWOULDBLOCK" logic to be ineffective.
To resolve this, we apply an appropriate* amount of pre-processor fun to define a new macro, FA_NONBLOCKING_SOCKCALL which does the "right thing" depending on if you're on Windows or not, resolving the bug.
(*opinions may vary)
This PR bumps the version of the capnp package to 0.5.7, and provides a new pre-compiled module for the "cross-compilation" workflow, which is the output of the code contained in this PR.
I have tested that this resolves the reported bug by manually building a client containing this node module.

Thanks 😄


Move Python formatting to a supported formatter

We currently format all Python code using yapf. This is done via the tools/yapf.sh and tools/yapf.bat wrapper scripts.
Unfortunately, yapf is abandoned and only works with Python versions up to and including Python 3.10. As OS and local Python installations are upgraded, this will become more and more problematic. On some Linux distributions side-by-side python installations are unsupported, for example.
Workarounds
Those on macOS can work around this by installing homebrew then:
brew install<EMAIL_ADDRESS>
Related

https://github.com/flutter/engine/pull/55905

yapf was only chosen because it was already used for LUCI recipes, and we already had a mirror set up for it. One thing to investigate would be whether pylint has options to also enforce formatting, which we might currently have disabled so that yapf and pylint don't fight with eachother.

It looks like the Python Software Foundation currently supports black: The uncompromising Python code formatter.
We would need to (a) import it via a mirror, (b) change our internal tooling to support it, and (c) potentially reformat the repo.
If this is something we want to do, I can look into getting started with (a).

I think we should do something about this.
Working on Linux is frustrating since the latest Ubuntu/Debian LTS releases no longer offer the option to install older python3 versions compatible with our version check.
Not to kick the can further down the road, but I was able to successfully run the check using python3.13 with the 2to3 package installed separately. It seems like we might be able to widen the version check further if we needed to, but this isn't sustainable long-term.
@matanlurey I can do a trial run later this week sometime to see how black looks locally on my machine. If it looks like it'd be a reasonable fit, I can report back then and look at wiring up the mirroring unless anyone has any objection.

That would amazing, thanks Chris!

cipd ls -r | grep black

There are some existing black cipd pacakges.


Model vector access

Implemented
_get_primal_values(), _get_constraint_values(), _get_reduced_costs() and _get_shadow_prices() and updated respective interfaces.
see
https://github.com/biosustain/optlang/issues/98

I updated the glpk vector functions where I found them, but could not find vector functions for the MIP problems.
Only thing remaining is that the rounding_to_bounds is not yet vectorized for the primals.
My local builds are passing, also AppVeyor, not sure what is going on with travis.

This is not ready. Tests not passing. But the basic structure is there. Will finish as soon as I find some more time.

Codecov Report

Merging #101 into devel will decrease coverage by 20.69%.
The diff coverage is 65%.

@@            Coverage Diff             @@
##            devel     #101      +/-   ##
==========================================
- Coverage   89.18%   68.48%   -20.7%
==========================================
  Files           8        8
  Lines        2431     2437       +6
  Branches      550      548       -2
==========================================
- Hits         2168     1669     -499
- Misses        163      678     +515
+ Partials      100       90      -10

Impacted Files
Coverage Δ

optlang/cplex_interface.py
2.15% <0%> (-88.64%)
:arrow_down:

optlang/interface.py
89.65% <100%> (-0.41%)
:arrow_down:

optlang/glpk_interface.py
86.01% <71.42%> (-0.25%)
:arrow_down:

optlang/exceptions.py
88.88% <0%> (-11.12%)
:arrow_down:

optlang/expression_parsing.py
70.83% <0%> (-8.34%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a4030fa...bc03c03. Read the comment docs.

This is ready to merge. All tests pass.
Not sure what is going on with the codecov, looks like the cplex tests are not run on travis.
This improved the FBA speed by 50% for me.
Should be the same for the cobrapy get_solution function when using the list functions instead of the OrderedDict properties.
If I can use numpy arrays I would return numpy arrays instead of lists. Otherwise good as it is.

Thanks.
On Sat, Apr 8, 2017 at 10:10 AM, Kristian Jensen<EMAIL_ADDRESS>wrote:

Merged #101 https://github.com/biosustain/optlang/pull/101.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/biosustain/optlang/pull/101#event-1035083853, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA29ulSub1dzDioXDtptmX7VDpl0hdWqks5rt0EEgaJpZM4M22-a
.

--
Matthias König
Junior Group Leader LiSym - Systems Medicine of the Liver
Humboldt-University Berlin, Institute for Theoretical Biology
https://www.livermetabolism.com
<EMAIL_ADDRESS>Tel: +49 30 20938450
Tel: +49<PHONE_NUMBER>0


Is there a reason gap_open has to be > gap_extend and not >= gap_extend?

If I try to use
let gap_penalties = Gaps { open: -1, extend: -1 };

I get an assertion error.  I understand that the way scoring works that gap-open needs to include the gap extension, since a gap of length one is scored as just the open penalty.  However, I would like to be able to do alignments where e.g. a 1bp gap costs -1 and a 2bp gap costs -2, etc. which is not possible with the current check.
Is there any reason to require that open > extend instead of open >= extend?

IIRC the reason is because it saves a couple instructions when computing tracebacks (we can make certain assumptions on the scores). I'll look into allowing open >= extend.

Ah, I see.  I would appreciate it.  It's nice to be able to create alignments where the cost per bp of a gap and the cost of a mismatch are the same.  I'm using block-aligner to align PacBio reads to other PacBio reads and my expectation is that 1-2bp indels are about as likely as mismatches.

This probably won't be implemented because I'm fairly certain adaptive-banding-like approaches don't work well when open == extend (this was measured in the original adaptive banding paper, and I think its probably the case for block aligner too). Also, it probably makes more sense to use a different approach for linear gap penalty alignments because block aligner has a lot of overhead that allows it to handle affine gap penalities.


CSV file: String value NA not imported

I tried to load some CSV data into a node table. The CSV file was very simple and contains a single column with 2-letter country codes, no header and the values are NOT quoted.
CSV sample:
MY
MZ
NA
NC
NE
The data was correctly imported except for the value: NA
` kuzu> create node table Country(code string, primary key(code));
kuzu> copy Country from "data/countries-test.csv";

5 number of tuples has been copied to table: Country.

`

Result:
` kuzu> match(c:Country) return c.code;

c.code

MY

MZ

NC

NE

(5 tuples)
Time: 9.82ms (compiling), 0.59ms (executing)
`
I assume that NA is interpreted as "not available" respectively null. If the String is quoted then the value is correctly imported.
Uwe

Hi @uwegeercken , thanks for reporting this! Yes, you're right, the reason behind this "bug" is because NA is not correctly interpreted as a non-null string value. We currently use arrow's CSV reader, and by default configures it to allow a set of "null_values" (including NA) interpreted as NULLs for strings. We're changing this soon, and will update you here soon as it's done.
Thanks!


Check for non Junit5 usage in CI

Fixes #2586
This PR adds a script that runs during CI that will fail if any new non-Junit5/Jupiter tests are added.
There are probably some improvements to be made here. I used find-unapproved-chars.sh as a reference when writing this script.

Your changes look good to me. Thanks for the improvements and review, @ctubbsii.


ros2: backport removal of viewpoint object

Account for differences between main and ros2.
Remove camera and viewpoint publishers and subscribers.

Closing in favour of force pushing to target PR.


Win release contains Trojan:Win32/Wacatac.D2!ml signatures

I tried to use the prebuild release but windows defender is detecting a trojan:
Trojan:Win32/Wacatac.D2!ml

the virus total report https://www.virustotal.com/gui/file/98d43ecf894f25fa4db5e283b9c216b3c64245cf0ce3cdbae3b9d27e5d1a7b58/detection
other miner like nbminer got properly detected as PUA miner.


DocsArchive-<IP_ADDRESS>Auto archive content

Auto archive content from https://github.com/OPS-E2E-PPE/docs-archive-test-source

Docs Build status updates of commit ae3df8f:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch Release_Archive_master_2020-01-09-03-57-57 has been deleted or has been force pushed: fatal: couldn't find remote ref refs/heads/Release_Archive_master_2020-01-09-03-57-57

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Question] Do you have any good ideas on how to clearly distinguish the front and back of a model object?

As the title suggests, I want to clearly distinguish the front and back of a model object. I want to try using OpenPose, but my model object is not a human figure; it's a piece of clothing.

Based on my experience, if the shape does not have a clear distinction between the front and back, it is difficult to create a sampling image as intended.
Therefore, it comes down to luck.

If you have any good ideas, please let me know.


Support for Bessel functions in nopython mode

Thank you for your good work on numba, this fantastic piece of software has been making my life so much easier (I code scientific tools).
It would be great to have support for Bessel functions in nopython mode. They are defined as ufunc in scipy.special. Thanks

I agree, this would certainly be a useful feature! Thanks.

See "scumba" discussion here: https://github.com/numba/numba/issues/4292

This now works if the Numba extension package numba-scipy is installed:
from numba import njit
import numpy as np
import scipy

@njit
def foo(x):
    return scipy.special.j0(x)

print(foo(0.2))

See: https://github.com/numba/numba-scipy


Consider allowing a flag to decode to more native Go types

So --go-type is wonderful to selectively convert, for example, _timestamptz=[]time.Time. However it'd be great if there could be a flag to do this wholesale, otherwise the number of flags might continually accrue.
I would be willing to look into implementing this if you would like.

Hi David,
Would you mind expanding the example? I'm not sure I follow.
As a concrete example, here's what our internal invocation looks like. How would it change?
--go-type _bool=[]bool
--go-type _date=[]time.Time
--go-type _int2=[]int16
--go-type _int4=[]int32
--go-type _int8=[]int
--go-type _float8=[]float64
--go-type bool=bool
--go-type boolean=bool
--go-type bytea=[]byte
--go-type citext=string
--go-type currency=github.com/arryved/simc/base/proto/moneypb.Currency
--go-type decimal_xl=github.com/shopspring/decimal.Decimal
--go-type float4=float32
--go-type float8=float64
--go-type int2=int16
--go-type int4=int32
--go-type int8=int
--go-type lnglat=github.com/jackc/pgtype.Point
--go-type ltree=string
--go-type numeric=github.com/shopspring/decimal.Decimal
--go-type text=string
--go-type text16=string
--go-type text64=string
--go-type text128=string
--go-type text256=string
--go-type text1k=string
--go-type text4k=string
--go-type timestamptz=time.Time
--go-type upc=string

Sorry for the vague issue. Basically I'd want a flag like --use-native-types to provide nearly all of the mappings automatically. I currently maintain a less exhaustive list and I imagine a lot of other people do too but it'd be nice to turn it all on with a single flag.


Do I need to download the entire FFHQ dataset？

Thank you for sharing such excellent work！
I am currently generating data according to your data production script. However, I noticed that the FFHQ dataset is very large. Could you please tell me which files within it are essential?

Hello,
To train the model, you need to download images from the FFHQ dataset with a size of 1024x1024 (89.1GB).
In our research, we utilized all 70,000 images, but if you prefer a smaller subset, downloading around 10,000 images should be sufficient.


Reverse the order of changelog

Changelogs usually have newest releases at the top of the file.

Whoops, thanks for the catch!


Waypoint: Comparisons with the Logical And Operator

Based on the instructions given, I don't see what's wrong with my code.
Instructions: Combine the two if statements into one statement which will return "Yes" if val is less than or equal to 50 and greater than or equal to 25. Otherwise, will return "No".
My code:
function myTest(val) {
if (val >= 25 && <= 50) {
return "Yes";
}
return "No";
}
myTest(10);
Can someone please show me the error of my ways? Thank you

found it
if (val >= 25 && <= 50) old
if (val >= 25 && VAL <= 50) new


avoid duplication in LSP document symbols

Motivation

This has come up both as a known bug in Sorbet's document symbol implementation and as a bug pointed out by people experimenting with document symbols.  The problematic case is laid out in comments in the commits for this PR.
Test plan

I don't know that our test harness is equipped to handle this case: we want multiple files, but we only want to test the output of some LSP operation on a single file.
I plan to test this on Stripe's codebase using files that people have pointed out as having this problem.
I am not 100% sure that the refactoring that retains a list of candidates is behavior-preserving, since our test harness doesn't test multi-file cases at the moment.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_M8S0GPuF3yKoZd
→ https://go/builds/bui_M8S0fym8tY6LUQ

This does seem to produce the desired deduplication on files in Stripe's codebase.

@froydnj does this PR close this issue? https://github.com/sorbet/sorbet/issues/4638


CORS Pre-flight check failing

What I'm trying to do:
Im trying to update meta_data of a product. Below is the sample code
import WooCommerceRestApi from '@woocommerce/woocommerce-rest-api';

const WC = new WooCommerceRestApi({
  url: 'http://localhost:8888/wordpress', // Your store URL
  consumerKey: <key>, // Your consumer key
  consumerSecret: <secret>, // Your consumer secret
  version: 'wc/v3', // WooCommerce WP REST API version
});

export const fetchProducts = () => new Promise((resolve, reject) => {
  const params = {
    per_page: 50,
    meta: true,
  };

  WC.get('products', params).then((response) => {
    resolve(response.data);
  }).catch((error) => {
    reject(error.response.data);
  });
});

export const updateProduct = (id, data) => new Promise((resolve, reject) => {
  // data param contains an object {meta_data: [{},{},....,{}]}
  WC.put(`products/${id}`, data).then((response) => {
    console.log(response.data);
    resolve(response.data);
  }).catch((error) => {
    console.log(error);
    reject();
  });
})

I am able to fetch products without any issues through get request. But when I try to update meta_data, I get following error.

Access to XMLHttpRequest at 'http://localhost:8888/wordpress/wp-json/wc/v3/products/816?oauth_consumer_key=1111222233334444&oauth_nonce=E3kDMWQOzCMwnmq1fFQR2QKNpETKdRYW&oauth_signature_method=HMAC-SHA256&oauth_timestamp=1566826887&oauth_version=1.0&oauth_signature=b05%2FvJVrAANDckfdPnjOpLJDmYVdtxBJNifGwYIgkHk%3D' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.

Can you please help me get through this. Regards.

CORS policy is configured on your server, or in the browser in some cases, I found this guides that may help you:

https://crunchify.com/how-to-fix-access-control-allow-origin-issue-for-your-https-enabled-wordpress-site-and-maxcdn/
https://stackoverflow.com/questions/46522749/how-to-solve-redirect-has-been-blocked-by-cors-policy-no-access-control-allow

Still something that can't be by passed on this library, so you need to fix on your end.


Update the release scripts to include Python 3.10

Update the release scripts to include Python 3.10, and drop Python 3.6. Manylinux2010 images (centos6) are now provided instead of manylinux1 (centos5) in order to support Python 3.10
(apologies to @Thrameos for missing this in https://github.com/jpype-project/jpype/pull/1061#issuecomment-1126713213, I started fixing it once I saw your comment, and then realised you'd already solved most of the problems. Here are the only 2 things that weren't addressed I think (including a release of a Python 3.10 manylinux wheel))
I have bumped from manylinux1 to manylinux2010. The former is no longer actively supported, and there is no Python 3.10 available in them. If you are concerned about dropping manylinux1 for older Python versions, we should do some more rationalisation of the release script I think.

Thanks.  I will back port this and run the release script again.

Sadly that did not work out of the box.
auditwheel: error: cannot repair "wheelhouse/JPype1-1.4.0-cp310-cp310-linux_x86_64.whl" to "manylinux1_x86_64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.

It appears that the PLAT variable also needs to be updated as well.  I made the additional update and copied the wheels to the github release.  @marscher there are two more wheels that need to be pushed.


Many standard library types not found

I just updated to the latest nightly compiler and rust-analyzer (eb8f76a86fe88fcad370577b344f397dd6118cfd). I'm using i686-pc-windows-msvc on Windows 10: rustc 1.40.0-nightly (87cbf0a54 2019-11-01)
For me currently, rust-analyzer can't find many items in std like vec![], Vec, or Option. On the other hand, it can find Range and HashMap. I thought these are supposed to work and did before I updated. If I run analysis-stats on rust-analyzer:
Total expressions: 78546
Expressions of unknown type: 21367 (27%)
Expressions of partially unknown type: 3931 (5%)

which is a lot worse than other people recently posted. Could this be related to any recent nightly changes or that I'm on Windows?
I'm not using rustup (installing msi and unpacking the src archive manually), but I think the src is in the right place since ra does find some types in std.

I did try it in my Windows machine (eb8f76a) with rustc 1.40.0-nightly (87cbf0a54 2019-11-01) :
cargo +nightly run --release -p ra_cli -- analysis-stats .

Here is the result :
Total expressions: 89603
Expressions of unknown type: 5678 (6%)
Expressions of partially unknown type: 3407 (3%)
Type mismatches: 3724
Inference: 29.0840339s, 0b allocated 0b resident
Total: 40.423245s, 0b allocated 0b resident

:thinking:
One thing to check would be to use 'go to definition' on an std type that does resolve (e.g.) HashMap and see where it takes you -- does it go to the correct installation of rust?
Apart from that, you could try creating a very small test project that references just e.g. Vec, and run ra_cli with RUST_LOG=ra_hir::nameres=debug. That will probably still produce a lot of logs, but might give a hint as to what's happening.

@edwin0cheng Thanks for trying it. It was useful to know it had nothing to do with Windows or the version of Rust.
@flodiebold Yes, HashMap and other types took me to the definition in the right file in the right version of Rust. I created a project and ran ra_cli as you suggested, and it dumped 27k lines of output. From searching for Vec in it, it seems like nameres can resolve vec::Vec in some of the standard library files, but not in the test file.
It seems it is because I'm not using Rustup. I tried Rustup in a VM and found that the source archive it downloads is different than the source archive on https://forge.rust-lang.org/infra/other-installation-methods.html.
Download page:   https://static.rust-lang.org/dist/rust-nightly-src.tar.gz
Rustup manifest: https://static.rust-lang.org/dist/2019-11-04/rust-src-nightly.tar.gz

The one on the download page seems to contain many more files and is 20MB, but the one Rustup downloads is only 3.6MB. Looking at it more, it seems like the one on the download page is from Jan 2, 2015! That's what the modified dates in the archive are, and the files have "Copyright 2014" headers. The link must have been changed to point at 2015 source in the past couple months because I still have the previous file (rustc-nightly-src.tar.gz) and it's definitely from 2019. rust-analyzer works much better with the 2019 source archive. Thanks for the help and all the work on rust-analyzer!


Can you update settings for production?

Any Chance you can update settings for production with ec2?

Any Chance of updating settings for production?

Did you look at the Readme? In particular "experimental" and "not for production".
This codebase is old and based on old versions of Django and Nuxt. I would suggest forking but honestly there are probably better approaches these days.


ex_rabbit_pool or ex_rabbitmq_pool?

Hi,
I was trying this out and noticed that the docs reference ex_rabbitmq_pool while the app that gets generated is ex_rabbit_pool.
The project name is also ex_rabbit_pool and is referenced as such from the blog post (https://www.erlang-solutions.com/blog/ex_rabbit_pool-open-source-amqp-connection-pool.html).

Hi @donaldducky thank you for point out this, really appreciate it, indeed there is an error in docs.


Error bubbling or better error documentation

Currently many (possibly most or even all) errors encountered in R do not bubble up to Python.  As a result, users get error messages with vaguely mysterious error codes (I've seen this specifically on denoise) (e.g, returned non-zero exit code -11).
I think we should do one of the following:

Provide clear documentation somewhere of the significance of the error codes

If such docs exist externally (i.e, in the actual dada2 docs), point to them somewhere from our own docs
... or,

Bubble those errors up and use a dictionary or something to convert them into useful error messages, e.g,

DADA2 returned non-zero exit code -11.  This probably means there is a problem with X, Y, or Z.  For more, see www.our_awesome_dada2_docs.biz

Agree this would be useful. Right now the only communication from R to python is happening via the exit code, and its only being used for one specific type of error (no reads passed filter).
I'll note that the R code is printing to stdout useful progress and error information. That's suppressed by the plugin though (unless verbose mode is turned on).

Sorry I haven't looked into this more, but is stderr exclusively used for errors in R and dada2? I know this isn't the case for fasttree or mafft, but we could probably just naively pass along stderr if it's clean enough. That alone could be a huge improvement.

It appears that Rscript invocations (which is how the plugin is calling the dada2 code) output print and cat statements to stdout, and message, warning and stop messages to stderr.
The message output wouldn't be desired (that doesn't indicate the kind of problem usually associated with stderr) but capturing and reporting the warning and stop messages in stderr would make sense, as they correspond to the kind of messages generally expected in stderr from shell commands.
So, I think this is a good idea that is definitely worth trying. If there are too many messages, we also can modify the R code to suppress them to keep things cleaner.

This may be a non-issue. It appears that stdout/stderr is being captured and displayed if there's an error raised in R. I chatted with @maxvonhippel and showed him the current behavior and it appears to be working as expected.
The confusion was related to the CLI capturing stdout/stderr and dumping it in a log file. For example, this is what users will see when a command fails within R:
$ qiime dada2 denoise-single --i-demultiplexed-seqs demux.qza --p-trim-left 0 --p-trunc-len 5000 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza --p-n-threads 0

Plugin error from dada2:

  Command '['run_dada_single.R', '/tmp/qiime2-archive-6jn0vrm3/b0aa4c34-
  2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmprc6rl9ev/output.tsv.biom',
  '/tmp/tmprc6rl9ev', '5000', '0', '2.0', '2', 'consensus', '1.0', '0',
  '1000000']' returned non-zero exit status 2

Debug info has been saved to /tmp/qiime2-q2cli-err-g3hwwrqf.log.

Here's what's in the debug log created by the CLI:
$ cat /tmp/qiime2-q2cli-err-g3hwwrqf.log
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/qiime2-archive-6jn0vrm3/b0aa4c34-2cfe-4d84-a04c-1185cf3d31dc/data /tmp/tmprc6rl9ev/output.tsv.biom /tmp/tmprc6rl9ev 5000 0 2.0 2 consensus 1.0 0 1000000

R version 3.3.1 (2016-06-21)
Loading required package: Rcpp
Warning messages:
1: multiple methods tables found for ‘arbind’
2: multiple methods tables found for ‘acbind’
3: replacing previous import ‘IRanges::arbind’ by ‘SummarizedExperiment::arbind’ when loading ‘GenomicAlignments’
4: replacing previous import ‘IRanges::acbind’ by ‘SummarizedExperiment::acbind’ when loading ‘GenomicAlignments’
5: multiple methods tables found for ‘left’
6: multiple methods tables found for ‘right’
DADA2 R package version: 1.4.0
1) Filtering The filter removed all reads: /tmp/tmprc6rl9ev/L2S204_1_L001_R1_001.fastq.gz not written.
xThe filter removed all reads: /tmp/tmprc6rl9ev/L5S155_2_L001_R1_001.fastq.gz not written.
x
Error: No reads passed the filter (was truncLen longer than the read length?)
Traceback (most recent call last):
  File "/home/jairideout/dev/qiime2/q2cli/q2cli/commands.py", line 222, in __call__
    results = action(**arguments)
  File "<decorator-gen-240>", line 2, in denoise_single
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 200, in callable_wrapper
    output_types, provenance)
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 297, in _callable_executor_
    output_views = callable(**view_args)
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 126, in denoise_single
    run_commands([cmd])
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 35, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/jairideout/miniconda3/envs/qiime2/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/tmp/qiime2-archive-6jn0vrm3/b0aa4c34-2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmprc6rl9ev/output.tsv.biom', '/tmp/tmprc6rl9ev', '5000', '0', '2.0', '2', 'consensus', '1.0', '0', '1000000']' returned non-zero exit status 2

This same info is available when passing --verbose to the CLI command:
$ qiime dada2 denoise-single --i-demultiplexed-seqs demux.qza --p-trim-left 0 --p-trunc-len 5000 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza --p-n-threads 0 --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/qiime2-archive-4l0ksjei/b0aa4c34-2cfe-4d84-a04c-1185cf3d31dc/data /tmp/tmpwe3m5tmm/output.tsv.biom /tmp/tmpwe3m5tmm 5000 0 2.0 2 consensus 1.0 0 1000000

R version 3.3.1 (2016-06-21)
Loading required package: Rcpp
Warning messages:
1: multiple methods tables found for ‘arbind’
2: multiple methods tables found for ‘acbind’
3: replacing previous import ‘IRanges::arbind’ by ‘SummarizedExperiment::arbind’ when loading ‘GenomicAlignments’
4: replacing previous import ‘IRanges::acbind’ by ‘SummarizedExperiment::acbind’ when loading ‘GenomicAlignments’
5: multiple methods tables found for ‘left’
6: multiple methods tables found for ‘right’
DADA2 R package version: 1.4.0
1) Filtering The filter removed all reads: /tmp/tmpwe3m5tmm/L2S204_1_L001_R1_001.fastq.gz not written.
xThe filter removed all reads: /tmp/tmpwe3m5tmm/L5S155_2_L001_R1_001.fastq.gz not written.
x
Error: No reads passed the filter (was truncLen longer than the read length?)
Traceback (most recent call last):
  File "/home/jairideout/dev/qiime2/q2cli/q2cli/commands.py", line 222, in __call__
    results = action(**arguments)
  File "<decorator-gen-240>", line 2, in denoise_single
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 200, in callable_wrapper
    output_types, provenance)
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 297, in _callable_executor_
    output_views = callable(**view_args)
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 126, in denoise_single
    run_commands([cmd])
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 35, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/jairideout/miniconda3/envs/qiime2/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/tmp/qiime2-archive-4l0ksjei/b0aa4c34-2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmpwe3m5tmm/output.tsv.biom', '/tmp/tmpwe3m5tmm', '5000', '0', '2.0', '2', 'consensus', '1.0', '0', '1000000']' returned non-zero exit status 2

Plugin error from dada2:

  Command '['run_dada_single.R', '/tmp/qiime2-archive-4l0ksjei/b0aa4c34-
  2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmpwe3m5tmm/output.tsv.biom',
  '/tmp/tmpwe3m5tmm', '5000', '0', '2.0', '2', 'consensus', '1.0', '0',
  '1000000']' returned non-zero exit status 2

See above for debug info.

The error handling/display works the same way if an error is raised on the Python side:
$ qiime dada2 denoise-single --i-demultiplexed-seqs demux.qza --p-trim-left 0 --p-trunc-len -1 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza --p-n-threads 0

Plugin error from dada2:

  Argument to 'trunc_len' was -1, should be non-negative.

Debug info has been saved to /tmp/qiime2-q2cli-err-t7y794ah.log.

Contents of debug log file:
$ cat /tmp/qiime2-q2cli-err-t7y794ah.log
Traceback (most recent call last):
  File "/home/jairideout/dev/qiime2/q2cli/q2cli/commands.py", line 222, in __call__
    results = action(**arguments)
  File "<decorator-gen-240>", line 2, in denoise_single
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 200, in callable_wrapper
    output_types, provenance)
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 297, in _callable_executor_
    output_views = callable(**view_args)
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 114, in denoise_single
    _check_inputs(**locals())
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 81, in _check_inputs
    % (param, arg, explanation))
ValueError: Argument to 'trunc_len' was -1, should be non-negative.

When run with --verbose:
$ qiime dada2 denoise-single --i-demultiplexed-seqs demux.qza --p-trim-left 0 --p-trunc-len -1 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza --p-n-threads 0 --verbose
Traceback (most recent call last):
  File "/home/jairideout/dev/qiime2/q2cli/q2cli/commands.py", line 222, in __call__
    results = action(**arguments)
  File "<decorator-gen-240>", line 2, in denoise_single
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 200, in callable_wrapper
    output_types, provenance)
  File "/home/jairideout/dev/qiime2/qiime2/qiime2/sdk/action.py", line 297, in _callable_executor_
    output_views = callable(**view_args)
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 114, in denoise_single
    _check_inputs(**locals())
  File "/home/jairideout/dev/qiime2/q2-dada2/q2_dada2/_denoise.py", line 81, in _check_inputs
    % (param, arg, explanation))
ValueError: Argument to 'trunc_len' was -1, should be non-negative.

Plugin error from dada2:

  Argument to 'trunc_len' was -1, should be non-negative.

See above for debug info.

It appears that error handling is consistent on both Python and R sides, and we're capturing stdout/stderr from R. Thus, I don't think there's much to change here, but perhaps we can tweak things to make it a little clearer to users. @maxvonhippel and I chatted about:

Prepend some text to the non-zero exit status message users see when an error occurs in R:

There was an error when running DADA2 in R:

Command '['run_dada_single.R', '/tmp/qiime2-archive-4l0ksjei/b0aa4c34-
  2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmpwe3m5tmm/output.tsv.biom',
  '/tmp/tmpwe3m5tmm', '5000', '0', '2.0', '2', 'consensus', '1.0', '0',
  '1000000']' returned non-zero exit status 2

Add a section to the q2cli tutorial describing --verbose and the debug log.

Add a blank line to the CLI's log file text to highlight the debug log

$ qiime dada2 denoise-single ...

Plugin error from dada2:

  <error message>

See above for debug info.

$

@benjjneb @ebolyen @maxvonhippel does this all make sense, or am I missing something here? Any other suggestions/tweaks that can be made to make things clearer?
Having an explicit mapping of DADA2 error codes to user-friendly messages would be great, but we'd need to have a stable exit code "API" across DADA2 versions to pull it off. @benjjneb are the DADA2 exit codes generally stable across package versions?

@jairideout this all makes sense.  I think my confusion was primarily because this:
Command '['run_dada_single.R', '/tmp/qiime2-archive-4l0ksjei/b0aa4c34-
  2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmpwe3m5tmm/output.tsv.biom',
  '/tmp/tmpwe3m5tmm', '5000', '0', '2.0', '2', 'consensus', '1.0', '0',
  '1000000']' returned non-zero exit status 2

... is not an informative, immediately helpful message to the average, non-technical user in the way that this is:
Argument to 'trunc_len' was -1, should be non-negative.

That said, as we discussed, for now, mapping exit codes to nice summary messages for dada2 is probably not worthwhile.  I do think however that adding There was an error when running DADA2 in R:, as you mention in your first bullet point, would really help.  This would give non-technical users a piece of information they can immediately understand, without needing the intuition of:

Ok, this array of commands is what was given to os in python and then executed in a shell I didn't see, and this is the exit code from that process I didn't see in my shell visibly in front of me, and I need to look at the R configuration to debug this rather than look at Qiime2

.

@jairideout my understanding was the goal was simply to elevate R's stderr into here:
$ qiime dada2 denoise-single ...

Plugin error from dada2:

  <Whatever R produced on stderr here>

Debug info has been saved to /tmp/qiime2-q2cli-err-g3hwwrqf.log.

We're not arguing that what it is doing is wrong, it's just that the following exception:
Command '['run_dada_single.R', '/tmp/qiime2-archive-4l0ksjei/b0aa4c34-
  2cfe-4d84-a04c-1185cf3d31dc/data', '/tmp/tmpwe3m5tmm/output.tsv.biom',
  '/tmp/tmpwe3m5tmm', '5000', '0', '2.0', '2', 'consensus', '1.0', '0',
  '1000000']' returned non-zero exit status 2

is essentially useless to the end use (maybe the exit code is worthwhile, but beyond that...)

@ebolyen I agree, but the concern is that the error that would be elevated seems to be pretty likely to be very long (looking at the debug log files produced at least).  Which would kind of seem to undermine the whole point of the verbose flag, if the output was naively almost as verbose as when the user specifies verbose be True.  Therefore it seems like we would need to make exit codes to shorter summary messages, as we do here, which could be a lot of work and also could be onerous if/when dada2 updates its exit codes etc.

The initial point of this conversation was that stderr is already reasonably short, and if it isn't it could be made shorter by slightly augmenting the R scripts. --verbose collates both stdout and stderr so it really isn't the same thing here.
Basically instead of passing check to the subprocess call, we would check the exit code ourselves and construct something like:
process = subprocess.run(...)
if process.exit_code:
    raise RException(process.exit_code, process.stderr)

The only thing that might need to change would be to stop emitting messages to stderr like @benjjneb noted.
This is going to take some coordination on both the R and Python side, but it's easier than an exit-code map (which is also a fine idea).

Chiming in a bit late, the current behavior @jairideout described does seem reasonable, but I also agree that having a more informative output immediately to the user (not just the log file) upon failures would be helpful.
There seem to be two options: emit the output of stderr from the R script upon a failure, or map errors to exit codes that then trigger useful text on the python side.
I think that the stderr output is the better way to go, especially as I suspect that messages can be redirected from stderr to stdout on the R side, leaving just the warnings and errors as desired.
Rscript calls do not to my knowledge naturally provide useful exit codes, so capturing the error and assigning an exit code would all have to be done by hand, which I think will be quite a bit harder to do in a way that covers all possible errors.
So if I fix the message issue, is it fairly easy on the python side to emit the stderr from the Rscript call upon a failure?

Thanks for the discussion, @maxvonhippel @ebolyen @benjjneb!
@ebolyen @benjjneb, what you are describing makes sense (i.e. capturing and displaying stderr in the Python exception message). I understand that the R error message will usually be short, and thus appropriate for inclusion in the Python exception message. However, warnings are also printed to stderr, which are potentially of use to the user but are too long to include in a Python exception. For example, here's what the R stdout and stderr look like:
stdout:
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/qiime2-archive-5gyimnjn/b0aa4c34-2cfe-4d84-a04c-1185cf3d31dc/data /tmp/tmpi11vx2_t/output.tsv.biom /tmp/tmpi11vx2_t 5000 0 2.0 2 consensus 1.0 0 1000000

R version 3.3.1 (2016-06-21)
DADA2 R package version: 1.4.0
1) Filtering xx

stderr:
Loading required package: Rcpp
Warning messages:
1: multiple methods tables found for ‘arbind’
2: multiple methods tables found for ‘acbind’
3: replacing previous import ‘IRanges::arbind’ by ‘SummarizedExperiment::arbind’ when loading ‘GenomicAlignments’
4: replacing previous import ‘IRanges::acbind’ by ‘SummarizedExperiment::acbind’ when loading ‘GenomicAlignments’
5: multiple methods tables found for ‘left’
6: multiple methods tables found for ‘right’
The filter removed all reads: /tmp/tmpi11vx2_t/L2S204_1_L001_R1_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpi11vx2_t/L5S155_2_L001_R1_001.fastq.gz not written.
Error: No reads passed the filter (was truncLen longer than the read length?)

Running with @benjjneb's idea, we could modify the R scripts to send message and warning to stdout, so that only the R error message is included in stderr. However, from searching around it doesn't appear possible to redirect all message, warning, stop, and quit calls to stdout (e.g. see this post and this post). We can update the R scripts in this repo to use cat (stdout) instead of message, warning, etc. but AFAIK we aren't able to control how other R libraries/scripts behave in this respect (i.e. those invoked by the dada2 R scripts).
@benjjneb do you have any ideas for how to accomplish what I'm describing above? My apologies if I'm missing something obvious here (I'm fairly clueless when it comes to R).

@jairideout
There are some commands in R eg. suppressWarnings that can be used to simply not write out warnings, and will easily suppress those unimportant "multiple methods" warnings. I will start by cleaning some of that up, with the goal of getting to a point where the stderr is clean enough to be reported by the plugin when it exits with an error.

Another option might be to just echo the last line of stderr.

I will start by cleaning some of that up, with the goal of getting to a point where the stderr is clean enough to be reported by the plugin when it exits with an error.

@benjjneb thanks, that'll be really helpful!

Another option might be to just echo the last line of stderr.

That's a good idea! It should work with the current error messages produced by the R scripts.
Another idea is to use some sort of identifier/tag in the error message(s) we want displayed in the Python exception. Then on the Python side, stderr can be easily parsed and the relevant part of the error message displayed. For example, the R scripts could emit error messages starting with "R DADA2 Error: ..." or something like that.
With either of these solutions, the existing stdout/stderr behavior in the R scripts could remain as-is and we wouldn't have to suppress certain warnings/messages/etc. @ebolyen's solution sounds simpler and may be enough for our purposes here. If we want to get fancier than that, explicit exit codes is probably the better way to go in the long run. What do you guys think?

Another idea is to use some sort of identifier/tag in the error message(s) we want displayed in the Python exception. Then on the Python side, stderr can be easily parsed and the relevant part of the error message displayed. For example, the R scripts could emit error messages starting with "R DADA2 Error: ..." or something like that.

I think one of the more compelling reasons for this change is to handle situations where R itself crashes (often because it is out of memory). What kind of error you get when that happens is pretty hard to predict. For the most part there really isn't any good reason for the R script to fail (other than the one exit code we are catching) as we validate everything in Python, so this would ideally target situations where we don't really have that level of control.

I'm not sure I understand your followup @ebolyen -- are you proposing we go with your solution (parsing the final line), the solution I proposed, or something different?

I think stderr capture of some kind is probably necessary to handle the unexpected errors from the language/lower level libraries. But I don't expect parsing out a substring or exit codes to really work in that situation, as we don't have much control over what happens when things catastrophically fail.
This is mostly what we noticed in the LV workshop, the errors were never really something the dada2 code was producing, it usually came from elsewhere. So it seems like either making stderr shorter, or only grabbing the last line which we assume to relate to whatever fatal error the script encountered would be the most robust way of handling this.

Thanks for clarifying! It sounds like error parsing (using tags or the final line) nor exit code mappings aren't going to work great in all situations. Handling stdout/stderr from external applications will get tackled in https://github.com/qiime2/qiime2/issues/224 when we have explicit support for plugins with external commands -- it's probably better to come up with a consistent strategy then and update existing plugins to conform to it.
For now, how about leaving the behavior as-is, since it's consistent with how other plugins work and the current behavior isn't lossy (i.e. the user has access to all stdout/stderr produced by the command)? Having @benjjneb clean up the stdout/stderr would be useful, and those improvements could be made at any time.
If we decide to leave the behavior as-is, I'd be happy to make those other updates @maxvonhippel proposed (listed in my comment) to help users find the error log produced by q2cli.

I am good with that plan. It does look like duplicating stderr so that the last line can even be read is not easy at all. It would required a re-implementation of tee with all kinds of fun os.fork and os.dup2 calls. This assumes we want stdout and stderr to be written and echo'ed in real-time (which I think is pretty important so that you can follow the entire process of what happened and when/where the failure occurred).

Note: @ebolyen's merged PR implements @maxvonhippel's first suggestion from a previous comment:

Prepend some text to the non-zero exit status message users see when an error occurs in R:

I created an issue on the docs repo to track @maxvonhippel's second suggestion:

Add a section to the q2cli tutorial describing --verbose and the debug log.

I created another issue to discuss ways of making q2cli's debug output easier to notice:

Add a blank line to the CLI's log file text to highlight the debug log

Thanks everyone for the discussion!


Default snippet for 'c'

I just noticed that if I type c and hit Tab, it autocompletes to the CDATA wrapper.
Since this is rarely needed any more, but <c>...</c> is used quite frequently, is there a way to change things so that c+Tab produces code formatting as the default?

@oscarlevin do you support cdata completion snippet? IF no I think it is a vscode xml

I am pretty sure that I have a cdata snippet enabled, so I will look into seeing if there is anything that can be done about the order.  This is yet another argument for removing snippets and using only completions, so we can use vscode-xml's schema aware completion suggestions.

I'm still trying to unlearn habits from 3 or 4 years of using Atom. Hitting either c or m and then tab is pretty ingrained muscle memory at this point!

we can use vscode-xml's schema aware completion suggestions.

I think it us the right direction to avoid polluating completion whith snippets which are not allowed.
But please keep in mind you will loose your advanced snippet which generate some non required attributes.
If you want to keep it it will require some work on vscode-xml.  See
https://github.com/oscarlevin/pretext-tools/issues/3#issuecomment-1406481716

This is now fixed with our new snippets as completion feature.


Adds more reasoning for symbolic maps and some tests demonstrating desired behavior

@yzhang90 and @daejunpark please add any tests you want to make sure the symbolic map passes to this PR. I figure this file can become our "specification of what symbolic maps in K mean".

The following test scenario should be a good start.
(Note that ?= means neither == nor != can be proved.)
Here I consider the ideal symbolic map reasoning that tries to simplify a term as much as possible, i.e., reducing to a canonical term, at the cost of time overhead.
An efficient symbolic map reasoning may not fully simplify the term, but should be equivalent to the canonical term.
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 3                    when y == x /\ z == y /\ z == x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 2 z |-> 3            when y == x /\ z != y /\ z != x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 2 [z <- 3]           when y == x /\ z ?= y /\ z ?= x

M x |-> 1 [y <- 2] [z <- 3] => M x |-> 3 y |-> 2            when y != x /\ z != y /\ z == x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 y |-> 2 z |-> 3    when y != x /\ z != y /\ z != x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 y |-> 2 [z <- 3]   when y != x /\ z != y /\ z ?= x

M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 y |-> 3            when y != x /\ z == y /\ z != x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 y |-> 2 [z <- 3]   when y != x /\ z ?= y /\ z != x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 y |-> 2 [z <- 3]   when y != x /\ z ?= y /\ z ?= x

M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 [y <- 3]           when y ?= x /\ z == y /\ z ?= x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 z |-> 3 [y <- 2]   when y ?= x /\ z != y /\ z != x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 [y <- 2] [z <- 3]  when y ?= x /\ z != y /\ z ?= x

M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 [y <- 2] [z <- 3]  when y ?= x /\ z ?= y /\ z == x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 [y <- 2] [z <- 3]  when y ?= x /\ z ?= y /\ z != x
M x |-> 1 [y <- 2] [z <- 3] => M x |-> 1 [y <- 2] [z <- 3]  when y ?= x /\ z ?= y /\ z ?= x

-------------------------------------------------------------------------------------------

M x |-> 1 [y <- 2] [z] => 2                                 when y == x /\ z == y /\ z == x
M x |-> 1 [y <- 2] [z] => M [z]                             when y == x /\ z != y /\ z != x
M x |-> 1 [y <- 2] [z] => M x |-> 2 [z]                     when y == x /\ z ?= y /\ z ?= x

M x |-> 1 [y <- 2] [z] => 1                                 when y != x /\ z != y /\ z == x
M x |-> 1 [y <- 2] [z] => M [z]                             when y != x /\ z != y /\ z != x
M x |-> 1 [y <- 2] [z] => M x |-> 1 [z]                     when y != x /\ z != y /\ z ?= x

M x |-> 1 [y <- 2] [z] => 2                                 when y != x /\ z == y /\ z != x
M x |-> 1 [y <- 2] [z] => M y |-> 2 [z]                     when y != x /\ z ?= y /\ z != x
M x |-> 1 [y <- 2] [z] => M x |-> 1 y |-> 2 [z]             when y != x /\ z ?= y /\ z ?= x

M x |-> 1 [y <- 2] [z] => 2                                 when y ?= x /\ z == y /\ z ?= x
M x |-> 1 [y <- 2] [z] => M [z]                             when y ?= x /\ z != y /\ z != x
M x |-> 1 [y <- 2] [z] => M x |-> 1 [z]                     when y ?= x /\ z != y /\ z ?= x

M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y ?= x /\ z ?= y /\ z == x
M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y ?= x /\ z ?= y /\ z != x
M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y ?= x /\ z ?= y /\ z ?= x

Another thing. I think we should use krun to test this behavior. kprove involves some implicit simplification over both LHS and RHS of the spec, so not adequate to test the function evaluation.

@daejunpark I've tested as many of your rules as I can by translating them into kprove tests. I first tested them with no partial revert (so keeping @yzhang90 's #290), and the result was this: 3dbb676. But then, by importing MAP-SYMBOLIC from domains.k, it was able to prove more of them: 81961e3. Perhaps we can add enough lemmas to MAP-SYMBOLIC that they all pass.
I think we need to be able to turn off evaluation of the RHS before attempting matching, to make sure that the simplification happens to exactly the RHS and no further. You suggested doing this by running with krun instead of kprove, but I think it may be easier to just tell the prover "make sure it matches exactly what I wrote, and not some normal form of what I wrote".
We need to also need to find out why the negative proof cases are passing, because they probably shouldn't be.

updated test scenario expected output with test spec id:
M x |-> 1 [y <- 2] [z] => 2                                 when y == x /\ z == y /\ z == x  16
M x |-> 1 [y <- 2] [z] => M [z]                             when y == x /\ z != y /\ z != x   3
M x |-> 1 [y <- 2] [z] => M x |-> 2 [z]                     when y == x /\ z ?= y /\ z ?= x   1

M x |-> 1 [y <- 2] [z] => 1                                 when y != x /\ z != y /\ z == x  15
M x |-> 1 [y <- 2] [z] => M [z]                             when y != x /\ z != y /\ z != x   9
M x |-> 1 [y <- 2] [z] => M x |-> 1 [z]                     when y != x /\ z != y /\ z ?= x   7

M x |-> 1 [y <- 2] [z] => 2                                 when y != x /\ z == y /\ z != x   8
M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y != x /\ z ?= y /\ z != x   6
M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y != x /\ z ?= y /\ z ?= x   5

M x |-> 1 [y <- 2] [z] => 2                                 when y ?= x /\ z == y /\ z ?= x  12
M x |-> 1 [y <- 2] [z] => M [z]                             when y ?= x /\ z != y /\ z != x  14
M x |-> 1 [y <- 2] [z] => M x |-> 1 [z]                     when y ?= x /\ z != y /\ z ?= x  13

M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y ?= x /\ z ?= y /\ z == x  10
M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y ?= x /\ z ?= y /\ z != x  11
M x |-> 1 [y <- 2] [z] => M x |-> 1 [y <- 2] [z]            when y ?= x /\ z ?= y /\ z ?= x   4

@ehildenb do you think you can have another set of the 15 assignment tests where you provide the pre-condition saying that x, y, and z are not in the keys of the map?  such a test set will exercise more interesting behaviors.

hmm, @ehildenb, now i realized that it is not trivial to add the in_keys side-condition to the other rules.  so i'll not ask you to complete the MAP-SYMBOLIC in this pr.

looks good to me. but i'll let @denis-bogdanas to review the last commit.

@ehildenb @daejunpark PR is now rebased on top of 2 other PRs and still has issues. Do not merge until prerequisites are merged.


Validation SumStats once deposition app is back up

This is a reminder to validate SumStats submitted by Matthew Sampson
For the study with PMID: 29903748
The submission ID is 5e613a18b8b8fd000143a5a2

I have checked the submission and the backend only contains the empty SS template file. It seems like the user has not filled in the template and submitted it. As a result, the submission is still in STARTED.

Checked the ftp server and the files are there but not gzipped. The checksums provided are for gzip files and do no match the checksums of the files even if I gzip them.


What is the sense of String.erase( int position, int chars )?

Godot version:
Any
OS/device including version:
Any
Issue description:
What is the sense of String.erase( int position, int chars ) if it does not return anything?
https://docs.godotengine.org/en/stable/classes/class_string.html#class-string-method-erase

It changes the string :)

Seems like string in gdscript are indeed mutable ;)

Looks like this is the only String method that modifies the String instead of returning a new one 🤔

I'd vote for revising this method to make it immutable like other String methods.

Yeah, this is confusing for me. It is better when all functionality works the same way.

I'd vote for revising this method to make it immutable like other String methods. What should we rename it to?

Maybe something like: splice?

@MWFIAE Sounds good to me. The method would behave roughly like JavaScript's Array.splice then.

In Javascript, splice() changes the original array whereas slice() doesn't but both of them return array object. So, maybe we should go for something like slice() to maintain immutability of String.

@theoway We already have Array.slice() which has a different behavior, I fear giving a different behavior to String.slice() would make it confusing.

@theoway We already have Array.slice() which has different behavior, I fear giving a different behavior to String.slice() would make it confusing.

Indeed, it'd be confusing. Then it'd be a good option to implement something like splice(like in Javascript)while maintaining string immutability.

Indeed, it'd be confusing. Then it'd be a good option to implement something like splice(like in Javascript)while maintaining string immutability.
Why not just update erase to return the new string and not mutate the string as a backwards incompatible break for 4.0? Since going forward you'd need to do something with 'erase' anyway. It seems cleaner then cluttering the API up with another method name.

Indeed, it'd be confusing. Then it'd be a good option to implement something like splice(like in Javascript)while maintaining string immutability.

Why not just update erase to return the new string and not mutate the string as a backwards incompatible break for 4.0? Since going forward you'd need to do something with 'erase' anyway. It seems cleaner then cluttering the API up with another method name.

Makes sense. But the method name needs to be changed, as it won't be erasing anymore(after changing the method)

@Calinou Or should we change it for now so that it doesn't mutate the original string, and returns a copy of the erased string, like @denormative said.

Upon thinking beyond the method itself, we have a similar method String.insert(), which is immutuable and returns a new String...

as it won't be erasing anymore

Neither insert would insert a new substring into existing one, so perhaps it also has to be renamed? Not sure... 🤔

Neither insert would insert a new substring into existing one, so perhaps it also has to be renamed? Not sure... 🤔

Indeed. If anyone has suggestions as to what it should be renamed to, drop 'em here. I'll make a PR to change the erase() to something like splice and change the name of insert()

Maybe we should remove String.erase() entirely in 4.0? Mutable string methods can cause more trouble than they solve.

Maybe we should remove/unexpose String.erase() entirely in 4.0? Mutable string methods can cause more trouble than they solve.

That's a good idea. It can be replaced with a similar method, like splice(in Javascript), making sure that it doesn't mutate the string. How does that sound?

@theoway Isn't JavaScript's Array.splice() the same as Array.slice() but in a mutable form?

@theoway Isn't JavaScript's Array.splice() the same as Array.slice() but in a mutable form? Going by this question on Stack Overflow, I can't see a difference other than the mutability.

Also, splice can add elements at the index from where old elements were removed.
So, in a way, it can do the job of insert() & erase()

@theoway In this case, I guess it makes sense to implement an immutable version of String.splice().

@theoway In this case, I guess it makes sense to implement an immutable version of String.splice().

Should I make the PR of implementation or make a proposal first?

@theoway I think you can open a pull request directly against the master branch and remove String.erase() at the same time.

@Calinou  I had one more question. Since, we're implementing splice, it'll make insert() redundant in gdscript string. I was wondering if we should get rid of it as well.

@theoway I guess it makes sense to remove String.insert() as well in this case.

@Calinou Now there are many references to erase() and insert() throughout the project. So, should I remove these methods and make changes everywhere or add the new method(i.e. splice()) while keeping these two, but removing their references from engine's scripting languages(gdscript, gdnative etc.)?

I am currently trying to testing RichTextLabel nodes in 4.0 master because the new RTL got a whole bunch of features I really need in my project. My project is very heavy in dynamic text. Which means I need to format BBCode text of the RTL a lot, which in turn means I need to erase and insert (not necessarily replace) a lot of strings.
Right now, I can't really do any of my testing because the latest master build appears to have erase() removed, but splice does not exist either.
Also, just to note, erase is still featured in the online API documentation, however not in the build in API documentation. splice() is mentioned nowhere. The current master also gives me a showstopping error saying "this property does not exist".
I would really love to test the new RTL BBCode features, but without being able to edit strings properly, there really is no point. It would be really nice if someone could reinstate erase() until we actually have splice() or any better solution.

Right now, I can't really do any of my testing because the latest master build appears to have erase() removed, but splice does not exist either.

Also, just to note, erase is still featured in the online API documentation, however not in the build in API documentation. splice() is mentioned nowhere. The current master also gives me a showstopping error saying "this property does not exist".

The erase() method was probably not exposed since it mutates the String. The new core binding system made it difficult or impossible to expose some methods: https://github.com/godotengine/godot/issues/43311
Closing as a duplicate of https://github.com/godotengine/godot/issues/43311, since we'll have to see whether we *can * actually expose a String.erase() method for 4.0.


Permit to customize LINK regex

This proposal permit to customize link regex
(i would like to be able to grab them into inline comment so i need to change the regex to ^(\.{1,2}[/\\])?([^:# ]+)?(:\d+|#[\w-]+)?, eg :
<!-- LINK example.html -->
<?php // LINK example.php ?>
<?php /* LINK example.php */ ?>

Looks like a good change to me. Would you be so kind to additionally declare this new setting in the package.json as well as the README.md?
From a code style perspective it would be preferred to leave the default regex as a constant as it was originally, however renamed to DEFAULT_LINK_REGEX.
Nevertheless, thanks for the PR!


🅱 Horizontal FlashList Bug (Layout Problem for height)

Current behavior
🅱  Horizontal FlashList Bug (Layout Problem for height)
create Flashlist horizontal with any data try to switch from portrait - landscape - portrait again
you will see thats list height is not updated correctly (cropped!) and sounds cache previous height for landscape
regarding some investigation i made console.log from inside _setFinalDimensions
console.log(this._totalWidth, ' - ', this._totalHeight)
recyclerlistview/dist/reactnative/core/layoutmanager/LayoutManager.js
1250  -  952
530  -  952
500  -  952
500  -  696
❌ 500  -  696 (should back again to correct height 952)
also if you tried that from (onLayout) you will see that width and height not updated correctly
<FlashList         onLayout={(event) => {           console.warn(event.nativeEvent.layout.width, '-', event.nativeEvent.layout.height)         }}         horizontal
bug.webm
Expected behavior
List height should be full as it was in beginning before rotate.
To Reproduce
try to switch horizontal flashlist from portrait - landscape - portrait again
you will see list is cropped in height when back again to portrait
you can also check width and height from onLayout to check wrong values.
Platform:

[ ] iOS
[ ] Android

Environment
"@shopify/flash-list": "^1.4.1",
"expo": "~47.0.12",

Any solutions or ideas of how to overcome this issue?

Thanks for flagging the issue. This is a duplicate of https://github.com/Shopify/flash-list/issues/581 that unfortunately hasn't been solved yet 😞 PRs are welcome!


[Bug] Using AOT with UMA and TFLITE

Hi everyone,
found a possible issue.
When executing a testcase using UMA and generating the test .c files via aot.py, it seems that the pointers of the offloaded functions are not in the right order.
Can anyone help?
Thanks,
Michael
CC: @cgerum @PaulPalomeroBernardo  @manupa-arm @areusch @lhutton1 @d-smirnov @mehrdadh @SebastianBoblest @sezgin1947
Expected behavior
Correct execution of provided NN imported from TFLITE file.
Actual behavior
Segmentation fault, due to mixed up pointers in generated file.
In particular: in default_lib2.c the pointers for placeholder_1, placeholder are mixed up
Environment
Ubuntu 18.04
TVM c3c7c4ccc3e3ebb7cc4dbb55e8dde579ac52c949
Steps to reproduce
Run tests/python/contrib/test_uma/test_uma_pipeline.py from this branch
https://github.com/MichaelJKlaiber/tvm/tree/issue/uma-tflite
Generated file looks like this:
#include "/home/michael/Documents/code/tvm/apps/uma/_template/conv2dnchw.cc"// tvm target: vanilla_accelerator
#define TVM_EXPORTS
#include "tvm/runtime/c_runtime_api.h"
#include "tvm/runtime/c_backend_api.h"
#include <math.h>
#ifdef __cplusplus
extern "C"
#endif
TVM_DLL int32_t tvmgen_default_vanilla_accelerator_main_0(float* placeholder, float* placeholder_1, float* T_matmul_NT, uint8_t* global_const_workspace_4_var, uint8_t* global_workspace_5_var) {
  for (int32_t i1 = 0; i1 < 10; ++i1) {
    for (int32_t i2 = 0; i2 < 784; ++i2) {
      if (i2 == 0) {
        T_matmul_NT[i1] = 0.000000e+00f;
      }
      T_matmul_NT[i1] = (T_matmul_NT[i1] + (placeholder_1[i2] * placeholder[((i1 * 784) + i2)]));
    }
  }
  return 0;
}

when changing the function signature manually to
TVM_DLL int32_t tvmgen_default_vanilla_accelerator_main_0(float* placeholder_1, float* placeholder, float* T_matmul_NT, uint8_t* global_const_workspace_4_var, uint8_t* global_workspace_5_var)
the test is successful

this looks odd--it's late for today for me to investigate but hopefully we can look into it next week.

Seems the bug is in the lowering step from  relay_prim_func to tir_prim_func.
When collecting the input arguments, the order gets messed up. By replacing the function _get_tensors here
https://github.com/MichaelJKlaiber/tvm/blob/issue/uma-tflite/python/tvm/relay/backend/contrib/uma/api/lower.py#L83
by
            return list(te_cached_func.inputs) + list(te_cached_func.outputs)

everything works perfectly fine for all the test case we provided including the tflite import
Does anyone know if it is sufficient to rely on the .inputs parameters from te_cached_func ?

@MichaelJKlaiber is this one fixed?

yes, the bugfix is in main


fast debug drawing and capsule height and width fix

This has faster debug drawing which improves performance significantly when is_debugging_collisions_hint is turned on.
Also includes capsule fixture fix.

Just note on this PR, this is kinda clobbering this:  (reverted some things).
			if (body && is_contact_monitor_enabled()) {
				for (int i = 0; i < get_contact_count(); i++) {
					Vector2 point = get_box2dworld_transform().xform_inv(get_contact_world_pos(i));
					draw_rect(Rect2(point + Point2(-1.0f, -1.0f), Size2(2.0f, 2.0f)), Color(1.0f, 1.0f, 0.0f));

Up for discussion how to do this, but this looks like we just want to show contacts as well for each body?

I hadn't really decided whether we should or how to go about drawing debug contacts. In the future I think they should be made useful for debugging advanced collision logic.


chore(clang-tidy): remove deprecated/unsupported AnalyzeTemporaryDtors setting

Description
This PR aims to make Clang-Tidy usable in IDEs, e.g. with Clangd as a language server. For VS Code, the default Clangd version is 18, which does not support the AnalyzeTemporaryDtors option anymore, which was deprecated in version 16.
This PR removes the setting which, according to the Clang 16 release notes is no longer in use:

Deprecate the global configuration file option AnalyzeTemporaryDtors, which is no longer in use. The option will be fully removed in clang-tidy version 18.

Related links:

LLVM GitHub: Discussion about removal of the option
Clang 16 Release Notes: Deprecation message

Tests performed
Before the change, Clangd 18 would show errors when parsing .clang-tidy:
E[10:48:30.356] tidy-config error at /home/.../autoware/.clang-tidy:205:0: unknown key 'AnalyzeTemporaryDtors'
E[10:48:30.356] Error parsing clang-tidy configuration in /home/.../autoware/.clang-tidy: Invalid argument

After the change, the errors are gone.
Effects on system behavior
Not applicable.
Interface changes
Not applicable.
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

NOTE:
The default clang-tidy version in Ubuntu 22.04 is 14, but the default config for AnalyzeTemporaryDtors seems to be false so this modification shouldn't have impact on environments with older versions of clang-tidy as well.


Identity Default UI Manage page can potentially lead to user lock outs

The default Identity UI has a bug with both changing email and phone number in the Manage/Index page. For example, if the user enters an email in the form field, SetEmailAsync is naively called which marks the user's email as unconfirmed.
var email = await _userManager.GetEmailAsync(user);
if (Input.Email != email)
{
    var setEmailResult = await _userManager.SetEmailAsync(user, Input.Email);
    if (!setEmailResult.Succeeded)
    {
        var userId = await _userManager.GetUserIdAsync(user);
        throw new InvalidOperationException($"Unexpected error occurred setting email for user with ID '{userId}'.");
    }

    // In our UI email and user name are one and the same, so when we update the email
    // we need to update the user name.
    var setUserNameResult = await _userManager.SetUserNameAsync(user, Input.Email);
    if (!setUserNameResult.Succeeded)
    {
        var userId = await _userManager.GetUserIdAsync(user);
        throw new InvalidOperationException($"Unexpected error occurred setting name for user with ID '{userId}'.");
    }
}

If the user now logs out on a site that requires a confirmed email, they will not be able to log in again. To add insult to injury, a confirmation email is not sent automatically, but instead requires the user clicking a link after being returned to the page from the post. If the user neglects to do so, then once they're logged out, they're now locked out. Sending the confirmation email immediately without user intervention would be better as now they have a chance to recover. However, even that only works if the email is good. If the user made a mistake and entered an erroneous email address, they are now completely locked out without hope of ever being able to recover without intervention from the site owner.
Long and short, this code needs to be changed to accomodate using ChangeEmailAsync instead. A confirmation email needs to be sent on post, instead of immediately setting the new email address, and then a confirm email change page needs to be  added to actually handle the change.
This likewise applies to the setting of a new phone number, which follows the same broken logic as setting an email address.
var phoneNumber = await _userManager.GetPhoneNumberAsync(user);
if (Input.PhoneNumber != phoneNumber)
{
    var setPhoneResult = await _userManager.SetPhoneNumberAsync(user, Input.PhoneNumber);
    if (!setPhoneResult.Succeeded)
    {
        var userId = await _userManager.GetUserIdAsync(user);
        throw new InvalidOperationException($"Unexpected error occurred setting phone number for user with ID '{userId}'.");
    }
}

Will be fixed in #8356


The feasibility of modding Tekken in this way.

This is more of a question for devs.
I see that you can get memory from hardcoded address this way, and that they're gotten from Cheat Engine. What more can you do? Can you call functions maybe or press button? Something simple like force restart training (Select+A)
Or could it be taken further is a player lobby, like if both players were running a modified version of Tekkenbot to make a match have an infinite timer, and heal damage? To create some kind of online training mode.
I'm considering attempting this, but how realistic is that, and what would you suggest looking at first?

Infinite timer and infinite health can be activated by both players using trainer to activate infinite options.
if used by one player and the other player does not, then there will be sync error and gets disconnected before the match ends.
You cannot write memory and run input by python without code injection.