[QUERY] Is the TableTransactionBatch intended to capture entity state?

Query/Question
I am moving from WindowsAzure.Storage SDK to Azure.Data.Tables and I have many places in my code where I use a transaction entity batch to reduce round trips to Table Storage and to leverage atomic updates to many entities within a partition.
In WindowsAzure.Storage, it's possible to built up a transaction batch and then modify the entities prior to submitting the batch and have those latest entity changes be reflected in the submitted change. In other words, this works:

Add entity to batch
Modify entity
Submit batch
See changes on step 2 in the resulting entity.

In Azure.Data.Tables, it looks like step 1 immediately captures all of the OData properties from the entity at the moment, in a deep-clone type fashion, making any changes (i.e. step 2) have no affect the batch that gets submitted.
Is this an intentional design decision? It's certainly not an insurmountable problem. For example step 1 can actually be adding logical entity operations to a list like List<Action<TableTransactionBatch>> that I manage and then at step 3 I actually create the batch and add all of the collected operations into the batch at that time, thus capturing the latest entity state.
If this was intentional, do you think it would be worth enhancing the XML docs on the TableTransactionBatch methods to say that the current entity state is captured and future changes to the ITableEntity will not affect the batch? It caught me off guard in the migration to the new SDK so I'm wondering if others might fall into the same trouble.
Environment:

Name and version of the Library package used: Azure.Data.Tables 12.0.0-beta.6
Hosting platform or OS and .NET runtime version:

Windows 10
.NET 5 SDK

.NET SDK (reflecting any global.json):
Version:   5.0.201
Commit:    a09bd5c86c

Runtime Environment:
OS Name:     Windows
OS Version:  10.0.19042
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\5.0.201\

Host (useful for support):
Version: 5.0.4
Commit:  f27d337295

.NET SDKs installed:
1.1.14 [C:\Program Files\dotnet\sdk]
2.1.814 [C:\Program Files\dotnet\sdk]
3.1.407 [C:\Program Files\dotnet\sdk]
5.0.100-dev [C:\Program Files\dotnet\sdk]
5.0.104 [C:\Program Files\dotnet\sdk]
5.0.200 [C:\Program Files\dotnet\sdk]
5.0.201 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.0-preview.8.20359.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.0-preview.8.20358.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.0-rc.1.20366.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.0-preview.8.20352.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download

IDE and version: Visual Studio 16.10 Preview 1

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Hi @joelverhagen - Thanks for filing this issue. This behavior wasn't a specific decision, but I agree that adding some docs to clarify the behavior would be helpful.

If this is helpful, here's a diff where I moved a simple module from WindowsAzure.Storage to Azure.Data.Tables:
https://github.com/joelverhagen/ExplorePackages/compare/28762403101995e518463213e5789aad8cb085b0..9f95e495b2fded7e98032f0537263eb265cbd073#diff-fbd0ef9457ec5339f8446a2fd91ba401fd76c452b415f952e0b3b5784a035f9dL145-R223
I've tried to link to the specific part of the changeset that is interesting. Main pain points:

No Count/enumeration API on the batch type to determine a no-op. Submitting an empty batch results in an HTTP 400 from Table storage, which the Azure.Data.Tables APIs allows forcing the caller to protect against this.
Entities must be modified prior to adding them to the batch. This causes some duplication in code where previously I could update all of the entities in one place.

No Count/enumeration API on the batch type to determine a no-op. Submitting an empty batch results in an HTTP 400 from Table storage, which the Azure.Data.Tables APIs allows forcing the caller to protect against this.

Would it be your expectation for SubmitBatch to throw InvalidOperationException if it contains no operations? I think this makes sense.
Outside of that, would you find it helpful to have the operation count available either way?

Entities must be modified prior to adding them to the batch. This causes some duplication in code where previously I could update all of the entities in one place.

This one I am less sure about since I wonder if being able to manipulate entities already added to the batch might give the expectation that after execution the entities would be updated based on the batch results. Do you have any thoughts about this?

Would it be your expectation for SubmitBatch to throw InvalidOperationException if it contains no operations? I think this makes sense.

I would expect that, yes. The SDK can save some unnecessary IO on a case that is likely unintended by the caller.

Outside of that, would you find it helpful to have the operation count available either way?

The only reason I have used the count is for an "is empty" check. I can't think of anything additional except perhaps, in the future and as needed, an enumeration API to inspect the items in the batch. But TBH I can't think of a scenario for that more extensive API.

This one I am less sure about since I wonder if being able to manipulate entities already added to the batch might give the expectation that after execution the entities would be updated based on the batch results. Do you have any thoughts about this?

Hmm... interesting point. I think my vision is clouded/biased based on legacy SDK behavior which would do both of those things. That's something I hadn't even considered yet since I haven't gotten to the part in my own codebase where batch updates are done and I need the subsequent etags.
Is it feasible to get all of the new etags from the successful result of batch of entity updates?

Would it be your expectation for SubmitBatch to throw InvalidOperationException if it contains no operations? I think this makes sense.

I would expect that, yes. The SDK can save some unnecessary IO on a case that is likely unintended by the caller.

Cool, I'll add an issue to make that minor change.

Is it feasible to get all of the new etags from the successful result of batch of entity updates?

Yes! Although I don't think we have a good sample showing this yet.
Example:
TableBatchResponse response = await batch.SubmitBatchAsync().ConfigureAwait(false);
// Get the new ETag for the updated entity
response.GetResponseForEntity(entity.RowKey).Headers.ETag;

Actually, we do show this in the samples here:
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/tables/Azure.Data.Tables/samples/Sample6TransactionalBatch.md#adding-entities-with-a-transactional-batch
https://github.com/Azure/azure-sdk-for-net/blob/533ee2530f783f081d04c4bbcb79d483d9ef16a6/sdk/tables/Azure.Data.Tables/tests/samples/Sample6_TransactionalBatchAsync.cs#L74


Breaking change, update intl to 0.18.1.

Bug description
math_keyboard package won't run with newest flutter sdk.
see error below:
[flutter_lib_test] flutter pub get
Resolving dependencies...
Because math_keyboard >=0.0.1 depends on flutter_localizations from sdk which depends on intl 0.18.1, math_keyboard >=0.0.1 requires intl 0.18.1.
So, because flutter_lib_test depends on math_keyboard ^0.1.9 which depends on intl ^0.17.0, version solving failed.
exit code 1

To Reproduce

create a flutter app on the latest Flutter 3.10 sdk.
add math_keyboard: ^0.1.9 as dependency package.
run flutter pub get
see error.

Updating math_keyboard's intl package dependency to 0.18.1 should solve the issue.

Fixed with math_keyboard 0.2.0


feat: add FAQ section

Related Issue
#340
Description
I have added an FAQ section using accordions which displays the most frequently asked question answers.It will help clear the common queries of users.I have also added some sample question answers based on git-re from my own.
Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)

https://github.com/ashutosh-rath02/git-re/assets/122633300/7ee36541-222a-443f-96f3-7c3a767af08c
Checklist:

[x] I have performed a self-review of my code
[x] I have read and followed the Contribution Guidelines.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]

Thank you for submitting your pull request! 🙌 We'll review it as soon as possible. In the meantime, please ensure that your changes align with our CONTRIBUTING.md. If there are any specific instructions or feedback regarding your PR, we'll provide them here. Thanks again for your contribution! 😊

@komal-agarwal5 resolve this please and remove the underline please

@komal-agarwal5 resolve this please and remove the underline please

@ashutosh-rath02 I made the corrections, is it working now?

@ashutosh-rath02 sir can you please add the gssoc and level labels?


Add Logo

Since logoA is not being used, it should be removed. By that, you can also rename logoA2 to simply logo.


feat!: diasble acr admin account

Disable Azure Container Registry admin account by default to follow Azure recommendation.
BREAKING CHANGE: admin account disabled by default. To upgrade, set admin_enabled to true.

Tests pass ⭐


Fixing AM0012 false failures for newly on-boarded addons

Summary
Prevents false failures when a new addon is on-boarding and does not have any bundles yet.

/retest


First "az vm run-command invoke" returns successful, automation kicks off second "az vm run-command invoke" returns failure.

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
In our automation, we are using az vm run-comand invoke multiple times to check if VMs are up and ready.
For this current case we use it to run cloud-init status --wait.
That returns successful, have a grep for checking. Then a second command is kicked off in automation,
this one is tailing our loud-init-output.log file, to make sure everything is set up correctly. Sometimes this command is failing with the following error:
ERROR: (Conflict) Run command extension execution is in progress. Please wait for completion before invoking a run command.
I have also seen it fail later on in our automation, with the same error.
We are hitting it more often and I have been able to hit it manually as well.
To Reproduce
az vm create (with custom data file - cloud-init file)
az vm run-command invoke                  \
      --resource-group ${RESOURCE_GROUP_NAME} \
      --name ${VM_NAME}                      \
      --command-id RunShellScript             \
      --scripts "cloud-init status --wait"    \
      --query 'value[0].message' --output tsv | tee ${cmdlog}

az vm run-command invoke                          \
    --command-id     RunShellScript                 \
    --name           ${VM_NAME}                     \
    --resource-group ${RESOURCE_GROUP_NAME}         \
    --scripts "tail -100 /var/log/cloud-init-output.log" \
    --query 'value[0].message' --output tsv | tee /tmp/cloud-init_output.log

Expected behavior
az vm run-command invoke should not think another run-command is running if the previous command returned successful.
Environment summary
Pipelines Run on Azure Devops:
Agent name: 'Azure Pipelines 2'
Agent machine name: 'fv-az234-671'
Current agent version: '2.189.0'
Operating System
Ubuntu
20.04.2
LTS
Virtual Environment
Environment: ubuntu-20.04

Compute


Dribbblish seens broken

Dribblish Broken Features
The song's image preview doesn't work as should, and the time bar it's invisible. The controls (Pause, Shuffle, Next song etc) are inverted with the Song's name and author.
To Reproduce
I installed everything correctly, in Windows 10, everything updated.
Then installed spicetify via Powershell, and also choose the Theme + Colour scheme via powershell.
Screenshots

Logs
The FetchEvent for "https://xpui.app.spotify.com/glue-resources/fonts/GoogleSansDisplayMedium.woff2" resulted in a network error response: the promise was rejected.
The FetchEvent for "https://xpui.app.spotify.com/glue-resources/fonts/GoogleSansDisplayRegular.woff2" resulted in a network error response: the promise was rejected.
service-worker.js:1 Uncaught (in promise) no-response: no-response :: [{"url":"https://xpui.app.spotify.com/glue-resources/fonts/GoogleSansDisplayMedium.woff2","error":{}}]
at D._handle (https://xpui.app.spotify.com/service-worker.js:1:20428)
at async D._getResponse (https://xpui.app.spotify.com/service-worker.js:1:10198)
service-worker.js:1 Uncaught (in promise) no-response: no-response :: [{"url":"https://xpui.app.spotify.com/glue-resources/fonts/GoogleSansDisplayRegular.woff2","error":{}}]
at D._handle (https://xpui.app.spotify.com/service-worker.js:1:20428)
at async D._getResponse (https://xpui.app.spotify.com/service-worker.js:1:10198)
vendor~xpui.js:29 MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 change listeners added. Use emitter.setMaxListeners() to increase limit
at c (https://xpui.app.spotify.com/vendor~xpui.js:29:2656773)
at t.o.addListener (https://xpui.app.spotify.com/vendor~xpui.js:29:2659055)
at https://xpui.app.spotify.com/vendor~xpui.js:29:2091056
at ka (https://xpui.app.spotify.com/vendor~xpui.js:29:2946534)
at Ps (https://xpui.app.spotify.com/vendor~xpui.js:29:2970386)
at t.unstable_runWithPriority (https://xpui.app.spotify.com/vendor~xpui.js:29:3190652)
at hi (https://xpui.app.spotify.com/vendor~xpui.js:29:2916042)
at Is (https://xpui.app.spotify.com/vendor~xpui.js:29:2970165)
at ws (https://xpui.app.spotify.com/vendor~xpui.js:29:2954936)
at vs (https://xpui.app.spotify.com/vendor~xpui.js:29:2953858)
c @ vendor~xpui.js:29
GoogleSansDisplayMedium.woff2:1 Failed to load resource: net::ERR_FAILED
GoogleSansDisplayRegular.woff2:1 Failed to load resource: net::ERR_FAILED
The FetchEvent for "https://xpui.app.spotify.com/glue-resources/fonts/RobotoMedium.woff2" resulted in a network error response: the promise was rejected.
service-worker.js:1 Uncaught (in promise) no-response: no-response :: [{"url":"https://xpui.app.spotify.com/glue-resources/fonts/RobotoMedium.woff2","error":{}}]
at D._handle (https://xpui.app.spotify.com/service-worker.js:1:20428)
at async D._getResponse (https://xpui.app.spotify.com/service-worker.js:1:10198)
The FetchEvent for "https://xpui.app.spotify.com/glue-resources/fonts/Roboto.woff2" resulted in a network error response: the promise was rejected.
service-worker.js:1 Uncaught (in promise) no-response: no-response :: [{"url":"https://xpui.app.spotify.com/glue-resources/fonts/Roboto.woff2","error":{}}]
at D._handle (https://xpui.app.spotify.com/service-worker.js:1:20428)
at async D._getResponse (https://xpui.app.spotify.com/service-worker.js:1:10198)
RobotoMedium.woff2:1 Failed to load resource: net::ERR_FAILED
Roboto.woff2:1 Failed to load resource: net::ERR_FAILED
DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/service-worker.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/vendor~xpui.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/xpui.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/998.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/xpui-desktop-modals.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
Specifics:

OS: Windows 10 Home
<IP_ADDRESS>4.gc0d1ad42
v2.5.0
Dribblish Purple

We are tracking issues relative to Dribbblish after last update here
https://github.com/morpheusthewhite/spicetify-themes/issues/507


scss tweaks for contrast issues, img presentation role, still more to do

Handling a few WCAG issues towards AA level, will need to discuss changes to minimal mistakes framework/variables, might need to open issues there in particular with contrast in template presets
#66

@rentry Do we need to care about the netlify deploy failures before merging this PR?

@curtisk This branch is showing the next election as November.  Did you branch from the latest version of Master?

yeah, realized what happened, I'll re-submit

@shannonmcharg No need to worry about the Netlify alerts. They only indicate that the Netlify build failed, but you're production deployment is GitHub Pages, so it shouldn't affect anything.


Add a failpoint for delaying proposals

This adds a failpoint simulating a delay in generating an appendable block when a node is about to propose, for better testing of consensus.

bors r+


Enable condition evaluation for jira issue config and job recipe

updates demodata
enables 'when' evaluation for Jira issue config
enables 'when' evaluation for job recipe
updates _merge_key()  to be able to merge lists

LGTM


Support for NPM private modules

pnpm install should support npm private modules -- https://www.npmjs.com/private-modules

Closing in favor of #161


Variable name uniqueness issue in inlining (introduced in Numba 0.54)

Reporting a bug

[x] I have tried using the latest released version of Numba (most recent is
visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
[ ] I have included a self contained code sample to reproduce the problem.
i.e. it's possible to run as 'python bug.py'.

mk_unique_var was removed from the line below in Numba 0.54. This is causing variable name conflict issues in Bodo due to heavy use of inlining. Reproducer is not easy to provide but possible if necessary (doesn't seem necessary to me at the moment). What was the reason for removing mk_unique_var?
https://github.com/numba/numba/blob/7f72315f0bf672729b5a6eaf004aa9061e8bf0f5/numba/core/inline_closurecall.py#L398

It was removed in this PR: https://github.com/numba/numba/pull/7031

Reproducer is not easy to provide but possible if necessary (doesn't seem necessary to me at the moment).

Can you provide some more detail about the conditions that cause the problem please?

Variable names of the inlined function clash with the caller.

@ehsantn thanks for reporting this. I think it'd be good to have a reproducer so it's possible to see what is going wrong and from there work out the best way to fix it. A reproducer would also be used as part of the unit tests for any proposed fix with view of preventing similar issues in future. Thanks.
IIRC mk_unique_var was removed in favour of using the scope.redefine to handle https://github.com/numba/numba/issues/6969#issuecomment-840909275.

import numpy as np
from numba import jit, generated_jit, prange

my_jit = jit(cache=True, nogil=True, nopython=True, inline="always")

def factory():

    def impl(col):
        for i in range(col.size):
            col[i] *= 2

    return my_jit(impl)

@generated_jit(cache=True, nogil=True, nopython=True, parallel=True)
def outer(a):

    inner = factory()

    def impl(a):
        for i in prange(a.shape[0]):
            inner(a[i])

    return impl

if __name__ == "__main__":

    a = np.arange(100).reshape(10,10)
    outer(a)

I am not sure if this is identical, @ehsantn. Is the problem in your case also related to prange? I am also running into issues.

The traceback for the above is:
Traceback (most recent call last):
  File "test.py", line 31, in <module>
    outer(a)
  File "/venvs/nbenv/lib/python3.8/site-packages/numba/core/dispatcher.py", line 485, in _compile_for_args
    error_rewrite(e, 'unsupported_error')
  File "/venvs/nbenv/lib/python3.8/site-packages/numba/core/dispatcher.py", line 423, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.UnsupportedRewriteError: Failed in nopython mode pipeline (step: convert to parfors)
Overwrite of parallel loop index

File "test.py", line 10:
    def impl(col):
        for i in range(col.size):
        ^

Is this a MWR (based on the above)?
import numpy as np
from numba import njit, prange

@njit(inline="always")
def bar(x):
    for i in range(x.size):
        x[i] += 1

@njit(parallel=True)
def foo(a):
    for i in prange(a.shape[0]):
        bar(a[i])

a = np.ones((10, 10))
foo(a)

if so, I think the issue is that the inliner renames the variable i in the bar function as a version of i and adds it to the scope. Then in parfors, the failure comes from here:
https://github.com/numba/numba/blob/7f72315f0bf672729b5a6eaf004aa9061e8bf0f5/numba/parfors/parfor.py#L2673-L2684
where the scope is checked and if it is in the scope the unversioned name is returned and this collides with the unversioned induction var name.
Since #7031, in the inliner uses scope.redefine to create a variable with a versioned name and put it in scope, whereas previously the mk_unique_var was creating a variable with a similarly versioned name but does not put it into the scope.
I think the code ends up looking a bit like:
import numpy as np
from numba import njit, prange

@njit(parallel=True)
def foo(a):
    for i in prange(a.shape[0]):
        for i in range(a.shape[1]):
            a[i] += 1

a = np.ones((10, 10))
foo(a)
print(a)

which is valid and fails the same way under parfors.

Thanks for simplifying @stuartarchibald - I threw together that example in a hurry. I am not surprised that that is the source of the error - it is consistent with what I imagined.
Is this to be considered a bug, it is it a user error? I ask because ensuring uniqueness of variable names in complicated code could be challenging.

Thanks for simplifying @stuartarchibald - I threw together that example in a hurry. I am not surprised that that is the source of the error - it is consistent with what I imagined.

No problem.

Is this to be considered a bug, it is it a user error?

It's a bug, but one of those bugs where a decision will have to be made about where to fix it. Parfors should probably just handle this use case, but that is easier said than done. CC @DrTodd13 what do you think?

I ask because ensuring uniqueness of variable names in complicated code could be challenging.

@JSKenyon What's the reason for setting inline='always' in your code? Obviously it should "just work" but if you can get away without using it that'd be a work around for this case. Also, it's entirely unreasonable for Numba to expect users to implement a version of SSA (a bit like unique naming) in their code to ensure "uniqueness" of variables, Numba needs to fix this!!
@ehsantn I'm not convinced that the above is the same as the problem you are seeing?

@ehsantn I'm not convinced that the above is the same as the problem you are seeing?

Apologies if I derailed the issue!

@JSKenyon What's the reason for setting inline='always' in your code? Obviously it should "just work" but if you can get away without using it that'd be a work around for this case. Also, it's entirely unreasonable for Numba to expect users to implement a version of SSA (a bit like unique naming) in their code to ensure "uniqueness" of variables, Numba needs to fix this!!

It is not a roadblock for me at present. The inline='always' was there because in my real code this is about 7 loop levels deep and I wanted to be sure that the function call was elided. This may be an error on my part - I would be lying if I said I perfectly understood the if and when's of inlining.

@ehsantn I'm not convinced that the above is the same as the problem you are seeing?

Apologies if I derailed the issue!

Not at all, thanks for reporting it, I'm fairly sure it's a related problem, but just perhaps not exactly the one @ehsantn can see. I've opened https://github.com/numba/numba/issues/7380 for more discussion.

@JSKenyon What's the reason for setting inline='always' in your code? Obviously it should "just work" but if you can get away without using it that'd be a work around for this case. Also, it's entirely unreasonable for Numba to expect users to implement a version of SSA (a bit like unique naming) in their code to ensure "uniqueness" of variables, Numba needs to fix this!!

It is not a roadblock for me at present. The inline='always' was there because in my real code this is about 7 loop levels deep and I wanted to be sure that the function call was elided. This may be an error on my part - I would be lying if I said I perfectly understood the if and when's of inlining.

I see, there's a reasonable chance that LLVM would inline it anyway. There's some notes on what the Numba inliner does (and what LLVM does) here: https://numba.readthedocs.io/en/stable/developer/inlining.html#notes-on-inlining. Feel free to post on https://numba.discourse.group/ if you've any questions about it. Thanks!

@stuartarchibald your reproducer captures the issue well. I just tried it and adding mk_unique_var fixes the issue. The problem is name conflicts during inlining. Bodo has a lot of IR nodes inlined in various ways in several transformations in general.
Just adding an extra mk_unique_var in this line doesn't seem to hurt anything. Is the intention to break existing code so everyone is forced to use the Scope object properly because it is super critical now?


Fix indentation to make consistent with previous examples

Minor update to fix example in example in README.md

Nice one, thanks a bunch @mattkram 👏


Some cmdlets and config require writing to $PSHOME\SessionConfig folder

Summary of the new feature/enhancement
For Snap and AppX packages of PSCore6, $PSHOME is effectively read-only.  Any configuration stored in $PSHOME needs to be in a different location.  On Windows 10, it would make sense to be in $env:SystemDrive\ProgramData.
Register-PSSessionConfiguration stores the config file in $PSHOME\SessionConfig folder.
powershell.config.json for the system is stored in $PSHOME, this means that admins can't set system wide config and cmdlets like Enable-ExperimentalFeature -Scope AllUsers won't work.
Update-Help -Scope AllUsers also won't work as it's stored in $PSHOME.
When this is fixed, test changes relying on Test-CanWriteToPsHome should be reverted.

Long time Windows exposes UAC virtualization for registry keys and file system.
If we could use it it would solve this problem.

@iSazonov that would potentially work for Windows, but doesn't solve the same problem with Snaps.

I like the way for Windows. I think it is right way.
I guess there is a solution in Snaps too - PowerShell is not single application with such behavior. My quick search https://forum.snapcraft.io/t/snap-layouts/7207 "Creating new files and directories in read-only spaces". So I believe Snaps has that we need.

What parts of snap would we map?  Just because we can, doesn't mean we should.  We should not make any part of $PSHome writable on linux.

See also https://github.com/PowerShell/PowerShell-Snap/issues/32

Can we just use /etc/ for this? Or does Snap "classic" et al not have a way to do this?

@joeyaiello The problem is this should be done in PowerShell. /etc is where the file should be for all PowerShell installations.

For reference #4080

Known locations that write into $PSHOME - In-progress

Item
Location

Register-PSSessionConfiguration
$PSHOME\SessionConfig

Update-Help -scope currentuser
May just be a bug

Update-Help -scope AllUsers
?

Enable-ExperimentalFeature -Scope AllUsers
$pshome\powershell.config.json

Set-ExecutionPolicy -Scope LocalMachine
$pshome\powershell.config.json

Proposal
Rational
It looks like the majority of the problem is writing to the configuration file.  We should move it under /etc for Linux and macOS, and `%programdata% for Windows.
We can make similar changes for $PSHome\SessionConfig and $PSHome\en-US (and related folders) later.
Details
Security
These folders should be secured similarly to the currently location the file is located at when the product is installed via DEB, RPM, or MSI.
The configuration file is system wide configuration and only system administrators should be able to modify it.  This also eliminates pwsh creating it on first launch.
Linux and macOS
We would move the system configuration to /etc/powershell/powershell.config.json
We would use the folder so that any addition items will be group together.
Packages would have to be updated to create this and the zip instruction would have to be update or a script created.
Windows
We would move the system configuration to %programdata%\Microsoft\PowerShell\powershell.config.json
MSI could be update to create the file.
MSIX - @stevel-msft?
zip - instructions would have to be updated
Issues

What do you do on upgrade?  Currently we overwrite the file, but this is not the correct behavior.
Not sure how to set the default config after MSIX install.

Possible solution is to include a script that admins can run after install.  This would simplify zip and tar.gz install as well.

For PowerShell.config.json, it might be better to have cmdlet(s) to manage that file with -Scope AllUsers/CurrentUser.  This would work cross-platform.

Cmdlets is an option. The presence of configs implies supporting direct editing.

@SteveL-MSFT  I don't understand your point at all.  Most of those already exist.  The problem is that the AllUser file needs to be writable.

@TravisEz13 system wide config can exist, but needs to be under ProgramData (on Windows) and requires elevation to set.  My point of the cmdlets isn't for setting individual settings, but something like Get-PowerShellConfig which returns an object representing the config and Set-PowerShellConfig to save it, but with -Scope parameter so the user doesn't need to know where in the filesystem it goes.

We would move the system configuration to /etc/powershell/powershell.config.json
We would move the system configuration to %programdata%\Microsoft\PowerShell\powershell.config.json

This means different versions of pwsh will share the same configuration file, which is problematic. Even with sub-folder named after the version, it means different instances of the same version will share the same configuration file, which is still a breaking change from today's behavior.

@SteveL-MSFT I still don't see how this addresses this issue at all.  I think you have a fundamental mis-understanding of the problem.  That might be a useful new feature, but it doesn't help solve this problem.

@daxian-dbw System wide configuration should be actually system wide, this is a reason to move it into a common folder.  I can see reason to have different files for preview and stable.

@PowerShell/powershell-committee reviewed this, we agree not to take any changes for RC.  We should relnote the limitations and address post 7.0.  The proposal moving the system level config to /etc and %programdata% makes sense.

In last days I make some experiments with the config and new Core Json API (to remove dependency from Newtonsoft Json.NET) and discover some inconsistencies in the config file format and serialization/deserialization. I hope we fix this too in 7.1.

I think we need to make this a priority for 7.1, but particularly implementing the /etc/powershell/powershell.config.json . The scenario is essentially that, as on Windows with the GP store, Unix admins need to be confident that Computer-wide Policy is applied to all PowerShells on a box, irrespective of whether they were bin-dropped as portable copies or installed to XDG-compliant locations via a package.
I can follow up with offline justification if needed

Now .Net Core 5.0 has new enhancements in JSON implementation and this raises a question - will we move on .Net Core 5.0 in 7.1 timeframe and benefit from new .Net Core enhancements and improvements?
PowerShell Core 7.0 LTS is based on .Net Core 3.1 LTS - release in 2020 year
PowerShell Core 7.1 GA is based on .Net Core 5.0 GA - release in 2021 year
PowerShell Core 8.0 LTS is based on .Net Core 6.0 LTS - release in 2022 year

@iSazonov the plan is to move to .NET 5 for 7.1 as soon as they have a public preview.

@SteveL-MSFT - would you release a 7.1 on a pre-release version of .NET core 5?

@doctordns Release 7.0 will be on release .Net Core 5.

https://github.com/PowerShell/PowerShell/issues/9278#issuecomment-487822446
The suggested workaround here does not work with snaps in classic containment (full access to the host)

Not seeing much reference to also being unable to create a global profile.ps1 due to $PSHome being unwritable for Windows.  I found issue #12519 "Unable to create profile.ps1" that was opened about this, and later closed referencing this issue, but never had feedback noted here...  So adding that here now myself as I've also come across the same problems as well..

This issue has been open now for 4 years.  When you go to Microsoft's page for installing Powershell, the very first section is entitled: Install PowerShell using Winget (recommended)
If you do so, the installation is broken because nobody can adjust things like the execution policy, including administrators.
Is this ever going to be fixed?  MS is directly telling everybody to deploy PS using an install which doesn't work and hasn't worked for years.

This issue has been open now for 4 years. When you go to Microsoft's page for installing Powershell, the very first section is entitled: Install PowerShell using Winget (recommended)
If you do so, the installation is broken because nobody can adjust things like the execution policy, including administrators.
Is this ever going to be fixed? MS is directly telling everybody to deploy PS using an install which doesn't work and hasn't worked for years.

Is there any news on this issue? What is the reason for the difficulty in fixing it already?

+1.

Just ran into this issue.  Incredibly annoying to discover this has been a known problem for 4 years and MSFT hasn't bothered to fix it.

Just ran into this issue. ... reinstalled from MSI and now problems.

Just encountered this as well on a new build and installed via the msix package instead of the msi. Windows Store version still prevents changing the execution policy.


Id 221 ship create api skeleton for admin UI

Notes

New admin server ()
Docker-compose for the admin server
makefile updated

LGTM. Just added a request change to unify the middleware, but consider it more a suggestion than a request change. It if it not easy to move, let's go on and we iterate over it later.


[AutoPR hdinsight/resource-manager] [HDInsight] - Update configuration related APIs

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/5477

This PR has been merged into https://github.com/Azure/azure-sdk-for-js/pull/1950


[JN-453] Add new questions to a survey

This adds the ability to add new questions to a survey page. Not included in this PR is the autofill "scratch box" functionality. That will be coming in the next PR.

TO TEST:
Play around with the question designer. Make sure that switching between types works correctly and that the questions are saved correctly by viewing the generated JSON in the JSON Editor, as well as the final result in the Survey Preview tab.

Finally got a chance to check this out - really like how you added all the descriptive helper text -- made it much easier to navigate

It doesn't have to be in this PR, but we really should enforce non-null name/stableIds -- I accidetnally created one with a blank name, and it was almost impossible to find

Going to merge this as-is so Erin can play around with it, but will improve the destructive nature of question-switching in a future PR.


Invalid casts among primitive types

Hi, in some cases casts are invalid. For example in class com/paypal/android/foundation/presentation/model/ConsentUriParams.java type boolean is converted to byte using explicit cast
    public void writeToParcel(android.os.Parcel parcel, int i) {
        parcel.writeByte((byte) this.showConsent);
        parcel.writeByte((byte) this.showOneTouch);

but the following code should be generated instead
    public void writeToParcel(android.os.Parcel parcel, int i) {
        parcel.writeByte(this.showConsent ? 1 : 0);
        parcel.writeByte(this.showOneTouch ? 1 : 0);

APK: https://drive.google.com/file/d/16O--dtXt35zB1Ahx-9qEYvCmyeF1n0AG/view?usp=sharing

@asashour
Was it fixed by the PR? I see that the given test case still fails

I have rechecked from the GUI and now the code generated is:
public void writeToParcel(Parcel parcel, int i) {
    parcel.writeByte(this.showConsent ? (byte) 1 : 0);
    parcel.writeByte(this.showOneTouch ? (byte) 1 : 0);
    parcel.writeString(this.secureIdToken);
    parcel.writeString(this.challengeUri);
    parcel.writeString(this.consentData.toString());
}

I hope you are checking with the PR branch, not with master, since it is not merged yet.

@asashour
Seems to be not completely fixed, file com/paypal/android/foundation/cashin/model/PayPalCashRetailerDetail.java
    public void writeToParcel(android.os.Parcel parcel, int i) {
        //...
        parcel.writeInt(this.lastVisited); // declaration: private boolean lastVisited
    }

Haha, this is funny, in this method cast don't even added, bytecode just use boolean as int.
@sergey-wowwow thanks for sample, I have waited it for a long time ))
I think this issues must be catch by type inference and cast must be inserted and later cast will replaced with ternary using @asashour PR.

@skylot
Always welcome to fix my bugs! :D

I went into TypeInferenceVisitor and have to say I am lost. The logic is mainly to compare types and decide which one to set, but there is no place/example of inserting InsnNodes

In TypeInferenceVisitor already collected all usages of variable, so I think it must be easy to find this case. Also, current type inference specially adjusted for such cases and type errors with primitive types just omitted, this also must be fixed.
And actually there is one place with inserting instruction and case is very similar.

The logic is mainly to compare types and decide which one to set

Haha, this is a definition of "type inference" :

Type inference refers to the automatic detection of the data type of an expression in a programming language.


socket hang up

Hello,
I got a pretty weird problem:
I created this proxy:
axios: {
        credentials: true,
        proxy: true,
        baseURL: process.env.NUXT_PUBLIC_BASE_URL,
        debug: process.env.NODE_ENV && process.env.NODE_ENV === 'development',
    },
    proxy: {
        '/backend': {
            target: process.env.NUXT_PUBLIC_APP_URL, // http://api.test
            changeOrigin: true,
            secure: process.env.NODE_ENV && process.env.NODE_ENV === 'development',
            credentials: true,
            rewrite: (path) => path.replace(/^\/backend/, ''),
            common: {
                'Accept': 'application/json, text/plain, */*'
            },
        },
    },
    ```

Now If I do something like:

return useFetch(/backend${url}, {
server: true,
credentials: "include", // Allow browser to handle cookies
headers,
...opts,
pick: pick
});

I get this error:

http proxy error: Error: socket hang up
at connResetException (node:internal/errors:693:14)
at Socket.socketOnEnd (node:_http_client:478:23)
at Socket.emit (node:events:539:35)
at endReadableNT (node:internal/streams/readable:1344:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
timestamp: true,
error: Error: socket hang up
at connResetException (node:internal/errors:693:14)
at Socket.socketOnEnd (node:_http_client:478:23)
at Socket.emit (node:events:539:35)
at endReadableNT (node:internal/streams/readable:1344:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
code: 'ECONNRESET'
}
}

If I use

return useFetch(http://api.test/${url}, {
server: true,
credentials: "include", // Allow browser to handle cookies
headers,
...opts,
pick: pick
});

everything works like a charm.

same here

same here


2.0.0 beta2

Most of changes below are to make things consistent with the Vapor MySQL counterpart and to help support features in the PostgreSQL Driver.
Changes:

Rename connection parameter from host to hostname. This allows for supporting master and read replica in PostgreSQL Driver.
Rename connected boolean to isConnected.
Rename connection pointer to cConnection.
Rename error var to lastError.
Move all connection tests into the ConnectionTests file.

New:

Add Error class for better error messaging.
Add Context class with isPostgreSQL var.

Codecov Report

Merging #36 into master will decrease coverage by 3.05%.
The diff coverage is 69.2%.

@@            Coverage Diff            @@
##           master     #36      +/-   ##
=========================================
- Coverage   92.65%   89.6%   -3.06%
=========================================
  Files          14      16       +2
  Lines        1879    1962      +83
=========================================
+ Hits         1741    1758      +17
- Misses        138     204      +66

Impacted Files
Coverage Δ

Sources/PostgreSQL/ConnectionInfo.swift
66.66% <100%> (ø)
:arrow_up:

Sources/PostgreSQL/Notification.swift
100% <100%> (ø)
:arrow_up:

Tests/PostgreSQLTests/PostgreSQLTests.swift
96.44% <100%> (+0.21%)
:arrow_up:

Sources/PostgreSQL/Context.swift
40% <40%> (ø)

Sources/PostgreSQL/Error.swift
5.26% <5.26%> (ø)

Sources/PostgreSQL/Connection.swift
79.41% <56%> (-5.86%)
:arrow_down:

Tests/PostgreSQLTests/Utilities.swift
53.57% <85.71%> (ø)
:arrow_up:

Sources/PostgreSQL/Database.swift
84.61% <87.09%> (ø)
:arrow_up:

Tests/PostgreSQLTests/ConnectionTests.swift
87.23% <90.47%> (+7.23%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b11b11...ac659e1. Read the comment docs.

Most of changes below are to make things consistent with the Vapor MySQL counterpart and to help support features in the PostgreSQL Driver.
Changes:
Rename connection parameter from host to hostname. This allows for supporting master and read replica in PostgreSQL Driver.
Rename connected boolean to isConnected.
Rename connection pointer to cConnection.
Rename error var to lastError.
Move all connection tests into the ConnectionTests file.
New:
Add Error class for better error messaging.
Add Context class with isPostgreSQL var.


Bump gradle-plugin-publish to 0.21.0

Codecov Report

Merging #475 (3c59625) into master (5a8cc27) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #475   +/-   ##
=========================================
  Coverage     58.87%   58.87%
  Complexity      861      861
=========================================
  Files            69       69
  Lines          6451     6451
  Branches       1006     1006
=========================================
  Hits           3798     3798
  Misses         2360     2360
  Partials        293      293

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a8cc27...3c59625. Read the comment docs.


Only download last version if you have not already

First... THANK YOU for LastVersion! It is amazing!!
Please close this ASAP as its more a feature request than an issue.
I am trying to keep software updated once per week, but i do not want to download a gigabyte every week to find out half of it is not updated from last week (same version still) is there OR could there be a way too download my list of releases only if it is a new version? (not just last version) Thank you for your work, its appreciated!

Hi @5c0tt-b0t ,
It depends:
If you "install" an RPM asset from GitHub, using lastversion install <reponame> then it should not download something in vain. It will consult the RPM database, will check what is the current version of the thing you installing, and skip downloading if it's not newer on GitHub. This is a bit limited because it makes the assumption that package name equals <reponame> . Plus, you have to install lastversion itself via RPM so it has access to RPM Python modules. So this is quite limited.
E.g. if you do this in CentOS 8:
sudo yum -y install https://extras.getpagespeed.com/release-latest.rpm
sudo yum install lastversion
...
sudo lastversion install mailspring

Then it will work as intended. It finds Mailspring on Github, finds its .rpm assets, checks if "mailspring" package is installed, and see if it's older than on Github, before downloading.
If you're not using RPM-based install of lastversion or not downloading .rpms with it, then this won't work and you have to script whichever way to compare what version you already have before downloading.
You'll use something like this.
Just need to store the "last downloaded version" somewhere, e.g.
EXISTING_VER=$(cat version.txt)
LASTVER=$(lastversion repo/owner -gt ${EXISTING_VER})
if [[ $? -eq 0 ]]; then
  # LASTVER is newer
  lastversion download repo/owner
  echo $LASTVER > version.txt
fi

This uses -gt switch which compares what's latest to a given version, e.g. lastversion repo/owner -gt 1.2.3 will return 0 exit code if the version on GitHub is newer than 1.2.3.


Further plans for project

I like the way you are creating this project, standards should be a base for further app modules.
I have a few questions:
Are you going to implement a parsing of files for certain logic objects, like tasks?
Will there be some kind of inline math (mathjax, latex-like)?
Will there be some kind of native save-file - link it support (for example a science article in pdf, that you annotated in note will be saved in some attachments directory?
Will there be some kind of highlight tool for any kind of documents?
I am asking those questions because I want to help with project development.
What is your roadmap for this project at this moment?

I like the way you are creating this project, standards should be a base for further app modules.

;)

Are you going to implement a parsing of files for certain logic objects, like tasks?

Most likely. It's been on my todo list for a while, especially since the addition of the calendar widget and deterministic periodic notes. I'm not entirely sure of what it will look like. Happy to discuss ideas.

Will there be some kind of inline math (mathjax, latex-like)?

Not currently planned.

Will there be some kind of native save-file - link it support (for example a science article in pdf, that you annotated in note will be saved in some attachments directory?

Almost definately! I was working on assets support before I got side-tracked with the recent keyboard-layout issues (DE, RU). I'll post more information about this soon - would love your feedback and ideas.

Will there be some kind of highlight tool for any kind of documents?

Hmmm... not sure. Are you thinking about something minimal or more in the lines of zotero ?

I am asking those questions because I want to help with project development.

That's awesome! Any help would be greatly appreciated.

What is your roadmap for this project at this moment?

I don't have a publicly accessible roadmap. I'm currently tracking everything in my own notes (managed by Notesium). I should probably make one.

Thanks for your reply.
Objects
Tasks
First of all I think it is very helpful that markdown (by default don't tell - [ ] is a standart tasks syntax) because:

it is not easy to use if you are not familiar with modal editing (in vim or helix just r to x between [ ])
you should always watch that there is space between [ ]
it is not easy to understand if you are doing task or done by one char (this is for source editing)

Alternatives to - [ ]

org-mode inline tasks

Personally used in Logseq, easy to read, easy to modify. In my opinion, complex syntax for bare-hand writing.
Will not recomend to use as a standalone task without shortcut infrastructure around it.
Have multiple line syntax.
I haven't seen many mobile apps for this syntax.

todo.txt

Easy to read, easy to parse. Easy to add personal completion state.
Many dedicated apps (android, linux, windows, ios).
May be a little too long to read in plain-text editor.
There is a complete, stabilized format that is not changing very long.

plain markdown BOLD status for task

Easy to write, read. Easy personalization of tasks.

obsidian dataview tasks

I just don't think that tasks should be marked with any kind of emoji, because it is not always easy to write them from cli, so to use it with cli apps.
I love how it look when it is parsed. It is easy to read in this way.

My thoughts
I think that by default there should be todo.txt tasks. All other formats should be able to be implemented by some kind of side-parser.
I don't completely know how it is better to implement but here is the idea. There is a binary that can parse specialized type of object (org-mode tasks) and output it in stdout (json-format, newline, ... idk)
Then app just should merge those tasks in some kind of unified table with tags from where did it come. (Parse of all notes dir or just one file, that was specified)
User that can write a simple rule, what task he want: urgent and important, importand and not on time... etc.
This way, I think may be implemented any kind of object parsing. But there should be a BASE what should be always parsed first.
As for me, I found out that I only need several tools for note-taking and productivity:

Google calendar (without tasks, they are bad in Gcal)

just for basic time management, not task planning, because task planning is always wrong

Basic notes app (for jotting lecture notes or my personal ideas)

here I have Projects, Areas, Resources and Archive
Projects and Resources sometimes merged.
Projects have most of tasks.
Project always have a lot of external links. External pdfs, in what I highlight something.
Archive is just a bunch of old files that is currently not useful in any area of life.
Areas sometimes have tasks too.

Tasks app where I sort tasks by four categories:

urgent and important
not on time and important
urgent and not important
not on time and not important

As you can see there only four objects that I definetly use:

Calendar date
Task
Any kind of external file
Link (about it later)

I don't know what any user definetly need. Those are my needs. I'd like to see parsing of them in app by default.
Links
I'had a lot of links to old sites in my old Logseq database. Now a lot of small cool sites are gone and info from them too.
I think there is only way to solve this problem - saving such info by yourself. Any site can change. Your notes should refer something that you meant when was writing.
I've searched a lot for easy to use tools with such ability, but found only several:

omnivore
zotero
logseq pdf referencing
But all of them lack of minimalisticity - i just need 3 formats to export and highlight into:
pdf (standart, will be always easy to read, not something like doc wich will not be readable in libreoffice and even newer word)
image (vector and rastor)
plain-text (may be basic markdown, not some fancy javascript based it implementation)

I just need ability to save anything and then just add my thoughts to any part of that info i saved.
Zotero is not easy to use and in my opinion, more specialized (academic).
Links (like object) should refer:

date when it was done
place - local file or just link to a site
context - relative place in whole place what you are talking about (optional)

Currently I am just thinking of how it will be implemented in general way, Do one thing, and do it well.
How it should work with external tools? I don't currently know. I think this should be external, but easy to implement in other tools.
However, basic []() markdown link is a must-have in terms of fast and easy-to-use writing. My teacher always said: Every tool should do one thing right—not everything poorly..


Mono No errors reported for async functions, game continues

Godot version:

c6b587636b1e3cf27e566e6e5b10859316cbefb6
OS/device including version:

Ubuntu 18.10, Mono 5.16
Issue description:

When an async function fails to execute due to an error, it will not report the error to the console, and will not stop execution of the game.
Steps to reproduce:
Do something erroneous in an async function; observe that your mistake is not reported.
using Godot;
using System.Threading.Tasks;

public class TestScript : Node
{
    public override void _Ready()
    {
        AsyncFunc();
    }

    private async Task AsyncFunc()
    {
        GD.Print("Running...");
        var myArray = new int[3];
        myArray[4] = 0; //Whoopsie-doodle!
        GD.Print("Done.");
    }
}

Minimal reproduction project:

NoAsyncError.zip

I think this is inherent to the Task library. You're not actually waiting for the Task so there's no place the exception would actually be reported.
If you change your example code in this way:
AsyncFunc().Wait()
Then it'll actually report the underlying exception.


The connection is unstable.

I am using ESP32 in AP mode to create an FTP server. When connecting with FileZilla software and transferring images, the connection gets interrupted midway.

What about STA mode?

@nopnop2002 I'm also facing the same issue,
In STA mode. It's working perfectly, but in AP mode the connection is getting interrupted.
Files are transferring after the forceful disconnection from FileZilla

After the connection established, waiting for welcome message line, none of the files are transferring, after the force disconnection, it's transferring files but not all the selected files, few files were missing.

@iamnarendrans
What kind of logs are displayed on the FTP server side?

I was unable to reproduce the problem using AP mode + FileZilla, so I will close it.


Updating Kong ACL Plugin with HTTP PATCH method doesn't working as expected?

Components versions

Kong: 1.1.2

Summary
Using HTTP PATCH to update a service ACL groups plugin causes all groups being replaced by the contents of the request payload, whereas it was expected the new groups only be appended to the existent group list.
Steps to reproduce

Create the ACL plugin in a service  with two groups in the while list:

$ curl -X POST http://kong:8001/services/{service}/plugins \
    --data "name=acl"  \
    --data "config.whitelist=group1" \
    --data "config.whitelist=group2" \
    --data "config.hide_groups_header=true"

Update the white list with a third group using a HTTP PATCH request:

$ curl -X PATCH http://kong:8001/services/{service}/plugins/{acl-plugin-id} \
    --data "config.whitelist=group3"

Actual result (the first two groups were replaced by the third group):

$ curl -X GET http://kong:8001/services/{service}/plugins/{acl-plugin-id}

{
...
"config": {
    "hide_groups_header": true,
    "blacklist": null,
    "whitelist": [
      "group3"
    ]
  }
...
}

Expected result (third group being appended to the list):

$ curl -X GET http://kong:8001/services/{service}/plugins/{acl-plugin-id}

{
...
"config": {
    "hide_groups_header": true,
    "blacklist": null,
    "whitelist": [
      "group1",
      "group2",
      "group3"
    ]
  }
...
}

Is that the right behavior?
Thanks!

The current behavior is expected. I'll mark this as a feature request.

For JSON we could implement merge_patch but for other content types, there is no standard.


letsencrypt-nginx-proxy-companion is removing out of its context symlinks in /etc/nginx/certs

I'm not able to have a mixed configuration with domains that are under letsencrypt-nginx-proxy-companion management and domains that are not, having in mind that jwilder/nginx-proxy is the same and using the same certs dir for all domains. What's happening is that letsencrypt-nginx-proxy-companion is removing my custom certificates since it considers that they are unsed.
log:
letsencrypt-nginx-proxy-companion    | Symlinked domains: yyy.com xxx.com
letsencrypt-nginx-proxy-companion    | Enabled domains: xxx.com
letsencrypt-nginx-proxy-companion    | Disabled domains: yyy.com
letsencrypt-nginx-proxy-companion    | Some domains are disabled. Check them to remove unused symlinks.
letsencrypt-nginx-proxy-companion    |
letsencrypt-nginx-proxy-companion    | Checking domain yyy.com:
letsencrypt-nginx-proxy-companion    | Checking yyy.com.crt - removing.
letsencrypt-nginx-proxy-companion    | Checking yyy.com.key - removing.
letsencrypt-nginx-proxy-companion    | Checking yyy.com.dhparam.pem
letsencrypt-nginx-proxy-companion    | Checking yyy.com.chain.pem - removing.
letsencrypt-nginx-proxy-companion    | Unused domains checking is finished.

docker-compose.yml:
services:
  letsencrypt_nginx_proxy_companion:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: letsencrypt-nginx-proxy-companion
    volumes_from:
      - load_balancer
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "${LETSENCRYPT_CERTS_PATH}:/etc/nginx/certs:rw"
    environment:
      - NGINX_DOCKER_GEN_CONTAINER=load_balancer
  load_balancer:
    image: jwilder/nginx-proxy
    container_name: load_balancer
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./docker/load-balancer/nginx-proxy-vhost.d:/etc/nginx/vhost.d
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /usr/share/nginx/html
      - "${LETSENCRYPT_CERTS_PATH:/etc/nginx/certs:ro"
    labels:
      com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
    environment:
      - ENABLE_IPV6=true
    restart: always
  website:
    container_name: website
    image:  ${IMAGE_WEBSITE}
    ports:
      - 5555:80
    environment:
      - VIRTUAL_HOST=xxx.com
      - LETSENCRYPT_HOST=xxx.com
      -<EMAIL_ADDRESS>  custom_certificates_website:
    container_name: custom-certificates-website
    image:  ${IMAGE_CUSTOM_CERTIFICATES_WEBSITE}
    ports:
      - 5556:80
    environment:
      - VIRTUAL_HOST=yyy.com,*.yyy.com

Originally posted by @ejbp in https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/issues/319#issuecomment-437718806

Can you describe how do you organize your custom certificates in /etc/nginx/certs ? Exactly like the LE companion does (a folder named after the certificate's base domain and symlinks pointing to the cert and private key inside) ? Please try to be as precise and detailed as possible.
Are they obtained through Let's Encrypt ? I guess they are.

Okay.
So the issue at hand is that the LE companion can't really discriminate the certificate it creates from other obtained through another method when they use a similar structure (symlinks to files in a subfolder).
The best solution I can think of right away would be to include a dot file next to the certs handled by the companion during creation / renewal (most importantly before the symlink cleanup function). Then the symlink cleanup function check for the presence of this dot file. If it's not there, those files are not handled by the companion, don't touch the symlinks.
Does that seem sound to you ?

I can't think in a better solution. Probably is what you have in mind but that dot file should be inside each directory not in the root (near the sym link) for better user visualization of the root dir.
Though, in the moment of transition from current version to this one, some symlinks may stay zombies if:

Docker with current version is running
Stop the docker and update it
Remove an existent companion managed host
Launch the docker with the new image

In my opinion is not a reason to implement a more complex and eventually worst solution so I would advance with this one.


Workflow for contributing to a vendored package

Hi
The vendored packages don't seem to carry the .git metadata from the upstream.
I was wondering workflow do you use when package has a bug and I fix it locally (inside the vednored folder) and want to make a pull request.

Personally I do this:

Delete the package from my vendor folder temporarily. I just leave the dep config files alone and don't run dep ensure for a bit.
Work on the package in my GOPATH so that it's picked up automatically by the go tools.
Once my fix is upstream, or I'm ready to vendor my fork, I tweak my manifest to either use the fork by running dep ensure, or update to the latest version upstream by running dep ensure -update X to repopulate my vendor.

Essentially I would recommend against ever working in your vendor directory since dep will overwrite any changes. It's too easy to lose work that way.

Closing this since we've updated the FAQs in #755 to include an answer to this question.


Unable to send command in serial connection

I have installed the images onto two NodeMCU ESP8266s
The first configured as an AP, and works fine
The second I tried to configure an an Repeater with automesh
That one however never seems to work.
Connecting with a serial connection I see this information, repeated every 2-3 seconds
Also, no ability give any command (like factory reset) I cannot even type anything. That also woks fine when connected to the AP installed one. (using Putty as client)
Any idea what is wrong and how to correct it? Reflashing did nothing.
--------- Start serial data ------------
ets Jan  8 2013,rst cause:4, boot mode:(3,7)
wdt reset
load 0x40100000, len 1328, room 16
tail 0
chksum 0x12
load 0x3ffe8000, len 604, room 8
tail 4
chksum 0x34
csum 0x34
rBoot v1.4.2 -<EMAIL_ADDRESS>Flash Size:   32 Mbit
Flash Mode:   QIO
Flash Speed:  40 MHz
Booting rom 0.
▒▒▒g▒s▒▒o|▒
l
lllc▒▒|{▒l▒g▒▒g▒d▒▒{▒l▒l▒
lXH▒H▒▒▒Repeater V2.2.14 starting
running rom 0
Config found and loaded (3216 Bytes)
---------End -----------

I have a very similar problem, I have a NodeMCU based on ESP8266MOD. I can't send any commands to the chip. Also, the wifi network is working but connecting to <IP_ADDRESS> times out. Hopefully, we'll get some attention.

Excuse me, why do I always report errors with this project compiler?

Corrupted binary? Did you download the zip? HW Problem? Different flash modul?
No other idea - obviously, the binary works for nearly all of the downloaders....

There is no problem with the binaries. I want to compile this project, but it always reports an error.

Which error? Which compile environment?

The compiler software I use is ESP_IDE .The .h file is always missing when the project is compiled。

I compile my project exactly right, and the official routine is ok.

This esp_open_sdk works fine for me: https://github.com/xxxajk/esp-open-sdk

Thanks. I'll give it a try

Hello Martin.
For my problem I downloaded the binaries, and they work just fine. I have used them to flash one other ESP with no problem, that one works just fine.
My problem seems to be that I cannot erase the configuration I set on the problem ESP. Not by connecting to it with serial, as it does not accept input. And reflashing does not work either as it just take up the config it had before.
I think I have read som comment that the configuration is stored in a different part of the memory that is not overwritten by flashing the new binary, but I am not skilled enough to know how to find and reset that part.
But if I could get help with that I have a way to start from zero and test further.

Use the flash tool to erase the complete memory.

Thank you for the prompt answer.
I had used a different flash tool, as I had issues with the one you recommended, and that tool had no ERASE button.
When I got the recommended tool working and Erased then flashed again, then it works brilliantly.
Thanks again for the help
Please categorize this problem with Error Code 60 (problem located 60 cm from the screen) :D


Ensure tickets are either immature or live

There is currently a primitive check in /feeaddress to ensure a ticket is not older than TicketMaturity+TicketExpiry, but we should be able to validate tickets a bit more carefully.
We should not issue fee addresses, or broadcast fee transactions, for tickets which are expired, missed or revoked.

Discussed with @davecgh and this is more tricky than expected. It seems like we cannot accurately determine if a ticket has already voted or not. I think the best vspd can do is check:

Ticket is not expired
Ticket is not missed and unrevoked

We cannot detect tickets which are missed and revoked, or tickets which have already voted. This will have to be properly documented and validating the tickets will have to be the responsibility of clients.


Figure out the causation in calculating the lateral forces

The lateral forces look like:
$F_y = F_z(\bar{u}, \bar{q})*\alpha(\bar{u},\bar{q})$
and the equations of motion are then:
$\mathbf{M} \dot{\bar{u}} = f(\bar{u}, \bar{q}, F_y)$
but to calculate $F_z$ we end up with auxiliary equations that are of this form: $F_z = f(\dot{\bar{u}}, \bar{u}, \bar{q}, F_y)$ and to solve for $F_z$ you typically would append this equation to the dynamical equations and solve them simultaneously because they are linear in $\dot{\bar{u}}$ and $F_z$. But to solve them simultaneously you need $F_y$, so there is a circular causation. It implies that we have to iteratively solve for $F_y$ at each time step, but I don't think that should be the case.
I'm currently formulating $F_y$ numerically instead of specifying it as a symbolic function. If I replaced the symbolic $F_y$ with the first equation above then I would eliminate $F_y$ from the symbolics. That seems like it would solve it. But I'm not sure why I can't manage that numerically.

I wrote the lateral forces as symbolic functions of the vertical tire forces. This means you only have the vertical forces as specified in the dynamic equations. The equations should also be then linear in the vertical force. Then you can solve for all the u' and vertical forces simultaneously with the dynamic equations and the auxiliary equations.

This is not an issue anymore, but could arise if I want to be able to supply purely numerical functions for the lateral forces and self-aligning moments. We can also calculation the normal forces when the bike is static and just make the slip coefficients constant instead of time varying with normal force (a common simplification).


rs-server not work

Required Info |
-- | --
Camera Model | D430
Firmware Version | realsense-viewer not detecting camera
Operating System & Version | Ubuntu 16
Kernel Version (Linux Only) | 4.15.0-142-generic
Platform | NVIDIA Jetson Nano
SDK Version | Tried v2.50.0, v2.47.0, v2.49.0
Segment | Robotics

I want to use a network connection  to use RS-server
on Ubuntu  build
cmake ../ -DFORCE_RSUSB_BACKEND=true -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true -DCMAKE_INSTALL_PREFIX=../Cinstall -DBUILD_NETWORK_DEVICE=ON
run  rs-server
./rs-server
Rs-server is running
usb realsense-viewer  add network device ip <IP_ADDRESS>
Received error
[getStreams] error:404 Stream Not Found 404

@yuxue521 Did you follow the steps in https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-2-6-setting-static-ip ？

@RealSenseSupport  Thanks very much for your assistance with the question by @yuxue521   :)

@RealSenseSupport  I don't have PI4 I just want to test the network under Ubuntu. I tried local IP (<IP_ADDRESS>) and <IP_ADDRESS> failed

Hi @yuxue521  Are you still experiencing this problem and require further assistance, please?  Thanks!

@MartyG-RealSense I need, i haven't succeeded

I am not familiar with the IP address connection method that you used: usb realsense-viewer add network device ip <IP_ADDRESS>
I note that you are not using a Raspberry Pi 4.
Is rs-server installed on a non-Pi remote computer that the camera is attached to, and you are connecting to that computer with ssh from a central host computer?

@MartyG-RealSense Both rsserver and realsense-viewer work on the local PC, I was to test the rsserver network, If it succeeds, other devices should be able to connect.

rs-server is intended to be built on a second Ubuntu-equipped computer / computing device so that a camera attached to that computer can be communicated with over a network connection from a main computer (the central 'host') that has realsense-viewer installed on it.
For example, the open-source ethernet networking white-paper describes a setup where there is a central host PC and a remote computer (Pi 4) that has the camera attached to it and rs-server installed on it.
https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras

If only one computer (the local PC) was being used then there would be no need for setting up the camera as a Network Camera, as the camera could be accessed by realsense-viewer directly thorugh the USB cable.
Perhaps I am misunderstanding something about your one-computer networking project.  If I am then please provide more information.  Thanks!

For the convenience of testing, I use my local PC for testing. Why doesn’t this work? I tried to use two ubuntu PC and the test also failed.
I think his working principle should be to obtain video data from USB and forward it to the network (rtsp/rtp) through LIVE555. For the network, it should not be distinguished between the local machine and other PC. Isn't that the case?

I see that at the top of this case the camera model is listed as D430.  Is that correct please?  If it is then you would not be able to stream RGB video from a D430 as that model (a caseless Depth Module Kit) does not contain an RGB sensor.

@MartyG-RealSense Yes, I use D430, how do I use Z16 to RGB or Y8 / Y16 to RGB, have a document?

The D430 does not have RGB available from the left infrared sensor either (the D415 and D450 / D455 models do).  I therefore cannot see any way for you to obtain RGB unless you use a second camera to provide the RGB source and sync it with the RealSense camera.
You could obtain a monochrome RGB-like image from the left infrared sensor (Y8) if you minimized the Laser Power setting or turned the projector off in order to make the IR dot pattern disappear from the image.  It would not be a true RGB format image though.

Thank you, I understand


ARROW-16700: [C++][R][Datasets] aggregates on partitioning columns

This updates the Scanner node such that it will use the guarantee expression to fill out columns missing from the dataset but guaranteed to be some constant with appropriate scalars, rather than just inserting a null placeholder column. In case both are available, the dataset constructor prefers using the scalar from the guarantee expression over the actual data, since the latter would probably be an array that unnecessarily repeats the constant value.
This is the other part of what was uncovered while analyzing ARROW-16700, the more direct cause being a duplicate of ARROW-16904 (see also https://github.com/apache/arrow/pull/13509 for my fix for that).

I confirmed that the C++ unit tests validate the fix provided in this PR. Still working on an R test that correctly validates the fix.
https://github.com/apache/arrow/blob/6592ad2472bad6f268448cdb9a0e6c2c0dd4a6e4/cpp/src/arrow/dataset/scanner.cc#L911-L916

I'm sorry but I believe I have sent you on a wild goose chase.  I had suggested an R test but, upon re-reading the JIRA, realize that I had forgotten this is not reproducible from R (because it always inserts a project node prior to an aggregate node) and so no R test is needed.


feat: pass deserialization exceptions in gateway event handler to loop exception handler

Summary
Changes DiscordWebSocket.received_message to pass payload deserialization exceptions to the running loop's exception handler instead of letting them propagate up and crash the shard/bot.
This prevents bots from just outright crashing if an event couldn't be parsed;
Ideally, deserializing the payloads should always work, but history has shown that unannounced API changes unfortunately happen from time to time (even if accidental, like text-in-voice in December :^) ), and the library can't always perfectly account for that.
note that this is really more of a proposal, feel free to suggest alternative implementations or ideas
Checklist

[x] If code changes were made, then they have been tested

[ ] I have updated the documentation to reflect the changes
[x] I have formatted the code properly by running task lint
[x] I have type-checked the code by running task pyright

[ ] This PR fixes an issue
[x] This PR adds something new (e.g. new method or parameters)
[ ] This PR is a breaking change (e.g. methods or parameters removed/renamed)
[ ] This PR is not a code change (e.g. documentation, README, ...)

Not exactly sure which error handler this should redirect to, eg should an on_error be dispatched, or if this should have an opt-out option as someone might have code reliant on the old system.

Not exactly sure which error handler this should redirect to, eg should an on_error be dispatched,

Dispatching on_error is difficult since we'd need to await it, blocking further gateway event handling. Running it in a separate task isn't an option either since you wouldn't be able to access the exception in the task using sys.exc_info like usual.

or if this should have an opt-out option as someone might have code reliant on the old system.

I can't really think of anything that would break due to this change, but I'm open to marking it as breaking just in case

I ended up causing an issue where parse_message_create event could not deseralise the message. This would cause every single message to cause an exception, and by default I don't think that's desired behavior.

That still seems better than outright crashing the bot on the first exception, you're not going to have a good time either way :p

What we might be able to do is add an argument for controlling the behavior here: default to off and opt-in, and in the future we can default this to True. Notate in the documentation this is not finalised and could change between releases.

Adding a toggle for this sounds good, though it should be opt-out (mainly for lib dev, I suppose?) instead of opt-in, since the goal of this was to have most/all clients protected from unannounced breaking API changes (like the automod change a few days ago) or other mishaps.

*It gets worse in that some exceptions in an event are fine to continue but others are not.

Could you elaborate on that? Not quite sure which exceptions this is about.

I still don't see a toggle :P

*It gets worse in that some exceptions in an event are fine to continue but others are not.

Could you elaborate on that? Not quite sure which exceptions this is about.

This is about some errors in some bots without intents, mostly issues that @Skelmis has discovered: we shouldn't hide that messages will error their creation, but I'm not entirely sure. It is possible to glitch the bot out in a way that will error on every message creation, and at that point I believe the bot is unstable to continue to operate. However, your mileage may vary, and this could be a beneficial addition. We could add it, make it opt out by default and change it to opt in if we get reports (but honestly I don't think we will).

I personally feel it's better to raise an error on every exception vs crashing the shard/bot. This is because, while it's a pain to have that many exceptions it would still allow other aspects of the bot to function vs outright crashing.
N.b. this also comes into affect in environments where a developer has say, automated docker restarts. Personally I use them for if the server goes down n comes back without me noticing, but in the case of an exception resulting in a process restart that would very quickly eat into the session start limit which is not something good.

I personally feel it's better to raise an error on every exception vs crashing the shard/bot. This is because, while it's a pain to have that many exceptions it would still allow other aspects of the bot to function vs outright crashing.

That makes sense, but I'm concerned some developers might ignore or miss them if they have other logs enabled. However, these concerns could be alleviated by adding a default logging configuration that doesn't require the user to set their own loggers if they want to use the default.

N.b. this also comes into affect in environments where a developer has say, automated docker restarts. Personally I use them for if the server goes down n comes back without me noticing, but in the case of an exception resulting in a process restart that would very quickly eat into the session start limit which is not something good.

Exponential back-off is your friend!

You could possibly send the exceptions to stderr rather then on_error? To ensure they show up.
Although possibly a new event could be warranted, on_critical_error where: This event is dispatched for errors originating outside of the control of the library.
..note
Ideally you should never need this event as it is reserved for serious issues such as API regression or plausible process crashes.
And exponential backoff how? It's a full container restart. What your suggesting appears to be something I'd need to implement in process

You could possibly send the exceptions to stderr rather then on_error? To ensure they show up.

That's fair, we need to redo some of our logging and printing anyways.

Although possibly a new event could be warranted, on_critical_error where: This event is dispatched for errors originating outside of the control of the library.

That might be useful, but perhaps on_gateway_error is a better name. @shiftinv thoughts?

..note Ideally you should never need this event as it is reserved for serious issues such as API regression or plausible process crashes.
And exponential backoff how? It's a full container restart. What your suggesting appears to be something I'd need to implement in process

This was a mention about your set-up, its not a great idea to always restart on an error although I can't really comment much on that since I used to do that.

I think it's better to catch critical errors instead of crashing the client, because if some developers encounter a regularly appearing critical error they won't be able to ignore it unless they mokeypatch the lib internals. Speaking of an event for this - yes, we should provide that, because this will allow bot developers to decide whether they want to do some specific actions on each critical error (e.g. logging out or modifying the even payload).

You could possibly send the exceptions to stderr rather then on_error? To ensure they show up.

That's fair, we need to redo some of our logging and printing anyways.

Although possibly a new event could be warranted, on_critical_error where: This event is dispatched for errors originating outside of the control of the library.

That might be useful, but perhaps on_gateway_error is a better name. @shiftinv thoughts?

..note Ideally you should never need this event as it is reserved for serious issues such as API regression or plausible process crashes.
And exponential backoff how? It's a full container restart. What your suggesting appears to be something I'd need to implement in process

This was a mention about your set-up, its not a great idea to always restart on an error ~although I can't really comment much on that since I used to do that.~

Technically our current restart policy is None due to the issue we've faced, but the point stands. Not all users will run a perfect setup so ideally the library would at-least provide some form of assistance in that regard.

I think it's better to catch critical errors instead of crashing the client, because if some developers encounter a regularly appearing critical error they won't be able to ignore it unless they mokeypatch the lib internals. Speaking of an event for this - yes, we should provide that, because this will allow bot developers to decide whether they want to do some specific actions on each critical error (e.g. logging out or modifying the event payload).

I agree with this, monkey patching library internals just to try stop our bot crashing wasn't exactly a highlight of the week.
As to what kind of event, I still think it should be a new event which is dispatched (and prints to stderr by default) but after that I feel that would be enough from the libraries side in terms of prevention. Anything further and its probably just a case of the end developer providing logs to Disnake in order to fix upstream if its a Discord issue, otherwise the developer can choose to handle it as they please

Although possibly a new event could be warranted, on_critical_error where: This event is dispatched for errors originating outside of the control of the library.

That might be useful, but perhaps on_gateway_error is a better name. @shiftinv thoughts?

The loop's default exception handler would always call logger.error, so it'd show up even if logging isn't configured, but a separate event is still a good idea.
It's implemented in https://github.com/DisnakeDev/disnake/pull/401/commits/9fd26d24f3559ffd830bf8699e29030a133347da, still need to make the entire thing configurable though.


Provide a way to coordinate services

This is more of a discussion issue for now, and a braindump of some ideas that came to mind after noticing about #1. Once reading a whole configuration from the command line is supported, there a few more things that can be done without much effort — note how things build on each other in small incremental steps:

Allowing using dmon --config as a shebang. Maybe even with a different name like dservice by checking argv[0]. Then service descriptions can be contained in a file instead of having to specify every option in the command line.
Providing a named Unix socket for controlling the instance.
Accepting sysvinit-style script arguments. For example, for a “script” with the dservice shebang at /etc/dservice/ntpd, the following should be possible:

/etc/dservice/ntpd status
/etc/dservice/ntpd start
/etc/dservice/ntpd stop
In  general, dservice <servicefile | servicename> <command> [args...] will be equivalent to <servicefile> <command> <args...>.

Support deriving certain settings from the name of the configuration file, for example if a file is called /etc/dservice/ntpd:

PID file: /run/dservice/ntpd.pid
Control socket: /run/dservice/ntpd.socket
Destination of log files:

In dlog single-file mode: /var/log/ntpd.log
In drlog rotated-dir mode: /var/log/ntpd/*
In dslog syslog mode: service ntpd in daemon log

Others?

Service dependencies: A possible command over the control socket would wait-start: a process writes the command to the socket, and then waits for a message, which is sent once the process is known to have started or failed.

Multiple dependencies? Just open and do wait-start on their control sockets: the operation is idempotent, so for already-started services a response is generated immediately.
How would stopping services with dependencies work? For example, having a service ntpd depending on networkmanager raises the following doubts:

Does stopping ntpd also stop networkmanager if no other service needs it?

Some services may continue to run even if they are not required anymore by other services. In the example networkmanager should continue working so the users of the computer can continue accessing the network.
Maybe it makes sense to have “explicitly enabled” services (is “explicitly stopped” services better naming?).

Does stopping networkmanager also stop ntpd because the latter cannot work without a network connection.

Some services may continue working even after some of their dependencies stopped. Maybe specifying those dependencies does not make sense, or the way to go os differentiating between “soft dependencies” and “hard dependencies”.

Emulate sd_notify() to a certain degree.

The dservice command is a great idea and great name.
In the part of service dependencies, I think that the better solution is use soft and hard dependencies to manage stop. You can use soft-depend and hard-depend as command, or wants and requires as systemd.


[Bug]: MySQL 8.0.28 Container Fails to Start with Testcontainers 3.8.0

Testcontainers version
3.8.0
Using the latest Testcontainers version?
Yes
Host OS
Windows/Ubuntu
Host arch
x86/ARM
.NET version
8.0
Docker version
Docker Desktop 4.28.0 (139021)

Docker info
❯ docker info
Client:
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1-desktop.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.6-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.24
    Path:     C:\Program Files\Docker\cli-plugins\docker-debug.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.22
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-feedback.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.0.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.5.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: <IP_ADDRESS>-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 15.48GiB
 Name: docker-desktop
 ID: 231921ac-6bb9-409e-b0d6-ae72b231b261
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  <IP_ADDRESS>/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

What happened?
When attempting to start a MySQL 8.0.28 container using Testcontainers 3.8.0, the operation fails with a Docker.DotNet.DockerApiException, indicating that the container is not running. This issue occurs during the execution of integration tests, specifically when trying to establish a connection to the MySQL container.

I'm including a simplified test scenario that demonstrates the problem. The test attempts to start a MySQL 8.0.28 container and then perform two operations: opening a database connection and executing a simple SQL script. Both tests highlight the failure to start the container, as indicated by the Docker.DotNet.DockerApiException.
namespace Testcontainers.MySql;

using MySqlConnector;
using System.Data;
using System.Data.Common;

public class MySqlContainerTest : IAsyncLifetime
{
    private readonly MySqlContainer _mySqlContainer;

    public MySqlContainerTest()
    {
        _mySqlContainer = new MySqlBuilder().WithImage("mysql:8.0.28")
                                            .Build();
    }

    public Task InitializeAsync()
    {
        return _mySqlContainer.StartAsync();
    }

    public Task DisposeAsync()
    {
        return _mySqlContainer.DisposeAsync().AsTask();
    }

    [Fact]
    public void Should_Connect_When_CreateConnectionToContainer()
    {
        // Arrage
        using DbConnection connection = new MySqlConnection(_mySqlContainer.GetConnectionString());

        // Act
        connection.Open();

        // Assert
        Assert.Equal(ConnectionState.Open, connection.State);
    }

    [Fact]
    public async Task Should_RunSuccessful_When_ExecScript()
    {
        // Arrage
        const string scriptContent = "SELECT 1;";

        // Act
        var execResult = await _mySqlContainer.ExecScriptAsync(scriptContent)
                                              .ConfigureAwait(true);

        // Assert
        Assert.True(0L.Equals(execResult.ExitCode), execResult.Stderr);
        Assert.Empty(execResult.Stderr);
    }
}

Relevant log output
2024-03-20 11:58:05 2024-03-20T10:58:05.673661Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.28) initializing of server in progress as process 42
2024-03-20 11:58:05 2024-03-20T10:58:05.684913Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-03-20 11:58:06 2024-03-20T10:58:06.012863Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2024-03-20 11:58:07 2024-03-20T10:58:07.036748Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2024-03-20 11:58:05 2024-03-20 10:58:05+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
2024-03-20 11:58:05 2024-03-20 10:58:05+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-03-20 11:58:05 2024-03-20 10:58:05+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
2024-03-20 11:58:05 2024-03-20 10:58:05+00:00 [Note] [Entrypoint]: Initializing database files
2024-03-20 11:58:09 2024-03-20 10:58:09+00:00 [Note] [Entrypoint]: Database files initialized
2024-03-20 11:58:09 2024-03-20 10:58:09+00:00 [Note] [Entrypoint]: Starting temporary server
2024-03-20 11:58:09 mysqld: Error on realpath() on '/var/lib/mysql-files' (Error 2 - No such file or directory)
2024-03-20 11:58:09 2024-03-20T10:58:09.680732Z 0 [ERROR] [MY-010095] [Server] Failed to access directory for --secure-file-priv. Please make sure that directory exists and is accessible by MySQL Server. Supplied value : /var/lib/mysql-files
2024-03-20 11:58:09 2024-03-20 10:58:09+00:00 [ERROR] [Entrypoint]: Unable to start server.

Additional information
No response

Does it fail with 3.7.0 too? Our CI should use the latest patch version too 🤔. Maybe it is a regression from uploading the /etc/mysql/my.cnf file (#1116) /cc @0xced. I am wondering about:

mysqld: Error on realpath() on '/var/lib/mysql-files' (Error 2 - No such file or directory)

I can take a look at it end of next week.

I've tested it with 3.7.0, and it's working well. It seems like the issue might not be related to the version itself. Maybe the regression from uploading the /etc/mysql/my.cnf file, as mentioned in #1116, doesn't affect this version?
I'll keep an eye out for any related errors, but so far, so good.

I just submitted #1144 which should address this issue.
Note that this problem only exist for mysql:8.0.28 and earlier versions. mysql:8.0.29 onwards images have an empty /var/lib/mysql-files directory and the containers start fine.
This is also the reason why this was not caught by continuous integration since the default image against which the test are run is mysql:8.0 which currently corresponds to mysql:8.0.36.

@0xced Thanks for the PR. @macalbert you can work around the issue until a new version is available by adding the following line to the builder configuration:
WithResourceMapping(Array.Empty<byte>(), "/var/lib/mysql-files/gh-issue-1142")

This will create the directory upon container start.