(gen-ai): JumpStartSageMakerEndpoint not working

Describe the bug
I'm unable to deploy the SageMaker endpoint using the JumpStartSageMakerEndpoint construct.
Expected Behavior
The endpoint gets created.
Current Behavior
CloudFormation throws the following error when trying to create the endpoint:
Could not find model data at s3://jumpstart-cache-prod-us-east-1/huggingface-asr/huggingface-asr-whisper-base/artifacts/inference-prepack/v1.0.1/. (Service: AmazonSageMaker; Status Code: 400; Error Code: ValidationException; Request ID: 8e830934-e61a-4c1e-96e5-1663211204b0; Proxy: null)

Reproduction Steps
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import {
  JumpStartModel,
  SageMakerInstanceType,
  JumpStartSageMakerEndpoint,
} from "@cdklabs/generative-ai-cdk-constructs";

export class SagemakerJumpStartFmEndpointStack extends cdk.Stack {
  jumpStartEndpoint: JumpStartSageMakerEndpoint;

  constructor(scope: Construct, id: string, props: SagemakerJumpStartFmEndpointStackProps) {
    super(scope, id, props);

    this.jumpStartEndpoint = new JumpStartSageMakerEndpoint(this, "JumpStartEndpoint", {
      model: JumpStartModel.HUGGINGFACE_ASR_WHISPER_BASE_2_0_0,
      instanceType: SageMakerInstanceType.of("ml.g5.2xlarge"),
    });
}

Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.111.0
Framework Version
0.1.11
Node.js Version
21.4.0
OS
Mac OS
Language
Typescript
Language Version
No response
Region experiencing the issue
us-east-1
Code modification
No
Other information
No response
Service quota

[X] I have reviewed the service quotas for this construct

Hi @LeonLuttenberger, thank you for reporting this ! I was able to reproduce
Issue:
The construct is expecting to get the model artifacts as a single gzip compressed tar archive (.tar.gz suffix)
However, for some models, it is not the case. It seems to affect all models in version 2_0_0 or above.
The construct needs to handle this edge case


'remove-gitlab-registration' script can invalidate in-use runner authentication token

Describe the bug
We're in the process of upgrading to 7.2.2 + the new runner registration workflow, and discovered an unfortunate race condition with remove-gitlab-registration.
Sometimes when cycling the agent instance, the AWS ASG will spin up the new instance before tearing down the old one. In that case, the following sequence of events can occur:

Old Instance
New Instance

starts
n/a

obtains Runner Token 1 from Gitlab
n/a

adds Runner Token 1 to SSM
n/a

running
n/a

running
starts

running
fetches and validates Runner Token 1 from SSM

running
finishes startup

begins shutdown
running

remove_gitlab_registration.sh runs and invalidates Runner Token 1
running

finishes shutdown
receives 403 from Gitlab because Runner Token 1 is now invalid

To Reproduce
The exact steps to reproduce seem to be tricky because the ASG seems inconsistent in how it's cycling the instances, but I believe it's:

Upgrade module to 7.2.2
Use the new runner registration workflow in the gitlab-runner.tftpl script

To enable this, we're passing in:
runner_gitlab.access_token_secure_parameter_store_name
runner_gitlab_registration_config.type
runner_gitlab_registration_config.project_id

Run terraform apply
Wait for the ASG to stabilize
Make a change to the agent's user-data (e.g. by modifying runner_install.pre_install_script), and re-run terraform apply
Wait for the ASG to stabilize
Verify that the new agent instance cannot contact Gitlab because of Forbidden errors

Expected behavior
The new agent instance can continue to run jobs from Gitlab.
Additional context
As a concrete example, the new instance fetched and validated the token at Jan 05 18:49:04 , but the the original instance then started shutdown at Jan 05 18:49:17 and invalidate the token, after which the new instance started logging the following:
Jan 05 18:50:51 gitlab-runner[2957]: {"level":"error","msg":"Checking for jobs... forbidden","runner":"<redacted>","status":"POST https://gitlab.com/api/v4/jobs/request: 403 Forbidden","time":"2024-01-05T18:50:51Z"}
Jan 05 18:50:54 gitlab-runner[2957]: {"level":"error","msg":"Checking for jobs... forbidden","runner":"<redacted>","status":"POST https://gitlab.com/api/v4/jobs/request: 403 Forbidden","time":"2024-01-05T18:50:54Z"}
Jan 05 18:50:57 gitlab-runner[2957]: {"level":"error","msg":"Checking for jobs... forbidden","runner":"<redacted>","status":"POST https://gitlab.com/api/v4/jobs/request: 403 Forbidden","time":"2024-01-05T18:50:57Z"}
Jan 05 18:50:57 gitlab-runner[2957]: {"level":"error","msg":"Runner \"<redacted>\" is unhealthy and will be disabled for 1h0m0s seconds!","time":"2024-01-05T18:50:57Z","unhealthy_requests":3,"unhealthy_requests_limit":3}

As a temporary workaround, we're passing the following into runner_install.start_script:
# This can be removed once https://github.com/cattle-ops/terraform-aws-gitlab-runner/issues/1062 is resolved.
echo 'Disabling "Remove the GitLab Runner from GitLab at shutdown" job...'
chmod a-x /opt/remove_gitlab_registration.sh || true
systemctl disable remove-gitlab-registration.service || true
systemctl stop remove-gitlab-registration.service || true
echo 'Done!'

We're also seeing this issue with the old registration token workflow. 2 more options I can think of for a fix:

Don’t re-use runner auth tokens, instead generate a new one every time a new runner spins up. Instance refresh would take a bit longer since it’ll need to re-register the runner in GitLab every time, not sure if there are other side effects.
Skip runner de-register during instance refresh. The remove_gitlab_registration.sh script would need logic to figure out if an instance refresh is the cause of instance termination and skip de-registering. Not sure how possible/reliable it would be since that script would need to know which ASG + instance refresh to look in, and if an ASG gets torn down while a refresh is in progress then de-registration may never happen.

Tangential to the current issue, but in the previous version of the module we were using the gitlab_runner resource (now gitlab_user_runner) to manage the token, and then passing it in to the cattle-ops/gitlab-runner/aws module.
That way, the Gitlab Runner + token were entirely managed by Terraform, rather than the shell script.
Although it wouldn't solve the issue for users who want the module to manage their Gitlab Runners/tokens automatically, providing a way to pass in the registration token directly to the module rather than have it managed by the shell script would solve the issue for us.

Hi, can anyone think of any more solutions other than the ones proposed? Or is there a preference on the ones proposed? We're running into this issue almost every time we make an update to our runners and would love for it to be fixed, though I'm not sure which solution is the best since they all have some caveats.

In some testing, this race bites us > 50% of the time.  Seems better to take the suggestion of skipping the deregister and leaving cleanup to a manual task than the current situation which creates potential (likely) outages.

Having downtimes is not acceptable here. I do not see any as I update the Runners in the early morning before the first developers are arriving.
Depending on how many Runner restarts you have, there can be many unused Runner registration in GitLab. What about the following proposal:
Instead of storing the Runner Token as plain string, we extend the secret with a in-use flag. Then we are able to skip the deregistration in case another Runner started slightly before the registration. So the secret would look like
{
  "token": "null",
  "in_use": false
}

Old secrets are converted during Runner startup.
What do you think, @eyablonowitz, @long-wan-ep, @schmidt-galen-heb

If the in-use flag is set to true during runner start up, and then in the deregister script it checks the flag before deregistering, when would the flag be set to false to allow deregister to happen?

Oh yes, a boolean does not work here. We need a number. The token is released if the number is zero.
In the example above, it is 1, then the next runner increments it to two and the first runner decrements it to one and does not release the token as it is still in use.

Ok gotcha, I think that's the best one out of the options proposed. As you mentioned, it's possible for it to fail since there's no atomic operation in SSM, but it should be pretty rare.

@long-wan-ep Kindly ask for a quick review of #1102


[BUG]  React Hook useEffect has a missing dependency in ./pages/search.js

Description
This issue came up as a warning in my vs code terminal after I ran git commit on an issue that I fixed. The warning can be recreated by going to ./pages/search.js
This can be fixed by just adding the missing dependencies.
Screenshots

Do you want to work on this issue?
Yes
Additional information
No response

Thanks for raising! I have assigned this to you @saram7619

Thank you @SaraJaoude

Unassigned for the reason mentioned here: https://github.com/EddieHubCommunity/BioDrop/pull/9508

hello can you assign me this issue?

hello can you assign me this issue?

@Roshanchau I will assign this to you

I cannot find the same timer function where the useEffect hook has missing dependency as in the screenshot.

Hello! I've noticed that the code causing the issue has been removed as part of PR #9248. The terminal no longer shows any missing dependencies. It looks like we might be able to close this issue now, but I'd love to get your thoughts on it.

Hello! I've noticed that the code causing the issue has been removed as part of PR #9248. The terminal no longer shows any missing dependencies. It looks like we might be able to close this issue now, but I'd love to get your thoughts on it.

Good spot, thank you 👍


Faster mapping execution

I assume this issue is about improving indexing time. First we shoud try to answer questions such as:

What subgraphs are representative of the use pattern we want to speed up? How complex are their mappings? Do they call contracts or IPFS? How many events are there per block? How many entities are there per block?
What speedup are we targeting?
Are we considering a warm database with blocks loaded (probably from indexing a previous version of the subgraph) or cold?
Do we want to speed up indexing from genesis or indexing the newest block in a subgraph that's already synced?

Case studies
I went ahead and did three case studies: Compound, ENS and Uniswap. One case I didn't cover is IPFS, which none of these use.
Compound makes 3-4 contract calls per event. The time to make a contract call is dependent on the latency to the eth node, so I can't measure on my machine, we'd need production timings, but I'd suspect that the contract calls dominate the indexing time.
ENS does no contract or ipfs calls and has very simple mappings. From a cold db, fetching blocks from the eth node dominates the indexing time. From a warm DB, it gets more subtle. The three major steps to process a block (and my local measurments for ENS) are:

Get block from the block stream 3-10ms
Process events in the runtime ~3ms*n of events in block
Apply operations to the DB 10-20ms, proportional to n of operations
Repeat.

Steps 1 and 3 take a while because they involve the DB, while step 2 is the interaction with wasm.
Uniswap like ENS doesn't do contract or ipfs calls. However it has more complex mappings that take ~15ms per event, making the runtime the slowest step in many blocks.
Possible solutions
From a cold db I don't have a better idea other than making sure there's good networking to the eth node, and then there are also bigger ideas such as local contract execution. From a warm DB there are a few things:
For Compound, caching contract calls might help.
For ENS step 3 is significant, applying the operations to the DB seems to be a bottleneck if the mapping is simple and each block has only a few events. We could pipeline operations to the DB so that we could start processing the next block while the previous one is being applied. This would require architectural changes, currently the block stream cannot proceed while the previous block has not been fully processed. Could be an up to 2x speed up for some blocks.
For Uniswap, Step 2, the runtime, becomes the bottleneck. I see two things that we can improve: First being smarter with allocations and pre-allocating memory when passing objects to AS.  Then there is swapping out wasmi for something more performant, Cranelift + wasmtime or wasmer. This is more work but should give at least a 10x speed up to the runtime.

As a part of this, I've been observing the timings of the Decentraland subgraph in production. Observations:

store.get times vary a lot, from 2-3ms to 40-60ms for a same operation. Possible reasons:

Varying DB load, but the DB has a lot more resources than it uses right now.
Entities of a same type having drastically different sizes, I don't think this is the case for this subgraph.
Contention in the DB connection pool in the indexing node.

We should check for this.

Varying DB performance for some other reason.

Factoring out store.get, event processing time still varies a lot. Possible reasons:

Contention in the Tokio thread pool in the indexing node.

We should check, DB operations in tasks could cause this. We should also log the size of the pool.

Varying CPU performance for some reason, even though we have more CPU than we use and it's not spiking.

Looking at logs in the hosted service, DB contention doesn't look like a big issue, so I don't have new or confirmed ideas for why store.get performance varies.
On general event processing time, thread pool contention is clearly being a factor, #926 will give clear confirmation of this. My plan there would be:

Set the tokio threadpool size to 100, this should confortably allow for 20 subgraphs to run in a single node.
Solve #905 by moving all store operations into a separate, blocking threadpool, and have them return futures. This is not hard but it's laborious and will have a big diff.
Try going back to the default threadpool size.

We should do 1 now, and if it works leave 2-3 in the technical debt bucket.

Old, also we switched to wasmtime.


feat(#377): list failing tests

Description
Adds a listing of failing tests after a test run as proposed on #377

Type of Change

[x] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🗑 Chore

Changed it to have an output including line and column, as well as the path of to the suite.
The only problem is that the suite is always identified as ./test/.test_runner.dart when optimization is on (default).
Without optimization, the output is:
Failing Tests:
 - <path to cwd>/test/src/widgets/the_real_test_file.dart:30:5

With optimization, the output is:
Failing Tests:
 - <path to cwd>/test/.test_runner.dart:30:5

Update: Changed to retrieve the test location from the stack trace. In this way, the right path is recovered regardless of the optimization being true of false .
Also added a prefix to show if the test failed  or resulted in an error:
Failing Tests:
 - [FAILED] test/example_test.dart 4:5
 - [ERROR] test/example_test.dart 5:5


FR: Lock issues after they are fixed

@wilhuff proposed a feature for the bot to lock issues when conversation is no longer productive.  It would probably involve the bot leaving a comment like:

Hey everyone.  It has been {time interval} since this issue was fixed so I am locking the thread.  If you're still having problems please open a new issue (link) and we're happy to discuss.

The main to-do here is to determine the criteria for locking.  Should we just do closed for {x} time?  Should we require a special label?  A milestone?

Why not use the github ui?

Gil would like to do this automatically after some period of time post-closing to leave time for people to chime in about short-term regressions, I think.

Oh, to force people to open new issues instead of adding semi-similar issues to closed existing issues?  That would be useful - maybe only for closed with a merge versus closed for staleness? @wilhuff

Yeah, twice in the last two weeks I've had to prompt people to open new issues for things we fixed months ago. In one case a few weeks back, the issue was almost a year old and someone posted, "me too".


RECT_TO_CURV function on variables that vary in time/depthy

Reported by Ansley B Manke on 16 Nov 2007 21:17 UTC
Wei Cheng reports that rect_to_curv gives results that don't correctly vary in
time when the variable being regridded varies in both time and depth.
Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/1553

Comment by<EMAIL_ADDRESS>on 19 Nov 2007 21:49 UTC
Fixed; a loop index was incorrect in
fer/efi/rect_to_curv.F


ITVHUB

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x] I've searched the bugtracker for similar questions including closed ones

Question

is there anyway to grab higher video than 504p from ITVHUB?
Thank You.

@KiaN2020 no there isn't. Apparently HD versions are protected by DRM.


deal with typescript

figure out a better solution to making vscode happy than slapping // @ts-ignore where it complains
......because any time it has complained, it is a complete waste of time (and wrong)

i never understand why reinventing the whell .. overcomplicating with this node thing..
older site already has all, just must to be ported to micro php framweork and that's all
i suggest avloid database and relly fully in git.. using mirrors sync files is easy.. all the management is delegated to git managers like git(crap)hub or gitlab or codeberg so we can promote good practices.. users proposed skins using pull recuests


cURL error 35 + 52

I'm running a batch of 32 continuous requests and occasionally I'm seeing the following exceptions:
cURL error 35: SSL connect error [./vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:49]
cURL error 52: Server returned nothing (no headers, no data) [./vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:49]

Once the application gets to this point the only way to resume Guzzle requests is to restart it. I'm using Guzzle Pool with a pool_size of 32, and I suspect the problem may be with some timeout configuration.
Has anyone faced the same issue? Can I overcome this with any Guzzle/cURL setting?

http://stackoverflow.com/questions/4073404/curl-errno-35-unknown-ssl-protocol-error-in-connection-to-secure-site443
http://guzzle.readthedocs.org/en/latest/clients.html#config
$client->get('/', [
    'config' => [
        'curl' => [
            CURLOPT_SSLVERSION => 3 // replace 3 with whatever SSL version the remote server is using
        ]
    ]
]);

no?

@kamilsk Unfortunately no, I'm still facing the same issues.

I'm not sure how to fix these at the curl level, but you could just retry these failures using something like the retry-subscriber: https://github.com/guzzle/retry-subscriber

@alixaxel did you ever find the cause of your Curl Error 52?

@konrness No, at some point it just disappeared. I suspect the server/firewall dropped those requests.

@kamilsk Very helpful for me.


PageTransitionType is not working

No matter what I write for page transition or remove the entire page transition line the output is nothing.
Splash animation is working but page transition is not working. I have already imported the page_transition.dart file

I also am having this problem. I'm using leftToRight transitions and they never work.

I'm also have this problem.

Exactly the same problem PageTransitionType doesnt work.. the creator needs to fix the issue its been from a long time!


[Bug]: Icons svg updated and old icon not working

Describe the bug
Icons svg updated and old icon not working
Screenshots

What browser are you using?
Opera GX 98 (64 bit)
What version of AdashimaaTube are you using?
v.2.2.6

Fixed in version 2.2.7. Please let me know if you are still having this issue.


ForestDB CBL crash.

Hi Folks,
Continuous Pull and Push sync is active, random CBL forest DB crash seen.
CBL logs:
[06/15 03:10:11.794] [1240] RemoteRequest: RemoteRequest{GET, https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true}: executeRequest() finally
[06/15 03:10:11.821] [971] Sync: com.couchbase.lite.support.RemoteRequestRetry$1@1e35f518: RemoteRequestRetry inner request finished, url: https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true
[06/15 03:10:11.822] [971] Sync: com.couchbase.lite.support.RemoteRequestRetry$1@1e35f518: RemoteRequestRetry was successful, calling callback url: https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true
[06/15 03:10:11.822] [971] Sync: PullerInternal{unknown, pull, 673e5}: pullRemoteRevision add rev: {s919879872020_ptx1465983604397_meta.doc #3-82f3249a40ffcab9fd0add2ca5f39a1d @22} to batcher: com.couchbase.lite.support.Batcher@235637df
[06/15 03:10:11.822] [971] Batcher: com.couchbase.lite.support.Batcher@235637df: queueObjects called with 1 objects (current inbox size = 0)
[06/15 03:10:11.823] [971] Batcher: com.couchbase.lite.support.Batcher@235637df: scheduleWithDelay 250 ms, scheduled ...
[06/15 03:10:11.823] [971] RemoteRequest: RemoteRequest{GET, https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true}: executeRequest() finally
[06/15 03:10:11.863] [651] Sync: com.couchbase.lite.support.RemoteRequestRetry$1@369e0d56: RemoteRequestRetry inner request finished, url: https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true
[06/15 03:10:11.864] [651] Sync: com.couchbase.lite.support.RemoteRequestRetry$1@369e0d56: RemoteRequestRetry was successful, calling callback url: https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true
[06/15 03:10:11.864] [651] Sync: PullerInternal{unknown, pull, 673e5}: pullRemoteRevision add rev: {s919879872020_ptx1465983604397_meta.doc #3-82f3249a40ffcab9fd0add2ca5f39a1d @22} to batcher: com.couchbase.lite.support.Batcher@5f6bb79
[06/15 03:10:11.865] [651] Batcher: com.couchbase.lite.support.Batcher@5f6bb79: queueObjects called with 1 objects (current inbox size = 0)
[06/15 03:10:11.865] [651] Batcher: com.couchbase.lite.support.Batcher@5f6bb79: scheduleWithDelay 250 ms, scheduled ...
[06/15 03:10:11.865] [651] RemoteRequest: RemoteRequest{GET, https://bsyncg.kodiakhsqc.com:443/ptxdata/s919879872020_ptx1465983604397_meta.doc?rev=3-82f3249a40ffcab9fd0add2ca5f39a1d&revs=true&attachments=true}: executeRequest() finally
[06/15 03:10:12.044] [1214] Batcher: com.couchbase.lite.support.Batcher$2@fe011d7: call processNow ...
[06/15 03:10:12.046] [1214] Batcher: com.couchbase.lite.support.Batcher@1e560f8d: processNow() called, inbox size: 1
[06/15 03:10:12.048] [1214] Batcher: com.couchbase.lite.support.Batcher@1e560f8d: invoking processor com.couchbase.lite.replicator.PullerInternal$2@372f2f2b with 1 items
[06/15 03:10:12.049] [1214] Sync: PullerInternal{unknown, pull, 673e5} inserting 1 revisions...
[06/15 03:10:12.057] [1214] Sync: com.couchbase.lite.replicator.PullerInternal$9@2b214ac4: inserting s919879872020_ptx1465983604397_meta.doc [3-82f3249a40ffcab9fd0add2ca5f39a1d, 2-b70fdea84e727094468a98a7f531f393, 1-8a87b30dbca09f81b51e40d7cbda5f1d]
[06/15 03:10:12.059] [1214] Database: INSERT {s919879872020_ptx1465983604397_meta.doc #3-82f3249a40ffcab9fd0add2ca5f39a1d @4}, history[3]
[06/15 03:10:12.065] [1214] CBLite: ---> Added: {s919879872020_ptx1465983604397_meta.doc #3-82f3249a40ffcab9fd0add2ca5f39a1d @0} as seq 0
[06/15 03:10:12.066] [1214] Sync: com.couchbase.lite.replicator.PullerInternal$9@2b214ac4: finished inserting 1 revisions
[06/15 03:10:12.067] [1214] Sync: PullerInternal{unknown, pull, 673e5}: Setting lastSequence to 196027::196258 from(196027::196256)
[06/15 03:10:12.070] [1214] Sync: com.couchbase.lite.replicator.PullerInternal$9@2b214ac4: inserted 1 revs in 16 milliseconds
[06/15 03:10:12.071] [1214] Sync: com.couchbase.lite.replicator.PullerInternal$9@2b214ac4 insertDownloads() updating completedChangesCount from 1 -> 2
[06/15 03:10:12.073] [1214] Sync: PullerInternal{unknown, pull, 673e5}: Incrementing completedChangesCount count from 1 by adding 1 -> 2
[06/15 03:10:12.073] [683] Batcher: com.couchbase.lite.support.Batcher$2@198cbdad: call processNow ...
[06/15 03:10:12.075] [1214] CBLite: setPaused: false
[06/15 03:10:12.076] [683] Batcher: com.couchbase.lite.support.Batcher@235637df: processNow() called, inbox size: 1
[06/15 03:10:12.078] [683] Batcher: com.couchbase.lite.support.Batcher@235637df: invoking processor com.couchbase.lite.replicator.PullerInternal$2@3287f86e with 1 items
[06/15 03:10:12.087] [683] Sync: PullerInternal{unknown, pull, 673e5} inserting 1 revisions...
[06/15 03:10:12.090] [683] Sync: com.couchbase.lite.replicator.PullerInternal$9@2385a8e2: inserting s919879872020_ptx1465983604397_meta.doc [3-82f3249a40ffcab9fd0add2ca5f39a1d, 2-b70fdea84e727094468a98a7f531f393, 1-8a87b30dbca09f81b51e40d7cbda5f1d]
[06/15 03:10:12.092] [683] Database: INSERT {s919879872020_ptx1465983604397_meta.doc #3-82f3249a40ffcab9fd0add2ca5f39a1d @22}, history[3]
[06/15 03:10:12.115] [639] Batcher: com.couchbase.lite.support.Batcher$2@1a29f473: call processNow ...
[06/15 03:10:12.116] [683] Database: ForestDB Error: forceInsert()
com.couchbase.cbforest.ForestException: Forestdb Handle is being used by another thread
	at com.couchbase.cbforest.Document.init(Native Method)
	at com.couchbase.cbforest.Document.<init>(Document.java:97)
	at com.couchbase.cbforest.Database.getDocument(Database.java:46)
	at com.couchbase.lite.store.ForestDBStore$4.run(ForestDBStore.java:961)
	at com.couchbase.lite.store.ForestDBStore.inTransaction(ForestDBStore.java:1371)
	at com.couchbase.lite.store.ForestDBStore.forceInsert(ForestDBStore.java:955)
	at com.couchbase.lite.Database.forceInsert(Database.java:1809)
	at com.couchbase.lite.replicator.PullerInternal$9.run(PullerInternal.java:568)
	at com.couchbase.lite.store.ForestDBStore.runInTransaction(ForestDBStore.java:297)
	at com.couchbase.lite.replicator.PullerInternal.insertDownloads(PullerInternal.java:550)
	at com.couchbase.lite.replicator.PullerInternal$2.process(PullerInternal.java:115)
	at com.couchbase.lite.support.Batcher.processNow(Batcher.java:383)
	at com.couchbase.lite.support.Batcher.access$300(Batcher.java:16)
	at com.couchbase.lite.support.Batcher$2.run(Batcher.java:317)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
	at java.util.concurrent.FutureTask.run(FutureTask.java:237)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
	at java.lang.Thread.run(Thread.java:818)

[06/15 03:10:12.119] [683] Sync: com.couchbase.lite.replicator.PullerInternal$9@2385a8e2: failed to write {s919879872020_ptx1465983604397_meta.doc #3-82f3249a40ffcab9fd0add2ca5f39a1d @22}: status=-1
[06/15 03:10:12.119] [683] Sync: PullerInternal{unknown, pull, 673e5}: Progress: set error = org.apache.http.client.HttpResponseException
[06/15 03:10:12.120] [683] Sync: com.couchbase.lite.replicator.PullerInternal$9@2385a8e2: finished inserting 1 revisions
[06/15 03:10:12.121] [683] Sync: com.couchbase.lite.replicator.PullerInternal$9@2385a8e2: inserted 1 revs in 32 milliseconds
[06/15 03:10:12.122] [683] Sync: com.couchbase.lite.replicator.PullerInternal$9@2385a8e2 insertDownloads() updating completedChangesCount from 21 -> 22
[06/15 03:10:12.123] [683] Sync: PullerInternal{unknown, pull, 673e5}: Incrementing completedChangesCount count from 21 by adding 1 -> 22
[06/15 03:10:12.117] [639] Batcher: com.couchbase.lite.support.Batcher@5f6bb79: processNow() called, inbox size: 1
[06/15 03:10:12.134] [639] Batcher: com.couchbase.lite.support.Batcher@5f6bb79: invoking processor com.couchbase.lite.replicator.PullerInternal$2@16fe26b3 with 1 items
[06/15 03:10:12.134] [639] Sync: PullerInternal{unknown, pull, 673e5} inserting 1 revisions...
[06/15 03:10:12.135] [683] CBLite: setPaused: false
[06/15 03:10:12.250] [1237] Batcher: com.couchbase.lite.support.Batcher@39cdc2ea: waitForPendingFutures is called ...
[06/15 03:10:12.251] [1237] Batcher: com.couchbase.lite.support.Batcher@39cdc2ea: waitForPendingFutures done
[06/15 03:10:12.252] [1238] Batcher: com.couchbase.lite.support.Batcher@1563d000: waitForPendingFutures is called ...
[06/15 03:10:12.253] [1238] Batcher: com.couchbase.lite.support.Batcher@1563d000: waitForPendingFutures done
[06/15 03:10:12.254] [1239] Batcher: com.couchbase.lite.support.Batcher@d4b1c5: waitForPendingFutures is called ...
[06/15 03:10:12.255] [1239] Batcher: com.couchbase.lite.support.Batcher@d4b1c5: waitForPendingFutures done
[06/15 03:10:12.650] [1237] Batcher: com.couchbase.lite.support.Batcher@1e560f8d: waitForPendingFutures is called ...
[06/15 03:10:12.650] [1238] Batcher: com.couchbase.lite.support.Batcher@235637df: waitForPendingFutures is called ...
[06/15 03:10:12.650] [1239] Batcher: com.couchbase.lite.support.Batcher@5f6bb79: waitForPendingFutures is called ...
[06/15 03:10:12.769] [8] Listener: authHeader is null
[06/15 03:10:12.770] [8] Listener: Not enforcing basic auth -- allowedCredentials null or empty
[06/15 03:10:12.771] [8] Sync: PullerInternal{unknown, pull, 99681} [fireTrigger()] => STOP_GRACEFUL
[06/15 03:10:12.771] [1219] Sync: firing trigger: STOP_GRACEFUL
[06/15 03:10:12.772] [1219] Sync: PullerInternal{unknown, pull, 99681} [onExit()] IDLE => STOPPING
[06/15 03:10:12.772] [1219] Sync: State transition: IDLE -> STOPPING (via STOP_GRACEFUL).  this: PullerInternal{unknown, pull, 99681}
[06/15 03:10:12.772] [1219] Sync: PullerInternal{unknown, pull, 99681} [onExit()] IDLE => STOPPING
[06/15 03:10:12.772] [1219] Sync: PullerInternal{unknown, pull, 99681} [onEntry()] IDLE => STOPPING
[06/15 03:10:12.773] [1219] Sync: PullerInternal{unknown, pull, 99681} STOPPING...
[06/15 03:10:12.773] [1219] ChangeTracker: ChangeTracker{https://---:---@bsyncg.kodiakhsqc.com:443/pttdata, OneShot, @35fb2d43}: Changed tracker asked to stop
[06/15 03:10:12.773] [1219] CBLite: setPaused: false
[06/15 03:10:12.774] [1219] ChangeTracker: ChangeTracker{https://---:---@bsyncg.kodiakhsqc.com:443/pttdata, OneShot, @35fb2d43}: Changed tracker aborting request: org.apache.http.client.methods.HttpPost@304a4f30
[06/15 03:10:12.775] [1222] ChangeTracker: ChangeTracker{https://---:---@bsyncg.company:443/pttdata, OneShot, @35fb2d43}: Change tracker run loop exiting
[06/15 03:10:12.775] [1222] ChangeTracker: ChangeTracker{https://---:---@bsyncg.company:443/pttdata, OneShot, @35fb2d43}: Change tracker in stopped()
[06/15 03:10:12.775] [1222] ChangeTracker: ChangeTracker{https://---:---@bsyncg.company:443/pttdata, OneShot, @35fb2d43}: Change tracker calling changeTrackerStopped, client: PullerInternal{unknown, pull, 99681}
[06/15 03:10:12.778] [1219] Batcher: com.couchbase.lite.support.Batcher@16c30ca2: flushing all objects (wait=false)
[06/15 03:10:12.779] [1219] Sync: State transition: IDLE -> STOPPING (via STOP_GRACEFUL).  this: PullerInternal{unknown, pull, 99681}
[06/15 03:10:12.780] [1219] Sync: changeTrackerStopped.  lifecycle: ONESHOT
[06/15 03:10:12.780] [1219] ChangeTracker: ChangeTracker{https://---:---@bsyncg.company:443/pttdata, OneShot, @35fb2d43}: getLastError() null
[06/15 03:10:12.794] [1241] Sync: PullerInternal{unknown, pull, 99681} [fireTrigger()] => STOP_IMMEDIATE
[06/15 03:10:12.795] [1241] Sync: PullerInternal stop.run() finished
[06/15 03:10:12.796] [1219] Sync: firing trigger: STOP_IMMEDIATE
[06/15 03:10:12.796] [1219] Sync: PullerInternal{unknown, pull, 99681} [onEntry()] STOPPING => STOPPED
[06/15 03:10:12.799] [1219] Sync: PullerInternal{unknown, pull, 99681}: clearDbRef() called
[06/15 03:10:12.810] [1219] Sync: PullerInternal{unknown, pull, 99681}: clearDbRef() setting db to null
[06/15 03:10:12.810] [1219] Sync: State transition: STOPPING -> STOPPED (via STOP_IMMEDIATE).  this: PullerInternal{unknown, pull, 99681}
[06/15 03:10:12.812] [1242] Sync: [waitForPendingFutures()] STARTED - thread id: 6480
[06/15 03:10:12.812] [1242] Sync: PullerInternal{unknown, pull, 99681} [fireTrigger()] => STOP_GRACEFUL
[06/15 03:10:12.813] [1219] Sync: firing trigger: STOP_GRACEFUL
[06/15 03:10:12.813] [1242] Sync: [waitForPendingFutures()] END - thread id: 6480

**ADB traces:**

06-15 15:10:12.874 I/DEBUG   (  282): signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------

06-15 15:10:12.874 D/StatusBarManagerService(  955): manageDisableList userId=0 what=0x0 pkg=WindowManager.LayoutParams

06-15 15:10:12.884 I/ActivityManager(  955): Displayed com.android.documentsui/.DocumentsActivity: +199ms

06-15 15:10:12.884 W/ContextImpl(  955): Calling a method in the system process without a qualified user: android.app.ContextImpl.sendBroadcast:1687 com.android.server.InputMethodManagerService$6.run:2738 java.lang.Thread.run:818 <bottom of call stack> <bottom of call stack>

06-15 15:10:12.894 D/Documents(32579): Before filtering 0, after 0

06-15 15:10:12.894 D/Documents(32579): Before filtering 1, after 0

06-15 15:10:12.904 D/Documents(32579): Before filtering 14, after 2

06-15 15:10:12.904 I/Timeline(32579): Timeline: Activity_idle id: android.os.BinderProxy@2b32d90 time:5238324

06-15 15:10:12.904 D/Documents(32579): Before filtering 46, after 0

06-15 15:10:12.904 D/Documents(32579): Found 4 of 4 recent queries done

06-15 15:10:12.944 D/PanelView( 1186): mClearAll.setVisibility - mIsFullyOpened : false isShade : true mHasNotification : true mIsUpwardFling : false mQsFullyExpanded : false isShadeLocked : false mClearAllVisible : true

06-15 15:10:12.944 D/PanelView( 1186): mClearAll.setVisibility - mIsFullyOpened : false isShade : true mHasNotification : true mIsUpwardFling : false mQsFullyExpanded : false isShadeLocked : false mClearAllVisible : true

06-15 15:10:12.964 D/PanelView( 1186): mClearAll.setVisibility - mIsFullyOpened : false isShade : true mHasNotification : true mIsUpwardFling : false mQsFullyExpanded : false isShadeLocked : false mClearAllVisible : true

06-15 15:10:12.964 I/DEBUG   (  282):     r0 00000000  r1 00004846  r2 00000006  r3 00000000

06-15 15:10:12.964 I/DEBUG   (  282):     r4 6faf3db8  r5 00000006  r6 0000006e  r7 0000010c

06-15 15:10:12.964 I/DEBUG   (  282):     r8 12fbb9a0  r9 9f915800  sl 00000001  fp 13ed9820

06-15 15:10:12.974 I/DEBUG   (  282):     ip 00004846  sp 6faf3680  lr b6eb2fc9  pc b6ed6acc  cpsr 600f0010

06-15 15:10:12.974 I/DEBUG   (  282):

06-15 15:10:12.974 I/DEBUG   (  282): backtrace:

06-15 15:10:12.974 I/DEBUG   (  282):     #00 pc 00037acc  /system/lib/libc.so (tgkill+12)

06-15 15:10:12.974 I/DEBUG   (  282):     #01 pc 00013fc5  /system/lib/libc.so (pthread_kill+52)

06-15 15:10:12.974 I/DEBUG   (  282):     #02 pc 00014be3  /system/lib/libc.so (raise+10)

06-15 15:10:12.974 I/DEBUG   (  282):     #03 pc 00011529  /system/lib/libc.so (__libc_android_abort+36)

06-15 15:10:12.974 I/DEBUG   (  282):     #04 pc 0000fcb4  /system/lib/libc.so (abort+4)

06-15 15:10:12.974 I/DEBUG   (  282):     #05 pc 001c7419  /data/app/com.app.company-1/lib/arm/libCouchbaseLiteJavaForestDB.so (__gnu_cxx::__verbose_terminate_handler()+220)

06-15 15:10:12.974 I/DEBUG   (  282):     #06 pc 00197187  /data/app/com.app.company-1/lib/arm/libCouchbaseLiteJavaForestDB.so (__cxxabiv1::__terminate(void (*)())+2)

06-15 15:10:12.974 I/DEBUG   (  282):     #07 pc 001971b7  /data/app/com.app.company-1/lib/arm/libCouchbaseLiteJavaForestDB.so (std::terminate()+10)

06-15 15:10:12.974 I/DEBUG   (  282):     #08 pc 000f3fed  /data/app/com.app.company-1/lib/arm/libCouchbaseLiteJavaForestDB.so

Thanks
Nithin

Hi @NitzDKoder,
Thank you for bug report. Could you please provide CBL version number?

@hideki below forestdb cbl 1.2 was used..
http://files.couchbase.com/maven2/com/couchbase/lite/couchbase-lite-android/0.0.0-700/

Worngly closed..

@NitzDKoder
v1.2.0 has crush issue with ForestDB. Could you please try 1.2.1 or latest build from master branch 0.0.0-xxx?

Sure..any obivious issues pointing to crash stack trace?

most of ForestDB related crashes were caused by accessing ForestDB after closed ForestDB by other threads or accessing ForestDB from multiple threads concurrently. ForestDB is not thread safe, so we added thread safety in wrapper of forestdb. But it was not sufficient.

Used patch build from https://github.com/couchbase/couchbase-lite-java-core/issues/1177..
Alright lets see 1.21 or master will address this? So any fixed planned for thread safe?

We believe latest version is thread-safe. However, we might not cover all scenario. If you find any forestdb issue, please let us know.

Hi @NitzDKoder,
Did you have chance to test  with build from latest master branch?

I will close this ticket. If you are still encountering this issue, please reopen.


Implement settings and user preferences

              // TODO: Implement settings and user preferences

https://github.com/mind-co/comind/blob/e7a958d2c6cef4ae22bb3eb47cbd222647a5a553/lib/stream.dart#L730

Partial status -- this works for colors only.


Unmarshaler / UnmarshalRQL is not implemented.

UnmarshalRQL does nothing.

Yeah I never got round to implementing this interface. Ill try to either add it in the next patch (or remove the interface if its not possible with the new method of decoding)
Thanks for reporting the issue and sorry for taking so long to reply.


Implement support for having rendering and script execution on different threads

We want rendering and script execution to run in parallel as much as possible.
For this to work, and work efficiently, we have to do a few things:

[ ] keep buffers on the rendering thread, but have proxies for them on the script thread. That means, e.g., that a BitmapData element exists on the script thread, but its buffer is on the rendering thread. In case of script manipulating this buffer, we can pull it over via the synchronous channel, and move it back again after the script execution has finished and before rendering starts again.
[ ] move path data for shapes from the script thread to the rendering thread before rendering starts. We don't need to copy this data, as we don't need it in the script thread, anymore. (Except for the rare case where the path data is queried by script. Then we have to move or copy it back over again.)
[ ] enable text rendering and metrics calculations on the rendering thread. This is somewhat complicated by the fact that AS3 has synchronous APIs for gathering various text metrics. These metrics are already calculated lazily before rendering or when they're queried, so in the ideal case where they aren't queried by script, we can just keep them on the rendering thread. If they are queried, however, we have to copy them over using the sync channel.
[ ] move video data to the rendering thread after it has been parsed. For RTMP.

Nope


Project Properties are never updated once they have been set.

Project properties are set when the project is saved but are not updated anymore after that when the user modifies some settings.
For instance, the model_depth for a maxwell 2D transient cannot be updated after the project has been saved.
The behavior of the property project_properies in Design() must be updated.

Hi @MaxJPRey , is this issue fixed?

this has been fixed

Yes @Samuelopez-ansys it has been fixed. Thanks to you and @maxcapodi78 for closing it.


Fix pypi badge

Pypi badge was referencing pypika.

Pull Request Test Coverage Report for Build 5

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 97.034%

Totals

Change from base Build 3:
0.0%

Covered Lines:
229

Relevant Lines:
236

💛  - Coveralls


mini-fix in cluster.Get

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Pull request type

Please check the type of change your PR introduces:

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Other information

Codecov Report

Merging #127 (99eb87f) into master (07d8bd0) will decrease coverage by 25.97%.
The diff coverage is 0.00%.

:exclamation: Current head 99eb87f differs from pull request most recent head dc67917. Consider uploading reports for the commit dc67917 to get more accurate results

@@             Coverage Diff             @@
##           master     #127       +/-   ##
===========================================
- Coverage   45.22%   19.24%   -25.98%
===========================================
  Files         126      106       -20
  Lines       13740    10993     -2747
===========================================
- Hits         6214     2116     -4098
- Misses       7210     8720     +1510
+ Partials      316      157      -159

Flag
Coverage Δ

1.13.x
?

1.14.x
?

1.15.x
?

1.16.x
?

1.17.x
19.24% <0.00%> (-25.98%)
:arrow_down:

e2e
19.24% <0.00%> (-25.73%)
:arrow_down:

integration
19.24% <0.00%> (-25.73%)
:arrow_down:

macOS-latest
?

ratelimiter
15.76% <0.00%> (-0.02%)
:arrow_down:

scripting
14.96% <0.00%> (-0.02%)
:arrow_down:

table
?

ubuntu-latest
19.24% <0.00%> (-36.35%)
:arrow_down:

unit
?

windows-latest
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

internal/cluster/cluster.go
25.22% <0.00%> (-41.30%)
:arrow_down:

internal/conn/conn.go
58.33% <0.00%> (-19.26%)
:arrow_down:

internal/table/trailer.go
0.00% <0.00%> (-100.00%)
:arrow_down:

internal/scheme/options.go
0.00% <0.00%> (-100.00%)
:arrow_down:

internal/table/keep_in_cache.go
0.00% <0.00%> (-100.00%)
:arrow_down:

internal/table/transaction.go
0.00% <0.00%> (-86.14%)
:arrow_down:

internal/table/client.go
0.00% <0.00%> (-85.72%)
:arrow_down:

internal/table/data_query.go
0.00% <0.00%> (-80.77%)
:arrow_down:

internal/deadline/context.go
0.00% <0.00%> (-80.00%)
:arrow_down:

internal/table/retry.go
0.00% <0.00%> (-79.77%)
:arrow_down:

... and 70 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07d8bd0...dc67917. Read the comment docs.


✨ Update webapp guidance for AP

Describe the feature request.
Based off the issues experienced in https://github.com/ministryofjustice/data-platform-support/issues/544 I thought it would be good to list where the current guidance, and templates can be improved.
Update the template to include the healthcheck

build-push-deploy-dev.yml
build-push-deploy-prod.yml

Need the following added before line 164:
--set WebApp.AlternativeHealthCheck.enabled="true" \
--set WebApp.AlternativeHealthCheck.port=9999 \

This is to prevent healthcheck issues
Update the guidance to state the port needs to be 9999
The healthcheck and other endpoints are sent to port 9999, but this normally isn't the default or norm port when people build webapps. So the guidance needs to specific that they need to use port 9999.
A good place to include this is probably where the Dockerfile is first mentioned here.
Add note to make sure user doesn't set the namespace to be the same as the repo name
The Cloud Platform namespace should be different to the github repository name the app is in; this isn't explicitly stated so I think it would be good to just mentioned in here.
Examples of working webapps using different app languages
Would be good to include a list of known, working webapps at the end of the guidance for the various types of webapp people typically create. It's very RShiny heavy at the moment, so examples of Flask, Dash and streamlit would be useful too.
Describe the context.
No response
Value / Purpose
No response
User Types
No response

We would look into the suggestions for the guidance.

https://github.com/moj-analytical-services/user-guidance/pull/424


Add support for osx

Rplidar does currently only work on linux.
Supports need to be added for osx, and the usb library in rdk might need to be extended for that. This code snippet might come in handy when working on this issue:

USBInfo: &usb.Identifier{
    Vendor:  0x10c4,
    Product: 0xea60,
}

Actually, I just realized you may actually need to install drivers for this under OSX.
https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers
If you've already done that and/or the device works with other software already though, then I'm probably wrong.

That's assuming the device path isn't hard coded like in the current PR, and instead actually uses the usb.Search() functions in RDK, because the Mac device name will be different from the linux one that works on the pi.

Tried running it on osx after installing the driver; didn't work. (Changed it to use the search function before testing this).

I don't have an RPLidar, so I'll leave it up to you (or anyone else that has one.) If you want to troubleshoot, just let me know and I'll try and help out.

Sounds good, thank you!


Optimise performance of descendant segment

One of the examples and a corresponding test, when uncommented, each take several minutes to run because they involve the descendant segment. This is probably because the current algorithm computes all (12!, i.e. over 400 million) permutations of descendants and then filters this collection according to the rules in the RFC.
This inefficient approach was taken because it fairly clearly conforms to the RFC.
Is there an effective optimisation which also fairly clearly conforms to the RFC?
To reproduce uncomment the example and issue:
stack run

or uncomment the test and issue:
stack test

Note: the expected value in the test is incorrect and the test should fail.

Changing:
l :: Nodelist <- map (map snd) $ filter validDescendantOrdering $ permutations $ ([], n):descendants [] n

to:
l :: Nodelist <- map (map snd) $ map (([], n):) $ filter validDescendantOrdering $ permutations $ descendants [] n

improved the speed of the problematic example by a factor of 8.

Merged and tidied up: 94eac8304bf4d2a3bef130c5086a53ebc8979249


Add GpuMapConcat support for nested type keys.

Signed-off-by: remzi<EMAIL_ADDRESS>This is a subtask of #5559.
The GpuMaxConcat could support array typed keys and struct typed keys.

build

The changes look good, but I think we should also add integration tests so that we test this on all supported platforms.


Get-EntraUser degrades gracelessly compared to Get-MGUser because of hardcoded $select

Describe the bug
Right now, if you run Get-EntraUser without an explicit list of -Properties you'd like to select, the function is hard-coded to decide for you that you must be explicitly trying to select the following property list:
Id,AccountEnabled,AgeGroup,OfficeLocation,AssignedLicenses,AssignedPlans,City,CompanyName,ConsentProvidedForMinor,Country,CreationType,Department,DisplayName,GivenName,OnPremisesImmutableId,JobTitle,LegalAgeGroupClassification,Mail,MailNickName,MobilePhone,OnPremisesSecurityIdentifier,OtherMails,PasswordPolicies,PasswordProfile,PostalCode,PreferredLanguage,ProvisionedPlans,OnPremisesProvisioningErrors,ProxyAddresses,RefreshTokensValidFromDateTime,ShowInAddressList,State,StreetAddress,Surname,BusinessPhones,UsageLocation,UserPrincipalName,ExternalUserState,ExternalUserStateChangeDateTime,UserType,OnPremisesLastSyncDateTime,ImAddresses,SecurityIdentifier,OnPremisesUserPrincipalName,ServiceProvisioningErrors,IsResourceAccount,OnPremisesExtensionAttributes,DeletedDateTime,OnPremisesSyncEnabled,EmployeeType,EmployeeHireDate,CreatedDateTime,EmployeeOrgData,preferredDataLocation,Identities,onPremisesSamAccountName,EmployeeId,EmployeeLeaveDateTime,AuthorizationInfo,FaxNumber,OnPremisesDistinguishedName,OnPremisesDomainName,IsLicenseReconciliationNeeded,signInSessionsValidFromDateTime,SignInActivity

Adding a -Debug switch to Get-MgUser, on the other hand, reveals that it simply does not include a $select=... at all in the HTTP request to the Graph API if you do not bother to explicitly specify which properties you would like to query.
The result is that:

Get-MgUser without an explicit -Property succeeds if (Get-EntraContext).Scopes includes something that allows some user details to be accessed through https://graph.microsoft.com/v1.0/users/ (Graph Explorer example here).
Get-EntraUser without an explicit -Property list fails if (Get-EntraContext).Scopes.Contains('User.Read.All') is false, even if the logged-in user's scope otherwise allows some user details to be accessed through https://graph.microsoft.com/v1.0/users/, because Get-EntraUser tries to download an explicit kitchen-sink list of user properties, some of which people without User.Read.All scope are not allowed to view.

To Reproduce

Execute Connect-Entra -TenantId SOME_TENANT_ID_HERE with a user for whom the following are all true:

 (Get-MgUser -UserId 'SOME_USER_IDENTIFIER_HERE').Count -eq 1

 (Get-MgUser -UserId 'SOME_USER_IDENTIFIER_HERE' -Property @('Id','UserPrincipalName')).Count -eq 1

 $my_temporary_error_output = @()
 Get-MgUser -UserId 'SOME_USER_IDENTIFIER_HERE' -Property @('Id','SignInActivity') -ErrorAction 'SilentlyContinue' -ErrorVariable 'my_temporary_error_output'
 $my_temporary_error_output.FullyQualifiedErrorId -eq 'Authentication_RequestFromUnsupportedUserRole,Microsoft.Graph.PowerShell.Cmdlets.GetMgUser_Get'

 (Get-EntraUser -ObjectId 'SOME_USER_IDENTIFIER_HERE' -Property @('Id','UserPrincipalName')).Count -eq 1

Execute and validate that the following is true (errors out as Authentication_RequestFromUnsupportedUserRole):$my_temporary_error_output = @()
Get-EntraUser -ObjectId 'SOME_USER_IDENTIFIER_HERE' -ErrorAction 'SilentlyContinue' -ErrorVariable 'my_temporary_error_output'
$my_temporary_error_output.ErrorRecord.ErrorDetails.Message -Match 'Authentication_RequestFromUnsupportedUserRole'

Expected behavior

(Get-EntraUser -ObjectId 'SOME_USER_IDENTIFIER_HERE').Count -eq 1 should be true and simply only return the data that a $select-less Graph API call would return, not get forced into an Authentication_RequestFromUnsupportedUserRole by deciding for the person using Get-EntraUser exactly which properties they meant to include.

(It is fine to try the full property list, but at least fall back to trying a $select-less version if it errors out this way, so that it behaves equivalently to Get-MgUser.)

Debug Output
Get-EntraUser -ObjectId<EMAIL_ADDRESS>-Debug

DEBUG: GET<EMAIL_ADDRESS>HTTP/1.1
HTTP: graph.microsoft.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Microsoft Windows 10.0.19045; en-US) PowerShell/7.4.3 Invoke-MgGraphRequest

DEBUG: GET<EMAIL_ADDRESS>HTTP/2.0 200 OK
Cache-Control: no-cache
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000
request-id: OBFUSCATED
client-request-id: OBFUSCATED
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"OBFUSCATED"...}}
x-ms-resource-unit: 1
odata-version: 4.0
Date: Wed, 24 Jul 2024 18:07:53 GMT
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true; IEEE754Compatible=false; charset=utf-8
Content-Encoding: gzip

{"@odata.context":"https://graph.microsoft.com/v1.0/$metadata#users/$entity","businessPhones":["+1<PHONE_NUMBER>"],"displayName":"A Name","givenName":"TheFirstName","jobTitle":"A Job Title","mail":"an_email@example.com","mobilePhone":"+1<PHONE_NUMBER>","officeLocation":"Sunny Street","preferredLanguage":null,"surname":"TheLastName","userPrincipalName":"an_email@example.com","id":"the_user_id"}
DEBUG: ============================ TRANSFORMATIONS ============================
DEBUG: Uri : https://graph.microsoft.com/v1.0/users/the_user_id?$select=Id,AccountEnabled,AgeGroup,OfficeLocation,AssignedLicenses,AssignedPlans,City,CompanyName,ConsentProvidedForMinor,Country,CreationType,Department,DisplayName,GivenName,OnPremisesImmutableId,JobTitle,LegalAgeGroupClassification,Mail,MailNickName,MobilePhone,OnPremisesSecurityIdentifier,OtherMails,PasswordPolicies,PasswordProfile,PostalCode,PreferredLanguage,ProvisionedPlans,OnPremisesProvisioningErrors,ProxyAddresses,RefreshTokensValidFromDateTime,ShowInAddressList,State,StreetAddress,Surname,BusinessPhones,UsageLocation,UserPrincipalName,ExternalUserState,ExternalUserStateChangeDateTime,UserType,OnPremisesLastSyncDateTime,ImAddresses,SecurityIdentifier,OnPremisesUserPrincipalName,ServiceProvisioningErrors,IsResourceAccount,OnPremisesExtensionAttributes,DeletedDateTime,OnPremisesSyncEnabled,EmployeeType,EmployeeHireDate,CreatedDateTime,EmployeeOrgData,preferredDataLocation,Identities,onPremisesSamAccountName,EmployeeId,EmployeeLeaveDateTime,AuthorizationInfo,FaxNumber,OnPremisesDistinguishedName,OnPremisesDomainName,IsLicenseReconciliationNeeded,signInSessionsValidFromDateTime,SignInActivity
DEBUG: Method : GET
DEBUG: =========================================================================

DEBUG: GET /v1.0/users/the_user_id?$select=Id,AccountEnabled,AgeGroup,OfficeLocation,AssignedLicenses,AssignedPlans,City,CompanyName,ConsentProvidedForMinor,Country,CreationType,Department,DisplayName,GivenName,OnPremisesImmutableId,JobTitle,LegalAgeGroupClassification,Mail,MailNickName,MobilePhone,OnPremisesSecurityIdentifier,OtherMails,PasswordPolicies,PasswordProfile,PostalCode,PreferredLanguage,ProvisionedPlans,OnPremisesProvisioningErrors,ProxyAddresses,RefreshTokensValidFromDateTime,ShowInAddressList,State,StreetAddress,Surname,BusinessPhones,UsageLocation,UserPrincipalName,ExternalUserState,ExternalUserStateChangeDateTime,UserType,OnPremisesLastSyncDateTime,ImAddresses,SecurityIdentifier,OnPremisesUserPrincipalName,ServiceProvisioningErrors,IsResourceAccount,OnPremisesExtensionAttributes,DeletedDateTime,OnPremisesSyncEnabled,EmployeeType,EmployeeHireDate,CreatedDateTime,EmployeeOrgData,preferredDataLocation,Identities,onPremisesSamAccountName,EmployeeId,EmployeeLeaveDateTime,AuthorizationInfo,FaxNumber,OnPremisesDistinguishedName,OnPremisesDomainName,IsLicenseReconciliationNeeded,signInSessionsValidFromDateTime,SignInActivity HTTP/1.1
HTTP: graph.microsoft.com
User-Agent: PowerShell/7.4.3 EntraPowershell/0.11.0 Get-EntraUser

DEBUG: GET https://graph.microsoft.com/v1.0/users/the_user_id?$select=Id,AccountEnabled,AgeGroup,OfficeLocation,AssignedLicenses,AssignedPlans,City,CompanyName,ConsentProvidedForMinor,Country,CreationType,Department,DisplayName,GivenName,OnPremisesImmutableId,JobTitle,LegalAgeGroupClassification,Mail,MailNickName,MobilePhone,OnPremisesSecurityIdentifier,OtherMails,PasswordPolicies,PasswordProfile,PostalCode,PreferredLanguage,ProvisionedPlans,OnPremisesProvisioningErrors,ProxyAddresses,RefreshTokensValidFromDateTime,ShowInAddressList,State,StreetAddress,Surname,BusinessPhones,UsageLocation,UserPrincipalName,ExternalUserState,ExternalUserStateChangeDateTime,UserType,OnPremisesLastSyncDateTime,ImAddresses,SecurityIdentifier,OnPremisesUserPrincipalName,ServiceProvisioningErrors,IsResourceAccount,OnPremisesExtensionAttributes,DeletedDateTime,OnPremisesSyncEnabled,EmployeeType,EmployeeHireDate,CreatedDateTime,EmployeeOrgData,preferredDataLocation,Identities,onPremisesSamAccountName,EmployeeId,EmployeeLeaveDateTime,AuthorizationInfo,FaxNumber,OnPremisesDistinguishedName,OnPremisesDomainName,IsLicenseReconciliationNeeded,signInSessionsValidFromDateTime,SignInActivity
HTTP/2.0 403 Forbidden
Cache-Control: no-cache
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000
request-id: OBFUSCATED
client-request-id: OBFUSCATED
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"OBFUSCATED"...}}
x-ms-resource-unit: 1
Date: Wed, 24 Jul 2024 18:07:54 GMT
Content-Type: application/json
Content-Encoding: gzip

{"error":{"code":"Authentication_RequestFromUnsupportedUserRole","message":"User is not in the allowed roles","innerError":{"date":"2024-07-24T18:07:54","request-id":"OBFUSCATED","client-request-id":"OBFUSCATED"}}}
Invoke-MgGraphRequest: GET
https://graph.microsoft.com/v1.0/users/the_user_id?$select=Id,AccountEnabled,AgeGroup,OfficeLocation,AssignedLicenses,AssignedPlans,City,CompanyName,ConsentProvidedForMinor,Country,CreationType,Department,Display
Name,GivenName,OnPremisesImmutableId,JobTitle,LegalAgeGroupClassification,Mail,MailNickName,MobilePhone,OnPremisesSecurityIdentifier,OtherMails,PasswordPolicies,PasswordProfile,PostalCode,PreferredLanguage,ProvisionedPlans,OnPremisesProv
isioningErrors,ProxyAddresses,RefreshTokensValidFromDateTime,ShowInAddressList,State,StreetAddress,Surname,BusinessPhones,UsageLocation,UserPrincipalName,ExternalUserState,ExternalUserStateChangeDateTime,UserType,OnPremisesLastSyncDateTi
me,ImAddresses,SecurityIdentifier,OnPremisesUserPrincipalName,ServiceProvisioningErrors,IsResourceAccount,OnPremisesExtensionAttributes,DeletedDateTime,OnPremisesSyncEnabled,EmployeeType,EmployeeHireDate,CreatedDateTime,EmployeeOrgData,p
referredDataLocation,Identities,onPremisesSamAccountName,EmployeeId,EmployeeLeaveDateTime,AuthorizationInfo,FaxNumber,OnPremisesDistinguishedName,OnPremisesDomainName,IsLicenseReconciliationNeeded,signInSessionsValidFromDateTime,SignInAc
tivity HTTP/2.0 403 Forbidden Cache-Control: no-cache Vary: Accept-Encoding Strict-Transport-Security: max-age=31536000 request-id: OBFUSCATED client-request-id: OBFUSCATED
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"OBFUSCATED"...}} x-ms-resource-unit: 1 Date: Wed, 24 Jul 2024 18:07:54 GMT Content-Type: application/json
Content-Encoding: gzip  {"error":{"code":"Authentication_RequestFromUnsupportedUserRole","message":"User is not in the allowed
roles","innerError":{"date":"2024-07-24T18:07:54","request-id":"OBFUSCATED","client-request-id":"OBFUSCATED"}}}

Module Version
ModuleType Version    PreRelease Name
---------- -------    ---------- ----
Script     0.11.0     preview    Microsoft.Graph.Entra

Microsoft.Graph.Entra 0.11.0 preview
Environment Data
Name                           Value
----                           -----
PSVersion                      7.4.3
PSEdition                      Core
GitCommitId                    7.4.3
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Additional context
I suppose this is probably in that gray zone between a bug and a feature; sorry if I chose the wrong template.  :)

@kkgthb Thanks for raising the bug, we are looking into it.

Can we clarify the concern here -- I don't think it is a design goal to emulate Get-MgUser. In fact, the goal was to behave like Get-AzureAdUser. If this command behaves like the latter, this seems by design. If there is a workaround like explicitly specifying Properties then it seems like there is no issue here; we should though clarify the design philosophy in the repository and / or documentation.

In short term, Entra PowerShell goal should be an easier migration from the AzureAD module to the Microsoft Graph-based commands, however, it shouldn't blindly follow AzureAD's design decisions.
There must be a reason why Microsoft Graph PowerShell SDK commands return just a subset of Entra objects' properties by default, right?
The problem with Graph PowerShell is that specifying the -Select/-Property parameter will return only the specified properties and neglect the default properties. That's absolutely unexpected behavior for PowerShell users. Azure PowerShell team fixed that introducing the -AppendSelected parameter.
For example:
Get-AzADUser returns default properties
Get-AzADUser -Select returns ONLY selected properties
Get-AzADUser -Select -AppendSelected returns default+selected properties
Why couldn't this be implemented in Entra PowerShell and give users a user-friendly control on returned properties?
Most of the users of the Get-EntraUser command wouldn't even notice the difference between Get-MgUser and Get-EntraUser, if SignInActivity property (returned by default with Get-EntraUser) didn't require an additional permission, AuditLog.Read.All.
That means that Microsoft is forcing users of Get-EntraUser to have User.Read.All and AuditLog.Read.All permissions even when users aren't interested in the SignInActivity property. How is that going with the least privilege model?

@alexandair : The -AppendSelected feature is part of our priority list in a quarter or two.

In short term, Entra PowerShell goal should be an easier migration from the AzureAD module to the Microsoft Graph-based commands, however, it shouldn't blindly follow AzureAD's design decisions.
There must be a reason why Microsoft Graph PowerShell SDK commands return just a subset of Entra objects' properties by default, right?
The problem with Graph PowerShell is that specifying the -Select/-Property parameter will return only the specified properties and neglect the default properties. That's absolutely unexpected behavior for PowerShell users. Azure PowerShell team fixed that introducing the -AppendSelected parameter.
For example: Get-AzADUser returns default properties
Get-AzADUser -Select returns ONLY selected properties
Get-AzADUser -Select -AppendSelected returns default+selected properties
Why couldn't this be implemented in Entra PowerShell and give users a user-friendly control on returned properties?
Most of the users of the Get-EntraUser command wouldn't even notice the difference between Get-MgUser and Get-EntraUser, if SignInActivity property (returned by default with Get-EntraUser) didn't require an additional permission, AuditLog.Read.All.
That means that Microsoft is forcing users of Get-EntraUser to have User.Read.All and AuditLog.Read.All permissions even when users aren't interested in the SignInActivity property. How is that going with the least privilege model?

I think -AppendSelected is a great idea -- looking forward to it! And agreed that optimizing for AzureAD expectations is probably the most expedient way to go.
The last observation about permissions is not strictly true -- Get-EntraUser does not fail if you don't have those permissions -- those properties simply don't get returned. This is in turn just a reflection of how the underlying MS Graph API functions (https://graph.microsoft.com/v1.0/users/).
As far as "why does Get-MgUser act the way it does with fewer properties, as far as I can tell there is actually not a reason for this other than "That's how the MS Graph API works," i.e. Get-MgUser == 'https://graph.microsoft.com/v1.0/users/. So if you try that URI in Graph Explorer for instance you *should* see the same properties that are getting returned by default with Get-MgUser. (I am saying this without looking at the code for Get-MgUser, perhaps somehow it does indeed limit the properties, but it is well-known that v1.0` only returns "default" properties).
One thing to try with Get-MgUser -- does Get-MgBetaUser return more properties? If so, that's not a surprise either because the underlying beta API version of MS Graph returns all properties by default.
So the claim I'm making here is that Get-MgUser just happens to do whatever the underlying API does. The behavior of MS Graph API was optimized for generic app scenarios and not specifically PowerShell use cases for instance (certainly not AzureAD PS parity). So to me Get-MgUser default behavior is not a benchmark for that of other graph-based PowerShell experiences.

A little more about default properties in MS Graph API on which my Get-MgUser theory is based: https://stackoverflow.com/questions/48229949/get-all-user-properties-from-microsoft-graph.
Maybe just look at your request URI when issuing Get-MgUser (via -debug?) to see if it's limiting the result set itself or going with default MS Graph API behavior.

A little more about default properties in MS Graph API on which my Get-MgUser theory is based: https://stackoverflow.com/questions/48229949/get-all-user-properties-from-microsoft-graph.
Maybe just look at your request URI when issuing Get-MgUser (via -debug?) to see if it's limiting the result set itself or going with default MS Graph API behavior.

I forgot to include -debug information from Get-MgUser.

Get-MgUser -debug reveals that it does, indeed, merely call https://graph.microsoft.com/v1.0/users/the_user_id.
Whereas Get-EntraUser -debug calls https://graph.microsoft.com/v1.0/users/the_user_id?$select=long_list_of_hard_coded_properties.

This is why I suggested that:

Even if for some reason Microsoft is absolutely positively convinced that people want Microsoft to offer a mechanism by which they make it clear which properties are "all the properties" (that is, be the ones to maintain a trustworthy, official copy of the string "Id,AccountEnabled,...,SignInActivity" somewhere handy)...
Microsoft should still at least make Get-EntraUser try behaving like Get-MgUser if the hardcoding returns an Authentication_RequestFromUnsupportedUserRole before completely erroring out.

Get-EntraUser should gracefully handle exceptions arising from Get-AzADUser's slightly awkwardly paternalistic "if you don't specify a $select, I'm going to pick one for you" behavior...
...and try one more time with Get-MgUser's "hey, let's just not presume we know what $select the user meant if they didn't specify one and leave it off the underlying Graph call altogether" approach.

Or, if that would break a contract that Microsoft can't go back on their word about saying something like:

Get-EntraUser will, forever and for all time, never behave more maturely than Get-AzADUser did, so that no one ever ever ever has to refactor code... (ahem:)

In short term, Entra PowerShell goal should be an easier migration from the AzureAD module to the Microsoft Graph-based commands, however, it shouldn't blindly follow AzureAD's design decisions.

... Then Microsoft should at least add something like a concise Get-EntraUser -NoSelect or Get-EntraUser -MgStyle flag that lets people who currently have no problems using Get-MgUser enjoy Get-EntraUser without having to type 50-bazillion things into -Properties by hand to get around Get-EntraUser's insistence on telling you "you'll take these $selects and you'll like it!" default behavior.

Because clearly not everyone does "like it."

The current set is really bad!
Please don't ask hard coded for everything but at least check if the user has permissions to read a property

Changing topics slightly to clear up a detail:

Most of the users of the Get-EntraUser command wouldn't even notice the difference between Get-MgUser and Get-EntraUser, if SignInActivity property (returned by default with Get-EntraUser) didn't require an additional permission, AuditLog.Read.All.
That means that Microsoft is forcing users of Get-EntraUser to have User.Read.All and AuditLog.Read.All permissions even when users aren't interested in the SignInActivity property. How is that going with the least privilege model?

For what it's worth, Get-MgUser doesn't only work without AuditLog.Read.All.
Get-MgUser and the underlying Graph API also work gracefully even without User.Read.All.
As documented in the original bug report, when I run Connect-Entra without a -Scopes flag at all, my (Get-EntraContext).Scopes includes neither User.Read.All nor AuditLog.Read.All.
That is, (Get-EntraContext).Scopes.Contains('User.Read.All') is false for me.
I'm not sure which of the 19 scopes I end up with is responsible for https://graph.microsoft.com/v1.0/users/the_user_id (URL-parameterless; no $select at all) and thereby Get-MgUser working for me, but clearly the Graph API itself doesn't require me to even have User.Read.All.


nixos/dhcpcd: Expose less information to servers

Motivation for this change
dhcpcd exposes a lot of information which is IMO not necessary and a possible security risk because of exposed version numbers.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

Sounds OK, superficially – I know almost nothing about dhcpcd or the protocol.

@vcunat To elaborate more: The data is sent during DHCPv6 solicitation (Wireshark Info: Solicit). The package looks like this (look at the last two):

It surely works here with my router, but I don't know if other routers could fail. I doubt it but you never be sure…

I don't think we should make this change because some routers may reject this vendorclassid, and the user cannot override it in extraConfig. As per dhcpcd.conf(5):

vendclass en data
Add the DHCPv6 Vendor Indetifying Vendor Class with the IANA assigned Enterprise Number en with the data.  This option can be set more than once to add more data, but the
behaviour, as per RFC(3925) is undefined if the Enterprise Number differs.

So I'm afraid it's up to the user to set this manually if they want to expose less information.

@xeji So do you think it would be okay if I do it as optionalString?

I could see this being an optional value but it being open to the user to customize. In a datacenter, pxebooting machines could declare some other options to the dhcp server for it to take some other action. But I agree with @xeji that the default shouldn't change.

@cransom That's not a new feature compared to the currently existing extraConfig functionality. I'll close this PR, users can add the appropriate line to their configuration if they wish.

Totally agree. We should rather keep the options clean


azdata-cli installation fails when invoked from within Azure Data Studio or from command line

Azure Data Studio Version: Version: 1.27.0
Commit: 765c7064d46253d88393e0e1ff78874b6fe8fa64
Date: 2021-03-16T21:18:54.187Z
VS Code: 1.51.0
Electron: 9.4.3
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 20.4.0

Steps to Reproduce:

Launch Azure Data Studio;
Immediate upon start, Azure Data CLI extension complains that "Could not find Azure Data CLI, install it now? If not then some features will not be able to function."
Error is obtained (always): "Error installing Azure Data CLI: Unexpected exit code from command: Error: Cannot install under Rosetta 2 in ARM default prefix (/opt/homebrew)! To rerun under ARM use: arch -arm64 brew install ... To install under x86_64, install Homebrew into /usr/local. (1)"

Are you using an M1 ARM-based Mac? The error seems to be from that - could you try following the instructions on this page to install the CLI directly?
https://docs.microsoft.com/en-us/sql/azdata/install/deploy-install-azdata-macos?view=sql-server-ver15
And then if/when you get the same error above as I expect try running arch -arm64 brew install azdata-cli instead?

Affirmative. I already did that a couple of weeks ago and as stated at the top of the article, macOS support is only provided up to 10.15, excluding this way 11.x Big Sur. It looks to me there is already plenty of Microsoft software running on arm64, why not AzData?
Regarding the reference to brew that was also attempted (specifying the target architecture by prefixing the command with "arch -arm64"), the same errors as before:
==> /opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/bin/pip install -v --no-deps --no-binary :all: --ignore-installed /private/tmp/azdata-cli--gssapi-20210412-3257-a5w0fb/gssapi-1.6.12
Last 15 lines from /Users/user/Library/Logs/Homebrew/azdata-cli/112.pip:
requirement.install(
File "/opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 856, in install
six.reraise(*exc.parent)
File "/opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/lib/python3.8/site-packages/pip/_vendor/six.py", line 703, in reraise
raise value
File "/opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/lib/python3.8/site-packages/pip/_internal/operations/install/legacy.py", line 74, in install
runner(
File "/opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 273, in runner
call_subprocess(
File "/opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess
raise InstallationError(exc_msg)
pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-req-build-sfsa10xl/setup.py'"'"'; file='"'"'/private/tmp/pip-req-build-sfsa10xl/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-0li4bpvv/install-record.txt --single-version-externally-managed --compile --install-headers /opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/include/site/python3.8/gssapi Check the logs for full command output.
WARNING: You are using pip version 20.2.4; however, version 21.0.1 is available.
You should consider upgrading via the '/opt/homebrew/Cellar/azdata-cli/20.3.2/libexec/bin/python3 -m pip install --upgrade pip' command.
Removed build tracker: '/private/tmp/pip-req-tracker-5va5wyyk'
Do not report this issue to Homebrew/brew or Homebrew/core!

I talked with the owners of the azdata CLI and the status currently is that they don't support Big Sur/ARM64 yet but will be looking into that going forward.
As this is an external issue I will close this for now since there's nothing we can do from the ADS side. If I have any updates I'll let you know!


#525 Update six requirement to <=1.15.0

Changelog: (Fix): Update six requirement to <=1.15.0

[x] Refer to the issue that supports this Pull Request.
[x] If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
[x] I've read the Contributing guide.
[x] I've followed the PEP8 style guides for Python code.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Rishi Theivendran seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Log errors in plugin construction and use.

Currently, the test suite is cluttered with the new console logs. Not sure how y'all would like to resolve that. @nnance ?

@estk You can override console.error with a function that does nothing just before you call the function you expect to error, then set it back to console.error right after.

@estk on second thought, I don't think we should log errors this way. I think the proper way to do it is to either
a) log all the errors in a middleware - this requires no code changes.
b) log all the errors with the help of the logFunction which gets passed to Apollo Server.
If you want to do a), no PR is needed. If you want to do b), please open a new PR, I'll close this one.


Add sieve rules

Refs:

https://support.tigertech.net/sieve
https://protonmail.com/support/knowledge-base/sieve-advanced-custom-filters/
https://xenoworld.de/knowledge/sieve.html
https://arstechnica.com/information-technology/2014/03/taking-e-mail-back-part-3-fortifying-your-box-against-spammers/
https://github.com/leagris/githubsieve
https://dovecot.org/list/dovecot/2014-September/097846.html

Duplicate #410


Migration from self hosted gitlab does not work with project name in URL

Migration from self hosted gitlab-ce (checked many different versions with api v4: 9.x, 10.x, 11.x, 12.x and finally 13.3.7) to gitea v1.13 does not work if source repository URL contains project namespace + name like
https://my.gitlab.domain/me/myproject.git
https://my.gitlab.domain/me/myproject
Gitea throws error:

modules/task/task.go:51:handle() [E] Run task failed: GET https://my.gitlab.domain/api/v4/projects/me/myproject: 404 failed to parse unknown error format

Opening this URL in web browser returns

{"error":"404 Not Found"}

URL-s that work in web browser:

list of repos: https://my.gitlab.domain/api/v4/projects
repo id=10 details: https://my.gitlab.domain/api/v4/projects/10

Tested that migration from such gitlab to gitea works when source URL is specified with repo ID not name like this:
https://my.gitlab.domain/10
Please verify and fix or (if using ID is required) update instruction in UI for users to know what to enter as URL.

I'm also having problems with mirroring from a self-hosted gitLab instance. However using the ID does not work for me. Have you tested the ID method with private repos?

Have you tested the ID method with private repos?
As far as I remember there were internal repos also (limited access for logged in users with repo permissions assigned) but all were from "organization" namespaces (which user had permissions to) not user private namespace. Unfortunatelly no source gitlab available to test it.

I tested a gitlab migration and it worked on my side ... (v1.14.0-dev 553a62cb389aee83d7dea230dca6f5d1c8cea423)

same issue happend here: https://github.com/go-gitea/gitea/pull/13646#issuecomment-731973386
I think in this case it's not gitea's fault, since the migrator relay on a working api on the gitlab side ...

after 4 years, the issue is still there when  I am using Gitlab CE v17.3 .   Also, I checked gitlab API document as following https://docs.gitlab.com/ee/api/projects.html#get-single-project
it seems that Gitlab don't have api like https://gitlab.server.domain/api/v4/projects/group/project-name ,  and it has only api like  https://gitlab.server.domain/api/v4/projects/:id


Fixing some outdated code

Hi,
this contains some bugfixes due to, I'm assuming, newer versions of the dependencies.
The name_scope change might not be what you intended, I'm happy to change it as requested.
I might also make some more changes to update the test + notebook.
Best

Awesome! Thanks! Let me know when you're happy with it and I'm happy to merge.

Awesome! Thanks! Let me know when you're happy with it and I'm happy to merge.

Hi,
upon noticing that there is a function tfp.math.batch_interp_regular_nd_grid() that seems to do the same, I decided that I will be using the tfp function (my project already depends on tfp anyway). So, I probably won't get back to updating this PR further, sorry.
You may still merge, since this makes it at least usable with recent versions of tensorflow. But I can't really comment on whether e.g. the change with tf.name_scope() captures the original intent properly, since I'm rather new to tensorflow.
Best, Thomas

Hi,
upon noticing that there is a function tfp.math.batch_interp_regular_nd_grid() that seems to do the same, I decided that I will be using the tfp function (my project already depends on tfp anyway). So, I probably won't get back to updating this PR further, sorry.
You may still merge, since this makes it at least usable with recent versions of tensorflow. But I can't really comment on whether e.g. the change with tf.name_scope() captures the original intent properly, since I'm rather new to tensorflow.
Best, Thomas

No worries at all. Thanks!

No worries at all. Thanks!


Flutter ImagePicker Plugin (MissingPluginException - No implementation found for method pickImage)

I am using image_picker 0.6.1+8 . I am getting the following exception:
_[VERBOSE-2:ui_dart_state.cc(148)] Unhandled Exception: MissingPluginException(No implementation found for method pickImage on channel plugins.flutter.io/image_picker)
#0      MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:319:7)
<asynchronous suspension>
#1      ImagePicker.pickImage (package:image_picker/image_picker.dart:62:40)
<asynchronous suspension>_

Here are my observations:

Happens only on iOS. The plugin works perfectly on android.

I debugged the plugin a little bit and found its "registerWithRegistrar" method to be called correctly inside the plugin. However, the "handleMethodCall" never gets invoked in the plugin. The exception gets thrown when InvokeMethod is called.

A handler in our appdelegate.m gets called if we register a handler using the plugin channel there. However, the plugin "handleMethodCall" is not invoked.

Other plugins like shared_preferences also have the same issue.

My flutter doctor output
flutter doctor -v
[✓] Flutter (Channel beta, v1.10.7, on Mac OS X 10.14.3 18D109, locale en-SA)
    • Flutter version 1.10.7 at /Users/hashir/Developer/flutter
    • Framework revision e70236e36c (3 weeks ago), 2019-10-02 09:32:30 -0700
    • Engine revision 9e6314d348
    • Dart version 2.6.0 (build 2.6.0-dev.0.0<PHONE_NUMBER>)

[!] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/hashir/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 28.0.3
    ✗ Android SDK file not found: /Users/hashir/Library/Android/sdk/platforms/android-29/android.jar.
    • Try re-installing or updating your Android SDK,
      visit https://flutter.dev/setup/#android-setup for detailed instructions.

[✓] Xcode - develop for iOS and macOS (Xcode 10.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.3, Build version 10G8
    • CocoaPods version 1.8.0

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 39.0.3
    • Dart plugin version 191.8423
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] Connected device (1 available)
    • iPhone Xʀ • 4A93D602-957E-4BE2-8E5E-2F5B931590CB • ios • com.apple.CoreSimulator.SimRuntime.iOS-12-4 (simulator)

we had converted an existing iOS app into flutter. We did it long time back and it was not as per the guidelines specified in https://github.com/flutter/flutter/wiki/Add-Flutter-to-existing-apps.
Following the guidelines mentioned in the wiki to convert existing ios app into flutter solves the missing plugin exceptions.

MissingPluginException(No implementation found for method  pickImage on channel plugins.flutter.io/image_picker)


Add callback to obtain handshake response as client in case of handshake failure.

This provides a listener callback to obtain the server's handshake response in case it was rejected with a HTTP status code != 101.
E.g. if the server rejects the WebSocket upgrade request by issuing a
HTTP response with 503 status code, then this HTTP response data can be
obtained via
WebSocketListener.onWebsocketHandshakeReceivedAsClientFailed(WebSocket,
ClientHandshake, ServerHandshake).
If there already is any other way of obtaining the server's response including http status code, please enlighten me :) Thanks / Christoffer

This issue is solved with #572


README.md: point out that termbox is downloaded at compile time

(This commit also adds a link to the termbox repository to the readme file.)
I'm not sure if this commit might come across a little snarky; if so, I apologize.

I didn't consider it snarky at all, thanks for this!


Support berlin hardfork

Expected Behavior
ganache-cli --hardfork berlin should work and behave like other Ethereum nodes running on Berlin hardfork.
Current Behavior
Error: Hardfork with name berlin not supported
Possible Solution
I guess the solution is obvious, the EIPs in https://github.com/ethereum/eth1.0-specs/blob/master/network-upgrades/berlin.md need to be implemented and followed when that hardfork setting is there. As I guess this is a ganache-core issue at the core, I filed https://github.com/trufflesuite/ganache-core/issues/858 as well.
Context
I like the fact that with tests running against ganache, I can run the same EVM code I deploy on Ethereum testnets and mainnet and that I get the same gas usage numbers so I can accurately estimate what gas usage on live setups will be.
Currently, that's not possible for Berlin hardfork.
Your Environment

Version used: Ganache CLI v6.12.2 (ganache-core: 2.13.2)
NodeJS Version: v10.24.0
Operating System and version (include distro if Linux): openSUSE Linux

Hi,
When is this expected to close?


ion-app freezes parent page when navigating back

In a Vue app with ionic 4 components
In a page (parent) when I navigate to another page (child) that has  as the first tag after

+1


Handicap fix

Coverage decreased (-0.04%) to 92.894% when pulling c5d8b5ba008dd985bc67d88ec21a41c1a07b9d64 on handicap-fix into 605cceca87b6fab7734c789027ef3bf7a034ec13 on master.


clear render interval on unload

With the most recent version of react, was getting a console full of errors from Canvas.js, trying to get the 2d context of a non-existent canvas node.
Not sure if this is the best solution, but it seems to do the trick. Just clearing the 30ms interval on unmount.

Thanks, since it works for now it'll do!  I dont have much time to work on this as of late..  Thanks again for the fix.

@MattMcFarland , please merge this

Merged, thanks!!!


Basin-hopping changes attempt 2

I should have been more careful in https://github.com/scipy/scipy/pull/7954
I added a test, and it failed, because the energy values fed to Metropolis are arrays, not floats, and causing a RuntimeWarning.  This should fix it.
Also made some changes to the docstrings
(Also all the whitespace and PEP8 things that Spyder complains about)

Any issues with this?

Looks fine to me. One question: why did you delete the test func1d_nograd?

Note that we're quite short on reviewer power (and I have almost no time myself until early Jan), so if something is really ready to go an extra poke is a good idea.

why did you delete the test func1d_nograd?

because it's 1. not being used and 2. identical to func1d, which doesn't make sense since it's supposed to be a nograd version

Okay thanks. In it goes then. Merged.


Demo webstite not found - error 404

Bug Report
Demo webstite not found - error 404
http://www.reactvirtualized.com/

Screenshots

Maybe should be URL address be edited to 👉 https://bvaughn.github.io/react-virtualized/ ? 🤔


TypeError: 'ST7565' object does not support 'protocol_framebuffer'

The framebufferio module doesn't work with this display class, and yet somehow the very similar SharpMemoryDisplay does…
import framebufferio

f = framebufferio.FramebufferDisplay(mydisplay)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'ST7565' object does not support 'protocol_framebuffer'

I was trying to get all the fancy displayio stuff like groups and tilegrids to work with a display that has this chipset, and based on what I've been able to do with the Sharp Displays and their deceptively similar driver code I wound up here… It's not make-or-break for my project to have framebufferio, but looking at this code versus the Sharp Display driver I don't see how one is using a framebuffer and the other isn't since they both subclass adafruit_framebuf.FrameBuffer

@tannewt other way around - its framebuf derivative not displayio - we don't stock anything with this chipset anymore so we havent ported it to displayio!

Awesome, thank you!


Layers: Inactive dynamic bindings may interfere with dyamic_offset validation

Performing dynamic_offset checks against bindings which include inactive bindings returns unexpected results.
It appears that the check in descriptor_sets.cpp fails because the index-based mapping between setBindings and dynamic_offsets can get broken if there are inactive bindings.

Example:
I'm using 2 dynamic UBOs which bind to memory buffer A (binding 0, binding 1), and another dynamic UBO (binding 2) bound to a smaller buffer B. upon draw each binding is bound with an offset. The Ubo for Binding 1 is not used by the shader, and might have been optimised away.
The error message - using current github master validation layers compiled from source is the following:
DS 0x3e encountered the following validation error at vkCmdDrawIndexed() time: Dynamic descriptor in binding #2 at global descriptor index 2 uses buffer<PHONE_NUMBER>000023 with dynamic offset 16777472 combined with offset 0 and range 48 that oversteps the buffer size of 16777216.

When I check my data, it appears that offset, range, and buffer size are correct, and the error message prints out correct and expected values for these. But dynamic_offset (the value '16777472' above) refers to the buffer offset previous to the buffer that needs to be checked.
My hypothesis is that whilst only active binding slots are checked for dynamic_offset correctness, the vector of dynamic_offsets passed to cvdescriptorset::DescriptorSet::ValidateDrawState may contain offsets for active and inactive slots alike.
Printf-style debugging when compiling validation layers locally showed me that in this particular case there were indeed 2 bindings being checked (the descriptorSetLayout for this set contains 3, of which one inactive) against 3 dynamic_offsets.
In core_validation.cpp where this validation is prepared, the bindings appear to be grabbed only for pPipe->active_slots.
Might be relevant for #880

@tgfrerer Your hypothesis was correct. We always have the full dynamic offset array during draw-time validation, but we only looped over the active bindings. Added a mapping between bindings and dynamic offset array to fix the bug which has been pushed to master. Thanks!

Brilliant, thanks =)


cut section from drawing

is it possible to add a feature which allows you to create secondary internal borders, or failing that slice a drawinging in two so the edges of the slice can be worked?

Technically possible of course, but the problem is that the holes would vanish if the module is disabled. Also other modules working with the shape of the drawings wouldn't know of the holes, because technically the shape can only be polygon, circle/ellipse, rectangle (a pixi.js shape); so this might cause problems too. Initially I experimented with this idea, but I decided against it for those reasons and the fact that I would need override a lot making a bigger mess.


regular expression fails if there is .raw file in the folder

The following code for max number in the folder fails if there is .raw file inside.
r = re.compile(rf"^{filename}_(\d*)")
# regex objects which match the upper statement

reg = map(r.search, captures_list)

# getting max from the group 1 <=> index

max_index = max(list(map(lambda match: int(match.group(1)), reg)))

The reason is that "^{filename}_(\d*)" regular expression group (\d*) also takes letters within and than the int cast fails.

Fixed and improved!
r = re.compile(rf"^{filename}_(\d+)")
# regex objects which match the upper statement
reg = filter(lambda item: item is not None, map(r.match, captures_list))
# getting max from the group 1 <=> index
max_index = max(map(lambda match: int(match.group(1)), reg))


Expand AudioOffloadListener functionality

Use case description
The current interface doesn't provide actual status of offload (whether is enabled on the AudioTrack, not whether it's sleeping). It doesn't allow apps making decisions to keep using offload support after a write error, which may be transient.
Requests:

call a new method AudioOffloadListener.isOffloadedPlayback with the value of AudioTrack.isOffloadedPlayback.  This may be true, even when sleepingForOffload is false.
Allow AudioOffloadListener to be involved in the DefaultAudioSink maybeDisableOffload decision, such as trying additional times with offload enabled.
Expose the reason for offload being disabled, such as Audio Init errors, or write errors, or not supported for format.

Proposed solution
Add additional methods to AudioOffloadListener that affect the
Alternatives considered
Expose AudioTrack and additional logging.

Potentially also, allow an offload only mode DefaultSudioSink.OFFLOAD_MODE_ENABLED_ONLY ?

I think we agreed I'd put up a speculative CL with just

call a new method AudioOffloadListener.isOffloadedPlayback with the value of AudioTrack.isOffloadedPlayback. This may be true, even when sleepingForOffload is false.

and another one that avoids the disabling retry on a write error.

Offload integration is still experimental for now and doesn't expose all the configuration needed.

The final state will likely have a getter similar to `isOffloadedPlayback. I'll keep this enhancement open to track this.
Error recovery could be made configurable, but it's less important and also not currently done for decoder fallback and similar logic elsewhere. (We only have configurable error handling on the loading side at the moment).


Add new repos to repo_list.md

Add PostgreSQL-related projects to the repo_list.md.

Hi @Baoyuantop , PTAL. Thanks!

Hi @Baoyuantop could you help review this PR?


Drop wildcard support for address/label search in AddressBookPage

Wildcard support for address/label search in AddressBookPage was introduced in bitcoin/bitcoin#12080, and it is available in release binaries since v0.17.
But this feature has never being documented or mentioned in the GUI.
Thus, there are no reasons to believe that users rely on this feature.
Required for migration to Qt 6, because:

There is no direct way to do wildcard matching in QRegularExpression.

Tests included.
Based on bitcoin-core/gui#591, so only last 2 commits belong to this PR.

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts
Reviewers, this pull request conflicts with the following ones:

#585 (Replace QRegExp with QRegularExpression by prusnak)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

cc @promag

"However, the wildcardToRegularExpression method is provided to translate glob patterns into a Perl-compatible regular expression that can be used for that purpose."
Seems like it should be trivial to retain this feature, if we want to?

~0 on this. As I've said on IRC I don't particularly care for this feature to go (I don't think the entire Address Book is particularly useful, searching in the Transactions tab should cover it?), though "being compatible with Qt 6" is a strange motivation. As @luke-jr says it would still be possible by rewriting the expression.

Rebased on top of the merged #591.

@luke-jr

"However, the wildcardToRegularExpression method is provided to translate glob patterns into a Perl-compatible regular expression that can be used for that purpose."
Seems like it should be trivial to retain this feature, if we want to?

Would you mind provide a working example please?

Combined into #593.


Consider using Astronomy package

@meteor-gazelle/core-team This is a more full bodied package compared to simple schema which provides a model layer.
https://github.com/jagi/meteor-astronomy
It may be beneficial to use this as it provides some additional structure, especially as the code base grows.
Going to research this a bit more.

Seems interesting, I think this will be very beneficial in abstracting lot of data manipulation.

Oh that looks nice!

Definitely worth a look in to, what are the alternatives for this? How is this currently handled and reasons for including this?

This looks cool, definitely worth digging into more.

I refactored much of my code last night from SimpleSchema's to Astronomy classes. So far Im really enjoying what this model layer package has to offer. Document level methods, hooks into various document events (beforeInsert, afterInsert, and much more), Document inheritance, custom validators.
So far its a big +1 from me...

@belairWCD the primary alternative is simple-schema.[0] This does a lot of the same thing, however with more features and (IMO) a nicer API. I suggest reading the README for astronomy, it covers the why and most of what the library can do. I especially like the object layer. The code is also pretty friendly; I expect we'll probably need to contribute back upstream a bit, as tends to happen when a library is a large part of your code. :)
So far it doesn't seem like anyone is against this, and we've started implementing announcements[1] as a small proof of concept. Should we just roll ahead with this?
[0] https://github.com/aldeed/meteor-simple-schema
[1] https://github.com/meteor-gazelle/meteor-gazelle/pull/151/files

@jimrollenhagen We're rolling. Closing issue.