How to provide AVP configuration in Sidecar Container

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots/Verbose output
If applicable, add screenshots to help explain your problem.
If you've tried running argocd-vault-plugin generate with --verbose-sensitive-output to help debug, please include that output here after redacting any secrets.
Additional context
Add any other context about the problem here.

In the method via argocd-cm ConfigMap i am supplying my vault configuration as environment varible in argocd repo server as mentioned in below code
 envFrom:        - secretRef:            name: argocd-vault-plugin-credentials

But how can we apply these vault configuration  via sidecar ?

Is this Method will work without providing plugin '{}' in application manifest?

Yes, this part is missing in the documentation and should be added

@NitishGupta9282 You can see the Argo CD docs here https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/#using-environment-variables-in-your-plugin
The system environment variables (of the repo-server container for argocd-cm plugins or of the sidecar for sidecar plugins). If using sidecar, you attach the env vars to the sidecar container.

I am having a similar issue while I am trying to setup ArgoCD with AWS secret manager and vault plug-in has been setup as sidecar container where I am getting below error:
time="2023-06-27T10:02:45Z" level=error msg="argocd-vault-plugin generate . failed exit status 1: Error: Must provide a supported Vault Type\nUsage:\n  argocd-vault-plugin generate <path> [flags]\n\nFlags:\n  -c, --config-path string   path to a file containing Vault configuration (YAML, JSON, envfile) to use\n  -h, --help                 help for generate\n  -s, --secret-name string   name of a Kubernetes Secret in the argocd namespace containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD). The namespace can be overridden by using the format <namespace>:<name>" execID=df3e0
this is configuration with ENV variable which is being shown on argoCD-repo server:
kubectl exec argo-cd-argocd-repo-server-7c4599c44d-qbqsk -it /bin/sh -n argo-cd
$ env | grep AVP
AVP_TYPE=awssecretsmanager
$ env | grep AWS
AWS_REGION=us-west-2

here is how i have built sidecar container with ArgoCD helm chart value file:
configs:
  cmp:
    create: true
    plugins:
      argocd-vault-plugin:
        allowConcurrency: true
        discover:
          find:
            command:
              - sh
              - "-c"
              - "find . -name '*.yaml' | xargs -I {} grep \"<path\\|avp\\.kubernetes\\.io\" {} | grep ."
        generate:
          command:
            - argocd-vault-plugin
            - generate
            - "."
        lockRepo: false
      argocd-vault-plugin-helm:
         generate:
          command: ["sh", "-c"]
          args: ["helm template . | argocd-vault-plugin generate -"]
      argocd-vault-plugin-helm-with-args:
         generate:
          command: ["sh", "-c"]
          args: ["helm template ${helm_args} . | argocd-vault-plugin generate -"]
repoServer:
  volumes:
  - name: argocd-cmp-cm
    configMap:
      name: argocd-cmp-cm
  extraContainers:
    - name: avp1
      command: [/var/run/argocd/argocd-cmp-server]
      image: <vault-plugin_image>
      securityContext:
        runAsNonRoot: true
        runAsUser: 999
      volumeMounts:
        - mountPath: /var/run/argocd
          name: var-files
        - mountPath: /home/argocd/cmp-server/plugins
          name: plugins
        - mountPath: /tmp
          name: tmp
        # Register plugins into sidecar
        - mountPath: /home/argocd/cmp-server/config/plugin.yaml
          subPath: argocd-vault-plugin.yaml
          name: argocd-cmp-cm
    - name: avp2
      command: [/var/run/argocd/argocd-cmp-server]
      image: <vault-plugin_image>
      securityContext:
        runAsNonRoot: true
        runAsUser: 999
      volumeMounts:
        - mountPath: /var/run/argocd
          name: var-files
        - mountPath: /home/argocd/cmp-server/plugins
          name: plugins
        - mountPath: /tmp
          name: tmp
        # Register plugins into sidecar
        - mountPath: /home/argocd/cmp-server/config/plugin.yaml
          subPath: argocd-vault-plugin-helm-with-args.yaml
          name: argocd-cmp-cm
    - name: avp3
      command: [/var/run/argocd/argocd-cmp-server]
      image: <vault_plugin_image>
      securityContext:
        runAsNonRoot: true
        runAsUser: 999
      volumeMounts:
        - mountPath: /var/run/argocd
          name: var-files
        - mountPath: /home/argocd/cmp-server/plugins
          name: plugins
        - mountPath: /tmp
          name: tmp
        # Register plugins into sidecar
        - mountPath: /home/argocd/cmp-server/config/plugin.yaml
          subPath: argocd-vault-plugin-helm.yaml
          name: argocd-cmp-cm
  env:
    - name: AWS_REGION
      value: us-west-2
    - name: AVP_TYPE
      value: awssecretsmanager
  serviceAccount:
    automountServiceAccountToken: true

@harshchauhan1988 did you resolve this?


Simplify Error type

Error needed to implement Clone at some point, but it doesn't anymore. So it can be simplified a bit.

LGTM


squid:S2259 - Null pointers should not be dereferenced

This pull request is focused on resolving occurrences of Sonar rule
squid:S2259 - Null pointers should not be dereferenced.
You can find more information about the issue here:
https://dev.eclipse.org/sonar/coding_rules#q=squid%3AS2259
Please let me know if you have any questions.
George Kankava

Hi, Can you please re-submit the PR ?

I've merged it into the main branch... Thnks for the PR !!!

Hi,
Do you want me to just close this PR and create new one once again? Or maybe some other tips for doing this?
Thanks.

I’ve merged it into the main branch.
Thanks for the PR...
--
Sachin Handiekar
Sent with Airmail
On 11 February 2016 at 08:44:44, George Kankava<EMAIL_ADDRESS>wrote:
Hi,
Do you want me to just close this PR and create new one once again? Or maybe some other tips for doing this?
Thanks.
—
Reply to this email directly or view it on GitHub.

Great! happy to hear that. Thank you.
On Thu, Feb 11, 2016 at 12:45 PM, Sachin Handiekar<EMAIL_ADDRESS>
wrote:

I’ve merged it into the main branch.
Thanks for the PR...
--
Sachin Handiekar
Sent with Airmail
On 11 February 2016 at 08:44:44, George Kankava<EMAIL_ADDRESS>wrote:
Hi,
Do you want me to just close this PR and create new one once again? Or
maybe some other tips for doing this?
Thanks.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
https://github.com/sachin-handiekar/jInstagram/pull/145#issuecomment-182768685
.


ALSA library configuration in CentOS

I am running this package under CentOS.  However, I got the following error messages.
ALSA lib pcm.c:2501:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2501:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2501:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map
ALSA lib pcm.c:7476:(snd_pcm_slave_conf) missing field rate
ALSA lib pcm.c:7476:(snd_pcm_slave_conf) missing field rate
ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp
ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp

Is these error message related to the incorrect ALSA configuration?

Hi @chyan26
Quiet just uses PortAudio internally to interface with the sound card. I know some amount of spam here is normal, but the last two lines (Cannot open device /dev/dsp) look concerning. You might have luck searching for PortAudio and that error to see if you get any results.
Also worth noting that quiet will try to just attach to the default sound device but you could change the code so it doesn't. You can find where it picks a device id in encode_soundcard.c

But why the error message comes from ALSA library instead of the PortAudio,


Notebook Editor: julia starts with wrong cwd on remote host

Bug: Notebook Editor: julia starts with wrong cwd on remote host

Steps to cause the bug to occur
Opening a (Julia) .ipynb on a remote host (via Remote-SSH) starts a (remote) julia kernel as expected. The kernel process gets a<EMAIL_ADDRESS>which is good and tells julia to search for an environment defining Project.toml in the cwd.
However, the actual working directory of the julia kernel is set to my home directory on the remote host and not the directory of the ipynb. This way the kernel starts up with the wrong julia environment, which is very inconvenient (i.e. useless) and inconsistent to the behavior in jupyter lab.

Your Jupyter and/or Python environment
Please provide as much info as you readily know

Jupyter server running: N/A
Extension version: 2020.8.109390 (15 September 2020)
VS Code version: 1.49.0

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

The julia extension for VS code should be handling running Julia kernels now.


Tutorial “Index a logging dataset locally” is missing attribute “tenant_id” inside example log snippet

While working through the “Index a logging dataset locally” tutorial (https://quickwit.io/tutorials/tutorial-hdfs-logs/), I noticed a small omission: The example snippet for the log entry is missing the “tenant_id” attribute.
Here's how the first entry of the dataset looks like:
❯ curl --silent https://quickwit-datasets-public.s3.amazonaws.com/hdfs-logs-multitenants.json.gz | gunzip | head -n 1 | jq .
{
  "timestamp":<PHONE_NUMBER>,
  "severity_text": "INFO",
  "body": "PacketResponder: BP-108841162-<IP_ADDRESS>-1440074360971:blk_1074072698_331874, type=HAS_DOWNSTREAM_IN_PIPELINE terminating",
  "resource": {
    "service": "datanode/01"
  },
  "attributes": {
    "class": "org.apache.hadoop.hdfs.server.datanode.DataNode"
  },
  "tenant_id": 58
}

I'll create a quick PR.

Closed by #1705


User login

Merging user_login into user_profile_page so I can continue working on the user_profile_page with the latest changes in the user_login branch such as the navigation bar, splash screen and updated login/signup pages

Self Merging so I can start working and this PR doesn't need a formal review


Method not found on Azure Functions: "void Grpc.Core.CallOptions..ctor(Grpc.Core.Metadata, System.Nullable1<System.DateTime>, System.Threading.CancellationToken, Grpc.Core.WriteOptions, Grpc.Core.ContextPropagationToken, Grpc.Core.CallCredentials)"

What version of gRPC and what language are you using?
gRPC 1.20.1, CSharp
What operating system (Linux, Windows,...) and version?
MacOS 10.14.4 (18E226)
What runtime / compiler are you using (e.g. python version or version of gcc)
$ dotnet --version
2.2.106

$ func version
2.7.1149

$ dotnet build
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
...

What did you do?
I wrote an Azure Function to use gRPC 1.20.1 and connect to an external service.  When that didn't work, I rolled back to gRPC 1.18.0 and that did work. I made an example repro with instructions here: (https://github.com/jess-sheneberger/grpc-method-not-found-repro)
What did you expect to see?
I expected gRPC 1.20.1 and 1.18.0 to behave identically.
What did you see instead?
Azure Function with gRPC 1.20.1 throws an exception:
System.Private.CoreLib: Exception while executing function: TestHttpTriggerFunc. client-azure-function: Method not found: 'Void Grpc.Core.CallOptions..ctor(Grpc.Core.Metadata, System.Nullable1<System.DateTime>, System.Threading.CancellationToken, Grpc.Core.WriteOptions, Grpc.Core.ContextPropagationToken, Grpc.Core.CallCredentials)'.

Anything else we should know about your project / environment?
Interestingly, a similar client (https://github.com/jess-sheneberger/grpc-method-not-found-repro/tree/master/client-console) written to run as a console app does not exhibit the same behavior, so it seems to be the combination of something that changed from 1.18.0 to 1.20.1 and the Azure Function runtime.

This looks a lot like #18889 and may be a dupe, but my environment seemed sufficiently different to create a new issue. I found that issue first while googling my problem. Noticed there was a request for a repro, so I put one together here: (https://github.com/jess-sheneberger/grpc-method-not-found-repro)

i have the same proble, i try to init server from function az and response error Method not found: 'Grpc.Core.ServerServiceDefinition HotelsService.Hotels.BindService(HotelsBase)'.

https://github.com/grpc/grpc/issues/18889#issuecomment-495657430 mentioned that upgrading visual studio has helped. What VS version are you using and can you try to upgrade?

I actually don't have VS installed, just the .NET Core SDK, which is fairly recent at 2.2.106 (2.2.107 is most recent here: https://dotnet.microsoft.com/download)...I think msbuild gets installed as part of that bundle, so I'm not certain how to get to 16.1.* instead of 15.9.20 that msbuild shows

Is this still a problem?

Is the latest release still impacted?

Sorry, I'm not working on the Azure Function that was using GRPC anymore and I don't have the environment I was using.


Backport to 0.1: Shared must relinquish control to the executor if repolled

This backports #2136 to Futures 0.1. There isn't much to add on top of
what's mentioned in #2136: the same bug exists in Futures 0.1, and it'll
fail in the same way when polled in recent versions of Tokio (#2418).
Backporting to 0.1 allows codebases that still have some Futures 0.1
code around to still use newer versions of Tokio.

Any thoughts? :)

#2122 hasn't been released yet, so it's nice to publish the next release of 0.1 after merging this.
(We need to decide how long we should maintain 0.1 branch, but this should be fine given that we have another backport already merged.)

Could we get this merged and released? Having futures 0.1 being broken on newer versions of Tokio makes upgrading Tokio more painful than it should be :(

Thanks, I'll prepare the next release.

Thanks a lot @taiki-e !

Hey @taiki-e , I'm sorry to be a bit of a bother here, but has this made it into a new release yet? Thanks :)

Published 0.1.30, sorry for the delay.

Thanks very much!
On Mon, 5 Oct 2020 at 17:55, Taiki Endo<EMAIL_ADDRESS>wrote:

Published 0.1.30, sorry for the delay.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/futures-rs/pull/2154#issuecomment-703757707,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANIHVQJLTYFPESWHHN7PODSJH27HANCNFSM4M7RLPPQ
.


Can we remove workflow code from PowerShell Core?

ActionPreference.Suspend is not supported in PowerShell Core (it's only used for workflows, which are not in PowerShell Core). Today it just adds confusion to the code with checks to ensure that value isn't used where there doesn't need to be any. The checks should also be performed by a PSSA rule rather than in source code (similar to ActionPreference.Ignore, as was decided in #4348). Do we need to keep ActionPreference.Suspend checks around at this point or can we just remove the half-dozen checks related to it from the source code?
Further, if we remove these checks (for the same reason we removed the checks for ActionPreference.Ignore), do we need the ActionPreference.Suspend enumeration value kept around at all?
@SteveL-MSFT: For committee review.
[Update]
As per the committee review, all workflow code should be removed. Here is the list of items requiring removal:

[x] workflow keyword
[x] parallel keyword (used for parallel activities in workflow)
[x] sequence keyword (used for sequential activities in workflow)
[x] inlinescript activity block (used for activities outside of workflow)
[x] -parallel parameter on foreach statement
[x] Checkpoint-Workflow activity (command)
[x] Suspend-Workflow activity (command)
[x] ActionPreference.Suspend enumeration
[x] plus all related classes/methods

If this does not work in any case it can be removed.
We could use our suggestion system to inform users.

@PowerShell/powershell-committee reviewed this and agree but should broaden this to reviewing all the code and remove all PSWorkflow code

There is a lot of such code. Maybe step-by-step will be suitable.
Should we be worried about remoting?

Is there really a lot? Here is the list of things that need to be removed that I am aware of:

[ ] workflow keyword
[ ] parallel keyword (used for parallel activities in workflow)
[ ] sequence keyword (used for sequential activities in workflow)
[ ] inlinescript activity block (used for activities outside of workflow)
[ ] -parallel parameter on foreach statement
[ ] Checkpoint-Workflow activity (command)
[ ] Suspend-Workflow activity (command)
[ ] ActionPreference.Suspend enumeration
[ ] plus all related classes/methods

That's not that long of a list.

@KirkMunro Thanks for the review! You could move your check list in the issue description and rename header.
I expected that almost everything was already removed there since we had already cleaned the code several times. Therefore, I was a little surprised that there is still relatively a lot of unnecessary code.
I believe we will make the cleanup step-by-step. Perhaps we will need some discussions because there is public APIs that I already catched in #9618.

@iSazonov Good idea, and done.

There are good arguments for not removing keywords:

You can provide better error messages (workflow not supported vs. command not found)
It wouldn't be a breaking change to use the keyword in the future.

I also do not think that the keywords or parameter names should be moved entirely, and they are all already flagged as errors which is the right thing to do, in my opinion.   The only downside to this is that means the workflow keyword cannot be used for a function name without using & (call).

Part of the reasoning by @PowerShell/powershell-committee to remove is to have the errors at parse time/compile time (for C# code) vs at runtime.  There is no plan to ever bring back workflow.  Instead, Jeffrey has a proposal for workflow-like state management that I'll publish a RFC for eventually.

There is a public method on the Debugger class called GetWorkflowDebuggerFunctions that is not used internally and seems intended only for workflow. Given that is the case, can this (and the related const string function definitions) be removed?

@KirkMunro we've removed other workflow specific code that impacts public API, so now would be the time to remove it

@SteveL-MSFT PowerShellGet internally uses the IsWorkflow property of FunctionDefinitionAst objects. How do you want that handled? I have that property removed from PR #10223 at the moment, which causes some Pester tests to fail because that property does not exist.
Futher, since PowerShellGet is managed independently of PowerShell itself, why are there PowerShellGet tests that use whatever latest version a user has installed in the PowerShell test suite?

It is not time critical to leave the property as dummy, then cleanup PowerShellGet, and then remove the property from ast.

I'm concerned about removing the workflow keyword. An error message like Workflow is not supported in PowerShell 6+ is better than The term 'workflow' is not recognized as the name of a cmdlet ....

I'm concerned about removing the workflow keyword.

@daxian-dbw Please look discussion in #10223

It would be cool if PowerShell Core provided modularity on the workflow engine choice so that endusers could opt to choose their workflow adapter assembly. I have floated this idea in the UiPath/CoreWF discussion.


Issues with log levels in flee.py (i.e. camp, init, conflict) when set to 1 in simsetting.yml (FabFlee).

Camp or camp?

Traceback (most recent call last):
  File "/home/csstdds/Codes/flee/runscripts/run_par.py", line 78, in <module>
    e.evolve()
  File "/home/csstdds/Codes/flee/flee/pflee.py", line 21, in wrapper
    return func(*args, **kwargs)
  File "/home/csstdds/Codes/flee/flee/pflee.py", line 621, in evolve
    a.finish_travel(time=self.time)
  File "/home/csstdds/Codes/flee/flee/pflee.py", line 21, in wrapper
    return func(*args, **kwargs)
  File "/home/csstdds/Codes/flee/flee/pflee.py", line 104, in finish_travel
    super().finish_travel(time=time)
  File "/home/csstdds/Codes/flee/flee/flee.py", line 184, in finish_travel
    if self.location.Camp is True:
AttributeError: 'Location' object has no attribute 'Camp'. Did you mean: 'camp'?

Solution: flee.py
line 184: if self.location.Camp is True: change to lower letter c -> if self.location.camp is True:
line 516: if loc.Camp is True:  change to lower letter c -> if loc.camp is True:

Set all log levels to 1 and out.csv has inconsistencies at the start (see attachment).
out.csv

Thanks for flagging this up. I now fixed the first problem:
https://github.com/djgroen/flee/commit/fe0259703e8521aa66c7a037c40cc1c370f7647d

I also made some other redirects now, solving the other issues. @cspgdds could you test it out perhaps?
https://github.com/djgroen/flee/commit/a2667271c4241d45c231ddfc052c8a7964f1d7c6

@djgroen, I set all log levels to 1, which produced agents.out.0 and links.out.0, as well as printed 'new total spawn weight' in the terminal.

Sounds like it works then.


[backport] gas: update gas of memory.size  (#17)

Description
What is the purpose of the change?
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Brief of your solution
Please describe your solution to solve the issue or feature request.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Codecov Report

Merging #18 (f8268a0) into memory_grow (4624bc1) will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           memory_grow      #18   +/-   ##
============================================
  Coverage        44.39%   44.39%
============================================
  Files                8        8
  Lines              428      428
============================================
  Hits               190      190
  Misses             204      204
  Partials            34       34

Impacted Files
Coverage Δ

exec/interp_gas_table.go
100.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4624bc1...f8268a0. Read the comment docs.


How about the grouplinear?

Hi，i noticed that in our multi-stream implementation of group linear, we didn't use batchgemm or groupgemm. Is there any particular reason for this?

Batched gemm doesn't support different GEMM sizes.
For grouped gemm, we have two (potential) implementations:

cublasGemmGroupedBatchedEx(). The performance is not as good as the multi-stream implementation, and it doesn't support FP8 for now.
Cutlass. We evaluate the performance with the GEMM sizes from popular MoE models, including Mixtral 8x7B, 8x22B, Qwen2-57B-A14B, and DeepSeek v2, multi-stream calls to cuBLASLt shows better performance in most cases.

We will keep exploring the performance of grouped gemm, and would add them to TE if there were better implementaions.


CS2 maps sizes

As per the CS2 documentation, to create a custom map, a 14.336x14.336km, 4096x4096 resolution, grayscale, 16 bit .png or .tiff is needed, plus an optional 57.334x57.334km heightmap for the worldmap.

I can't input a 14.336 size in your tool, so i can't properly generate the required heightmaps. Could you fix this? Thank you

Set the map size to include the unplayable area for both CS1 and CS2.
That is, 17.28km for CS1 and 57.334km for CS2.
In CS2, if you set the map size to 57.334km, a playable map of 14.336km will be automatically generated.

Set the map size to include the unplayable area for both CS1 and CS2. That is, 17.28km for CS1 and 57.344km for CS2. In CS2, if you set the map size to 57.344km, a playable map of 14.336km will be automatically generated.

This isn't right, the game requires a 14.336km 4096px heightmap file. Your app only downloads a 57.355km 4096px raw file. Please just allow for a lower value

A little confused, can you show in a screenshot how to load the world map? I only see the button to load the altitude map.

@BrokVid in the CS2 editor, under Modify Terrain, there are button to import the Heightmap and Worldmap. You may need a mod to unlock the editor. I use MOOB by Cities2Modding (https://github.com/Cities2Modding/MOOB).

Official map editing is coming next week


Adding custom column didnt work

The given instructions didnt work for me. I had to update the query to say:
Text.From([Document.Location.coordinates]{1})&","&Text.From([coordinates]{0})

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: db43c478-6301-7b69-131a-c4289797c49d
Version Independent ID: 5f991bdb-919e-f712-1fe0-050eba25ebaf
Content: Power BI tutorial for Azure Cosmos DB connector
Content Source: articles/cosmos-db/powerbi-visualize.md
Service: cosmos-db
GitHub Login: @SnehaGunda
Microsoft Alias: sngun

@abcdino Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

#please-close

We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Search query slow with filter

If the following are true then a search query on a QueryableIndex can be slower than expected.

There is a filter
The search dimension is high cardinality
Many values of the search dimension match the query

This is because for QueryableIndex, the search query uses an index-based algorithm that does a bitmap intersection op for every value of the search dimension that matches the query before applying the filter. This can be more expensive than the cursor-based algorithm that applies the filter first, especially if the filter is very selective or if the search dimension is high cardinality.
Ideally the search query should use some heuristics to choose between the index-only vs. cursor-based algorithms, or at least provide a context flag to let you choose which one.
See also https://groups.google.com/d/topic/druid-user/gjhh3rfdrUw/discussion.

Sounds interesting! I'm working on it.

@gian not sure why we need some heuristics, i think if the query contains a filter we should apply the filters first.

We always use the filter bitmap, the only question is do we intersect it with the bitmaps in the indexes of the search dimension (index-based algorithm), or do we use it to scan through the rows of the search dimension (cursor-based algorithm). Which one is faster would depend on the number of rows that match the filter and on the cardinality of the search dimensions.


:robot: Run examples on pipeline to ensure they are still valid

We don't have a mechanism to remind us that a certain change will require to update the existing examples. We should be able to easily run some script that builds the images in the examples and fails the pipeline in case one of the examples is broken.

Extend with multiple tickets related to each example


feat: add more recommendations (DA-2038,DA-2040,DA-2051,DA-2035)

closes (DA-2038,DA-2040,DA-2051,DA-2035)

Breaking this PR into three individual PRs:
https://github.com/prisma/docs/pull/6531
https://github.com/prisma/docs/pull/6532/files
https://github.com/prisma/docs/pull/6533


155 frontend   impact tracking  badges

Consolidated noun project api calls to one function that takes and passes an id
Fixed current event and profile icon setting to use new api call function
Added badges component to profile page using noun project icons

Looks good!


20230721 Fixes

Codecov Report

Merging #122 (fa48c8f) into prep-release (75d99eb) will increase coverage by 0.63%.
The diff coverage is 80.76%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@               Coverage Diff                @@
##           prep-release     #122      +/-   ##
================================================
+ Coverage         70.03%   70.67%   +0.63%
================================================
  Files               111      113       +2
  Lines              1792     1889      +97
================================================
+ Hits               1255     1335      +80
- Misses              537      554      +17

Impacted Files
Coverage Δ

src/account/admin.py
100.00% <ø> (ø)

src/account/forms.py
68.18% <ø> (ø)

...nagement/commands/aaactl_fix_personal_org_slugs.py
0.00% <0.00%> (ø)

src/account/models.py
100.00% <ø> (ø)

src/account/rest/views.py
82.11% <ø> (ø)

src/account/views/controlpanel.py
86.36% <33.33%> (+0.64%)
:arrow_up:

src/common/email.py
82.60% <57.14%> (-6.87%)
:arrow_down:

src/account/views/auth.py
44.44% <80.00%> (+0.93%)
:arrow_up:

...account/management/commands/aaactl_import_users.py
93.67% <93.67%> (ø)

src/account/signals.py
68.96% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[RFC]: metrics: density: measure runc containers

This metrics test just allow measure the memory density
about Clear Containers, it fails when tries to measure
PSS memory of components that only exists during a Clear
container execution (e.g. qemu instance). This commit
allows to measure the PSS of the workload launched by a
runc container.
Fixes: #480
Signed-off-by: Mario Alfredo Carrillo Arevalo<EMAIL_ADDRESS>

Hi @grahamwhaley , this commit includes a function that will get all instances of docker-containerd-shim launched by docker-containerd, then it will calculate the PSS average about the child process launched by each docker-containerd-shim, that in this case is a sh instance, these results can be affected just if there are more containers running, however we made a clean up before that
docker-containerd                 ├── docker-containerd-shim                                           └── workload <--1                 ├── docker-containerd-shim                                           └── workload <--2                 ├── docker-containerd-shim                                           └── workload <--N

lgtm.
Thansk for the update @MarioCarrilloA I think tracking the payload via the shim is the right thing to do :-)
Hi @sboeuf This PR is being blocked by the Jenkins CI :-) - can you let me know if that is live and needs a rebuild (looks like you aborted the build for this PR, so it is effectively 'failed'), and if not, I can always over-ride the blocker for now whilst we settle Jenkins down.


Rewrite for APIv3

In this pull request, I've rewritten the entire library to be compatible with TBA API v3.
The new library is instantiated differently than the prior version. Furthermore, as specified in #4, the rewrite uses a builder pattern to construct requests before sending them to TBA's API. For example:
tba.New("tokenhere") // Go best practices advocate using New for libraries with one exposed interface
team, err := tba.Team(1418).Simple().Get()
// ...
rankings, err := tba.District("chs", 2017).Rankings().Get()
// etc.

I've ignored concerns about backward compatibility based on the facts that (a) APIv2 will be shuttered on January 1st, 2018, and (b) I am the only current user of this library and am prepared to make changes to my projects.
I've aggressively tested my implementation and I can't find any issues with it. However, since you've been using Go longer than I have, you may see problems with standards conformity or the like. Please point these out if they at all exist and I'll happily fix them.
Lastly, I'd like to humbly request that you transfer this repository to my account. I recognize that you are in college now and are probably very busy; I don't want to waste your time by submitting pull request after pull request as I continue developing this library. Please let me know if this is something you'd be comfortable with.

Also, it's probably best to squash this into one commit when you merge.

This looks really great! I am pleased to see that the builder design patter could be put to good use here.
I would be happy to transfer the repository to you.

You will need to delete your fork before I can perform the transfer: https://help.github.com/articles/about-repository-transfers/

Done. Didn't want to delete until this was merged.
Thanks so much for your help!


Add a cli option to specify backend services host if they are not running locally

e.g. Services running in Docker containers on Mac OS X need to be accessed via
Docker VM IP address instead of OS X's localhost.

On Mac OS X, dockerized services are not running as localhost. They need to be accessed via Docker VM (boot2docker or Docker machine) which usually has a different IP (e.g. <IP_ADDRESS>)
Added this cli option to specify the backend service host. So the proxy could find the correct backend service.

@flanker thanks for the PR. this looks good to me, but i have a concern: since backend_host is no longer under invokers' control to check if invoker setup has been run properly or not, i suspect this will create more problems. @gnufied @nisanth074 what do you guys think?

This makes ton of sense. @flanker I just merged https://github.com/code-mancers/invoker/pull/146/files which implements this partly.
Can you update this branch with latest master? I will be happy to look and merge.


Revert "Revert "Add an event for health check failures.""

This reverts commit fc4e711a8ca66a0e0883fad053e3a038ee3517a5.
I think the failures that @saad-ali saw were a flake, revert the revert.

Will merge on green.


[PE-D][Tester C] Incomplete Example for Add Command

Description: The example for the add command in the command summary does not include optional parameters for social media handles and schedule details, which are part of the format in the user guide.
Location: Command Summary Section
Expected: Example should include optional parameters like [cs/CAROUSELL_HANDLE], [fb/FACEBOOK_HANDLE], [ig/INSTAGRAM_HANDLE], [sn/SCHEDULE_NAME], [sd/SCHEDULE_DATE], [st/SCHEDULE_TIME], and [t/TAG]....
Actual: "e.g., add n/James Ho p/22224444<EMAIL_ADDRESS>a/123, Clementi Rd, 1234665 t/friend t/colleague"

Labels: type.DocumentationBug severity.Medium
original: LiHaixin000/ped#12

We do have an example on social media


instructions on installing index/assets plugins

tossed in (simple) instructions on picking adapters appropriate to their environment. Not sure whether installation is the right chapter for that. Since it's 0.4.x, I didn't bother too much (let's get installation instructions / user flow through discovering all plugins & commands super clear for 0.5)
not sure about the link formatting by the way... didn't work with GH pages before

thanks! in the future I think you just want to use links like "../configuration".


Fix php-fpm PID check for php8.1

Summary
make drupal-database during make starter TAG=2.0.0 hangs while waiting to find the php-fpm PID file, which is in a new location on php 8.1. This PR updates the check to work for php 8.1, and updates the buildkit and starter site tags to use php 8.1 and drupal 10.
How to test
First, https://github.com/Islandora-Devops/isle-dc/pull/346 needs to merge and this branch needs rebased.
Checkout this PR
cd /path/to/your/isle-dev/copy
gh pr checkout 347 --repo Islandora-Devops/isle-dc

Make sure php 8.1 + drupal 10 comes up
make starter
curl -v https://islandora.traefik.me/ > /dev/null

Moving to draft until https://github.com/Islandora-Devops/isle-dc/pull/346 merges this can't be tested.

I have both PRs  on my local machine. I can run make local without a problem but neither of the make starter TAG=xxx commands.

Testing this required a change to the sample.env#L52 file. This pr isn't pulling in recent changes like those to the sample.env file. If I manually modify CODEBASE_PACKAGE=islandora/islandora-starter-site:dev-main to current CODEBASE_PACKAGE=islandora/islandora-starter-site:0.8.0 the 2nd command (make starter TAG=2.0.0) works as expected. But the first doesn't and probably won't because of the composer.lock file in starter.

@DonRichards - did you have the latest commits of this branch checked out? I merged master into this branch before publishing the PR and this branch has v0.8.0 set for starter site
FWIW I only can get
make starter TAG=2.0.1 CODEBASE_PACKAGE=islandora/islandora-starter-site:1.1.0

to work properly. The 2.0.1 tag for the docker images has the drush path set correctly for drush 12. So I updated both those env vars in this PR.

@joecorall I see https://github.com/Islandora-Devops/isle-buildkit/pull/289 has been merged.
I tried making the changes in this PR then
make starter TAG=2.0.1 CODEBASE_PACKAGE=islandora/islandora-starter-site:1.1.0
The make process got a fair way but then errored:
 [success] Installation complete.
docker compose exec -T drupal with-contenv bash -lc "drush -l https://islandora.traefik.me user:role:add fedoraadmin admin"
In EntityTypeManager.php line 139:
  The "features_bundle" entity type does not exist.
make[1]: *** [starter-finalize] Error 1
make: *** [starter] Error 2

docker compose exec -T drupal with-contenv bash -lc "drush -l https://islandora.traefik.me ws"
gives
In EntityTypeManager.php line 139:
  The "features_bundle" entity type does not exist.

as does drush cr etc.

make demo TAG=2.0.1 breaks early with:
✔ Container isle-dc-fcrepo-1           Started
✔ Container isle-dc-drupal-1           Started
docker compose exec -T drupal with-contenv bash -lc 'composer install; chown -R nginx:nginx .'
s6-envdir: fatal: unable to envdir: No such file or directory
make[1]: *** [local] Error 111
make: *** [demo] Error 2

make local dies at the same point.

@kayakr did you run make down clean first? I am wondering if you had an existing codebase in your isle-dc directory? Or was this error with a fresh clone of this repo/PR

@kayakr @DonRichards OK now that https://github.com/Islandora-Devops/isle-buildkit/pull/289 has merged and a tag was made, if you pull down the changes from this PR running
git pull origin php-81-fpm-pid
make down clean
make starter

Should get you a local d10 site running with the latest starter site install.

To get solr working, we also need this PR to merge https://github.com/Islandora-Devops/isle-buildkit/pull/290

I'm able to run this with starter_dev and it works, but with a few changes. I set the tag to main instead of 2.0.2 because the pr mentioned above has been merged.
@kayakr I did run into the same issue:
docker compose exec -T drupal with-contenv bash -lc 'composer install; chown -R nginx:nginx .'
s6-envdir: fatal: unable to envdir: No such file or directory
To fix that error I ran this command
docker compose exec -T drupal with-contenv bash -lc "chown -R nginx:nginx . ; su nginx -s /bin/bash -c 'composer install'"
Which is from this pr from @DonRichards (although it was not able to run in the Makefile itself, I had to use the command line.)
Then I ran make starter-finalize ENVIRONMENT=starter_dev
This gave me a Drupal 10.1.2 site running PHP 8.1.22

This PR now uses islebuildkit:2.0.3 which should fix the solr issues.

I was able to get this but only after pulling in the code @aOelschlager mentioned

Which is from this pr from @DonRichards (although it was not able to run in the Makefile itself, I had to use the command line.)

make starter
curl -v https://islandora.traefik.me/ > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying <IP_ADDRESS>:443...
* Connected to islandora.traefik.me (<IP_ADDRESS>) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [325 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [15 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [3975 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=traefik.me
*  start date: Sep 18 13:06:49 2023 GMT
*  expire date: Dec 17 13:06:48 2023 GMT
*  subjectAltName: host "islandora.traefik.me" matched cert's "*.traefik.me"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: islandora.traefik.me]
* h2 [:path: /]
* h2 [user-agent: curl/8.1.2]
* h2 [accept: */*]
* Using Stream ID: 1 (easy handle 0x15200a800)
> GET / HTTP/2
> Host: islandora.traefik.me
> User-Agent: curl/8.1.2
> Accept: */*
>
  0     0    0     0    0     0      0      0 --:--:--  0:00:29 --:--:--     0< HTTP/2 504
< content-type: text/plain; charset=utf-8
< content-length: 15
< date: Wed, 27 Sep 2023 22:02:43 GMT
<
{ [15 bytes data]
100    15    0    15    0     0      0      0 --:--:--  0:00:30 --:--:--     4
* Connection #0 to host islandora.traefik.me left intact

:shipit:

make starter TAG=2.0.1 CODEBASE_PACKAGE=islandora/islandora-starter-site:1.1.0 works as well!

Thanks @joecorall


Graphs for evals

create an /evals directory and store your evals there then run the cells in the jupyter to graph your data

Merging for the sake of having this in so other folks can start iterating on it.


[T2A2][T04-A2]Song Zhiwen

@zzzzwen your pull request is not in the correct naming format. please rename it using the correct tutorial id and team id. going forward, it it is not in the correct format, it may not be reviewed!

in general good work on this. however, there is scope for improvement,

in general, pay more attention to coding standards violation.

some useful ones include checking how conditional statements are written, naming conventions etc.,

try to make commit messages meaningful
e.g., "Add switch" you can include where (or which file/method) may be why as well!


Added: Enhance MatchingPairs game with tutorial overlay

Introduce a tutorial overlay for the MatchingPairs game, allowing users to receive guidance.
Refactor MatchingPairs to avoid prop mutation.
Update the Button component to support multiple clicks when specified (so the tutorial overlay can be closed multiple times).
Integrate MSW for API mocking in Storybook.
Add relevant tests for the new features.

Resolves #1338

This works and looks really wel. Two things come to mind:
* We should turn off the tutorial in `matching_pairs_lite`

I turned it off for all matching pairs rules files, and created a new MatchingPairs2025 (which we from now on can regard as the working version of matching pairs 2.0) rules file that does have tutorial overlay text messages.

* We now log a timestamp for every action (click) in the game, so maybe we should also log if a turn showed the tuturial popup, because there's going to be some extra reading time.

I looked into this but this would be kind of hard to manage from the frontend (assuming we'd send this information from the intermediate_score endpoint) since we use the response of that api endpoint to (not) show a tutorial message in an overlay. 🤔


[TFLite] add support for float16

Add support for float16
TFLite converts float32 models to float16 models by introducing a Dequantize op in every op that contains a float32 values (weights, biases, and constants etc. ).
So conv2d op should account for the weights and bias as an input tensor instead of a weight value attached to itself.

It would be nice to supply testing.

I've added a test.

Is there something else I should fix or change?

@euntaik could your pr solve this issue ? https://github.com/apache/tvm/issues/5823 If it could, we could associate this pr with that issue and close.
@giuseros could you have another round of look? Thanks!

@euntaik Could you help to check your pr could solve issue #5823? IMO, your pr is enough.

Thanks @euntaik @giuseros @FrozenGene !


Document template variable quoting in Chronograf

From this issue: https://github.com/influxdata/chronograf/issues/4912
URL for relevant page?
https://docs.influxdata.com/chronograf/v1.7/guides/dashboard-template-variables/
We need to document when a user needs to quote template variables and when they do not. This has come up in our community slack: https://influxcommunity.slack.com/archives/CHACR8WJ2/p1581008702024400?thread_ts=1581001415.023700&cid=CHACR8WJ2
When you use pre-defined meta queries in Chronograf, you do not need to single quote the values when you use them in the queries. When you use any other option with strings, you need to single quote the template variable when used in a query.
Attached is a Dashboard that illustrates the behavior.
Template Variables Test.json.zip

@russorat, to be sure, it sounds like community saying the opposite?:
To query a variable that was created w custom meta query, use single quotes.
Otherwise, don’t use quotes. (Perhaps you're making a distinction between pre-defined meta query and custom meta query?)
Please confirm I'm not misreading-thank you!

@kelseiv you still need to use quotes if you use string values in CSV or Map. Try importing the attached dashboard into chronograf for a concrete example.


Visualization of rarefaction curves by a grouping variable causes system crash

🐛 Bug
The generation of rarefaction curves using the repDiversity function with the option .method = 'raref' seems to work fine with a few datasets that I have tried. However, when I try to visualize the results by a grouping variable using the .by = function, I have encountered a problem.
To Reproduce
Using the dataset provided with immunarch ...
1. Generate the rarefaction curve data.
divRare <- repDiversity(.data = immdata$data, .method = 'raref')

This step takes 1.35 sec on my computer. No problem.
2. Generate a plot of the rarefaction curves
divRare %>%
  vis()

This step takes 0.67 sec on my computer and a nice figure is generated with 12 rarefaction curves overlaid. Again, no problem.

3. Generate a plot of the rarefaction curves by a grouping variable
In the example immdata dataset in the package, there is a grouping variable called Status in the metadata. Here I try to make a plot by this grouping variable.
divRare_imm %>%
  vis(.by = c('Status'), .meta = immdata$meta)

This step takes 0.60 min on my computer and a nice grouped figure is generated.

What happens with my dataset
Here I reproduce the steps above using my dataset of mouse TCR sequences.
Generate rarefaction curve data
divRare <- repDiversity(.data = tcr_imm$data, .method = 'raref')

This takes 30.5 sec.
Plot rarefaction curves
divRare %>%
  vis()

This takes 10.8 sec and generates this busy plot that I intend on fixing by grouping, as in the example above.

Plot rarefaction curves by a grouping variable
In my dataset, there is a metadata variable called Arm that is a binary grouping variable.
divRare %>%
  vis(.by = c('Arm'), .meta = tcr_imm$meta)

This step crashes my computer every time I try it. The computer is computing for at least 10 min and becomes unresponsive. I cannot use Ctrl+Alt+Del to get Task Manager open to close RStudio.
Expected behavior
I would like the grouped rarefaction curves to generate or, at least, understand why they are not.
Really appreciate this awesome package and any help provided.
Additional context
Computer: AMD 3950X (16 core), 64 Gb RAM
RStudio 2022.07.0 Build 548 "Spotted Wakerobin" Release (34ea3031, 2022-07-06)

sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
other attached packages:
[1] forcats_0.5.2     stringr_1.4.1     purrr_0.3.4       readr_2.1.2
[5] tidyr_1.2.0       tibble_3.1.8      tidyverse_1.3.2   immunarch_0.7.0
[9] patchwork_1.1.2   data.table_1.14.2 dtplyr_1.2.2      dplyr_1.0.10
[13] ggplot2_3.3.6
loaded via a namespace (and not attached):
[1] googledrive_2.0.0   colorspace_2.0-3    ggsignif_0.6.3
[4] ellipsis_0.3.2      class_7.3-20        modeltools_0.2-23
[7] mclust_5.4.10       circlize_0.4.15     fs_1.5.2
[10] GlobalOptions_0.1.2 rstudioapi_0.14     ggpubr_0.4.0
[13] farver_2.1.1        graphlayouts_0.8.1  ggrepel_0.9.1
[16] flexmix_2.3-18      lubridate_1.8.0     fansi_1.0.3
[19] xml2_1.3.3          codetools_0.2-18    doParallel_1.0.17
[22] robustbase_0.95-0   knitr_1.40          shinythemes_1.2.0
[25] polyclip_1.10-0     rlist_<IP_ADDRESS>       jsonlite_1.8.0
[28] broom_1.0.1         cluster_2.1.4       kernlab_0.9-31
[31] dbplyr_2.2.1        pheatmap_1.0.12     ggforce_0.3.4
[34] shiny_1.7.2         httr_1.4.4          compiler_4.2.1
[37] backports_1.4.1     assertthat_0.2.1    Matrix_1.4-1
[40] fastmap_1.1.0       gargle_1.2.0        cli_3.3.0
[43] later_1.3.0         tweenr_2.0.1        htmltools_0.5.3
[46] tools_4.2.1         igraph_1.3.4        gtable_0.3.1
[49] glue_1.6.2          reshape2_1.4.4      fastmatch_1.1-3
[52] Rcpp_1.0.9          carData_3.0-5       cellranger_1.1.0
[55] vctrs_0.4.1         ape_5.6-2           nlme_3.1-157
[58] iterators_1.0.14    fpc_2.2-9           ggseqlogo_0.1
[61] ggraph_2.0.6        ggalluvial_0.12.3   xfun_0.32
[64] rvest_1.0.3         mime_0.12           lifecycle_1.0.1
[67] phangorn_2.9.0      googlesheets4_1.0.1 rstatix_0.7.0
[70] stringdist_0.9.8    factoextra_1.0.7    DEoptimR_1.0-11
[73] MASS_7.3-57         scales_1.2.1        tidygraph_1.2.2
[76] hms_1.1.2           promises_<IP_ADDRESS>    parallel_4.2.1
[79] RColorBrewer_1.1-3  yaml_2.3.5          gridExtra_2.3
[82] UpSetR_1.4.0        stringi_1.7.8       foreach_1.5.2
[85] shape_1.4.6         rlang_1.0.5         pkgconfig_2.0.3
[88] prabclus_2.3-2      evaluate_0.16       lattice_0.20-45
[91] labeling_0.4.2      tidyselect_1.1.2    plyr_1.8.7
[94] magrittr_2.0.3      R6_2.5.1            generics_0.1.3
[97] DBI_1.1.3           haven_2.5.1         pillar_1.8.1
[100] withr_2.5.0         abind_1.4-5         nnet_7.3-17
[103] crayon_1.5.1        modelr_0.1.9        car_3.1-0
[106] uuid_1.1-0          utf8_1.2.2          tzdb_0.3.0
[109] rmarkdown_2.16      viridis_0.6.2       grid_4.2.1
[112] readxl_1.4.1        reprex_2.0.2        digest_0.6.29
[115] diptest_0.76-0      xtable_1.8-4        httpuv_1.6.5
[118] stats4_4.2.1        munsell_0.5.0       viridisLite_0.4.1
[121] quadprog_1.5-8

Update:
I have been able to get a grouped plot of my dataset if I select a subset of the data further. When I examined the code for the vis.immunr_rarefaction that is responsible for the plotting of rarefaction curves, I notice something that I am not sure is correct.
This is the part of the function used for producing the grouped plot (slightly modified to facilitate readability here).
if (.mean && is_grouped) {
  data_proc <- .muc.res %>%
    group_by(Group, Size) %>%
    summarise(MinVal = quantile(Mean, 0.025, na.rm = TRUE),
              MaxVal = quantile(Mean, 0.975, na.rm = TRUE),
              MeanVal = mean(Mean, na.rm = TRUE))
  p <- p +
    labs(title = paste("Rarefaction analysis: by ", group_column)) +
    geom_line(aes(x = Size, y = MeanVal, color = Group), data = data_proc)

  if (.errors) {
    p <- p +
      geom_ribbon(aes(x = Size, ymin = MinVal, ymax = MaxVal, group = Group, fill = Group),
                  data = data_proc, alpha = 0.15)
  }
}

Note that .muc.res is the output of the process_metadata_arguments function and is a dataframe with 7 columns: Size, Q0.025, Mean, Q0.975, Sample, Type, and Group. This dataframe
contains the rarefaction curves output from repDiversity(.method = 'raref')  (Size = Sample size = X axis , Mean = Estimated diversity = Y axis), a 95% confidence interval for the diversity estimate (Q0.025, Q0.975), the name of the sample (Sample), the interpolation method (Type), and the grouping variable (Group). In creating the data_proc dataframe, the code above finds the average value of the estimated sample diversity (Mean) for each sample size (Size) found in each Group.
For most TCR sequencing datasets, I suspect that each Sample will have different values for Size. This means that MinVal = MaxVal = MeanVal for most values of Size. For instance, in a subsample of my dataset that consists of TCR sequencing of tumors from 7 mice (5 belonging to group 1 and 2 belonging to group 2), there were 33,362 rows in .muc.res which resulted in 33,360 rows in data_proc, indicating that the values of Size are almost all unique across samples.
> dim(data_proc)
[1] 33360     5
> dim(.muc.res)
[1] 33362     7

This results in a problem when one tries to plot the average rarefaction curve using the code above.  Take for example this extract/subset from  data_proc tibble which will demonstrate the issue nicely. Note two things:

MinVal = MaxVal = MeanVal
While Size is monotonically increasing, the value of MeanVal is not (i.e. a smooth curve is therefore not possible)

> data_proc[23459:23465, ]
# A tibble: 7 × 5
# Groups:   Group [1]
Group        Size   MinVal   MaxVal  MeanVal
<fct>       <dbl>    <dbl>    <dbl>    <dbl>
1 symphony 21.57726 1.879414 1.879414 1.879414
2 symphony 21.57976 1.540023 1.540023 1.540023
3 symphony 21.58091 1.740328 1.740328 1.740328
4 symphony 21.58372 1.879414 1.879414 1.879414
5 symphony 21.58693 1.540023 1.540023 1.540023
6 symphony 21.58737 1.947766 1.947766 1.947766
7 symphony 21.59018 1.879414 1.879414 1.879414

The issue is that when the code then calls ggplot and the geom geom_line, this results is a see-saw rarefaction curve which is not the average rarefaction curve for the group but a curve that essentially connects the lowest estimated values for diversity within the samples of the group to the highest estimated values for diversity within the samples of the group. This is shown for my samples here:
repDiversity(.data = tcr_fr$data, .method = 'raref')  %>%
  vis(.by = c('Arm'), .meta = tcr_fr$meta)

You can see how the line is essentially zig-zagging from ~ 1.5 to ~ 2.2 for sample size of about 20, which is seen from the data contained in the data_proc variable shown above.
Recommendation
I think that a different method of grouping rarefaction curves is required as the method contained in the function is probably not valid. It would seem to me that at least a couple of options exist:

Group the samples prior to calling repDiversity and generate a single rarefaction curve for the grouped samples, along with its 95%CI.
Fit a non-linear model to the individual rarefaction curves and estimate their group level mean and 95%CI.

I am not sure which is more valid but I hope this example shows a potential issue with repDiversity and grouped rarefaction curves in immunarch.

Hi @canuckafar !
Tnanks a lot for your detailed feedback on our packages.We will definitely take into account your recommendations to improve future versions of Immunarch.
Feel free to write any questions or recommendations further along:)
Enjoy your week,
Maria Samokhina


re-gen reports at 5 july

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 0.849%

Totals

Change from base Build 990260075:
0.0%

Covered Lines:
18

Relevant Lines:
2119

💛  - Coveralls


Https bad connection issue

Describe the Bug with repro steps

Go to Logic Apps Testa_Conexao_App12
Click on resubmit last process
Wait to finalize and show the error message

What type of Logic App Is this happening in?
Consumption (Portal)
Which operating system are you using?
Windows
Are you using new designer or old designer
New Designer
Did you refer to the TSG before filing this issue? https://aka.ms/lauxtsg
Yes
Workflow JSON
No response
Screenshots or Videos
No response
Browser

Microsoft Edge

Additional context
It was working normally until 18/10/2024 at noon.
After that, it's not working anymore.

Can you provide any more information or images of what you're seeing/doing? How is your workflow set up? We aren't able to do much with the information provided here.


Doubly compensating likelihoods in biallelic model

I thought I had fixed this in #134, but instead I fixed it in #127.

Er, nevermind. This was indeed fixed in #134.


feat: html button component

Implementation of the HTML Button component
Implementation of the Button component which contains 3 variants and each of the variant has 3 appearances. There is also the option the render a icon button by omitting the label prop. There is also the ability to add a disabled state whenever necessary
Props:
label
variant
appearance
size
disabled
icon
Screenshots

Looks great! Just missing the documentation update.

Hi @hamza14khan I was planning the add the docs site in the React PR so that we have both libs when I add it


add List collection quick start

This is a first pass at #3279
I only covered the List (instead of a survey of all collections) to keep the quick start reasonably short.
I'm going to create a second version designed to be run locally for A/B testing, and add to this PR.
/cc @rowanmiller @LadyNaggaga
Internal link

I really like the concepts you picked to keep building with - it definitely feels like it flows from the dot.net intro 😄. The IndexOf section might get a little complex... but let's see how it does in the user study, it will be good to have it there to see how folks go with it.
Good to go from my perspective.


Add the "display type" option for 1_n_2 parts back in.

See numbas/Numbas#320. I think the runtime code is in, so we just need to add the option back into the editor.

To do this:

in editor/static/js/question/edit.js, add an observable called displayType to the 1_n_2 part type's model, with default value 'radiogroup'. Also add a 'displayTypes' property with two values: 'radiogroup' and 'dropdownlist' (copy the one from m_n_x)
In [editor/templates/question/part_types/1_n_2.html], (https://github.com/numbas/editor/blob/master/editor/templates/question/part_types/1_n_2.html), add a select element for the displayType property (copy the one from the m_n_x part)


WIP: 1.30 docs for KEP-4222

KEP-4222 is currently targeting 1.30 for alpha.
/sig api-machinery

Hello @benluddy 👋 please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before Tuesday March 12th 2024 18:00 PST. Thank you!

Thanks, @drewhagen. I'm retargeting this KEP to 1.31 since not all of its alpha criteria have been satisfied in time for code freeze.

Heads up that dev-1.31 branch is created now, if you choose to reopen this for the next release.


Add usage, known limitations and performance sections

Ref #3

Hum, ATOMIC_REBUILD will be easy to implement with PostgreSQL :)


Segmentation fault when opened file has been moved

Issue Report Checklist

[ ] Searched the issues page for similar reports
[ ] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[ ] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[ ] Could not reproduce inside jupyter qtconsole (if console-related)
[ ] Tried basic troubleshooting (if a bug/error)

[ ] Restarted Spyder
[ ] Reset preferences with spyder --reset
[ ] Reinstalled the latest version of Anaconda
[ ] Tried the other applicable steps from the Troubleshooting Guide

[ ] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
Segmentation fault when opened file has been moved
What steps reproduce the problem?

Open 2 or more files
Move all it somewhere via desktop filemamaner
Spyder ask to close first inaccessible file. Say "Yes"
Spyder ask to close second inaccessible file. Say "Yes"
Segmentation fault

What is the expected output? What do you see instead?
Paste Traceback/Error Below (if applicable)

(5) kai@asus-ux360c:~/Projects/spyder/5$ spyder
No QCoreApplication instance found. Application patches not applied. You have to call load_stylesheet function after instantiation of QApplication to take effect.
Segmentation fault

Versions

Spyder version: latest installed via pip install -UI --pre spyder. But this behavior i know ~1 year
Python version: Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux
Qt version: 5.11.3+dfsg-3
PyQt version: PyQt5-5.12.3-5.12.10-cp35.cp36.cp37.cp38.cp39-abi3-manylinux1_x86_64.whl
Operating System name/version: Debian GNU/Linux 10 (buster)

Dependencies

latest

I could reproduce this with the steps described by @kai3341. We'll try to fix it in a future release.

This is basically the same as issue #18838, so I'm going to close it in favor of that one.


use nvidia-docker to make more portable?

Would it be possible to use nvidia-docker to make this more portable?

Does Nvidia docker work on intel or AMD graphics ?


UtilityFuncCRRA

This PR creates an object oriented CRRA utility function to better organize namespaces and add formal structure to models.

[x] Tests for new functionality/models or Tests to reproduce the bug-fix in code.
[x] Updated documentation of features that add new functionality.
[x] Update CHANGELOG.md with major/minor changes.

Codecov Report

Merging #1168 (7eb3e64) into master (69c5f9c) will decrease coverage by 0.01%.
The diff coverage is 68.29%.

@@            Coverage Diff             @@
##           master    #1168      +/-   ##
==========================================
- Coverage   73.58%   73.56%   -0.02%
==========================================
  Files          72       72
  Lines       11465    11485      +20
==========================================
+ Hits         8436     8449      +13
- Misses       3029     3036       +7

Impacted Files
Coverage Δ

HARK/ConsumptionSaving/ConsRiskyAssetModel.py
40.59% <28.57%> (ø)

HARK/ConsumptionSaving/ConsMarkovModel.py
85.83% <37.50%> (ø)

HARK/ConsumptionSaving/ConsPrefShockModel.py
79.68% <50.00%> (ø)

HARK/ConsumptionSaving/ConsMedModel.py
73.93% <66.66%> (-0.12%)
:arrow_down:

HARK/utilities.py
36.05% <76.47%> (+4.45%)
:arrow_up:

HARK/ConsumptionSaving/ConsGenIncProcessModel.py
82.45% <85.71%> (ø)

HARK/ConsumptionSaving/ConsIndShockModel.py
85.19% <100.00%> (-0.16%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

This looks like a good step forward to me!

To do from @llorracc: look for standardized function representation (with derivatives and inverses) in other python libraries

from scipy: https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.PPoly.html#scipy.interpolate.PPoly
from sympy: https://docs.sympy.org/latest/guides/custom-functions.html#differentiation
There does not seem to be a standard for mathematical functions.

To do: Think of design for CRRA(CD) and/or CRRA(CES).

Looks like there's significant overlap between this PR and #1177
This looks like a simpler feature, and unless there's some good reason I'm not thinking of, it would be best to finish this one in its own PR.

Fix merge conflict please.

My bad ...

@llorracc : advocates aligning this syntax with dolo's

Would like to have all econ utility functions in HARK.utilities, but I think it's useful for econ UtilityFunction objects to inherit from MetricObject so they can be compared, ie what is the distance between utility functions? However, when I add from HARK.core import MetricObject into HARK.utilities I get a circular reference error because core.py inherits from utilities.py.
I would like to propose that the file utilities.py contains only classes/methods that pertain to econ utility functions, and everything else that is more of a programming utility to go into a file called tools.py or something like that. This would avoid the circular reference.

TODO : labeled goods

@alanlujan91 See merge conflicts and test failures.
Punt the broader scope issues to #1035.
Lots of good stuff in this PR as is.

now addresses #1188

Changes in this PR:

Move econ utilities to rewards.py.
Rename parameter gam to rho in CRRAutility and derivatives.
Fix ZeroDivisionError in all utility functions.
Create cobb_douglas, cobb_douglas_p, cobb_douglas_pp, and cobb_douglas_pn functions.
Create const_elast_subs and const_elast_subs_p functions.
Implement abstract UtilityFunction class.
Implement UtilityFuncCRRA class with derivative and inverse.
Implement UtilityFuncCobbDouglas with derivative.
Implement UtilityFuncCobbDouglasCRRA.
Implement UtilityFuncConstElastSubs.
Replace instances of CRRAutility with UtilityFuncCRRA throughout codebase.

I'll review this and merge if I don't see any issues.

Great work! Merging.


Scheduler confirmation modal

Link to issue #796
I am not able to submit the formik form using the modal I created and need help there.

@theborakompanioni If you find time then take a look at this PR I also want to ask if I should create the relevant translation.json files for my files.
Thanks!

@theborakompanioni If you find time then take a look at this PR I also want to ask if I should create the relevant translation.json files for my files.
Thanks!

Can you please base your changes on branch devel? It seems to include changes from #791.

Yes please


[1.x] Make inertia response more extension friendly

This PR simply aims to make Inertia\Response more friendly to extension/customization by:

Further breaking down the toResponse method
Allowing the class to be resolved from the container

This unlocks, for example, customizing how the page.url get's resolved by Inertia.
In a particular case, it was necessary for me to ensure redirections keept the url fragment (/abc#123).
Previously I had to:

create a custom middleware
render the Inertia response
get it's orginal content
modify it
finally reset the page with the modified content
return the response.

Example implementation
<?php

declare(strict_types=1);

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Illuminate\Http\Response;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\RedirectResponse;

final class EnsureCorrectInertiaRedirectionWhenPossible
{
    /**
     * Handle an incoming request.
     * Ensure URL fragments on redirections are set whenever possible.
     *
     * @param  Closure(\Illuminate\Http\Request): (\Illuminate\Http\Response|\Illuminate\Http\RedirectResponse|\Illuminate\Http\JsonResponse)  $next
     */
    public function handle(Request $request, Closure $next): Response|RedirectResponse|JsonResponse
    {
        if (!$request->inertia()) {
            return $next($request);
        }

        $requestedFrom = $request->headers->get('x-requested-from');

        if (!$requestedFrom) {
            return $next($request);
        }

        $response = $next($request);

        if (!($response instanceof JsonResponse)) {
            return $response;
        }

        $parsedRequestedFrom = parse_url($requestedFrom);

        if (!is_array($parsedRequestedFrom)) {
            return $response;
        }

        if (!isset($parsedRequestedFrom['path'])) {
            return $response;
        }

        if (!isset($parsedRequestedFrom['fragment'])) {
            return $response;
        }

        $requestedFromUri = $parsedRequestedFrom['path'] . (
            isset($parsedRequestedFrom['query']) ? "?{$parsedRequestedFrom['query']}" : ''
        );

        $page = $response->getOriginalContent();

        if ($requestedFromUri !== $page['url']) {
            return $response;
        }

        $page['url'] = "{$page['url']}#{$parsedRequestedFrom['fragment']}";

        $response->setData($page);

        return $response;
    }
}

Now, a custom Response class overwritting the resolveUrl method could be resolved from the container whenever an Inertia response is returned.

Example implementation
use Illuminate\Http\Request;
use Illuminate\Foundation\Application;
use Illuminate\Support\ServiceProvider;
use Inertia\Response as InertiaResponse;

final class AppServiceProvider extends ServiceProvider
{
    /**
     * Bootstrap any application services.
     */
    public function boot(): void
    {
        $this->app->bind(InertiaResponse::class, CustomInertiaResponse::class);
    }
}

final class CustomInertiaResponse extends InertiaResponse
{
    public function resolveUrl(Request $request, array $props): string
    {
        $url = '...';

        return $url;
    }
}

Resolved any conflicts that arose after https://github.com/inertiajs/inertia-laravel/pull/641. This PR is now up to date with the 1.x branch.


The "id" argument must be of type string. Received undefined

I'm getting this error during the dev and build process.
ERROR #11321  PLUGIN

"gatsby-source-shopify-translations" threw an error while running the onCreatePage lifecycle:

The "id" argument must be of type string. Received undefined

TypeError: The "id" argument must be of type string. Received undefined

I've been ignoring it so far because it didn't seem to cause any problems but now I've tried to build the page and the process fails because of this error.
I'm wondering where this error comes from and If anybody else noticed it too.
I'm using v0.4.0 but the error has been happening on earlier versions too.

Can you tell me both gatsby and gatsby-source-shopify version? Do you upgrade to latest version? Because there have been some schema changes and I haven't had a chance to see exactly what is changed yet.
Release Note - https://www.gatsbyjs.com/docs/reference/release-notes/v4.11/
All id fields that come back from the Shopify API have now been mapped to shopifyId so that id is always intrinsic to Gatsby.

I'm using gatsby 4.10.3 and gatsby-source-shopify 6.10.2

This is what my gatsby-node.js file looks like:
exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions

  const productsQuery = await graphql(`
    {
      products: allShopifyProduct {
        edges {
          node {
            id
            title
            storefrontId
            handle
          }
        }
      }
      translatedProducts: allShopifyTranslatedProduct {
        edges {
          node {
            title
            locale
            storefrontId
            handle
          }
        }
      }
    }
  `)

  const products = productsQuery.data.products.edges
  const translatedProducts = productsQuery.data.translatedProducts.edges

  products.forEach(product => {
    const translatedProduct = translatedProducts.filter(
      tProduct => tProduct.node.storefrontId === product.node.storefrontId
    )

    translatedProduct.forEach(tProduct => {
      const newProduct = {
        ...product,
        node: {
          ...product.node,
          handle: slugify(tProduct.node.title),
        },
      }
      const { locale } = tProduct.node

      createPage({
        path:
          locale === 'de'
            ? `/shop/${slugify(tProduct.node.title)}`
            : `/${locale}/shop/${slugify(tProduct.node.title)}`,
        component: path.resolve('./src/templates/ProductPage/index.js'),
        context: {
          id: newProduct.node.id,
          locale: locale,
          title: newProduct.node.title,
          storefrontId: newProduct.node.storefrontId,
          originalPath: product.node.handle,
          handle: product.node.handle,
          isAlreadyTranslated: true,
        },
      })
    })
  })
}

the stack trace says the error happens in the onCreatePage.js at 31:23. I'm wondering since in there is nothing called id in the whole file.

I've figured out the error happened because I wasn't using the configPath key in the plugin config because I didn't need it. (see #8 )
So configLocales in onCreatePage.js was undefined.


Build TM config for #max_num peers in distinct regions

In GCP we pay different bandwidth costs when communicating with nodes in different regions: https://cloud.google.com/vpc/network-pricing#:~:text=a%20legacy%20network.-,Inter%2Dregion%20data%20transfer,-VM%2Dto%2DVME.g. NA<>NA is .02 cents per GIB, but NA<>EU is .05 cents, and other regions get to 14 cents.These costs are crazy high (even same region).We can make a config option for max_num_peers_in_distinct_regions, which acts as a filter on incoming/outbound connections, so we can control how muhc is same region. We can enforce some in-code minimum like 5 here (so we get cross-geo connectivity), but otherwise have most of our traffic same-region.

This got auto-created from an automated internal issue tracker here. Was always intended to just be in comet upstream


mcu/nordic: Require LF clock source to be specified

This is typically specified by BSP and omitting to do so is an error.

(Continued from Slack) My mistake was to explicitly use LFRC though and $notnull would not catch that. I was wondering whether a restriction like MCU_LFCLK_SOURCE == 'LFRC' && !BLE_CONTROLLER would work, since BLE clearly does not work with the low-accuracy LFRC.

(Continued from Slack) My mistake was to explicitly use LFRC though and $notnull would not catch that. I was wondering whether a restriction like MCU_LFCLK_SOURCE == 'LFRC' && !BLE_CONTROLLER would work, since BLE clearly does not work with the low-accuracy LFRC. (edit) It should be noted that the LFRC can be calibrated, which makes BLE reliably work for me.

yeap, I updated PR with this restriction


Add a number of collections

Hi, sweet app. I added the collections in my history that were missing.

this is awesome, thank you for the merge!


Implement content type propagation

Implements propagating content types to all of the cloud providers. Does so by adding a kwarg to all clients content_type_method that by default is set to the built-in mimetypes.guess_type.
Includes tests and documentation.
Changes looks bigger than it is because all of our Local testing resources and Mocks had to be updated to track content types that were set so that we could test them.
Closes #222

Codecov Report

Merging #226 (da23a4f) into master (403ce19) will decrease coverage by 0.4%.
The diff coverage is 92.6%.

@@           Coverage Diff            @@
##           master    #226     +/-   ##
========================================
- Coverage    94.7%   94.3%   -0.5%
========================================
  Files          21      21
  Lines        1248    1281     +33
========================================
+ Hits         1183    1208     +25
- Misses         65      73      +8

Impacted Files
Coverage Δ

cloudpathlib/local/localclient.py
96.4% <87.5%> (-1.0%)
:arrow_down:

cloudpathlib/s3/s3client.py
92.8% <90.9%> (-3.3%)
:arrow_down:

cloudpathlib/azure/azblobclient.py
94.2% <92.8%> (-0.4%)
:arrow_down:

cloudpathlib/gs/gsclient.py
92.7% <100.0%> (-1.6%)
:arrow_down:

Thanks @jayqi for review! Integrated all your comments.


Any suggestions on interacting with API with a service account?

In my scenario, I wish to have some sort of a service principal that will perform calls to API without a need for user involvement. Could you point me in the right direction on how to implement non-interactive interaction and what authentication setup is needed for this?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ac47cbcf-45ed-81aa-b532-e7ad0046658e
Version Independent ID: 7aa1f60e-069f-3d62-5cac-b65e4b1116ef
Content: Authorization using OAuth 2.0 - Azure DevOps
Content Source: docs/integrate/get-started/authentication/oauth.md
Product: devops
Technology: devops-ecosystem
GitHub Login: @chcomley
Microsoft Alias: chcomley

Thanks for your question. It looks like you're working through an issue with your scenario or implementation, rather than an issue with the documentation. Here are a couple of options where you might consider asking your question:

Azure DevOps on Stack Overflow
Azure DevOps Support Bot

I am closing this issue now.  You are welcome to @ mention me for any followup.
We hope to hear from you again.

@borys86 any luck with that? I am also looking into this.

Regarding the question from @borys86 this is definitely an issue with the documentation. There needs to be an example of service to service calls using OAuth2 in the documentation. In many use cases the app will want to be be able to make a REST call to say, for example, create a release, but we don't want to give the user access to that, just the app.

Yes a Client Credentials example is needed

@WilliamAntonRohm, looks like there are a number of us, me included, looking for documentation on calling these APIs from a service account. Rather than redirecting us elsewhere, do you have any comment on additional samples to do what we're looking to do?

@borys86, @mboutet, @paulnuk, @TerenceSansom, @slbentley -- thank you for clarifying.
@chcomley -- please look into this issue.

From what I've read (and I've done a lot of reading on this) there's actually no support for calling these APIs from a service principal, which is somewhat surprising to say the least. As it stands you have to get the user to authorise your app to use the selected scopes, and get a refresh token via the code that gets given after the user grants the authorisation. Then you have to store each user's refresh token somewhere secure and use that to get an access token, all of which is rather involved compared to using a service principal.
As far as the documentation goes I think it needs to explicitly clarify whether or not you can use a service principal, and if you can't then it would be handy to find out where we can put in a request to allow this.

@paulnuk -- thanks for your feedback.  You are welcome to put in your request here.

A while ago it was possible to call API's with basic auth, and a PAT token. Is this still possible, since its unrealistic to have the user sign in for a automated system

Hi all, (@borys86, @slbentley, @mboutet, @TerenceSansom, @paulnuk)
There are two types of apps that can integrate with Azure DevOps using out REST APIs.

An app that talks to Azure DevOps on behalf of their users using Oauth - https://docs.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/oauth?view=azure-devops
An app running on an identity talking to Azure DevOps. The recommended approach here is to use a PAT. The challenge today is that you can't create a PAT programmatically and this is something we will be fixing within the next few months.

Azure DevOps doesn't support using service principals to access our APIs. I'll be clarifying this within the article, which should go live sometime today. To submit this as a feature request, please go here, and then select Suggest a feature > Azure DevOps. Thank you!


[Core] Complete And Test branch function

Purpose

In issue#3757, Paimon has Support using table branches with $branch_ and support different schemas for different branches. Therefore, this PR complete the branch function and test it for spark.

Tests
IT Cases.

API and Format
No changes.

Documentation
Yes.

+1


int8 tflite model allocate_tensors() silently stop python process.

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
binary
TensorFlow version
2.8.1, 2.13.0, '2.14.0-dev20230706'
Custom code
Yes
OS platform and distribution
Windows 11, Windows 10 WSL with Ubuntu 20.04
Mobile device
No response
Python version
3.8
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
I have converted a Resnet18 model from onnx to tflite. (onnx > tf > tflite)
onnx to tf conversion is done by this repo
tflite is converted to int8 precision using post-training integer quantization link
Netron can display the converted int8 model correctly.
onnx model & tflite model link
tflite int8 model link
but when I try to do inference. calling the method allocate_tensors() stop the python process without showing any error/warning.
if the tflite model is converted with fp32, this issue  doesn't happen.
I have no idea how do to fix this issue or is there any workaround?
thanks
Standalone code to reproduce the issue
## tf to tflite conversion
import tensorflow as tf
import numpy as np

saved_model_dir = 'resnet18'
tflite_model_path = saved_model_dir + '.tflite'

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

def representative_dataset_gen():
    for _ in range(100):
        data = np.random.rand(1, 3, 224, 224)
        yield [data.astype(np.float32)]

converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()

# Save the model
with open(tflite_model_path, 'wb') as f:
    f.write(tflite_model)

## inference time
import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path="resnet18.tflite")
print('before')
interpreter.allocate_tensors()
print('after') # this line not displayed

Relevant log output
# inference time output (tf 2.14.0-dev20230706)
WARNING:tensorflow:From C:\Users\AI\miniconda3\envs\tf\lib\site-packages\tensorflow\python\ops\distributions\distribution.py:259: ReparameterizationType.__init__ (from tensorflow.python.ops.distributions.distribution) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use `tfp.distributions` instead of `tf.distributions`.
WARNING:tensorflow:From C:\Users\AI\miniconda3\envs\tf\lib\site-packages\tensorflow\python\ops\distributions\bernoulli.py:165: RegisterKL.__init__ (from tensorflow.python.ops.distributions.kullback_leibler) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use `tfp.distributions` instead of `tf.distributions`.
before
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

I found a workaround for this issue.
instead of converting from tf to tflite, convert from keras to tflite works for me. (onnx > keras > tflite)
onnx to keras conversion is done by onnx2keras package (pip install)
keras to tflite using the same code above (only replace with tf.lite.TFLiteConverter.from_keras_model)

Hi @bluesy7585
Thanks for the workaround.
I see the onnx-TF library has not been updated recently and the conversion to TF might be causing the issue.
Feel free to close the issue since it is resolved.
Thanks.


License ?

Hello, under what license is this project released under ? I would like to study it to learn from it. Thank you.

Feel free to work on it or redistribute. There is no licence for this code.
Just mention the main repo and the github link.


[ffi] Data corruption from native thread

Hi,
I'm having an issue where a NativeCallable.listener callback is receiving garbled data.
My callback is void Function(Pointer<Char> json) => json.cast<Utf8>().toDartString();
This fails ~80% of the time as it fails to decode the UTF-8 in the call to toDartString(). This string should always be valid UTF-8, and indeed in the remaining 20% of the time it succeeds and behaves as expected.
I believe it is similar to the issue described here.
Is there a workaround for this? I think the proposed NativeCallable.blocking might solve it but in the meantime is there anything I can do from the dart side?
Dart SDK version: 3.4.4
Thanks

I believe it is similar to the issue described here.

It does sound like the same issue, the String memory being released on the native code side before the Dart callback is run (asynchronously).

Is there a workaround for this?

Yes, malloc the String in C and pass ownership to Dart.

I think the proposed NativeCallable.blocking might solve it but in the meantime is there anything I can do from the dart side?

It would (and would make the callback synchronous instead of asynchronous).
No, you cannot do anything from the Dart side.
You'll have to write a C function that copies the String into malloced memory and pass that copy of the string to Dart so that then in Dart you can free it when you're done with it.

Ok I'll make the changes on the C side for now.
If I had a library for which I didn't have the source code (only binaries and headers) am I right in saying the only workaround for this at the moment to wrap that library in a layer that makes copies for all of the callbacks?

If I had a library for which I didn't have the source code (only binaries and headers) am I right in saying the only workaround for this at the moment to wrap that library in a layer that makes copies for all of the callbacks?

Correct 👍

Fair enough, thanks for getting back to me quickly! 🙂


Bugfix: if a @CamundaSelector is used with process element  and uuid

if a @CamundaSelector is used with process element  and the StrongUuidGenerator (which should be used for production) the selection fails because defintionKey expects a number after the second colon.
example:
@CamundaSelector(type = "intermediateThrowEvent", event = ExecutionListener.EVENTNAME_START, process = "process")
and in Setup:
...
this.idGenerator = new StrongUuidGenerator();
...
will fail without this patch.

Thanks! We cannot rely on the format of a process definition id anyway, so I will have to refactor this soon. If the pid gets to long, the format key:revision:deployment is not used at all.

thank you for accepting - much appreciated. Will you release a fixed version to central soon?

see #77 ... I fear your fix will not be enough, so I want to address this, but I have the code around here somewhere already, so yes: soon!


refactor: migrate vuex to pinia

Types of changes

[ ] New feature
[ ] Bug fix
[ ] Documentation change
[ ] Coding style change
[x] Refactoring
[ ] Performance improvement
[ ] Test cases
[ ] Continuous integration
[x] Typescript definition change
[x] Breaking change

Background and context
#5

Solution
Refactoring store directory.

How is the change tested?
Manually test all cases.

Changelog

Changelog(CN)
Changelog(EN)
Related issues

从 Vuex 迁移为 Pinia
Migrate from vuex to pinia
#5

Checklist:

[x] Provide changelog for relevant changes (e.g. bug fixes and new features) if applicable.
[x] Changes are submitted to the appropriate branch (e.g. features should be submitted to feature branch and others should be submitted to master branch)

Other information

@blackcater 近期更新后进行处理哈！感谢PR！

@blackcater hello。我们商讨目前先不添加开发logger，pinia-logger这个库，可以先暂时去掉哈。😄

@blackcater hihi。我们近期会发版哦。麻烦按照提示修改上传呀！感谢。


Support for Fairseq Translation Model

Hi, does DeepSpeed-MII support fairseq's translation model, such as transformer.wmt16.en-de or transformer.wmt19.en-de? as no task translation listed in the Supported Models and Tasks section.

@huzerD MII does not currently support this model, but we are adding support for more and more models. @jeffra is this model on our radar for support?


Start implementation of point cloud layer

Getting started

@kylebarron
I still need to actually use the NormalAccessor. But I need to read up on that Re: https://github.com/developmentseed/lonboard/issues/322#issuecomment-1900806891

You need to merge in or rebase on the latest main branch to resolve your merge conflicts

GTG
LGTU @kylebarron ?

It would also be great to create a minimal example in the same form as the existing examples directory so we can at least have a manual workflow to test that the layer works.

Awaiting last review for this PR @kylebarron

Thanks!


Including underlying deserialization errors in Status

This a fix for https://github.com/grpc/grpc/issues/24784 that supersedes proposed PR https://github.com/grpc/grpc/pull/24785
The original request was to expose DebugException in client-side response marshaller exceptions.
However the native API only accepts status code and description (and not the DebugException in the Status).
This PR create a new Status to encapsulate the deserialization exception.  A simple string representation of the DebugException is also provided  as part of the description so that the information from the exception is not lost.
In some situations the DebugException may be retained and available to the API user, such as a blocking unary call, so it is also included.
Tests for marshalling errors have also been updated

@jtattermusch for review
There seems to be lots of CI checks failing but none look relevant.


SIGSEGV on GC of actors at the end of execution

This crash was found while diagnosing #2975.
An adapted version of the code, run with flags --ponygcinitial 1 --ponygcfactor 1, would often cause some (but not all) of the Coordinator actors to be GC'd. In these cases, it would occasionally result in a segmentation fault after the 10 second timer in Main was finished.
$ lldb-3.9 tmp -- --ponygcinitial 1 --ponygcfactor 1
(lldb) target create "tmp"
Current executable set to 'tmp' (x86_64).
(lldb) settings set -- target.run-args  "--ponygcinitial" "1" "--ponygcfactor" "1"
(lldb) run
Process 24501 launched: '/home/eric/tmp/tmp' (x86_64)
Hello, let's spawn lots of actors(10) with many Noops(23)
That's 10 cycles, each with 25 actors.
Main is sleeping for 10 secs. Should see some GC now?
Fake work is done: 10 jobs. Nothing at all should stop GC from working.
GCing Coordinator 3
GCing Coordinator 0
Process 24501 stopped
* thread #2: tid = 24505, 0x0000000000423168 tmp`messageq_push + 24, name = 'tmp', stop reason = signal SIGSEGV: invalid address (fault address: 0x8)
    frame #0: 0x0000000000423168 tmp`messageq_push + 24
tmp`messageq_push:
->  0x423168 <+24>: movq   %rsi, 0x8(%rdx)
    0x42316c <+28>: retq
    0x42316d:       nopl   (%rax)
tmp`messageq_push_single:
    0x423170 <+0>:  movq   $0x0, 0x8(%rdx)
(lldb) bt
* thread #2: tid = 24505, 0x0000000000423168 tmp`messageq_push + 24, name = 'tmp', stop reason = signal SIGSEGV: invalid address (fault address: 0x8)
  * frame #0: 0x0000000000423168 tmp`messageq_push + 24
    frame #1: 0x0000000000423836 tmp`pony_sendv + 38
    frame #2: 0x000000000041fe50 tmp`cycle_dispatch + 496
    frame #3: 0x0000000000423a87 tmp`ponyint_actor_run + 119
    frame #4: 0x000000000041c5f7 tmp`run_thread + 215
    frame #5: 0x00007ffff79b96ba libpthread.so.0`start_thread + 202
    frame #6: 0x00007ffff72d541d libc.so.6`__clone + 109 at clone.S:109

The invalid address and backtrace were consistent over multiple segfaults. When either all or no Coordinators had been GC'd before the timer expires, this crash does not happen. Also, I could not reproduce it without either of these two flags.
Compiler info:
$ ponyc --version
0.25.0 [release]
compiled with: llvm 3.9.1 -- cc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
Defaults: pic=false ssl=openssl_0.9.0

The behavior can be triggered by running a benchmark on ponylang/http's sample server.
ab -c 8 -n 10000 http://localhost:50000/bob
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
apr_socket_recv: Connection reset by peer (104)
Total of 8436 requests completed

lldb
...
localhost - - [25/Jul/2019:03:49:26 +0000] "GET /bob HTTP/1.0" 200 0 "" "ApacheBench/2.3"
Process 25523 stopped
* thread #2, name = 'ponyfun', stop reason = signal SIGSEGV: invalid address (fault address: 0x8)
    frame #0: 0x00005555555c50f8 ponyfun`messageq_push + 24
ponyfun`messageq_push:
->  0x5555555c50f8 <+24>: movq   %rsi, 0x8(%rdx)
    0x5555555c50fc <+28>: retq
    0x5555555c50fd:       nopl   (%rax)
ponyfun`messageq_push_single:
    0x5555555c5100 <+0>:  movq   $0x0, 0x8(%rdx)

(lldb) bt
* thread #2, name = 'ponyfun', stop reason = signal SIGSEGV: invalid address (fault address: 0x8)
  * frame #0: 0x00005555555c50f8 ponyfun`messageq_push + 24
    frame #1: 0x00005555555c58ac ponyfun`pony_sendv + 44
    frame #2: 0x00005555555c20c9 ponyfun`cycle_dispatch + 569
    frame #3: 0x00005555555c5b0f ponyfun`ponyint_actor_run + 127
    frame #4: 0x00005555555ba11a ponyfun`run_thread + 218
    frame #5: 0x00007ffff7bfd57f libpthread.so.0`start_thread + 239
    frame #6: 0x00007ffff79c60e3 libc.so.6`__GI___clone + 67

Just another data point!

Eric says it happened with additional flags back then.
I can confirm that segfault can happen with no flags as well now.
Most recent work on GC was done in #3154 afaik.
ponyc.exe -version
0.29.0-d3fc84f1 [debug]
compiled with: llvm 7.0.1 -- msvc-15-x64 1916
Defaults: pic=false

here's debug stacktrace:
Exception thrown: write access violation.
**_Tgt** was 0x8. occurred

aoc5.exe!std::_Store_relaxed_8(volatile unsigned __int64 * _Tgt, unsigned __int64 _Value) Line 1766	C++
aoc5.exe!std::_Atomic_store_8(volatile unsigned __int64 * _Tgt, unsigned __int64 _Value, std::memory_order _Order) Line 1813	C++
aoc5.exe!std::atomic_store_explicit(std::_Atomic_address * _Atom, void * _Value, std::memory_order _Order) Line 474	C++
aoc5.exe!std::_Atomic_address::store(void * _Value, std::memory_order _Order) Line 620	C++
aoc5.exe!std::atomic<pony_msg_t *>::store(pony_msg_t * _Value, std::memory_order _Order) Line 231	C++
aoc5.exe!std::atomic_store_explicit<pony_msg_t *>(std::atomic<pony_msg_t *> * _Atom, pony_msg_t * _Value, std::memory_order _Order) Line 824	C++
aoc5.exe!messageq_push(messageq_t * q, pony_msg_t * first, pony_msg_t * last) Line 56	C++
aoc5.exe!ponyint_actor_messageq_push(messageq_t * q, pony_msg_t * first, pony_msg_t * last) Line 140	C++
aoc5.exe!pony_sendv(pony_ctx_t * ctx, pony_actor_t * to, pony_msg_t * first, pony_msg_t * last, bool has_app_msg) Line 578	C++
aoc5.exe!pony_send(pony_ctx_t * ctx, pony_actor_t * to, unsigned int id) Line 662	C++
aoc5.exe!check_blocked(pony_ctx_t * ctx, detector_t * d) Line 565	C++
aoc5.exe!cycle_dispatch(pony_ctx_t * ctx, pony_actor_t * self, pony_msg_t * msg) Line 844	C++
aoc5.exe!handle_message(pony_ctx_t * ctx, pony_actor_t * actor, pony_msg_t * msg) Line 199	C++
aoc5.exe!ponyint_actor_run(pony_ctx_t * ctx, pony_actor_t * actor, bool polling) Line 325	C++
aoc5.exe!run(scheduler_t * sched) Line 899	C++
aoc5.exe!run_thread(void * arg) Line 950	C++
[External Code]

what application are you running there as a test application @plprofetes ?

the same I used in #2975, you don't want to get into details again ;)
Then it was about eating up whole memory, now it just segfaults, 99% of the time.

Nope, sorry, I'm using the application that caused me to report #2975. The one with linked list of actors.
Let me test this issue with #2975 thoroughly.

@plprofetes can you provide a gist of the application in question? to make sure we have the right one to test with.

@EpicEric i'm unable to reproduce this with vendored LLVM 7.0.1 on Ubuntu bionic using the latest master with arch set to x86_64 which would rule out some optimizations being done.

do you want me to describe details and [no]sense of that, again?
https://gist.github.com/plprofetes/768163f6c27b22b9aeda6529ddeb6e21

@plprofetes nope. thanks, i have something that is consistently reproduceable. just tried. boom boom boom!

This probably dates back to https://github.com/ponylang/ponyc/pull/2709 and is more likely now. @dipinhora do you have any thoughts?

@SeanTAllen some debuggery has shown the following is occurring:

Promise[State] actors are being created by Unit
Promise[State] actors are being destroyed by the Cycle Detector
Unit is sending a block message to the Cycle Detector with a reference to at least some already destroyed Promise[State] actors
Cycle Detector asks a Promise[State] actor that is already destroyed if it is blocked resulting in a segfault

What I haven't figured out yet is whether the changes in #2709 broke things or if this was always a possibility (if an actor wasn't scheduled for some reason and so didn't send block/unblock messages) which was masked previously due to every actor always sending block/unblock messages all the time and #2709 only exposed the timing related issue already present but extremely unlikely to be encountered previously.

As far as I can tell, this is fixed by #3254 which will be merged soon and released not so long after that.
Thanks to everyone who contributed to finding and closing this.