imagejob was marked as failed

Environment:

Eraser version: v0.3.0 (scanner disabled)containers:
- args:
  - --leader-elect=false
  - --eraser-image=ghcr.io/azure/eraser:v0.3.0
  - --collector-image=ghcr.io/azure/collector:v0.3.0
  - --repeat-period=5m
  - --job-cleanup-on-success-delay=5m

Kubernetes version: (use kubectl version):> kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.6", GitCommit:"42a9a90338d705a1650fb68b7891f84b62adb0b0", GitTreeState:"clean", BuildDate:"2022-06-15T04:25:21Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.24) and server (1.22) exceeds the supported minor version skew of +/-1

What steps did you take and what happened:
Eraser didn't move on when imagejob was marked as failed. Is there any approach to debug why imagejob failed?
I see default job-cleanup-on-error-delay is 24 hours, does it mean failed job would be deleted after 24 hours waiting?
Logs:
> kgp -n eraser-system
NAME                                            READY   STATUS      RESTARTS   AGE
collector-aks-nodes-24729421-vmss000000-fqflx   0/1     Completed   0          11h
eraser-aks-nodes-24729421-vmss000000-6hqsm      0/1     Completed   0          18h
eraser-controller-manager-7754df6b7d-wdbzk      1/1     Running     0          21h

> kl -n eraser-system collector-aks-nodes-24729421-vmss000000-fqflx
{"level":"info","ts":1658157632.2651355,"logger":"collector","msg":"excluded configmap was empty or does not exist"}

> kl -n eraser-system eraser-aks-nodes-24729421-vmss000000-6hqsm
{"level":"info","ts":1658131161.465809,"logger":"eraser","msg":"excluded configmap was empty or does not exist"}
{"level":"info","ts":1658131162.1706002,"logger":"eraser","msg":"Removed","given":"sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911","digest":"sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5b911","digest":"sha256:7b178dc69474dd40a6471673c620079746e086c341b373fa723c09e043a5
b911"}
{"level":"info","ts":1658131162.1706374,"logger":"eraser","msg":"Image is running","image":"sha256:f936070357350bdb2d0866a41fc996f0caa761474dd4111e6199e617f7b4d9eb"}

> kl -n eraser-system eraser-controller-manager-7754df6b7d-wdbzk
{"level":"info","ts":1658157621.1683147,"logger":"controller","msg":"ImageCollector Reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.258417,"logger":"controller","msg":"Successfully created collector ImageJob","process":"imagecollector-controller","job":"imagejob-pzlnm"}
{"level":"info","ts":1658157621.2584496,"logger":"controller","msg":"done reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.4286213,"logger":"controller","msg":"Started collector pod on node","process":"imagejob-controller","job":"imagejob-pzlnm","node":"aks-nodes-24729421-vmss000000","nodeName":"aks-nodes-24729421-vmss000000"}
{"level":"info","ts":1658157621.4292846,"logger":"controller","msg":"Marking job as failed","process":"imagejob-controller","success ratio":1,"actual ratio":0}
{"level":"info","ts":1658157621.5052567,"logger":"controller","msg":"ImageCollector Reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.505326,"logger":"controller","msg":"failed phase","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.581916,"logger":"controller","msg":"done reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.5820005,"logger":"controller","msg":"ImageCollector Reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.5820642,"logger":"controller","msg":"failed phase","process":"imagecollector-controller"}
{"level":"info","ts":1658157621.5820723,"logger":"controller","msg":"Delaying imagejob delete","process":"imagecollector-controller","job":"imagejob-pzlnm","deleteAter":"2022-07-19 15:20:21 +0000 UTC"}
{"level":"info","ts":1658157621.5820847,"logger":"controller","msg":"done reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658188347.44031,"logger":"controller","msg":"ImageCollector Reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658188347.4405007,"logger":"controller","msg":"failed phase","process":"imagecollector-controller"}
{"level":"info","ts":1658188347.4405115,"logger":"controller","msg":"Delaying imagejob delete","process":"imagecollector-controller","job":"imagejob-pzlnm","deleteAter":"2022-07-19 15:20:21 +0000 UTC"}
{"level":"info","ts":1658188347.4405496,"logger":"controller","msg":"done reconcile","process":"imagecollector-controller"}
{"level":"info","ts":1658188347.5244133,"logger":"controller","msg":"Delaying imagejob delete","process":"imagelist-controller","job":"imagejob-cfx5z","deleteAter":"2022-07-19 07:59:09 +0000 UTC"}

Imagejob
> kd imagejob
Name:         imagejob-cfx5z
Namespace:
Labels:       <none>
Annotations:  <none>
API Version:  eraser.sh/v1alpha1
Kind:         ImageJob
Metadata:
  Creation Timestamp:  2022-07-18T07:59:08Z
  Generate Name:       imagejob-
  Generation:          1
  Managed Fields:
    API Version:  eraser.sh/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:generateName:
        f:ownerReferences:
          .:
          k:{"uid":"fda306a3-6c35-4170-b0ee-b9b1b04056f8"}:
      f:spec:
        .:
        f:template:
          .:
          f:metadata:
          f:spec:
            .:
            f:containers:
            f:restartPolicy:
            f:serviceAccountName:
            f:volumes:
    Manager:      eraser
    Operation:    Update
    Time:         2022-07-18T07:59:08Z
    API Version:  eraser.sh/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:deleteAfter:
        f:desired:
        f:failed:
        f:phase:
        f:skipped:
        f:succeeded:
    Manager:      eraser
    Operation:    Update
    Subresource:  status
    Time:         2022-07-18T07:59:09Z
  Owner References:
    API Version:           eraser.sh/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  ImageList
    Name:                  imagelist
    UID:                   fda306a3-6c35-4170-b0ee-b9b1b04056f8
  Resource Version:        66546
  UID:                     f084accc-62dd-449a-888d-71f627fba71f
Spec:
  Template:
    Metadata:
    Spec:
      Containers:
        Args:
          --imagelist=/run/eraser.sh/imagelist/images
          --log-level=info
        Image:              ghcr.io/azure/eraser:v0.3.0
        Image Pull Policy:  IfNotPresent
        Name:               eraser
        Resources:
          Limits:
            Cpu:     8m
            Memory:  30Mi
          Requests:
            Cpu:     7m
            Memory:  25Mi
        Volume Mounts:
          Mount Path:        /run/eraser.sh/imagelist
          Name:              imagelist-ghxp5
          Mount Path:        /run/eraser.sh/excluded
          Name:              excluded
      Restart Policy:        Never
      Service Account Name:  eraser-imagejob-pods
      Volumes:
        Config Map:
          Name:  imagelist-ghxp5
        Name:    imagelist-ghxp5
        Config Map:
          Name:      excluded
          Optional:  true
        Name:        excluded
Status:
  Delete After:  2022-07-19T07:59:09Z
  Desired:       1
  Failed:        0
  Phase:         Failed
  Skipped:       0
  Succeeded:     0
Events:          <none>

Name:         imagejob-pzlnm
Namespace:
Labels:       <none>
Annotations:  <none>
API Version:  eraser.sh/v1alpha1
Kind:         ImageJob
Metadata:
  Creation Timestamp:  2022-07-18T15:20:21Z
  Generate Name:       imagejob-
  Generation:          1
  Managed Fields:
    API Version:  eraser.sh/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:generateName:
        f:ownerReferences:
          .:
          k:{"uid":"30140bea-0650-4b8e-be7a-f9ce2d7ccea6"}:
      f:spec:
        .:
        f:template:
          .:
          f:metadata:
          f:spec:
            .:
            f:containers:
            f:restartPolicy:
            f:serviceAccountName:
            f:volumes:
    Manager:      eraser
    Operation:    Update
    Time:         2022-07-18T15:20:21Z
    API Version:  eraser.sh/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:deleteAfter:
        f:desired:
        f:failed:
        f:phase:
        f:skipped:
        f:succeeded:
    Manager:      eraser
    Operation:    Update
    Subresource:  status
    Time:         2022-07-18T15:20:21Z
  Owner References:
    API Version:           eraser.sh/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  ImageCollector
    Name:                  imagecollector-shared
    UID:                   30140bea-0650-4b8e-be7a-f9ce2d7ccea6
  Resource Version:        162062
  UID:                     36ce9251-9e5b-48e7-98f4-71b50b7bdf02
Spec:
  Template:
    Metadata:
    Spec:
      Containers:
        Image:              ghcr.io/azure/collector:v0.3.0
        Image Pull Policy:  IfNotPresent
        Name:               collector
        Resources:
          Limits:
            Cpu:     8m
            Memory:  30Mi
          Requests:
            Cpu:     7m
            Memory:  25Mi
        Volume Mounts:
          Mount Path:        /run/eraser.sh/excluded
          Name:              excluded
      Restart Policy:        Never
      Service Account Name:  eraser-imagejob-pods
      Volumes:
        Config Map:
          Name:      excluded
          Optional:  true
        Name:        excluded
Status:
  Delete After:  2022-07-19T15:20:21Z
  Desired:       1
  Failed:        0
  Phase:         Failed
  Skipped:       0
  Succeeded:     0
Events:          <none>

@jiashun0011 any reason you are setting repeat period and delay to 5mins for both? i think this is what's causing this issue

@jiashun0011 any reason you are setting both repeat period and delay to 5mins? i think this is what's causing this issue

Sounds reasonable and let me give a try.
Btw, may I know is there any approach to see the details behind
{"level":"info","ts":1658157621.505326,"logger":"controller","msg":"failed phase","process":"imagecollector-controller"}

@sozercan tested this with the new design and there was no error. Just want to confirm, it will wait 5 minutes after deletion of the job to repeat vs 5 minutes after deploying eraser.

Closing this. @jiashun0011 please verify if this still an issue with v0.4.0 and later.

Closing this. @jiashun0011 please verify if this is still an issue with v0.4.0 and later.

@sozercan It works well as expected and sorry for the late reply.


Add swagger info in AppBaseController

Creating in reference of https://github.com/InfyOmLabs/swagger-generator/issues/1

Now we are publishing AppBaseController with swagger info in latest version.


dpT calculation in waterBaseProp_dT

Reported by m.graeber on 22 Oct 2009 10:15 UTC
I suppose there is a bug in the calculation of dpT in
Modelica.Media.Water.IF97_Utilities.waterBaseProp_dT
For Region 4 there are to different lines where dpT is calculated:
aux.dpT := BaseIF97.Basic.dptofT(T);

and some lines below
aux.dpT := (vap.s - liq.s)*(if (liq.d <> vap.d)
	then (vap.s - liq.s)*liq.d*vap.d/(liq.d - vap.d)
	else BaseIF97.Basic.dptofT(aux.T));

The first algorithm should be the right one.

Migrated-From: https://trac.modelica.org/Modelica/ticket/224

Modified by dietmarw on 22 Oct 2009 16:40 UTC

Comment by hubertus on 23 Oct 2009 19:15 UTC
The second formula is the Clausius-Clapeyron equation: dp/dT = (s2-s1)/(v2-v1). In IF97, the analytical curve p = f(T) is an approximation, even though a good one. Using Clausius-Clapeyron instead makes dp/dT in my view more consistent with other parts of the fundamental equation. But you have a point in that the first one should be removed. It is no error, anyways. Check, e.g. http://theory.phy.umist.ac.uk/~judith/stat_therm/node44.html

Comment by m.graeber on 25 Oct 2009 16:09 UTC
Isn't there still an error in the equation? Shouldn't it be
aux.dpT := if (liq.d <> vap.d) then
    (vap.s - liq.s)*liq.d*vap.d/(liq.d - vap.d) else
    Modelica.Media.Water.IF97_Utilities.BaseIF97.Basic.dptofT(aux.T);

without the term (vap.s - liq.s) in front of the if equation?

Comment by hubertus on 25 Oct 2009 20:16 UTC
No, this is the right formula. Compare it against the derivative of the approximation function dpofT.

Modified by dietmarw on 26 Oct 2009 07:17 UTC

Comment by hubertus on 28 Oct 2009 23:36 UTC
fixed in d7a0945b0c63a0bf33eb04b3ab7be369da6315fb in trunk. Will be merged to maintenance later, with other media changes

Comment by dietmarw on 25 Nov 2009 11:53 UTC
Milestone MSL3.1.1 deleted


Why sbt publish instead of sbt release

the command sbt publish uses the version number from version in thisBuild (version.sbt) which is typically a SNAPSHOT version. As you might know publishing a snapshot version to bintray isn't possible.
'sbt release' updates the version number (snapshot -> actual version) before releasing. After releasing a tag is pushed to the repo.
So I was wondering why sbt publish is used, or is there a way to set the version number correctly while publishing?

@jorkzijlstra The use of release and publish terminology in SBT matches convention for Maven repos.
What you need to do is have the publish step a part of you release process, so that the publish takes place while the version number is non-SNAPSHOT, before it's bumped to a new SNAPSHOT.
The default release process includes the following three steps, in this order:

setReleaseVersion, which bumps version a.b.c-SNAPSHOT to a.b.c
publishArtifacts, which does the artifact publication (and, if you're using the sbt-bintray plugin, is linked up to publishing to bintray)
setNextVersion, which bumps the next development version to a.b.c+1-SNAPSHOT (although the +1 might be in a different place depending on which version bump strategy you're using).

Hope this helps clarify.

@oliverlockwood, indeed, I already implemented it like that.
https://github.com/kaliber-scala/scala-mailer/blob/master/build.sbt
but thanks for the reply.

Sure thing.  @dwijnand, I guess this issue can be closed?


How do I pre-set the card members name

I have the card holders name and have it pre-populated in the form.  How do I set it in the Cart object?

Good question, but it's possible as I have seen it be done recently... What have you tried already?
@theotherzach was the one that I know has made this work... perhaps they could share.

I have a same question,
I have the card holders name and have it pre-populated in the form. How do I set it in the Cart object?
Things tried by me after creating card object

Triggered change, keyup, focus and blur event on card holder name input
Added and removed class .jp-card-flipped on .jp-card element

None of the above steps worked for me.
It would be great if some one guide me how to achieve this.
Thanks

i need this feature

Hello,
i'm using this dirty solution. I'don't have time to search what events are triggering (propably keypress, keydown, blur, focus), so I use all possible events :)
`            (function($) {
$.fn.extend({
triggerAll: function (events, params) {
var el = this, i, evts = events.split(' ');
for (i = 0; i < evts.length; i += 1) {
el.trigger(evts[i], params);
}
return el;
}
});
})(jQuery);
        $('#cardModalName').val('tlykken').triggerAll(" blur  change  change  focus  select  submit  reset  keydown  keypress  keyup  keyup  keydown  keyup");`

See my answer and the entire thread here.

Bump


Enable Configuring of Reconnect Attempts and Delays

This change enables configuration of reconnect attempts and number of retries for both DCP channels and Configuration Provider.
In addition, it stops attempts to reconnect once the conductor.stop() has been called.
The client interface must have CompleteHandler and ErrorHandler for the client to be able to deliver terminal errors and normal end of stream events.

@amoudi87 I think its good to make those values configurable, but here is some input:

in the change i don't see where the config provider settings are used, and also I think it might be better to not configure the attempts at all, since shouldn't it be: try forever until a certain condition hits? for the config provider its: shutdown and for the dcp channels its: channel is not part of the server config anymore. Since what would you do if the attempts run out but the node is still part of the config?

@daschl
Check out HttpStreamingConfigProvider. The config provider settings are used there but there was no way for the client user to configure those. I just enabled that.
For the dcp channels, attempt until the node is not part of the config doesn't work since we get into situations where we disconnect but the dcp channel keeps trying to connect and fail forever, with no way to stop it.

@amoudi87 does this change have JDCP ticket associated?

@avsej This fixes JDCP-28. How do we associate the commit with the ticket?

@amoudi87 could you put it into the commit message title?

Done. I don't think I have write access to this repo.
Also, It is crucial that we have something similar to onError and OnComplete for the client since we might get some terminal events/errors.
But that would be in a subsequent change


Fixed release instructions on closing milestone

No review needed.

coverage: 25.088%. remained the same
when pulling fbd773f182873c052c6b38decbb0c9c4933e5495 on andy/fix-close-milestone
into 5f032acd9e9ec3ba8d28a749a810700eb88383bd on master.


Exploit the second console.dir argument

Summary

Fixes #10176. Now console.dir repeats the behavior of the Node.js Console class.
Test plan

beforeafter

I'm sorry this was forgot about :/ - https://github.com/facebook/jest/pull/10638 was just opened and is up to date on master. You're still a commit author in there for when that is merged :)

@jeysal then I'll close this PR.


CI/CD

Establish a process that every merge to dev branch will lead to deploy to dev environment
Establish a process that every merge to main branch will lead to deploy to prod environment

CI for dev ENV is now applied on each push/pull request to "dev" branch -
In short, every change that has been pushed to "dev"  trigger’s this workflow - https://github.com/tikalk/10n1/actions/workflows/CI-using-ghub-reg.yaml
The workflow, build the docker images(UI & DS) and push them to github registry.
You can find the images here - https://github.com/orgs/tikalk/packages?repo_name=10n1


Part 2 downgrade Playsysupdater Yellow Screen

Plaisysupdater give me a yellow screen all times i tried to execute it. I've tried Safesysupdater as well, and it freezes in "Restoring slapheap backup".  And sysupdater give me a infinite black screen.

Sysver?

Closed for no response.


Update debugger tcd

Small change to update the Debugger to also use dark mode colors.

@tylerthetiger, thank you for the contribution!


More architectures JDA-NAS

Adds support for more architectures, will follow up with PR to udp-queue natives by @aikaterna .

These changes should be submitted together with the natives

I assume a PR to this directory would be optional https://github.com/sedmelluq/jda-nas/tree/master/udp-queue-natives/dist?

Not without a change in dependencies

Understood, I wait for the upstream PR and will bump version on this repo.

closing as this is implemented in https://github.com/freyacodes/Lavalink/pull/692


Lots of client side errors in demo

I spun up the demo with some slight modifications to use amazon bedrock. I ran the tests and several of the links result in the following error:

Full config:
# This configuration compares LLM output of 2 prompts x 2 GPT models across 3 test cases.
# Learn more: https://promptfoo.dev/docs/configuration/guide
description: 'My first eval'

prompts:
  - 'Write a tweet about {{topic}}'
  - 'Write a very concise, funny tweet about {{topic}}'

providers:
  - bedrock:completion:anthropic.claude-v2

tests:
  - vars:
      topic: bananas

  - vars:
      topic: avocado toast
    assert:
      # For more information on assertions, see https://promptfoo.dev/docs/configuration/expected-outputs
      - type: icontains
        value: avocado
      - type: javascript
        value: 1 / (output.length + 1) # prefer shorter outputs

  - vars:
      topic: new york city
    assert:
      # For more information on model-graded evals, see https://promptfoo.dev/docs/configuration/expected-outputs/model-graded
      - type: llm-rubric
        value: ensure that the output is funny

Are you able to share the contents of ~/.promptfoo/output/latest.json?  That will help me troubleshoot, as I don't have access to the claude-v2 model on bedrock.

Sure!
latest.json

Thanks, very helpful!  It looks like Claude v2 may be returning a different format from expected.  The client-side error occurs because the score value is null.
I'll handle the client side error so it doesn't crash the page, but this will also require an update to the Bedrock provider code.

One potentially temporary workaround is to add a transform directive to your test cases to unpack the nested completion value in the Claude response.  That would look something like this:
defaultTest:
  options:
    transform: JSON.parse(output).completion

Thanks a lot @typpo!

Unfortunately, the transform didn't work.
    │ [FAIL] SyntaxError: Unexpected token o in JSON                     │ [FAIL] SyntaxError: Unexpected token o in JSON                     │
│                                                                    │ at position 1                                                      │ at position 1                                                      │
│                                                                    │                                                                    │                                                                    │
│                                                                    │ SyntaxError: Unexpected token o in JSON at position 1              │ SyntaxError: Unexpected token o in JSON at position 1              │
│                                                                    │ m                                                                  │ m                                                                  │
│                                                                    │     at JSON.parse (<anonymous>)                                    │     at JSON.parse (<anonymous>)                                    │
│                                                                    │     at eval (eval at transformOutput (/Users/sean.hellebu          │     at eval (eval at transformOutput (/Users/sean.hellebu          │
│                                                                    │ sch/.npm/_npx/81bbc6515d992ace/node_modules/promp...               │ sch/.npm/_npx/81bbc6515d992ace/node_modules/promp...

Ok. I should be able to get you a confirmed solution (and a proper fix) as
soon as my model access request is approved :)
On Thu, Feb 15, 2024, 8:46 AM Sean Hellebusch @.***>
wrote:

Unfortunately, the transform didn't work.
│ [FAIL] SyntaxError: Unexpected token o in JSON                     │ [FAIL] SyntaxError: Unexpected token o in JSON                     │

│                                                                    │ at position 1                                                      │ at position 1                                                      │
│                                                                    │                                                                    │                                                                    │
│                                                                    │ SyntaxError: Unexpected token o in JSON at position 1              │ SyntaxError: Unexpected token o in JSON at position 1              │
│                                                                    │ m                                                                  │ m                                                                  │
│                                                                    │     at JSON.parse ()                                    │     at JSON.parse ()                                    │
│                                                                    │     at eval (eval at transformOutput (/Users/sean.hellebu          │     at eval (eval at transformOutput (/Users/sean.hellebu          │
│                                                                    │ sch/.npm/_npx/81bbc6515d992ace/node_modules/promp...               │ sch/.npm/_npx/81bbc6515d992ace/node_modules/promp...
—
Reply to this email directly, view it on GitHub
https://github.com/promptfoo/promptfoo/issues/474#issuecomment-1946560141,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACLYJXIHE3OB6VJGJYUMB3YTY3X3AVCNFSM6AAAAABDIQSS2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBWGU3DAMJUGE
.
You are receiving this because you were mentioned.Message ID:
@.***>

Sorry, it's been a while and I still don't have access to Claude 2.1 on Bedrock.  However, I think I noticed the issue in code (a030796476d52236b02e487f7c7f264e70b2e902).  The fix should be out in 0.43.1


UnauthorizedBlobOverwrite error even though the blob write is successful and no overwrite attempt is made.

Describe the bug
Command
sudo az storage blob upload -f filename -c container -n path --account-name account-name --sas-token -**---**************************************************** --no-progress;
Output
Client-Request-ID=cc820fe8-f9db-11eb-9bd8-000d3a0613b5 Retry policy did not allow for a retry: Server-Timestamp=Tue; 10 Aug 2021 13:06:38 GMT; Server-Request-ID=7556b92d-001e-005a-17e8-8df5ad000000; HTTP status code=403; Exception=This request is not authorized to perform blob overwrites. ErrorCode: UnauthorizedBlobOverwriteUnauthorizedBlobOverwriteThis request is not authorized to perform blob overwrites.RequestId:7556b92d-001e-005a-17e8-8df5ad000000Time:2021-08-10T13:06:39.0249485Z. ERROR: This request is not authorized to perform blob overwrites. ErrorCode: UnauthorizedBlobOverwrite UnauthorizedBlobOverwriteThis request is not authorized to perform blob overwrites. RequestId:7556b92d-001e-005a-17e8-8df5ad000000 Time:2021-08-10T13:06:39.0249485Z;
However, the blob did get uploaded to the container

Why are we getting this error then? Is there some kind of inbuilt retry that didnt get the response from the operation and retried to upload but since the blob was already uploaded successfully it fails with UnauthorizedBlobOverwrite exception.
To Reproduce
sudo az storage blob upload -f filename -c container -n path --account-name account-name --sas-token -**---**************************************************** --no-progress;
Expected behavior
Expected the Az output to show it as a success, since the blob was uploaded successfully.
Environment summary
Az installed through - curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
Version -
az --version
azure-cli                         2.27.0 *
OS version - Ubuntu 18.04.5 LTS
Shell type - bash
Additional context
This is not a consistent behavior, however the number of times this issue is occurring is alarming.

Storage

Hi @StutiSri, thanks for reaching out.
Actually the blob didn't get uploaded as you expected. You can check the timestamp and find that it's an existing blob and didn't get updated by your uploading.
The error msg shows that you are not authorized to perform blob overwrites. Please make sure the sas token you are using has the write permission so that you can successfully overwrite existing blob.

We are getting this error on first time uploads.
Here is a snippet of the storage account

And the error message with the timestamp
Command - sudo az storage blob upload -f /var/local/data/be447a8e-90f0-4578-95e2-ccb1f24215f8.csv -c eastus -n cpu/standard_d2s_v3/2021/08/24/data/1629781200/be447a8e-90f0-4578-95e2-ccb1f24215f8.csv --account-name stbk2test2eastus2 --sas-token -**--**-**************************************************** --no-progress;
Time - 2021-08-24 05:00:55.9529890 UTC
Error Message - CPU_DATA_PLANE_TEST - UPLOAD_RECORD - ERROR: Output for failed run - ERROR: Client-Request-ID=4d5d814e-0498-11ec-a789-000d3a140b8c Retry policy did not allow for a retry: Server-Timestamp=Tue; 24 Aug 2021 05:01:29 GMT; Server-Request-ID=c7617c1d-801e-00f8-60a5-98a784000000; HTTP status code=403; Exception=This request is not authorized to perform blob overwrites. ErrorCode: UnauthorizedBlobOverwriteUnauthorizedBlobOverwriteThis request is not authorized to perform blob overwrites.RequestId:c7617c1d-801e-00f8-60a5-98a784000000Time:2021-08-24T05:01:29.9405835Z. ERROR: This request is not authorized to perform blob overwrites. ErrorCode: UnauthorizedBlobOverwrite UnauthorizedBlobOverwriteThis request is not authorized to perform blob overwrites. RequestId:c7617c1d-801e-00f8-60a5-98a784000000 Time:2021-08-24T05:01:29.9405835Z;
Time - 2021-08-24 05:01:30.5173270

May I know how you generated the sas-token which is used in this command?

Its generated in the C# code that's run in the Function App using AccountSasBuilder and BlobServiceClient. It is then passed to the VM to use.

Can you use az storage container generate-sas -n $ContainerName --expiry $Expiry --permissions acdlrw --account-name $StorageAccount to generate a new one and try az storage blob upload with this new sas-token?

Is there some existing issue in the sas-token generation with C# code?

Here's the sample create-a-service-sas-for-a-blob-container
You need to refine this line
sasBuilder.SetPermissions(BlobContainerSasPermissions.Read)

to set enough permissions (Better to set all permissions including Add, Create, Delete, List, Read, Write)

Our default behavior is not of overwrite. Most of the times the command works, but sometimes it gives the overwrite exception in the first time upload.
We only have the Create permission set during sas token generation.

Then it should be service issue. Can you share the debug log with --debug?


/bin/sh: 1: None/bin... not found

/bin/sh: 1: None/bin/subfinder: not found
Something went wrong with the command below:
None/bin/subfinder -d site.com -t 100 -o /opt/Osmedeus/workspaces/site.com/subdomain/site.com-subfinder.txt -nW

/bin/sh: 1: None/bin/gobuster: not found
Something went wrong with the command below:
None/bin/gobuster -m dns -np -t 100 -w /opt/Osmedeus/plugins/wordlists/shorts.txt -u site.com -o /opt/Osmedeus/workspaces/site.com/subdomain/raw-site.com-gobuster.txt

/bin/sh: 1: None/bin/amass: not found
Something went wrong with the command below:
None/bin/amass -active -d site.com -o /opt/Osmedeus/workspaces/site.com/subdomain/sitecom-amass.txt

I have the same error, im in kali.

That's happened because your current shell didn't sourcing ~/.bashrc.
Simply open a new terminal or run command source ~/.bashrc.
P/s: I'm also fixed that on install script so just clean the folder and clone the repo again.

That's happened because your current shell didn't sourcing ~/.bashrc.
Simply open a new terminal or run command source ~/.bashrc.
P/s: I'm also fixed that on install script so just clean the folder and clone the repo again.

Delete repo and clone again, same error. What did it for me was sourcing bash rc file, now I will try this interesting tool.


It is not working after 1.25.0 patch.

The timer is not detecting the true crown.

Apologies for the delay. JesterCap should now work with the 1.25.0b version of the game with the latest release:
https://github.com/crashb/jestercap/releases/tag/1.25.0b_01


command 'extension.liveServer.goOnline' not found

Issue Type: Bug
GoLive suddenly disappears.
So I tried command+L, command+O.
VSCode shows "command 'extension.liveServer.goOnline' not found".
I tried to solve the problem, but it has not been solved.
uninstall LiveServer, restart Mac, install Live Server.
Please help to clear this problem.
Extension version: 5.6.1
VS Code version: Code 1.49.0 (e790b931385d72cf5669fcefc51cdf65990efa5d, 2020-09-10T17:39:53.251Z)
OS version: Darwin x64 19.6.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz (4 x 1600)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 10, 11

Memory (System)
4.00GB (0.22GB free)

Process Argv
-psn_0_40970 --crash-reporter-id e6dc7ca3-0803-49c7-b04b-7a734db941de

Screen Reader
no

VM
0%

Thank you. I solved this problem.
I uninstall Easy Sass, and did reinstall Liveserver and EasySass.
I could not understand the cause. Thank you.


Port FAQ Page From Old Website

Port FAQ Page from the old website over. Make sure to work on the correct branch.
Old Github repo.

 Add FAQ Page


Theme loading and "editor.tokenColorCustomizations" support.

Fixes: Issue#1294
TODO:

[x] Load themes
[x] Load existing ralsp-prefixed overrides from "workbench.colorCustomizations".
[x] Load overrides from "editor.tokenColorCustomizations.textMateRules".
[x] Use RA tags to load vscode.DecorationRenderOptions (colors) from theme & overrides.
[x] Map RA tags to common TextMate scopes before loading colors.
[x] Add default scope mappings in extension.
[x] Cache mappings between settings updates.
[x] Add scope mapping configuration manifest in package.json
[x] Load configurable scope mappings from settings.
[x] Load JSON Scheme for text mate scope rules in settings.
[x] Update Readme.

Borrowed the theme loading (scopes.ts) from Tree Sitter with some modifications to reading "editor.tokenColorCustomizations" for merging with loaded themes and had to remove the async portions to be able to load it from settings updates.
~Just a PoC and an idea I toyed around with a lot of room for improvement.~
For starters, certain keywords aren't part of the standard TextMate grammar, so it still reads colors from the ralsp prefixed values in "workbench.colorCustomizations".
But I think there's more value making the extension work with existing themes by maping some of the decoration tags to existing key or keys.

These will merge with the default ones coming with the extension, so you don't have to implement all of them and works well with overrides defined in settings.
    "editor.tokenColorCustomizations": {
        "textMateRules": [
            {
                "scope": "keyword",
                "settings": {
                    "fontStyle": "bold",
                }
            },
        ]
    },

Edit: The idea is to work with 90% of the themes out there by working within existing scopes available that are generally styled. It's not to say I want to erase the custom Rust scopes - those should still remain and eventually worked into a custom grammar bundle for Rust specific themes that target those, I just want to make it work with generic themes offered on the market place for now.
A custom grammar bundle and themes for Rust specific scopes is out of... scope for this PR.
We'll make another round to tackle those issues.
Current fallbacks implemented
    [
        'comment',
        [
            'comment',
            'comment.block',
            'comment.line',
            'comment.block.documentation'
        ]
    ],
    ['string', ['string']],
    ['keyword', ['keyword']],
    ['keyword.control', ['keyword.control', 'keyword', 'keyword.other']],
    [
        'keyword.unsafe',
        ['storage.modifier', 'keyword.other', 'keyword.control', 'keyword']
    ],
    ['function', ['entity.name.function']],
    ['parameter', ['variable.parameter']],
    ['constant', ['constant', 'variable']],
    ['type', ['entity.name.type']],
    ['builtin', ['variable.language', 'support.type', 'support.type']],
    ['text', ['string', 'string.quoted', 'string.regexp']],
    ['attribute', ['keyword']],
    ['literal', ['string', 'string.quoted', 'string.regexp']],
    ['macro', ['support.other']],
    ['variable', ['variable']],
    ['variable.mut', ['variable', 'storage.modifier']],
    [
        'field',
        [
            'variable.object.property',
            'meta.field.declaration',
            'meta.definition.property',
            'variable.other'
        ]
    ],
    ['module', ['entity.name.section', 'entity.other']]

Hm, can with remove our own settings completely in favor of TextMate definitions?
I'd rather have one way to config the coloring, and I agree that theme files are better for this.
If the existing rust tm grammar does not contain tags that we need (like mutable variabels), should we ship our own .tm?
(Note: I haven't looked at the grammar files for VS Code at all, so I might be suggesting something stupid here)

@matklad Sure!
Tree Sitter ships their own grammar where they pattern match.
But honestly, very few themes will support custom Rust scopes.
My suggestion is to work within the existing grammars for the scopes that are very custom.
                decoration('keyword.unsafe'),
                decoration('builtin'),
                decoration('macro'),
                decoration('variable.mut', 'underline'),

These are the current "custom" scopes, though I am not sure about "builtin".
Surely there's has to be a decent compromise for highlighting them with existing grammar.
I'm not 100% what Tree Sitter is doing here , but I know they also ship their own grammar files for each language they support.

@matklad I actually thought of a plan forward here so you can abandon the old settings.
So the issue is, some of the scopes aren't part of the general grammars. Not to mention many themes won't bother supplying styling for them.
Something like 'variable.mut' isn't going to be in a lot of user made themes not to mention isn't part of the regular grammars supplied.
So I was thinking of a mapper.
That apart from supplying a pre-configured default mapper, lets the user override the scopes.
"rust-analyzer.themingScopes" : {
  "keyword.unsafe" : "keyword.control",
  "macro" : "invalid. illegal",
   "variable.mut" : "variable",
  "builtin" : "support. type"
}

I am sure there are better scopes to map them to, but this is a start at least to get RA to work with most of the supplied themes on the VSCode extension marketplace.

LGTM!
Let's also add a short note here about how to configure colors now.

Sure, but I wasn't... done. :)
It's still a work in progress, I just wanted some feedback on some of the ideas I had in mind to not.. half-ass it.

So I was thinking of a mapper.

Hm, I think this can backfire. Like, something like invalid.illegal sounds pretty inappropriate :) I think it's important that we preserve the ability to do custom things like variable.mut. It would be best if we can somehow define our own scopes, but, if that fails, just custom rust-specific override seettings seem ok

@matklad That was just an example. It doesn't have to backfire unless the user wants to.
I think to be safe, RA would pick sane standards.
variable -> variable
keyword.unsafe -> keyword.control
I agree that keeping the existing scopes is better, that's why I didn't suggest to change their names at all.
This is an intermediate step until a better solution comes up that allows custom tags/scopes to work with existing themes designed.

The important thing here is that we support constructs that exist only in Rust, and not in other languages. In particular we should man if, while and firends to keyword.control, and for unsafe we really need a dedicated rust.unsafe tag (which, ideally, should fallback to keyword).

@matklad Yeah, and with the mapper, it can fallback to keyword out of the box.
If it's undefined, then it will be keyboard.unsafe and letting some theme picks it up.
But that's a bit more work with supplying a grammar bundle.
This is more intermediate that lets it work today.

So the plan I had in mind:

Let RA use existing scopes for the keywords defined by RA that matches both.  [DONE]

Let RA allow mapping custom scopes to existing scopes without removing the custom ones from the code. [WIP]

Let RA supply a grammar bundle with custom scopes and pattern match the tokens that fit.

The last portion would let RA work with variable.mut and keyword.unsafe and is also what the tree-sitter does.

SGTM, although I still have a feeling that it probably makes sense to just skip step 2.

@matklad Maybe I'm the one missing something here. But if you skip 2 you will be leaving keyword.unsafe and other custom scopes unmapped unless the theme supports it?
That's a big unless as my original plan to make it work with 90% of the themes out there in a nice compromise that lets the user fix those compromises if they so wsh without having to find a Rust-specific theme .

The important thing here is that we support constructs that exist only in Rust, and not in other languages. In particular we should man if, while and firends to keyword.control, and for unsafe we really need a dedicated rust.unsafe tag (which, ideally, should fallback to keyword).

This already happens now as the names are the same, so not really changing anything.
I wasn't talking about mapping everything - just the ones that were custom. This also makes the config portion easier to reason about from user PoV.

Hm, yeah, convinced that the mapping is a nice intermediate step! Sorry for the back and forth here :)

Bit off-topic, but this tree-sitter demo is pretty cool: https://twitter.com/oni_vim/status/1167475446222479361.

@seivan @matklad This looks really nice! Thanks for putting an effort into this.
I'm currently using Clion as my daily driver, and I really like their highlighting for Rust.
I was just playing with RA's syntax highlighting engine this weekend when I saw this PR!
I really like the flexibility of RA's syntax engine, I find it much nicer to play around with than the Textmate regexes VSCode has on by default.
Currently I have something like this (in a small fork I'm playing with), this is from the html based tests.

Just some thoughts - some of those are relevant to the highlighting engine in rust-analyzer and some to the vscode frontend this PR is modifying.

If we take https://macromates.com/manual/en/language_grammars as reference:

#[derive] and other attibutes should be marked as entity
stuff inside macro should be meta? I'm not entierly sure about that one, but it seems reasonable if we would want to implement a feature such as unsafe scopes (it's so good!) as meta.rust.unsafe or something to be able to decorate it.

It would be ideal if we could make highlighting scopes would be a bit more granular than available today.
For example - Type aliases, generics, and lifetime annotations should be allowed to be colored differently, they could default to what's available today, although I find Clion's choices more tasteful.

If any help is necessary on the rust end of things (matching different categories for AST entities) I'd be happy to assist :)

It would be ideal if we could make highlighting scopes would be a bit more granular than available today.
For example - Type aliases, generics, and lifetime annotations should be allowed to be colored differently, they could default to what's available today, although I find Clion's choices more tasteful.

The issue is working with existing themes that do not supply any theming for the Rust specific scopes.
A good compromise is to use the existing scopes as much as we can and also support any Rust "specific" theme that would want granular scopes.
This is yet to be implemented.
What exists as of today (with this pr) is this

Work as much as we can with existing scopes that are shared.
These include "keyword", "variable" and etc.

Allow overriding scopes by defining editor.tokenColorCustomizations .
This lets you add "fontStyle": "bold" or "foregroundColor": "#FF0000"

Allow overriding Rust specific scopes with ralsp prefixed tags under workbench.colorCustomizations.

This lets you target the custom scopes with your own colour, obviously this is theme independent and not the best approach .
BUT it does let you have granular control over what unsafe or variable.mut looks like, which might be important to some. It's critical we don't remove the granular control while working with 90% of the themes out there.

If any help is necessary on the rust end of things (matching different categories for AST entities) I'd be happy to assist :)

I'd love some help on what good sane defaults could be mapped to the Rust specific scopes.
 'keyword.unsafe'
'builtin'
'macro'
'variable.mut'

These are the scopes currently not targeted by either user supplied themes or editor.tokenColorCustomizations  however you can still target them with statically defined colours under workbench.colorCustomizations.
So what's missing today is

Good sane default mapping to existing scopes
Code that takes care of said mapping

Eventually in a different PR also supply a grammar bundle for future Rust specific themes.

#[derive] and other attibutes should be marked as entity
stuff inside macro should be meta? I'm not entierly sure about that one, but it seems reasonable if we would want to implement a feature such as unsafe scopes (it's so good!) as meta.rust.unsafe or something to be able to decorate it.

From reading that document, I think you swapped the meaning of meta and entity:

meta — the meta scope is generally used to markup larger parts of the document. For example the entire line which declares a function would be meta.function and the subsets would be storage.type, entity.name.function, variable.parameter etc. and only the latter would be styled.

@bjorn3 I think my reasoning is correct.
so if we have:
unsafe {
    ffi_fn();
}

From the explanation -
For example the **entire** line which declares a function would be meta.function .
We would have keyword.unsafe followed by a meta scope of meta.unsafe.
Inside this scope you would have another function call etc..
The same reasoning goes for stuff that's inside macros and so forth.

@seivan
IMO some sane defaults (unrelated to rust) would be:
unsafe -> storage.modifier
'builtin' -> variable.language
'macro' -> support.other.
'variable.mut' just a regular variable.

@omerbenamram Appreciate the help.
Unless I misunderstand what  builtin is wouldn't support be a better mapping?

support — things provided by a framework or library should be below support.

function — functions provided by the framework/library. For example NSLog in Objective-C is support.function.
class — when the framework/library provides classes.
type — types provided by the framework/library, this is probably only used for languages derived from C, which has typedef (and struct). Most other languages would introduce new types as classes.
constant — constants (magic values) provided by the framework/library.
variable — variables provided by the framework/library. For example NSApp in AppKit.
other — the above should be exhaustive, but for everything else use support.other.

I just want to point out that dealign with macro implementations is out of the scope for this PR.
The idea is to target "regular" Rust code for now and make it work with 90% of the themes out there.
Adding a custom grammar bundle and working with Rust specific scopes is... out of scope (pun!) for now.

@seivan I feel like support is relevant to things from std.
If I understand correctly, builtin types are primitives (i32, bool, u64, str etc..), which would make sense as part of language.

@seivan I feel like support is relevant to things from std.
If I understand correctly, builtin types are primitives (i32, bool, u64, str etc..), which would make sense as part of language.

Ah that makes sense, thanks!

So what I am doing now is going through this list and checking what part of my code they actually target because it's unclear for me
                decoration('comment'),
                decoration('string'),
                decoration('keyword'),
                decoration('keyword.control'),
                decoration('keyword.unsafe'),
                decoration('function'),
                decoration('parameter'),
                decoration('constant'),
                decoration('type'),
                decoration('builtin'),
                decoration('text'),
                decoration('attribute'),
                decoration('literal'),
                decoration('macro'),
                decoration('variable'),
                decoration('variable.mut', 'underline'),
                decoration('field'),
                decoration('module')

Maybe @matklad could give some input on what they actually are?
For instance things like field will target both field definitions in a struct as well as referring o the field. While module will not target module definitions, but only when referring to modules.

For instance type seems kinda janky.

I set it to black and those are the ones targeted.

Oof, I think some of these might be my fault. I just hacked at them until the colorization looked reasonable, e.g. field affecting both field references and declarations was intentional.

@lnicola with recent work by @viorina on classify_name / classify_name_ref, I think it should be possible to implement "color references like the declarations they resolve to" logic relatively easy.

@matklad @omerbenamram
So I added support for fallback scopes in case a theme is missing it.
It's a bit of a shame that the theming loading and scopes aren't more intelligent as they need to match the exact keyname. This is probably due to the fact that we're loading the raw values from a theme and try to do a key look up. Tree Sitter does this as well.
This will allow multiple attempts to find a scope within a theme before failing and fallback to the hardcoded rlsp color tags.
    ['keyword.unsafe', ['storage.modifier', 'keyword.other', 'keyword.control']],
    ['function', ['entity.name.function']],
    ['parameter', ['variable.parameter']],
    ['type', ['entity.name.type']],
    ['builtin', ['variable.language', 'support.type', 'support.type']],
    ['text', ['string', 'string.quoted', 'string.regexp']],
    ['attribute', ['keyword']],
    ['literal', ['string', 'string.quoted', 'string.regexp']],
    ['macro', ['support.other']],
    ['variable.mut', ['variable']],
    ['field', ['variable.object.property']],
    ['module', ['entity.name.section']]

Any input on what would be better to map against? I just want to point out not all themes support these so there's still a chance of failure. BUT you can always define them yourself in
"editor.tokenColorCustomizations"
Meaning, you can always fill in the gap until we supply a language grammar for Rust.

The fallbacks for these
    ['parameter', ['variable.parameter']],

    ['macro', ['support.other']],

    ['field', ['variable.object.property']],

    ['module', ['entity.name.section']]

Seem to fail a lot, so need to add more fallbacks.
@omerbenamram Any input on this?

@seivan I took a few minutes to write a small github scraper to search through some textmate themes for fun.
here are some top used scopes for themes that mention rust and their semantic meaning (they are sorted by number of usages).
Here are some that use support.
69: support.function -> ['Library Function', 'Built-in functions / properties', 'Library function']
67: support.type -> ['Builtin Types', 'Class, Support', 'Support Type']
67: support.class -> ['Blue text', 'Class, Support', 'Class']
66: support.constant -> ['[VSCODE-CUSTOM] Support Constant', 'Library Constant', 'Number, Constant, Function Argument, Tag Attribute, Embedded']
44: support.other.variable -> ['Library variable', 'Variable, String Link, Regular Expression, Tag Name']
44: support -> ['Rust: Extern Crate', 'Support', 'Language Built-ins']
23: support.variable.dom.js -> ['[VSCODE-CUSTOM] JS/TS Support Variable Dom', 'Normal']
23: support.variable.property.dom.js -> ['Member variables/functions', '[VSCODE-CUSTOM] JS/TS Support Variable Property Dom']
21: support.function.any-method -> ['Support Function Any Method', 'Function call']
20: support.other.escape.special.regexp -> ['Escape/constant']
20: support.other.module.haskell -> ['Data modifier']
20: support.other.namespace.use.php -> ['Data modifier']
20: support.function.builtin.rust -> ['Functions']
20: support.function.C99 -> ['Functions']
20: support.function.core.ruby -> ['Functions']
20: support.function.go -> ['Functions']
20: support.function.js -> ['Functions']
20: support.function.log.rust -> ['Functions']
20: support.function.magic.php -> ['Functions']
20: support.function.std.rust -> ['Functions']
19: support.function.be.latex -> []
19: support.function.general.tex -> []
19: support.function.section.latex -> []
17: support.function.builtin.python -> ['Functions']
16: support.function.builtin.shell -> ['Functions', 'Bold text']
14: support.function.definition.latex -> []
14: support.function.builtin.go -> ['Functions']
12: support.macro.rust -> ['Text', 'Function Call', 'Rust macro']
11: support.function.rust -> ['Bold text', 'Rust function']
11: support.constant.property-value.css -> ['Identifier', '[VSCODE-CUSTOM] Css Support Constant Value']
11: support.type.property-name.json -> ['Constants (JSON)', '[VSCODE-CUSTOM] JSON Property Name']
10: support.function.source.rust -> ['Function call']
10: support.type.property-name -> ['Built-in functions / properties', '[VSCODE-CUSTOM] Support Type Property Name']
10: support.variable -> ['Built-in magic functions and constants', 'Support variable']
10: support.function.magic -> ['Built-in magic functions and constants', 'Magic function']
9: source.cssproperty-name.support -> ['Source Css Property Name Support,source Css Property Value Support']
9: source.cssproperty-value.support -> ['Source Css Property Name Support,source Css Property Value Support']
9: support.type.property-name.jsonpunctuation -> ['[VSCODE-CUSTOM] JSON Punctuation for Property Name']
9: support.variable.dom.ts -> ['[VSCODE-CUSTOM] JS/TS Support Variable Dom']
9: support.variable.property.dom.ts -> ['[VSCODE-CUSTOM] JS/TS Support Variable Property Dom']
9: support.variable.property.js -> ['[VSCODE-CUSTOM] JS/TS Variable Property']
9: support.variable.property.ts -> ['[VSCODE-CUSTOM] JS/TS Variable Property']
9: support.variable.property.tsx -> ['[VSCODE-CUSTOM] JS/TS Variable Property']
9: support.type.exception -> ['Exception']
9: source.clojuresupport.other.keyword -> []
9: support.function.name -> ['Function call']
9: support.function.builtin -> ['Builtin function']
9: support.punctuation.expression -> ['String Quotes']
9: meta.declaration.module.haskellsupport.other.module.haskell -> ['Module name']
9: meta.import.haskellsupport.other.module.haskell -> ['Imported module']

And for entity:
89: entity.name.tag -> ['Keyword Common', 'Entity Name Tag', 'Tag name']
87: entity.other.attribute-name -> ['Tag Attribute', 'HTML/CSS attribute names', 'Entity Other Attribute Name']
69: entity.other.inherited-class -> ['Inherited classes', 'String, Symbols, Inherited Class, Markup Heading', 'Strings, Inherited Class']
66: entity.name.class -> ['Class, Support', 'Class', 'Entity Name Class,entity Name Type Class']
58: entity.name.function -> ['Function names', 'Entity Name Function', 'Function name']
42: markup.headingentity.name -> ['Markdown: heading']
22: entity.name.type.class -> ['Class, Support', 'Class', 'Entity Name Class,entity Name Type Class']
20: entity.name.type.go -> ['Data modifier']
20: entity.name.type.namespace.php -> ['Data modifier']
14: entity.name.section.puppet -> ['Data modifier']
14: entity.name.function.decorator.python -> ['Data modifier']
13: entity.other.attribute-name.id -> ['entity.other.attribute-name.id', 'Attribute IDs', 'Entity Other Attribute Name Id']
13: entity.name.impl.rust -> ['Function Call', 'Bold text', 'Rust impl']
12: entity.name.type -> ['Entity Name Type', 'Types', 'Type Alias']
11: entity.name.section.markdown -> ['[VSCODE-CUSTOM] Markdown headings', 'Heading (Markdown)']
11: entity.name.module -> ['Ruby module', 'Module Name']
11: entity.name.struct -> ['Rust struct', 'Struct Name']
10: entity.name.section -> ['Entity Name Section', 'Headings']
10: entity.name.namespace -> ['Namespace']
10: entity.name.tag.css -> ['CSS: tag-name', 'Keyword, Storage']
9: entity.name.function.makefile -> ['Function definition', 'Bold text']
9: punctuation.definition.identity -> ['Punctuation Definition Heading,punctuation Definition Identity']
9: source.gfmlinkentity -> ['Source Gfm Link Entity']
9: entity.name.lifetime.rust -> ['Storage Modifier Lifetime Rust,entity Name Lifetime Rust']
9: constant.character.entity -> ['Constant.character.entity']
9: entity.other -> ['Entity-Other']
9: meta.function-callentity.name.function.js -> ['Function call']
9: entity.name.package -> ['Package Name']
9: meta.declaration.exports.haskellentity.name.function.haskell -> ['Exported function']
9: entity.other.task-tag -> ['Task tag']
9: meta.method.identifierentity.name.function.java -> ['Java method definition']
9: entity.name.type.new -> ['New Object - JavaScriptNext']
9: entity.name.function.program -> ['Program name']
9: meta.constantentity.name.constant -> ['Ruby constant']
9: meta.function.decoratorentity.name.function.decorator -> ['Decorator']
9: storage.type.templatetag.djangoentity.tag.tagbraces.django -> ['Django template tag braces']
9: storage.type.variableentity.tag.tagbraces.django -> ['Django variable tag braces']
9: entity.other.jinja.delimiter -> ['Jinja tag delimiter']
9: entity.other.attribute-name.localname -> ['Local name']
9: constant.character.entity.html -> ['HTML entity']
9: constant.character.entity.htmlpunctuation -> ['HTML entity']
9: entity.name.tag.doctype -> ['doctype']
9: meta.tagentity.name -> ['Tag']
9: source.adbentity.name.function -> ['Function']
9: entity.name.filename.adb -> ['Info']
9: entity.name.tag.logcat.tag -> ['logcat Tag']
9: entity.name.function.logcat -> ['logcat Verbose']
9: entity.name.class.logcat -> ['logcat Info']
9: entity.other.attribute-name.class.css -> ['CSS: .class']
9: entity.other.attribute-name.class.csspunctuation -> ['CSS: .class']
9: entity.other.attribute-name.id.css -> ['CSS: #id']
9: entity.other.attribute-name.id.csspunctuation -> ['CSS: #id']
9: entity.other.attribute-name.pseudo-element -> ['CSS: pseudo-element']
9: entity.other.attribute-name.pseudo-class -> ['CSS: pseudo-class']
9: entity.other.attribute-name.tag.pseudo-class -> ['CSS: pseudo-class']
9: entity.name.tag.wildcard -> ['CSS: tag wildcard']
9: entity.other.attribute-name.universal -> ['CSS: tag wildcard']
9: entity.other.attribute-name.placeholder-selector.sass -> ['SASS: Placeholder Selector']
9: entity.other.attribute-selector.sass -> ['SASS: Attribute Selector']
9: entity.name.filename.find-in-files -> ['Find In Files: filename']
9: source.ramlmeta.key-valueentity.name.function.yaml -> ['RAML Method name']
9: source.ramlmeta.resourceentity.name.tag.yaml -> ['RAML Resource name']
9: entity.other.control -> ['Regex Control Anchors']
9: meta.function-declaration.swiftentity.type.function.swift -> ['Swift Function Definition']
7: meta.functionentity.name.function -> ['Function definition']
7: entity.name.function.abp -> ['Function definition']
7: entity.name.function.go -> ['Function definition']
7: meta.class-method.jsentity.name.function.js -> ['Function definition']
7: entity.name.function.fortran -> ['Function definition']
4: entity.name.function.preprocessor -> ['Preprocessor', 'Bold text']
4: entity.name.struct.rust -> ['Function Call', 'Bold text']
4: entity.name.module.rust -> ['Function Call', 'Bold text']

The script is here if it could be of any further use for you :)
https://gist.github.com/omerbenamram/f5952e1a02228a9ed1223c71a288bb9a

@omerbenamram @matklad
Alright latest commit will log in the console (web inspector) the missing scopes from a theme.
The mapper tries to be as generous as possible to a broad scopes map so it won't miss anything that might be themed.
If your themes look odd, check the web inspector to see what scopes weren't mapped to anything viable.
Then check if your current theme isn't styling any of the scopes mapped to in scopes_mapper.ts
It will always fall back to the hardcoded values under "workbench.colorCustomizations". Eventually we can remove that all together once some sane defaults have picked.
In the extreme case, where a scope is missing, the user can still target themselves as they would with "workbench.colorCustomizations" but instead using "editor.tokenColorCustomizations" - which is the more "correct" approach.

@matklad It's ready for review - except, how do you want error handling since I'm not allowed to log?
I updated the docs/user/ README.md#settings and added configuration to package.json with schemed backed objects to make sure we match VSCode schemas.

@seivan note that the CI is failing. I'll try to review this PR some time, but I am afraid this won't happen too soon.
If someone familiar with VS Code API wants to give a review in the meantime, that would be really helpful!

@matklad I did in fact see that - but I can't figure out what portion of the CI is making the failure.
I thought it was the linter at first but this commit should have taken care of that.
 Seem like at least there are some formatting issues in the CI failure.

@edwin0cheng I did now, thanks!
I would have suspected vscode doing the formatting on its own - I wonder if it's related to tslint being deprecated?
I was wondering earlier as well since it didn't seem to complain about inconsistent semicolon use.

@matklad Alright, no worries. I'm actually running this fork now since I need
    "editor.tokenColorCustomizations": {
        "textMateRules": [
            {
                "scope": "keyword",
                "settings": {
                    "fontStyle": "bold",
                }
            },
        ]
    },

and it's been working great :)
That being said, you still haven't brought up how you wanna do logging.
Some of the mappings don't really work on a few themes I got, I think we'll be catching 50-80% of the scopes. So users will still fill in the gaps if they're missing to their own themes.
But it's better than the existing solutions as we're working directly with vscode.
Meaning you can define more customizations than what ralsp prefixed values in "workbench.colorCustomizations" allows - which is only colour for now.
It would be nice to be able to log these for debugging purposes.
I've currently commented out the parts that log the missing stuff but it's useful to have around since it lets you know exactly what's missing.
My solution to this is to have a smarter mapping that does globs instead of fixed key lookups.

Hm, yeah, looks like there's no built-in support for logging the errors: https://github.com/Microsoft/vscode/issues/10063

@matklad How about just logging to the console?
Check the commented parts. It will indicate what scopes are currently missing any themes so you can fix them and we get a better understanding at what scopes are commonly used.

I could test the pull request a bit. I used this code and configuration to test.

main.rs
use std::fs::File;
use std::io::Read;
use std::os::unix::process::CommandExt;

fn main() {
    unsafe {}
}

settings.json
"workbench.colorTheme": "Solarized Light",
"rust-analyzer.trace.cargo-watch": "error",
"rust-analyzer.highlightingOn": true,
"editor.tokenColorCustomizations": {
  // "[Solarized Light]": {
  "textMateRules": [
    {
      "scope": [
        "keyword.unsafe"
      ],
      "settings": {
        // "foreground": "#DC322F",
        "foreground": "#ff00ff",
        "fontStyle": "bold"
      }
    },
    {
      "scope": "keyword.operator",
      "settings": {
        "foreground": "#ff00ff"
      }
    }
  ]
  // }
},
"rust-analyzer.scopeMappings": {
  "keyword.unsafe": "keyword.unsafe"
}

I use Solarized Light and have some problem styling the keyword.unsafe scope. If the color is pink in the left pane then the colorization works. Green means the default Solarized Light color is taken. Orange/Red is the ralsp default.
Using editor.tokenColorCustomizations with textMateRules does not work:

I need to force a mapping using scopeMapping to make it work, even if the mapping is between identical names.

However, it breaks again, if I scope the textMateRules under a [Solarized Light] scope, such that the rules should only apply when selecting this color scheme it no longer works.

Just to show that putting the textMateRules under [Solarized Light], I can color the keyword.operator scope as seen with the :: in the imports.

Sidenote: the coloring of std::os::unix::process:CommandExt is weird, since unix and process, which are both modules  are not colored black while CommanExt is not colored blue, even though it is a trait like std::io::Read. This might be unrelated to the changes in this PR though.

@jonasbb Thanks a lot for testing!

Using editor.tokenColorCustomizations with textMateRules does not work:
You can't use custom RA scopes under editor.tokenColorCustomizations until we supply a bundle language. You can only target the scopes your language bundle supports.

I need to force a mapping using scopeMapping to make it work, even if the mapping is between identical names.

Yes, that's the idea. To get RA to style its custom scopes you need to map it to a scope your language bundle understands. But this is good, that means I should be adding keyword.unsafe to the default map 👌

However, it breaks again, if I scope the textMateRules under a [Solarized Light] scope, such that the rules should only apply when selecting this color scheme it no longer works.

Right now, only values part of "editor.tokenColorCustomizations.textMateRules" are loaded.

Just to show that putting the textMateRules under [Solarized Light], I can color the keyword.operator scope as seen with the :: in the imports.

Yeah, that's because your language bundle understands keyword.operator as it's a common scope.
Loading values from
I am still investigating on how to manage "editor.tokenColorCustomizations.[DYNAMIC THEME NAMES]"` is out scope for now until I figure out decent mappings.
I'll upload with the unsafe as a default now.
Thanks again!

@jonasbb Btw, did you strip out the comments out of Solarized Light or did it work out of the box?

@seivan I looked into your commit history and figured out how to parse jsonc and plugged that in for testing. It does not work out of the box. Out of the box everything falls back to the ralsp.* scopes.

Using editor.tokenColorCustomizations with textMateRules does not work:
You can't use custom RA scopes under editor.tokenColorCustomizations until we supply a bundle language. You can only target the scopes your language bundle supports.

I need to force a mapping using scopeMapping to make it work, even if the mapping is between identical names.

Yes, that's the idea. To get RA to style its custom scopes you need to map it to a scope your language bundle understands. But this is good, that means I should be adding keyword.unsafe to the default map ok_hand

Let me try to clarify this point a bit. I am not sure if we quite understand each other here.
These are the rules how a color is chosen as I understand it right now:

Theme supports Scope
tokenColorCustomization supports Scope
Applies color from

Default scope mappings

✅

Theme color

✅
Default scope mappings

✅
✅
Custom colors

For the third case I would argue that instead of applying the default scope mapping rules it should read the customization colors. This would allow the user to add a new scope to tokenColorCustomization and have the extension pick it up and use it directly.
Currently, it only works if in addition to specifying the scope in tokenColorCustomization you also add an entry to scopeMappings (difference between first and second screenshot above).

Just to show that putting the textMateRules under [Solarized Light], I can color the keyword.operator scope as seen with the :: in the imports.

Yeah, that's because your language bundle understands keyword.operator out of the box as it's a common scope. I am not parsing those portions at all... yet.

Looking at the Solarized theme link above, it doesn't seem like keyword.operator is supported. The only difference I see is that keyword.unsafe has a default scope mapping specified, whereas keyword.operator has not.
This means that keyword.unsafe falls back to ralsp.keyword.unsafe (even though the customization is enabled) and keyword.operator stays keyword.operator allowing me to style it.
Thank you so much for the work here :)

I left a couple of comments. I didn't test this yet, but it looks good overall. My main concern is that the tagging we do should be much more expressive in order to map better to the TM scopes, but that's not something to do in this PR. One could argue that Code might get support for semantic highlighting in the future, and it will not necessarily follow TextMate, but I'm not sure how probable that is.

I agree, I like the fact that you got your own clear and granular scopes, I would have wished for more to be honest :)

As a nit, I found the functional style [x].map(x => f(x)).forEach(x => g(x)) a bit awkward (ignore the eta conversion, that's not my point), but that's probably my personal preference.

I prefer this, and since the Rust code follows the same pattern I thought it was appropriate, do you feel strongly about it?

Note that CI seems to be failing again, but I'm not sure why. I see some Cargo.lock changes in the CI log, but I'm not sure what's running cargo update. A rebase might help here.

I've just been merging in master from time to time, since I just my fork :)
What's your suggestion on fixing this, rebase against what?

Looks like the last master fixed the CI stuff.

@lnicola Looks like latest master fixed the CI stuff.

I prefer this, and since the Rust code follows the same pattern I thought it was appropriate, do you feel strongly about it?

In Rust it is indeed idiomatic to chain iterator methods (I'm not saying that it's not idiomatic in JS/TS, but it's the first time I see it), but it's used with collections. The moral equivalent would be vec![x].into_iter().map(f).for_each(g), which certainly isn't common. Also, Rust iterators are lazy, while I imagine JS makes an intermediate allocation on each of the steps (including the single element array). It might not matter during theme loading, but it's probably something we wouldn't want during tagging.

I finally gave this a try with some of the default themes. There are some differences from the current highlighting, but nothing too bad -- most are on keywords. See the attached archive.
To summarize my previous feedback:

this seems like a desirable change
Code might get better semantic highlighting "soon"; we don't know how it will look like, but I wouldn't be surprised if it followed the TextMate scopes conventions
there are some code idioms I'm not personally a fan of (and make the code a little convoluted)
the TM scope mappings can be improved
we might want a finer-grained highlighting support in RA

The last two points above can be done in a future PR, since this one has been idling here for a long time. I think we can merge this now (or soon-ish) and try to fine-tune it more in the future.
tm.zip

Thanks for the review, @lnicola !
Yeah, let's merge this then!
bors r=lnicola

@matklad @lnicola Just a moment, I need to address some of the issues brought up. But I am kinda short on time right now. Please don't merge just yet. :)

bors r_
On Wed, 4 Dec 2019 at 13:03, Seivan Heidari<EMAIL_ADDRESS>wrote:

@matklad https://github.com/matklad @lnicola
https://github.com/lnicola Just a moment, I need to address some of the
issues brought up. But I am kinda short on time right now. Please don't
merge just yet. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/pull/2061?email_source=notifications&email_token=AANB3M6S7FWH35HQJLKSC3TQW6MANA5CNFSM4JEVXL62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF4Y5PY#issuecomment-561614527,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANB3M2HIY2OJEYRAV2VBATQW6MANANCNFSM4JEVXL6Q
.

bors r-

@matklad you made a typo there

@matklad Sorry for the long delay, I've been out of commission but I'm back and merging in master and dealign with the last comments! Just wanted to update you.

Also see https://github.com/rust-analyzer/rust-analyzer/issues/604#issuecomment-568491774. Looks like there's some native support in the LSP for this already?

@matklad @lnicola Alright, done.
I won't be able to add any other changes myself now for some time, working on other things.
But as far as I know, I have nothing else to add.

@matklad I looked at that - unless I have misunderstood completely (pretty big chance) this means that scopes.ts functionality would be unnecessary;

Loading active themes
Creating decorations (inside highlighter.ts) based on the active theme rules

However it would still need scopes_mapper.ts to map Rust scopes to TM scopes - unless you want to rename them, which I advice against, I like your names more as they're easier to understand.
Either way, I am done with the PR - fixed the review comments with this exception
but I see value of that because it makes sure the scopes_mapper and scopes are not coupled. So you could get rid of the portion of the code that deals with VsCode hacks (e.g loading themes, etc) while keeping the code that maps rust scopes to TM scopes which will still be useful even with the LSP changes currently pending.

Let's merge this and fix any fallout separatelly!
bors r+


[nrf toup] boards: improve nrf52_pca20020 devicetree

[nrf toup] boards: improve nrf52_pca20020 devicetree
Signed-off-by: Prestegård<EMAIL_ADDRESS>

@prestegaard it looks like just the name of the pull request was changed, but the commit itself was not changed. Can you please fix it up using git commit --amend?

Hi,
Commit message is now changed to suggested content.

DNM until Upmerge #296 will be merged

This PR is very old so I'm going to close it. I think I just got a ping because of the rebase. Please rebase and reopen if still relevant.


one of the main design issue is that the top navigation bar is not draggable, i think problem is with the CSS

with the new commit, this issue has been resolved.


Deploy question?

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'start',
1 verbose cli   'server.js' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart<EMAIL_ADDRESS>6 info start<EMAIL_ADDRESS>7 verbose unsafe-perm in lifecycle true
8 info<EMAIL_ADDRESS>Failed to exec start script
9 verbose stack Error<EMAIL_ADDRESS>start: grunt "server.js"
9 verbose stack Exit status 1
9 verbose stack     at EventEmitter. (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:213:16)
9 verbose stack     at EventEmitter.emit (events.js:110:17)
9 verbose stack     at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:24:14)
9 verbose stack     at ChildProcess.emit (events.js:110:17)
9 verbose stack     at maybeClose (child_process.js:1015:16)
9 verbose stack     at Process.ChildProcess._handle.onexit (child_process.js:1087:5)
10 verbose pkgid<EMAIL_ADDRESS>11 verbose cwd /opt/mean
12 error Linux 2.6.32-504.12.2.el6.x86_64
13 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "start" "server.js"
14 error node v0.12.3
15 error npm  v2.9.1
16 error code ELIFECYCLE
17 error<EMAIL_ADDRESS>start: grunt "server.js"
17 error Exit status 1
18 error Failed at the<EMAIL_ADDRESS>start script 'grunt "server.js"'.
18 error This is most likely a problem with the meanjs package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error     grunt "server.js"
18 error You can get their info via:
18 error     npm owner ls meanjs
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

@golmic, it looks like you're trying to start the project with
grunt "server.js"

try this instead
grunt

or
node server.js

@golmic Are you still having any issues?

no.thanks~

Close?


Build fails in release with: ld: Undefined symbols: static (extension in SystemPackage):SystemPackage._StrSlice.== infix(A, A) -> Swift.Bool

Any package which depends on swift-system and uses the == on FilePath.Component will fail to build in release mode (debug is fine).
Consider this example program
import SystemPackage

print(FilePath("/x").lastComponent! == FilePath("/x").lastComponent!)

with the following dependencies/targets
    dependencies: [
        .package(url: "https://github.com/apple/swift-system.git", from: "1.0.0"),
    ],
    targets: [
        .executableTarget(
            name: "repro",
            dependencies: [
                .product(name: "SystemPackage", package: "swift-system"),
            ]
        ),
    ]

debug works
$ swift run
Building for debugging...
Build complete! (0.34s)
true

release fails
$ swift run -c release
Building for production...
error: link command failed with exit code 1 (use -v to see invocation)
ld: Undefined symbols:
  static (extension in SystemPackage):SystemPackage._StrSlice.== infix(A, A) -> Swift.Bool, referenced from:
      _repro_main in main.swift.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: fatalError
[0/1] Linking repro

Also reproes in Xcode.
This is Xcode 15A221 but also reproduces on anything else I tried.

-> rdar://113495818