Regression of 2.0.11

i have 2 version of odb 2.0.11-SNAPSHOT which build in Jun 10th. another is 2.0.11 which build yesterday.
Running on 2.0.11-SNAPSHOT, do load testing (20 concurrent request to insert data), all data insert correctly.
But if i run 2.0.11, run single thread sending request sequentially, looks ok. But if i sending concurrent requests, most of record saving failed, check on server log :
Error during insertion of key in index input : {"patchId":9990000016,"@class":"bodyAngle","_cid":293,"raw":"#278:1467","meanZ":0.3885992,"syncLog":"#189:72","meanY":0.1529076,"tzone":"0","time":"1434323024741","meanX":0.8929008,"value":-24.048149,"user":"#106:170"} Error during insertion of key in index
And after this happen, no matter single thread or multi thread request, all failed

Hi, what is log statement ?

Just send you my test case, please give a try.

what's the progress for this issue

Hi, Henry
Sorry for delay. During database creation script I got:
orientdb {db=pharos}> create property event.metric LINK metric
Error: com.orientechnologies.orient.core.exception.OCommandExecutionException: Error on execution of command: sql.create property event.metric LINK metric
Error: java.lang.IllegalArgumentException: No enum constant com.orientechnologies.orient.core.metadata.schema.OType.METRIC
Could you provide fixed script ?

Hi, Henry.
Do you have any update on this issue ?

Hi Andrey, sorry about the late, i updated sql script just remove create
property event.metric. Please give a try. Thanks,
Henry,
2015-07-10 5:14 GMT-07:00 Andrey Lomakin<EMAIL_ADDRESS>
Hi, Henry.
Do you have any update on this issue ?
—
Reply to this email directly or view it on GitHub
https://github.com/orientechnologies/orientdb/issues/4453#issuecomment-120397238
.

Hi, Henry.
Any updates on this issue ?

Hi Andrey, I send u a email yesterday with updated script, have u got it?
Sent from my iPhone

On Jul 13, 2015, at 3:59 AM, Andrey Lomakin<EMAIL_ADDRESS>wrote:
Hi, Henry.
Any updates on this issue ?
—
Reply to this email directly or view it on GitHub.

I send u a email yesterday with updated script, not sure if got it or not. I will send it again anyway.

Fixed.

Is this fix going to get back ported to 2.0x?

that is 2.0.x we backported it to 2.1 and I assure you that design of 2.1 is more robust than 2.0. So I suggest to at least test your application on 2.1 version.


Can't make commit in lazygit integrated to AstroNvim.

I start to make commit using 'c' key.
I fill commit summary and message and then... I just can't understand how to finish editing and do actual commit.
Sorry for such newbie question...

What happens when you make a commit using c? I don't use AstroNvim, so not exactly sure what is going on.

Commit dialog appears with two fields: summary and message. I could enter text and switch field with TAB, but I don't get how to finish test edition and perform actual commit.

What happens when you press "enter" while your cursor is in the commit message section?

iirc, enters new line (when I'm in message body). I don’t remember what happens when I'm message title.

I believe while being in message title and pressing "enter" should create the commit. And then by pressing P (shift + p) this commit should be pushed to the origin.

I'm also experiencing this issue, not running in Astrovim though, just in a more vanilla setup of nvim.
The workaround I have is hitting "tab" once I've written the commit body, which takes my. cursor back to the title and then I hit ENTER. This commits the change. It's lame but it works.


Revised translation-jp.json to be consistent with Google Docs Extension translation

I am sorry to bother you.
At last I understand there are 2 files to translate. I should have read your advices on Github.

please ignore the weblate version, which I made it to 100% just before I find teh second file.


Identifiers linking ro-crates inside a repository

From a conversation with the UTS folks including @ptsefton, Mike Lynch and others
Say I create two ro-crates on my desktop: A and B. A is a collection with identifier "A" and B is an item with identifier "A/B" (this naming scheme is from PARADISEC). B is an item within the collection A.
When I create these crates I can stamp the identifiers as "/paradisec.org.au/A" and "/paradisec.org.au/A/B" (the domain name prefix will be explained later) and I can link the item (B) in the collection (A) by using "hasMember" and the collection (A) to the item (B) via "memberOf". Here's the two crates (cut down for clarity).
{
  @graph: [{
    @type: 'Dataset',
    @id: '/paradisec.org.au/A',
    hasMember: [{ @id: '/paradisec.org.au/A/B' }]
  }]
}

{
  @graph: [{
    @type: 'Dataset',
    @id: '/paradisec.org.au/A/B',
    memberOf: [{ @id: '/paradisec.org.au/A' }]
  }]
}

At the time of creation I know that they need to be linked but I don't know exactly where they will be hosted so I can't use a URI to link them: e.g. https://{some.place.com}/paradisec.org.au/A - some.place.com is unknown at the moment of creation.
Although not relevant in the ro-crate layer, stamping the ID's with a domain prefix is akin to how DOI's are minted. Provided a user creating the crate ensures their id's are unique within their namespace, their crates can live in repositories with crates from other communities without resulting in ID clashes. This is essential when hosting on an OCFL filesystem as the ID is used to create the path for the OCFL object.
This issue is about considering this problem. What is the best way to define these references between crates?
Option 1: continue as defined above
The current PARADISEC RO-crate / OCFL demonstrator works with links in this way. When the portal encounters one of these hasMember / memberOf id's it knows how to transform that into a local OCFL path and load the object. So, it assumes that it's relative to the current host even though they're not valid id's and they imply a path within the crate rather than a link to some other crate in the same hosting service.
However, by doing this I can move those two crates to another system and the interlinking will continue to work provided that the target system makes the same assumption.
Option 2: rewrite ID's on the way into / out of the repository
A solution could be to rewrite these id's as the crates get ingested into a repo. They could be rewritten to the name of the repo so that /paradisec.org.au/A becomes https://{some.host.com}/paradisec.org.au/A. But rewriting can fail and it also needs to be done as the crates move around repositories during their lifetime.
Option 3: invent a scheme - use ARCP - but it needs work, maybe...
The UTS folks went down the path of something like _:local-id:ocfl-object:/paradisec.org.au/A which is valid JSON-LD, and identifies that it's a local identifier in an ocfl object. However, it's not a standard so we talked about using ARCP identifiers. The one that stood out was arcp://name,paradisec.org.au/A but this has some problems:

it still looks like a file ref
it suggests a local repository url but doesn't make it explicit

So now we come to inventing things...
Ideally, I would encode that fact that this reference is a URL that is to be resolved in the current host in a way that is unambiguous. So, if we mash the UTS approach with ARCP we might birth something like:
arcp://host:local,url:/paradisec.org.au/A
Requirements of any solution
Guide rails are always good to have.... :-)

Encodes in link that it should be resolved within the hosting system
Doesn't require prior knowledge of the hosting system when creating
Doesn't involve rewriting into / out of the hosting system as the object moves around during its lifetime
Doesn't hash the bits so they're incomprehensible - otherwise we'll need some other property in the crate that allows the user to identify the id as 'A', '/A/B'

https://s11.no/2018/arcp.html#uuid-based


Allow reporting on quarantined packages

The PyPI JSON API returns a 404 for quarantined projects (source here), which makes the _validate_pypi check fail with a 404
https://github.com/vipyrsec/dragonfly-mainframe/blob/14edb7d5a294d0a03f3da316b4889007316e3d85/src/mainframe/endpoints/report.py#L141-L149
However, the HTTP endpoint will still return a 200, and we still want to be able to report quarantined packages to help PyPI administrators triage quarantined packages faster. I propose that if the JSON endpoint returns a 404, we then make a GET request to the HTTP endpoint to see if we get a 200. If we do, then the package is quarantined and the _validate_pypi check should pass, allowing a report.

Could we just only request the HTML page?

Didn't think of that, yes that sounds better


update link to 'The Deprecation HTTP Header' internet draft

Please check if the PR https://github.com/zalando/restful-api-guidelines/pull/622 introduces changes which are relevant to the Zally project.

Zally not effected.

Zally not effected.


Help wanted: solution for secp256k1 being disabled

With the upgrade of the jdk version, the secp256k1 curve has been gradually disabled. Are there plans to use other curves to replace secp256k1?

If your JDK/JRE version later than 8u300.

Path to '{JRE_HOME}/jre/lib/security/';
Open 'java.security' with text editor;
Find the key word 'jdk.disabled.namedCurves =' and where you can find 'secp256k1';
Remove 'secp256k1' from the config value list.

If your JDK/JRE version earlier than 8u300, maybe not.

A simpler solution: Add JVM start options -Djdk.tls.namedGroups="secp256k1"

Or, add JVM start options: -Djdk.tls.namedGroups="secp256k1

Reference: https://www.java.com/zh-CN/download/help/release_changes.html

jdk release note can reference to https://www.oracle.com/java/technologies/javase/jdk-relnotes-index.html

can reference to #536


Get an error without clear message when creating a rule

[NOTE]: # ( ^^ Provide a general summary of the issue in the title above. ^^ )
Description
This is the first time we try to create a rule to aggregate a custom event. After clicking "Create Rule", we see an error message without clear details.
Steps to Reproduce
Expected Behaviour
Relevant Logs / Console output
Your Environment

NR1 CLI version used:
Browser name and version:
Operating System and version:

Additional context

Prestion Millett brought this to my attention so I'm going to see if the DevEx team hero can take a look at this problem.

Prestion Millett brought this to my attention so I'm going to see if the DevEx team hero can take a look at this problem.

Looks like this was related to permissions: It seems to be related to permission. I can navigate and follow through the steps to configure the aggregation rule, and it only fails at the last creation step - [9:57 AM] {“data”:{“eventsToMetricsCreateRule”:{“__typename”:“EventsToMetricsCreateRuleResult”,“failures”:[{“__typename”:“EventsToMetricsCreateRuleFailure”,“errors”:[{“__typename”:“EventsToMetricsError”,“description”:“user not authorized”,“reason”:“USER_NOT_AUTHORIZED”}],“submitted”:{“__typename”:“EventsToMetricsCreateRuleSubmission”,“accountId”:2991421,“name”:“JmeterEvent LatencyAverage summary”,“nrql”:“FROM JmeterEvent SELECT summary(LatencyAverage) AS ‘JmeterEvent.LatencyAverage.summary’  FACET RequestName, TestName“}}],“successes”:[]}}} . We should look into providing a visual warning to users who do not have permissions to create rules.

Looks like this was related to permissions: It seems to be related to permission. I can navigate and follow through the steps to configure the aggregation rule, and it only fails at the last creation step - [9:57 AM] {“data”:{“eventsToMetricsCreateRule”:{“__typename”:“EventsToMetricsCreateRuleResult”,“failures”:[{“__typename”:“EventsToMetricsCreateRuleFailure”,“errors”:[{“__typename”:“EventsToMetricsError”,“description”:“user not authorized”,“reason”:“USER_NOT_AUTHORIZED”}],“submitted”:{“__typename”:“EventsToMetricsCreateRuleSubmission”,“accountId”:2991421,“name”:“JmeterEvent LatencyAverage summary”,“nrql”:“FROM JmeterEvent SELECT summary(LatencyAverage) AS ‘JmeterEvent.LatencyAverage.summary’  FACET RequestName, TestName“}}],“successes”:[]}}} . We should look into providing a visual warning to users who do not have permissions to create rules.


Use On-demand capacity rather than provisioned.

thanks for submitting the feature request for incorporating ddb on-demand capacity. we have added it to our backlog items. the changes in PR looks good, we will try to address it with next update.

updated with v1.3.1


Fail to open config.pb.txt when running yolo_camera_detector_test

System information
**OS Platform and Distribution **: Host: Linux Ubuntu 16.04/Docker: 14.04
**Apollo installed from **: docker
**Apollo version **: 3.0

I am trying to run the yolo_camera_detector_test in module/perception/obstacle/camera/detector/yolo_camera_detector after building it with Bazel to see my own image result, but I got the following error:
_[NVBLAS] NVBLAS_CONFIG_FILE environment variable is NOT set : relying on default config filename 'nvblas.conf'
[NVBLAS] Cannot open default config file 'nvblas.conf'
[NVBLAS] Config parsed
[NVBLAS] CPU Blas library need to be provided
Running main() from gmock_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from YoloCameraDetectorTest
[ RUN      ] YoloCameraDetectorTest.model_init_test
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0907 15:41:44.330113  5446 file.h:97] Failed to open file modules/perception/model/camera/yolo_camera_detector_config.pb.txt in text mode.
E0907 15:41:44.330178  5446 file.h:140] Failed to open file modules/perception/model/camera/yolo_camera_detector_config.pb.txt in binary mode.
F0907 15:41:44.330260  5446 yolo_camera_detector.cc:41] Check failed: GetProtoFromFile(FLAGS_yolo_camera_detector_config, &config_)
*** Check failure stack trace: ***
Aborted (core dumped)_

It looks like there is problem with the yolo_camera_detector_config.pb.txt, but I checked that file and it
seems normal. Could somebody help me with this issue? Or if anyone know a better way to run the perception module with my own data, please share with me. Thanks!

You could set the path of nvblas.conf by using export NVBLAS_CONFIG_FILE=/usr/local/cuda.
You could not find the file in /usr/local/cuda. You may need to create one first.

Hi muleisheng, thanks for your reply. I created the nvblas.conf, now the first 3 lines of warning became:
_[NVBLAS] NVBLAS_CONFIG_FILE environment variable is set to '/usr/local/cuda'
[NVBLAS] Config parsed
[NVBLAS] CPU Blas library need to be provided_

I put
NVBLAS_CPU_BLAS_LIB  /usr/lib/libopenblas.so

in the nvblas.conf, but it still says 'CPU Blas library need to be provided', and the rest of the error messages are still the same. Could you share with me if you have any idea what is going on? Thank you very much!

I am closing this as the rest of the problem seems to because the file path is wrong

@sopXx, hey, how do you fix this problem, I also meet the same problem as you, can you share your method here?

@xinwf export NVBLAS_CONFIG_FILE=/usr/local/cuda

@HouYuu, I have done this, it seems that works right, but another problem CPU Blas library need to be provided still exists, how to solve it?

I have never seen the problem like that，you should post your detailed  operation steps and system version information
| |
houyu
Samsung Electronics  -
Software Engineer
18189118087
<EMAIL_ADDRESS>|
Signature is customized by Netease Mail Master
On 11/21/2018 13:53, 辛文飞 wrote:
@HouYuu, I have done this, it seems that works right, but another problem CPU Blas library need to be provided still exists, how to solve it?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

detailed info is at this issue, #6013
if I set the NVBLAS_CONFIG_FILE=/usr/local/cuda，then the log is config is parsed, but the CPU BLAS library problem still exists, if I set the set the NVBLAS_CONFIG_FILE=/usr/local/cuda/nvblas.conf, it won't generate such log, no tips about the CPU, it looks like everything is ok, but when I search this problem in issues, they all answer this question with the reply setting the NVBLAS_CONFIG_FILE=/usr/local/cuda, so, I set it with this common setting, keeping consistent with all of you, I don't konw which is right.

@xinwf Have you tried to use the released version of Apollo 2.5 or 3.0?

@HouYuu, no, I always use develop version from 2.5 to 3.0


update egress policy FAQ

Description

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @ahrkrak ! @bcreane can you take a look?

@ahrkrak could you perhaps make this change also in master? That would ensure that it persists across future releases. This will require you to squash your commits, however. So, if you prefer, you could make a separate PR for this purpose and avoid the squashing.


fix: export outOfContext and storage

Export the relevant object so that can reuse nestjs-pino in non-http environments.
Need to reuse outOfContext to keep the options set in LoggerModule.forRoot(options).
import { outOfContext, storage as loggerStorage, Store as LoggerStore } from 'nestjs-pino';
import type { Logger as PinoLogger } from 'pino';

...

@On({ event: 'something' })
public async webSocketEvent(): Promise<void> {
  const store = new LoggerStore((<PinoLogger>outOfContext).child({
    foo: 'bar',
    user: 'test',
  }));

  await loggerStorage.run(store, async () => {
    // logic...
    this.logger.log('hello'); // {"foo":"bar","user":"test","msg":"hello" ... }
  });
}

This is not a solution to this issue(https://github.com/iamolegga/nestjs-pino/issues/803), but it can be used as a temporary measure.

I would prefer better developer experience instead of exporting internals. Now we will export it, after we should deprecate it and release new major version with better solution. Moreover there are no docs and tests for such usage.
Sorry, but I cannot merge it in the current state.


Create Notification Publishers

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

New or affected resource(s)
Please provide a list of the new and/or affected resources/data sources, for example:

pingfederate_notification_publisher

Is your feature request related to a problem? Please describe.
I'm unable to create notification publishers which are needed by other resources such as notification publishers settings
Describe the solution you'd like
The ability to create a notification publisher using terraform
Describe alternatives you've considered
API calls to create publishers.
Potential Terraform configuration
References

@colinhuckstep, thank you for the issue submission! The notification_publisher resource currently has a Pull Request , and is approved. This will be included in the provider very soon!

Great, thanks for the update. I'll keep an eye out for it.

This resource has been added in the new v0.14.0 release - https://registry.terraform.io/providers/pingidentity/pingfederate/latest


fix: change message to "Hello, cold world!"

This Pull Request changes the message from "Hello, world!" to "Hello, cold world!"

Changes approved by BiancaTulio.


update doc for local-up-openyurt

What type of PR is this?
/kind documentation
What this PR does / why we need it:
Add tutorial for local_up_openyurt.sh and update the e2e tests instructions.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

/lgtm
/approve


Vercel Integration

Hey! Prospective user of both flags and analytics here, what you've built looks super useful.
If you haven't already considered it, I think it would be a nice experience to create a Vercel Integration to streamline the onboarding process, particularly around setting environment variables. I'd imagine there would be exposure benefits to being on the Vercel integration marketplace as well.

Sounds good! Excited for this stuff to be native to Vercel someday ;)


Database could not be revived

Hello,
I'm trying to revive the database using a (slightly modified) manifest generated by vdb-gen:
apiVersion: vertica.com/v1beta1
kind: VerticaDB
metadata:
  name: xxx
spec:
  autoRestartVertica: true
  communal:
    credentialSecret: gcs-creds
    endpoint: ""
    path: gs://xxx/
    region: US-WEST1
  dbName: xxx
  image: vertica/vertica-k8s:11.0.2-0
  initPolicy: Revive
  kSafety: "1"
  licenseSecret: vertica-license
  local:
    catalogPath: /var/lib/verticadb/catalog
    dataPath: /var/lib
    depotPath: /var/lib/verticadb/depot
    requestSize: 65536Mi
    storageClass: ssd
  reviveOrder:
  - podCount: 6
    subclusterIndex: 0
  shardCount: 6
  subclusters:
  - affinity: {}
    isPrimary: true
    name: default
    resources:
      requests:
        cpu: 2
        memory: 8Gi
      limits:
        cpu: 4
        memory: 16Gi
    size: 6
    serviceType: LoadBalancer
    serviceAnnotations: {}
  superuserPasswordSecret: xxx-su-passwd
  temporarySubclusterRouting:
    template:
      affinity: {}
      isPrimary: false
      name: ""
      resources: {}
      size: 0
  upgradePolicy: ""

I see 6 pods created after I apply it, but the database revive command fails with the following error (the below logs come from v_xxx_node0001):
Attempting to retrieve file: [gs://xxx/metadata/xxx/cluster_config.json]

Validated 6-node database xxx defined at communal storage gs://xxx/.
Cluster lease has expired.
Preparation succeeded all hosts

Calculated necessary addresses for all nodes.
Starting to bootstrap nodes. Please wait, databases with a large catalog may take a while to initialize.
>>Calling bootstrap on node v_xxx_node0002 (<IP_ADDRESS>)
>>Calling bootstrap on node v_xxx_node0003 (<IP_ADDRESS>)
>>Calling bootstrap on node v_xxx_node0004 (<IP_ADDRESS>)
>>Calling bootstrap on node v_xxx_node0005 (<IP_ADDRESS>)
>>Calling bootstrap on node v_xxx_node0006 (<IP_ADDRESS>)

Database could not be revived.
Error: Not enough primary nodes to revive the database.
Successful primary nodes: None

Right before this, I see a correct revive_db command with 6 addresses listed after the --hosts option:
sudo --preserve-env su dbadmin -- /opt/vertica/bin/admintools -t revive_db \
    --hosts=<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS> \
    --communal-storage-location=gs://xxx/ \
    --communal-storage-params=/home/dbadmin/auth_parms.conf \
    --database xxx

Is this reproducible: yes
Operator version: 1.9.0
Database version: 11.0.2-18

Usually what the 'Not enough primary nodes' error means is that we hit problems trying to revive at a majority of primary nodes (all in your case). It's not an error saying there aren't enough primary nodes. We have validated that there are 6 primaries and the CR has 6 primaries.
There might be an indication of what is going on by looking at /opt/vertica/log/adminTools.log. Can you provide your adminTools.log? Maybe start with the pod that is driving the revive.

Thank you, the problem was that image version vertica-k8s:11.0.2-0 was older than the database version 11.0.2-18. Since there is no image for 11.0.2-18, I built my own using the Dockerfile in docker-vertica, but now I'm facing a different issue: the operator is trying to run a command
sh -c cat > /opt/vertica/config/accept_eula.py && /opt/vertica/oss/python3/bin/python3 /opt/vertica/config/accept_eula.py

but it fails with
sh: 1: cannot create /opt/vertica/config/accept_eula.py: Permission denied

 /opt/vertica/config is mounted as a volume owned by root. In the 11.0.2-0 image, there were commands in the entrypoint script chowning the mounted directories to dbadmin, but the new image doesn't have them. Is there a way to fix that?

I managed to move a bit further by building the image from commit e7208d8 (tag v1.9.0), now it's failing with
Attempting to retrieve file: [gs://xxx/metadata/xxx/cluster_config.json]

Validated 6-node database xxx defined at communal storage gs://xxx/.
Cluster lease has expired.

Database could not be revived.
Error: Database xxx already exists.
If you want to force a new revive from communal storage on this cluster, please clean up the existing database first. You can drop the existing database by running: [admintools -t drop_db -d xxx]

/opt/vertica/config is mounted as a volume owned by root. In the 11.0.2-0 image, there were commands in the entrypoint script chowning the mounted directories to dbadmin, but the new image doesn't have them.

Yes, this was changed to use the fsGroup property in the pod's security context. It sounds like you got past this part, but could you confirm that your vertica pods have this set?
spec:
  ...
  securityContext:
    fsGroup: 5000

If it doesn't have that, you may need to upgrade the operator to the latest version (1.9.0).

re: Error: Database xxx already exists.

This happens when the database is already in /opt/vertica/config/admintools.conf. If you haven't already, can you try deleting the VerticaDB and the PVCs. Then recreating the VerticaDB again?

Thank you for your response!
I already have the latest version of the operator
➜  ~ helm ls -n verticadb
NAME     	NAMESPACE	REVISION	UPDATED                             	STATUS  	CHART                   	APP VERSION
verticadb	verticadb	1       	2023-02-10 14:11:27.333138 -0800 PST	deployed	verticadb-operator-1.9.0

I checked - pods don't have fsGroup set in their security context. The pod security context is empty, and the container security context is
    securityContext:
      capabilities:
        add:
        - SYS_CHROOT
        - AUDIT_WRITE
        - SYS_PTRACE

I tried adding fsGroup to the CR spec, but got the following error when attempted to apply it:
error: error validating "xxx-cr.yaml": error validating data: ValidationError(VerticaDB.spec.securityContext): unknown field "fsGroup" in com.vertica.v1beta1.VerticaDB.spec.securityContext; if you choose to ignore these errors, turn validation off with --validate=false

The database is indeed already in /opt/vertica/config/admintools.conf. Yes, I deleted the vdb resource and all the PVCs before redeploying, and also made sure the PVs were all gone before I reapplied the manifest, so the PDs are pristine, but I am still getting the same error.

It sounds like your CRD is old. Did you upgrade from a prior release to get to 1.9.0 of the operator? If so, did you apply the CRDs manually? helm won't ever overwrite CRDs. So, that steps needs to be done manually.
If you haven't already, the steps are:
kubectl apply -f https://github.com/vertica/vertica-kubernetes/releases/latest/download/verticadbs.vertica.com-crd.yaml
kubectl apply -f https://github.com/vertica/vertica-kubernetes/releases/latest/download/verticaautoscalers.vertica.com-crd.yaml

No I installed it from scratch on 2023-02-10, and as my previous comment shows, it's the 1st revision.
I applied the above two manifests - unfortunately, it didn't help, I'm still getting both errors (unknown field "fsGroup" and  Database xxx already exists).

spec.securityContext is for the container level. Where as fsGroup is for the pod level. If you have recreated the image from the 1.9.0 tag, that should be enough to get around the permission issues and so you don't need to set fsGroup.
As for the Error: Database xxx already exists. error, can you send me the vertica operator log?
kubectl logs --selector control-plane=controller-manager --container manager --tail=-1
If you don't want to share it in this issue, you can send it to me via email at<EMAIL_ADDRESS>

Thank you, I've sent you the log via email.

Thanks @unix-way. I think I know what's happening in your environment. I'll try to repro in house and work on a fix.

If you want to try this fix out before the next version of the operator. You can follow these instructions to use the helm chart and image created by the CI:
Go to this download page and download the 'release-artifact' in the Artifacts section:
https://github.com/vertica/vertica-kubernetes/actions/runs/4216047644
In a shell where the release-artifacts.zip was downloaded:
$ unzip release-artifacts.zip
$ kubectl apply -f verticaautoscalers.vertica.com-crd.yaml
$ kubectl apply -f verticadbs.vertica.com-crd.yaml
$ helm install vdb-op verticadb-operator-1.10.0.tgz --set image.repo=ghcr.io,image.name=vertica/verticadb-operator:e84715f74d1e6dc115e613d023c

Thank you! I applied the manifests from release-artifacts.zip and upgraded the helm chart as described above. Now the DB revive is failing with a different error:
2023-02-21T20:18:49.220Z	INFO	controllers.VerticaDB	Event logging	{"eventtype": "Warning", "reason": "ReviveDBFailed", "message": "path '/var/lib/verticadb/ssd' is not a valid vertica path"}
2023-02-21T20:18:49.220Z	INFO	controllers.VerticaDB	aborting reconcile of VerticaDB	{"verticadb": "verticadb/xxx", "result": {"Requeue":true,"RequeueAfter":0}, "err": null}
2023-02-21T20:18:49.220Z	DEBUG	events	path '/var/lib/verticadb/ssd' is not a valid vertica path	{"type": "Warning", "object": {"kind":"VerticaDB","namespace":"verticadb","name":"xxx","uid":"1050b988-13f7-411f-8f85-0687f487409b","apiVersion":"vertica.com/v1beta1","resourceVersion":"364865325"}, "reason": "ReviveDBFailed"}

Now it looks like if there is some data path that does not follow the regex pattern defined in function extractPathPrefix, it's not accepted as a valid data path.
I have the following storage locations defined in the DB, so /var/lib/verticadb/ssd is a TEMP location.
dbadmin=> SELECT NODE_NAME, LOCATION_PATH, LOCATION_USAGE FROM STORAGE_LOCATIONS ORDER BY 1, 2;
     NODE_NAME     |                      LOCATION_PATH                      | LOCATION_USAGE
-------------------+---------------------------------------------------------+----------------
 v_xxx_node0001    | /var/lib/verticadb/depot/xxx/v_xxx_node0001_depot       | DEPOT
 v_xxx_node0001    | /var/lib/verticadb/ssd                                  | DATA,TEMP
 v_xxx_node0002    | /var/lib/verticadb/depot/xxx/v_xxx_node0002_depot       | DEPOT
 v_xxx_node0002    | /var/lib/verticadb/ssd                                  | DATA,TEMP
 v_xxx_node0003    | /var/lib/verticadb/depot/xxx/v_xxx_node0003_depot       | DEPOT
 v_xxx_node0003    | /var/lib/verticadb/ssd                                  | DATA,TEMP
 v_xxx_node0004    | /var/lib/verticadb/depot/xxx/v_xxx_node0004_depot       | DEPOT
 v_xxx_node0004    | /var/lib/verticadb/ssd                                  | DATA,TEMP
 v_xxx_node0005    | /var/lib/verticadb/depot/xxx/v_xxx_node0005_depot       | DEPOT
 v_xxx_node0005    | /var/lib/verticadb/ssd                                  | DATA,TEMP
 v_xxx_node0006    | /var/lib/verticadb/depot/xxx/v_xxx_node0006_depot       | DEPOT
 v_xxx_node0006    | /var/lib/verticadb/ssd                                  | DATA,TEMP
                   | gs://xxx/                                               | DATA
(13 rows)

I could probably retire it before migrating the database, but this location is legit and we plan to continue using some form of a TEMP location in the Kubernetes setup.

@spilchen Can you take a look please? Not sure if the issue needs to be reopened.

As a reminder, the paths configuration in my CR manifest is as follows, maybe I need to change something in it:
    catalogPath: /var/lib/verticadb/catalog
    dataPath: /var/lib
    depotPath: /var/lib/verticadb/depot

Thanks for sharing your storage paths. That was a case I hadn't considered. I'll work on fixing this.

@unix-way if you want to try out this latest fix.
Go to this download page and download the 'release-artifact' in the Artifacts section:
https://github.com/vertica/vertica-kubernetes/actions/runs/4255009543
In a shell where the release-artifacts.zip was downloaded:
$ unzip release-artifacts.zip
$ kubectl apply -f verticaautoscalers.vertica.com-crd.yaml
$ kubectl apply -f verticadbs.vertica.com-crd.yaml
$ helm install vdb-op verticadb-operator-1.10.0.tgz --set image.repo=ghcr.io,image.name=vertica/verticadb-operator:0f4e03dfc3d540612d02abd7357f838a694a4e73

Set dataPath to /var/lib/verticadb and it finally worked! Thank you so much!

Thanks for confirming!


ISSUE/QUESTION - jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'components/schemas/head'

I am trying to validate the JSON with the expected Open API JSON schema(generated by FastAPI/Pydantic). However, for the $ref nodes in the schema, I receive an error message - "Unresolvable JSON pointer: %r" % fragment and jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'components/schemas/head'

Hi. There isn't nearly enough information to help here, but also please try to use the Slack channel for questions, and the issue tracker for actual bugs (after first checking to see if any of the existing ones sounds like your issue).


Add sourcegraph cheatsheets to documentation/techbits

Sourcegraph is an awesome companion which lets you browse your favourite repos without having to clone them to your local machine with ease and IDE super powers :)
Write a cheat-sheet to help openebs newbies better use this powerful tool.
It'll be a markdown formatted file in here

You can install sourcegraph extension on your chromium-alike browser.


Undefined provider in the success callback

Issue
In the success callback the provider is undefined.
Reproduction
sst 3.4.3
const app = authorizer({
  subjects,
  providers: {
    code: CodeAdapter<{ email: string }>(
      CodeUI({
        sendCode: async (claims, code) => {
          console.log('Code adapter sendCode:', code);
          return Promise.resolve();
        }
      })
    )
  },
  success: async (ctx, value) => {
    // Expect value.provider to be "code" but got undefined
    console.log('Success handler value:', value.provider);

    return ctx.subject('user', {
      email: 'some-email'
    });
  }
});

export const handler = handle(app);

Auth flow is triggered by accessing to:
https://auth.domain.com/authorize?redirect_uri=http://localhost:3000&response_type=token&client_id=local
 console.log('Success handler value:', value.provider); will log un undefined value

i just tried recreating this and i can't - you can see how we're using it at terminal https://github.com/terminaldotshop/terminal/blob/dev/packages/functions/src/auth2.ts#L111
and try it
https://auth.terminal.shop/authorize?redirect_uri=https://localhost&response_type=code&client_id=xxx

I found the cause.
hono > 4.0.0 is required have the provider defined.
With version 3.0.0 provider is undefined.


add Smoke grenade and tear gas grenade

About the PR

What should have been a long time ago, a smoke grenade for the syndicate and a tear gas grenade for sec.
Media

https://github.com/space-wizards/space-station-14/assets/60344369/3be6e814-3f40-45d9-9506-87cd64d0a6e1

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl: VigersRay

add: Added smoke and tear grenade.

this should be using a chem payload like the rest of the grenades. its not really an issue if people can replace whats in it

And then what's the point of making a copy of the modular grenade but with a different name and sprite?

because these would have prefilled contents?

Wouldn't it be easier then to just put a regular modular grenade in the uplink and the sb belt that will be filled?
Wow free modular grenade in uplink real.

Jokes aside, you cant crack open a frag grenade for example like you would with a modular grenade from sci. Uplink sells traitor items for a reason. Another difference is that your smoke grenade doesnt beep unlike a modular grenade.
Just don't allow extracting the payload from the grenade and its okay to use ChemicalPayloadComponent.

since the crowbaring bit is on modular grenade's main component if you dont have that you wont have it do anything

Okay then, there is a problem in that if I use a payload, it is difficult or impossible for me to regulate the life and smoke density.  How do I even get the right amount of elements to create a thick smoke screen that will last 30 seconds?  Here I am without a clue.  And why can FleshOnTrigger exist but SmokeOnTrigger can't?

Since taking the payload out of the smoke grenade would be impossible I believe you can go goofy with the mixture (or make an exclusive chemical that you can finetune), alternatively make a "smoke payload" similarly to the "explosive payload" we already have in-game

this should be using a chem payload like the rest of the grenades. its not really an issue if people can replace whats in it

So, after a little thought, I realized that using a chemical payload is not an option since no chemical reaction in the game produces a thick enough smoke that can exist for more than 30 seconds. The only option would be to create something like FlashPayload but SmokePayload using SmokeOnTrigger

Re-open when you come back to it.

Re-open when you come back to it.

the fact is that I still think that the SmokeOnTriggerComponent should exist, in particular it will be needed for blob spores, but I think I will open a new pr where there will only be changes in the smoke code with support for colored smoke, and then I will open a separate pr for the smoke grenade. What about the tear grenade...


⚠ Bofrost.de has degraded performance

In fb5008d, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 735 ms

Resolved: Bofrost.de performance has improved in 37e6a78 after 6 minutes.


Fix GraphQL example

https://twitter.com/joshleong/status/1256019423489482752

📝Changed routes:

/

(view full size)

Commit 733dccbcbca53a0dc5dfa111050e8d0dbc84e047 (https://swr-site-git-shuding-patch-1.zeit.now.sh).


Model objects do not currently notify DataSource of changes within setters.

Not sure how to address this, so this is an RFC.
Our DataSources have a public void notifyDataChanged() which should be called within model objects (Documents specifically).  The idea is that when a Document changes, the DataSource should know so it can push to the server, or cache the dirty ones to a file, or whatever.
Having the existing notifyDataChanged() is one way to do it, but we would need to change Document to have a DataSource attribute, and subclasses of Document would need to call the notify method within each setter.
Alternatively, Document could inherit from Observable, and call updateObservers() within each setter.  DataSource would implement the Observer interface and add itself as an observer to each Document it returns.  I prefer the latter, since Document would not be coupled to DataSource.  Here's a usage example:
public class DummyDataSource extends Observable<DummyDataSource> implements DataSource, Observer {

	private Collection<Claim> claims;
	private Collection<User> users;
	private Collection<Item> items;
	private Collection<Tag> tags;

	@Override
	public void addClaim(User user, ResultCallback<Claim> callback) {
		Claim claim = new Claim(UUID.randomUUID());

		// TODO associate claim with user

		claim.addObserver(this);  // <---- here is the magic line

		callback.onResult(claim);
	}

I think we already ruled out observer pattern for model objects like the Documents.
The notifyDataChanged() doesn't have to go in the setters for those model objects. There is going to have to be some method in the activities that takes the updated text entry fields and puts them into the model object. Since the activity will already have a reference to DataSource, the activity can call notifyDataChanged() after it makes a change to a model object.

My only concern with notifyDataChanged() being called from the Activity is that it moves away from active-model MVC, which seems to be what we're expected to use.

@TheLastBanana you are right about that. Do you have an alternative in mind or do you prefer the observer pattern that @kdbanman suggested?

Ideally, I would probably do what you suggested. But given the course expectations, I think something like @kdbanman's observer pattern is our best bet.

If we are confined within the active mvc model and cannot deviate then I think the observer pattern is the way to go as well.

I think we already ruled out observer pattern for model objects like the Documents.

Sorry I misunderstood that - I thought we ruled out activities/views observing model objects, not the DataSource itself.

Since the activity will already have a reference to DataSource, the activity can call notifyDataChanged() after it makes a change to a model object.

The active model problem is my concern.  I just asked for clarification on the class forum about whether or not we're confined to active model MVC.
Something to watch out for with the observer pattern I described is that the DataSource doesn't know anything about non-Document model objects.  Destination, ApproverComment, and Receipt are the ones I'm talking about.  They're not just simple enums, and they are not CRUDded by DataSource.  Hence, the DataSource has no place to add itself as an observer to them.
For example, if we were to call setPhoto(...) on a Receipt object of an Item, the DataSource wouldn't get notified because the Item itself isn't mutated.  The way I suggest dealing with that is not having setters on the non-Document objects.  To change them, reconstruct them and use the setters on the Document objects.  Then every change is observable by the DataSource.

We could call addObserver() within the Document objects, too.  It'd look like this:
class Item extends Document {    // Document extends Observable
  // ...

  @Override
  public void addObserver(Observer obs) {
    super.addObserver(obs);

    // need to override addObserver and add obs to each non-Document attribute
    if (this.receipt != null) this.receipt.addObserver(obs);
  }

  public void setReceipt(Receipt receipt) {
    if (this.receipt != receipt) {
      this.receipt = receipt;

      // also need to add observers to any new non-Document attribute
      this.receipt.addObservers(this.observers);
    }
  }

  // ...
}

It comes down to whether or not we think we'll need to mutate objects that are contained within Documents. At the moment, all of those objects could easily be re-created if we're changing them. Do we anticipate that changing? If we do, we should probably just add observers to them.

At the moment, all of those objects could easily be re-created if we're changing them. Do we anticipate that changing?

Where do you believe the threshold lies for "too complex to recreate instead of using setters"?

I'm going to implement the "only Documents are observers, recreate every non-Document" option.
From a maintenance perspective, if we change our minds later need to switch, none of our code will break by adding observers.  But if we start with observers everywhere, then things could break if we move to a recreation pattern.
Besides, it's a less complex change from what we have right now :-)


More positioning options

Should support:

Vertical bars, ie ability to position on the left and right.

This will require any Orientation::Horizontal boxes in widgets to automatically switch to Orientation::Vertical

Ability to disable anchoring to edges, ie allowing for centered docks

Have started work on this. Currently blocked by some issue with the popup system.


Add payment method on confirmation

When you get the confirmation page you receive the shipping and Payer address. But for Payer it would be nice if the payment method would also show. If they add a PO# we can decide to show that or not, but the payment method could show like that under the address:
Payment method: Credit Card

Not a problem. I will add that


Prepare for JPMS - rename packages

Preparation for modularization in #10157. This is merely a refactoring PR that renames various Java packages and migrates some projects from Scala to Java.
Pull Request Description
The renames are necessary to avoid problems with conflicting package names of JPMS modules in the future.
Important Notes

Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:

[x] The documentation has been updated, if necessary.
[x] Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
[x] All code follows the
Scala,
Java,
TypeScript,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
[x] Unit tests have been written where possible.

There is native image compilation error.


any doc?

Hey @caicaicai
I'm going to close this because you didn't provide any information about what you're looking for.
If you are still in need of help, could you please re-make the issue over at the Platform repo and explain what you're having issues with.
Thanks


Support Autocomplete Brackets for String Interpolation

This PR is to implement 54759
Changes

Added an attribute to autoClosingPairs hash called "onlyIn". It's an array that takes 3 values: "string", "comment", or "regex", and ensures that the autoClosingPair ONLY works within that scope. (Necessary for getting string interpolation to work).
Added another attribute to autoClosingPairs hash called "cursorPosition". Allows the user to input a position where they want the cursor to be moved to after autocomplete is done. The value must be within the range of 0 and close.length. Else, the cursor will default to the original behavior (cursor located just before the "close" string). This helps to move the cursor inside the brackets after the brackets are autocompleted for interpolation.
Added definitions and functions to other files to ensure that the above attributes are accessible/usable.
Added tests to test the above implementations.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: phaskins sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thank you for the PR. I believe these changes are no longer necessary as it appears this feature works OK for e.g. JavaScript, so Ruby could use the same config options as JavaScript to get the feature working:


Using GoLand, which I understand to use delve under the hood (see GoLand ticket where someone else is hitting this same issue), I attempted to start a debug session for the Trivy project: https://github.com/aquasecurity/trivy, running the program at ./cmd/trivy.
I'm able to reproduce this without GoLand, too, via:
dlv debug ./cmd/trivy/main.go

I'm running on macOS 14.4.1, on an M1 processor.
cc: @knqyf263 in case this interests you!
What did you see happen?
$ dlv debug ./cmd/trivy/main.go
 # command-line-arguments
runtime.textsectionmap: reloc 1 (R_ADDR) to non-macho symbol runtime.text.1 type=1 (STEXT)
runtime.textsectionmap: unsupported obj reloc 1 (R_ADDR)/8 to runtime.text.1
panic: machorelocsect: size mismatch

goroutine 3363 [running]:
cmd/link/internal/ld.machorelocsect(0x14000118000, 0x14000dfee00, 0x1401935b9e0, {0x140227d0000?, 0x14000000002e9bed?, 0x102d54800?})
        cmd/link/internal/ld/macho.go:1256 +0x214
cmd/link/internal/ld.relocSectFn.func1.1()
        cmd/link/internal/ld/asmb.go:203 +0x4c
created by cmd/link/internal/ld.relocSectFn.func1 in goroutine 1
        cmd/link/internal/ld/asmb.go:202 +0x1e4
exit status 1

(Note: I also had to temporarily adjust Trivy's go.mod to use toolchain go1.22.2 so it matched my installed toolchain.)
What did you expect to see?
I expected to be able to start a debugging session successfully, via either dlv or GoLand.
(I can't reproduce this issue using go1.21.9, FWIW.)

@luhring Thanks for raising!
We've also faced the same issue and looked into it. It's even easier to reproduce it.
$ go build -gcflags "all=-N -l" github.com/aquasecurity/trivy/cmd/trivy
# github.com/aquasecurity/trivy/cmd/trivy
runtime.textsectionmap: reloc 1 (R_ADDR) to non-macho symbol runtime.text.1 type=1 (STEXT)
runtime.textsectionmap: unsupported obj reloc 1 (R_ADDR)/8 to runtime.text.1
panic: machorelocsect: size mismatch

goroutine 3226 [running]:
cmd/link/internal/ld.machorelocsect(0x14000118000, 0x14001046700, 0x1401700b140, {0x1402a95c000?, 0x0?, 0x1030007a0?})
        /opt/homebrew/Cellar/go/1.22.1/libexec/src/cmd/link/internal/ld/macho.go:1256 +0x20c
cmd/link/internal/ld.relocSectFn.func1.1()
        /opt/homebrew/Cellar/go/1.22.1/libexec/src/cmd/link/internal/ld/asmb.go:203 +0x4c
created by cmd/link/internal/ld.relocSectFn.func1 in goroutine 1
        /opt/homebrew/Cellar/go/1.22.1/libexec/src/cmd/link/internal/ld/asmb.go:202 +0x1e4

We tried Go 1.22.0, 1.22.1, and 1.22.2, but the same error occurred with all of them. However, CGO_ENABLED=0 works around it.
CGO_ENABLED=0 go build -gcflags "all=-N -l" github.com/aquasecurity/trivy/cmd/trivy

Awesome, thanks for the added context! 🙌

I met the same error when building bytebase by running sh scripts/build_bytebase.sh with CGO_ENABLED=1.
Step 2 - building Bytebase backend...
# command-line-arguments
runtime.textsectionmap: reloc 1 (R_ADDR) to non-macho symbol runtime.text.1 type=1 (STEXT)
runtime.textsectionmap: unsupported obj reloc 1 (R_ADDR)/8 to runtime.text.1
panic: machorelocsect: size mismatch

goroutine 205 [running]:
cmd/link/internal/ld.machorelocsect(0x14000114000, 0x140063ae7e0, 0x1400006c480, {0x14024dd8000?, 0x110000000058630c?, 0x1045d47a0?})
        /opt/homebrew/Cellar/go/1.22.2/libexec/src/cmd/link/internal/ld/macho.go:1256 +0x214
cmd/link/internal/ld.relocSectFn.func1.1()
        /opt/homebrew/Cellar/go/1.22.2/libexec/src/cmd/link/internal/ld/asmb.go:203 +0x4c
created by cmd/link/internal/ld.relocSectFn.func1 in goroutine 1
        /opt/homebrew/Cellar/go/1.22.2/libexec/src/cmd/link/internal/ld/asmb.go:202 +0x1e4

In one of my projects, I was able to narrow down this error happening as soon as I use https://github.com/aws/aws-sdk-go-v2 in go 1.22+.
My workarounds are to exclude the library and all its usages via build flag, or to run the program in a container, which is obviously in Linux VM.
I also investigated for a while and thought maybe I have some old linked library somewhere.
Thank you for creating this issue.

@randall77 would you help take a look? Thanks!

@cherrymui This looks like a linker issue. That failing check is off by 8.

Thanks for reporting. The direct problem is runtime.textsectionmap: reloc 1 (R_ADDR) to non-macho symbol runtime.text.1 type=1 (STEXT), that is, an (8-byte) relocation failed to apply, which causes a size mismatch by 8. The underlying issue is that the text section-split-code path doesn't interact well with the plugin-enabled code path. If the code doesn't (transitively) import the plugin package, it would work fine, but it does. I'll work on a fix.

Thanks for reporting. The direct problem is runtime.textsectionmap: reloc 1 (R_ADDR) to non-macho symbol runtime.text.1 type=1 (STEXT), that is, an (8-byte) relocation failed to apply, which causes a size mismatch by 8. The underlying issue is that the text-section-split code path doesn't interact well with the plugin-enabled code path. If the code doesn't (transitively) import the plugin package, it would work fine, but it does. I'll work on a fix.

Wish I could still send Kudos for your help. Thank you!

Also curious why it happens to Darwin/arm64 only?

Also curious why it happens to Darwin/arm64 only?

Because on ARM64 the C linker has a section size limit, so we need to split text sections. There is no such limit on AMD64. As for other OSes, the ELF linking is just different from Mach-O linking.

@d-bytebase  Based on the error message the problem you're reporting is not related to the one described here. Would you please open another issue? Thanks. Be sure to include details instructions on how to reproduce.

@d-bytebase Based on the error message the problem you're reporting is not related to the one described here. Would you please open another issue? Thanks. Be sure to include details instructions on how to reproduce.

Thanks. I figured it out as it has nothing to do with golang but clang, simply put macos/arm64 is too bad. Golang's linker improvement is exactly solving this kinds of issue, and I found a way out.

@gopherbot please backport this to Go 1.22. This causes build failures on darwin/arm64.


Format JSON benchmark code

Ran cargo fmt on master. This should help Travis CI builds not to fail.

Coverage remained the same at 88.248% when pulling c740d44b281127b8d16a44bd0a534cd43d002b99 on murarth:fmt-json-bench into a1db24b2814df1ccc4af1a5e00317faefb023828 on Geal:master.

thanks


[RFC] Backend System Evolution

Status: Open for comments
Authors: @backstage/maintainers
Need
The new Backend system stems from a couple of needs identified prior to writing this RFC.
Some of the areas covered in this RFC need much more work, but we would like to open up for comments and feedback for this proposal before moving forward.
Simplified Installations
The perhaps most important of these is to make it easy for integrators to install new backend plugins without having to jump around in multiple files to wire up the plugin.
The current process is both time consuming and prone to errors. Ideally the installation should not require more than a few lines of code excluding the configuration itself.
Sane Defaults
It should be easy to stay up to date with new backend plugins when upgrading.
Today, plugins often have their dependencies constructed in the plugin's setup file, which leads to a high risk of breaking changes and manual labour during upgrades when a plugin starts taking additional dependencies or when constructor parameters change.
We need to be able to introduce new default APIs without having to change the existing code in order to consume them.
Extending Plugins
It should be easy for integrators to install extensions for existing plugins in order to gain new functionality.
Plugin developers should be able to provide extension points for other modules to use.
Developer Friendly
The concept of installing, extending and providing extension points should be similar across the system and easy for developers to understand.
The development experience should be easy and quick.
Proposal
The proposal is divided into several sections, as the use cases for integrators and plugin developers are different.
Plugin Usage
In its simplest form new plugins can be installed by adding them to the backend.
import { createBackend } from '@backstage/backend-common';
import { catalogPlugin } from '@backstage/plugin-catalog-backend';
import { catalogGithubModule } from '@backstage/plugin-catalog-github-module';

async function main() {
  const backend = createBackend();
  // Adds the catalogPlugin
  backend.add(catalogPlugin());
  // Installs GitHub org discovery, which adds the entity provider to the catalog
  backend.add(catalogGithubModule.orgDiscovery());
  await backend.start();
}

Plugin authors can expose options that can be used for altering the default setup.
backend.add(catalogPlugin({ disableProcessing: true }));

Plugin Authors
New plugins are created by exporting the result of createBackendPlugin.
createBackendPlugin accepts a register function which takes care of wiring up the plugin to the backend once it has been installed.
The register function is passed a backend environment (env) parameter. The environment can then be used to register the plugin's init function which is called on startup.
The init function uses a dependency injection system similar to the Utility APIs found in the frontend core library. For now we refer to these as "Services" rather than "APIs", but naming is to be determined. This is to help separate the concepts, as they don't function in the exact same way.
Dependencies to the plugin are provided in the deps section by mapping them to a name that then can be referenced in the init function and the corresponding ServiceRef. The backend framework takes care of initializing dependencies prior to calling the init function.
export const examplePlugin = createBackendPlugin({
  id: 'com.my-company.example',
  register(env) {
    env.registerInit({
      deps: {
        router: httpRouterServiceRef,
      },
      async init({ router }) {
        // plugin specific setup code.
        router.use('/hello', async (_req, res) =>
          res.json({ message: 'Hello World' }),
        );
      },
    });
  },
});

Plugins Providing Extension Points
There is a need for plugins to expose extension points which can be extended by other plugins. The software catalog is an example of such a plugin, which today is extended with custom processors and entity providers.
A plugin can register multiple extension points, each of which can be depended on by other plugins through a service reference.
Please note that the ServiceRef for the extension point is imported from the plugin's node package, as we should avoid plugin-to-plugin imports.
// exported from @backstage/plugin-example-hello-world-node

export interface GreetingsExtensionPoint {
  addGreeting(greeting: string): void;
}

export const greetingsExtensionPoint =
  createServiceRef<GreetingsExtensionPoint>({
    id: 'com.my-company.example.greetings',
  });

import { greetingsExtensionPoint } from '@backstage/plugin-example-hello-world-node';

export const examplePlugin = createBackendPlugin({
  id: 'com.my-company.example',
  register(env) {
    // implements the GreetingsExtensionPoint
    const greetingsExtensions = new GreetingsExtensions();

    env.registerExtensionPoint(greetingsExtensionPoint, greetingsExtensions);

    env.registerInit({
      async init() {
        // The greetingsExtensions instance is in scope as it's created above.
        // Note how this uses the private .greet() method which is
        // not exposed in the public API.
        greetingsExtensions.greet();
      },
    });
});

Providing Services
There is going to be a set of default services provided by the framework and surrounding packages, for the purposes of for example routing, logging and many other aspects. These are common utilities that are intended to be usable by all backend plugins. Regardless of what this system ends up looking like, we intend for there to be a package that brings together sensible defaults for the average backend, just like @backstage/app-defaults does for the frontend. These services can be overridden with custom implementations or configurations at the backend level.
What's common for many of these services is that there is a need to scope the service implementation to each plugin. For example the logging service is more useful when it's able to tell which plugin is producing a particular log line. To accommodate this use case, the service factory always creates an instance scoped to the plugin requesting the service.
Service factories are created using the createServiceFactory function connecting the factory implementation to the ServiceRef. The factory function returned is expected to produce an instance of the service given a plugin ID.
export const loggerServiceRef = createServiceRef<LoggerService>({
  id: 'core.logging',
});

const loggerServiceFactory = createServiceFactory({
  api: loggerServiceRef,
  deps: {},
  factory: async () => {
    const rootLogger = new RootLogger();
    return async (pluginId = 'root') => rootLogger.child({ pluginId });
  },
});

// The backend is then passed the loggerFactory to register it with the backend.
const backend = createBackend({
  apis: [loggerServiceFactory],
});

Similar to frontend Utility APIs, service factories may depend on other services. The dependency mechanism is slightly different though, as we receive a factory function rather than the concrete implementation of the service dependency. This factory function is then used to request a service instance for a given plugin ID. Services may also support un-scoped instances, which are retrieved by omitting the plugin ID.
export const dbServiceRef = createServiceRef<DbService>({
  id: 'core.db',
});

const dbServiceFactory = createServiceFactory({
  api: dbServiceRef,
  deps: {
    loggerFactory: loggerServiceRef,
  },
  factory: async ({ loggerFactory }) => {
    const rootLogger = await loggerFactory();
    const dbManager = new DbManager(rootLogger);

    return async (pluginId?: string) => {
      if (!pluginId) {
        throw new Error('DB Service must be scoped to a plugin');
      }
      const logger = await loggerFactory(pluginId);
      return dbManager.forPlugin(pluginId, { logger });
    };
  },
});

Developer Experience
Many installations have several backend plugins running together in the same main process. To accommodate for a leaner and less noisy development experience it is desirable to have an option to run only a specific subset of plugins wired up to any given backend instance.
For example this would just start the catalog and scaffolder backend.
yarn backstage-cli start-backend --backend catalog --backend scaffolder

Alternatives
Several experiments on different setups where conducted in the backend-system-exploration repository where we looked at different approaches for wiring up plugins and providing dependencies.
Roadie have also helped out prototyping what the backend system would look like with an off-the-shelf dependency injection framework added on top of the existing backend system. See #9165 for more information.
Risks
Massive Backend Plugin Migration
The intent is to gradually let existing plugins implement the new Backend API while still exposing the old API for backwards compatibility.
There should be a way to install and wire a "legacy" plugin into the new backend system. We don't see a risk supporting that use case as the existing plugin setups are mostly relying on the old environment for configuration.
Restrictions Introduced by the Framework
The current way of building and structuring backend plugins is extremely open, with very few restrictions or guardrails. This means that you might not be fighting the framework that much, but it also makes it difficult for the framework to help you. The suggestions in this RFC changes this balance by a fairly large margin. It introduces a lot more structure, benefitting utility and easy of installation, but at the cost of flexibility and stricter boundaries enforced by the framework.

Amazing proposal! I really like how it encapsulates much of the same pattern from the frontend system which provides integrators the same familiar interface and a consistent configuration experience throughout the entire system.
Can you elaborate on the plugin extensions point design a bit more ie. what would an example look like for a catalog extended with a custom processor/provider?
I also resonate with the framework restrictions risk. For us, one of the main obstacles for other teams helping implement custom plugins is the learning curve involved with learning the "Backstage framework". Backend plugins were a lot easier to "sell" since you only really needed to know how to implement an Express router (though according to this proposal, it shouldn't be that different).

Amazing proposal! I really like how it encapsulates much of the same pattern from the frontend system which provides integrators the same familiar interface and a consistent configuration experience throughout the entire system.

Thank you! Yep that is indeed part of what we were going for, making sure existing knowledge of the Backstage frontend can be reused.

Can you elaborate on the plugin extensions point design a bit more ie. what would an example look like for a catalog extended with a custom processor/provider?

They're a completely open interface, a blank canvas, but we do have some plans. As part of the backend system design we've explored a couple of different concepts for bridging together plugins and modules. One of them is "hooks", another is "feature collections". Both are things that we thought could potentially be the interaction point, but in the end we realized that it was the wrong abstraction level and we should let plugins choose instead.
So as part of this system we would likely include a couple of convenient building blocks for creating extension points. I'll give a super quick intro to the two concepts above, but let's please keep the focus of the discussions in this RFC around the original proposal and not these APIs as it could quite easily derail otherwise.
Hooks
A way to replace or add behavior deep inside plugins, inspired by the Webpack architecture.
// in catalog register()
const engineExtensionPoint = {
  hooks: {
    stitchEntity: createAsyncHook<{entity: Entity}>(),
    beforeEntityProcessing: createSyncHook<{entity: UnprocessedEntity}>(),
  },
}

env.registerExtensionPoint(catalogEngineExtensionPoint, engineExtensionPoint);

// inside catalog stitcher
await engineExtensionPoint.hooks.stitchEntity.callHook({entity})

// inside a different module
({
  deps: { catalogEngine: catalogEngineExtensionPoint },
  init({ catalogEngine }) {
    catalogEngine.hooks.tap(async ({ entity }) => {
      // custom logic
    })
  }
})

Feature collections
Inspired by all different places where we have various features that we want to add to or replace, such as catalog processors, entity providers, search collators, etc.
// in catalog register()
const ingestionExtensionPoint = {
  providers: createFeatureCollection<EntityProvider>()
}

env.registerExtensionPoint(catalogIngestionExtensionPoint, ingestionExtensionPoint);

// inside catalog init
const providers = ingestionExtensionPoint.providers.get()

// inside a different module
({
  deps: { catalogIngestion: catalogIngestionExtensionPoint },
  init({ catalogIngestion }) {
    catalogIngestion.providers.add(new GitHubOrgProvider())
    // or, perhaps more likely to be called directly in the backend setup
    catalogIngestion.providers.remove('url-reader')
  }
})

I also resonate with the framework restrictions risk. For us, one of the main obstacles for other teams helping implement custom plugins is the learning curve involved with learning the "Backstage framework". Backend plugins were a lot easier to "sell" since you only really needed to know how to implement an Express router (though according to this proposal, it shouldn't be that different).

Yep, well strive to keep things lean with hopefully minimal amount of code needed for a simple express route plugin. No fancy wrapping of the express API or anything like that. Hopefully this system will in the end make it easier to build even a simple plugin with few external interactions. Very much aware of the risk here though

Great proposal! Understood why and what's expected with examples.
To be honest I haven't used Backstage and haven't gone through the code nor the system design docs. But with some experience in using other frameworks with similar problems (i guess), the proposed way would help the community, strengthen contributors a lot.
Thanks.

Implementation is underway, tracked through a meta issue at https://github.com/backstage/backstage/issues/13081

Have we given any thought to having teardown in addition to setup be a core part of the API. I think this is critical to be able to do things that most mature plugin/addon systems have that allow you to programmatically add/remove or restart it. To use se the example from the readme, this could look like:
import { greetingsExtensionPoint } from '@backstage/plugin-example-hello-world-node';

export const examplePlugin = createBackendPlugin({
  id: 'com.my-company.example',
  register(env) {
    // implements the GreetingsExtensionPoint
    const greetingsExtensions = new GreetingsExtensions();

    env.registerExtensionPoint(greetingsExtensionPoint, greetingsExtensions);

    env.registerInit({
      async init() {
        // The greetingsExtensions instance is in scope as it's created above.
        // Note how this uses the private .greet() method which is
        // not exposed in the public API.
        greetingsExtensions.greet();
      },
      async drop() {
        // do teardown specific to this exertion point.
      },
    });
  },
  async unregister() {
    // remove databases, clear out custom directories, etc...
  },
});

@cowboyd other than for local development we didn't (seriously) consider any kind of system for hot-swappable plugins or anything like that. I don't agree that most mature plugin systems have the ability to add/remove plugins at runtime. Are there any particular use-cases you have in mind?
We did consider startup/teardown hooks though, but are currently thinking that we can expose those as services if needed. So init would depend on for example lifecycleServiceRef and you could do something like lifecycle.onShutdown(() => ...) from within init.

Yep we'll have a look at local development more properly before settling things and happy to bounce ideas in that area too. We did some early experiments where we concluded that the HMR that we have right now may be a bit overkill, and restarting the entire server on every change might be alright. The backend has to be and generally is stateless, with all state stored in the DB. An exception to that during development is SQLite, which we'd need some form of solution for.

The backend has to be and generally is stateless, with all state stored in the DB.

I should qualify what I mean by state. I'm talking about things like database connections, bound server ports, open file descriptors, and active http connections, and from that perspective, the backend is extremely stateful.

Closing this as the first versions of this are in alpha already, and the rest of the work is tracked in other issues.


Placeholder to show name of Player's job

Description of new idea or improvement on existing one
Currently, there is a placeholder to display the levels and EXP of each job for players, but no placeholder to display the NAME of the job, to correspond.
Detailed information what should do what and when
For instance, I can use %jobsr_user_jlevel_2% to show the level for the player's second job. Lets say it's Builder. But there is no placeholder to show the Name for player's second job. If I used %jobsr_name_2%, it would show the second job in the config, which might be Farmer.
So I can't have a menu, scoreboard, or gui, that shows the player's second job, with the level of the job next to it.
It'd be great if there were a placeholder that did this.
So basically, something like %jobsr_user_name_(jname/number)%.

So now you want to show the job name or the user name in the job?

I'd like to show the name of the job in X slot, for the player.
So if the player has three jobs: Farmer, Builder, Digger, I could use a placeholder like %jobsr_jname_2% to return "Builder", or %jobsr_jname_3% to return "Digger".

Added in this commit.


Delete Script Should Have a HTTP Method Option

Hello,
I am currently using your fantastic editor on a simple blogging app I'm developing. So far it's been great for me, but I have one little thing bugging me:
The deleteScript is calling the URL with POST by default.
Now this should be not a big deal, but I want to use DELETE because I want to directly delete the resource using the same URL as my insert, as a part of being a Restful app. Currently, this is impossible.
Does this sounds like a good idea to you? If so, I may be able to help by sending a pull-request or two. Thank you very much.

That sound like a good idea, you're right.
But we need to avoid breaking change, so the best solution should be to:

add an option (like deleteScript) called deleteMethod which should be at POST by default (to keep BC)
add a condition in Images.prototype.deleteFile to use either POST or DELETE depending on the deleteMethod option

You can make an ajax DELETE request with jQuery like that:
$.ajax({
    url: this.options.deleteScript,
    type: 'DELETE',
    data: { file: file }
});

Ok, I'm currently working on it. I can't assign myself at waffle.io though, but I'll send the pull request here as soon as I finished working.


Add support to execute queries with concurrent futures

The proposed PR aims to enhance the functionality of the existing codebase by adding support to execute queries with concurrent futures.
This feature will allow multiple queries to be executed simultaneously, which can significantly improve the performance of the application.

@anthonywainer
Can you provide one or two examples where parallel DDL executions will be useful?
Also, how do you exceptions when one or more SQL files have errors during execution? What is the expected behavior ?

I'm not OP but I see value on this feature as well.
One example is a situation where the project is using lots of Repeatable scripts, then applying lots of files with same prefix could be done in parallel and reduce overall time.
I've recently deployed about 200 scripts and took about 10 minutes. I'm quite confident it could be drastically reduced with some parallel execution.
Error handling is tricky, would the transaction rollback feature work if all future threads share the same session/connection?
One thing I noticed that is not handled in this PR is that the order of scripts are not respected (e.g.: we can't run R_00001_xx.sql and R_0002_xx.sql in parallel).
The orchestration of parallel runs should take care of different groups, running groups sequentially, but groups contents in parallel.
Other things is that it is trying to create a ThreadPoolExecutor with as many workers as scripts to run. That doesn't seem to be a good idea. My suggestion would be to have the number of max_workers to be actually part of the cmdline arguments.

The gain in performance, anecdotally does not, justify the complexity of managing exceptions. We will not be including this change into the main branch.


"-ltiff" missing in Makefile

There is no "-ltiff" switch in the Makefile, thus causing unresolved references of TIFF library functions.

Sorry, I wasn't aware of similar closed issue.


Reduce number of annotations needed to make it work

At the moment every method in a backing bean must get an own id for mapping.
This could be done by directly using the backing bean method name as a fallback for lookup.
This approach has a drawback as well, it's less refacting safe if backing bean interface is changed.

FluentApiBackingBeanField annotations are now optional.
Field method names will be used as fallback id if not set explicitly.


Enhancement: add is_final , default True  to the Transcript event

my stt provider (google)  provides the ability to get partial transcript updates as the AudioChunks are processed.
my app takes advantage of of those by giving the user visualization of their dictation progress
if we set the default to True, then existing transcript users will not be impacted (they wouldn't be checking for that attribute anyhow)

that workes...  (I made an xTranscript event)
can now get partial results.


chore: Minor typographical fixes

Fixes include:-

typo fixes
grammatical corrections

What kind of change does this PR introduce?

refactoring
Did you add tests for your changes?
N/A

Does this PR introduce a breaking change?
Nope

What needs to be documented once your changes are merged?
N/A

cc @sokra

cc @sokra

cc @sokra

Thanks


Can't add integration with config flow wizard

Describe the bug
Can't add a robot because of an error message from the wizard dialog.
To Reproduce
Try to add the integration from the integrations list.
The wait spinner starts to show. After some time it results in this error dialog:
(Config flow could not be loaded)
Der Konfigurationsfluss konnte nicht geladen werden:
 <title>504 Gateway Time-out</title>
504 Gateway Time-out

nginx/1.18.0

Expected behavior
The wizard should start to show available options.
Additional Information (please complete the following information)

HA version: 2023.3.1
integration version: 0.16.4   179f6e1
Model Name: dreame.vacuum.p2029

Having the same issue in comparable configuration, any workarounds?

How is the camera involved, when the wizard should show up?
Also, Dreame F9 and L10 Pro have no camera ;-)

I don't know what happened, today it works.
HA is still 2023.3.1 with integration version: 0.16.4
maybe a reboot of HA did the job already.


Tensorflow 2.1.0 causing all RAM to be consumed on model.fit

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock
example script provided in TensorFlow):

OS Platform and Distribution (e.g.,
Linux Ubuntu 16.04): Linux Ubuntu 19

Tensorflow version: 2.1.0cp37

CUDA/cuDNN version: - GPU model and memory:
1070 GTX 8 GB GDDR5

Describe the current behavior
I am using tf.data dataset API. When using model.fit, on the first epoch, I am already consuming 16 GB of RAM when I run with GPU, but when I run with CPU the memory consumption is very small.
EDIT: tensorflow-cpu actually has the same issue with high memory consumption
Same dataset, same code. This only happens in Tensorflow 2.>
Is this a known issue? What can I do as a temporarily fix?
Standalone code to reproduce the issue
Just a simpel model, with a dataset which loads images:
model = models.Sequential()
model.add(Conv2D(4, (9, 9), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(8, (9, 9), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(16, (9, 9), activation='relu'))
model.add(MaxPooling2D((2, 2)))

model.add(Flatten())
model.add(Dense(5, activation='sigmoid'))

model.summary()

model.compile(optimizer='adam', loss=losses.binary_crossentropy, metrics=['accuracy'])

history = model.fit(ds_train,
                    steps_per_epoch=steps_per_epoch,
                    validation_data=ds_val,
                    validation_steps=validation_steps,
                    epochs=10)

I should also mention that the images I am using are concatenated images that are 320px * 4 in width and 180 in height

Actually even when I use tensorflow-cpu, memory consumption is still very high:

@HadiSDev
Could you please share colab link or simple standalone code with supporting files to reproduce the issue in our environment. It helps us in localizing the issue faster. Thanks!

Here is the colab link: https://colab.research.google.com/drive/19p1PTKnje-tldQfYCTPimBT99bYK1y7N
It is working on colab but not on my own pc.

Exact same code and the same pictures

Might be related to: https://github.com/tensorflow/tensorflow/issues/35030

It gets worse when I use concatenated images where it is 320px * 4 in width.

Yes @HadiSDev Looks like it is indeed duplicate of #35030

Did you try using the latest tf-nightly version i.e., 2.2dev0 @HadiSDev ?

I was going to try but when I try to import tensorflow i get:
Python 3.7.5 (default, Nov 20 2019, 09:21:52)
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hadi/.local/lib/python3.7/site-packages/tensorflow/__init__.py", line 48, in <module>
    from ._api.v2 import compat
  File "/home/hadi/.local/lib/python3.7/site-packages/tensorflow/_api/v2/compat/__init__.py", line 39, in <module>
    from . import v1
  File "/home/hadi/.local/lib/python3.7/site-packages/tensorflow/_api/v2/compat/v1/__init__.py", line 34, in <module>
    from . import compat
  File "/home/hadi/.local/lib/python3.7/site-packages/tensorflow/_api/v2/compat/v1/compat/__init__.py", line 39, in <module>
    from . import v1
  File "/home/hadi/.local/lib/python3.7/site-packages/tensorflow/_api/v2/compat/v1/compat/v1/__init__.py", line 673, in <module>
    _module_dir = _module_util.get_parent_dir_for_name(_estimator_module)
  File "/home/hadi/.local/lib/python3.7/site-packages/tensorflow/python/tools/module_util.py", line 62, in get_parent_dir_for_name
    if not spec.origin:
AttributeError: 'NoneType' object has no attribute 'origin'

@HadiSDev Can you try reinstalling tensorflow 2.2.0 and let me know if the issue still persists. Thanks!

Closing this issue as it has been inactive for more than 3 weeks. Please add additional comments for us to open this issue again. Thanks!


[Bug]: Hot reload not triggered for certain theme files

Please confirm that you have:

[x] Searched existing issues to see if your issue is a duplicate. (If you’ve found a duplicate issue, feel free to add additional information in a comment on it.)
[x] Reproduced the issue in the latest CLI version.

In which of these areas are you experiencing a problem?
Theme
Expected behavior
Changing a file should sync the file and trigger a hot reload in the browser
Actual behavior
The file gets synced but a hot reload does not get triggered
Verbose output
2024-09-26T14:01:08.203Z:
Running system process:
  · Command: npm prefix
  · Working directory: /path/to/theme/dir

2024-09-26T14:01:08.556Z: Obtaining the dependency manager in directory /path/to/theme/dir...
2024-09-26T14:01:08.821Z:
Running system process:
  · Command: ruby -v
  · Working directory: /path/to/theme/dir

2024-09-26T14:01:08.824Z: Request to https://theme-kit-access.shopifyapps.com/cli/admin/api/unstable/themes/128731349105/assets/bulk.json completed in 647 ms
With response headers:
 - cache-control: no-cache
 - content-type: application/json; charset=utf-8
 - server-timing: processing;dur=306, socket_queue;dur=5.863, cfRequestDuration;dur=368.999958
 - x-request-id: 404652dd-32bb-4fee-9e1d-da16107c64b1-1727359268

2024-09-26T14:01:08.824Z: File Upload Results:
-templates/search.json: success
2024-09-26T14:01:08.914Z: Request to https://theme-kit-access.shopifyapps.com/cli/admin/api/unstable/themes/128731349105/assets/bulk.json completed in 743 ms
With response headers:
 - cache-control: no-cache
 - content-type: application/json; charset=utf-8
 - server-timing: processing;dur=421, socket_queue;dur=7.395, cfRequestDuration;dur=594.999790
 - x-request-id: c4eaaac1-1512-418c-81ca-68ccc6b25801-1727359268

2024-09-26T14:01:08.914Z: File Upload Results:
-locales/pt-BR.json: success
-locales/pt-PT.json: success
-locales/da.json: success
2024-09-26T14:01:09.009Z: Request to https://monorail-edge.shopifysvc.com/v1/produce completed in 156 ms
With response headers:
 - x-request-id: 4f19715c-651d-41d3-a633-316566efaa8f

2024-09-26T14:01:09.012Z: Analytics event sent: {
  "command": "theme dev",
  "time_start":<PHONE_NUMBER>068,
  "time_end":<PHONE_NUMBER>198,
  "total_time": 2130,
  "success": true,
  "cli_version": "3.67.2",
  "ruby_version": "2.6.10",
  "node_version": "20.11.1",
  "is_employee": false,
  "uname": "darwin amd64",
  "env_ci": false,
  "env_plugin_installed_any_custom": false,
  "env_plugin_installed_shopify": "[\"@shopify/cli\"]",
  "env_shell": "fish",
  "env_device_id": "45ac8e8547bf908f5d1325fcaa6aedbb15a7a228",
  "env_cloud": "localhost",
  "env_package_manager": "yarn",
  "env_is_global": true,
  "cmd_app_warning_api_key_deprecation_displayed": false,
  "cmd_all_timing_network_ms": 1896,
  "cmd_all_timing_prompts_ms": 0,
  "cmd_all_launcher": "unknown",
  "cmd_all_topic": "theme",
  "cmd_all_plugin": "@shopify/theme",
  "cmd_all_force": false,
  "cmd_all_verbose": true,
  "cmd_all_path_override": true,
  "cmd_all_path_override_hash": "7b29505f8d9e1cab721d6a3fd8fa024055af932c",
  "cmd_all_timing_active_ms": 233,
  "cmd_all_exit": "ok",
  "args": "--environment=feat1 --verbose",
  "cmd_all_environment_flags": "{\"password\":\"*****\",\"theme\":\"128731349105\",\"store\":\"my-dev-store\",\"ignore\":[\"templates/index.json\"]}",
  "env_plugin_installed_all": "[\"@shopify/cli\"]",
  "metadata": "{\"extraPublic\":{},\"extraSensitive\":{}}"
}
2024-09-26T14:01:09.012Z: Completed command theme dev
2024-09-26T14:01:09.026Z: Request to https://theme-kit-access.shopifyapps.com/cli/admin/api/unstable/themes/128731349105/assets/bulk.json completed in 856 ms
With response headers:
 - cache-control: no-cache
 - content-type: application/json; charset=utf-8
 - server-timing: processing;dur=530, socket_queue;dur=4.65, cfRequestDuration;dur=792.000055
 - x-request-id: 042617e5-46b8-4a2e-9b42-62e8dc6fb042-1727359268

2024-09-26T14:01:09.028Z: File Upload Results:
-locales/de.json: success
-locales/es.json: success
-locales/fr.json: success
-locales/nl.json: success
-locales/en.default.json: success
2024-09-26T14:01:09.256Z: Request to https://theme-kit-access.shopifyapps.com/cli/admin/api/unstable/themes/128731349105/assets/bulk.json completed in 1082 ms
With response headers:
 - cache-control: no-cache
 - content-type: application/json; charset=utf-8
 - server-timing: processing;dur=893, socket_queue;dur=7.323, cfRequestDuration;dur=958.999872
 - x-request-id: eb26a6d5-886e-448c-b53b-30f2c3334a01-1727359268

2024-09-26T14:01:09.260Z: File Upload Results:
-templates/404.json: success
-templates/article.json: success
-templates/blog.json: success
-templates/cart.json: success
-templates/page.contact.json: success
-templates/list-collections.json: success
-templates/collection.json: success
-templates/password.json: success
-templates/product.json: success
-templates/page.json: success
2024-09-26T14:01:09.269Z: Ignoring theme file config/settings_schema.json via .shopifyignore...
2024-09-26T14:01:09.302Z: Ignoring theme file templates/index.json via --ignore...
2024-09-26T14:01:09.303Z: Files to be deleted:

GET /
2024-09-26T14:01:10.726Z: → Rendering https://theme-kit-access.shopifyapps.com/cli/sfr/?_fd=0&pb=0 (with )...
2024-09-26T14:01:11.212Z: Request to https://theme-kit-access.shopifyapps.com/cli/sfr/?_fd=0&pb=0 completed in 485 ms
With response headers:
 - cache-control: max-age=0, private, must-revalidate
 - content-type: application/json; charset=utf-8
 - etag: W/"e85e94237ed788b7ea14d7f704819cbb"
 - server-timing: processing;dur=315, socket_queue;dur=7.705, cfRequestDuration;dur=381.000042
 - x-request-id: 2b125ed0-c70f-4a9c-9e8d-6e38275b3744-1727359270

2024-09-26T14:01:11.213Z: ← 200 (request_id: 2b125ed0-c70f-4a9c-9e8d-6e38275b3744-1727359270)
2024-09-26T14:01:17.463Z: Reading the content of file at sections/my-section.liquid...
2024-09-26T14:01:18.352Z: Request to https://theme-kit-access.shopifyapps.com/cli/admin/api/unstable/themes/128731349105/assets/bulk.json completed in 878 ms
With response headers:
 - cache-control: no-cache
 - content-type: application/json; charset=utf-8
 - server-timing: processing;dur=478, socket_queue;dur=7.054, cfRequestDuration;dur=558.999777
 - x-request-id: 94c20d70-82c7-4431-b116-99a0baee49d9-1727359277

• 10:01:18 Synced » update sections/my-section.liquid

Reproduction steps

Clone latest Dawn repo
Add a new file in sections/ called rich-text-2.liquid (the name does not seem to matter)
Paste this into the file: https://gist.github.com/nikitaourazbaev/8c8d6d20ce4fb990c145ffc3005c3289
Make changes to the file
Hot reload will not be triggered for this file, while it will be triggered for other files

Operating System
macOS Sonoma 14.7
Shopify CLI version (check your project's package.json if you're not sure)
3.67.2
Shell
Warp, fish
Node version (run node -v if you're not sure)
v20.11.1
What language and version are you using in your application?
Liquid

Hi @jamesmengo, yes, I’m rendering it:

index.json
{
  "sections": {
    "rich_text_2_DY3XVH": {
      "type": "rich-text-2",
      "settings": {
        "title": "Rich text",
        "rich_text": "
Use this text to share information about your brand with your customers. Describe a product, share announcements, or welcome customers to your store.
"
      }
    },
    "image_banner": {
      "type": "image-banner",
      "blocks": {
        "heading": {
          "type": "heading",
          "settings": {
            "heading": "Image banner",
            "heading_size": "h0"
          }
        },
        "text": {
          "type": "text",
          "settings": {
            "text": "Give customers details about the banner image(s) or content on the template.",
            "text_style": "body"
          }
        },
        "button": {
          "type": "buttons",
          "settings": {
            "button_label_1": "Shop all",
            "button_link_1": "shopify://collections/all",
            "button_style_secondary_1": true,
            "button_label_2": "",
            "button_link_2": "",
            "button_style_secondary_2": false
          }
        }
      },
      "block_order": [
        "heading",
        "text",
        "button"
      ],
      "settings": {
        "image_overlay_opacity": 40,
        "image_height": "large",
        "desktop_content_position": "bottom-center",
        "show_text_box": false,
        "desktop_content_alignment": "center",
        "color_scheme": "scheme-3",
        "image_behavior": "none",
        "mobile_content_alignment": "center",
        "stack_images_on_mobile": false,
        "show_text_below": false
      }
    },
    "rich_text": {
      "type": "rich-text",
      "blocks": {
        "heading": {
          "type": "heading",
          "settings": {
            "heading": "Talk about your brand",
            "heading_size": "h1"
          }
        },
        "text": {
          "type": "text",
          "settings": {
            "text": "
Share information about your brand with your customers. Describe a product, make announcements, or welcome customers to your store.
"
          }
        }
      },
      "block_order": [
        "heading",
        "text"
      ],
      "settings": {
        "desktop_content_position": "center",
        "content_alignment": "center",
        "color_scheme": "scheme-1",
        "full_width": true,
        "padding_top": 40,
        "padding_bottom": 0
      }
    },
    "featured_collection": {
      "type": "featured-collection",
      "settings": {
        "title": "Featured products",
        "heading_size": "h2",
        "description": "",
        "show_description": false,
        "description_style": "body",
        "collection": "all",
        "products_to_show": 8,
        "columns_desktop": 4,
        "full_width": false,
        "show_view_all": false,
        "view_all_style": "solid",
        "enable_desktop_slider": false,
        "color_scheme": "scheme-1",
        "image_ratio": "adapt",
        "image_shape": "default",
        "show_secondary_image": true,
        "show_vendor": false,
        "show_rating": false,
        "quick_add": "none",
        "columns_mobile": "2",
        "swipe_on_mobile": false,
        "padding_top": 28,
        "padding_bottom": 36
      }
    },
    "collage": {
      "type": "collage",
      "blocks": {
        "collection-0": {
          "type": "collection",
          "settings": {
            "collection": ""
          }
        },
        "product": {
          "type": "product",
          "settings": {
            "product": "",
            "second_image": false
          }
        },
        "collection-1": {
          "type": "collection",
          "settings": {
            "collection": ""
          }
        }
      },
      "block_order": [
        "collection-0",
        "product",
        "collection-1"
      ],
      "settings": {
        "heading": "Multimedia collage",
        "heading_size": "h2",
        "desktop_layout": "left",
        "mobile_layout": "collage",
        "card_styles": "product-card-wrapper",
        "color_scheme": "scheme-1",
        "padding_top": 36,
        "padding_bottom": 36
      }
    },
    "video": {
      "type": "video",
      "settings": {
        "heading": "",
        "heading_size": "h1",
        "enable_video_looping": false,
        "video_url": "https://www.youtube.com/watch?v=_9VUPq3SxOc",
        "description": "",
        "full_width": false,
        "color_scheme": "scheme-1",
        "padding_top": 36,
        "padding_bottom": 36
      }
    },
    "multicolumn": {
      "type": "multicolumn",
      "blocks": {
        "column1": {
          "type": "column",
          "settings": {
            "title": "Column",
            "text": "
Pair text with an image to focus on your chosen product, collection, or blog post. Add details on availability, style, or even provide a review.
",
            "link_label": "",
            "link": ""
          }
        },
        "column2": {
          "type": "column",
          "settings": {
            "title": "Column",
            "text": "
Pair text with an image to focus on your chosen product, collection, or blog post. Add details on availability, style, or even provide a review.
",
            "link_label": "",
            "link": ""
          }
        },
        "column3": {
          "type": "column",
          "settings": {
            "title": "Column",
            "text": "
Pair text with an image to focus on your chosen product, collection, or blog post. Add details on availability, style, or even provide a review.
",
            "link_label": "",
            "link": ""
          }
        }
      },
      "block_order": [
        "column1",
        "column2",
        "column3"
      ],
      "settings": {
        "title": "",
        "heading_size": "h1",
        "image_width": "third",
        "image_ratio": "adapt",
        "columns_desktop": 3,
        "column_alignment": "center",
        "background_style": "none",
        "button_label": "",
        "button_link": "",
        "color_scheme": "scheme-1",
        "columns_mobile": "1",
        "swipe_on_mobile": false,
        "padding_top": 36,
        "padding_bottom": 36
      }
    }
  },
  "order": [
    "rich_text_2_DY3XVH",
    "image_banner",
    "rich_text",
    "featured_collection",
    "collage",
    "video",
    "multicolumn"
  ]
}

Is there anything else I can do to narrow down the cause on my end?


[Admin] (design) Fix link to animation guidelines

Addresses issue https://github.com/OfficeDev/office-js-docs-pr/issues/1189.

Docs Build status updates of commit 7613530:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/design/using-motion-office-addins.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Removing Shared list - missing name in header of dialog

"The list name is displayed in the dialog header"
When removing a shared list it states Remove undefined from shared lists.
As you can see from the screenshot, the attempt was to delete "Jody Test - Edit" shopping list

couldn't reproduce this one

Now a pass


Cursor Disappearing on Non-Interactive Areas & navbar

Hi @MohitKambli @manoj9896 @khandelwal20sid @vatsaljain-hub @AdityaBavadekar
On the Lisbook website, the mouse cursor disappears when entering black or white spaces outside of the main template areas. This makes it difficult for users to navigate and click on links efficiently.
Suggested Improvement:
Ensure the cursor remains visible across the entire page, including non-interactive areas, to improve navigation and overall user experience.
https://github.com/user-attachments/assets/59cfda73-671d-4218-8b6f-bd5439dc8e16

if you think it is a useful, please assign this issue to me and my teammate @jayasri2111.

Go ahead @Saiharitha3. Don't forget to 🌟 the repo

Thank you! sure.

Hi team! I'm new to contributing here, and I noticed this issue is still open. I've put together a pull request with a potential fix. @Ctoic would you be able to review it when you have a chance? Thank you!


frontend: refactor passphrase.tsx

This is a followup on #2713 that refactors the component to be more concise, and easier to understand/maintain.

Set passphraseEnabled to decide whether to display the disable or the enable wizard.
When one wizard is rendered it knows the status of the passphraseEnabled. Also the disable wizard does not need to know about the steps. The enable wizard knows where to start end stop and can manage that state on its own.
SetPassphrase is more explicit and less error prone than toggling it.
Remove unnecessary API request after receiving a successful response after setting the passphrase on the device.
Remove some code duplication when rendering steps

I used t(..) in the object directly, and reformatted the renderSteps return. PTAL @thisconnect :)