apply transforms to epi image

Hi,
As I mentioned on my previous neurostars post (https://neurostars.org/t/sdcflows-applytopup/19516), I am trying to use sdcflows to correct a BOLD image before applying fMRIPrep to it. Following our last discussion, I have run fMRIPrep to obtain only the slicetime and motion correction. The command I used was the following:
singularity run --cleanenv                                                      \
	-B ${TEMPLATEFLOW_HOST_HOME}:${SINGULARITYENV_TEMPLATEFLOW_HOME}            \
	-B /data/HCP/hcp_test_bids:/in                                              \
	-B ${TMPDIR}/out:/out                                                       \
	-B ${TMPDIR}/wrk:/wrk                                                       \
	${container} /in /out/${s} participant                                      \
	--participant_label ${s}                                                    \
	-w /wrk/${s}                                                                \
	--nthreads 4 		                                                    \
	--mem_mb 48000                                                              \
	--fs-license-file ${license}                                                \
	--fs-no-reconall     	 						    \
	--skip-bids-validation                                                      \
	--notrack                                                                   \
	--ignore fieldmaps 					                    \
	--omp-nthreads 3

The output looks like this

I have also created this script to merge the transformations obtained from sdcflows, the motion correction parameters, and transform the EPI image.
You can see the script I used to merge the transformations here (https://github.com/JessyD/sdcflows-test/blob/master/apply_sdcflows.py). However, the outputs do not look like what I expected. I tried all permutations on the way that the transformations are passed to merge_xforms, but they all look similar to the images below:

affine --> motion correction --> fieldwarp,

motion correction --> affine --> fieldwarp

Am I missing something or maybe passing the incorrect transformations (https://github.com/JessyD/sdcflows-test/blob/master/apply_sdcflows.py)?
@effigies @mgxd
Thank you!

I think this is a problem - I'm still trying to fully diagnose it but it looks like a bug indeed.

Okay, I've managed to diagnose the issue. The fieldmap output from the unwarping workflow is indeed a fieldmap (therefore, just a 3D file, in Hz units).
In order to feed this into antsApplyTransforms and generate an actual unwarping transform, the fieldmap needs to be converted (which means, it needs to be set in the right dimensions and scale it by the readout time and pixel size along PE).
Working on a fix now.


Test: improved editor "preview" behaviour

Refs: https://github.com/microsoft/vscode/issues/109779 https://github.com/microsoft/vscode/issues/85636

[ ] anyOS @alexdima
[ ] anyOS @eamodio

Complexity: 3
Create Issue

Release Notes
Preview editors are enabled by default e.g. when you just click on a file from the explorer and appear with italic font. The idea of preview editors is that the next editor replaces the previous preview editor so that you don't accumulate tabs. An editor turns into a non-preview editor based on many conditions (e.g. making it dirty). In this milestone we tweaked this functionality to put an editor into non-preview mode right from the beginning or guide the user how to change the settings.
Note: if you have preview editors typically disabled, please enable them for testing.
Verify:

workbench.editor.enablePreviewFromQuickOpen is false by default
workbench.editor.enablePreview appears in "Commonly Used" section of settings editor and is easy to understand
you find a action in the top right "..." editor overflow menu to always keep editors and this turns off editor preview
when you trigger "Goto definition" and that brings you to another file, the editor you start from gets out of preview automatically

While this is outside the scope of this, when I went looking for the workbench.editor.enablePreview setting in the Commonly Used section it took me a bit to find it at first because for I wasn't looking for the "Workbench" part.

All looks good to me!


Styling 2: Return of the Styling

Gotta love all the whitespace diffs.

@bignell The routing works with the buttons. I like the color! We will play around with finishing touches on the rounded corners etc.

Did you try to add a margin between the upper buttons to differentiate, or decide not to, or did this suggestion get lost in the reopening of the PR?


Fix create namespace error

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

main

PR #79

PR #85   👈

PR #86

This stack of pull requests is managed by Graphite.


Conditional incoherent amplitude displays are showing wrong magnitudes

Example: https://algassert.com/quirk#circuit={"cols"%3A[["H"%2C"H"%2C"H"]%2C["•"%2C"Z"]%2C["Amps1"]%2C[]%2C["Amps1"%2C1%2C"•"]%2C[]%2C["Amps1"%2C1%2C"◦"]]}

Adding controls results in the total squared amplitude being less than 100%.

Wong tracker?

Whoops.


Support for PostgreSQL schemas broken

I updated an old project and found that Kuery-ORM no longer recognizes PostgreSQL schemas:
struct Person : Codable, Model {
    static var tableName: String = "folks.person" //folks is the schema and person the table

    var id: Int
    var firstName: String
    var lastName: String
    var jobTitle: String

    enum CodingKeys : String, CodingKey {
        case id
        case firstName = "first_name"
        case lastName = "last_name"
        case jobTitle = "job_title"
    }
}

Error:  706 : Query execution error:
ERROR:  relation "folks.person" does not exist
LINE 1: SELECT * FROM "folks.person"
                      ^
 For query: SELECT * FROM "folks.person"

This is an unexpected consequence of the changes we made to support case sensitivity in the database plugins. I can recreate the issue outside Kuery by simply creating a schema, creating a table within it and then wrapping the schema.tablename in double quotes in a SELECT statement.
This one may be tricky to fix in an elegant fashion, will see what I can come up with.

Having spent some time looking at this issue in more depth I have concluded schema's are not something that we can support or that were intentionally supported in the past.
My reasoning for this is the fact that the three supported databases for SwiftKuery all have different implementations of SCHEMA that we are not able to implement support for in a plugin agnostic manner. While an implementation could be made it would result in application code that was not portable between database plugins which would be contradictory to SwiftKuery being an abstraction of the underlying plugin implementations.
@fwgreen, unfortunately this is not something we will be able to resolve. It is possible to use raw queries to interact with schema's via the executeQuery methods on a model but the utility functions such as save and find will not work.

@kilnerm I'm sorry to hear that.  I had assumed that the support for PostgreSQL came from its Swift-Kuery plugin and not from Swift-Kuery or Swift-Kuery-ORM.
Adding the ability to perform a raw query as part of the database configuration, while not elegant, would mostly solve the problem:  The user could then set the search_path for the rest of the session.

I'm marking this as closed. If anything changes with regard to our ability to implement a solution we can re-open.


External domain access list : MS teams and SfBOlnine

Is the External Domain Access list shared between Microsoft Teams and Skype for Business Online or are they maintained as separate lists?
If they are common, can I use the New-CsEdgeAllowList cmdlet to manage the Microsoft Teams allowed domains list?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 31588a25-f3ee-e888-aa67-203be3d34489
Version Independent ID: 3b498bcf-69a9-01ac-973a-4d5dbeea587c
Content: Manage external access (federation) in Microsoft Teams
Content Source: Teams/manage-external-access.md
Service: msteams
GitHub Login: @LolaJacobsen
Microsoft Alias: lolaj

@officedocsbot assign @scanum

Hi @Arcass
Thanks for reaching out!
Yes, external domain access list is shared between Microsoft Teams and Skype for Business Online. So you can manage the teams external list using the "New-CsEdgeAllowList" cmdlet as well.
Hope this information is helpful. Looking forward to your confirmation. Thank you.

@Arcass  Thank you for submitting this feedback.
From our understanding the issue has been resolved. If you feel it hasn't been resolved please re-open this issue.

@officedocsbot close


Tests for scrabble-score's optional extensions

So I set myself the challenge of doing scrabble-score's extensions for :double- and :triple-scoring letters.
In the course of doing so, I wrote tests. I thought these may be useful to the source, so here is my offer to include them upstream.
Comments, suggestions, nits and other (constructive) criticism welcome; this is my first contribution to exercism.io, so I may need some guidance ;-)

Aha, the Travis tests fail because my new tests fail.
It seems the reference implementation doesn't actually do the double and triple letter scoring.

Oh yeah! and before I forget, I have to ask to ignore all but the first test, as I did in the other PR

I think this is a good start. Thanks.

Thanks for the feedback! I'll try to incorporate it!
The #[ignore] will be added, but probably only after I've passed the Travis round at least once.
(Or is travis smart enough to un-ignore the tests?)
I may be slow to respond, but that won't mean I've abandoned this, it's just that I'm doing this in the brief lost minutes during my commute :-)
I'm undecided on :garbage;
I was considering that :doubleSomeword would be the syntax for a double-scoring word; The spec leaves open how this would be expected.

(Or is travis smart enough to un-ignore the tests?)

Yes, it does. I should have explained it, my bad for not doing so!
You don't necessarily need to look at this script, but if you are curious, our Travis CI runs https://github.com/exercism/xrust/blob/master/_test/check-exercises.sh#L42, thus unignoring all tests.

I was considering that :doubleSomeword would be the syntax for a double-scoring word; The spec leaves open how this would be expected.

Ah! This could be a fine idea. You are right about the spec.
As in the other PR, this could be a thing all languages can benefit from, so it could be worth opening an issue in x-common to discuss how the spec should be interpreted. And if we reach an agreement, a PR can be sent to x-common.
If agreement can't be reached in x-common, we could possibly clarify the specification for our track by adding a exercises/scrabble-score/HINTS.md - this is a special file that gets appended to the README.

update!

I've #[ignored] all newly added tests
example code updated to pass all tests (locally ran check_exercises.sh, passed!)
slight tweak of one test name

feel free to nitpick over variable names and comments! It's the example after all, so it should be clear!
I'm opening a discussion issue on x-common next.

a delimiter like [] works for me

double []

triple <>

aw[e]some

awsome

[awesome]

<aw[e]some>

etc.

And I'm fine having tests in place to handle the "extra credit" problems, even if those tests aren't in canonical.

Ah, delimiters! I like how simple the implementation could become:
(I wonder if that compiles...? Mutating the multiplyer inside the match AND reading it in very the same expression looks maybe a bit too compact to work. But maybe Rust lets us have nice things!)
score += Match c {
   '[' => multiplyer *= 2; 0,
   ']' => multiplyer /= 2; 0,
   '<' => multiplyer *= 3; 0,
   '>' => multiplyer /= 3; 0,
   'a'|'e'|'u'.... => 1,
   ... => 2,
   _ => 0
} * multiplyer

If I'm reading this PR correctly, there are just a few small changes @petertseng mentioned in comments. Are those blocking merge? Or can we move on this PR without those changes?

I had a different reading of this PR: I thought all of my comments are unimportant, and what we would want to discuss before merge is:

In-line notation seem super awkward to me. Parsing seems fraught with problems, as you mention. I think a more robust structure is necessary.

And that whatever notation we use should be added to HINTS.md

Ah, yes. I saw the comment approving the ideas of delimiters, but hadn't checked the code to see if delimiters had been implemented.

After the discussion on the common track, I understood that the optional exercise was intentionally left vague. Figuring out a syntax was explicitly said to be part of the exercise.
That's why I haven't pursued this further.

Ok. Sounds like we can close, then. Thanks for your work on this, @juleskers!


Validate RC pipeline with DDP

aferritto created the following on Apr 21:
DDP will help with training larger models
Original issue: https://github.ibm.com/ai-foundation/PrimeQA/issues/143

@bhavani105 : please update the status and the pipeline label for this issue


Extension host terminated unexpectedly...

We have written the needed d
Issue Type: Bug
after the latest update this error is occurring with my vs code for several times.. so that no extensions are working ........
getting rid of it....
so much irritating...
 Please paste.

after reinstalling the vs code works perfectly but after the restart same problem is arising so please do some permanent solution....

Check if your antivirus is interfering with VSCode files.

Duplicate of https://github.com/microsoft/vscode/issues/94807, please read the issue here for how to fix: https://github.com/microsoft/vscode/issues/94858#issuecomment-611905765

i have found the solution for this...
go to the quick heal
->file folders
->exclude file folder from scanning
->add file i.e C:\Users\hp\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js
->save changes...
It is working perfectly....


Update to Go 1.22

Bumps Go version to 1.22, which also required a bump of golang.org/x/tools to resolve a a invalid memory address or nil pointer dereference panic.
Closes https://github.com/derision-test/go-mockgen/issues/51
I'm not sure what else would be required, or if this is even the best approach to resolving the panic 🤔  perhaps a Go version somewhere between 1.18 and 1.22 resolves the issue as well.

@efritz - Thanks for merging! Can you make a new tag to include this merge?

@unknwon v2.0.0 🎉


No way to provide ID

React Router has an ID tag in both IndexRouteObject and NonIndexRouteObject as you can see here. This provides an easy way of defining routes so allow easier isActive classes for elements outside React Router. It would be great to have this added as well
https://github.com/remix-run/react-router/blob/de5e57b1fc7309617208a6dd2ddcbd68fdca48b4/packages/react-router/lib/context.ts#L22-L40
I'm trying to setup Storybook in our project and that seemed like the easiest way to handle links in the tabs here
https://github.com/packit/dashboard/pull/240

Hello 👋 thank you for opening an issue.
That's reasonable and easy to do, I'll add it this week.

@SpyTec support for route id has been added in version 1.0.2. Can you give it a try and let me know if it fulfil your need ?

I'm closing this, if you have an issue please re-open it.


Accessibility

Address accessibility issues: https://wave.webaim.org/report#/https://theairlab.org/

Can somebody volunteer for this?

I added a language tag that was missing in this commit 09120b0.
I think the rest of the issues WAVE reports are ok to ignore. Most of the are the following:

No text on some links, but these have images instead, for example the twitter icon link at the bottom of the page.
Alt text on images is the same as the title of the articles they link to. I think this is fine.
Some text looks like a heading but isn't using a heading tag. The text still looks ok.
It thinks some of the buttons on the Publications don't have text but they actually do.

Commit 124ab8cc8659c4249c369994f889fd1b32fbc086 fixed most of the errors. Now we just have occasional warnings, as mentioned in the above comment by John. We could periodically revisit this and check the website.


New version: Brave.Brave.Nightly version <IP_ADDRESS>

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.4 schema?

Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/116366&drop=dogfoodAlpha

/azp run


Move Identity Cookie code into separate functions and cleanup invalid identity cookies

Q
A

Is bugfix?
no

New feature?
yes

Breaks BC?
no

Tests pass?
yes

Fixed issues
#8795

I've this code in my application: https://ideone.com/oygl4K
It was necessary to me to separate the parse cookie logic, because you can have a console application that uses cookies (e.g. websocket).

Generally looks good. Could you add a CHNAGELOG line, please?

OK.  I made those changes.  Thank you!

Merged https://github.com/yiisoft/yii2/commit/7249a6c99e96cafae522da379fc7c609ce910bc3 with minor adjustments https://github.com/yiisoft/yii2/commit/38be74446e943f238f13be02ac93f1880451548c

Thank you @maine-mike!


Refine Redis Connector

Split Redis api in smaller groups (Hash, List, Sets, Streams...)
Create at least one integration test for each the modules mentioned above
Unit test for all of the redis operations
Update documentation with an examples for each of the redis api

WIP


Documenting Admin features for Jenkins Adminstration

Document what are some of the hooks, settings that are configured that will allow a stuck Jenkins job to be kicked off.
In general document Jenkins configuration guide.

From: Mark Cowlishaw
Sent: Tuesday, May 2, 2017 11:47 AM
Subject: FW: Update Microsoft.Azure.KeyVault.WebKey dependency on JSON.NET to unblock ASP.NET Core 2.0 preview1
It looks like there was a separate configuration for the BUILD triggers for this job, so that it wasn’t kicking off with your comments.  I added you, so you should be able to kick things off with github comments now.  I had previously added you to the global list, but that apparently did not percolate to this project.
If you notice that the ‘test this please’ comments are not working for any project in the future, lmk and I will show you how to add yourself.

Ack.  I am going to put this documentation in our team OneNote.

Docs around debugging are complete.  Now working on finishing up docs on setting up workers.


Regression Suite Run on Deploy

User Story - Business Need
We need to run the regression suite in all lower environments upon deploy, so we can catch issues before a production deployment.

[x] Ticket is understood, and QA has been contacted (if the ticket has a QA label).

User Story(ies)
As a business owner
I want quality checks built into the deployment pipeline
So that the platform can automatically detect and raise issues before changes get to Production
Additional Info and Resources

Work was attempted on this in #1327, however there was an issue with using github runner and public endpoints which resulted in timeouts and failure. This was rolled back and canceled until further work could be done on this.
The Edge team manages the edge routers - the routers through which all VA network traffic from the internet enters our network.
Example of deployment status reported to slack
Private Zenhub Image

Engineering Checklist

[x] Open a ticket with Edge
[ ] Determine if there are alternatives to how runner is being used or an alternative to runner if that isn't going to work
[ ] We need to run the regression from these API actions against the private endpoints

[ ] Deploy to ENV action .github/workflows/dev_deploy.yml
[ ] Tag Push to Perf .github/workflows/tag_trigger.yml
[ ] Pre-release to Staging .github/workflows/prerelease_trigger.yml

[ ] This action needs to be modified or removed and replaced
.github/workflows/regression-public.yml

Acceptance Criteria

GIVEN the deployment is triggered to env DEV/STAGING/PERF

[ ] Regression run with errors
WHEN the regression suite completes successfully, but returns errors
THEN report the update to slack in va-notify-automatic-alerts channel with a status of Regression Failed
AND further details can be found in the linked workflow

[ ] Regression run success
WHEN the regression suite completes successfully with no errors
THEN report the update to slack in va-notify-automatic-alerts channel with a status of Regression Passed
AND further details can be found in the linked workflow

[ ] Regression run technical failure
WHEN the regression suite fails to complete
THEN report the update to slack in va-notify-automatic-alerts channel with a status of Regression Error
AND further details can be found in the linked workflow

QA Considerations
For QA to populate. Leave blank if QA is not applicable on this ticket.

[ ] the regression runs and reports out its status when the Deploy to Env API action is used for dev, perf, staging
[ ] the regression runs and reports out its status when the tag Push to Perf action is used
[ ] the regression runs and reports out its status when the pre-Release trigger deploys code to Staging

Potential Dependencies
It is preferred to use the private endpoints.
If we can only use the GitHub runner space to run the code and running it against the public routes, then we are dependent on Edge resolving whatever issue is causing timeouts at the Edge routers for the public endpoints
It is recommended to not use self-hosted runners in public repos.
Out of Scope
We haven't yet discussed advanced deployment logic, like a rollback, when the regression suite fails, so for now the deployment will complete regardless or regression suite failure.
For full reporting of output (not truncated results) we might need to wait until the html report is created and stashed as an artifact in GitHub. See Spike. So, truncated output should not be a blocker.

Hey team! Please add your planning poker estimate with Zenhub @cris-oddball @EvanParish @k-macmillan @kalbfled @ldraney @nikolai-efimov

We set this to an 8 because both Cris and Lucas will be working on this together. They are hoping it's only 3ish days, but ya never know.

Lucas and I agreed to pick this up on Monday

Me and Cris synced on this.  She discovered that using a self-hosted runner with a public repo is fine, so long as we follow some security practices that shouldn't be an issue in our case.  That enough for me to move forward on investigating if we can simply add the public repo to the self-hosted runner, and what that would entail.  It's important I don't disrupt work that is currently using the runner, so I'm hoping for a clear answer without needing to play with our current runner.
If there's a path forward, I will need to update the runner at some point, so I'll coordinate with QA and team at that point.

From the self-hosted runner documentation:

You can add self-hosted runners at various levels in the management hierarchy:

Repository-level runners are dedicated to a single repository.
Organization-level runners can process jobs for multiple repositories in an organization.
Enterprise-level runners can be assigned to multiple organizations in an enterprise account.

Which means we need to add an organization-level runner if we want it to be accessible by both repos.
As our organization is department-of-veterans-affairs, we may have some issues with that; but I pinged another DevOps engineer (Jeremy) to hear his thoughts.  I may end up messaging the oddball-va channel to be sure!
That's not a likely possibility, so now I'm going to investigate using a PAT (rather than deploy_keys, which has already been attempted) for repo to repo communication.  As long as this allows for a beautiful workflow_call at the end of our deploy_to_env workflow to regression-public.yml, I'll be happy with this solution.

I'll send this comment to @cris-oddball and @k-macmillan to see if things look good.  If so, I'll ask Kyle to add the PAT to our repo as QA_REPO_ACCESS_TOKEN.
I can use a self-hosted runner for our private repository in our public repository's workflow via a workflow_call, assuming we have the Personal Access Token (PAT) set up. Here's how I can approach this.  Btw, this is a fairly strong hypothesis/approach, based on docs and another oddball engineer (Jeremy), who will be reading this tomorrow morning as well:

Reusable Workflows: I can use a reusable workflow by another workflow if it's in the same repository, stored in a public repository (and our organization allows using public reusable workflows), or stored in a private repository (with settings that allow it to be accessed). More information can be found in GitHub Docs on Reusable Workflows.

Accessing Self-Hosted Runners used by another repo: Called workflows that are owned by the same user or organization as the caller workflow can access self-hosted runners from the caller's context. This means a called workflow can access self-hosted runners that are in the caller repository or in the caller repository's organization, provided that the runner has been made available to the caller repository. Detailed information is available on the same GitHub Docs page.

Calling a Reusable Workflow: To call a reusable workflow, I'll use the uses keyword within a job. I can reference reusable workflow files using the syntax {owner}/{repo}/.github/workflows/{filename}@{ref} for workflows in public and private repositories. The {ref} can be a SHA, a release tag, or a branch name.

The best security practices for this are outlined in GitHub Docs on Security Hardening for GitHub Actions.  When a reusable workflow is triggered by a caller workflow, the github context is always associated with the caller workflow. The called workflow is automatically granted access to github.token and secrets.GITHUB_TOKEN. For more information about the github context, see Contexts.  I’m not sure what this implies yet, but it seems to basically be saying that a public repo now has access to the private repo’s actions’ permissions, so be wise.

Passing Inputs and Secrets: For inputs, the data type of the input value must match the type specified in the called workflow (either boolean, number, or string). Workflows calling reusable workflows in the same organization or enterprise can use the inherit keyword to implicitly pass the secrets, as detailed in GitHub Docs on Passing Inputs and Secrets to a Reusable Workflow.

With the above outline, I can leverage our self-hosted runner in our private repository for workflows in our public repository, using workflow_call and properly configured PAT in our secrets. This approach ensures secure and efficient workflow management across different repository types within our organization.
Workflow File Example
Notification-api (public repo) Workflow proposal:
name: Trigger Regression Tests

on: [push, pull_request]  # Define your trigger events

jobs:
  call-regression-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Call Regression Testing Workflow
        uses: department-of-veterans-affairs/notification-api-qa/.github/workflows/run-regression-tests.yml@master
        with:
          environment: ${{ input.env }}'
          branch: ${{ github.ref }}  # Use the current branch
        secrets:
          VAEC_AWS_ACCESS_KEY_ID: ${{ secrets.VAEC_AWS_ACCESS_KEY_ID }}
          VAEC_AWS_SECRET_ACCESS_KEY: ${{ secrets.VAEC_AWS_SECRET_ACCESS_KEY }}
          VAEC_DEPLOY_ROLE: ${{ secrets.VAEC_DEPLOY_ROLE }}
          QA_REPO_ACCESS_TOKEN: ${{ secrets.QA_REPO_ACCESS_TOKEN }}

The QA_REPO_ACCESS_TOKEN is used to authenticate access to the private repository.
The environment and branch inputs are passed to run-regression-tests, as it needs..

Security and Best Practices

The use of QA_REPO_ACCESS_TOKEN ensures secure access to the private repository.
I didn’t end up using the secrets.inheret because in this case it makes more sense to be explicit.

Yea, the peter-evans repository-dispatch was what I was looking at, too. i'll re-read my note to see if I actually found something that did not require a 3rd party workflow.

No, no, I think my original format was correct.  An example from the same documentation (Again, I'm concerned about how the PAT is going to be used):
jobs:
  reusable_workflow_job:
    runs-on: ubuntu-latest
    environment: production
    steps:
    - uses: actions/labeler@v4
      with:
        repo-token: ${{ secrets.envPAT }}
        configuration-path: ${{ inputs.config-path }}

Time to make a POC in my own repo!

Okay, I keep getting this error:
Invalid workflow file: .github/workflows/test-workflow-call-pvt.yml#L8
error parsing called workflow
".github/workflows/test-workflow-call-pvt.yml"
-> "ldraney/private-repo/.github/workflows/private-workflow.yml@master"
: workflow was not found. See https://docs.github.com/actions/learn-github-actions/reusing-workflows#access-to-reusable-workflows for more information.

Here's my code in my public repo:
name: Call Private Repo Workflow

on:
  workflow_dispatch:

jobs:
  reusable_workflow_job:
    uses: ldraney/private-repo/.github/workflows/pizza-workflow.yml@master
    secrets:
      envPAT: ${{ secrets.envPAT }}

here's the code in my private repo:
name: Hello World Workflow

on:
  workflow_call:
    secrets:
      envPAT:
        required: true
  workflow_dispatch:

jobs:
  say-hello:
    runs-on: ubuntu-latest
    steps:
      - name: Print Hello World
        run: |
          echo "Hello World! The workflow has been successfully triggered."

Which isn't too surprising at this point :(.  The documentation says:

Configure the repository to allow access to workflows in other private repositories.

Also, the settings explicitly say:

I'm going to make sure by doing the same set up with a private repo instead of public, won't take long.
Then, since workflow_call doesn't work, the next best result in our timeframe would be to provide a link in the workflow to the regression suite; but this won't report the passed/failed regression.  This is what I'm thinking about.

Cris and I decided to approach the repository dispatch option before approaching a second self-hosted runner.  I will follow the model outlined in this third-part action in my own repos first to see if the end result is what we want.  If so, we will likely use the third-party action as a model for proper use of the respository_dispatch trigger, as it seems the marketplace version isn't much more than a template configuration.

Just recording here that I got a successful repository_dispatch hello world with the following code in my personal repos:
public repo:
name: Call Private Repo Workflow

on:
  workflow_dispatch:

# jobs:
#   reusable_workflow_job:
#     uses: ldraney/private-repo/.github/workflows/private-workflow.yml@master
jobs:
  trigger-private-workflow:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger workflow in private repo
        run: |
          curl -XPOST -H "Authorization: token ${{ secrets.envPAT }}" -H "Accept: application/vnd.github.v3+json" \
          -d '{"event_type": "trigger-private-workflow"}' \
          https://api.github.com/repos/ldraney/private-repo/dispatches

private repo:
name: Hello World Workflow

on:
  workflow_call:
    secrets:
      envPAT:
        required: true
  workflow_dispatch:
  repository_dispatch:
    types: [trigger-private-workflow]

jobs:
  say-hello:
    runs-on: ubuntu-latest
    steps:
      - name: Print Hello World
        run: |
          echo "Hello World! The workflow has been successfully triggered."

Some thoughts... This is good work. We will meet in the morning to discuss

have the workflow wait until the regression is done (likelywith the gh cli)
meanwhile, has a link to the currently running workflow
Once regression completes, the action recognizes either a passed or failed regression (likely with the gh cli)
retrieve an artifact URL from the QA repo regression output if user wants to see regression results.

Let's look into this trigger-workflow-and-wait action as well, though.
For your step #4, we might need to create that HTML report and use that as the artifact.

Got the first part of the above working with:
public repo:
name: Call Private Repo Workflow

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment (dev, staging, or perf)'
        required: true
        type: string

jobs:
  trigger-private-workflow:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger workflow in private repo
        run: |
          curl -XPOST -H "Authorization: token ${{ secrets.envPAT }}" -H "Accept: application/vnd.github.v3+json" \
          -d '{"event_type": "trigger-private-workflow", "client_payload": {"environment": "${{ github.event.inputs.environment }}"}' \
          https://api.github.com/repos/ldraney/private-repo/dispatche

private repo:
name: Hello World Workflow

on:
  repository_dispatch:
    types: [trigger-private-workflow]

jobs:
  say-hello:
    runs-on: ubuntu-latest
    steps:
      - name: Print Hello World
        run: |
          echo "Hello World from ${{ github.event.client_payload.environment }}! The workflow has been successfully triggered."

So I'll now check out trigger-workflow-and-wait before syncing with Cris.

Successful run with the following code:
public repo:
name: Call Private Repo Workflow

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment (dev, staging, or perf)'
        required: true
        type: string

jobs:
  trigger-private-workflow:
    runs-on: ubuntu-latest  # specify the runner
    name: first job
    steps:
    - name: Trigger Workflow
      uses<EMAIL_ADDRESS>      with:
        owner: ldraney
        repo: private-repo
        github_token: ${{ secrets.envPAT }}
        # github_user: github-user
        workflow_file_name: private-workflow.yml
        ref: master
        wait_interval: 1
        client_payload: '{"environment": "${{ github.event.inputs.environment }}"}'
        propagate_failure: true
        trigger_workflow: true
        wait_workflow: true

Me and Cris successfully got the new action to work!
But, it's a third party action.  Thankfullly, its only seven files, and has a lenient MIT licensing that Cris will handle the dynamics of.  So, i'll be moving this all over to notification-api... after I run one more thing setup -- getting a link to the exact regression that ran with this deployment.

Hey Cris,
I’ve made branch 1526-regression-deploy and updated it with what is working in my public repo.
You can see the changes on the PR here.  You can see I’ve added new files according to the repo we copied here.
probably don’t have to worry about this, but I had some trouble adding these and getting them working in my own repo.  I beat my head against this for hours.  I was extremely frustrated with what should have been a simple copy paste.  I’m 90% sure once a repo is git cloned, it is then registered as a submodule, and i had to rename the directory i cloned it into to get it working in my public repo.  You’ll like have to do something similar.  The new files appear on the draft PR, so again you probably don't have to worry about this.
Things left to do:

Test the workflow and make sure it works.  Since the workflow is not on master, be sure to select it from our branch:

master, in this case, means branch of notification-api that will be tested.
As you will see in the run-regression.yml, this input for the branch will influence which branch of the QA repo, in other words, which version of the regression will run.

MIT stuff, as you mentioned.
Make sure things are as you like.  I believe this will be working solution!  Here’s hoping for smooth testing.

some notes:

Workflow_dispatch means there will be an interface for the workflow like the one above.
Workflow_call is what enables the run_regression to appear as its nice own block on the actions workflow progress graphic \

today was consumed with testing Ed's lambda changes and then refactoring the QA repo to use the new secrets.
i will review what Lucas has and see if there is anything I can do from here.

As you will see in the run-regression.yml, this input for the branch will influence which branch of the QA repo, in other words, which version of the regression will run.

Turning my attention back to this - if what Lucas said above is true, then this will not work. In the API workflows (actions) the branch is meant to indicate the branch of the API that gets deployed, not what branch in the QA repo runs. I'll pull the code down and take a look to see if this is true. If it is, I'll try to modify it such that it always runs master in the QA repo (which is its own problem but there you go).

Merged master into this branch to capture the recent changes.
Yes, that workflow passes the branch to the QA repo, which we do not want.
Will run the workflow to see if it even runs the regression. If it does, will then try to modify the thing so that it does not send the branch over to the QA repo.

this is also a problem: FROM alpine:3.15.0 in .github/actions/workflow-trigger-and-wait/Dockerfile We do not want another outdated container.

Trying to run the Deploy to Dev action using the workflow from the branch Lucas created resulted immediately in

Invalid workflow file: .github/workflows/dev_deploy.yml#L71
The workflow is not valid. .github/workflows/dev_deploy.yml (Line: 71, Col: 11): Input branch is required, but not provided while calling.

commented out these lines from run-regression.yml
on:
  workflow_call:
    inputs:
      environment:
        description: 'Notification-api deployment environment to test (dev|perf|staging)'
        required: true
        type: string
      # branch:
        # description: 'Will run latest SHA from branch'
        # default: 'master'
        # required: false
        # type: string
  workflow_dispatch:
    inputs:
      environment:
        description: 'Notification-api deployment environment to test (dev|perf|staging)'
        required: true
        type: string
      # branch:
        # description: 'Will run latest SHA from branch'
        # default: 'master'
        # required: false
        # type: string

and hardcoded the branch of the regression suite to run
jobs:
  trigger-regression-tests:
    runs-on: ubuntu-latest  # specify the runner
    name: Trigger Regression Tests
    steps:
    - name: Trigger Regression Workflow
      uses: ./.github/actions/workflow-trigger-and-wait
      with:
        owner: department-of-veterans-affairs
        repo: notification-api-qa
        github_token: ${{ secrets.envPAT }}
        workflow_file_name: regression.yml
        ref: ${{ github.event.inputs.branch }}
        wait_interval: 10
        client_payload: '{"environment": "${{ github.event.inputs.environment }}", "branch": "master"}'
        propagate_failure: true
        trigger_workflow: true
        wait_workflow: true

and now the action is at least deploying the api to dev

Failed to trigger the regression:

Probably because I deployed master and master doesn't have any of that stuff. So, good thing I yanked the branch out of the call over to run regression since the QA suite wouldn't know what to do with that branch anyway.
Redeploying with the branch set to the 1526 branch.

also, those failures are not being reported back to Slack, looks like only setup-env-{env} and deploy-api steps get reported out to slack?

Same result on Trigger Regression Workflow, even when deploying from the workflow and the branch.

Looking into error.

Progress? Adding
jobs:
  trigger-regression-tests:
    runs-on: ubuntu-latest  # specify the runner
    name: Trigger Regression Tests
    steps:
    - uses: actions/checkout@v4 # added this line

Does get farther but the new error is
Error: Github token is required. You can head over settings and
under developer, you can create a personal access tokens. The
token requires repo access.

So, yea, need to add the PAT.  Holding off until speaking with Kyle.

Confirmed that I do not have access to create a PAT for the QA Repo, so will wait and talk to Kyle.
Also cleaned up some edges in the PR.

Spoke with Kyle:

OK to use PAT since we don't really have a choice, when ready Kyle will set that up (I do not have permissions)
Kyle needs to know the explicit permissions to set on the PAT
NOT OK to use the currently copied action code. We need to go back to the CURL solution, make sure we get some output to return in the public workflow (link to the QA private workflow showing the results or to an HTML report artifact

Given that, marking this ticket as blocked until Lucas returns and dropping it back into the TODO column. I will sync with Lucas when he returns next week.
cc / @ldraney @npmartin-oddball @mjones-oddball @k-macmillan

I am able to make a fine-grained PAT that the dept of veteran's affairs owns.

The comment shows the code that uses CURL. If I have time I'll see if I can't get that going with a PAT Kyle creates.

FRIDAY UPDATE
I have a proof of concept working in my public and private personal repos that is documented here.
Kyle and Lucas have access to my private repo now and both are linked in that document.
The fine-grained permissions are also described in that document.
Lucas uses the CURL method while mine uses a GitHub scripting action (from GitHub and verified). The latter may be more extensible given that it is a script and there is a ping/pong method described in the document that might also be of use.
I will get with @ldraney on Monday to review our options and consider next steps.

PAT made and added to the API repo that has access to the QA repo.

Me and Cris had a meeting to go over the details.
My POC copied a bash script from an existing public action, but Cris reported that was shut down by Kyle due to... security concerns?  The script seems safe enough to me, so may ping Kyle about that.
Cris has a cleaner approach though (as there are things in the script that don't directly pertain to this ticket), so I will first respect Kyle's input and see if I can get the same results in her POC repos.

Still just letting this sit.  Cris did some cool things with her POC.
but, in summary, all thats left to figure out it:

Public repo needs to detect failure in called workflow
Public repo should get a beautiful, easy to see report from the private repo’s regression

There’s nothing in github-script that offers what we need, so next step is probably to look closer at what I originally copied over to get the functionality that we want;
So essentially, I'll look at that copied action's script and re-write it specifically for our purposes... and then see how/if I'll implement it with what Cris already has.

I just made a branch on Cris’ POC repo to play with her workflow file.
The final solution will be a renamed regression-public.yml to run-regression.yml in notification-api repo.  It will have a nice looking action step that looks something like this (shoutout to Cris for refining this):
    - name: Trigger Regression Workflow
      uses: ./.github/actions/workflow-trigger-and-wait
      with:
        owner: department-of-veterans-affairs
        repo: notification-api-qa
        github_token: ${{ secrets.envPAT }}
        workflow_file_name: regression.yml
        ref: ${{ github.event.inputs.branch }}
        wait_interval: 10
        client_payload: '{"environment": "${{ github.event.inputs.environment }}", "branch": "master"}'
        propagate_failure: true
        trigger_workflow: true
        wait_workflow: true

Which will use the action.yml and trigger_script.sh which reside in the workflow-trigger-and-wait directory, as shown above.
Hypothesized Next Steps

Finalize trigger_script.sh:

Implement the logic to continuously check the workflow's status until completion.
Handle the extraction of the workflow run URL and the final status (success or failure).
Go over any needed error handling and/or security practices in the script.

Update action.yml:

Define the inputs and outputs for the custom action, matching it with the trigger_script.sh.
Ensure the action metadata aligns with the GitHub Actions requirements.

Testing in the POC Public Repository:

Deploy the changes to the public repository and test the entire workflow.
Monitor the actions dashboard in both hello-world and hello-universe to verify the integration behaves as expected.

Test in the POC Public Repository:

Hopefully just a few renaming of script inputs is all that is needed here.

And, we’ll get to that point. It might be ready to open for a PR after another discussion with Cris at that point.

PAT requires a permissions update (will inform @k-macmillan when we get the POC working).

Just trying to figure out why these curl commands worked in my and Cris's POC, respectively:

the one i had working that i borrowed, and was working

response=$(curl --fail-with-body -sSL \
      "${GITHUB_API_URL}/repos/${INPUT_OWNER}/${INPUT_REPO}/actions/$path" \
      -H "Authorization: Bearer ${INPUT_GITHUB_TOKEN}" \
      -H 'Accept: application/vnd.github.v3+json' \
      -H 'Content-Type: application/json' \
      "$@")

The one cris had working

          const result = await github.rest.repos.createDispatchEvent({
            owner: 'cris-oddball',
            repo: 'hello-world',
            event_type: 'run_regression',
            client_payload: {"environment": "${{ inputs.environment }}"}
          })
          console.log(result);

The new one I try to write

  response=$(curl -L -sSL -w "%{http_code}" -X "$method" \
    -H "Accept: application/vnd.github+json" \
    -H "Authorization: Bearer ${GITHUB_TOKEN}" \
    -H "X-GitHub-Api-Version: 2022-11-28" \
    "${GITHUB_API_URL}/repos/${OWNER}/${REPO}/actions/$path" \
    -d "$data")

But, there may be something else at play here, as I'm not sure all the differences between Cris's POC and my own.  Hers wasn't using an action.yml or a dockerfile, for example.

@ldraney
some end of day thoughts for you before you start any work tonight:
Repository Dispatch

In hello-universe, trigger-private-workflow.yml uses a job, run-regression, to make a GET request to hello-world at https://api.github.com/repos/cris-oddball/hello-world/dispatches

The job uses actions/github-script@v6 and awaits the response from github.rest.repos.createDispatchEvent. ←- this is from OctoKit!  Read about on this page, scroll down to (or search for) “Create a repository dispatch event”.
octokit.rest.repos.createDispatchEvent({
  owner,
  repo,
  event_type,
});

Except we can just use github.rest.repos.createDispatchEvent because GitHub must use OctotKit or be based on it or something since my workflow worked right out of the box.
In turn, documentation on that REST API endpoint is here. The only required body parameter is event_type. In this parameter, we enter the respository_dispatch type that we want to invoke, here as run_regression. The client_payload is optional, but we use it to send the environment over to the regression.
The workflow in hello-world, run-tests.yml, has a repository dispatch section with a type of run_regression. It seems like you could add that event_type to any workflow in the private repo and that webhook call above will invoke any worfklow that contains it, because I are not even bothering to describe the name of the private workflow in my public workflow.
Our problem is that we want to get something back from hello-world to let us know that our run was successful, be that the output, a link to the action, an artifact - something.
Workflow Dispatch
Lucas’ action uses a workflow dispatch (I think?). Happily, OktoKit also provides something for that and is described on this page, scroll down to (or search for) “Create a workflow dispatch event” on that page.
So, we can use this to await the response using github.rest.actions.createWorkflowDispatch which invokes a GET request to https://api.github.com/repos/OWNER/REPO/actions/workflows/WORKFLOW_ID/dispatches - See GitHub documentation here.
Note that path parameters owner, repo and workflow_id are required. Body parameter ref is also required (so, we should not have taken it out) and it needs to be a branch or tag name. For now, the ref should just be hard-coded to main (for these POC repos) or master (for the real repos).
octokit.rest.actions.createWorkflowDispatch({
  owner,
  repo,
  workflow_id,
  ref,
});

At this point, I think we need to do a few things:

Drop the idea of reusable workflow at this point. That is like the advanced version of what we want to do. First let’s get the thing working using specific values just for the regression, then later we can determine how a reusable workflow can benefit us.
Decide on workflow dispatch or repository dispatch. Frankly, the later seems sufficient to me but I’d like to understand the difference and why Lucas prefers the former.
Determine how exactly that bash script enables returning something usable from the private repo and just incorporate it into either the repository dispatch or the workflow dispatch.

Summary of progress:
We spent some time figuring out where the bug was, but ultimately we were trying to modify and even build from scratch (via bash) what we needed, but the bug we are experiencing is somewhere in the nuance between using bash in a github action context; Cris found a better approach for our problem and future problems, which is with the action/github-scripts and the doors it opens with its validation and prebuilt functionality.  It solves the problems I’m facing of actually making the call and request validation and github action nuances, and enables a more focused approach regarding communicating with github’s REST API.
Next steps:
So, my next goal is to extend upon Cris’s implementation and learning with a new POC that builds on her work; again, with the critical point being communication between our public repo to a private repo and getting useful regression feedback in the public repo output.  Next steps are likely:
Get an understanding of the interactions with the GitHub API via Octokit and/or maybe a javascript module, focusing on maintainability, readability, error handling, and reusability. This will be done with a documentation overview in reference to the code Cris is already using, and how the documentation recommends extending its functionality.
Also, rest api documentation includes the javascript code examples for each route
Design a method to monitor the triggered workflow's status and output
May involve this API documentation for
downloading workflow_run attempt logs
Get workflow run attempt
Download workflow run logs
Get a workflow run
Presenting the results in a clear and useful manner within the GitHub Action logs and Summary page
I’ll update here as I make progress, but the above is my current POA.

I made some progress on using github-script, and have the foundation for a working POC in cris-oddball/hello-universe (mimicking our public notification-api repo) which calls the cris-oddball/hello-world (mimicking the private regression repo).
My changes successfully call and wait for the regression to run!   Also, if the called action fails, it causes our public repo workflow to fail--which is very good!  Now, the last two necessities are to get the actual output from the called private repo workflow, and I also want a direct link to the workflow that was called.  We'll also want a nice workflow summary with this info and link as a part of the final product.

@k-macmillan we will need an update to the PAT - @ldraney has the info, he will need the name of that PAT. Thanks!

Things are looking so good!  We just need to make sure we have the right name of that PAT.  I've either got a permissions error or a no input provided error for DISPATCH_PAT and envPAT, respectively.
But I expect a successful run after that.

I've added this workflow_call to the relevant workflows as well, so those will be ready to test once we get our first manual test running.  Again, gotta figure out what is going on with this PAT.  I'll be surprised if its something else.

I can’t get past this permissions issue. We need to open up one of the permissions or something.
Relevant docs and webpages:

"Resource not accessible by personal access token" error documentation
Github community discussion - Github permissions force you to give write permissions for this situation - stack overflow
Documentation for required permissions for workflow_dispatch
PAT permissions documentation - maybe there are implications for using a self-hosted runner with this action?

i tested this and nope and this problem is with other workflows as well, even when there are no inputs involved.

Hotfix merged and the workflow run handles the regression.
Lucas and Kyle to meet to discuss next steps. Leaving ticket to sit here in the meantime.

Lucas and Kyle decided to leave the merge in master and to create a new ticket to look into using type: environment
Removed the user stories for sending messages to the Slack channel since that work was not performed.
Tested by pushing to perf - passed
Will test the deploy to dev action
Cannot test the pre-release to staging until the pre-release is created next Tuesday.

Pre-release to staging failed.
The workflow is not valid. .github/workflows/prerelease_trigger.yml (Line: 32, Col: 13): Job 'run-qa-suite' depends on unknown job 'run-deployment'.

line 32 should be needs: [pre-release]
Alerted @ldraney and awaiting another hotfix.

Hotfix has two approvals but the unit tests are failing (not because of this PR). Once that is cleaned up and it's merged, this can be pre-released to staging to test.

Pre-release sent to staging, awaiting the regression run.

staging pre-release failed due to the api deploy timing out after 30 minutes. re-running that job.

Kyle cut a hotfix for bumping the CPU on those tasks. Created a 3rd pre-release to staging that includes this so that deployments don't take forever or die.

sigh. send the wrong tag. sending the correct tag as a 4th pre-release to staging .

finally: success!

closing ticket.


setup security policy for every project and cookiecutter

in form of a SECURITY.adoc like the one that github's ui generates in e.g. https://github.com/JonasPammer/cookiecutter-pypackage-test/security/policy
also document the maintenance need of this of this file in release/version documentation in development.adoc
for my projects<EMAIL_ADDRESS>can be used for reporting vulns, i look into it
more info on this practice may be found in e.g. the following ossf scorecard issue scan result https://github.com/JonasPammer/cookiecutter-pypackage-test/security/code-scanning/60

When creating a "draft security advisory" for a project (under Security (top) -> Security advisories (left) -> New draft security advisory) github pre-fills the "description" segment of it with the following text:
### Impact
_What kind of vulnerability is it? Who is impacted?_

### Patches
_Has the problem been patched? What versions should users upgrade to?_

### Workarounds
_Is there a way for users to fix or remediate the vulnerability without upgrading?_

### References
_Are there any links users can visit to find out more?_

### For more information
If you have any questions or comments about this advisory:
* Open an issue in [example link to repo](http://example.com)
* Email us at [example email<EMAIL_ADDRESS>
Other fields of an Security Advisory include:

Affected product(s)
** Ecosystem (GitHub Actions, npm ..)
** Package name
** Affected version
** Patched version
Severity (Low, Moderate, High, Critical, Asses using Common weakness enumarator CWE)
CVE Identifier (Request ID later, i have an existing one)
Title

Posting this here as this may help form the contents of SECURITY.adoc i.e. with a boilerplate text of the above

When creating SECURITY.md from GitHub UI, this boilerplate is given:
# Security Policy

## Supported Versions

Use this section to tell people about which versions of your project are
currently being supported with security updates.

| Version | Supported          |
| ------- | ------------------ |
| 5.1.x   | :white_check_mark: |
| 5.0.x   | :x:                |
| 4.0.x   | :white_check_mark: |
| < 4.0   | :x:                |

## Reporting a Vulnerability

Use this section to tell people how to report a vulnerability.

Tell them where to go, how often they can expect to get an update on a
reported vulnerability, what to expect if the vulnerability is accepted or
declined, etc.

TIL, thanks to https://mailtolink.me/ (project available on github which i just starred for giving me this knowledge and being very to the point but modern and nice looking page and OSS), mailto links can have an very extensive body and there is an app that can generate these mailto links. so not only can i make a ``` with the boilerplate but also make the mailto link automatically fill in the boilerplate (but beware: don't only do one thing. some may click on the mailto, some may not and just go ahead writing email. some mail programs will understand it, some not. so still have normal ```)

closed as per above PRs


Strange entry when using fzf-tmux

When I call fzf-tmux there is strange first line input:
Ptmux;]4;0;rgb:2b/30/3b\Ptmux;]4;1;rgb:bf/61/6a\Ptmux;]4;2;rgb:a3/be/8c\Ptmux;]4;3;rgb:eb/cb/8b\Ptmux;]4;4;rgb:8f/a1/b3\Ptmux;]4;5;rgb:b4/8e/ad\Ptmux;]4;6;rgb:96/b5/b4\Ptmux;]4;7;rgb:c0/c5/ce\Ptmux;]4;8;rgb:65/73/7e\Ptmux;]4;9;rgb:bf/61/6a\Ptmux;]4;10;rgb:a3/be/8c\Ptmux;]4;11;rgb:eb/cb/8b\Ptmux;]4;12;rgb:8f/a1/b3\Ptmux;]4;13;rgb:b4/8e/ad\Ptmux;]4;14;rgb:96/b5/b4\Ptmux;]4;15;rgb:ef/f1/f5\Ptmux;]4;16;rgb:d0/87/70\Ptmux;]4;17;rgb:ab/79/67\Ptmux;]4;18;rgb:34/3d/46\Ptmux;]4;19;rgb:4f/5b/66\Ptmux;]4;20;rgb:a7/ad/ba\Ptmux;]4;21;rgb:df/e1/e8\Ptmux;]10;rgb:c0/c5/ce\Ptmux;]11;rgb:2b/30/3b\Ptmux;]12;rgb:c0/c5/ce\lib/tasks/recreate.rake

Any idea what can be wrong? I can live with that, but it is kind of irritating.

I never had the problem. Please provide the exact steps to reproduce the issue.

I am not sure what is the issue. My TMux version is tmux 2.1.
It seems that this is issue with fzf-tmux itself as ag --nocolor -g "" | fzf-tmux works correctly.

I also use tmux 2.1 and standalone fzf-tmux works nicely. Do you have FZF_DEFAULT_COMMAND set? What about FZF_DEFAULT_COMMAND= fzf-tmux, or FZF_DEFAULT_COMMAND="seq 100" fzf-tmux?

All of them returns that strange line. What can be seen is that only first entry is mangled. What is stranger currently it returns valid entry when select the mangled one, so:
env FZF_DEFAULT_COMMAND='seq 1 100' fzf-tmux
<Enter>
1

I am using Fish 2.2.0 with TMux 2.1.

It seems that fzf is also affected.

Seems that this is problem with base16 thoughtbot/vim-rspec#91.
Closing as non related.

This solves problem: chriskempson/base16-shell#48


Inject can't observer removed data

Online demo: https://codepen.io/kwzm/pen/MxPMGK?editors=0010
I found just used inject could not observer data been removed, the component could not re-render.
But if you add observer everything is ok.
Ps: I used a func in inject, so it will auto add observer.

inject does not automatically apply observer, so the correct definition is, const App = inject((store) => ({data: store.data}))(observer(List)), which fixes the issue.
The observer you are seeing in the source is the observer of the wrapping HOC component, that runs the inject function (which might also be reactive), but it is not the child component itself, which needs to be an observer as well if is consuming observables.

@mweststrate Ok, I see. Thank you.


Fix BotConnector solution

Fix tests and bot connector sample (#126)
Change tests project target framework to netcoreapp2.0

Thanks Guys! This looks great.


docker build cache hack not working anymore

yeah it is it just keeps changing since we're reving packages a lot.


Ensure that DateTimePattern is stringified as its pattern

What: DateTimePattern is casted to string by it's own JVM identifier, but it should be the pattern.
Why: Several places seem to be replacing the (datetime) pattern with an actually value
Checklist:

[ ] Documentation added to the README.md OR link to PR on https://github.com/specmatic/specmatic-documentation N/A
[ ] Tests
[ ] Sonar Quality Gate

Issue ID:
Closes: #580
I've been trying to find a nice place to test this, but it seems to fall a bit between the cracks of the test-suite. Any pointers on where it would be appropriate to add tests would be very appreciated.

@androa Thanks for your Pull Request. I think you should be add a test like this to UrlMatcherTest.kt
@Test
    fun `should stringify date time query param to date time pattern`() {
        val urlMatcher = URLMatcher(mapOf("before" to DateTimePattern), pathToPattern("/pets"), "/pets")
        assertThat(urlMatcher).isEqualTo("/pets?before=(datetime)")
    }

One you have added this test and once the build is passing we will test the changes and merge the PR. Please let us know if you have any questions on this.

Thank you @harikrishnan83 for the help! Added the test case (it needed to be converted to string to satisfy the equal check). Works perfect.
Also tested it with the repro YAML from the issue and it generates samples and stubbing perfectly now.


Problem: (Fix #136) old Ledger app version used in desktop wallet and cli tutorials

Fix #136 by updating the Ledger app version in the tutorials.

/staging

Also removing -- "Crypto.org Chain Ledger Application Is In Development Mode"


Creating multiple style files per scope vars

Hello, thanks for this plugin again.
    themePreprocessorPlugin({
      scss: {
        multipleScopeVars: [
          {
            scopeName: 'theme-primary',
            path: path.resolve('src/theme/primary-vars.scss'),
          },
          {
            scopeName: 'theme-client1',
            path: path.resolve('src/theme/primary-vars.scss'),
          },
          {
            scopeName: 'theme-client2',
            path: path.resolve('src/theme/primary-vars.scss'),
          },
          {
            scopeName: 'theme-client3',
            path: path.resolve('src/theme/client3-vars.scss'),
          },
        ],
        defaultScopeName: 'theme-primary',
        extract: true,
      },
    }),
  ],

This doesn't build .css files for every theme, as I get: GET /assets/theme-client2.css net::ERR_ABORTED 404 (Not Found).
Am I doing it wrong, or should I create multiple files with different paths? (with symlink for example)

It working well in my demo.  https://github.com/GitOfZGT/dynamic-theme-demos .
@zougt/vite-plugin-theme-preprocessor v1.4.4  is  latest version.

This doesn't build .css files for every theme, as I get: GET /assets/theme-client2.css net::ERR_ABORTED 404 (Not Found). Am I doing it wrong, or should I create multiple files with different paths? (with symlink for example)

I was wrong.
Yes, you need  different paths.  and different  SASS var values.

If you have to, you can try to adjust the order:

Would it be possible if every multipleScopeVar would generate their own css-file?
Consider my use case: 3 different companies use my application. Company 1 and Company 2 have unique themes. Company 3 uses the same theme as Company 1.
I can say this in my JavaScript client bundle:
if (route.params.company === "Company 3") {
  useTheme("Company 1")
} else {
  useTheme(route.params.company);
}

But note; in this case my client bundle would leak customer information for everyone to see. If someone were to look at my client source code, they would know that I have Company 1 and Company 3 in my application.
If every multipleScopeVar was to generate their own css-file, I could just say:
useTheme(route.params.company);

That being said; I can also work around this issue by creating symlink. If you think this is not worth implementing, the symlink way is also easy.

First file must different others.  It working.


[doc]Some suggestions on documentation

在单机上部署的时候，资源中心的文档部分写的再详细点

For better communication, please use English. Thx.

For better communication, please use English. Thx.

which part？

which part？

Regarding the documentation, which part of the documentation you think is needed, or which documentation is not complete, you can reply here ISSUE. Your suggestions are welcome

Regarding the documentation, which part of the documentation you think is needed, or which documentation is not complete, you can reply here ISSUE. Your suggestions are welcome


Using the library on web

Hi Folks,
I have a project which is using common code for native and web. I am trying to use this library to pull the userAgent information on the 3 platforms - iOS, Android and Web.
My project has been setup to use 'metro-react-native-babel-preset'.
When I run the code i get the following error -
Module../node_modules/react-native-device-info/lib/module/internal/nativeInterface.js
TypeError: Cannot assign to read only property 'exports' of object '#'
5 |
6 | // @ts-ignore
7 | if (Platform.OS === 'web' || Platform.OS === 'dom') {

8 |   RNDeviceInfo = require('../web');
9 | }
10 |
11 | if (!RNDeviceInfo) {

Is there any configuration I am missing to use this on web?
thanks,
mohammed

@m0hamm3d-ac sorry - this library under my maintainership has not had a good web focus at all, as I don't use it that way personally yet so I sometimes miss that it is completely broken even. If you'll look at the PR queue here you'll see there are a couple from @robertherber which look great and are on-deck for merge but I haven't had time yet and am actually traveling now for the next week, so won't have time for a while either. If you could check them to see if they address your concern and/or potentially collaborate with Robert, I'm definitely to anything that helps with the web support, including taking on anyone that is willing to commit to it as a maintainer here - the more people helping the merrier

@mikehardy  thanks for your response. I will have a look at @robertherber 's PRs and see if I can fix the issue and raise a pull request. I will close this issue and add an update if required.

I am having same issue now.


Building with local dependencies is broken In recent updates.

I recently updated this tool from version 0.1.52 to 0.1.59. Before the update, I could have something like this:
[dependencies]
local = {path="./local_path"}

Then, after updating the code in local_path, v0.1.52 of cargo chef cook would not trigger a full rebuild; it would just rebuild the local dependency.
After updating to 0.1.59, I get an error telling me that local_path/Cargo.toml has not been found. Fixing this in the Dockerfile would trigger full rebuilds on each update of local_path, which would be a huge pain for my project.
It seems like 0.1.59 is actually attempting to build the full local dependency, instead of waiting until cargo build is called. Is that right? Shouldn't it only build a dummy version of it with remote dependencies?
P.S.: My repo is private, so I can't point to a concrete example. For now, switching back to 0.1.52 works for me.

Is the failing local crate some kind of vendored dependency?

This PR seems to disable the local deps.

I know that's the change that caused the issue, what I'm trying to understand is why you're experiencing it.
Is this crate part of the workspace you are using cargo-chef for?
A minimal reproducible case is needed to triage.

Thanks for looking into it so quickly. The crate is not part of the workspace. Here is rough structure of my app:
myapp/
......src/main.rs
......Cargo.toml
......Cargo.lock
......external_crates/external_crate_A/
.......................................src/main.rs
.......................................Cargo.toml

Here is the root Cargo.toml
[package]
name = "myapp"

[dependencies]
...
external_crate_A = { path = "./external_crates/external_crate_A" }
...

OK, then you need to copy it in before executing cargo chef cook.

The extra COPY does fix the issue. Thanks!
But why didn't we need to do the extra copy in version 0.1.52?

The old version of cargo-chef was using file traversals to identify which crates should be considered and "emptied out".
This is somewhat error prone, we have therefore switched to a different default: all the crates in the current workspace are considered "local" and will therefore be emptied out, all the other local dependencies are just dependencies and must be copied in (and will be part of the cache, which wasn't the case before).
I'll close this as resolved, let me know if you bump into other issues.


[Parse] Don't error on overloads in a conditional compilation block

What's in this pull request?
Modifies the parser to not throw an error on function overloads in a conditional compilation block. Previously, an error was thrown on the following:
class MyClass {
   #if NOT_NOW
   func hello(one: Int, two: Int) {
   }
   func hello(one: Int, three: Int) {
   }
   #endif
}

This was because the conditional compilation block was being treated as a braced scope, which is a resolvable scope, and therefore it was hitting a code path intended for shadowed variables.
Note: I'm not entirely sure I understand what it means by a "resolvable" scope, and I don't understand the parser very well, so I'm mostly just assuming here that Parser::parseDeclIfConfig is only ever invoked when the current scope is non-resolvable.
Resolved bug number: (SR-826)

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    OS X platform               | @swift-ci Please test OS X platform
    Linux platform              | @swift-ci Please test Linux platform

Note: Only members of the Apple organization can trigger swift-ci.

@swift-ci please test

I think the right thing to do here is take the parent's scope.  Something like:
diff --git a/lib/Parse/ParseDecl.cpp b/lib/Parse/ParseDecl.cpp
index 21f277e..e5f059c 100644
--- a/lib/Parse/ParseDecl.cpp
+++ b/lib/Parse/ParseDecl.cpp
@@ -2662,7 +2662,8 @@ ParserResult<IfConfigDecl> Parser::parseDeclIfConfig(ParseDeclOptions Flags) {

     Optional<Scope> scope;
     if (!ConfigState.isConditionActive())
-      scope.emplace(this, ScopeKind::Brace, /*inactiveConfigBlock=*/true);
+      scope.emplace(this, getScopeInfo().getCurrentScope()->getKind(),
+                    /*inactiveConfigBlock=*/true);

     SmallVector<Decl*, 8> Decls;
     if (ConfigState.shouldParse()) {

I can continue this patch through another pull request or you (@kballard) can take it and run with it.

I considered trying to adopt the parent's scope, but I was a little concerned about the fact that I don't understand SavedScope or delayed processing and wasn't sure if there was any subtlety there that would affect this. But I'm probably worrying over nothing. This evening I'll try that approach and see if it works.

@meadori PR updated with your suggestion. It passes the normal test suite on my machine.

LGTM.

@swift-ci please test

Yes, this looks good. Thanks!


Add API doc. Use autosummary to generate docs for each module.

This PR enables the automatic generation of API docs. It uses the autodoc and the autosummary extensions.
New top-level modules have to be added to docs/sphinx/api/sphinx_polyversion.rst.

Awesome! Would it be possible to implement the following two stylistic choices?

[ ] Exclude __init__ from the docs. It is already documented in the class description.
[ ] Document items of a module on the same webpage instead of seperate pages. That is include those below the list of all items in a module.

Thank you very much!


Load code from GitHub repo (into playground)

Embeds currently can load snippets from a GitHub repo, but in some cases it might be valuable to load snippets into the main dart.dev page.
Related issues:

https://github.com/flutter/gallery/issues/264
https://github.com/dart-lang/dart-pad/issues/1763

The main page also isn't able to load from the API Doc server, though the embeds are. Consider adding that to the playground as well, to give them parity.

Closing this in favor of https://github.com/dart-lang/dart-pad/issues/2702#issuecomment-1791322473 which tracks a lighter weight embedding mechanism that supports an "Open in DartPad" feature.


Can't feed my model with my image data tensor

I'm making a vector with full of my tensor images and pass to my model but it doesn't work. I did use your codes. Where is the problem?
Code: https://stackoverflow.com/questions/70947854/can't-feed-my-model-with-my-image-data-tensor please
@alantess

torch::Tensor CVtoTensor(cv::Mat img,int unsqueeze_dim = 0) {
    cv::resize(img, img, cv::Size{ img.rows, img.cols }, 0, 0, cv::INTER_LINEAR);
    cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
    auto img_tensor = torch::from_blob(img.data, { img.rows, img.cols, 3 }, torch::kByte);
    img_tensor = img_tensor.permute({ 2, 0, 1 }).toType(torch::kFloat).div_(255);
    std::cout << "tensor shape: " << img_tensor.sizes() << std::endl;

    img_tensor.unsqueeze_(unsqueeze_dim);
    std::cout << "tensors new shape: " << img_tensor.sizes() << std::endl;
    return img_tensor;
}

I wrote this function again and I did use this functşon to feed the model and it worked!


Linking from complex attributes

The use case
Consider the following cases:

A resource representing a product has a "price" attribute, which is an object of the form: {"amount": ..., "currency": ...}, where the "currency" property refers to a particular currencies resource.

A resource representing a project has a "team-members" attribute, which is an array of objects of the form: {"person": ...., "role-played": ...}, where the "person" and "role-played" properties refer to particular people and roles resources.

In both of the cases above, the attribute mentioned is an object or an array of objects that includes a link to another resource. But, crucially, the values for these attributes are only likely to occur in, or are only relevant in the context of, one resource. That is the (amount, currency) pair that represents a product's price doesn't make sense outside the context of that product, and the (person, role) pair that describes the role of a person on a particular project likely does not need to be considered outside of that project.
Accordingly, there's little semantic or functional justification for making separate, addressable resources from these  "complex attributes"'s values. Meanwhile, turning them into separate resources can have a performance cost for backends implemented with document databases, which allow the complex attributes' values to be stored directly in the main resource's document. Therefore, the JSON-API spec shouldn't mandate that these complex attributes be turned into separate resources. But, if it's not going to mandate that, it has to somehow allow these complex attributes' links to other resources to be expressed in the response.
Prior Discussions
This issue has been discussed many times before, including in #238, #276, and #311. But many of the proposed solutions no longer apply post rc2. Post rc2, though, @bintoro made two proposals that offer good starting points for discussion.
My initial preferred approach
As I said previously, I think @bintoro's first proposal makes the a lot of sense. It's syntactically consistent with the current approach, and it seems easier to implement server-side than splitting up the complex attribute would be. Its downside, of course, is that it makes a parser's search for "links" somewhat more complicated.
An in between option, that keeps it reasonably simple to find links but still keeps a complex attribute's value together, could be the following:
{
  “data”: {
    “id”: “1”,
    “type”: “people”,
    “name”: “Victor Mayonnaise”,
    “links”: [“employer”, “address.city”],
    “employer”: {
      “relationship”: “http://example.com/people/1/links/employer”,
      “resource”:  “http://example.com/companies/2”
      “type”: “companies”,
      “id”: “2”
    },
    “address”: {
      “street”: “12 Mean Street”,
      “city”: {
        “type”: “cities”,
        “id”: “123”
      }
    }
  }
}

Here, the link objects have been treated just like any other attribute value, so links out from simple attributes are now at the top-level the resource. The "links" key just lists an array of paths that hold link objects.
I have a feeling this might get strong pushback, though, as people seem to like the idea of the "links" key holding an object. I like that too, in theory, but I wonder whether, if the alternative is having multiple "links" keys throughout the resource, the above isn't a better compromise. I don't know.
Thoughts?

Great analysis @ethanresnick. I like your "in-between" approach a lot!
One aspect I've been trying to push for is uniformity of representations. It would be very elegant if a client didn't have to parse the same thing in different ways depending on the HTTP endpoint it happens to use. (This principle is not limited to complex attributes. I've brought this up elsewhere, too.)
Complex attributes that have their own links bunch would obviously fit this bill, since they are completely self-contained.
In the in-between proposal, the representation per se isn't uniform (because links switches context), but it doesn't matter. The crucial thing is whether the parsing logic can remain uniform, and your proposal accomplishes this perfectly. The links array will always be at the top level of the resource object, so /person/1 and /person/1/address can be parsed in identical ways.
GET /person/1/address

HTTP/1.1 200 OK

{
  “data”: {
    “links”: [“city”],
    “street”: “12 Mean Street”,
    “city”: {
      “type”: “cities”,
      “id”: “123”
    }
  }
}

Furthermore, a client shouldn't have to care whether address is internally a complex attribute or a linked resource. Requesting /person/1/address should return the same kind of representation regardless. This is also true in your scheme.
Nice job!

@bintoro Thanks!
And I'm glad you're thinking about parsing uniformity. That's been an ongoing concern for many of us—see, e.g., #276—and I think the spec is much better now in this regard. I still haven't taken the time to wrap my head around all the new cases introduced by rc2, but I'm happy to hear that the in between solution appears to work with them.
One thing, though: I'm not sure that GET /people/1/address is actually a response we need to be concerned with. The point with the complex attributes case, as I understand it, is that the data in question really is just an attribute, and not a distinct resource that it would make sense to address separately. So, in the same way we wouldn't have GET /people/1/first-name, I don't think we should have it for address.
The other requests that I'm most worried about are GET /people/1/links and GET /people/1/links/address.city. Here's how I think the in between solution could apply to those:
GET /people/1/links

{
  "data": {
    "links": ["employer", "address.city"],
    "employer": {
      "relationship": "http://example.com/people/1/employer",
      "href":  "http://example.com/companies/2",
      "type": "companies",
      "id": "2"
    },
    "address": {
      "city": {
        "type": "cities",
        "id": "123"
      }
    }
 }
}

A few notes here. First, the "links" key under "data" is arguably unnecessary, given the request, but I've included it for the parsing uniformity benefits. Second, I decided to make an "address" key and then nest a "city" key inside of it. An alternate approach would be:
{
  "data" {
    // ...
    "address.city": {
      "type": "cities",
      "id": "123"
    }
  }
}

I'm not sure which makes more sense.
If we go the nested "links" bags route, I think a similar question arises. Would GET /people/1/links look like:
"data": {
  "employer": { //link object },
  "address": {
    "links": {
      "city": { //link object }
    }
  }
}

Or would it look like:
"data": {
  "employer": { //link object },
  "address.city": { //link object }
}

Now, the second request, GET /people/1/links/address.city. I think this would look exactly the same that requesting a link from a simple attribute would. So GET /people/1/links/address.city is no different in structure than GET /people/1/links/employer. What that structure is, though, needn't be discussed here, since that's already happening in #357.

One thing, though: I'm not sure that GET /people/1/address is actually a response we need to be concerned with. The point with the complex attributes case, as I understand it, is that the data in question really is just an attribute, and not a distinct resource that it would make sense to address separately.

In this particular case, sure, it may never be necessary to target address directly. But more generally, it would be highly desirable to be able to switch from complex attributes to normalized resources (or vice versa) without affecting what the response looks like.
When you think of it this way, a term like "embedded resource" or "subresource" might be more apt than "complex attribute".

The other requests that I'm most worried about are GET /people/1/links and GET /people/1/links/address.city.

I would expect GET /people/1/links to respond with this:
{
  "data": {
    "links": ["employer", "address.city"],
    "employer": {
      "relationship": "http://example.com/people/1/employer",
      "href":  "http://example.com/companies/2",
      "type": "companies",
      "id": "2"
    },
    "address.city": {
      "type": "cities",
      "id": "123"
    }
  }
}

In the "nested links bags" scheme, I would expect any links in the substructures to be completely excluded, simply because they are not, in fact, part of the links object you're targeting. If it's desirable to target all of the links in the entire tree in a single request, then let's scrap the nested scheme right now. My first impression is that I like your proposal better anyway.
By the way, I like that you've included both the relationship-specific resource URL as well as the canonical URL (href) in the link object. I'm trying to accomplish the same thing in #379.

@bintoro Whoops, I totally didn't mean to put relationship and href keys in my payload earlier. Not sure how that happened! I think the discussion in #379 is a good one to have, but for simplicity here I meant to just assume that the structure of a link object stays constant. I've edited my earlier post accordingly.
Re...

But more generally, it would be highly desirable to be able to switch from normalized resources to complex attributes (or vice versa) without affecting what the response looks like.
Can you explain this a bit? As I see it, there are cases where a separate resource makes sense semantically and other cases where what you want to specify is really a complex attribute. But I can't imagine one case changing into another. If we really do start thinking about embedded resources, a la HAL, I think stuff's gonna get messy really fast.

I was thinking of something you said in #341:

by far, the most annoying aspect of the current API I've built on this spec is the extra, normalized resources I've had to create

It feels like an appealing property of a system if the representation of a resource can remain the same when it appears as a part of a larger resource. For example, you might take a snapshot of some independent, linked resource and permanently store it as part of some context. In this respect, the "nested links" route is more complete.
I suppose you're right that actually requesting a wholly dependent subresource directly is mostly just a theoretical concern.
Looking at this again, one thing that occurred to me is that the "in-between" approach has a property that might be dissatisfying to some:
Now:

/people/1/employer doesn't conflict with any actual member called employer.
/people/1/links matches the links structure perfectly.

The in-between model:

/people/1/employer coincides with an employer member yet doesn't match its structure.
/people/1/links no longer reflects the structure of the links member.

So, it kind of breaks the correlation between URLs and the corresponding objects.
I guess the choice between the alternatives depends at least partly on how useful it really is to enumerate all the relationships at the top level of the main resource (see the addendum I edited on my previous comment).

Let me back up a step. The reason I want to keep the concept of a "complex attribute", rather than moving to the notion of a "subresource", is that I think having both the notion of a "subresource" and an "included resource" will get confusing. For example, I think it'll make people ask why a request for GET /people/1?include=employer can't include the "employer" as a subresource within the person's resource object, rather than in a separate "include" key. And I'm not inclined to (re)start all those discussions when we're so close to 1.0.
That said, for consistency and easy parsing, I agree with you that it makes sense to structure "complex attributes" in the same way we structure resource objects. And, I agree with your points about the in-between model breaking a lot of nice correspondences between the URLs and the payload.
So, now, I'm thinking that one option would be to stick with the nested-links approach—which has all the advantages for parsing and url–object correspondence—and simply prevent the concept of a complex attribute from collapsing into the concept of a subresource by saying explicitly that a complex attribute MUST NOT be accessible at its own url, in the same way that simple attributes (like name) aren't.
The only problem I see with this is that, if the complex attribute doesn't have a url, how do you request the link inside of it? I.e., the above strategy implies that /people/1/links would not include an address.city member, in order to keep the correspondence with the "links" structure from the /people/1 payload. But if /people/1/links doesn't include address.city, then how is a user supposed to get the city's link object? The nested-links payload structure would suggest /people/1/address/links/city, but that becomes really awkward if, per the above, we've stipulated that GET /people/1/address shouldn't work.
So what are our other options? We could go back to your second proposal, but I agree that splitting up the attribute is suboptimal.
A radical option would be to simply require the server to serialize the "address" attribute as a series of key-value pairs that live at the same level as the simple attributes, so we'd get "address.street", "address.city", etc. top-level members, rather than having an object live at the "address" key. But I think this creates more problems than it solves.
Alternatively, we could stick with the in-between approach, but change the url of /people/1/employer to /people/1/linked/employer. This could be interpreted as requesting the resource linked to from the link object at the employer member. With this change, none of the urls "under" /people/1 directly map to sub-parts of the /people/1 payload; instead both /type/id/linked and /type/id/links become special namespaces with special behavior. Of course, this throws away the direct url/payload path correspondence, but at least it does so uniformly. The bigger concern, I think, is that this rename would imply that it should be possible to do GET /people/1/linked and have that return all the linked resources. Whether being able to request that is a good thing or not, I'm not sure. But I don't think it'd be that hard to implement on the server, since the requisite query is the same as that for GET /people/1?include=all,the,linked,properties and the server should have a schema to know what those property names are.
Thoughts? Other options?

I think having both the notion of a "subresource" and an "included resource" will get confusing

Sorry, I was being unclear... I'm not proposing changing the term unless there's some particular reason. You said:

The point with the complex attributes case, as I understand it, is that the data in question really is just an attribute

...so I used the term "subresource" to explain my thinking.

saying explicitly that a complex attribute MUST NOT be accessible at its own url, in the same way that simple attributes (like name) aren't

But what's the value in prohibiting it? The spec could just stay silent on the issue just like in the case of simple attributes.

if the complex attribute doesn't have a url, how do you request the link inside of it? [...] how is a user supposed to get the city's link object?

By requesting /people/1, and there you have it in data.address.links.city.
If there's no need to request /people/1/address directly (because it's wholly dependent), then certainly there should never be a need to access something inside it directly, right?

So, now, I'm thinking that one option would be to stick with the nested-links approach—which has all the advantages for parsing and url–object correspondence—and simply prevent the concept of a complex attribute from collapsing into the concept of a subresource by saying explicitly that a complex attribute MUST NOT be accessible at its own url, in the same way that simple attributes (like name) aren't.

Not being addressable is precisely the use case I want to support. Anyone who says 'you can just normalize' should go and do RDF tripples because thats where you end up. I don't want to implement a tripple store so I want attributes which have no identity in their own right but which may be conplex to be able to link out. There are countless basic data modeling use case drivers for this.
As I've said previously the real problem stems from segregation of fields and links which has dubious justification, if any, at all. Its a handrail just like typed collection keys gave people comfort until they realized the handrail actually  gets in the way.

Not being addressable is precisely the use case I want to support.

No one's saying complex attribute must be addressable. I just don't see the point of prohibiting it either. If someone wants to retrieve "1" by issuing GET /people/1/id, I don't mind.
And links bunches need not be addressable ever. The spec says:

If a link object contains a relationship URL, then the server MUST respond to requests to that URL to update the relationship.

If.
But presumably people will want to do this, and there's a dearth of sensible options apart from making the nested links objects addressable. And what's the harm in that? An application is free to not implement and go through the main object or the other side of the relation instead.

@bintoro I just wanted to spell out that the major driver is for rich nested attributes is that they are not addressable as they don't have an id, and the fact the server does not support that level of granularity on fields.
I do see a problem with interoperability with addressing "sub resources" / fields  like you describe. Having clients support both styles where on some servers they may be able to address with fine granularity and others only at the owning resource is quite a different api.
If however we limit 'jsob-api' defined addressibility to a resource, we still have the fields query param to control what we want returned. If you want to have finer grained addressability then I would say that would be outside of json-api defined semantics and more a local/proprietary behavior.

I understand the concern. Let's not veer off the main point because for such cases the spec already has it laid out:
A valid JSON API document must consist of data objects. A wholly dependent complex attribute that doesn't have an ID cannot, own its own, constitute a valid data object. There is absolutely no way the base spec permits the kind of granular access you're worried about.
However, what must be resolved is how to access the outbound links in such an attribute. For purposes of addressing, I don't see an issue with a complex attribute having a URL structure.
The way I would go about it is return a 204 or 403 at /resource/:id/complexattr but make the links available normally at /resource/:id/complexattr/links/*.
@ahacking, are you also opposed to this?
@ethanresnick, the same principle applies to your modified proposal:

The bigger concern, I think, is that this rename would imply that it should be possible to do GET /people/1/linked and have that return all the linked resources.

I don't think it implies anything. There's absolutely no rule saying all segments along a URL path must be directly accessible.
We already have an example of this today. Fetching /people/1/links would have to produce a links object or perhaps an array of link objects, but neither is valid for primary data. The client is supposed to always target an individual relationship.
The bigger problem with the rename is that it reintroduces the "links"/"linked" confusion, this time inside resource objects, and leads to ugly URLs because you'd have to repeat "linked" at each level; consider /posts/1/linked/comments/23/linked/user vs. /posts/1/comments/23/user today.

One more thing: it's important to keep in mind that the resultant URL structure in the "nested links" scheme is nothing more than a logical extension of the recommended URL structure. The base spec doesn't mandate any specific format.
 Between all the changes in RC2 and the fact that I've been out of this spec for a while until recently, I'm still reorienting myself to a bunch of the details. E.g. in the above discussion, I was operating under the assumption that GET /people/1/links was a valid request whose response was already defined in the spec. (I've updated my earlier comments accordingly, so this thread is easier to read.)
Given, then, as you point out, that we already have some recommended urls for which dropping segments from the end produce an invalid url, I'm fine with this same principle applying to links from complex attributes.  I'm fine with that!
So I think we just go with the nested links option.
There's one more case I want to consider, though: the case where the complex attributes holds an array of objects, each of which links out. For example, take a slightly-simplified version of the team-members case from my original post, which, with the nested links strategy, might look like this:
{
  "id": "1",
  "type": "projects",
  "name": "My awesome github project",
  "team-members": [{
    canPush: true,
    "links": {
      person": {
        //link object
      }
    }
  }, {
    canPush: false,
    "links": {
      person": {
        //link object
      }
    }
  }]
}

In this case, what are the recommended urls for those link objects?

Interesting example, @ethanresnick. My first instinct is the same, i.e. to use the array index.
As for recommendations: since complex attributes may contain arbitrary structures, I think we should either refer to the JSON Pointer spec on how to construct the URL path or say nothing. Applying the JSON Pointer transformation would yield the same /team-members/<index>/person/links/... result and cover everything else too.
In any case, the discussion about addressing stuff inside complex attributes should definitely come with a clear warning about mutability. An index or a key is not a forever-unique ID. Good point there.
Another thing I'd like the final spec to spell out more clearly is the principle that a client can't make any assumptions about URLs.
If the team members' links structures didn't have self members, then either

the links are not addressable, or
the client and server follow some application-specific convention.

Either way, no problem.

Or maybe we can find a scheme to recommend that better picks out the desired link object entity?

In the absence of IDs it's quite simply impossible to guarantee you're targeting the right entity. A client can fetch a link URL, but by the time it issues a request to it, the parent resource may have moved. Unfortunately there's nothing JSON API can do about it, so it's up to the application to prevent such race conditions somehow.

@bintoro I totally agree with all that! JSON Pointer, plus a warning about mutability, sounds good.
@dgeb I think @bintoro and I are in agreement here on how links from complex attributes should work. The final payload we're proposing looks like the below. Does this look good for submitting a PR?
(I've used non-numeric ids below just to make it clearer what's linking to what without having to add comments.)
GET /projects/ember

{
  "data": {
    "id": "ember",
    "type": "projects",
    "name": "Ember",
    "links": {
      "parent-organization": {
         "self": "http://example.com/projects/ember/links/parent-organization",
         "resource": "http://example.com/projects/ember/parent-organization",
         "type": "organizations",
         "id": "tilde"
      }
    }
    "team-members": [{
       "startDate": "2014-09-01T00:00:00.000Z",
       "isLead": true,
       "links": {
          "member": {
            "self": "http://example.com/projects/ember/team-memberships/0/links/member",
            "type": "people",
            "id": "ykatz"
          }
       }
    }, {
       //more team members, including ones who aren't leads and who may have an `endDate`
    }]
  }
}

This could be an additive change post-1.0.

This could be an additive change post-1.0.

But not very easily unless reserved keys are prefixed as suggested in #313. Linking from within complex attributes in the way proposed here means that links becomes a reserved key in any JSON object nested within a complex attribute. It's a backward-incompatible change unless we go with @links etc.

Per our contribution guide, I am closing this discussion here because it is not on our roadmap for 1.1.
I encourage you to continue this dialog @ http://discuss.jsonapi.org.
We appreciate the suggestions greatly, but the core team only has so much bandwidth to deal with new feature requests. Because of this, we are focusing our efforts on empowering the community to create extensions without our direct involvement (until a critical mass of adoption makes it clear that something like this belongs in the base specification).

/ref #794

Sounds like this is to be supported through an extension? Has anyone started on that?


chore(main): release 1.2.0

:robot: I have created a release beep boop
1.2.0 (2023-02-02)
Features

兼容各设备屏幕大小 (b5f2ec1)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/School-of-Website-Engineering/TinyWaste/releases/tag/v1.2.0 :sunflower: