New iteration of signed report changes (with Bogdan)

Use the new report upload format in our test env:

[x] Use the new unsigned report provided by @bogtieba
[x] Successfully produce signatures/digests
[x] DO NOT change the file name
[x] Successfully verify
[x] Notify @lenkan

@2byrds We had the internal meeting yesterday and we decided to accept *_signed.zip files. Also, we agreed that the signing tool shoud accept any type of file (zip, pdf, etc) and build the final zip as in the example.
As structure, we have the following zip file as the zip with the signatures + the received files.
Basically, the signature tool should generate a zip with 2 elements

META-INF/reports.json -> where we find/store the signature data
<OriginalFile>.<AnyExtension>

Let me know if I can help you in any way to move forward with this.
@lenkan do you need any more info to start working on this? Let me know
DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip

Thank you @bogtieba i should have a nice set of updates today and will share a signed version of that file for you (and Nord) to review/agree.

@2byrds We had the internal meeting yesterday and we decided to accept *_signed.zip files. Also, we agreed that the signing tool shoud accept any type of file (zip, pdf, etc) and build the final zip as in the example.
As structure, we have the following zip file as the zip with the signatures + the received files.
Basically, the signature tool should generate a zip with 2 elements

META-INF/reports.json -> where we find/store the signature data
<OriginalFile>.<AnyExtension>

Let me know if I can help you in any way to move forward with this.
@lenkan do you need any more info to start working on this? Let me know
DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip

So, my interpretation of this is that the reports.json will look like this:
{
    "documentInfo": {
        "documentType": "http://xbrl.org/PWD/2020-12-09/report-package",
        "signatures": [
            {
                "file": "../DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip",
                "aid": "EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw",
                "sigs": [
                    "AADoIV4XlPPBKeDQXdcVdIwdytUTuAbgFc3cYI4WbbAI_XY2QVRUWU97H92wL31t9RnhJGEaPoOcGHSqS8jw7bEK"
                ],
                "digest": "sha256-edf68f0116f7007647ea9c5417b7a30c8e4485eff73c18fe2e2a5d8e9cd92645"
            }
        ]
    }
}

@bogtieba can you confirm?

Also, here is a signed file from my local dev
DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip
It has been signed by the EBA Data Submitter from the leaflet, so the AID should be known by your test environment.

@lenkan That is correct. This is what we are expecting. The verifier needs updated as well, and I created a PR for @2byrds to approve.

@lenkan @bogtieba @tibor19 I'm close to finishing the end-to-end changes from reg-pilot data generation/testing, through the api and verifier. It was a little more involved than I had hoped. I'll work on it later today (Saturday) and provide and update tonight.

@lenkan @bogtieba @tibor19 I improved things in the verifier but a few test cases are still failing. I'll continue to work on it on Monday.

@2byrds today I've seen a couple of things that don't match with previous discoussions we had with you and nordlei regarding the reports.json fields.

Reports.json has a digests which seems to be a data duplication of the info we found in signatures. I see that verifier is expecting the reports.json->digests. Is there any reason we want this extra field?
dig vs digest fields are not consistent as field names.
sha256- vs sha256_. Based on previous chats/issues sha256- should be used, but verifier is expecting _.

cc: @lenkan

Thank you for reporting @bogtieba my latest changes from Friday will resolve the first 2 digest issues you mention.
I will do a sweep related to the prefixing format and include that in my current changes.

@bogtieba @tibor19 @lenkan
PRs merged and test file provided was added https://github.com/GLEIF-IT/reg-pilot/blob/main/signify-ts-test/test/data/orig_reports/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip
Signed zip preview:

Signed reports.json
{
  "documentInfo": {
    "documentType": "http://xbrl.org/PWD/2020-12-09/report-package",
    "signatures": [
      {
        "file": "../reports/FilingIndicators.csv",
        "digest": "sha256-b35bbe84ff0ced85c9a71e6d0a5b11b6513051ac008b1a0fc5f7b90b51d3b2b6",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAC4f9Qx80c2_xUhdfTNdzy5FPyrJ-2M84lAyoIQWYIWIZH12A8k5FUKKMRALL1qzzJI5EsRjWLLSLa3zyLg_DgI"
        ]
      },
      {
        "file": "../reports/k_04.00.a.csv",
        "digest": "sha256-0949ddf85dd63587e366ee2f4ad82346908e8d7d98be04d58d7b5bbf150548ef",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAOg9MXWlox6WAPDgSHbKk1iAdnzqrS8XrZgzU64r3fgA6vlu_OUKrAqWi1uZNm8Iihv4uSmqgnihIZLd4ZqkwC"
        ]
      },
      {
        "file": "../reports/k_04.00.b.csv",
        "digest": "sha256-2aae92645def99429894b7004f0d75cf55fa3fe4f4356ac297325a0afad0a725",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAKoK_wsnMlhAfhkq8iKOY8_tTGXK9RMQPpS5diQKmw-A5ivQQPzL7zgqKwJp0H7KvLiWlXUfWQZwrD0ybB-E0M"
        ]
      },
      {
        "file": "../reports/k_05.00.a.csv",
        "digest": "sha256-997cd86232a3e6111b09a621a7a5b62f679fb95cbc7b43bda9cde4842743353e",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAw4nxApsNLo-pKyWzn2XkKrzksDQJzp8aZDNYbw59F6CbrBoY2Wet7n_dTwJGVG9Rx7AEvY2vKl_-_NCfC9ggD"
        ]
      },
      {
        "file": "../reports/k_05.00.b.csv",
        "digest": "sha256-114f19e701e16969d71bc2ef8abe3c8fcbd7644ff660bbb2e7f19753948c62a9",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAw-SX4kmDjmQisRga8Rt1YyYFEF-oEGTqfzUflK2Y4z8ZUw5S6sWey4M_Dc5m3vJZKj2X47_BksoLGv9n24YgD"
        ]
      },
      {
        "file": "../reports/k_06.00.csv",
        "digest": "sha256-18485357308bca911f92317bd8e64ef0f1a6633057fb32384ca205612c10b504",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADwC97BI0p833UcHl-TzMsvO378nde1I1v8pq7QCiKTTqkXZxD_C-75BWfqefX-pID8BAVtqrzaKciELoGKqtcM"
        ]
      },
      {
        "file": "../reports/k_07.00.csv",
        "digest": "sha256-4033d3c3c22252de908bbe1b4458969d4965626c0d18783734254fb564baeaa5",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAzh4e8TS1RrTRGvoTx_2nIwd7_Oc0zx0H4wO09SI2PSTIUXEs4LJF-o0s0YNZBPcqZNtxMZtA7U3hAzMLbziIG"
        ]
      },
      {
        "file": "../reports/k_08.00.csv",
        "digest": "sha256-039eced3c2069769a71e522c909271ff8193800b819b56972d51c6d2df54f707",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADfDF-ni5NKjduByzQpn8e17QHv90nu37qve4zg0C3PpTGmvfASOpX4OYrpAWX0Ff3WNbsQkBntGwE5kShmRH8B"
        ]
      },
      {
        "file": "../reports/k_26.00.a.csv",
        "digest": "sha256-23156746b6ea0151430f4ff59dee6385ef71df8155a6b7f3eb1f289228fd4ec9",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AACGAtMFnu0upRv0xN-qYzxfs-U4Y7VBGie2kxpun_bxwjW0H2FfLegyQfgyNbNQB2j5GKH8RGrOexvjNpOKHBkN"
        ]
      },
      {
        "file": "../reports/k_26.00.b.csv",
        "digest": "sha256-709c4b287119bb379e44e9424cf695c0198f0f7e18e7022c381dc2c50423fb8b",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AACqUQMJ3vGQR0kXEZJUyHVD2Z2u_Ur-qh3aZCxVBrQR6wPYCUpiggD8B4ggoeiaFtLU_7j45Y7TuIpjTvvyKcgN"
        ]
      },
      {
        "file": "../reports/k_26.01.csv",
        "digest": "sha256-c3f9a911b5f49777916d5228f20cdb764b00537a491eff3a3d8287dd0ede179b",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADUV-tRPdJDJoYA-VXO1tuY40lfjnMMDeg1MPVTfVwBuUgcUKkyvebck-GHQfe_CCzlLtU_ZR9-O5Cs0uCunUML"
        ]
      },
      {
        "file": "../reports/k_28.00.csv",
        "digest": "sha256-a5f583350d096b4856ba1cc79dd1a16de57e7b837faa1c7787468ceb03c13f24",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAcEnRE2SDBhxBs98dr2qmHc-WNr57k5TZRxDarhA0s68mZyv_eYV99bP_5LbflPKoH0giWXP7I9KKzNbeGE5wB"
        ]
      },
      {
        "file": "../reports/k_29.00.csv",
        "digest": "sha256-36fea4cd655d9e47ae628010cdd70085c0d97673fee56ce1fd119dceb67f32d4",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADzH0uNNdqz-jU7_oe-W46tNTwhDHHPd6HKGeVIt-ymICdClYIuzmaXEm8lUgiIpGLFTnHQgn4x558r67piGE4N"
        ]
      },
      {
        "file": "../reports/k_29.01.a.csv",
        "digest": "sha256-4c9230494e92973757941aae1395f186180b4b4bc9c3e6faff3ec852462b80c5",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAD0_d-Sb9L9QEmMH5DD6uAkpZSkC6C9Fx3wJ6QBerx3uXUZI3J41KcRv28tf_K8slYG_3Lg1nUlTZaeyt2VCvAP"
        ]
      },
      {
        "file": "../reports/k_29.01.b.csv",
        "digest": "sha256-3a2e8d806f30ba0473cf6a7e1906d4e5af4459d5749c62bd5f9863d64d6c6f73",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADQY7gLEg9DWB8h20nxF7mUxV6eRfVmsBIOTXP86vP2aSkdPPchOWyUW0T4ngcZDPYqyfaM8FHFnLAUfR2sWXwN"
        ]
      },
      {
        "file": "../reports/k_60.00.a.csv",
        "digest": "sha256-3f5187c95a6a632b1889a4cd95cc0d64a7ecc2f74834130701a1feda2635a354",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAB5qJ2WdURDlCfJKabkh5rgLLSUi98rZPgZ_1uYluYFDSoRTiyAxiRKY1LmJ2yy_SVErY23zYIM_vdPoojYxa0M"
        ]
      },
      {
        "file": "../reports/k_60.00.b.csv",
        "digest": "sha256-8e21ef80091ed99a56c4137486e794566413664b877ae87a8e7c796c44b4019b",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAAzjHme-073TDrD6cFaG_oghi7eg-Ghob2n8Ub5I3Hjqgud3xXR_pXZVeMVxbJh-B9kF17CReEbM4HbT6y07k0F"
        ]
      },
      {
        "file": "../reports/k_60.00.c.csv",
        "digest": "sha256-42c419631c105eca121097ae5e76fc37290d270ce8cf98c9c9115086e51880da",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AABpsIi51rpik9NKCF-1cCRidufOjpCPFMprIdj8BGa1Qa6LSndRlubXey66vHGnPqvn7obyVXQ0D-iAcOiUASIA"
        ]
      },
      {
        "file": "../reports/k_61.00.csv",
        "digest": "sha256-ea8f293c874d5bbf34d1b71f04befb6e82ae7a31054cc5f9dd8309bbb1439088",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAA7dh7AsW3ep8888uJk8e43L-3rlD2K7kP5V6C-9uGeOgOa68BG9hZ28g2EmY-t8eX8cU177P6NTAg3sZ94QYwH"
        ]
      },
      {
        "file": "../reports/parameters.csv",
        "digest": "sha256-3c87d34bc0197a21853cf7d5a5d03f64d644b6bebe38152a94e086c01b1ba72c",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADc-V1QHV8N7VW6jBAx8lLWKWdAyRGRS9OLSi-oLzUEF3CwzmKHg2nnWZOCluWi3b1iuDUpiwTXRxY_WuFMVtgH"
        ]
      },
      {
        "file": "../reports/report.json",
        "digest": "sha256-97fe5bd8741d26ee77892d39271d2dddcd1d48ed72ed28b46ca397d899ad2c13",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AADv5WVj0TXNWQ-7HfoVED-8dbJAnlhvhLQqKG5GhKpNw0FlFJP1_KrY5KswiJnq4CCdNclv9MIh2BvVU2gXByQL"
        ]
      }
    ]
  }
}

verifier output:
....
keri: processing signature {'file': '../reports/k_61.00.csv', 'digest': 'sha256-ea8f293c874d5bbf34d1b71f04befb6e82ae7a31054cc5f9dd8309bbb1439088', 'aid': 'EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX', 'sigs': ['AAA7dh7AsW3ep8888uJk8e43L-3rlD2K7kP5V6C-9uGeOgOa68BG9hZ28g2EmY-t8eX8cU177P6NTAg3sZ94QYwH']}
keri: File /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/reports/k_61.00.csv not found in /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/META-INF
keri: Finding file ../reports/k_61.00.csv in zip files...
keri: File ../reports/k_61.00.csv found in zip, extracted to /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000/reports/k_61.00.csv
keri: processing signature {'file': '../reports/parameters.csv', 'digest': 'sha256-3c87d34bc0197a21853cf7d5a5d03f64d644b6bebe38152a94e086c01b1ba72c', 'aid': 'EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX', 'sigs': ['AADc-V1QHV8N7VW6jBAx8lLWKWdAyRGRS9OLSi-oLzUEF3CwzmKHg2nnWZOCluWi3b1iuDUpiwTXRxY_WuFMVtgH']}
keri: File /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/reports/parameters.csv not found in /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/META-INF
keri: Finding file ../reports/parameters.csv in zip files...
keri: File ../reports/parameters.csv found in zip, extracted to /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000/reports/parameters.csv
keri: processing signature {'file': '../reports/report.json', 'digest': 'sha256-97fe5bd8741d26ee77892d39271d2dddcd1d48ed72ed28b46ca397d899ad2c13', 'aid': 'EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX', 'sigs': ['AADv5WVj0TXNWQ-7HfoVED-8dbJAnlhvhLQqKG5GhKpNw0FlFJP1_KrY5KswiJnq4CCdNclv9MIh2BvVU2gXByQL']}
keri: File /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/reports/report.json not found in /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/META-INF
keri: Finding file ../reports/report.json in zip files...
keri: File ../reports/report.json found in zip, extracted to /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmptcfv6yfl/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000/reports/report.json
keri: All 20 files in report package have been signed by submitter (EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX).

Attaching signed file
DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip

Hm, @2byrds. This is not the same as I posted above  https://github.com/GLEIF-IT/reg-pilot/issues/79#issuecomment-2348889822. What we did there was to only sign the zip file. Then bundle the zip file together with a new reports.json in a new zip.

Hm, @2byrds. This is not the same as I posted above #79 (comment). What we did there was to only sign the zip file. Then bundle the zip file together with a new reports.json in a new zip.

@lenkan okay the verifier still supports verification of a single signature/file (like in the xbrl xml case), but i would just need to make it configurable whether it should recurse into the zip package or not. I'll add that now.

If the input file is a ZIP, as this one https://github.com/GLEIF-IT/reg-pilot/blob/main/signify-ts-test/test/data/orig_reports/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip, then the reports.json should be:
{
  "documentInfo": {
    "documentType": "http://xbrl.org/PWD/2020-12-09/report-package",
    "signatures": [
      {
        "file": "../DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip",
        "digest": "sha256-b35bbe84ff0ced85c9a71e6d0a5b11b6513051ac008b1a0fc5f7b90b51d3b2b6",
        "aid": "EASQ3NoDbzrOqiI6W0SSEAq9oBkUphgDkYlS7cMx4ufX",
        "sigs": [
          "AAC4f9Qx80c2_xUhdfTNdzy5FPyrJ-2M84lAyoIQWYIWIZH12A8k5FUKKMRALL1qzzJI5EsRjWLLSLa3zyLg_DgI"
        ]
      }
    ]
  }
}

@2byrds , per @tibor19, we should not open the zip file. We take, we generate the digest, signature and that's it. We do not look into it.
As simple as possible.

@2byrds , per @tibor19, we should not open the zip file. We take, we generate the digest, signature and that's it. We do not look into it. As simple as possible.

Thank you for confirming @bogtieba i'm adding that simple case now. I'll report back with the simple version of the signed file/verification ASAP.

Based on what @tibor19 requested, yes, this seems relative to the ROOT.
Let's go with this.
CC: @lenkan .

cc @daviddm

@lenkan @bogtieba @tibor19 @daviddm I have updated the verifier locally but need to merge another PR that will take a little time tomorrow. I'll post again here when the verifier is merged/available

Thank you @2byrds!
Please ping me when we have the new version so I can test and confirm.
If usefull for you, https://sign.vlei.tech/ nordlei deployed the changes to this staging environment.

@tibor19 @lenkan @tibor19 @daviddm I have created a new verifier release and used the NordLEI signing tool to test the newest verifier release.

API output
"@path": /upload/EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw/sha256-e562cbbcca2875ca5c6ea688e124b1b0cf6319ce881fe9106b2a08ddca1b8473
"signify-resource": EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw
"signify-timestamp": 2024-09-18T19:09:09.981000+00:00
"@signature-params: (@method @path signify-resource signify-timestamp);created=1726686549;keyid=BHerQd_5W7xwEf7_3hN7xFhh3xtjEmPdOlI5zunAt2cb;alg=ed25519" cig=0BAP1JMPVx8WM7VDnnukL-EksOLVhx4SIzHi16O2fX8wRfhMin7Nbiz1L8EuXEtwFvG4_l-SW-ddOf8JG-w6KxkH
Verify header sig started aid = EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw, cig = 0BAP1JMPVx8WM7VDnnukL-EksOLVhx4SIzHi16O2fX8wRfhMin7Nbiz1L8EuXEtwFvG4_l-SW-ddOf8JG-w6KxkH, ser = "@method": GET
"@path": /upload/EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw/sha256-e562cbbcca2875ca5c6ea688e124b1b0cf6319ce881fe9106b2a08ddca1b8473
"signify-resource": EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw
"signify-timestamp": 2024-09-18T19:09:09.981000+00:00
"@signature-params: (@method @path signify-resource signify-timestamp);created=1726686549;keyid=BHerQd_5W7xwEf7_3hN7xFhh3xtjEmPdOlI5zunAt2cb;alg=ed25519"....
posting to http://localhost:7676/request/verify/EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw
Verify sig response {"msg": "Signature Valid"}
VerifySignedHeaders.on_post: response {'msg': 'Signature Valid'}
checking upload: aid EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw and dig sha256-e562cbbcca2875ca5c6ea688e124b1b0cf6319ce881fe9106b2a08ddca1b8473
getting from http://<IP_ADDRESS>:7676/reports/EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw/sha256-e562cbbcca2875ca5c6ea688e124b1b0cf6319ce881fe9106b2a08ddca1b8473
upload status: {"submitter": "EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw", "filename": "DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip", "status": "verified", "contentType": "application/zip", "size": 35814, "message": "All 1 files in report package have been signed by submitter (EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw)."}

Verifier output:
keri: File /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmp50fh3v1d/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip w/ digest sha256-edf68f0116f7007647ea9c5417b7a30c8e4485eff73c18fe2e2a5d8e9cd92645 has valid digest
keri: Processing DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip:
        Type=application/zip
        Size=35814
keri: processing signature {'file': 'DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip', 'aid': 'EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw', 'sigs': ['AADoIV4XlPPBKeDQXdcVdIwdytUTuAbgFc3cYI4WbbAI_XY2QVRUWU97H92wL31t9RnhJGEaPoOcGHSqS8jw7bEK'], 'digest': 'sha256-edf68f0116f7007647ea9c5417b7a30c8e4485eff73c18fe2e2a5d8e9cd92645'}
keri: File /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmpgszzf45d/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000.zip found in /var/folders/dc/hfckgfqd5w3c9c9gz371g7_00000gn/T/tmpgszzf45d
keri: All 1 files in report package have been signed by submitter (EFE8-Km32lJzOa51K3IWMcctJCX8Ifu5f4BaUWdfgSbw).
[DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip](https://github.com/user-attachments/files/17049204/DUMMYLEI123456789012.CON_FR_PILLAR3010000_CONDIS_2023-12-31_20230405102913000_signed.zip)


Add usage example using React Hooks

Fixes: #269

They stopped the repairs. I had to do it myself, in the same way, with a new version of typeScript. And fix these problems for everyone to use. add react hooks、vue3 example
npm i body-scroll-lock-upgrade

repair log，Refer to the releases page.

Your naming is wrong!


Paginator

in some version this package 'white-october/pagerfanta-bundle' was replaced with custom code, but it is not work correctly.
if you have 20 pages it show this
Previous 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Next
if you have 100 it will show all 100 but it should show like in google style
Previous 1 2 3 4 5 6 7 8 9 10 Next  and you click Next it should show next 11..20
Previous 11 12 13 14 15 16 17 18 19 20 Next

any comments about this issue?,
just set private const PAGE_SIZE = 1; to see how it looks

Most probably it just was not implemented because we didn't have enough results for noticing this problem. If so, I think it would be good to implement this

any chance to resolve this? or maybe to use another bundle like BabDevPagerfantaBundle ?


c_int instead of c_uint for CLONE_*

These are the correct definitions:
pub const CLONE_NEWUTS: ::c_int = 0x04000000;
pub const CLONE_NEWIPC: ::c_int = 0x08000000;
pub const CLONE_NEWUSER: ::c_int = 0x10000000;
pub const CLONE_NEWPID: ::c_int = 0x20000000;
pub const CLONE_NEWNET: ::c_int = 0x40000000;
pub const CLONE_IO: ::c_int = 0x80000000;
These are #define in vi /usr/include/linux/sched.h
They must be c_int because:

int unshare(int flags);
int clone(int (*fn)(void *), void *child_stack, int flags, ...

Ah probably a mistake on my part, feel free to submit a PR!

I believe this was fixed.


feat: use latest vuepress and remove unused features

see comments

redirects are removed

Hi @Mister-Hope
Marc here. I had the same initial reaction to your PR as Lars. It came out of nowhere and has no description at all. This is a very large diff too and I didn't see the redirects either. You have to scroll way down to find them.
Like Lars, I would have rejected this.
Your account name looks rather anonymous, too. So it's not apparent that you are from the Vue team. Now that I saw your profile the PR makes more sense. Some short description here would have definitely avoided this misunderstanding.
I will take a look at this update later in more detail.

The color is meant to fix the background/text contrast, so we made a update for this, technically now a set of theme colors are automatically generated from the given theme color.
Users can still override them in palette file, I will upgrade that now.

Ref: https://dequeuniversity.com/rules/axe/4.9/color-contrast

That makes a lot of sense. I'm all for improving accessibility. Thanks again!


read_csv_arrow stops reading the rest of the file during a conversion error

Describe the usage question you have. Please include as many useful details as  possible.
When reading a csv file in R using read_csv_arrow with a schema, the program will stop and not finish reading in the whole file if it encounters a conversion error.  Is there a setting that will force all the records to be read in even if a conversion error is encountered?
Component(s)
R

Hi @kmkramer23.  There isn't a setting for this, as the conversion is part of the reading process, but perhaps if you could share a bit more information about the error you're encountering, we can help you fix it?

Some files we get may have a character value in one of the integer fields representing missing data.  Since the file is coming from an outside organization it is not something I can have fixed.  I just need to read it in and convert it to numeric setting any of those records with the missing character codes to NA.  I can read everything in as character and do some pre processing but it would have been nice if read_csv_arrow could have forced the non int values to NA to avoid the extra step.


Feature Proposal: Centralized Swagger UI for OpenAPI Specification Files

Feature Proposal
Summary
In the context of the .NET Aspire project, I propose to add a new feature aimed at consolidating all OpenAPI specification files across the project into a unified Swagger UI interface. This would greatly enhance the developer experience by providing a single, centralized location to view and interact with the API documentation generated from OpenAPI specifications.
Problem Statement
Currently, OpenAPI specification files are scattered throughout various parts of our project, making it challenging to locate, view, and test different APIs efficiently. Developers and API consumers have to navigate through multiple Swagger UI instances or look into different directories to find the relevant API documentation. This fragmentation hinders productivity and can lead to inconsistencies in how APIs are understood and used.
Proposed Solution
Implement a mechanism within the .NET Aspire project that automatically scans the project directories for OpenAPI specification files (*.json or *.yaml). Once identified, these files will be aggregated into a single Swagger UI instance. This unified Swagger UI will be automatically updated to reflect changes in the OpenAPI specifications, ensuring that the documentation is always current.
Benefits

Centralization: Provides a one-stop-shop for all API documentation, making it easier for developers to find and use the APIs they need.
Improved Developer Experience: Facilitates a smoother development process by reducing the time spent searching for API documentation.
Consistency: Ensures that all API documentation is presented in a consistent manner, aiding in understanding and integration efforts.
Automation: Reduces manual effort required to maintain API documentation visibility as the project evolves.

Implementation Considerations

The scanning mechanism should be configurable to allow for inclusion or exclusion of specific directories or files.
Consideration should be given to how the unified Swagger UI is hosted within the project (e.g., as part of the build process, within a container, etc.).
Security implications of aggregating and exposing API documentation in a single location should be assessed and mitigated.

Request for Comments
I invite the community to provide feedback on this proposal. Any insights on potential challenges, additional benefits, or alternative approaches to achieving this goal would be greatly appreciated.

Cool idea, related to https://github.com/dotnet/aspire/issues/2980

If we do this we should also support it in Visual Studio.

Just some (half-baked) general thoughts to help spur discussion

Dedicated Swagger service container

var builder = DistributedApplication.CreateBuilder(args);
var swagger = builder.AddSwaggerHost("swagger");

builder.AddProject<Projects.MyCoolProject>("web")
       .WithReference(swagger);

A few discovery mechanism ideas

var swagger = builder.AddSwaggerHost("swagger")
    .WithSwaggerFile("**/*.swagger.json"); // With glob to point to swagger files

var someService = builder.AddNonDotNetService("foo")
    .WithSwagger(); // Search output path for swagger file. Possibly accept glob pattern again. Intended scenario is including a swagger file from a docker image or something similar.

var web = builder.AddProject<Projects.MyCoolProject>("web")
    .WithReference(swagger);

// In MyCoolProject.Project.cs
var builder = WebApplication.CreateBuilder(args);
builder.AddSwagger("swagger"); // Wraps standard swagger packages to produce the swagger json file, then provides that back to the swagger host.

My Questions

What kind of ergonomics do we want here?
Should supporting DotNet projects (a la AddProject<TProject>(string)) feel different than supporting non-DotNet projects (e.g. a PHP web service)?

If so, how? Why?

How should we handle IDP support, e.g. if Keycloak is used as an auth source for the application overall, how do we secure Swagger with it?

This likely falls under the umbrella of IDP integration with Aspire overall and might be a little out of scope.

@Foxtrek64 I like the way that you've modeled this implementation as an integration in your proposal, with both a client and hosting component.
In .NET 9, we added  built-in support for OpenAPI document generation via the Microsoft.AspNetCore.OpenApi package. We could consider what it would look like to add a client integration for this, that could help solve issues like this one.
With regard to the hosting side of this, I saw it being inverted a bit:
var apiService1 = builder.AddProject<>();
var apiService2 = builder.AddProject<>();

var swagger = builder.AddSwaggerUi()
  .WithReference(apiService1)
  .WithReference(apiService2);

This way a user could navigate to a certain gateway that incorporated the OpenAPI documentation for all the API services in their application. You could potentially use the document switcher UI to keep things truly centralized.
Side note: although Swagger UI is popular, I've become quite partial to Scalar over the past few months. It would be interesting to explore adding an Aspire hosting integration for Scalar that provided this functionality. @xC0dex -- what are your thoughts on prototyping an Aspire hosting integration around Scalar and seeing what it would look like?

Should supporting DotNet projects (a la AddProject(string)) feel different than supporting non-DotNet projects (e.g. a PHP web service)?
If so, how? Why?

I'm inclined to say the ergonomics should be the same across all project types, especially if the API reference document is an independent resource in-and-off itself instead of something that gets added via a WithReference on the API service....or do we support both models?
One more thing, I was originally gonna close this issue in favor of centralizing all the discussion in #2980 but I think we should keep these two as independent and separate ideas.

This issue is about adding hosting/client integrations for OpenAPI document + Swagger UI/Scalar UI/etc.
The other issue is about providing an in-dashboard experience for ad-hoc testing and would require spinning up a new UI altogether.

I think the first issue (this one) is a lot more tractable given the APIs that we currently support and the prototyping that has already been done in this project.

@captainsafia I like the inverted model. By providing the service reference to the service with the open api spec, it makes it really easy to keep the ergonomics consistent regardless of whether the service generating the open api doc is .NET or not.
Also, I think the best way to handle the question of swagger vs scalar or other potential implementations could be something like
var openApi = builder.AddOpenApi();
var swagger = openApi.WithSwaggerUi();
var scalar = openApi.WithScalarUi();
// others

Essentially, the AddOpenApi() piece would produce an OpenApiBuilder which is responsible for aggregating all of the input documents. Vendors, be it Swagger or Scalar or someone else, can then come in and extend OpenApiBuilder to provide output to their web interface.
This also means a really nice tie-in to #2980 - the in-dashboard experience would simply be another output source, just like Swagger and Scalar.
builder.AddOpenApi().WithDashboardUi();

Also, I think the best way to handle the question of swagger vs scalar or other potential implementations could be something like
var openApi = builder.AddOpenApi();
var swagger = openApi.WithSwaggerUi();
var scalar = openApi.WithScalarUi();
// others

I assume this is code that someone would write in their app host. If so, what purpose does the AddOpenApi call and OpenApiBuilder serve? Is it mostly so you can chain different front-end to the same resource instance?
I think it is a comparison between:
var openApi = builder.AddOpenApi()
    .WithScalarUi()
    .WithSwaggerUi()
    .WithReference(apiService1)
    .WithReference(apiService2);

OR
var scalar = builder.AddScalarUi();
var swagger = builder.AddSwaggerUi();

scalar
  .WithReference(apiService1)
  .WithReference(apiService2);

swagger
  .WithReference(apiService1)
  .WithReference(apiService2);

With the first option being DRYer but requiring more new API. I don't think there's a technical need for an OpenApiBuilder (for example, some requirement to share state between the two that isn't available on IResourceBuilder) but not sure...

Hey @captainsafia,
this sounds like a great idea! I'm a fan of Scalar too, so I would be happy to work on an integration for Aspire. I haven’t worked with Aspire before, so I might need a little support 👀.

@xC0dex Nice! I'd recommend checking out https://github.com/davidfowl/AspireSwaggerUI as an example to start with. You'll notice if you look at this file that it uses an approach for serving the Swagger UI that is very familiar to what the Scalar.AspNetCore package currently does so you can use that as a building block for this.
Most of the Aspire-specific logic is in this file and deals with wiring up the endpoints directly to what the API service is actually listening on.
Note: the repo above doesn't use the inverted model that I proposed in https://github.com/dotnet/aspire/issues/3608#issuecomment-2438516461 but it's a good start nonetheless.

I assume this is code that someone would write in their app host. If so, what purpose does the AddOpenApi call and OpenApiBuilder serve? Is it mostly so you can chain different front-end to the same resource instance?

Essentially, yes. I think the primary concern here is the input logic - how we aggregate the various sources into a single resource instance that can be given to the chosen front-end. In my opinion, as someone implementing Swagger or Scalar or whatever else, I don't necessarily care where my input comes from or how it's created, I just care that I get input that is in the correct format. I also don't feel it's necessarily my responsibility to need to implement that logic myself. This would lead to a situation where there is inconsistency between builders.

I don't think there's a technical need for an OpenApiBuilder (for example, some requirement to share state between the two that isn't available on IResourceBuilder) but not sure...

I agree. Having looked at some other examples already in the ecosystem, it seems implementing our own builder type would be somewhat unusual. Take this example with Postgres, for example:
var database = builder.AddPostgres("database") // IResourceBuilder<PostgresServerResource>
    .WithPgAdmin();

Here, they're doing what we're trying to accomplish - some back-end service with a bolt-on GUI. I think it may be good to follow the example here - forego the OpenApiBuilder type for IResourceBuilder<OpenApiResource> or something of the like.
With this in mind, I think the first option makes the most sense, both from a consumer standpoint and from a vendor standpoint.
var openApi = builder.AddOpenApi() // IResourceBuilder<OpenApiResource>
    .WithDashboardUi()
    .WithResource(apiResource1)
    .WithResource(apiResource2);

I was bored over the weekend so I decided to throw together a quick partial prototype of this feature, in the current context that would be the builder.AddOpenApi().WithDashboardUi();.
I haven't worked with the aspire repo before so this is my first attempt at making something functional.
Forked repo here
Prototype showcase:
https://github.com/user-attachments/assets/ccfeccdb-0d41-4121-a478-b8c1c977441f

@mikekistler  FYI

If we do this we should also support it in Visual Studio.

+1 - personally, I think that the .http file support is a much better experience than the web based explorers. What is (or was the last time I looked) a good experience to go from a list of APIs on an endpoint to generating a block in the http file for calling that API.
For scenarios like this, I would like to improve the integration between VS and the Aspire dashboard. If an Aspire AppHost has the swagger annotations, what if that resulted in a "API Test" command in the resource view in the dashboard. Clicking it would take you to the .http file for that service, and enumerate the endpoints/APIs available.

For scenarios like this, I would like to improve the integration between VS and the Aspire dashboard. If an Aspire AppHost has the swagger annotations, what if that resulted in a "API Test" command in the resource view in the dashboard. Clicking it would take you to the .http file for that service, and enumerate the endpoints/APIs available.

To make sure I'm understanding your idea - if we take my example above, you're proposing that we take the AddOpenApi() and bake it in to the Aspire AppHost by default. And if it detects an OpenAPI file for any members, it automatically renders it on a built-in page in the dashboard.
I'd be on board with that approach, provided that we get a way to extract that OpenAPI information in case someone wants to expose Swagger or Scalar or some alternative UI, which could be good for production purposes where the Aspire Dashboard may be less appropriate.

With this in mind, I think the first option makes the most sense, both from a consumer standpoint and from a vendor standpoint.
var openApi = builder.AddOpenApi() // IResourceBuilder<OpenApiResource>
    .WithDashboardUi()
    .WithResource(apiResource1)
    .WithResource(apiResource2);

I've been thinking about this and I probably like this design the most because it leaves a room for easy configuration.
Let's say I would want to configure the path to the OpenApi document. I think something like builder.AddOpenApi(["/path/to/document.json"]) would be nice. It being an array, it would also allow to specify multiple paths.
Further more I think it would be nice to also have the ability to define specific paths for selected resources, so something like .WithResource(apiResource, ["/path/to/extra/document.json"]).
So putting it all together it would look something like this:
var openApi = builder.AddOpenApi(["/path/to/document.json"]) // IResourceBuilder<OpenApiResource>
    .WithDashboardUi()
    .WithResource(apiResource1)                                    // This has one document path.
    .WithResource(apiResource2, ["/path/to/extra/document.json"]); // This has two document paths.

Hey @captainsafia,
I’ve put together an MVP for the Scalar integration, based on the AspireSwaggerUI project. Currently, it uses the CDN version of the Scalar API Reference and isn’t configurable yet. From my understanding, the plan for how the UI will be added isn’t fully settled yet, so I didn't spend much time there.
Even though it’s still in a very early stage, I was wondering: should the UI/integration be fully configurable? And would it make sense to use the Scalar.AspNetCore package here, or should everything be shipped through Aspire or an Aspire integration package? I think it would also be possible to update the package or introduce a new one.
During the development, I thought it would enhance the developer experience if the OpenAPI documents from all services could be fetched automagically. I’m considering an endpoint that could provide all the necessary information needed for the integration.

I've been thinking about this and I probably like this design the most because it leaves a room for easy configuration.
Let's say I would want to configure the path to the OpenApi document. I think something like builder.AddOpenApi(["/path/to/document.json"]) would be nice. It being an array, it would also allow to specify multiple paths.
Further more I think it would be nice to also have the ability to define specific paths for selected resources, so something like .WithResource(apiResource, ["/path/to/extra/document.json"]).

I really like this configuration approach. A params string[] would work very nicely here. That said, I would really like to push for globbing via Microsoft.Extensions.FileSystemGlobbing. I think it'd be nice to have  something like this:
// Just include paths
builder.AddOpenApi(["**/*.document.json", "**/*.swagger.json"]);

// More complicated example. This is NOT intended to be equivalent to the
// preceding example.

Matcher matcher = new();
matcher.AddInclude("**/*.document.json");
matcher.AddExclude("**/*.swagger.json");
builder.AddOpenApi(matcher);

This would of course also apply to IResourceBuilder<OpenApiResource>#WithResource() as well, accepting either params string or a matcher instance.

Ooh, I wasn't aware of that class. I like it. This then gives me an idea of having two string[] in builder.AddOpenApi() method allowing to specify both include paths and exclude paths:
// First parameter are include paths, second parameter are exclude paths.
builder.AddOpenApi(["**/*.document.json"], ["**/*.swagger.json"]);

// This complicated example below now matches the preceding example.
Matcher matcher = new();
matcher.AddInclude("**/*.document.json");
matcher.AddExclude("**/*.swagger.json");
builder.AddOpenApi(matcher);

// Writing just exclude paths would be like this
builder.AddOpenApi([], ["**/*.swagger.json"]);

I just thought of a question "Would you be able to use multiple UIs at the same time?" I think I can explain it best with the following scenario.
The Scenario
I have some public API services and some internal API services along with possibly more services described in one AppHost.
var builder = DistributedApplication.CreateBuilder(args);

// ... some other services possibly

/* My projects */

var publicApiService1 = builder.AddProject<>();
var publicApiService2 = builder.AddProject<>();

var internalApiService1 = builder.AddProject<>();
var internalApiService2 = builder.AddProject<>();

/* OpenAPI */

builder.AddOpenApi()
    .WithSwaggerUi()
    .WithReference(publicApiService1)
    .WithReference(publicApiService2);

builder.AddOpenApi()
    .WithScalarUi()
    .WithReference(internalApiService1)
    .WithReference(internalApiService2);

builder.Build().Run();

The Question
Would I be able to use multiple UIs, in this case Swagger and Scalar, at the same time or would it throw an exception?
My initial thought is no as it would go against this feature proposal for a centralized UI. In other words it would no longer be compliant with:

Provides a one-stop-shop for all API documentation, making it easier for developers to find and use the APIs they need.

Why would you need two UIs?
Let's assume there are two companies A and B. "Company A" provides public API services. "Company B" would like to use "Company A's" public API services for their app development, for example. It would therefore be nice to give "Company B" a UI to be able to easily develop their app, while still keeping a UI for all "Company A's" internal API services.
There could also be a situation where "Company B" prefers to work with Swagger while "Company A" prefers to work with Scalar.

Just a thought - I wonder if a parameter to limit the environments the OpenApi engine is loaded in would be good, just because I feel like it would be a common option.
builder.AddOpenApi(); // Always add OpenApi
builder.AddOpenApi(builder.Environment.IsDevelopment() || builder.Environment.IsStaging());

Amendment 2: This could also be done by wrapping builder.AddOpenApi() in an if statement, but I feel like conditionally enabling/disabling OpenApi support will be such a common operation that it makes sense to have support for that built in.

Isn't this exactly how it's been done when Swagger was still part of ASP.NET template? Like you would do I believe this:
var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsDevelopment())
{
    builder.Services.AddOpenApi();
}

var app = builder.Build();

if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
}

app.Run();

I believe this is still a common practice amongst many developers and even in .NET 9 I don't think it has changed much.

I really like this configuration approach. A params string[] would work very nicely here. That said, I would really like to push for globbing via Microsoft.Extensions.FileSystemGlobbing. I think it'd be nice to have something like this:
// Just include paths
builder.AddOpenApi(["**/*.document.json", "**/*.swagger.json"]);

// More complicated example. This is NOT intended to be equivalent to the
// preceding example.

Matcher matcher = new();
matcher.AddInclude("**/*.document.json");
matcher.AddExclude("**/*.swagger.json");
builder.AddOpenApi(matcher);

This would of course also apply to IResourceBuilder<OpenApiResource>#WithResource() as well, accepting either params string or a matcher instance.

I re-read this feature proposal again and I feel like I'm missing something here. Correct me if I'm wrong, but globbing would require to know all valid url paths to each individual API services ahead of time at the AppHost level.

Let's say I would want to configure the path to the OpenApi document. I think something like builder.AddOpenApi(["/path/to/document.json"]) would be nice. It being an array, it would also allow to specify multiple paths.

Whereas what I was proposing was that you tell the AppHost exactly where to look for OpenApi documents because it knows addresses to each individual API services so it would do a simple HTTP request for all of them.
For example http://localhost:1234/path/to/document.json. This would either return Status200Ok meaning the data returned is likely an OpenApi document or it would return Status404NotFound meaning no OpenApi document exists.
So my question then is: How does the AppHost know where to look for OpenApi documents?

I re-read this feature proposal again and I feel like I'm missing something here. Correct me if I'm wrong, but globbing would require to know all valid url paths to each individual API services ahead of time at the AppHost level.

Perhaps I'm misunderstanding your concern, but perhaps something like this could resolve it?

The OpenApi document is being accessed from a location on disk. As an example, if my Aspire service is made up of three ASP.NET Core apps, and each of them create a Swagger definition file at $(MSBuildThisProjectDirectory)swagger.json, then I can set up a glob to look for **/swagger.json and it would return an enumerable of all of the swagger files in child directories of the working directory.
For web-based services, we already have a discovery model for this. When adding a database, service, you provide a connection name and consumers can reference it by name. I think we could do something similar with OpenAPI:

var builder = DistributedApplication.CreateBuilder(args);

Matcher serviceMatcher = new();
serviceMatcher.AddInclude("**/*.swagger.json");

var service1 = builder.AddProject<Projects.DiskService>(); // Dumps "{version}.swagger.json" to the project directory.
var service2 = builder.AddProject<Projects.WebService>("service2"); // Available at https+http://service2/path/to/swagger.json

var openApi = builder.AddOpenApi()
    .AddScalarUi()
    .WithLocalOpenApiDocument(service1, serviceMatcher) // Relies on copying the generated openapi document to the output location.
    .WithOpenApiUri(service2, "path/to/swagger.json") // An overload would accept a matcher instance, but I'm not sure how often this would be used.;

In both cases, both would get an enumerable of paths on disk. The WithOpenApiUri() would send a GET request and download the document to a well known working location, then return the path to that document.
There could be some cleanup/simplification here I'm sure, but I think this would at least cover our bases. Do let me know what you think or if I was able to answer your question.

I wasn't aware that you can generate OpenApi documents as a file. I'm only used to having them being served through HTTP GET method. This does explain the use of matcher nicely now.
Though, if I reading it correctly:

The WithOpenApiUri() would send a GET request and download the document to a well known working location, then return the path to that document.

This would mean saving it to a file, right? Is it necessary to do that? Not exactly sure about pros or cons of this solution.
In fact probably a better question: Is it better to have all OpenApi documents saved locally on a disk or have them all be available through HTTP GET method?

I wasn't aware you can generate OpenApi documents as a file.

I believe ASP.NET's OpenApi generator can spit out local files, but I wanted to provide this functionality to allow additional flexibility. There may be some services out there that provide the file on disk and you want to access it directly.

That would mean saving it to a file, right? Is it necessary to do that?

Good question. Technically speaking, you could save it in memory or just download it on the fly, but some sort of temporary local caching may be good if only for performance reasons. Some documents can be quite large depending on how they break up their APIs, which could lead to a large amount of bloat in memory.
I imagine mechanically, it would be simplest to provide a list of file paths to the front-end service and let it determine how to handle them.

Is it better to have all OpenApi documents saved locally on a disk or have them all be available through HTTP GET method?

I think even if we were to specify a preference, not all vendors will be able to comply with that preference. So the flexibility is important. As for providing them to our chosen front-end, I think the flat file method is probably the best just due to its simplicity.
As an aside, it could be interesting to parse all of the documents and merge them into some sort of in-memory database such as Redis, then have the database be consumed by whatever front-end. That would require vendor buy-in, but it could be a compact solution.

Technically speaking, you could save it in memory or just download it on the fly, but some sort of temporary local caching may be good if only for performance reasons.

I feel like there would probably have to be some kind of option to enable caching and disable caching. For example if I'm developing an ASP.NET web api service, I might not want caching on my local machine as I may be in very early stages of the development where things are changing constantly. However another example can be once an ASP.NET web api service is done and is put let's say on a staging environment on a remote server together with more web api services where I would want caching enabled.
The question then becomes: Would there also be different caching methods available? Would there just be one predefined?
I know the answer is probably: "It's better to let the developer have a choice." but sometimes I feel like too many choices may lead to a very complicated system. I guess I'm struggling to answer whether the pros outweigh the cost to build such system.

As an aside, it could be interesting to parse all of the documents and merge them into some sort of in-memory database such as Redis, then have the database be consumed by whatever front-end. That would require vendor buy-in, but it could be a compact solution.

This brings up a really good question as I have no idea who's job it would to be to parse all these documents. Considering there are both local and remote environments, I think local has an easy answer while remote environment currently doesn't really have an answer.
As far as I'm aware currently there is no such Aspire service that would act as a middle man between all the apis/services/databases and all the consumers like ones that collect OpenTelemetry or OpenApi documents.
I'm probably already going out of scope of this feature proposal but it almost feels like it would be great to have something like for the lack of a better word now: Aspire Proxy - a service that just acts like a middle man between all services and consumers, making it easier to hook up multiple consumers for the same purpose such as Swagger and Scalar for OpenApi documents or Aspire Dashboard and Seq for OpenTelemetry data.

For simplicity, the intial release may benefit from a lack of caching intitially. We could prvide an extension later to provide caching via MemoryCache/Redis/etc. That also provides a nice mechanism for enabling/disabling caching.
var openApi = builder.AddOpenApi().WithScalarUi();

if (Environment.IsStaging() || Environment.IsProduction())
{
    openApi.AddRedisCache();
}

For simplicity, the intial release may benefit from a lack of caching intitially.

Yes, I agree with this 100%.

Edit: I still think it should be the responsibility of the Ui implementations to parse and process these documents. Our open api components (via builder.AddOpenApi()) should only aggregate the documents for presentation.

I must be missing something extremely obvious. Where are the OpenApi documents going to be aggregated exactly if we're talking about a non local environment? I completely understand how it's gonna work on a local machine. I'm just really confused as to how it should work on other environments, if it's even supposed to work (I hope it does, otherwise I feel like it may be losing a full potential).
I created a .NET Aspire Starter App which has an API service and a Web service, along with AppHost and ServiceDefaults projects. When I do a manifest publish, I only get the two services:
{
  "resources": {
    "apiservice": {
      ...
    },
    "webfrontend": {
      ...
    }
  }
}

(I'm actually surprised the dashboard isn't included, but that's likely because I need to enable some kind of configuration)
Anyway what I mean by that there is no AppHost or any other service that would be able to do the job of "aggregating" all the OpenApi documents. Parsing the documents can be done by individual consumers, whether that's Swagger, Scalar or Dashboard, that's absolutely fine.
If this feature proposal means to create a new internal service, shouldn't such service be much more generic and have more purpose? Additionally shouldn't it also work the same on the local machine so there aren't two completely different ways of achieving the same thing?
(If it has been said before, I apologize but it seems like I'm starting to get a little bit lost in where exactly does this feature proposal aims to go)


Upgraded Angular 5 not working in production Mode

Hi,
Thanks for upgraded Angular 4 to Angular 5 But think is here,while publish or environment is in production,
webpack.config.js not able compile things properly which end up in error saying
Module Not Found:"Cannot resolve - ./$$_gendir/ClientApp/app/app.module.browser.ngfactory.ts
@ ./ClientApp/boot.browser.ts"
Have you come across the same if any

@tamizha09 I haven't tried production mode. I will try and locate the issue, but might not be soon. If you happen to find a solution before I get to look at it I am happy to accept a PR!

Thanks for quick response.
I tried to find some links
1.http://www.talkingdotnet.com/upgrade-angular-4-app-angular-5-visual-studio-2017/
2.https://github.com/long2know/angular-core-demo/
(Note: for running demo -> remove app_offline.htm in solution)
Above both sample works fine without server side pre rendering. but expected one is asp-prerender-module to load appsetting.json(appurl,identityurl).
if i include with pre-render-module and pre-render-data, it throws "No ResourceLoader implementation has been provided. Can't read the url "app.component.html"
Note:
In rule in webpack.config,remove include: /ClientApp/,
my package.json
"dependencies": {
"@angular/animations": "5.1.1",
"@angular/common": "5.1.1",
"@angular/compiler": "5.1.1",
"@angular/compiler-cli": "5.1.1",
"@angular/core": "5.1.1",
"@angular/forms": "5.1.1",
"@angular/http": "5.1.1",
"@angular/platform-browser": "5.1.1",
"@angular/platform-browser-dynamic": "5.1.1",
"@angular/platform-server": "5.1.1",
"@angular/router": "5.1.1",
"@types/webpack-env": "1.13.3",
"angular2-template-loader": "0.6.2",
"aspnet-prerendering": "3.0.1",
"aspnet-webpack": "2.0.1",
"awesome-typescript-loader": "3.4.1",
"bootstrap": "3.3.7",
"css": "2.2.1",
"css-loader": "0.28.7",
"es6-shim": "0.35.3",
"event-source-polyfill": "0.0.12",
"expose-loader": "0.7.4",
"extract-text-webpack-plugin": "3.0.2",
"file-loader": "1.1.6",
"html-loader": "0.5.1",
"isomorphic-fetch": "2.2.1",
"jquery": "3.2.1",
"json-loader": "0.5.7",
"preboot": "4.5.2",
"raw-loader": "0.5.1",
"reflect-metadata": "0.1.10",
"rxjs": "5.5.5",
"style-loader": "0.19.1",
"to-string-loader": "1.1.5",
"typescript": "2.6.2",
"url-loader": "0.6.2",
"webpack": "3.10.0",
"webpack-hot-middleware": "2.21.0",
"webpack-merge": "4.1.1",
"zone.js": "0.8.18"
},
"devDependencies": {
"@ngtools/webpack": "1.8.0",
"@types/chai": "4.0.10",
"@types/jasmine": "2.8.2",
"chai": "4.1.2",
"jasmine-core": "2.8.0",
"karma": "1.7.1",
"karma-chai": "0.1.0",
"karma-chrome-launcher": "2.2.0",
"karma-cli": "1.0.1",
"karma-jasmine": "1.1.1",
"karma-webpack": "2.0.9"
}
Still finding solution for pre rendering from server side,it will be helpful if anything finding from your side
Thanks

I was able to get publish to work, but I had to make some changes in the webpack.config.js file. Give this a shot and let me know if it works or not.
In the require section:
Before: const AotPlugin = require('@ngtools/webpack').AotPlugin;
After: const AngularCompilerPlugin = require('@ngtools/webpack').AngularCompilerPlugin;
Then in the rest of the file replace all of the AotPlugin with AngularCompilerPlugin.

Hi, when I published with this fix to Azure ClientApp page was not loaded - it generated error instead. This didn't happen with prior versions and it works OK locally.

Is this the same issue that issue #32 was opened for?

For those needing to get an Angular 4 app working, after a day of trying various things I had to do the reverse of @elanderson 's post and change (in webpack.config.js) line 4:
const AotPlugin = require('@ngtools/webpack').AngularCompilerPlugin;
to
const AotPlugin = require('@ngtools/webpack').AotPlugin;
For some reason the generated app had this as 'AngularCompilerPlugin'


Releasing failed in "Finalize Release" step

Turning on NPM provenance as id-token permission is set.
writing npm config to /home/runner/.npmrc
npm ERR! code E401
npm ERR! 401 Unauthorized - GET https://registry.npmjs.org/-/whoami
npm ERR! A complete log of this run can be found in: /home/runner/.npm/_logs/2024-03-16T19_55_29_604Z-debug-0.log
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.8/x64/bin/jupyter-releaser", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/cli.py", line 121, in invoke
    super().invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/cli.py", line 676, in publish_assets
    lib.publish_assets(
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/lib.py", line 379, in publish_assets
    util.run("npm whoami")
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/util.py", line 94, in run
    raise e
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/util.py", line 86, in run
    process = tee(cmd, **kwargs)
              ^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/tee.py", line [159](https://github.com/jupyterlab-contrib/jupyterlab-spreadsheet-editor/actions/runs/8310089368/job/22742184806#step:4:162), in run
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command 'npm whoami' returned non-zero exit status 1.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/actions/finalize_release.py", line [1](https://github.com/jupyterlab-contrib/jupyterlab-spreadsheet-editor/actions/runs/8310089368/job/22742184806#step:6:1)5, in <module>

I was able to publish the assets manually to both npm and PyPI and published the draft release:

https://www.npmjs.com/package/spreadsheet-editor/v/0.7.1
https://pypi.org/project/jupyterlab-spreadsheet-editor/0.7.1/
https://github.com/jupyterlab-contrib/jupyterlab-spreadsheet-editor/releases/tag/v0.7.1


DCEInst kill the same instruction twice.

In DCEInst, it is possible that the same instruction ends up in the
queue multiple times, if the same id is used multiple times in the
same instruction.
The solution is to keep the ids in a set, to ensure no duplication in
the list.
Fixes issue #1051

This fixes a crash in spirv-opt --eliminate-local-single-block that I observed earlier today.

I'll merge if the build bots are successful.