Upgrade IDIM Issuers to ACA-Py v0.10.1 and Askar only images.

[x] DEV

[x] Rotate RevReg: cred_def_id XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV

[x] SIT - Before September 6th

[x] Rotate RevReg: cred_def_id 7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT

[x] QA - Before September 6th

[x] Rotate RevReg: cred_def_id KCxVC8GkKywjhWJnUfCmkW:3:CL:20:PersonQA

[ ] PROD - Between September 8th and 12th

[ ] Rotate RevReg: cred_def_id RGjWbW1eycP7FrMf4QJvX8:3:CL:13:Person

Related task:

https://github.com/bcgov/DITP-DevOps/issues/103

Response from /revocation/active-registry/XpgeQa93eZvGSZBZef3PHn%3A3%3ACL%3A28075%3APersonDEV/rotate for dev:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9"
  ]
}

Response from /revocation/active-registry/XpgeQa93eZvGSZBZef3PHn%3A3%3ACL%3A28075%3APersonDEV/rotate for sit:
{
  "rev_reg_ids": [
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:50205010-00ae-4e47-b513-4ded5df05727",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:6b8710bb-79c8-4846-b257-378853336abf"
  ]
}

do we know the states of the registries (that are failing)?
if they are state=="init" nothing happens, and if they are not "active", then a new registry isn't created.

Where would I get that info?

Try: GET /revocation/registry/{rev_reg_id}

XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8 and XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9 are "state": "decommissioned"

Sigh, ok so they can get set to decommissioned and not kick off an init new registry. What we really need to know is (and I don't know how we could now...) is the state before the rotate call. It has to be "active" when you call rotate to kick off the init process.

They were active before the rotate call.  All were being actively used to issue and revoke credentials.

Looks to me like the RevReg rotation didn’t work. I don’t see any new RevRegDefs on CandyScan Dev, where the DIDs seem to be — not on Dev or SIT.  It looks like the RevRegs were decommissioned, but the new RevRegDefs were not created.

Link for SIT: https://candyscan.idlab.org/txs/CANDY_DEV/domain?page=1&pageSize=50&filterTxNames=["REVOC_REG_DEF"]&sortFromRecent=true&search=7xjfawcnyTUcduWVysLww5

They were active before the rotate call. All were being actively used to issue and revoke credentials.

That is the purpose of the rotate — to decommission the ones that were active, and to create new ones that become the active one.  However, it looks like the creation step didn’t work in this execution.

Well, someone put in a PATCH /revocation/registry/{rev_reg_id}/set-state?state=active API... it;s either that or create a new registry but don't think that's what you want?
I suppose if it doesn't get back to issuing then it will be POST /revocation/create-registry but that seems heavy handed.

They were active before the rotate call. All were being actively used to issue and revoke credentials.

That is the purpose of the rotate — to decommission the ones that were active, and to create new ones that become the active one. However, it looks like the creation step didn’t work in this execution.

Yeah, the only thing I can see is that kicking off the "init" process is dependent on notifications on the event bus... which is the same no matter when it's created but any chance of a message being missed? 🤷

They were active before the rotate call. All were being actively used to issue and revoke credentials.

That is the purpose of the rotate — to decommission the ones that were active, and to create new ones that become the active one. However, it looks like the creation step didn’t work in this execution.

Just to be clear, it will mark other registries as "decommissioned" but will only create/init a new one if the current state is active.  So just because it is marked decommissioned after the call doesn't mean that it was active before the call.

I confirmed the state by looking at the state in the prod environment, which has not been rotated yet.  All of the environments were identical, and the state in prod is active.

Agree — they must have been active.

So the fix in this case is to create new registries manually using /revocation/create-registry; correct?

I confirmed the state by looking at the state in the prod environment, which has not been rotated yet. All of the environments were identical, and the state in prod is active. SO I'd expect that new RevRegs would have been created.

OK, that sounds good and narrows it down... so more along the lines of what i put above and a hiccup on the notification...?
            init = IssuerRevRegRecord.STATE_ACTIVE == rec.state
            await self._set_registry_status(
                rec.revoc_reg_id, IssuerRevRegRecord.STATE_DECOMMISSIONED, init
            )

        if (state in IssuerRevRegRecord.TERMINAL_STATES) and init:
            return await self.init_issuer_registry(
                registry.cred_def_id,
                registry.max_cred_num,
                registry.revoc_def_type,
            )

terminal states are full and decommissioned, so if you pass in either of those AND that you want to init (prev state was active) then kick off the init process - which uses notifications and event bus.

Sorry, not sure what all that means.

So the fix in this case is to create new registries manually using /revocation/create-registry; correct?

I think so. POST body is something like this:
{
  "credential_definition_id": cred_def_id,
  "max_cred_num": 1000
}

Sorry, not sure what all that means.

just showing the key parts of the logic on rotate... if they were active then init was true and it should have kicked off the init_issuer_registry... since they were active and we didn't get a new registry was just looking at points of failure.  first thing I would think of is the notification didn't get pushed or received that does the work.

Ok, that covers the fix for our current situation.  Do you want a ticket in ACA-Py for tracking the permanent fix?  Would the fact that these agents are using an endorser service have anything to do with the new RevRegs not being created?  i.e. messages not getting to the endorser?

I hate to add more work, but can we back up DEV so we (might) have another try if this doesn’t work the first time?  I think this is the right thing as well.

I think so. POST body is something like this:
{
  "credential_definition_id": cred_def_id,
  "max_cred_num": 1000
}

Ok, that covers the fix for our current situation. Do you want a ticket in ACA-Py for tracking the permanent fix? Would the fact that these agents are using an endorser service have anything to do with the new RevRegs not being created? i.e. messages not getting to the endorser?

Yes — we need an ACA-Py ticket.  Nice to try it in the demo as the easiest way to verify the simple process works.  We can even try with an Endorser there, I think.

I'll create a ticket and link to this ticket for the details.

can we issue a few then rotate again? see if it repeatedly fails on dev?
so can you tell if the call didn't get to the endorser? or if the call didn't get back from the endorser? A lot of workflow creating the registry.

I hate to add more work, but can we back up DEV so we (might) have another try if this doesn’t work the first time? I think this is the right thing as well.

I think so. POST body is something like this:
{
  "credential_definition_id": cred_def_id,
  "max_cred_num": 1000
}

A backup is not going to help much if things get written tot he ledger.

A backup is not going to help much if things get written tot he ledger.

Agree — but our issue now is that things are not being written to the ledger.

The call to /revocation/create-registry returned 200 success, but nothing was written to the ledger.
{
  "result": {
    "state": "generated",
    "created_at": "2023-09-06T20:39:57.992060Z",
    "updated_at": "2023-09-06T20:40:01.104513Z",
    "record_id": "64293046-0895-403a-b695-6b3698e5f11b",
    "cred_def_id": "XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV",
    "issuer_did": "XpgeQa93eZvGSZBZef3PHn",
    "max_cred_num": 1000,
    "revoc_def_type": "CL_ACCUM",
    "revoc_reg_id": "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "revoc_reg_def": {
      "ver": "1.0",
      "id": "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
      "revocDefType": "CL_ACCUM",
      "tag": "64293046-0895-403a-b695-6b3698e5f11b",
      "credDefId": "XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV",
      "value": {
        "issuanceType": "ISSUANCE_BY_DEFAULT",
        "maxCredNum": 1000,
        "publicKeys": {
          "accumKey": {
            "z": "1 17D79CA6368D14B5A477BD5DB055DB3F723CA9CADF3429D414139308EE6B29C4 1 17EE55FCE54A448DBC92D2B2B0B9729C6005DB50F7154835DF77C088CF857430 1 06039690C3FB51CF04F104EBA451666C8CA743EAE0E632038545F7DDCBC7D7A2 1 177B8400364AAF2834B2BB8B0C4C057D80E8D7D9E0AF1013886FD0CCF8868729 1 1074873C05E4C528BD0CF6A4B8A81F23713FB3D01171D3FE370086459EAC175D 1 0F82C33B64D964C2482A22FD235631532414936965BCD03EEF1269B3A84C0CCF 1 06A2AC1C1B091066721878E5CCD7EFC2D5C5FC3795F80EF51C0D1E8485A47A49 1 22818EF0840B2D8E4E8D0B99F9D38EB1B328C0EC82CDA654CF5E475DFF9C6D90 1 15F857FADF48A3AEF47FB4F86BA4F84E78DB9D2A814DCABC7B3F132A389C63F6 1 0CA5EBD112CC7D2A87B23A0909A207E179C0749947404CD06DFE22F4F19CF9BA 1 103081758CA8CC9165A1DB256F5FD3FB1F6141EF82E20FB7C20DF5C7FF445DE8 1 09BEB8DDAF65EDC6616278A42E3D7CBD0F7C69E2ACBA2FFFC2EBE33E3F617FF8"
          }
        },
        "tailsHash": "FZScQrwGQWUfecfkMshLuoLFiMfJJPaXGLCeeo68m6RW",
        "tailsLocation": "/home/aries/.indy_client/tails/.hopper/FZScQrwGQWUfecfkMshLuoLFiMfJJPaXGLCeeo68m6RW"
      }
    },
    "revoc_reg_entry": {
      "ver": "1.0",
      "value": {
        "accum": "1 0391D9E0252B0239E20B10E14B410A03261C3C7E55B649D0785336EE8DA9CF88 1 1699333CCCD80488F67124DE4556CE56D0E874BA26E466CBB324ECB0F3A19140 1 20885B0A7CBF4970D1D9E9CDBDE1C3D61FA8FD50649D6BC738642DE31E7407B6 1 069CEA4B2FB138FABC87F3A92CC858BF8F4BDC18866E46069B0FAB86D61E5F88 2 095E45DDF417D05FB10933FFC63D474548B7FFFF7888802F07FFFFFF7D07A8A8 1<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000"
      }
    },
    "tag": "64293046-0895-403a-b695-6b3698e5f11b",
    "tails_hash": "FZScQrwGQWUfecfkMshLuoLFiMfJJPaXGLCeeo68m6RW",
    "tails_local_path": "/home/aries/.indy_client/tails/XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b/FZScQrwGQWUfecfkMshLuoLFiMfJJPaXGLCeeo68m6RW",
    "pending_pub": []
  }
}

Interesting — indicates perhaps that the /rotate was not the issue — the quietly failing non-ledger write is the issue.

Here are all of the registries the IDIM dev agent thinks it's created:
Of the 5 associated with XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV, only the two original ones were written to the ledger.
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25547:BC Person Credential (Dev):CL_ACCUM:467a8b7a-4b2a-46a7-8d3a-ee8195b685af",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person:CL_ACCUM:48864d46-0196-493f-9a6b-2d19a382f052",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:ca.bc.gov.iddev.verified_person:CL_ACCUM:6dca0943-f523-48c7-b507-9cb351977b34",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person:CL_ACCUM:d4006036-e86a-4163-a0ee-f290bad870ba",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25472:BC Person Credential (Dev):CL_ACCUM:ba29bf92-3207-45ca-bc77-0ac85148e8a1",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28060:Endorser Protocol Test:CL_ACCUM:553da5da-f50c-4701-89a0-e6f6beb230e2",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:26908:Person (Dev):CL_ACCUM:492c1ab1-455f-4e5c-a722-de0379bd70f5",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25547:BC Person Credential (Dev):CL_ACCUM:efe40df7-fd33-464d-92bd-de6c3621709e",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25472:BC Person Credential (Dev):CL_ACCUM:363558b8-859a-48ec-add7-4c21054f894d",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:Person (DEV):CL_ACCUM:5d4ef958-7073-4d19-9307-9aa3816c78fe",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:ca.bc.gov.iddev.verified_person:CL_ACCUM:e0ec320e-c096-4075-b3a5-264b677b7e90",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:26908:Person (Dev):CL_ACCUM:6186caed-4f00-4581-bd72-879a1e83f016",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25472:default:CL_ACCUM:59418b82-7364-4460-9fa0-0fb994b9377b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:Person (DEV):CL_ACCUM:89638367-d483-4fb6-a123-e73babc36c92",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person2:CL_ACCUM:71ff92a6-d7ad-4ae0-b824-a6659d0db4ee",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25472:BC Person Credential (Dev):CL_ACCUM:73b22f70-0dbe-4d69-9f99-75a5c15038de",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person4:CL_ACCUM:5135a8d7-8389-4b7a-8791-a314cd869f57",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person2:CL_ACCUM:33e66a7e-a925-44d9-9db1-ec5376ffc653",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:Person (DEV):CL_ACCUM:a2bde153-c624-4548-9b1d-94a9ec71d14f",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28060:Endorser Protocol Test:CL_ACCUM:c2e1c810-2e0b-4f19-93c8-fe28e1cfeae2",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person3:CL_ACCUM:16319413-7dfb-4899-9457-89a1d198f583",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:Person (DEV):CL_ACCUM:33ccad0e-8100-4ef7-b499-d5b60ea52101",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25361:verified_person4:CL_ACCUM:c69f5eaa-1405-4762-a5e2-118f42d94c49",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:25472:default:CL_ACCUM:0fd6902f-2ffd-4fc2-a971-f06b52b82400"
  ]
}

@usingtechnology, Can I leave this with you to investigate a little bit?  If you need anything from me just ask.

Well that is certainly an issue :-).  Not good that this was never noticed.

@usingtechnology, Can I leave this with you to investigate a little bit? If you need anything from me just ask.

@WadeBarnes — I would think this has to do with the deployment, since this works elsewhere. Endorser?  I think the logs are going to be needed at minimum.

I just realized what you meant with the list.  So all of the other RevRegs are there — just the last three — two that were attempted to be created via the “rotate” and the one you just created manually.

I just realized what you meant with the list. So all of the other RevRegs are there — just the last three — two that were attempted to be created via the “rotate” and the one you just created manually.

Yes

@WadeBarnes - if you can... the logs from the endorser and this agent would/may be helpful.

I'm not seeing any communication with the level logging is set to.  I'll have to increase the logging and try creating the registries.

I've updated https://github.com/bcgov/DITP-DevOps/issues/109#issuecomment-1709104949 with the list of RevRegs associated to XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV and their states.
I'm assuming the posted ones we'd expect to have been written to the ledger.  With the generated one, what are our expectations regarding that one?

generated means that it hasn't been written to the ledger, so we shouldn't see them on the ledger but they are on local storage.
flow is init -> generated -> posted.

Yeah, I have a few more steps to follow to get it posted; https://github.com/hyperledger/aries-cloudagent-python/blob/main/docs/GettingStartedAriesDev/CredentialRevocation.md#manually-creating-revocation-registries

I ran PATCH /revocation/registry/{rev_reg_id} to update the tails server URL, and then POST /revocation/registry/{rev_reg_id}/definition.
Received the error
500 Internal Server Error
Server got itself in trouble

and now XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b is in the posted state and not written to the ledger.
Error Logs:
aca-py-error.log

Not sure where to go from here.  Let me know if you've got any ideas.

Weird.
THere is this in the error log:
pydid.doc.doc.IdentifiedResourceMismatch: ID did:sov:4wvr7DC2XUtsHbpUpYaKvM#didcomm already found in Index and Items do not match
You've confirmed it's not the endorser, but may explain why nothing was getting out.
For whatever reason this duplicate is in the pydid did docs and failing sending outbound messages.
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/revocation/routes.py", line 1199, in send_rev_reg_def
    await outbound_handler(transaction_request, connection_id=connection_id)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/messaging/responder.py", line 101, in send
    return await self.send_outbound(
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/admin/server.py", line 147, in send_outbound
    return await self._send(profile, message)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/core/conductor.py", line 637, in outbound_message_router
    status: OutboundSendStatus = await self._outbound_message_router(
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/core/conductor.py", line 665, in _outbound_message_router
    return await self.queue_outbound(profile, outbound, inbound)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/core/conductor.py", line 697, in queue_outbound
    outbound.target_list = await self.dispatcher.run_task(
  File "/usr/local/lib/python3.9/asyncio/futures.py", line 284, in __await__
    yield self  # This tells Task to wait for completion.
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
    future.result()
  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step
    result = coro.send(None)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/connections/base_manager.py", line 609, in get_connection_targets
    targets = await self.fetch_connection_targets(connection)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/connections/base_manager.py", line 570, in fetch_connection_targets
    return await self.resolve_connection_targets(
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/connections/base_manager.py", line 362, in resolve_connection_targets
    doc, didcomm_services = await self.resolve_didcomm_services(did)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/connections/base_manager.py", line 269, in resolve_didcomm_services

so this agent is maybe not going to be able to send any outbound messages (as least when there is did lookup involved?)
may have to tag old school experts on how to fix the resolve_didcomm_services?

Weird. THere is this in the error log: pydid.doc.doc.IdentifiedResourceMismatch: ID did:sov:4wvr7DC2XUtsHbpUpYaKvM#didcomm already found in Index and Items do not match
You've confirmed it's not the endorser, but may explain why nothing was getting out. For whatever reason this duplicate is in the pydid did docs and failing sending outbound messages.

It's not the endorser's public DID, but it is the peer DID associated to the endorser connection.  Sorry for the confusion.

This might be a 0.10.1 issue if it is DID resolution.  Wade — can you tell if the Endorser was ever contacted? Or the ledger?  Seems not.
Jason — I assume the /rotate and the action that Wade DID should have gone through both the generated and posted steps?  E.g. there is not two actions that have to be done by the controller?  And if there are two, why there was no error surfaced in the operation.  I guess they are asynch, but there still should have been a notification of an error, I would think.

for the rotate and create, there may be errors in the logs but nothing would get surfaced because they are done via event/notification handlers. my guess (without the logs) is the same type of error happened when it finally sent something out the outbound queue to deliver to the endorser.

@dbluhm — could you pretty please take a look at the logs a few comments above?  This is an update to 0.10.1 and the connection to the endorser seems not to be working. Could this be a did resolution issue?

@WadeBarnes I've downloaded and scanned through the error log from the comment above. What is the result of GET /resolver/resolve/did:sov:4wvr7DC2XUtsHbpUpYaKvM?

did:sov:4wvr7DC2XUtsHbpUpYaKvM

{
  "did_document": {
    "@context": "https://w3id.org/did/v1",
    "id": "did:sov:4wvr7DC2XUtsHbpUpYaKvM",
    "authentication": [
      "did:sov:4wvr7DC2XUtsHbpUpYaKvM#1"
    ],
    "service": [
      {
        "id": "did:sov:4wvr7DC2XUtsHbpUpYaKvM#didcomm",
        "type": "did-communication",
        "priority": 0,
        "recipientKeys": [
          "did:sov:4wvr7DC2XUtsHbpUpYaKvM#1"
        ],
        "serviceEndpoint": "https://aries-endorser-agent-dev.apps.silver.devops.gov.bc.ca"
      },
      {
        "id": "did:sov:4wvr7DC2XUtsHbpUpYaKvM#didcomm",
        "type": "did-communication",
        "priority": 0,
        "recipientKeys": [
          "did:sov:4wvr7DC2XUtsHbpUpYaKvM#1"
        ],
        "serviceEndpoint": "wss://aries-endorser-agent-dev.apps.silver.devops.gov.bc.ca"
      }
    ],
    "verificationMethod": [
      {
        "id": "did:sov:4wvr7DC2XUtsHbpUpYaKvM#1",
        "type": "Ed25519VerificationKey2018",
        "controller": "did:sov:4wvr7DC2XUtsHbpUpYaKvM",
        "publicKeyBase58": "39msujHp1jPTPbWA5Ry9BnwGvbVw8WYGh8WYjoEryyQp"
      }
    ]
  },
  "metadata": {
    "resolver_type": "native",
    "resolver": "LegacyPeerDIDResolver",
    "retrieved_time": "2023-09-07T12:53:06Z",
    "duration": 42
  }
}

Shoot -- I should have caught this. The issue is the two service endpoints. The corrections from #2409 incorrectly assign the same ID for both. I'll open a PR to ACA-Py. @swcurran this will trigger a small cascade of things to correct this. I'll open an issue for this on ACA-Py.

As a result of the issues with ACA-Py 0.10.1 the IDIM agents have a number of decommissioned RevRegs which are written to the ledger, a number of posted RevRegs which never got posted to the ledger, and NO active RevRegs.
IDIM-Dev for example:
All:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

Posted:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

Decommissioned:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8"
  ]
}

Is there a way to get the posted ones written to the ledger?  If not, what is the recommended way to move forward with the RevReg rotation that was interrupted by 0.10.1?  I understand the /revocation/active-registry/{cred_def_id}/rotate endpoint will not create and post new RevRegs unless there are active ones.  Will it decommission the ones in the posted state?
cc @usingtechnology, @swcurran, @esune

I'm looking at using /revocation/registry/{rev_reg_id}/set-state to set the RevReg states so I can try reposting the ones that were created and not written to the ledger during the rotation.  I've tried to use /revocation/registry/{rev_reg_id}/set-state to set the state of the RevReg I manually created to decommissioned, but it does not accept that as a valid state; {"state": ["Must be one of: init, generated, posted, active, full."]}

@usingtechnology would calling the /rotate endpoint take care of this for @WadeBarnes, instead of having to manually fix things?

/rotate won't really help in this case. it will mark all as decommissioned but will not kick off a new registry process as there is no active reg to rotate out. I think @WadeBarnes setting the state is the first thing to try. if all is well then move on, but if it doesn't work, setting state to active then calling rotate should work.

@usingtechnology, With a "functional" setup the agent has two RevRegs written to the ledger the active state, yet when you call /revocation/active-registry/{cred_def_id} it only lists one.  How does AAC-Py know which one to use, does it just pick one (the frist), or do you need to explicitly tell it.  If you need to tell it how is that done?

active record is oldest published but not full.
there should be 2 active so we have an immediate fallover when we the working active is full.

Best way to decommission an unnecessary RevReg with /revocation/registry/{rev_reg_id}/set-state not understanding decommissioned would be to make it as full; correct?

that would be the way. just to be clear set-state does no additional processing.

OK, the idim-dev instance has been fixed.  The 2 RevRegs that were created during the rotation have been successfully posted to the ledger and activated.  The other one I created manually has been set to full.
Active:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

Full:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b"
  ]
}

Decommisioned:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8"
  ]
}

nice work @WadeBarnes !

nice work @WadeBarnes !

Thanks for the help!

IDIM is experiencing errors when publishing revocations to the new RegRegs.
idim-dev-revoke-error.log
@marcos-carretero, @swcurran, @usingtechnology, @esune, @shaangill025

With the new RevRegs being published manually, is there a step that I could have missed?  Does there need to be an initial RevRegEntry written to them?

I think I missed writing the tails file.

Tails files for the new registries have been published.

Wade Barnes @wade.barnes 11:01 AM
I think I missed writing the tails file. Looking.
Ok, try again. The tails files have been published.
Jay Cheng @jay.cheng 11:10 AM
I was able to revoke the credential issued earlier today; however, I issued another credential and getting http status 500 when revoking it
Timestamp would be around 11:07:50am

New log (time in UTC):
idim-dev-revoke-error.log

Reviewing the log in detail — the first errors in the sequence are from webhooks failing.  Those occur starting at 2023-09-08 18:07:49,424 to announce the completion of the issue-credential task. Then it looks like we get a Revocation Failure, and then an attempt to fix the revocation failure, which also fails.
Anyone know why the messages to the controller are failing? It would be nice to get that cleaned up so we know it is not an issue.
I think the attempt to fix the failure fails because the “fix” routine is not smart enough to handle the fixing a problem with the first publish revocation.  It expects it will get the “prev_accum” from the latest RevRegEntry, but in this case it needs to get it from the RevRegDef.
But that doesn’t solve why the RevRegEntry write fails in the first place.
There are also a bunch of errors at start up similiar to the DID Resolution fix from yesterday, but it appears that things proceed regardless. I’ll see if we can get those looked at.

Ah…the messages back to the controller are failing as 400s.  I bet that the controller is giving a 404 for the messages it is not prepared to receive/doesn’t care about.  Which is probably fine.  So I think we can safely ignore those.

I think I know what is going on.  The creation of a RevReg does two writes —  the RevRegDef and the first RevRegEntry that happens (usually) 3 seconds later. You can look at CandyScan here to see that.  If you look at the latest ledger entries.  The RevRegDefs were created along - no RevRegEntry.
I think the fix is to try to rotate the RevRegs again.  I hate that without really knowing, but I don’t see another option. Perhaps we take a database backup before...
My guess is the partial completion of the RevReg
2023-09-08 13:53:13,989 aries_cloudagent.admin.server DEBUG Match info: <MatchInfo {'rev_reg_id': 'XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a'}: <ResourceRoute [PUT] <DynamicResource  /revocation/registry/{rev_reg_id}/fix-revocation-entry-state> -> <function update_rev_reg_revoked_state at 0x7f1ac1587dc0>>
2023-09-08 13:53:13,989 aries_cloudagent.admin.server DEBUG Body: None
2023-09-08 13:53:13,989 aries_cloudagent.revocation.routes DEBUG >>> apply_ledger_update_json = false
2023-09-08 13:53:14,031 aries_cloudagent.revocation.routes DEBUG available write_ledgers = ['CANdyDev']
2023-09-08 13:53:14,031 aries_cloudagent.revocation.routes DEBUG write_ledger = <aries_cloudagent.ledger.indy_vdr.IndyVdrLedger object at 0x7f1aba0a9970>
2023-09-08 13:53:14,031 aries_cloudagent.revocation.routes DEBUG write_ledger pool = <aries_cloudagent.ledger.indy_vdr.IndyVdrLedgerPool object at 0x7f1ac06b2a60>
2023-09-08 13:53:14,031 aries_cloudagent.ledger.indy_vdr DEBUG Opening the pool ledger
2023-09-08 13:53:14,477 aries_cloudagent.admin.server ERROR Handler error with exception: 'NoneType' object is not subscriptable

=================
Traceback (most recent call last):
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/revocation/routes.py", line 928, in update_rev_reg_revoked_state
    ) = await rev_manager.update_rev_reg_revoked_state(
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/revocation/manager.py", line 180, in update_rev_reg_revoked_state
    return await rev_reg_record.fix_ledger_entry(
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/revocation/models/issuer_rev_reg_record.py", line 369, in fix_ledger_entry
    (rev_reg_delta, _) = await ledger.get_revoc_reg_delta(self.revoc_reg_id)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/ledger/indy_vdr.py", line 1066, in get_revoc_reg_delta
    "accum": response_value["accum_to"]["value"]["accum"],
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/admin/server.py", line 181, in ready_middleware
    return await handler(request)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/admin/server.py", line 218, in debug_middleware
    return await handler(request)
  File "/home/aries/.local/lib/python3.9/site-packages/aiohttp_apispec/middlewares.py", line 45, in validation_middleware
    return await handler(request)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/admin/server.py", line 338, in check_token
    return await handler(request)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/admin/server.py", line 450, in setup_context
    return await task
  File "/usr/local/lib/python3.9/asyncio/futures.py", line 284, in __await__
    yield self  # This tells Task to wait for completion.
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
    future.result()
  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step
    result = coro.send(None)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/revocation/routes.py", line 940, in update_rev_reg_revoked_state
    raise web.HTTPBadRequest(reason=str(err))
aiohttp.web_exceptions.HTTPBadRequest: 'NoneType' object is not subscriptable
2023-09-08 13:53:14,478 aiohttp.access INFO <IP_ADDRESS> [08/Sep/2023:13:53:13 +0000] "PUT /revocation/registry/XpgeQa93eZvGSZBZef3PHn%3A4%3AXpgeQa93eZvGSZBZef3PHn%3A3%3ACL%3A28075%3APersonDEV%3ACL_ACCUM%3A4a165d42-1215-41cd-b4dc-806855c37f1a/fix-revocation-entry-state?apply_ledger_update=false HTTP/1.1" 400 447 "https://idim-agent-admin-dev.apps.silver.devops.gov.bc.ca/api/doc" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"

hard to know which is empty/null/NoneType - response_value["accum_to"]["value"]["accum"]. could be either the accum_to or value.

I've confirmed the manual steps for creating the RevRegs is missing one final step, which is calling /revocation/registry/{rev_reg_id}/entry to post the initial accumulator value.  I've also confirmed this is only possible when there are no revoked credentials, i.e. before any other operations.  I'll submit a PR with the update to the docs.
I was able to post the initial RevRegEntry for the "backup" RevReg, but not for the "active" RegRev.  That leaves new RegRegs in a state where it's best to rotate them.
We can confirm this all again when I repair the sit environment.

After making a backup of the idim-dev wallet called /revocation/active-registry/{cred_def_id}/rotate.  This time the rotation completed successfully; https://candyscan.idlab.org/txs/CANDY_DEV/domain?page=1&pageSize=50&filterTxNames=["REVOC_REG_DEF","REVOC_REG_ENTRY"]&sortFromRecent=true&search=XpgeQa93eZvGSZBZef3PHn
Response from the call was:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

RevReg States for idim-dev:
Active:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:70093c2e-43f2-42fd-b376-5fd6b23a73b7",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:96fa823f-a320-4634-8708-b83ec0bddef2"
  ]
}

Decommissioned:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

So, the rotate set all of the previous RegRegs to decommissioned, including the one I had manually set to full.  @usingtechnology, is that the expected behavior?

Nice work!  Yes — that is the expected behaviour of rotate. That specific behaviour was discussed and agreed to when the feature was implemented.

At IDIM's request I've rotated the RevRegs on the dev instance again.
Response:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:70093c2e-43f2-42fd-b376-5fd6b23a73b7",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:96fa823f-a320-4634-8708-b83ec0bddef2",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

State:
Active:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:998398da-fb47-4c31-958b-1d730a581638",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1ae09ef3-acf7-4f3a-9a9b-f46374b82b21"
  ]
}

Decommissioned:
{
  "rev_reg_ids": [
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:70093c2e-43f2-42fd-b376-5fd6b23a73b7",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:a57e7adb-a44c-4ccb-9af1-b50ce77673be",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:1b6373a4-3200-4cbd-89c2-0984283e50f9",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:96fa823f-a320-4634-8708-b83ec0bddef2",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:d3f96e1c-951a-4a9b-9d31-6a6d921ee0c8",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:64293046-0895-403a-b695-6b3698e5f11b",
    "XpgeQa93eZvGSZBZef3PHn:4:XpgeQa93eZvGSZBZef3PHn:3:CL:28075:PersonDEV:CL_ACCUM:4a165d42-1215-41cd-b4dc-806855c37f1a"
  ]
}

Fixing idim-sit.  Current state of RevRegs for 7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT
Posted:
{
  "rev_reg_ids": [
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:ec28f7d7-23a0-474e-9f56-5c2fd3204d77",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:a81d6e90-b2ee-42ef-b0d9-fa81ffa2f3b8"
  ]
}

Decommissioned:
{
  "rev_reg_ids": [
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:50205010-00ae-4e47-b513-4ded5df05727",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:6b8710bb-79c8-4846-b257-378853336abf"
  ]
}

Steps to recover the RevRegs that were created during rotation, but not written to the ledger:

[ ] Set state generated
[ ] Post to ledger (tails URL is already set)
[ ] Write tails file
[ ] Post initial RevRegEntry

At IDIM's request I've rotated the RevRegs on the sit instance.
/revocation/active-registry/7xjfawcnyTUcduWVysLww5%3A3%3ACL%3A28075%3APersonSIT/rotate
{
  "rev_reg_ids": [
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:ec28f7d7-23a0-474e-9f56-5c2fd3204d77",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:a81d6e90-b2ee-42ef-b0d9-fa81ffa2f3b8",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:50205010-00ae-4e47-b513-4ded5df05727",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:6b8710bb-79c8-4846-b257-378853336abf"
  ]
}

Active:
{
  "rev_reg_ids": [
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:6280c6b5-36d3-419a-9e5b-0b1d993a9a8c",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:0e75ecfc-da6c-4265-8f92-da3fa3611397"
  ]
}

Decommissioned:
{
  "rev_reg_ids": [
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:ec28f7d7-23a0-474e-9f56-5c2fd3204d77",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:a81d6e90-b2ee-42ef-b0d9-fa81ffa2f3b8",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:50205010-00ae-4e47-b513-4ded5df05727",
    "7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:6b8710bb-79c8-4846-b257-378853336abf"
  ]
}

Txns for the new Registries:

https://candyscan.idlab.org/txs/CANDY_DEV/domain?page=1&pageSize=50&filterTxNames=[]&sortFromRecent=true&search=7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:6280c6b5-36d3-419a-9e5b-0b1d993a9a8c
https://candyscan.idlab.org/txs/CANDY_DEV/domain?page=1&pageSize=50&filterTxNames=[]&sortFromRecent=true&search=7xjfawcnyTUcduWVysLww5:4:7xjfawcnyTUcduWVysLww5:3:CL:28075:PersonSIT:CL_ACCUM:0e75ecfc-da6c-4265-8f92-da3fa3611397

IDIM reported testing was successful.

Agents have been upgraded to ACA-Py 0.10.2 using; the py3.9-0.10.2 image:

dev
sit
qa

The initial set of rotated RevRegs for KCxVC8GkKywjhWJnUfCmkW:3:CL:20:PersonQA have been recovered.
Active:
{
  "rev_reg_ids": [
    "KCxVC8GkKywjhWJnUfCmkW:4:KCxVC8GkKywjhWJnUfCmkW:3:CL:20:PersonQA:CL_ACCUM:351f78c4-27c1-44a0-a83d-5bea7d397ec8",
    "KCxVC8GkKywjhWJnUfCmkW:4:KCxVC8GkKywjhWJnUfCmkW:3:CL:20:PersonQA:CL_ACCUM:f0a718ef-af17-4360-a76c-649861de56b7"
  ]
}

https://candyscan.idlab.org/tx/CANDY_TEST/domain/56
https://candyscan.idlab.org/tx/CANDY_TEST/domain/58
https://candyscan.idlab.org/tx/CANDY_TEST/domain/57
https://candyscan.idlab.org/tx/CANDY_TEST/domain/59

Decommissioned:
{
  "rev_reg_ids": [
    "KCxVC8GkKywjhWJnUfCmkW:4:KCxVC8GkKywjhWJnUfCmkW:3:CL:20:PersonQA:CL_ACCUM:921ca010-04e7-4667-b613-5037e0459551",
    "KCxVC8GkKywjhWJnUfCmkW:4:KCxVC8GkKywjhWJnUfCmkW:3:CL:20:PersonQA:CL_ACCUM:b8ada6b0-c3c6-4e26-bb52-3b1aa9bef1b8"
  ]
}

IDIM prod has been upgraded to ACA-Py 0.10.2 using the py3.9-0.10.2 image, and the RevRegs have been rotated.
Active:
{
  "rev_reg_ids": [
    "RGjWbW1eycP7FrMf4QJvX8:4:RGjWbW1eycP7FrMf4QJvX8:3:CL:13:Person:CL_ACCUM:f954f297-07cd-444a-ba9b-96a126297ef2",
    "RGjWbW1eycP7FrMf4QJvX8:4:RGjWbW1eycP7FrMf4QJvX8:3:CL:13:Person:CL_ACCUM:0747da7c-1d0b-489e-aa84-9e94609e8174"
  ]
}

https://candyscan.idlab.org/tx/CANDY_PROD/domain/249
https://candyscan.idlab.org/tx/CANDY_PROD/domain/250
https://candyscan.idlab.org/tx/CANDY_PROD/domain/251
https://candyscan.idlab.org/tx/CANDY_PROD/domain/252

Decommissioned:
{
  "rev_reg_ids": [
    "RGjWbW1eycP7FrMf4QJvX8:4:RGjWbW1eycP7FrMf4QJvX8:3:CL:13:Person:CL_ACCUM:6a1fa86a-f5b2-4b26-ac94-ae165a63da10",
    "RGjWbW1eycP7FrMf4QJvX8:4:RGjWbW1eycP7FrMf4QJvX8:3:CL:13:Person:CL_ACCUM:badc49aa-6666-4a44-b375-060353258d0b"
  ]
}

This is finally complete


Count recursive calls

Hi,
I'm not able to count recursive calls:
describe "Count recursive calls" do
  class AClass
    def rec_function(try = 3)
      try > 0 ? rec_function(try - 1) : nil
    end
  end

  let!(:instance) { AClass.new }
  it {
    expect(instance).to receive(:rec_function).exactly(4).times
    instance.rec_function
  }
end

Is it normal ?
I use rspec-mocks 3.4.1
thanks.
Jules

Yes. The implementation doesn't exist. You need to add and_call_original

Yes thanks ! I found this ticket that had the answer : https://github.com/rspec/rspec-mocks/issues/917


convert option array to JS object

Another try. This paragraph, although informal, seems to imply that the "value0" approach is somewhat reliable.

Looks like we could consider moving the JS object creation to the PureScript side using https://github.com/purescript-contrib/purescript-argonaut-codecs/blob/master/src/Data/Argonaut/Combinators.purs


V1.5.3: Unrestricted File Upload Vulnerability

See PDF for details:
typemill-1.5.3-backstage.UploadVul.pdf

thank you for reporting, the upload-feature is for registered users only.
I will fix that asap and upload a hotfix version.

just published the hotfix <IP_ADDRESS>. I added a separate extension check and repeated the mimetype check when the file is uploaded to the temporary folder. If mimetype fails there, then the file is deleted immediately. Also updated the htaccess.
Vulnerability reported in #268 is still fixed.
The reason behind this security whole: Some environments did not support the mimetype extraction of a base64 string, so as a quick fix I made it conditiionally and this opened up for this vulnerability. Now solved properly be checking the mimetype of the stored file which works in all environments.
Thank you for reporting again!

Hi @trendschau
I'm new to Typemill, successfully installed Version <IP_ADDRESS> yesterday but found an issue with file upload. Folders and files have the appropriate permission and I can upload a picture with a FTP client to /media/... but not using the "image upload" button in the editor view. The image does not upload fully, meaning that the "loading" GIF below the uploaded image does not stop rotating waiting a few minutes. Similarly, for file upload, the file does not upload.
Since this might be related to this fix, I thought about leaving a comment here. Thanks for checking, I hope that it's not related to my infrastructure...

I cannot reproduce that error. Can you please open your developer tools in the browser, upload an image or file again and check the errors in the dev-tools? You should see something in the tabs "console" and/or "network".

Thanks for your quick response, here the error:

Can you click on "network" and then inspect the call to api/v1/image? Just click on the call and then open the tab "response" and please post the content of the response, there should be a detailed error message. Before you do that, please go to the settings in the admin area, scroll down to developer settings and activate the checkbox for report errors so that all error details are visible.

Thanks @trendschau for the hint. I found the error:
I got "Access denied by security policy"  coming from the WAF (mod security). Allowing the image upload process, I'm now able to upload images.


pdf needs special characters replaced

ahhh!

I can't figure this out, I'm going to have to leave the issue open as something that needs to be fixed.


Way to exclude pages from pages-by-date.html

May not want "Contact", "About", etc. pages in pages-by-date.html

Also exclude from Categories


Q-Learning - Documentation

I would like to add Q-learning - Reinforcement learning algorithm

I Would Like to Work on it . Plz Assign it to me.

Can I work on this @geekquad @ashwani-rathee @yukti845? I'll make the PR by today itself. Please assign it to me.


DCP dashboard enhancements

As per KT:
We had some requests for additional information to help us see what's in highest use currently on the Portal.  The suggestion was for the top 10 search requests and the top 10 downloaded/requested data sets. The goal is to see, for example, if most of the download requests are for things that are recently added to the DCP.

Obsolete.


[HOTFIX] Some clean-up in shuffle code.

Before diving into review #4450 I did a look through the existing shuffle
code to learn how it works. Unfortunately, there are some very
confusing things in this code. This patch makes a few small changes
to simplify things. It is not easily to concisely describe the changes
because of how convoluted the issues were, but they are fairly small
logically:

There is a trait named ShuffleBlockManager that only deals with
one logical function which is retrieving shuffle block data given shuffle
block coordinates. This trait has two implementors FileShuffleBlockManager
and IndexShuffleBlockManager. Confusingly the vast majority of those
implementations have nothing to do with this particular functionality.
So I've renamed the trait to ShuffleBlockResolver and documented it.
The aforementioned trait had two almost identical methods, for no good
reason. I removed one method (getBytes) and modified callers to use the
other one. I think the behavior is preserved in all cases.
The sort shuffle code uses an identifier "0" in the reduce slot of a
BlockID as a placeholder. I made it into a constant since it needs to
be consistent across multiple places.

I think for (3) there is actually a better solution that would avoid the
need to do this type of workaround/hack in the first place, but it's more
complex so I'm punting it for now.

/cc @aarondav and @rxin, with whom I discussed some of the existing design.

Test build #29456 has started for   PR 5286 at commit a406079.

Test build #29456 has finished for   PR 5286 at commit a406079.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29456/
Test FAILed.

Why is this a HOTFIX?

We have to put that whenever we don't create a JIRA or else the scripts we use get messed up. I can just make a JIRA for the overall clean-up.

At a high level LGTM

Test build #29458 has started for   PR 5286 at commit d1c0494.

Test build #29458 has finished for   PR 5286 at commit d1c0494.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29458/
Test FAILed.

LGTM too

Test build #29491 has started for   PR 5286 at commit f36edd5.

Test build #29491 has finished for   PR 5286 at commit f36edd5.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29491/
Test FAILed.

Jenkins, retest this pleas.e

Jenkins, retest this please.

Test build #29576 has started for   PR 5286 at commit f36edd5.

Test build #29576 has finished for   PR 5286 at commit f36edd5.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.
This patch removes the following dependencies:

RoaringBitmap-0.4.5.jar
activation-1.1.jar
akka-actor_2.10-2.3.4-spark.jar
akka-remote_2.10-2.3.4-spark.jar
akka-slf4j_2.10-2.3.4-spark.jar
aopalliance-1.0.jar
arpack_combined_all-0.1.jar
avro-1.7.7.jar
breeze-macros_2.10-0.11.2.jar
breeze_2.10-0.11.2.jar
chill-java-0.5.0.jar
chill_2.10-0.5.0.jar
commons-beanutils-1.7.0.jar
commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.1.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-digester-1.8.jar
commons-httpclient-3.1.jar
commons-io-2.1.jar
commons-lang-2.5.jar
commons-lang3-3.3.2.jar
commons-math-2.1.jar
commons-math3-3.1.1.jar
commons-net-2.2.jar
compress-lzf-1.0.0.jar
config-1.2.1.jar
core-1.1.2.jar
curator-client-2.4.0.jar
curator-framework-2.4.0.jar
curator-recipes-2.4.0.jar
gmbal-api-only-3.0.0-b023.jar
grizzly-framework-2.1.2.jar
grizzly-http-2.1.2.jar
grizzly-http-server-2.1.2.jar
grizzly-http-servlet-2.1.2.jar
grizzly-rcm-2.1.2.jar
groovy-all-2.3.7.jar
guava-14.0.1.jar
guice-3.0.jar
hadoop-annotations-2.2.0.jar
hadoop-auth-2.2.0.jar
hadoop-client-2.2.0.jar
hadoop-common-2.2.0.jar
hadoop-hdfs-2.2.0.jar
hadoop-mapreduce-client-app-2.2.0.jar
hadoop-mapreduce-client-common-2.2.0.jar
hadoop-mapreduce-client-core-2.2.0.jar
hadoop-mapreduce-client-jobclient-2.2.0.jar
hadoop-mapreduce-client-shuffle-2.2.0.jar
hadoop-yarn-api-2.2.0.jar
hadoop-yarn-client-2.2.0.jar
hadoop-yarn-common-2.2.0.jar
hadoop-yarn-server-common-2.2.0.jar
ivy-2.4.0.jar
jackson-annotations-2.4.0.jar
jackson-core-2.4.4.jar
jackson-core-asl-1.8.8.jar
jackson-databind-2.4.4.jar
jackson-jaxrs-1.8.8.jar
jackson-mapper-asl-1.8.8.jar
jackson-module-scala_2.10-2.4.4.jar
jackson-xc-1.8.8.jar
jansi-1.4.jar
javax.inject-1.jar
javax.servlet-3.0.0.v201112011016.jar
javax.servlet-3.1.jar
javax.servlet-api-3.0.1.jar
jaxb-api-2.2.2.jar
jaxb-impl-2.2.3-1.jar
jcl-over-slf4j-1.7.10.jar
jersey-client-1.9.jar
jersey-core-1.9.jar
jersey-grizzly2-1.9.jar
jersey-guice-1.9.jar
jersey-json-1.9.jar
jersey-server-1.9.jar
jersey-test-framework-core-1.9.jar
jersey-test-framework-grizzly2-1.9.jar
jets3t-0.7.1.jar
jettison-1.1.jar
jetty-util-6.1.26.jar
jline-0.9.94.jar
jline-2.10.4.jar
jodd-core-3.6.3.jar
json4s-ast_2.10-3.2.10.jar
json4s-core_2.10-3.2.10.jar
json4s-jackson_2.10-3.2.10.jar
jsr305-1.3.9.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.10.jar
kryo-2.21.jar
log4j-1.2.17.jar
lz4-1.2.0.jar
management-api-3.0.0-b012.jar
mesos-0.21.0-shaded-protobuf.jar
metrics-core-3.1.0.jar
metrics-graphite-3.1.0.jar
metrics-json-3.1.0.jar
metrics-jvm-3.1.0.jar
minlog-1.2.jar
netty-3.8.0.Final.jar
netty-all-4.0.23.Final.jar
objenesis-1.2.jar
opencsv-2.3.jar
oro-2.0.8.jar
paranamer-2.6.jar
parquet-column-1.6.0rc3.jar
parquet-common-1.6.0rc3.jar
parquet-encoding-1.6.0rc3.jar
parquet-format-2.2.0-rc1.jar
parquet-generator-1.6.0rc3.jar
parquet-hadoop-1.6.0rc3.jar
parquet-jackson-1.6.0rc3.jar
protobuf-java-2.4.1.jar
protobuf-java-2.5.0-spark.jar
py4j-<IP_ADDRESS>.jar
pyrolite-2.0.1.jar
quasiquotes_2.10-2.0.1.jar
reflectasm-1.07-shaded.jar
scala-compiler-2.10.4.jar
scala-library-2.10.4.jar
scala-reflect-2.10.4.jar
scalap-2.10.4.jar
scalatest_2.10-2.2.1.jar
slf4j-api-1.7.10.jar
slf4j-log4j12-1.7.10.jar
snappy-java-<IP_ADDRESS>.jar
spark-bagel_2.10-1.4.0-SNAPSHOT.jar
spark-catalyst_2.10-1.4.0-SNAPSHOT.jar
spark-core_2.10-1.4.0-SNAPSHOT.jar
spark-graphx_2.10-1.4.0-SNAPSHOT.jar
spark-launcher_2.10-1.4.0-SNAPSHOT.jar
spark-mllib_2.10-1.4.0-SNAPSHOT.jar
spark-network-common_2.10-1.4.0-SNAPSHOT.jar
spark-network-shuffle_2.10-1.4.0-SNAPSHOT.jar
spark-repl_2.10-1.4.0-SNAPSHOT.jar
spark-sql_2.10-1.4.0-SNAPSHOT.jar
spark-streaming_2.10-1.4.0-SNAPSHOT.jar
spire-macros_2.10-0.7.4.jar
spire_2.10-0.7.4.jar
stax-api-1.0.1.jar
stream-2.7.0.jar
tachyon-0.5.0.jar
tachyon-client-0.5.0.jar
uncommons-maths-1.2.2a.jar
unused-1.0.0.jar
xmlenc-0.52.jar
xz-1.0.jar
zookeeper-3.4.5.jar

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29576/
Test FAILed.

Jenkins, retest this please.

/cc @brennonyork - this is giving me strange dependency messages even though the patch doesn't touch any pom files.

Test build #29585 has started for   PR 5286 at commit c71fbc7.

Test build #29585 has finished for   PR 5286 at commit c71fbc7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29585/
Test PASSed.


add support for TLS

Is there a way to add TLS within the tower-web library even if just a workaround approach? I'm trying to add rustls to the application and running into issues.

This should already be possible today!
You would use the ServiceBuilder::serve function (docs). This lets you pass in your own acceptor. You can pass in a TLS acceptor if you want.
So, you would do something like:
let incoming = TcpListener::bind(...).unwrap()
    .incoming()
    .map(|tcp_stream| wrap_with_tls(tcp_stream));

tokio::run({
    ServerBuilder::new()
        // configure service
        .serve(incoming)
});

Hope this helps.

So when I use the serve endpoint I get the following error:

error[E0277]:  the trait bound tokio_rustls::TlsStream<tokio::net::TcpStream, rustls::server::ServerSession>: tower_web::net::Connection is not satisfied
--> src/main.rs:516:10
|
516 |         .serve(incoming)
|          ^^^^^ the trait tower_web::net::Connection is not implemented for tokio_rustls::TlsStream<tokio::net::TcpStream, rustls::server::ServerSession>
|
= note: required because of the requirements on the impl of tower_web::net::ConnectionStream for futures::stream::and_then::AndThen<futures::stream::map::Map<tokio::net::tcp::Incoming, [closure@src/main.rs:465:14: 465:59 wrap_with_tls:_]>, [closure@src/main.rs:466:19: 466:42], tokio_rustls::Accepttokio::net::TcpStream>

The only place I can get it working is to implement the Connection trait for the tokio_rustls::TlsStream is in the tower-web/src/net.rs file as such:
diff --git a/Cargo.toml b/Cargo.toml
index bcc20df..c5eb476 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -47,6 +47,7 @@ percent-encoding = "1.0.1"
 tokio = "0.1.6"
 tokio-fs = "0.1.2"
 tokio-io = "0.1.7"
+tokio-rustls = "0.8.0"
 tower-service = "0.1.0"
 void = "1.0.2"

diff --git a/src/lib.rs b/src/lib.rs
index 506c133..596c3a7 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -480,6 +480,7 @@ extern crate tokio;
 extern crate tokio_fs;
 extern crate tokio_io;
 extern crate tower_service;
+extern crate tokio_rustls;
 extern crate void;

 #[cfg(feature = "handlebars")]
diff --git a/src/net.rs b/src/net.rs
index 960f3d6..2e09579 100644
--- a/src/net.rs
+++ b/src/net.rs
@@ -7,6 +7,8 @@ use tokio_io::{AsyncRead, AsyncWrite};
 use std::io;
 use std::net::SocketAddr;

+use tokio_rustls::{TlsStream, rustls::ServerSession};
+
 /// A stream between a local and remote target.
 pub trait Connection: AsyncRead + AsyncWrite {
     /// Returns the socket address of the remote peer of this connection.
@@ -29,6 +31,12 @@ impl Connection for TcpStream {
     }
 }

+impl Connection for TlsStream<TcpStream, ServerSession> {
+    fn peer_addr(&self) -> Option<SocketAddr> {
+        TcpStream::peer_addr(self.get_ref().0).ok()
+    }
+}
+
 impl<T> ConnectionStream for T
 where
     T: Stream<Error = io::Error>,

I'm I doing something wrong? I'm thinking maybe there is another way to implement the wrap_with_tls function. I'm doing the following:
    let wrap_with_tls = {
        let mut config = ServerConfig::new(NoClientAuth::new());
        config.set_single_cert(load_certs(&*CERTS.public), load_private_keys(&*CERTS.private).remove(0))
            .expect("invalid key or certificate");
        TlsAcceptor::from(Arc::new(config))
    };

    let incoming = TcpListener::bind(&addr).unwrap()
        .incoming()
        .map(move |tcp_stream| wrap_with_tls.accept(tcp_stream))
        .and_then(|tls_stream| tls_stream);

Ah yes, you are correct.
The hack work around would be to define your own wrapper around TlsStream that implements Connection.
To do this better, we would want to add a feature flag to tower-web to add support for rustls.
Could you open a PR w/ this diff but make the rustls dependency optional?

will do


In windows, line breaking not work.

I insert \r\n in cell value. But The line breaking only work after I click the edit input or double click the cell.
How to mark line breaking work in windows ?

const wbout = XLSX.write(workbook, {bookType: 'xlsx', bookSST: true, type: 'binary'});
  const wblob = new Blob([s2ab(wbout)], {type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'});

Why closed? I met the same issue.
Did you figured out?

@NemoAlex
refer: https://github.com/protobi/js-xlsx/issues/73

Any progress on this issue?

Forcing a line break to properly render requires setting a style flag which is currently only supported in our Pro builds


vfio-setup script says that both dGPUs must be passed through

When running vfio-setup, I get to the following output:
These are your graphic cards, they have to be in separate groups.
The graphic card you want to passthrough cannot be in a group with other devices that
does not belong to itself. Both cards must also have unique hardware ids [xxxx:yyyy]!:

#------------------------------------------#
IOMMU Group  15: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
IOMMU Group  17: NVIDIA Corporation GA104 [GeForce RTX 3070 Ti] [10de:2482] (rev a1)
#------------------------------------------#

Press q to quit
Which group number do you want to check?:

I want to pass through my RTX 3070 Ti, so I enter '17' here and press ENTER, then receive the following output:
For this card to be passthrough-able, it must contain only:
* The GPU/Graphic card
* The GPU Audio Controller

Optionally it may also include:
* GPU USB Host Controller
* GPU Serial Port
* GPU USB Type-C UCSI Controller
* PCI Bridge (if they are in their own IOMMU groups)

#------------------------------------------#
IOMMU Group  15: Audio device: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
IOMMU Group  15: VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
IOMMU Group  17: Audio device: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1)
IOMMU Group  17: VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3070 Ti] [10de:2482] (rev a1)
#------------------------------------------#

To use any of these devices for passthrough ALL of them (except PCI bridges in their own IOMMU groups) has to be passed through to the VMs
To return to the previous page just press ENTER without typing in anything.
Do you want to use these devices for passthrough? [y/N]:

Is it normal behaviour for the script to say that both GPUs must be passed through, despite being in separate groups?

Ah, its one of the test cases i couldnt test for myself as i dont have 2 amd gpus or 2 nvidia gpus.
its an issue i need to fix in ls-iommu before i can fix it in vfio-setup. Thank you for catching that @pcrxn i will get to making a fix for that right away since i have a couple of minutes downtime. Worst case the fix will be out tomorrow


CV with Py3.8.6

Hi
After I have installed Visual Studio 2017 (I think C++ needed) it was starting, running on Py3.9.

But as I touched any object on the Graphics Window it has stucked.
After Installing 3.8.6 the GW is alive for 8-10sec, then stops to show the measurins while the program runs in the Command Window.
Can you tell me how to forward?

Can you share the current_viewer.log (if it's not too large, otherwise delete it and try to do a repro of the issue)?
From the description it almost sounds like the UI is getting stuck (I assume you did not press Pause?). It also looks like the sample speed is much lower than expected (110 vs 600-700), do you see any negative readings or erroneous floating point numbers coming from CR?

Hi,
This is the log of 3.9 and the 3.8.6.
There is a bit of overlapping.in the files.
The 3.9 was continuously working and displaying the graph.
All Captures made here by Snipping Tool.
As I touched any buttons, it crashed.
The 3.8 stopped very immediately of start (of course the Pause is not pressed)
Thanks, Csaba
current_viewer.log
current_viewer_39.log

The Captures.

At first look it seems there is a number USB data corruption errors, random characters interleaved with the sample data. Normally all the samples should be coming back in the format "1.234e-05\r\n". It would be a wild guess as to what's causing it, perhaps bad USB cable or grounding issue with the PC (are you running CR off LiPo?)

The corrupted data coming from the CR, if I use Putty and log the stream, it is randomly coming. My unit sending corrupted data since I have started the USB data collection. Not many corrupted records, I made a quick stat, in that case in 100k samples less than 10 pcs. was corrupted. The appearance looks absolutely stochastic. This is a good feature in CV that handles the filtering!
I have purchased a new CR, not yet arrived, I will check on that.
The cabling looks good quality, but I will change it see what happens!
The CR is on LiPo, I use optoisolator on the USB link, also opto on the DUT's terminal UART/USB converter, so ground loop is prevented. I am aware of ground loop, because many times it made crazy.
Besides the graphical visualization, the data logging (to a CSV file) would increase the value of CV. That would help the post analysis of the CR measurements.

I'm working on adding CSV export (and perhaps JSON too :) ). There are a few improvements in progress in regards to speed on slower CPUs (matplotlib is CPU intensive for large data sets) and those are a bit more involved, but hope to have something out in the next couple days. In the meantime, can you check if reducing the buffer size to 1000 (instead of the default 10,000) will increase the sample speed and the UI responsiveness on your machine? The parameter is here: https://github.com/MGX3D/CurrentViewer/blob/eda24dad39fee8ae4d52198455bba561af4081f5/current_viewer.py#L261

My logging PC is not a craft-machine:

Decreased the buffer to 1000, it works (wind sensor in sleep period):

I tried at csp=5000, was dead.
At csp=2500 is looks to work.

Regarding to opto isolator and data errors.
First I tried without the opto on separated the PC from the system ground.
The result was the same.
I wait for the new CR unit what should arrive in next days, then I can check the difference.
I am really happy about the new logging feature to to CSV file. The graphics window is very nice if I need to show the details for others. But more important is to analyze the happenings in 1,2,3.... days operation. Only the detailed log can handle that.

Check out the most recent version (1.0.1) - it adds CSV/JSON export, as well as many improvements for visualization speed.  Let me know how this works for you :)

This is perfect solution! THANKS!
You addressed all future emerging options!
Runs perfectly with default parameters at buffer 100.000!
There is a logging conversion problem, see in the files.
current_viewer.log
data2.txt
The used options: current_viewer.py -p COM18 -g -c -vvv
For the highest debug verbose -vvv is the option?
I have not checked the graphics functions yet.

Thanks for the nice words!
Errors like these are expected when the data is coming corrupted from hardware/USB (there is nothing CurrentViewer can do to parse it, these are not valid floating point numbers):
ValueError("could not convert string to float: '43.9531.45e-6\r\n'")
CurrentView filters out the obviously invalid ones, but indeed if the floating point number appear to be in the correct representation (such is the case with "2020-11-11 16:48:35.905941**,40.6**") it will still export it. Obviously this value is corrupted as well, but in a different way, as CurrentRanger cannot possibly measure 40Amps :).
Do you have the latest firmware on the CR? (share the output of the '?' command) Because the data is both corrupted and also coming 6x (!) slower than I would expect.

I will analyze the source of corrupted data as I will have the new purchased CR units in my hand.
The new one should work well!
The present CR unit has the latest firmware, was flashed on Nov 2.
The speed also will be examined using faster PC.
As I see the speed is set up by the CR and PC internal USB service communication.
Putty and Arduino Serial Monitor runs with the same speed (110SPS) while 115k baudrate has been setup!?
It will take some days to finish because I will have a 2 days program from tomorrow.
I will report asap. Thanks!

Some experiences:
48 hours of logging over 530MB file size at 110SPS, The no of records doesn't fit to an Excel page (1M lines), it is about 1.5M records.
I think the most of the data logs can be omitted from the logfile without lost information about the relevant power consumption of the DUT.
Most of the data arriving are the LowPower consumption, all around 40uA (in my case), what is really not important to log each of them. We just interested the higher an lower samples.
We could decrease the no of less important data content by an adaptive logging.
Range logging, when the sample is out of two threshold defined, will be logged.
Differential logging  where the defined relative change of the sample in % tells to write it to the log or not.
I have done once this kind of logger to record the current flow of a car battery, where the starter motor needed a fast logging, but other activities are not.
This kind of "data adaptive" logging would decrease IoT devices power logs file size about 98%, I suppose.
The other small thing:
if I stop the CV.py program and restart and I forget to change the -o file.csv filename it is purging the previous log content. I feel better to stop the run with the error message: file exist, or something like this.
Or just append a timestamp to the filename.

I made a graph of 1M samples. There were many negative value in the file and also unrealistic value. I have filtered:

Now my sampling is 5min, normally sampling is 15, 30 min. Maybe not 98% the saving but 50% looks realistic.

A bit more testing:
50.5 hours logging was perfectly stable. The file created is 703MB.
I have found a CSV splitter what chopped it to 20 pcs. of 1M line files: https://sourceforge.net/projects/splitcsv/files/split.exe/download
The analysis of the first 1M records:

There are 59 negative data, 79 is over 1A what are unrealistic as this DUT contains a RN2483 LoRa radio module. Some sensors can go up to 170mA peak one by one, but overall consumption surely under 1 Amp.
The 91% is in the range of Low Power 20-60 uA.
If the logging would be done using a simple data filtering the log can be decreased by 85%.

On the 3rd day it stops logging and continuously sending the same item to the current_viewer.log file.
Now I have this experience 3rd times.
I don't see any correlations in file sizes or record numbers.

I was not able to catch the edge when the logging stops, to see the preliminary circumstances.
The 1M logfile size was not enough to catch.
I have received the new CRs. One is installed, it has established a 105SPS link. The same errors in data stream.
I will continue testing.

I added a new option --log-size in 1.0.2 that allows you to customize the log size maximum
Increased log size for debugging

> python current_viewer.py -p COM9 -vvv --log-size 128

This increases the log maximum size to 128 megabytes from the default of 1 (alteratively --log-size 0.1 would cap the log at ~100Kb). It could be useful in capturing hard to reproduce errors. Note there are two logs on disk: the current one (current_viewer.log) and the rotated one (current_viewer.log.1), so the total log on disk is actually double this parameter.

As for the issue you're seeing, I'm beginning to suspect a problem on the PC side (either USB or USB drivers). Do you have another PC to test, or perhaps use a different USB port (check the device hierarchy in Device Manager / view by connection), or remove the USB hub if you have one. Here is how it looks for me (CR USB is on COM9 and BT is one of the 12-22 ports :) ):

Check out the latest version, some fixes around data corruption and exception handling. Note that now CV will abort faster in some scenarios, such as high data corruption rate, or USB serial exceptions (this also helps the logs from being rotated out and preserve the relevant errors from before - like it was happening in your case)
I did a bit more testing and the error 22 above is typically what follows a device unplug scenario. Hopefully the logs in the new version will contain the errors before error 22 (there is still some logging from other things in python, so you might still want to use the --log-size option I mentioned above).
A disconnect error will typically look like this in the log (and the chart title will update to  in red):
ERROR:2020-11-24 20:38:13,448:Thread-1:Serial read error: None: (<class 'serial.serialutil.SerialException'>, SerialException("ClearCommError failed (PermissionError(13, 'The device does not recognize the command.', None, 22))"), <traceback object at 0x0000024F0E333A00>)
INFO:2020-11-24 20:38:13,448:Thread-1:Telling CR to stop USB streaming
WARNING:2020-11-24 20:38:13,448:Thread-1:Was not able to clean disconnect from the device

Installing on my fastest PC, I am here:
As I understand the install is fine:

But stating the *.py there is some glitch:

What to do?

Check out this solution (and thread): https://developercommunity.visualstudio.com/solutions/1257144/view.html
Also make sure to refresh CV to 1.0.6 - should fix the random data corruption issues (it's a bug in python's libraries as well - readline() does not appear to be thread safe).  I guess remaining issue will be speed, curious how this turns out after trying the steps above

Thanks it worked.
Now I am struggling with the USB driver on the new PC install and also on the old Win10 install what worked till now.
I tried to install Arduino Zero driver, but no success.

Also I have the same issue on my Win7 install.
Do you know where I find a Win10 , Win7 driver for SamD21?

Not sure what devices you have there, as VID_0000 PID_0002 seems invalid hardware ID, besides:

Windows 10 has a generic USB serial driver built in, should work without installing any drivers (and in device manager driver will say usbser.inf)
CR for me shows up as HW ID USB\VID_04D8&PID_EE4C&REV_0100&MI_00  (and when in flashing mode is USB\VID_04D8&PID_EE44&REV_4201&MI_00)

Perhaps delete all the strange USB drivers and start fresh (from device manager click uninstall and check the "delete the driver,,")

Then unplug it and plug it back in and make sure it gets an inbox or WU driver.

Finally I have got over all USB isues on WIn10 and Win7 too.
I am running on my DELL G3 the CV:

It get over 1000SPS for a peak sometimes.
But then stops, python program die.

I try to copy some files for you to find the problem:

Then starting in with options: -p COM18 -g -c -vvv -o data1.csv --log-size 128
I got logging speed less than 100SPS.
I have changed to -vv:

I have got steady 1300+SPS :-) Wow!
I leave it till the morning, we will se the results.

Nice, congrats!
The log is completely corrupted, but the Shell Window Content shows you're running CV 1.0.5 - upgrade to 1.0.6 or newer which has a new serial read method that appears to work much better against these random data corruptions.

Congrats for you! 1.0.7. works perfect.

The logger produced 1GB file in 7 hours at 1300SPS with no error.
That is about 33M records. I don't know what app could analyze it, Excel can handle 1M records.
CV is sampling/logging according to the set CR ADC speed (low, avg, fast) at 230SPS, 650SPS, 1300SPS.
As I discussed earlier for long time logging most of the uninteresting data could be filtered from the logfile:

The LowPower state data should be averaged and logged as two datapoints in the file.

The annotated section of the incoming data could be represented in the data log file with the first measure entering the low power range (eg: 30 - 60 uA), and when the current goes out of the LP range the last logged LP data could be the average of the full annotated period.
This way the long logging could have a limited size with the relevant data.
What do you think?
I will test later with the slower PC the 1.0.7. version.

The 1.0.7 is running fine on the old PC. But runs at 110SPS and a lot of conversion error in runtime.
You can see the errors in the shell display:
SHELL_content.txt
There are no errors in the log files, probably because of -vv option instead of -vvv?
The GUI is works with default options, saving also fine:

I used the same USB hardware (cable, isolator) as with the fast PC running on 1300SPS.
It should be the USB hardware or the CPU of the PC what makes the difference?
I will test on a PC what is is between the two in performance.

It's possible but it would be hard to believe the CPU is the limiting factor here, unless it's much slower than a Raspberry Pi 4. For reference, a Raspbery Pi 400 does in excess of 2000 SPS  (with firmware mods to send more data). What kind of CPU we're talking about here?  (CV doesn't log the CPU speed yet).
Can you do this benchmark from a command prompt: "type COM3 >test.raw" and Ctrl-C after 30s, then divide the number of rows you got in test.row with the number of seconds it ran for?
The errors are logged always. -vvv just increases the verbosity for less severe messages. But yes you're right, those python exceptions can really slow things down. It seems there are still some data corruptions (eg newline missing), and I suspect they're coming from the python/serial libraries again, seems I haven't flushed out all of them.
I hear you on the need to reduce the data acquisition volume (and it is funny too, considering I was trying to collect as much data as possible up until now :) ), I'll think of a way to add smart filtering in next CV version but I'm also in talks with Felix as I believe this kind of feature would ideally live on the CurrentRanger: it's best not to send redundant/repeating data in the 1st place, as the USB-CDC speed is the limiting factor right now anyway. Plus it would help with other apps as well (eg arduino serial monitor).
I think the filtering method requires a bit more researching with real-world data patterns (eg, are the regular dots in your cart actual spikes in consumption or noise/interference) as to not filter out useful data.

This CPU:

This is fine for Arduino developments.
The command is not working:

Since I have changed to the new CR unit what I have purchased from Welectron in Germany last month, things are going smoother. And I am using an external self powered D-Link USB-Hub. Plus I use the USB isolator.
I have upgraded the new CV FW to 1.1.0. version, running in fast sampling mode, is better than the old CR with the same FW. There are no conversion errors anymore!

Data collection speed should be set for the different applications.
High speed is needed to analyze high speed data or deep details in time domain.
For the IoT device's peripherals debug the 1-2 kSPS logging speed is usually quite enough. CR and CV still fine for this kind of project.
But when I need MCU level fast (and short) logging I use the Saleae Pro what can log at 500MSPS. https://www.saleae.com/home2/
Low speed log of IoT devices, checking the operating situations in long range on the field. The environmental IoT sensors are active for some hundred of millisecs, so 10-100SPS should be enough to detect the issue if exists.
You are absolutely right it should be qualified in CR, and not sending unneccesary data. As I mentioned I did an implementation it in a 700A/20mA measuring range in the case of a car battery.


[FEATURE] Links do not have correct endpoint

Description
In the 3rd section of the how it works view are 4 links which are currently not directing to an end point. If possible (if the page already exists), link them to the right page. Otherwise let's keep them as they are.
Screenshots
Section

Additional information
No response

Please assign this to me

hello, can you assign me to this issue?

Sure @Nwakaego-Ego, happy to have you contributin to the project 😄

Hi @Nwakaego-Ego. Are you working on this ticket? Two weeks already passed. As mentioned in the contribution guide, assignees will get unassigned when there is no progress within 2 weeks.

Hi, @YuriDevAT I have had a lot of issues running this from my terminal, i just finally succeded, I want to jump right on it. I am very sorry about the delay


Fin team reports that sensors are not working out of the box on the Fin

@alanb128 the guys mentioned this to me in the hardware call but I've not tested yet, so just logging it here.

I've just tested joining the open fleet with a Fin and a BME680 - seems to work OK.

I just tested the simple-sensor with a BMP-280 on the Fin and it worked as well. I did not see the BME680 kernel module in balenaOS (2.80.3+rev1, prod) so I'm surprised that one worked @chrisys - which version of the OS did you use?

@alanb128 my device is in the open fleet if you want to take a look, it's balenaOS 2.80.3+rev1 and has the bme680 modules loaded!

I see that kernel module now, the manifest explorer dropdown must have been out of sync.


Comparison with EDVR. REDS dataset.

Hello,
In your paper you compare your model with EDVR.
Did you train EDVR on your dataset or only tested the pretrained model? In EDVR paper they use REDS dataset. For the sake of fare comparison could you please train your model with REDS dataset? It would be really good to see the CDVD-TSP and EDVR trained on same data.
I think everyone would highly appreciate it.

The EDVR's quantitative results are achieved by using its provided pre-trained model.
Our results on REDS dataset did not exceed EDVR, but we noticed that EDVR is overfitted on REDS dataset and it does not perform well on real datasets.

Thank you very much for fast reply! It is now much more clear to me. But I still have few questions:

So you mean that your model is not overfitted to your "real datasets"? And why REDS is not real one? It is created by the same person (Seungjun Nah) who created GoPro dataset.
Did you train your model on REDS or just tested the pretrained on another dataset one?
Could you please share the PSNR and SSIM which you got on REDS?
You compared your results to SRN-Deblur in your paper. The PSNR value given for SRN is almost the same as the original authors provide (30.26 vs 30.29). However, SSIM values were clearly different (0.9342 vs 0.9014). Does it happen because of different SSIM estimation (because Pytorch doesn't provide official SSIM function)? Did you train SRN model, or just tested the pretrained model?

The "real dataset" mentioned above is the dataset from "Sunghyun Cho, Jue Wang, and Seungyong Lee. Video deblurring for hand-held cameras using patch-based synthesis. ACM TOG, 31(4):64:1–64:9, 2012."
We noticed that the EDVR's deblurring results on the "real dataset" generated by using its provided pre-trained model are not well, that it seems to be overfitting on the REDS dataset.
We have not trained our model to convergence on REDS, which trained about 100epoch. The PSNR/SSIM we got is 32.080/0.9205. You also can train our model on the REDS dataset using the provided codes.
The SRN's quantitative results are achieved by using its provided pre-trained model. We use the PSNR/SSIM calculating function from EDVR's code.


Release 2.1.76

On the docket:

[ ] ...

Maybe:

[ ] Decouple PEX runtime interpreter selection from buildtime interpreter selection. #1020

Unlikely / backlog:

[ ] PEX_INHERIT_PATH+default shebangs don't interact well with virtualenvs #1344
[ ] Pex interpreter cache is not upgrade / downgrade safe. #1176
[ ] Provide a better resolve error message for a Platform DistributionTarget when --resolve-local-platforms. #1070
[ ] Remove PEX_PYTHON_PATH from build time interpreter selection. #1075

Split off from "Release 2.1.75 #1691" to make way for a fix for deadlock errors (#1694).


AST: Print opaque result types correctly in swiftinterfaces

Fixes a bug where references to the opaque types of other decls were printed with the some Type syntax, breaking the interface. Only the opaque type introduced by a declaration should be printed with the some Type syntax; other opaque types must always be printed as a stable addresses instead.
Resolves rdar://134582913.

@swift-ci please smoke test


debt - allow extHostSearch in common

@roblourens this can be verfied with "yarn web" (see extension.ts).
I noticed https://github.com/microsoft/vscode/issues/84110 as follow up

@roblourens ping

Thanks


Vulnerability: CVE-2022-42003 & CVE-2022-42889 in opensearchproject/opensearch:1.3.6

Describe the bug
Hi team,
In our trivy scan report there are 1 HIGH and 1 CRITICAL vulnerabilities in opensearchproject/opensearch:1.3.6.
Is there any plan to upgrade the version of libs to fix them? Thanks.

Library
Vulnerability
Severity
Installed Version
Fixed Version
Title

com.fasterxml.jackson.core:jackson-databind(jackson-databind-<IP_ADDRESS>.jar)
CVE-2022-42003
HIGH
<IP_ADDRESS>
<IP_ADDRESS>, <IP_ADDRESS>
jackson-databind:   deep wrapper array nesting wrt(jackson-databind-<IP_ADDRESS>.jar)   UNWRAP_SINGLE_VALUE_ARRAYS https://avd.aquasec.com/nvd/cve-2022-42003

org.apache.commons:commons-text   (commons-text-1.9.jar)
CVE-2022-42889
CRITICAL
1.9
1.10.0
apache-commons-text:   variable interpolation RCE https://avd.aquasec.com/nvd/cve-2022-42889

@jefftyn the CVEs have been addressed already:

https://github.com/opensearch-project/OpenSearch/pull/4785
https://github.com/opensearch-project/ml-commons/pull/520
https://github.com/opensearch-project/security/pull/2186

@jefftyn The next 1.3 release is scheduled for December 8 and will include these fixes

Closing this, since they are already fixed and will be pushed on next release.

When is the next release due? Our security folks are starting to complain.


[RFC] Moving from always trailing commas to always-multiline

RFC
It seems like I went a little too fast on the previous change (confused by prettier config "all").
We wanted to include end-of-line trailing commas at the end of multi-line statements (cf the compass update here).
But we did not mention much about the single-line statements.
So let's now vote for the single line statements :)

Keep always

export const supportedLanguages = ['en', 'fr', ]; // Enforced trailing commas here
export const supportedLanguages = [
  'en',
  'fr', // Enforced
];

Change to only-multiline

export const supportedLanguages = ['en', 'fr', ];
export const supportedLanguages = ['en', 'fr']; // Both format allowed for single-line statement
export const supportedLanguages = [
  'en',
  'fr', // Enforced trailing commas here
];

Change to always-multiline

export const supportedLanguages = ['en', 'fr']; // Enforced no trailing commas here
export const supportedLanguages = [
  'en',
  'fr', // Enforced trailing commas here
];

More info here.

ℹ Option 1 and 2 exposed to a conflict with another rule , ] (no space allowed before ]. If such an option is voted, we will need to find a solution to solve that.

Poll
Pool deadline: 4 november at 2pm (Thai/Vietnam).

It was resolved in https://github.com/nimblehq/eslint-config-nimble/pull/14


Image height of over 80,000 px on search results of "search"

Describe the bug
After a couple of scrolls, the search result of query search gives an image which is over 80,000px in height. See the video link below
To Reproduce
Steps to reproduce the behavior:

Go to https://search.creativecommons.org/search?q=search&provider&li&lt&searchBy
Scroll down a bit.

Screenshots

Desktop (please complete the following information):

OS: Linux mint
Browser: chrome

Hi guys, I'll like to work on this issue.

@Atanda1 I'm not sure if this issue is still relevant given that the design has changed, but if you can reproduce it, feel free to work on it.

Please post here if you are working on it.

Okay, but would it be merged officially, because it'll help with my GSoC
application.
On Feb 21, 2020 21:30, "Kriti Godey"<EMAIL_ADDRESS>wrote:

@Atanda1 https://github.com/Atanda1 I'm not sure if this issue is still
relevant given that the design has changed, but if you can reproduce it,
feel free to work on it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KOR5MO36WIGYSA57T3REA2VDA5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMT7XLA#issuecomment-589822892,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KIXJDOQCUAQ7KDFMHDREA2VDANCNFSM4H46C7DQ
.

@Atanda1 we'll evaluate any code you work on, whether it's merged or not. I recommend first checking if you can reproduce this issue and posting here if you're able to reproduce it.

Okay, I'll start working on it.

On my windows machine, there's a ton of eslint errors when I run "npm run dev" about expecting line breaks LF instead of CRLF. I've added *.js text eol=lf to my gitattributes file, according to eslint but its not still going through

http://eslint.org/docs/rules/linebreak-style
Actually linebreak errors

@akmadian @brenoferreira

I'm yet to get a response

Maybe we could just explicitly disable the rule in eslintrc?

Please do.  I'll really appreciate.
On Feb 26, 2020 03:24, "Ari Madian"<EMAIL_ADDRESS>wrote:

Maybe we could just explicitly disable the rule in eslintrc?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KJ6M6V537BFXDNKUT3REXHFDA5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6Q7UI#issuecomment-591204305,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KL5KWXZA3OAGPR6NFLREXHFDANCNFSM4H46C7DQ
.

@Atanda1 If you would like to create an issue and PR for this, I would be happy to assign you and approve the PR.

So I should create a PR and issue for removing eslint from the project now.
On Feb 26, 2020 04:07, "Ari Madian"<EMAIL_ADDRESS>wrote:

@Atanda1 https://github.com/Atanda1 If you would like to create an
issue and PR for this, I would be happy to assign you and approve the PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KKMW34GSGND5NUOAOTREXMGVA5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6TRXA#issuecomment-591214812,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KIF6KIJPFAXNFUYUQ3REXMGVANCNFSM4H46C7DQ
.

I'll be happy to do that.
On Feb 26, 2020 05:39, "Atanda David"<EMAIL_ADDRESS>wrote:

So I should create a PR and issue for removing eslint from the project now.
On Feb 26, 2020 04:07, "Ari Madian"<EMAIL_ADDRESS>wrote:

@Atanda1 https://github.com/Atanda1 If you would like to create an
issue and PR for this, I would be happy to assign you and approve the PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KKMW34GSGND5NUOAOTREXMGVA5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6TRXA#issuecomment-591214812,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KIF6KIJPFAXNFUYUQ3REXMGVANCNFSM4H46C7DQ
.

@Atanda1 No, we are definitely not removing eslint from the project entirely. I just meant adding a line to disable the linebreak-style rule in .eslintrc.js

Okay, I get you now. I'll do that.
On Feb 26, 2020 05:47, "Ari Madian"<EMAIL_ADDRESS>wrote:

@Atanda1 https://github.com/Atanda1 No, we are definitely not removing
eslint from the project entirely. I just meant adding a line to disable the
linebreak-style rule in .eslintrc.js
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KJNTQEC7SXHFGN7223REXX53A5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6Y3TA#issuecomment-591236556,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KN4JKNIYLFLWBNUX5LREXX53ANCNFSM4H46C7DQ
.

@akmadian so I should put this "image height" issue on hold for now.

@akmadian here is the issue https://github.com/creativecommons/cccatalog-frontend/issues/649
I've  requested a pull request already.

But on running npm run dev it compiles

And on the browser I get this

Go to http://localhost:8443/

Thanks. It worked!!
On Feb 26, 2020 1:09 PM, "hemanth kumar"<EMAIL_ADDRESS>wrote:

Go to http://localhost:8443/
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KNV4FKLLIDQQDOE5OTREZLYZA5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM77LNQ#issuecomment-591394230,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KIU4PFEZWEYAJQNWMDREZLYZANCNFSM4H46C7DQ
.

@akmadian heres the pull request https://github.com/creativecommons/cccatalog-frontend/pull/654

My first pull request just got merged,  would be done with the image height
in a bit.
On Feb 26, 2020 13:48, "Atanda David"<EMAIL_ADDRESS>wrote:

Thanks. It worked!!
On Feb 26, 2020 1:09 PM, "hemanth kumar"<EMAIL_ADDRESS>wrote:

Go to http://localhost:8443/
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423?email_source=notifications&email_token=AKZI4KNV4FKLLIDQQDOE5OTREZLYZA5CNFSM4H46C7D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM77LNQ#issuecomment-591394230,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KIU4PFEZWEYAJQNWMDREZLYZANCNFSM4H46C7DQ
.

@kgodey @akmadian After the new Design got merged, I don't think its an issue now, So this can be closed now!

Yes, it should.
On Apr 7, 2020 12:32 PM, "Abhishek Naidu B"<EMAIL_ADDRESS>wrote:

@kgodey https://github.com/kgodey @akmadian
https://github.com/akmadian After the new Design got merged, I don't
think its an issue now, So this can be closed now!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/creativecommons/cccatalog-frontend/issues/423#issuecomment-610334414,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZI4KO6TPYRCRNBQ5UJNELRLMFOTANCNFSM4H46C7DQ
.


Promises, fibers and a reorg

Good news: we have a lot more features.
Bad news: I had to reorganise things quite a bit to accommodate everything
Thoughts?

In related news, I'm thinking it would make sense to use synchronize.js lib built on top of node-fibers as it takes care of some of the boilerplate. I was experimenting with it here, and it seems to be more predictable for wrapping callbacks.

There are a few libraries in that domain. I think we may want to experiment with them, but what's here is useful for building primitives on.

Definitely, ideally I'd like to provide an intuitive API for the end users that makes it easy to wrap callback based code.