Create active_client_accounts

Related to https://github.com/bcgov/range-web/issues/418
Create user_id, client_id, type, where type is 'owner' or 'agent'

add linked by user as field
add changed date as field
Do we want history?

@micheal-w-wells Instead of history, we could possible just add an active column. Then instead of deleting links we don't want anymore, we just set active to false.

I'll say yes to active/inactive, and a changed date column.  I suppose 'inked by' is only going to say 'myrangebc' at this point anyway.

@micheal-w-wells I've found a potential roadblock with this solution. If a user is assigned to multiple client IDs, (for example, the user Leslie Knope is linked to the clients Leslie Knope and April Ludgate), and they go to sign a submission, how do we decide which client to attach the signature to? This would be especially tricky if both Leslie Knope and April Ludgate are agreement holders on the same RUP.

One possible solution I've though of is to display a dropdown in the navbar that would let the user select which client they want to act as. This could potentially come in handy for agency agreements as well.

I see what you are saying, but one user should only have to worry about one signature on each RUP.  I think if someone has two active client + loc codes in FTA for one RAN something is set up wrong on that end.  It would mean we are sending them two bills or they are a partner with themselves for the same agreement, or two AH are sharing a BCEID which isn't a thing.  The scenario you have given is test only as far as I understand.

@LisaMoore1 am I out to lunch, can you think of a time where one BCEID should be attached to multiple clients on one RUP other than testing?

@micheal-w-wells Would this be an example?

Joan and Frank are AHs — their grandson Marco is the agent for both of them
I’m not sure because I’m not sure what you’re thinking for differentiating AHs from agents. I can’t think of a straight up AH situation where the same BCeID would be used to identify to separate people.
wondering about a business but I don’t think that would apply because they just need one of their signing authorities to use the single business BCeID to represent the company

That helps @LisaMoore1, I just wanted to clear up @calebissharp 's concern here about Leslie and April both being clients for Leslie and being on the same RUP:

@micheal-w-wells I've found a potential roadblock with this solution. If a user is assigned to multiple client IDs, (for example, the user Leslie Knope is linked to the clients Leslie Knope and April Ludgate), and they go to sign a submission, how do we decide which client to attach the signature to? This would be especially tricky if both Leslie Knope and April Ludgate are agreement holders on the same RUP.

@micheal-w-wells and @calebissharp I’ve been taking the view that BCeID login/confirmation = a signature. Only one person can sign “their” signature therefore one BCeID = one client. In the case of a business they may have multiple signing authorities associated with that client. In range we aim to have a single person as our point of contact (the primary agreement holder where there are multiple AHs — meaning a primary AH has no other rights/obligations that the other AHs other than getting the physical mail). Where there is a ranch manager they act as agent for the company and they are our primary contact. This is why I keep going back to agency agreements being the trickiest aspect of linking accounts. If we solve that I believe we by default solve all other significant issues with account linking, client #s and locations.

Perhaps we should then move plan_confirmation to use the user id, rather than the client id. As well, maybe client_agreement should just be used to determine if a user has access to a given RAN based on the entries in active_client_account.

The gotcha scenario I have seen with that one @calebissharp is when an AH goes from a primary to secondary on a RUP (or otherwise use another client_id associated to that BCEID),  what do you do with their signature?  What we have now (with the last fix anyway) is that they would now have to sign again because of the new linkage (previously signatures were just broken in this scenario).  I'm not sure what the ideal is, and if its ok to carry signatures forward when its a new client_id but same bceid.

Moving from primary to secondary doesn't actual change the client ID though, does it? And plan confirmations reference ref_client.id.

@micheal-w-wells and @calebissharp I’ve been taking the view that BCeID login/confirmation = a signature. Only one person can sign “their” signature therefore one BCeID = one client. In the case of a business they may have multiple signing authorities associated with that client. In range we aim to have a single person as our point of contact (the primary agreement holder where there are multiple AHs — meaning a primary AH has no other rights/obligations than the other AHs other than getting the physical mail). Where there is a ranch manager they act as agent for the company and they are our primary contact. This is why I keep going back to agency agreements being the trickiest aspect of linking accounts. If we solve that I believe we by default solve all other significant issues with account linking, client #s and locations. I think the April and Leslie scenario is a testing anomalies.
Related...again from an agent perspective — how would we handle a situation where an agent is also an agreement holder? Their BCeID would be linked to their CLIENT# for sure...but that BCeID would also need to be identified as an agent-BCeID for a second (or perhaps even 3rd) client.

To clear up any potential confusion here:

1 BCEID = 1 Client, where Client (with a big 'c') here we mean the literal person.
1 Client/person, can have multiple client (little 'c') records in FTA.  There are plenty of examples.  I'm operating on the assumption we don't want to make these people make multiple BCEIDs.
1 Client/person, can have multiple signing authorities for themselves
1 Client/person, can be the signing authority for multiple other Clients.

Moving from primary to secondary doesn't actual change the client ID though, does it? And plan confirmations reference ref_client.id.

It does, because it's a new location code, so new loc + client number = new client_id as it is currently implemented.

So, if big 'c' Client represents a physical person/BCeID, than I think at least plan_confirmations should use the user id. (That's how it works in real life anyway. I assume you don't forge the signature of the person you're signing for as an agent). Then we just use little 'c' client records as a way of gating access to RANs for Client/person/BCeIDs (we could probably disambiguate things by adjusting naming). The trickiest thing I see with agency agreements so far is you still probably want a way to identify which Client/person the agent Client/person is signing for. Maybe we take the name from the client/FTA record and use that?

That works we just have to make sure not to carry a signature forward when it shouldn't.  I'd say put signee as one field and signed for as another user_id field.
I don't think we want to rely on name.  If I am an agent, the RUPS i can view are dictated by client + loc records assigned to me, so user_id : client_id, wherever we want to put that (plus a page/interface to do so).  I should not be signing for more than one party on a given RUP.  It should be straight forward from there to know based on the RUP I am viewing, and the agent user_id <> client_id link which record gets signed.

table: agent_client_link
agent user_id, client_id

Just to clarify @LisaMoore1, in the context of agency agreements, it is not possible to be the agent for multiple AHs on the same RAN?
@micheal-w-wells I think we have the user_id:client_id portion covered by active_client_account. I like the idea of signee and signed for on plan_confirmation. I assume those would be the same user.ids if someone signs for themselves. While we're on the topic, do you know of any other tables like plan_confirmation that are using ref_client.id right now but should be using user.id instead?

Also, after working with active_client_account a bit, I'd like to change the name. user_client_link seems more semantically useful.

user_client_links it is.  Yes it would be themselves in case of them signing it, you got it.  Can't think of anything else using client_id that's at issue right now, but will keep eyes peeled.
Above it looks like @LisaMoore1 had this example I missed Joan and Frank are AHs — their grandson Marco is the agent for both of them .  If we can legally have one agent sign for both parties in this case, they'd likely have a RUP together.  I think a low effort implementation that could work for now is sign it for all relevant parties if there is more than one, and if that doesn't work expand to a selection component.

@calebissharp

a non-AH child of the AHs (a mom and dad venture) could be the agent for either OR both parents
an AH child of the other AHs (family venture) could be the agent for either or both parents
a non- family member could take the same roles as “child” above
a business may have multiple signing authorities AND a ranch manager any of which could sign

Principles

we need to know if we have an agent versus AH signature
staff need to easily see (a table maybe?) for each RAN/RUP who has an agent and who it is
an agent may stop being an agent but if we look at the history we should be able to see that an agent (including their name) signed at a specific date on a specific version — the version remains legitimate if the agency agreement dissolves mid-way through a RUP term

@micheal-w-wells
Re. Agreement Holders

for the purposes of collections and agreement responsibilities there is a SINGLE agreement holder made up of all of the partners (whether Joint venture (JV), Ltd. Company or whatever)
the entity we call agreement holder (made up of all its people) is, as a single entity responsible for all obligations and share the rights equally
the individual partners that make up that entity do not have separate rights/responsibilities excepting for the one identified as primary getting the snail mail/phone calls
we do not consider the entity to have signed until all the partners that make up that entity have signed
if the entity gets shuffled around to have someone else get the mail/phone calls we don’t care (about collecting new signatures) so long as the ENTITY consists of the same collective of individuals
if a person is added to the partnership we consider that to be a new entity and we require a disposition form (as though they were transferring it to a completely different person) and a new plan/signatures is required — this disposition would result in a shift in FTA as the primary (if they stay the same) would get a new location code that includes all of the old JV partners + the new one (usually all the JV partners but the primary are using their -00 code)

@micheal-w-wells What do you think the best way to look up which users to sign for based on the client is? Right now we have the type column which can be owner or agent. I'm thinking we add a constraint to only allow 1 owner record per unique client_id, to allow us to do a reverse lookup on client_id to get the associated user account (which we'd use for the signed for column in plan_confirmation)

@calebissharp I think we would look it up the same we we looked up what RUP they can see.  This is my understanding anyway:
User BEN in user_client_links would be an agent for client_id 123.  We get 123's user_id from the same table.   I think your idea for a constraint works there.
We get then get RUPs he can see the same way we would get RUPs. for client 123.  We look in client_agreement and grab the records with matching client_id, and from there join on plan with agreement_id and we have the plan_id to go to plan_confirmation.
Again in the rare case you get more than one result (grandson signing for multiple) i'd say v1 just sign for both v2 make a checklist where you explicitly choose people to sign for.

@calebissharp @micheal-w-wells
Not sure if this adds to the understanding but you could also have a situation something like this:

agent 1 acts as agent for two of the three individuals on RAN071234 and one of the individuals on RAN075678
agent 2 acts as agent for the the remaining individual on RAN071234
agent 2 also has their own RAN077890 that they hold on their own

@micheal-w-wells We should be able to do sign for both. We may have to augment the data returned from /user/me to also include the relevant represented user accounts.

@calebissharp this one is done isn't it? or does the name of the issue not really match the work anymore?

Resolves by #162


[BUG] Replace Super-Linter with many different (used) linters

Describe the bug
Currently Super Linter fails with gitleaks, a linter written in go that checks for keys. Also it needs approval of the mantainer of the project every time a new contribuitor submmits a PR (which is uncomfortable).
To Reproduce
Steps to reproduce the behavior: just go to the current PRs, like #169  or #178

https://github.com/Rayji96/foundry-V5/pull/182 as a stop-gap currently

I did some tests, but I haven't had enough time to make it work :(

Hey, I kept going but I still can't pass the wall of the node-gyp error when installing the npm dependencies.
In this branch eslint-workflow I added a workflow to run only eslint, (I tried twice 🤣) but in both cases I found this error when running:
expand to see the error

Run npm install<EMAIL_ADDRESS>npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm WARN deprecated<EMAIL_ADDRESS>Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated<EMAIL_ADDRESS>See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated<EMAIL_ADDRESS>See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated<EMAIL_ADDRESS>https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated<EMAIL_ADDRESS>this library is no longer supported
npm WARN deprecated<EMAIL_ADDRESS>Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
npm WARN deprecated<EMAIL_ADDRESS>Use the original unforked package instead: postcss-markdown
npm WARN deprecated<EMAIL_ADDRESS>babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
npm WARN deprecated<EMAIL_ADDRESS>Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated<EMAIL_ADDRESS>This version of tar is no longer supported, and will not receive security updates. Please upgrade asap.
npm ERR! code 1
npm ERR! path /home/runner/work/foundry-V5/foundry-V5/node_modules/node-sass
npm ERR! command failed
npm ERR! command sh -c -- node scripts/build.js
npm ERR! Building: /usr/local/bin/node /home/runner/work/foundry-V5/foundry-V5/node_modules/node-gyp/bin/node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libsass_library=
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp verb cli [
npm ERR! gyp verb cli   '/usr/local/bin/node',
npm ERR! gyp verb cli   '/home/runner/work/foundry-V5/foundry-V5/node_modules/node-gyp/bin/node-gyp.js',
npm ERR! gyp verb cli   'rebuild',
npm ERR! gyp verb cli   '--verbose',
npm ERR! gyp verb cli   '--libsass_ext=',
npm ERR! gyp verb cli   '--libsass_cflags=',
npm ERR! gyp verb cli   '--libsass_ldflags=',
npm ERR! gyp verb cli   '--libsass_library='
npm ERR! gyp verb cli ]
npm ERR! gyp info using<EMAIL_ADDRESS>npm ERR! gyp info using<EMAIL_ADDRESS>| linux | x64
npm ERR! gyp verb command rebuild []
npm ERR! gyp verb command clean []
npm ERR! gyp verb clean removing "build" directory
npm ERR! gyp verb command configure []
npm ERR! gyp verb check python checking for Python executable "python2" in the PATH
npm ERR! gyp verb which failed Error: not found: python2
npm ERR! gyp verb which failed     at getNotFoundError (/home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:13:12)
npm ERR! gyp verb which failed     at F (/home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:68:19)
npm ERR! gyp verb which failed     at E (/home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:80:29)
npm ERR! gyp verb which failed     at /home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:89:16
npm ERR! gyp verb which failed     at /home/runner/work/foundry-V5/foundry-V5/node_modules/isexe/index.js:42:5
npm ERR! gyp verb which failed     at /home/runner/work/foundry-V5/foundry-V5/node_modules/isexe/mode.js:8:5
npm ERR! gyp verb which failed     at FSReqCallback.oncomplete (node:fs:202:21)
npm ERR! gyp verb which failed  python2 Error: not found: python2
npm ERR! gyp verb which failed     at getNotFoundError (/home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:13:12)
npm ERR! gyp verb which failed     at F (/home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:68:19)
npm ERR! gyp verb which failed     at E (/home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:80:29)
npm ERR! gyp verb which failed     at /home/runner/work/foundry-V5/foundry-V5/node_modules/which/which.js:89:16
npm ERR! gyp verb which failed     at /home/runner/work/foundry-V5/foundry-V5/node_modules/isexe/index.js:42:5
npm ERR! gyp verb which failed     at /home/runner/work/foundry-V5/foundry-V5/node_modules/isexe/mode.js:8:5
npm ERR! gyp verb which failed     at FSReqCallback.oncomplete (node:fs:202:21) {
npm ERR! gyp verb which failed   code: 'ENOENT'
npm ERR! gyp verb which failed }
npm ERR! gyp verb check python checking for Python executable "python" in the PATH
npm ERR! gyp verb which succeeded python /opt/hostedtoolcache/Python/3.11.1/x64/bin/python
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Command failed: /opt/hostedtoolcache/Python/3.11.1/x64/bin/python -c import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack   File "", line 1
npm ERR! gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
npm ERR! gyp ERR! stack SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?
npm ERR! gyp ERR! stack
npm ERR! gyp ERR! stack     at ChildProcess.exithandler (node:child_process:402:12)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at maybeClose (node:internal/child_process:1100:16)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
npm ERR! gyp ERR! System Linux 5.15.0-1030-azure
npm ERR! gyp ERR! command "/usr/local/bin/node" "/home/runner/work/foundry-V5/foundry-V5/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
npm ERR! gyp ERR! cwd /home/runner/work/foundry-V5/foundry-V5/node_modules/node-sass
npm ERR! gyp ERR! node -v v16.19.0
npm ERR! gyp ERR! node-gyp -v v3.8.0
npm ERR! gyp ERR! not ok
npm ERR! Build failed with error code: 1
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2023-01-11T11_12_05_050Z-debug-0.log
Error: Process completed with exit code 1.

Apparently is because it cannot find python in the virtual machine where it installs gyp, but I installed python in a previous step successfully and nothing changed 🤔 .
I will keep doing some other tests for this issue, but this particular task turn out a lot harder than I expected


help() in Code Interpreter throws an exception

Code Interpreter> help()
Traceback (most recent call last):
  File "/lib/python311.zip/_pyodide/_base.py", line 573, in eval_code_async
    await CodeRunner(
  File "/lib/python311.zip/_pyodide/_base.py", line 395, in run_async
    await coroutine
  File "<exec>", line 1, in <module>
  File "<exec>", line 183, in run
  File "<string>", line 1, in <module>
  File "<frozen _sitebuiltins>", line 103, in __call__
  File "/lib/python311.zip/pydoc.py", line 2007, in __call__
    self.interact()
  File "/lib/python311.zip/pydoc.py", line 2019, in interact
    request = self.getline('help> ')
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python311.zip/pydoc.py", line 2039, in getline
    return input(prompt)
           ^^^^^^^^^^^^^
OSError: [Errno 29 I/O error
]

Calling the help function with arguments seems to work fine. It's just the no-args invocation above that is an issue.

I just inspected the error, it's because help try to call input() and that is disabled in web-worker (a thread in the browser which does not have access to UI).
For now, I will do a patch for this function.


How To Add Buttons During Request Processing

const someFunction = (ctx) => ctx.someArray.map((el) => el);   someMenu.select('actionName', someFunction, actions);
It works. But I am not happy with the 'select'
How can I make a loop using ctx.someArray and add buttons?

You can implement 'someFunction' the way you want to.
function someFunction(ctx) {
  for (const el of ctx.someArray) {
    // do stuff
  }

  return ['1', 'tree', 'whatever']
  // or
  return {tree: 'Glorious Tree', whatever: 'Whatever you like' }
}

But I cant do like this
someMenu.button(someFunction, ...

The idea of this library is to know the general structure of the menu on startup.
You have some buttons that are always the same (menu.button) and some selections which might change. You know these are always there, you just dont know which exactly. Then you have some function returning these to menu.select.

Ok I understand it
How can I get buttons through the context to add another one?

You can base the return value of your someFunction on the ctx object. Base on that the buttons are created.
function someFunction(ctx) {
  if (ctx.from.id === 42) {
    return [ctx.from.first_name]
  }

  return ['foo', 'bar']
}


exe not creating

when i click compile this happens:

I've got the same issue, could you please explain how you fixed it?

im pretty sure i just changed from python 3.12 to 3.11.8

Turn on ALL of your bot intents, invite your bot to the server with ADMINISTRATOR perms (every permission)

Hey, just use the ModdedPysilon version, it covers every bug and it make sure your discord account never get banned.
Leave me a star if you like my project ;)


Add enabling auto merge in PRs

https://github.com/marketplace/actions/enable-github-automerge

Not sure about this one, it did help me not once to take a final glance on the PR before merging it. Anyway, currently in our settings only one review is required, while we typically wait for two reviews, I think this should be adjusted for auto-merges.


nxp: fix reset sequence

This fixes a race condition between the debugger and the MCU during reset.
Prior to this change, flashing would often fail.
After this change, I can reliably flash my Teensy 4.1 (which uses an NXP i.MX RT1062) using JTAG.
For reference, my original observations were captured here: https://github.com/probe-rs/probe-rs/pull/1174#issuecomment-1275568493
/cc @mciantyre

@Yatekii Yes -- thanks for reminding! Will have that in shortly.

@Yatekii Just added a changelog entry -- let me know if you'd like to tweak it in any way.

Thanks again for working through this. Just noting that I tested this commit the same way I tested #1174. LGTM.


Update admin-configforms.png

Changed options in Configure Form Processing page

@JohanFreelancer9 Please copy review

@JohanFreelancer9 Please copy review

Issue#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/3932

Issue#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/3932

@msbemba

@msbemba

@MikePlumleyMSFT Please merge this PR. Thanks.

@MikePlumleyMSFT Please merge this PR. Thanks.


Generate Zoom links automatically

Use this https://marketplace.zoom.us/docs/api-reference/zoom-api/meetings/meetingcreate
Will need to get access to BU's api (or get our own)

Replacing this with a Toggle Opportunity for Zoom or No Zoom https://github.com/BUUPE/Inquisitor/issues/149 (Closing this)


Feature/rate limiter docs

Documentation for the rate limiter feature at echo@1724

@lammel notice this line

If your app receives above 100 parallel requests at once then this might not be the best option

Is it safe to say that this is false considering your breakthrough with the middleware handling about 30,000 requests per sec conveniently?

@lammel notice this line

If your app receives above 100 parallel requests at once then this might not be the best option

Is it safe to say that this is false considering your breakthrough with the middleware handling about 30,000 requests per sec conveniently?

@lammel notice this line

If your app receives above 100 parallel requests at once then this might not be the best option

Is it safe to say that this is false considering your breakthrough with the middleware handling about 30,000 requests per sec conveniently?

Not necessarily. Handling 30000 req/s sequentially is different compared to handling 300req/s in parallel from 100 sources (which my test uses). This is where lock contention comes in.
The number 100 is of course just a very vague estimate and shall only act as an indicator, that for high load traffic the builtin RateLimiterMemoryStore and ratelimiter might negatively influence performance of the app.

@lammel notice this line

If your app receives above 100 parallel requests at once then this might not be the best option

Is it safe to say that this is false considering your breakthrough with the middleware handling about 30,000 requests per sec conveniently?

Not necessarily. Handling 30000 req/s sequentially is different compared to handling 300req/s in parallel from 100 sources (which my test uses). This is where lock contention comes in.
The number 100 is of course just a very vague estimate and shall only act as an indicator, that for high load traffic the builtin RateLimiterMemoryStore and ratelimiter might negatively influence performance of the app.

Thanks @iambenkay , adjusted some wordings and styling, think that's fine for now.

Thanks @iambenkay , adjusted some wordings and styling, think that's fine for now.


TypeHandler Scanning is not working in an Eclipse/OSGI environment

We are migrating our Eclipse RCP application from IBATIS 2.5 to the latest version of MyBatis. We have hit a snag with the TypeHandler scanning. I believe this is because the RCP application is running in an OSGI container and the way in which the default VFS is reading the list of resources for a given package is running into a security policy.
We also run the same exact code in a web application and that is working as expected.
This may be a similar issue to what is being encountered in JBOSS (which I believe is also running an OSGI based framework)
Any pointers on how to fix this would be greatly appreciated.
Here is the stack trace we are getting:
2014-02-12 15:36:01,917 ERROR - Could not read package: com/landacorp/lib/persistence
java.io.FileNotFoundException: D:\devel_V3.0.X\java\Framework\classes\com\landacorp\lib\persistence (Access is denied)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:120)
at org.eclipse.osgi.framework.util.SecureAction.getFileInputStream(SecureAction.java:123)
at org.eclipse.osgi.baseadaptor.bundlefile.FileBundleEntry.getInputStream(FileBundleEntry.java:56)
at org.eclipse.osgi.framework.internal.core.BundleURLConnection.connect(BundleURLConnection.java:53)
at org.eclipse.osgi.framework.internal.core.BundleURLConnection.getInputStream(BundleURLConnection.java:99)
at java.net.URL.openStream(URL.java:1010)
at org.apache.ibatis.io.DefaultVFS.list(DefaultVFS.java:90)
at org.apache.ibatis.io.VFS.list(VFS.java:193)
at org.apache.ibatis.io.ResolverUtil.find(ResolverUtil.java:216)
at org.apache.ibatis.type.TypeHandlerRegistry.register(TypeHandlerRegistry.java:339)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.typeHandlerElement(XMLConfigBuilder.java:283)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parseConfiguration(XMLConfigBuilder.java:103)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parse(XMLConfigBuilder.java:89)
at org.mybatis.spring.SqlSessionFactoryBean.buildSqlSessionFactory(SqlSessionFactoryBean.java:428)
at org.mybatis.spring.SqlSessionFactoryBean.afterPropertiesSet(SqlSessionFactoryBean.java:340)

Type alias scanning for entities does not seem to work in Websphere 8.5.5. Are there any temporary fixes available?

Hi Copperis, It's been a while since I had my head into this...but we experienced a similar problem reading type handlers from a spring boot application. The solution was to use a custom virtual file system that is aware of any classloading issues. In the case of websphere, you may need to have a VFS that knows how to traverse things in an OSGI based environment.
This is an example that was written to work for spring boot applications:
https://github.com/kazuki43zoo/spring-boot-mybatis-sample/blob/master/mybatis-typehandler/src/main/java/sample/infra/mybatis/vfs/SpringBootExecutableJarVFS.java
Obviously, this is not a direct fix, but should hopefully point you in the right direction.

Custom VFS can be specified since MyBatis 3.3.1 using configuration file via https://github.com/mybatis/mybatis-3/issues/325.
Note:
Custom VFS for spring boot application will provide by the mybatis-spring-boot 1.0.1 via https://github.com/mybatis/mybatis-spring-boot/issues/21.
However 1.0.1 has not been released yet. I hope early release mybatis-spring-boot 1.0.1 :smile:

The application is using a separate project as a maven dependency for entities and mapper xml's. It is included as a jar in the final WAR file under WEB-INF/lib. I don't think OSGI is related here.
Thank you both, I now understand how I can solve this.

Mapper resolution does not work with WebSphere 8.5.5.x if package name is provided - it seems the jar scanning works different in WAS. I've tried it with 3.4.0 and the ResolverUtil.find returns empty list. Is there any work on WAS VFS or some other solution to the problem? Please note, that the resolution works in a WAS TE within Eclipse with "Publish with resources from workspace" turned on. But in this case the file is parsed directly from the filesystem.

ok, after several tries the following VFS implementation works for me - may be the DefaultVFS implementation could be extended with this on a fall-down matter, because I think it is safe:
public List list(String path) throws IOException {
path = path + "/";
URL url = getClass().getResource(path);
list.addAll(list(url, path));
}


add benchmark, headless mode, and shutdown context to tui

This PR adds a benchmark to measure the performance of small updates to elements in the tui render. To allow the benchmark to run in CI, It also adds a headless mode and a context that allows the user program to shut down. It also adds the builder pattern and non-exhaustive to config, so that in the future adding more fields to config will not be a braking change.

Looks good - thank you :)


emphasis HTML time element over abbr element

README update
fixes #255

Thanks for trying, but this was just a gsub of <abbr> for <time>. It doesn't take into consideration the surrounding words that have meaning to the underlying discussion. Nor does it take into consideration the fact that <time> elements require a different attribute (datetime) as compared to the microformat-compliant <abbr> elements' title attribute.
Also, it completely ignores the website that drives most of the documentation (governed by index.html).

Hopefully fixed now

Close enough. I'll just fix the missed-comms from here.


Add missing dependency on cuda-cupti for CUDA 12 jaxlib builds

Fix for:

https://github.com/conda-forge/cuda-cupti-feedstock/issues/14
https://github.com/conda-forge/cuda-cupti-feedstock/issues/9

Checklist

[x] Used a static YAML file for the patch if possible (instructions).
[x] Only wrote code directly into generate_patch_json.py if absolutely necessary.
[x] Ran pre-commit run -a and ensured all files pass the linting checks.
[x] Ran python show_diff.py and posted the output as part of the PR.
[x] Modifications won't affect packages built in the future.

python show_diff.py (not sure why it contains clang-related changes):
(repodata) traversaro@IITBMP014LW012:~/conda-forge-repodata-patches-feedstock/recipe$ python3 show_diff.py
================================================================================
================================================================================
linux-armv7l
================================================================================
================================================================================
win-32
================================================================================
================================================================================
osx-arm64
osx-arm64::clang-18.1.5-default_h095aff0_0.conda
+  "constrains": [
+    "clang-tools 18.1.5.*",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
+  ],
osx-arm64::clang-tools-18.1.5-default_h7c89ad4_0.conda
-    "clangdev 18.1.5"
+    "clang 18.1.5.*",
+    "clangdev 18.1.5",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
================================================================================
================================================================================
linux-ppc64le
linux-ppc64le::clang-18.1.5-default_h60175c1_0.conda
+  "constrains": [
+    "clang-tools 18.1.5.*",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
+  ],
linux-ppc64le::clang-tools-18.1.5-default_hf6035ce_0.conda
-    "clangdev 18.1.5"
+    "clang 18.1.5.*",
+    "clangdev 18.1.5",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
================================================================================
================================================================================
linux-aarch64
linux-aarch64::clang-18.1.5-default_h33a044b_0.conda
+  "constrains": [
+    "clang-tools 18.1.5.*",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
+  ],
linux-aarch64::clang-tools-18.1.5-default_hb368394_0.conda
-    "clangdev 18.1.5"
+    "clang 18.1.5.*",
+    "clangdev 18.1.5",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
================================================================================
================================================================================
noarch
================================================================================
================================================================================
win-64
win-64::clang-18.1.5-default_hb53fc94_0.conda
+  "constrains": [
+    "clang-tools 18.1.5.*",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
+  ],
win-64::clang-tools-18.1.5-default_h3a3e6c3_0.conda
-    "clangdev 18.1.5"
+    "clang 18.1.5.*",
+    "clangdev 18.1.5",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
================================================================================
================================================================================
osx-64
osx-64::clang-18.1.5-default_h69aec76_0.conda
+  "constrains": [
+    "clang-tools 18.1.5.*",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
+  ],
osx-64::clang-tools-18.1.5-default_h69aec76_0.conda
-    "clangdev 18.1.5"
+    "clang 18.1.5.*",
+    "clangdev 18.1.5",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
================================================================================
================================================================================
linux-64
linux-64::clang-18.1.5-default_h90ac42e_0.conda
+  "constrains": [
+    "clang-tools 18.1.5.*",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
+  ],
linux-64::clang-tools-18.1.5-default_h127d8a8_0.conda
-    "clangdev 18.1.5"
+    "clang 18.1.5.*",
+    "clangdev 18.1.5",
+    "llvm 18.1.5.*",
+    "llvm-tools 18.1.5.*",
+    "llvmdev 18.1.5.*"
linux-64::jaxlib-0.4.23-cuda120py312hc008a70_201.conda
linux-64::jaxlib-0.4.23-cuda120py311hfb00743_200.conda
linux-64::jaxlib-0.4.23-cuda120py312h6027bbc_201.conda
linux-64::jaxlib-0.4.23-cuda120py310h3cc97ca_200.conda
linux-64::jaxlib-0.4.23-cuda120py312hc008a70_200.conda
linux-64::jaxlib-0.4.23-cuda120py39h443a19e_201.conda
linux-64::jaxlib-0.4.23-cuda120py310h3cc97ca_201.conda
linux-64::jaxlib-0.4.23-cuda120py311hfb00743_201.conda
linux-64::jaxlib-0.4.23-cuda120py311h5a49573_201.conda
linux-64::jaxlib-0.4.23-cuda120py39hfad2e6e_200.conda
linux-64::jaxlib-0.4.23-cuda120py310hb2a6d54_201.conda
linux-64::jaxlib-0.4.23-cuda120py39hfad2e6e_201.conda
+    "cuda-cupti >=12.0.90,<13.0a0",

I rebased on top of main, and now the diff is the expected one:
(repodata) traversaro@IITBMP014LW012:~/conda-forge-repodata-patches-feedstock/recipe$ python show_diff.py
================================================================================
================================================================================
linux-armv7l
================================================================================
================================================================================
win-32
================================================================================
================================================================================
osx-arm64
================================================================================
================================================================================
linux-ppc64le
================================================================================
================================================================================
linux-aarch64
================================================================================
================================================================================
noarch
================================================================================
================================================================================
win-64
================================================================================
================================================================================
osx-64
================================================================================
================================================================================
linux-64
linux-64::jaxlib-0.4.23-cuda120py312hc008a70_200.conda
linux-64::jaxlib-0.4.23-cuda120py310hb2a6d54_201.conda
linux-64::jaxlib-0.4.23-cuda120py311h5a49573_201.conda
linux-64::jaxlib-0.4.23-cuda120py311hfb00743_200.conda
linux-64::jaxlib-0.4.23-cuda120py311hfb00743_201.conda
linux-64::jaxlib-0.4.23-cuda120py39hfad2e6e_200.conda
linux-64::jaxlib-0.4.23-cuda120py312h6027bbc_201.conda
linux-64::jaxlib-0.4.23-cuda120py312hc008a70_201.conda
linux-64::jaxlib-0.4.23-cuda120py39hfad2e6e_201.conda
linux-64::jaxlib-0.4.23-cuda120py39h443a19e_201.conda
linux-64::jaxlib-0.4.23-cuda120py310h3cc97ca_201.conda
linux-64::jaxlib-0.4.23-cuda120py310h3cc97ca_200.conda
+    "cuda-cupti >=12.0.90,<13.0a0",

The PR is ready for review @conda-forge/core, thanks!

fyi @conda-forge/jax @conda-forge/cuda-cupti if you have any comment.

This looks good but shouldn't we also have a PR that add this dependency to the jaxlib recipe, too?

This looks good but shouldn't we also have a PR that add this dependency to the jaxlib recipe, too?

In theory the right way to handle would be to add a run_exports to cuda-cupti, we already did that on 12.4 but 12.0 is actually used, see https://github.com/conda-forge/cuda-cupti-feedstock/issues/14 . However, if we do not get feedback on  https://github.com/conda-forge/cuda-cupti-feedstock/issues/14 from @conda-forge/cuda-cupti probably we can just add the dependency on jaxlib itself.

We should definitely add it immediately to jaxlib as I would like to continue fixing other things there and that would probably produce broken packages again.

We should definitely add it immediately to jaxlib as I would like to continue fixing other things there and that would probably produce broken packages again.

Done: https://github.com/conda-forge/jaxlib-feedstock/pull/251 . Or if you are working on another PR, feel free to add it in that PR.


mumble: Fix 32-bit build

Description of changes
32-bit builds fail due to the following error. Apply a patch from master to fix the issue.
[ 25%] Building CXX object plugins/gtav/CMakeFiles/gtav.dir/gtav.cpp.o In file included from /build/source/plugins/gtav/Game.h:9,
                 from /build/source/plugins/gtav/gtav.cpp:6:
/build/source/plugins/gtav/structs.h:218:37: error: static assertion failed
  218 | static_assert(sizeof(CCameraAngles) == 0x408, "");
      |               ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
/build/source/plugins/gtav/structs.h:218:37: note: the comparison reduces to '(1028 == 1032)'

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build pkgsi686Linux.mumble

32-bit build succeeds

@ofborg test mumble botamusique


Transaction pool update by multiple clients

When multiple clients send user commands to a daemon, the inferred nonces could be duplicate if  user commands from one of the clients are still waiting to be added to the pool or if they send at the same time.
That is fixed in this PR by having the fields needed for a user command without the inferred nonce (User_command_input.t) written to a pipe from which the daemon generates a user command by inferring the nonce. The generated user commands are then written to another pipe that's consumed by the network pool. This way all the user command inputs are queued in a single pipe and the user commands created will have valid nonces
Checklist:

[ ] Tests were added for the new behavior
[ ] All tests pass (CI will check this if you didn't)
[ ] Serialized types are in stable-versioned modules
[ ] Does this close issues? List them:

Closes #4416

Want to highlight that this PR doesn't change batch-send-payment command. Couple of points regarding that:

batch-send-payment is in advanced commands and not sure if it is still used
batch-send-payments still uses rpc. With the send-payment and other payment related commands switched to graphql, the assumption is that for unsigned transactions the private key file is on the same machine as the daemon and daemon uses it to sign the transactions. In batch-send-payments, signing is done on the client machine. But with solution in this PR, that behavior would change and signing will have to be done by the  daemon, since nonce is determined by the daemon.
Perhaps we make the assumption regarding private key for  batch-send-payments as well and use approach implemented in this PR (and make another PR for it)? @bkase  @Schmavery

@deepthiskumar since it is in advanced, we should be okay :+1:

FWIW I would be inclined to believe that we might be able to get away with removing/deprecating batch-send-payments?  @yourbuddyconner do you know if we use this feature in our automation anywhere?
Not sure what the actual use case was intended to be for it, but it seems like with the introduction of this PR, there's no nonce issue that could be caused by writing a script that would send transactions in a loop.


[Feature Request] Make API return thumbnail URLs

Similar to #413, it would be great to get access to the thumbnail URL via the API so it can be displayed in an app.

Yes please


Folder (and content lost) upon folder rename

Hello,
Thank you so much for this Self-hosted LiveSync possibility ! This is awesome and so easy to setup.
I created a small vault on a Win10 Laptop (Laptop 1) and put one note in it.
I synched it to the server (docker running on a synology).
I then added a second obsidian on an iphone (this is so amazing how easy we can configure other devices thanks to your setup string/passphrase). I did not play much on the iphone. Mostly consultation and adding a picture and a pdf.
I then added a third obsidian on a second Win10 Laptop (Laptop 2).
All devices have Livesync enabled.
The iOS device is most of the time with the lock screen.
"Do not delete empty folder" is not set on devices.
Once I had everything in sync, I renamed a folder on Laptop 2 and got a case were the folder got duplicated on the Laptop 1 (old and new name).
I continued playing, trying to find a reproductible scenario. I first though that this was because the filename had en emoticon ("🚚 TestFolder" renamed into "TestFolder") but in fact, even without any special character, I get all kind of problem when I rename folder.
This is not systematic but is really easy to produce.
The worst case being that after renaming, the folder completely disappeared on both Laptops (including, the file that was in).
How can I best reproduce it for you to investigate ?
Is there any log that I should collect ? On the laptops ? On the server ?
Sorry for bringing such a worrying issue :-(
Thanks again for your huge work !

Ok. I found how to reproduce the problem.

Install and activate the plugin "Folder Note Core". Set Storage Strategy to "Inside Folder with same name"
Install and activate the plugin "AidenLx's Folder Note". Set Storage Strategy to "Inside Folder with same name"
Laptop1: Create a folder TempFolder.
Laptop2: The folder TempFolder does not happen because (empty) folders are not synced
Laptop1: Ctrl-click on created TempFolder folder to create a folder note. A TempFolder.md is created in the TempFolder folder.
Laptop2: The folder appears (because the folder TempFolder has a file TempFolder.md inside).
Laptop1: Rename the TempFolder to TempFolder2. This renames the folder and the TempFolder.md (to TempFolder2.md).
Laptop2:

Sometimes then TempFolder remains (emptied) while TempFolder2 appears with the TempFolder2.md inside.
Sometimes TempFolder and TempFolder2 are gone 😱

Laptop1: If I restart Obsidian:

Sometimes then TempFolder remains (emptied) while TempFolder2 appears with the TempFolder2.md inside.
Sometimes TempFolder and TempFolder2 are gone 😱

If I disable the folder, the problem disappear but honestly, this plugin is sooooo useful.
I may try the other storage strategies but this one is the recommended/default one.
I think this plugin just demonstrate a race condition effect that leads to data loss and should be further investigated.
I'm ready to help is you need any information.

I went a bit further in my testing and deactivated the "Folder Note Core" and "AidenLx's Folder Note" plugins.
I can reproduce the problem almost every time like this:

Laptop1: Open Obsidian (with livesync on)
Laptop2: Open Obsidian (with livesync on) and let it sync
Laptop1: Create folder "test" and create a note inside it.
Laptop2: "test" folder appears with the note inside
Laptop1: Rename "test" folder into "test2"
Laptop2: The folder "test" disappear and the "test2" folder never appear.
Laptop1: The folder "test2" disappear (and so the note inside is gone)

Thank you for the detailed procedure!
I will test it later!

Hello,
Thank you for the fast analysis.
I updated to v0.17.11 on the 3 devices and restarted obsidian.
I disabled hidden files and internal files because I'm seeing strange behaviours and don't want to mix 2 things.
It seems that a problem remain (see last trace here below) if:

"AidenLx's Folder Note" autosync is enabled
folder's note is renamed (wich automatically rename the folder as well)

With obsidian being opened only on 2 laptops (closed on my iphone), here is what I do:
Test with "AidenLx's Folder Note" autosync disabled
Laptop1 and Laptop2: Turn off "AidenLx's Folder Note" autosync
Laptop1: Restart Obsidian and wait until I see PM->LiveSync begin...
Laptop2: Restart Obsidian and wait until I see PM->LiveSync begin...
In the console of Laptop2, I see, regularly:
Access to fetch at '[https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4237-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXMuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADSfK_c&limit=250'](https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4237-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXMuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADSfK_c&limit=250%27) from origin 'app://obsidian.md' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
plugin:obsidian-livesync:15732          GET https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4237-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXMuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADSfK_c&limit=250 net::ERR_FAILED 307

This error is not seen on Laptop1.
Laptop1: Create folder test11 with note note11 under it.
Pensine:1/16/2023, 11:15:33 PM->Content saved:test11/Untitled.md ,chunks: 1 (new:0, skip:1, cache:0)
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:33 PM->DB <- STORAGE (plain) test11/Untitled.md
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:33 PM->Replication activated
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:41 PM->Content saved:test11/note11.md ,chunks: 1 (new:0, skip:1, cache:0)
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:41 PM->DB <- STORAGE (plain) test11/note11.md
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:41 PM->Replication activated
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:41 PM->entry removed:test11/Untitled.md-2-e0db6b3c1bf5ae7acffcfd76c8daec39
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:44 PM->Replication activated
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:44 PM->Applying 1 files
plugin:obsidian-livesync:19436 Pensine:1/16/2023, 11:15:44 PM->Applied test11/Untitled.md (3-50159d0fd6babf3505c73a66e188b986) change...

Laptop2: test11/note11 appears
Access to fetch at 'https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4237-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXMuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADSfK_c&limit=250' from origin 'app://obsidian.md' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
GET https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4237-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXMuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADSfK_c&limit=250 net::ERR_FAILED 307
Pensine:16/01/2023 23:15:31->TypeError:Failed to fetch
Pensine:16/01/2023 23:15:36->Replication activated
Pensine:16/01/2023 23:15:36->Applying 1 files
Pensine:16/01/2023 23:15:37->DB -> STORAGE (create,plain) test11/Untitled.md
Pensine:16/01/2023 23:15:37->Applied test11/Untitled.md (1-0209ccb53b7a099b96fef094d70e12b0) change...
Pensine:16/01/2023 23:15:43->Replication activated
Pensine:16/01/2023 23:15:44->Applying 2 files
Pensine:16/01/2023 23:15:44->DB -> STORAGE (create,plain) test11/note11.md
Pensine:16/01/2023 23:15:44->Applied test11/note11.md (1-ed509ffa62a0f25c07dd2467df52fd56) change...
Pensine:16/01/2023 23:15:44->xxx <- STORAGE (deleted) test11/Untitled.md
Pensine:16/01/2023 23:15:44->files: 1
Pensine:16/01/2023 23:15:44->Applied test11/Untitled.md (2-e0db6b3c1bf5ae7acffcfd76c8daec39) change...
Pensine:16/01/2023 23:15:44->deleteDB By path:test11/Untitled.md
Pensine:16/01/2023 23:15:44->entry removed:test11/Untitled.md-3-50159d0fd6babf3505c73a66e188b986
Pensine:16/01/2023 23:15:44->Replication activated
Access to fetch at 'https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4241-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXsuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADY7K_s&limit=247' from origin 'app://obsidian.md' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
GET https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4241-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXsuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADY7K_s&limit=247 net::ERR_FAILED 307
Access to fetch at 'https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4241-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXsuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADY7K_s&limit=250' from origin 'app://obsidian.md' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
GET https://obsidian-livesync.my.domain.com/myuser/_changes?style=all_docs&feed=longpoll&heartbeat=30000&since=4241-g1AAAACReJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5iYGBfXsuUIzd0swyxTTFDF09DhPyWIAkQwOQ-g83iOMW2CBDA4MkS-NUdG1ZADY7K_s&limit=250 net::ERR_FAILED 307
Pensine:16/01/2023 23:16:09->TypeError:Failed to fetch

Laptop1: Renamed folder test11 to test12
Pensine:1/16/2023, 11:24:05 PM->Content saved:test12/note11.md ,chunks: 1 (new:0, skip:1, cache:0)
Pensine:1/16/2023, 11:24:05 PM->DB <- STORAGE (plain) test12/note11.md
Pensine:1/16/2023, 11:24:05 PM->Replication activated
Pensine:1/16/2023, 11:24:05 PM->entry removed:test11/note11.md-2-1c5436fff21bf76eedf8b1edcf460f6e

Laptop2: After a while, folder test11 got renamed into test12
Pensine:16/01/2023 23:25:51->Replication activated
Pensine:16/01/2023 23:25:52->Applying 2 files
Pensine:16/01/2023 23:25:52->xxx <- STORAGE (deleted) test11/note11.md
Pensine:16/01/2023 23:25:52->files: 0
Pensine:16/01/2023 23:25:52->All files under the parent directory ([object Object]) have been deleted, so delete this one.
Pensine:16/01/2023 23:25:52->deleteDB By path:test11/note11.md
Pensine:16/01/2023 23:25:52->xxx <- STORAGE (deleted) test11
Pensine:16/01/2023 23:25:52->files: 6
Pensine:16/01/2023 23:25:52->Applied test11/note11.md (2-1c5436fff21bf76eedf8b1edcf460f6e) change...
Pensine:16/01/2023 23:25:52->entry removed:test11/note11.md-3-0e92fb903ec4e33f7cbe8753914dcb21
Pensine:16/01/2023 23:25:52->Replication activated
Pensine:16/01/2023 23:25:52->DB -> STORAGE (create,plain) test12/note11.md
Pensine:16/01/2023 23:25:52->Applied test12/note11.md (1-353358a73fa0d5915c48ba5a20d2e47e) change...

Laptop1: a bit later
I do not understand why it still manages test11...
Pensine:1/16/2023, 11:28:08 PM->Before LiveSync, start OneShot once...
Pensine:1/16/2023, 11:28:08 PM->Oneshot Sync begin... (pullOnly)
Pensine:1/16/2023, 11:28:08 PM->Replication activated
Pensine:1/16/2023, 11:28:08 PM->Replication completed
Pensine:1/16/2023, 11:28:08 PM->LiveSync begin...
Pensine:1/16/2023, 11:28:08 PM->Replication activated
Pensine:1/16/2023, 11:28:08 PM->Replication activated
Pensine:1/16/2023, 11:28:08 PM->Applying 1 files
Pensine:1/16/2023, 11:28:08 PM->Applied test11/note11.md (3-0e92fb903ec4e33f7cbe8753914dcb21) change...
Pensine:1/16/2023, 11:31:05 PM->Replication closed
Pensine:1/16/2023, 11:31:25 PM->Before LiveSync, start OneShot once...
Pensine:1/16/2023, 11:31:25 PM->Oneshot Sync begin... (pullOnly)
Pensine:1/16/2023, 11:31:25 PM->Replication completed
Pensine:1/16/2023, 11:31:25 PM->LiveSync begin...
Pensine:1/16/2023, 11:31:27 PM->Replication closed
Pensine:1/16/2023, 11:33:07 PM->Before LiveSync, start OneShot once...
Pensine:1/16/2023, 11:33:07 PM->Oneshot Sync begin... (pullOnly)
Pensine:1/16/2023, 11:33:07 PM->Replication completed
Pensine:1/16/2023, 11:33:07 PM->LiveSync begin...
Pensine:1/16/2023, 11:33:08 PM->Replication is already in progress.

Closed Obsidian on Laptop2
Closed Obsidian on Laptop1
Started Obsidian on Laptop1
No test11, only test12 👍
Started Obsidian on Laptop2
I re-did several this test and could not get a situation were test11 remained or where both test11 and test12 were deleted.
With "AidenLx's Folder Note" autosync enabled, rename folder14
Laptop1: Create a folder14 with a note folder14/folder14.md
Laptop2: folder14/folder14.md appears
Laptop1: rename the folder14 to folder15
Pensine:1/16/2023, 11:46:48 PM->Content saved:test15/note11.md ,chunks: 1 (new:0, skip:1, cache:0)
Pensine:1/16/2023, 11:46:48 PM->DB <- STORAGE (plain) test15/note11.md
Pensine:1/16/2023, 11:46:48 PM->Replication activated
Pensine:1/16/2023, 11:46:48 PM->entry removed:test14/note11.md-2-14de62709424b359bf9f0e069fe1c4a0
Pensine:1/16/2023, 11:46:48 PM->Content saved:test15/test15.md ,chunks: 2 (new:0, skip:3, cache:1)
Pensine:1/16/2023, 11:46:48 PM->DB <- STORAGE (plain) test15/test15.md
Pensine:1/16/2023, 11:46:48 PM->entry removed:test14/test14.md-3-dbf70d91ec8db1606ae8ca34cae0ecd3
Pensine:1/16/2023, 11:46:49 PM->Replication activated

Laptop2: After a while folder14 is gone and folder15 appears with the right content
Pensine:16/01/2023 23:48:01->Replication activated
Pensine:16/01/2023 23:48:01->xxx <- STORAGE (deleted) test14/note11.md
Pensine:16/01/2023 23:48:01->files: 1
Pensine:16/01/2023 23:48:01->deleteDB By path:test14/note11.md
Pensine:16/01/2023 23:48:01->entry removed:test14/note11.md-3-226159e4174ee235ddb8ca7e881d67e6
Pensine:16/01/2023 23:48:02->Applying 3 files
Pensine:16/01/2023 23:48:02->DB -> STORAGE (create,plain) test15/note11.md
Pensine:16/01/2023 23:48:02->Applied test15/note11.md (1-e2fb5995bc3daa6b64ac315e06b6ad67) change...
Pensine:16/01/2023 23:48:02->xxx <- STORAGE (deleted) test14/test14.md
Pensine:16/01/2023 23:48:02->files: 0
Pensine:16/01/2023 23:48:02->All files under the parent directory ([object Object]) have been deleted, so delete this one.
Pensine:16/01/2023 23:48:02->deleteDB By path:test14/test14.md
Pensine:16/01/2023 23:48:02->xxx <- STORAGE (deleted) test14
Pensine:16/01/2023 23:48:02->files: 7
Pensine:16/01/2023 23:48:02->Applied test14/test14.md (3-dbf70d91ec8db1606ae8ca34cae0ecd3) change...
Pensine:16/01/2023 23:48:02->entry removed:test14/test14.md-4-12525a6392bfd8a7ab1629afc5985658
no afitem found for path test14/test14.md, escaping...
no afitem found for path test14, escaping...
Pensine:16/01/2023 23:48:02->DB -> STORAGE (create,plain) test15/test15.md
Pensine:16/01/2023 23:48:02->Applied test15/test15.md (1-bbd4c758ad94806273a78598b350aa03) change...

With "AidenLx's Folder Note" autosync enabled, rename folder15/folder15.md into folder15/folder16.md which automatically gets renamed into folder16/folder16.md
Note: there is another md file in the folder: note11.md
Laptop1: Rename the folder15/folder15.md into folder15/folder16.md
The "AidenLx's Folder Note" plugin automatically renames folder15/folder16.md into folder16/folder16.md
Pensine:1/16/2023, 11:55:04 PM->Before LiveSync, start OneShot once...
Pensine:1/16/2023, 11:55:04 PM->Oneshot Sync begin... (pullOnly)
Pensine:1/16/2023, 11:55:04 PM->Replication completed
Pensine:1/16/2023, 11:55:04 PM->LiveSync begin...
Pensine:1/16/2023, 11:55:23 PM->Error:ENOENT: no such file or directory, open 'C:\Users\p.hilson\Documents\Obsidian\Pensine\Pensine\test15\test16.md'
Pensine:1/16/2023, 11:55:23 PM->Content saved:test16/note11.md ,chunks: 1 (new:0, skip:1, cache:0)
Pensine:1/16/2023, 11:55:23 PM->DB <- STORAGE (plain) test16/note11.md
Pensine:1/16/2023, 11:55:23 PM->Replication activated
Pensine:1/16/2023, 11:55:23 PM->entry removed:test15/note11.md-2-f4addd968a2887b8bbf8b142deb122fd
Pensine:1/16/2023, 11:55:23 PM->Content saved:test16/test16.md ,chunks: 2 (new:0, skip:3, cache:1)
Pensine:1/16/2023, 11:55:23 PM->DB <- STORAGE (plain) test16/test16.md
Pensine:1/16/2023, 11:55:26 PM->Replication activated
Pensine:1/16/2023, 11:55:26 PM->Applying 1 files
Pensine:1/16/2023, 11:55:26 PM->Applied test15/note11.md (3-598dbe0f5db373ff8e2d95e4efa78723) change...

**At this stage, Laptop1 UI does only show test16.
Laptop2: test16 folder appear with test16.md and note11.md inside. BUT test15 folder appears with test15.md inside (but no note11.md)
Pensine:16/01/2023 23:55:15->TypeError:Failed to fetch
Pensine:16/01/2023 23:55:24->Replication activated
Pensine:16/01/2023 23:55:24->Applying 1 files
Pensine:16/01/2023 23:55:25->DB -> STORAGE (create,plain) test16/note11.md
Pensine:16/01/2023 23:55:25->Applied test16/note11.md (1-77b7a2f625b65fa6c7ba655613125c75) change...
Pensine:16/01/2023 23:55:26->Applying 2 files
Pensine:16/01/2023 23:55:26->xxx <- STORAGE (deleted) test15/note11.md
Pensine:16/01/2023 23:55:26->files: 1
Pensine:16/01/2023 23:55:26->Applied test15/note11.md (2-f4addd968a2887b8bbf8b142deb122fd) change...
Pensine:16/01/2023 23:55:26->deleteDB By path:test15/note11.md
Pensine:16/01/2023 23:55:26->entry removed:test15/note11.md-3-598dbe0f5db373ff8e2d95e4efa78723
Pensine:16/01/2023 23:55:26->Replication activated
Pensine:16/01/2023 23:55:26->DB -> STORAGE (create,plain) test16/test16.md
Pensine:16/01/2023 23:55:26->Applied test16/test16.md (1-ef505e71655d5590a3f8db48c89d2f50) change...

Laptop1: Stop Obsidian
Laptop2: Stop Obsidian
Laptop1: Start Obsidian: test15 folder appear with test15.md inside
Pensine:1/17/2023, 12:06:30 AM->UPDATE STORAGE
Pensine:1/17/2023, 12:06:30 AM->Check or pull from db:test15/test15.md
Pensine:1/17/2023, 12:06:30 AM->DB -> STORAGE (create,plain) test15/test15.md
Pensine:1/17/2023, 12:06:30 AM->Check or pull from db:test15/test15.md OK
Pensine:1/17/2023, 12:06:30 AM->UPDATE STORAGE done.

Laptop2: Start Obsidian
Nothing special  happen

Sorry for the lengthy report.
In a few words, the problem I reproduce is:

With "AidenLx's Folder Note" plugin enabled
With "AidenLx's Folder Note" plugin autosync enabled,
When renaming the folder's note (eg: folder15/folder15.md => folder15/folder16.md)

Then:

The plugin renames automatically the folder15 into folder16 (hence: folder15/folder16.md => folder16/folder16.md).
After obsidian-livesync sync happen

Laptop1 has folder16/folder16.md
Laptop2 has folder16/folder16.md AND folder15/folder15.md
After obsidian restart, Laptop1 has folder16/folder16.md AND folder15/folder15.md

folder15/folder15.md shouldn't exist anywhere at the end.
The other thing I saw in the Laptop2 console were errors about cors but this is maybe unrelated and I will check https://github.com/vrtmrz/obsidian-livesync/issues/111 for this.

Reproduced! Thank you for this information! It's good to have a long one. Nothing to be worried about! So helpful!
It probably will be fixed in a couple of days!

Thank you for your patience! It has been fixed at 0.17.15.
This plugin caches changes and thinned if redundant to apply in batch. However, caches both deleted and created files had pointed to the same file. so it had been got weird.
Fixed queueing and caching behaviour.
I am really sorry for bothering you again, but would you mind if I ask for trying v0.17.15 again?

 And sure, I will test and provide feedback asap.
Thank you !

I changed my passphrase and overwrote my selfhosted couchdb db.
I deleted my vault on Laptop2 and my iphone and recreated a new setup URI from Laptop1.
I let Laptop1 sync finish with the couchdb.
Then I created the new vault on Laptop1 and the iphone, I installed the livesync plugin and did setup using the setup uri.
I then added and enabled the AidenLx's Folder Note (and Folder Note Core) plugin.
iphone
All my tests were successful 👍
Laptop2
It half synched then stopped syncing with a 'cors' error displayed in the logs.
This is strange but because of this, I cannot fully validate your correction. (and maybe you will prefer that this is followed in another ticket ?!?)
I restarted Obsidian with the console opened and see that it starts working with the database without error.
When I click the replicate button, it displays:
Pensine:19/01/2023 22:03:55->Oneshot Sync begin... (sync)
index.html:1 Access to fetch at 'https://obsidian-livesync.mydomain.be/mydb/' from origin 'app://obsidian.md' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.  The only sure thing is you didn't touch the server.
To check details, open inspector. plugin:obsidian-livesync:19760

Pensine:19/01/2023 22:03:56->Could not connect to server.

The only thing my pry eyes see is the double // on the line:
could not connect to https://obsidian-livesync.mydomain.be//mydb: TypeError:Failed to fetch
But this is probably a type in a printf to the console.
Any idea why I'm getting this 'cors' error ?

----remote config----
Requesting information to the remote CouchDB has been failed. If you are using
IBM Cloudant, it is the normal behaviour.

I am wondering why inside ----remote config---- had not been shown if you are using a self-hosted server.
If you once have synchronised, the configuration should be correct. (And looks fine)
Could you please check what will be shown on DevTools when you selected the errored request?
And, at nginx, we could not set add_header for CORS. May I see your configuration if you do not mind?

iphone is syncing nicely over the 4G network (no error in the log and ----remote config---- well reported).
This let me thing that the nginx and the couchdb server access is fine.
(note: once I'm confident and have a fully working sync, I'll share with you the docker-compose and nginx configuration)
Laptop2 is a corporate computer on which I'm admin but do not control everything. I suspect things being done under the wood like antivirus/proxying, ...
I can see that bitwarden all of a sudden does not sync with my self-hosted vaultwarden server.
I'll do my duty and make sure everything is ok on that side first.
Sorry for the annoyance.
I'll report here once I find the root cause.

I have so relieved to hear that!
LiveSync uses PouchDB for synchronisation.
And it will group the stored chunks together to some extent and transfer them in sequence.
If by chance, large chunks are then sent together in consecutive parts, CouchDB may reject them.
Of course, WAFs such as Akamai may reject consecutive requests, so in practice, it is often difficult to aim to reproduce this problem.
Looking forward to your report!

Setup

CouchDB running as a docker on a Synology listening on standard port 5984
Nginx reverse proxy running as a docker (SWAG) as well on the Synology and

Listening on https://obsidian-livesync.mydomain.be
Doing the SSL endpoint, geoip+fail2ban protections and transfering data to the CouchDB

pfSense router listening on port 443 on my registered domain name and forwarding incomming traffic to the nginx.

Obsidian + obsidian-livesync plugin are installed on:

Laptop1 (home Win10 laptop)
Laptop2 (corporate Win10 laptop)
iPhone

Problem Symptoms

obsidian-livesync on Laptop1 (home Win10 laptop) and iPhone are synchrozing correctly.
obsidian-livesync on Laptop2 (corporate Win10 laptop) is not synchronizing correctly (cors errors). The strangiest thing is that it sometimes sync a few things then stop for no reason.

Analysis
Digging further, I found out that, on Laptop2, I had trouble reaching other services running behind nginx (and therefore reached over https using the fqdn.
So for example, if I have another service setup which is reachable on nginx on https://niceservice.mydomain.be, then:

When Laptop2 was in the office, it could reach the niceservice using any browser.
From home, Laptop1 and iPhone (hence with a local ip address in the same subnet than the Synology) where able to reach the niceservice using any browser.
From home, Laptop2 (hence with a local ip address in the same subnet than the Synology)  was NOT able to reach the niceservice.

The error seen in the browser was:
McAfee Security Gateway - Bad Response -The proxy did not receive a valid response in time.
I first thought it was due to DNS Reflection but the strange thing was that Laptop1 and iPhone were not having the problem.
The problem is linked to McAfee running on my corporate Laptop2 that makes a check with the McAfee Security Gateway running at my company. The Security Gateway invalidates the request for some obscure resolution issues.
Solution
I then tried changing the way DNS resolution occured in my local network for my own services (exposed using xxx.mydomain.be).
I changed for NAT Reflection to Split DNS and all of a sudden I could reach all "hosts" declared in my DNS Resolver/Forwarder Overrides (and allowed in nginx) from Laptop2 (and Laptop1 and iPhone).
Conclusion
I know this problem has nothing to do with obsidian-livesync but I promised you the glory details once I would have solved my problem. Hopefully those can help other people.
Annex: Nginx couchdb configuration
The swag\nginx\proxy-confs\obsidian_livesync.subdomain.conf contains:
# Those are fake uri/ips
# Redirecting http[s]://obsidian_sync.mydomain.be to http://<IP_ADDRESS>:5984

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name obsidian-livesync.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    # enable for geo blocking
    # See /config/nginx/geoip2.conf for more information.
    if ($allow_list = yes) {
        set $allowed_country yes;
    }
    if ($allowed_country = no) {
        return 444;
    }

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_proto http;
        set $upstream_app <IP_ADDRESS>;
        set $upstream_port 5984;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

        proxy_max_temp_file_size 2048m;
    }
}

⚠ I should probably set the proxy_max_temp_file_size to the livesync plugin config. Any suggestion is welcome.

Thank you for your detailed information!
It might be great help for everyone! You know that the FAQ is not yet here.
enterprise environment makes us worried. I also had got stuck with the network gateway’s disconnection. Replication made a bit lot of reconnection, and looks like that felt them suspicious.


Query

Hi
Great module however what is the length property for and can the object keys be the name of host rather than the index number?
Thanks
Simon

The length property is for looping Hosts. So to find the Host section, you can use an old fashioned for loop:
for (var i = 0, len = config.length; i < len; i++) {
  var host = config[i]
  if (host.Host === theHostYouWant) {
    dealWith(host)
  }
}

By name of host you mean Host or HostName?

You can avoid setting the length as you can loop through the object keys like so;
var keys = Object.keys(config);
for (var i = 0,  i < keys.length; i++) {
  var host = keys[i]
  if (host.Host === theHostYouWant) {
    dealWith(host)
  }
}

However to find the relevant host you would need to set top level keys of object to the name of host, see this example config and the resulting object;
Host server1
     HostName server1.com
     User admin
     Port 22
     IdentityFile /path/to/key
Host server2
     HostName server2.com
     User admin
     Port 22
     IdentityFile /path/to/key

The object would look like this
var config = {
    server1: {
        HostName: 'server1.com',
        User: 'admin',
        Port: 22,
        IdentityFile: 'path/to/key'
    },
    server2: {
        HostName: 'server2.com',
        User: 'admin',
        Port: 22,
        IdentityFile: 'path/to/key'
    }
};

Are you willing to make this change?

That'll break current API. Would you give me a couple of hours to consider about it?

Yes sure

Sorry Simon, I think I'll stick with current structure.
The numeric properties and the .length were added because these are just sections within the parsed ssh config. And according to the man page of ssh_config, both Host and Match can start new sections.
The advantage of current structure is that it will retain the original structure of the ssh config text passed in. Given some ssh config like below:
IdentityFile ~/.ssh/id_rsa

Host *
  User nil

Host laputa
  User gulliver
  ForwardAgent true

If we parse it, then stringify the returned object, the result will be consistent with the original. Especially if the ssh config is long and has duplicated Host sections.
 Will your use case looks like below:
var sshConfig = require('ssh-config')

var config = sshConfig.parse(fs.readFileSync('~/.ssh/config'))
var settings = config.match('laputa.com')

// and settings will be an Object looks like:
//
//  {
//    IdentityFile: '~/.ssh/id_rsa',
//    Host: 'laputa.com',
//    User: 'gulliver',
//    ForwardAgent: true
//  }

And about .length. If an enumerable length property were confusing, I can just hide it via Object.defineProperty, which is the same way I'm gonna use to provide the .match method.

No problem I have spent some time reading more about config file structure (I only use the host part currently) .  Your suggested api  where you are returned the relevant host from the parsed object with match would work very well. :+1:

Great. I'm on it.

Cheers Jake

I have pulled the add-query branch down and cannot get query to work correctly

Can you provide more information?

See this snippet
var sshConfig = require('ssh-config');
var path = require('path');
var fs = require('fs');

var directory = path.join(process.env.HOME || process.env.USERPROFILE, '.ssh', 'config');

var results = fs.readFileSync(directory, 'utf8');

var parsed = sshConfig.parse(results);

var test = parsed.query('example1');
console.log(test);

It prints as an object of the first entry from config file and does not match the host called example1
Host example
     HostName example.com
     User root
     Port 22
     IdentityFile /path/to/key
Host example1
     HostName example1
     User user
     Port 15
     IdentityFile another/path/to/keyfile

Confirmed. Nice catch!

@simon-p-r I've pushed a new commit in add-query branch. I believe this is fixed now.
Besides, what do you consider about current behaviour of .query? I use ssh-config in a deployer script myself. The most use cases about the parsed config data is to query some host and get relative ssh parameters about that host. So I implemented .query this way.
If that's not exactly what you need, please let me know.

If .query suffices, I'll merge and release 0.2.0 now.

I have tried again however it still returns the incorrect entry.  I will write some tests on my forked repo to show you want I thought query api would be.

I have added a test and a new test file which breaks the query method as it fails on port parsing

Nice. I've checked out your code and looked inside. Your tests are based on an old version of add-query branch. The commit that fixes this issue is missed https://github.com/dotnil/ssh-config/commit/c71ffd4f8405df60b011d029cf4a66b2993d91e4

Ok I have pulled that in and it appears to work very well, I like the fact it merges other parsed config into returned object - that is very clever!  Look forward to using this excellent module in my project :+1:


UDP ports usage

The information might be outdated.
I have customer tenants with QoS disabled, but I can see on the meeting clients network traffic the UDP ports 3478-3481 in use...

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 470478ed-3b4e-ced8-0e28-6cda545b8400
Version Independent ID: 95b4044b-3a47-a511-c726-b5ee75863441
Content: Manage meeting settings - Microsoft Teams
Content Source: Teams/meeting-settings-in-teams.md
Service: msteams
GitHub Login: @SerdarSoysal
Microsoft Alias: serdars

Hello @TheITguy666 Luis, I am sorry, I am not sure I understand, those ports are required https://docs.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide
Let me know if I understood you correctly,

Sorry @get-itips
I open this on the wrong page :(


fix diagram: services speak to featureflag server not its ui

The UI is for a user to create/edit/toggle flags, services like the CheckoutService talk to the feature flag (grpc) server.

Is it a fair assumption that eventually almost all of the services will interact with featureflagbservice in some way?


feat(fluttium_interfaces): refactor to use addons semantics instead of actions

Status
READY
Description
Refactor to addons semantics instead of actions, part of #197
Type of Change

[x] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🗑 Chore

Looking good

Nit but technically isn’t it “add-on” which would mean in Dart it should be addOn or addOns?

Oh. Well dang, now I have to rethink the wording because I dont like addOns or add_ons for the pubspecl


Executing futures in wasm

the wasm-bindgen-futures crate provides for consuming promises as futures, and returning futures to JS as promises, but not executing futures in wasm.
I'm currently experimenting with borrowing the strategy for passing futures to js for my use-case.

I'm not sure it's actually possible to execute futures in wasm without the support of JS? Executing futures involves some degree of blocking but JS/wasm can't block right now I think? Do you have a particular API in mind though for this?

I'm playing with creating an indexeddb library. Eventually, I'd like to make it integrate into a virtual DOM library so an action
+------------+     +------------+
|click action+----->future chain|
+-----+------+     +-----+------+
      |                  |
      |                  |
+-----v------+           |
|state change<-----------+
+------------+

where you have your state, and functions describing how the state changes when the future is ready or rejected. This applies equally to fetch as much as indexedDB. The beauty of futures is that you can have quite a complex operation with many steps but treat it as a single atomic operation.
I'm just struggling to get my head around how js and wasm interact here. Do we need an executor in wasm using something like setTimeout(() => {..}, 0), or can we arrange to be woken up by the JS when a promise completes.
I always find the lack of determinism in js a bit frustrating (w.r.t. the order that things will happen in), and I feel like futures would help me get a handle on it, but the big picture is just eluding me.

I'm thinking I probably have to pass the function I want to run into the promise, so that JS knows what code to run. Maybe this means I can't use futures (at least not the standard rust ones).

The way that this is handled in stdweb is that it provides a spawn_local function, which lets you spawn a Future:
spawn_local(some_rust_future);

You can then write your code like this:
fn on_click() {
    state.merge(Action::LoadData);
    spawn_local(indexeddb::open("my_db", 1)
        .then(|db| db.start_transaction())
        .then(|(db, trans)| db.object_store("people"))
        .then(|store| store.get(0))
        .then(|record| {
            state.merge(Action::LoadSucceeded {
                name: record.name,
                age: record.age
            });
        }));
}

Internally spawn_local uses a Rust dequeue of Tasks, and it uses JavaScript's microtask queue to delay the Future execution by one tick.
The same sort of function should be implementable in wasm-bindgen as well.

@Pauan do you have to be running an event loop for that to work?

@derekdreery In asynchronous code, an event queue is mandatory (though it's hidden from the user).
JavaScript provides two built-in event queues: macrotask and microtask (Promises always use the microtask event queue). You can read more here:
https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/
So the question is whether spawning Futures directly uses the JS microtask queue, or whether it uses a Rust dequeue.
Using the JS microtask queue is a lot simpler, but using a Rust dequeue is multiple orders of magnitude faster.
Here is a very old stdweb Executor which doesn't use a Rust dequeue, it spawns Futures directly using the JS Promise microtask queue.
It was written for Futures 0.1, it's quite old, probably buggy, and quite slow, but it is also simple and easy to understand, so hopefully it's educational.

All of this event queue stuff is an implementation detail: the user just calls spawn_local and at some unknown point in the future the Future will run.
I say "unknown point", but it's usually very fast (a few milliseconds at most). The "unknown" part just means you can't rely upon it running at a deterministic point in time, since it's asynchronous.

Thanks for the microtask reading - I feel like I understand all this stuff much better now!

I am currently writing a small HttpRequest crate and ended up creating a wrapper struct with a custom Dropimplementation.
pub struct Request<T: Future + 'static = JsFuture>(Option<T>);

/// snip ..

impl<T: Future + 'static> std::ops::Drop for Request<T> {
    fn drop(&mut self) {
        if let Some(future) = self.0.take() {
            future_to_promise(future.and_then(|_| {
                future::ok(JsValue::null())
            }).or_else(|_| {
                future::err(JsValue::null())
            }));
        }
    }
}

Now the browser is doing most of the work.
Usage:
  Request::new(Method::Get, "example.org/test")
        .header("Accept", "text/plain").send()
        .and_then(|resp_value: JsValue| {
            let resp: Response = resp_value.dyn_into().unwrap();
            resp.text()
        })
        .and_then(|text: Promise| {
            JsFuture::from(text)
        })
        .and_then(|body| {
            println!("Response: {}", body.as_string().unwrap());
            future::ok(())
        });

@lcnr Running an asynchronous Future in the Drop impl seems pretty hacky to me.
If the goal is to cancel a fetch, you should use the standard AbortController, something like this:
pub struct Request<'a> {
    url: &'a str,
    init: RequestInit,
}

impl<'a> Request<'a> {
    pub fn new(url: &'a str) -> Self {
        Self {
            url,
            init: RequestInit::new(),
        }
    }

    pub fn send(mut self) -> RequestFuture {
        let controller = AbortController::new().unwrap();

        let init = self.init.signal(Some(&controller.signal()));

        let future = window().unwrap().fetch_with_str_and_init(self.url, &init).into();

        RequestFuture {
            controller,
            future,
        }
    }
}

pub struct RequestFuture {
    controller: AbortController,
    future: JsFuture,
}

impl Drop for RequestFuture {
    #[inline]
    fn drop(&mut self) {
        self.controller.abort();
    }
}

impl Future for RequestFuture {
    type Output = Result<JsValue, JsValue>;

    #[inline]
    fn poll(self: Pin<&mut Self>, waker: &LocalWaker) -> Poll<Self::Output> {
        self.future.poll_unpin(waker);
    }
}

Untested, but it should be close to correct.

I want to make requests during which I return control of the main thread back to to browser/javascript and don't help to store the future anywhere.

That happens naturally as part of the Promises/Futures system, you don't need to do anything special for that.
The best thing to do is to have Request::send return a Future (similar to what I showed in my post), and then you can use future_to_promise to spawn it in main:
#[wasm_bindgen(start)]
pub fn main() {
    future_to_promise(
         Request::new(Method::Get, "example.org/test")
            .header("Accept", "text/plain").send()
            .and_then(|resp_value: JsValue| {
                let resp: Response = resp_value.dyn_into().unwrap();
                resp.text()
            })
            .and_then(|text: Promise| {
                JsFuture::from(text)
            })
            .and_then(|body| {
                println!("Response: {}", body.as_string().unwrap());
                future::ok(JsValue::UNDEFINED)
            })
    );
}

No global state needed.
And then when spawn_local is supported you can just replace future_to_promise with spawn_local.
It's even nicer with async/await:
#[wasm_bindgen(start)]
pub fn main() {
    future_to_promise(async {
        let resp_value = await!(
            Request::new(Method::Get, "example.org/test")
                .header("Accept", "text/plain").send()
        )?;

        let resp: Response = resp_value.dyn_into().unwrap();
        let body = await!(JsFuture::from(resp.text()?))?;

        println!("Response: {}", body.as_string().unwrap());

        Ok(JsValue::UNDEFINED)
    });
}

Naturally you don't want to put everything into main, but that's not a problem: you can just split your program into multiple async functions:
async fn get_text(url: &str) -> Result<String, JsValue> {
    let resp_value = await!(
        Request::new(Method::Get, url).header("Accept", "text/plain").send()
    )?;

    let resp: Response = resp_value.dyn_into().unwrap();

    let body = await!(JsFuture::from(resp.text()?))?;

    Ok(body.as_string().unwrap())
}

async fn do_something() -> Result<(), JsValue> {
    let body = await!(get_text("example.org/test"))?;

    println!("Response: {}", body);

    Ok(())
}

#[wasm_bindgen(start)]
pub fn main() {
    future_to_promise(async {
        await!(do_something())?;
        Ok(JsValue::UNDEFINED)
    });
}

When you use await! on a Promise, it will yield control to the browser.
So in the above example, await!(Request::new(Method::Get, url).header("Accept", "text/plain").send()) and await!(JsFuture::from(resp.text()?)) will yield control.

P.S. async/await support will require #1105 to be fixed first (or I guess you can use the 0.1 to 0.3 Futures compatibility shim to make it work?).

FWIW JS at the fundamental level can't block, so it's almost always queueing up callbacks to execute at some later date. If you do blocking work at the base level there's likely some callback that gets invoked when the operation is finished (either successfully or not). In that sense we can queue up callbacks to run on events, and those callbacks could drive another event queue in Rust (much like futures work today with tokio and such).
Some of this may belong in the wasm-bindgen-futures crate, but otherwise much of this is largely stock futures and other crates which in theory already work.
Are there still points though that want to be clarified before closing this?

@alexcrichton what do you think about @Pauan's suggestion to add a spawn_local method to wasm-bindgen-futures?

It may be a good idea! I'll admit though that I don't fully understand the motivation after skimming over this issue again. Could you remind me the motivation though for adding a function like that?

(er also good to mention the context that future_to_promise is sort of like spawn_local, but it seems to me like spawn_local would likely have different performance characteristics, so I'm unclear if it's just that or if it also adds expressivity)

Ok just wanted to confirm. That seems reasonable to me to add to wasm-bindgen-futures!

I'll have a go at implementing the Queue and see what it looks like.

Very naive implementation in https://github.com/rustwasm/wasm-bindgen/pull/1148.

Just came back to this after a bit of time away...
Still haven't taken on the EventListener thing but I ended up just storing the closure in the Image struct and it seems to work fine so far
e.g. (partial code)
                img.set_src(url);

                let task = current();

                let closure = Closure::wrap(Box::new(move || {
                    console::log_1(&JsValue::from_str("loaded"));
                    task.notify();
                }) as Box<FnMut()>);

                img.set_onload(Some(closure.as_ref().unchecked_ref()));

                //closure.forget();

                self.img = Some(img);
                self.state = ImageState::Loading;
                self.closure = Some(closure);

@dakom are u sure it's not leaking memory?

The Image container thing gets dropped somehow once the future resolves, so I think it's all good... not 100% sure though and I don't get how that's really possible since the Future resolves an Image which still has the callback attached to it somehow?

this could throw an exception on the js side if the rust struct is dropped?

I'm not really sure about much of anything here, hehe
The one gotcha I'm having is that the closure can't access the containing struct since the closure outlives it... so not sure how to signal errors yet :\

Tried to workaround by using a channel... but seems I can't store that in a closure?
error[E0277]: expected a `std::ops::FnMut<()>` closure, found `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]`
  --> crates\web_sys_loaders\src\image.rs:72:49
   |
72 |                 let closure_err = Closure::wrap(Box::new(cb) as Box<FnMut()>);
   |                                                 ^^^^^^^^^^^^ expected an `FnMut<()>` closure, found `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]`
   |
   = help: the trait `std::ops::FnMut<()>` is not implemented for `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]`
   = note: wrap the `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]` in a closure with no arguments: `|| { /* code */ }
   = note: required for the cast to the object type `dyn std::ops::FnMut()`

code is basically:
let (sender_err, receiver_err):(Sender<JsValue>, Receiver<JsValue>) = channel();

//error callback
let task = current();
let cb = move || {
    sender_err.send(JsValue::from_str("unknown error!"));
    task.notify();
};

let closure_err = Closure::wrap(Box::new(cb) as Box<FnMut()>);

img.set_onerror(Some(closure_err.as_ref().unchecked_ref()));

Just came across this curious page on MDN: https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/queueMicrotask
o_O

@dakom That's very cool, and exactly what we need to remove the Promise overhead. But it's not standardized, so we can't use it right now.

@dakom That's very cool, and exactly what we need to remove the Promise overhead. But it's not standardized, so we can't use it right now.

@Pauan Today its not only standardized but also well supported https://caniuse.com/?search=queueMicrotask


Add specific test case for ReactorSession.dispose()

Can one of the admins verify this patch?

@conniey or @srnagar mind taking a look at this?

Thanks @AljoschaP


Prevent false positive on setting P element's font family

closes #6286
added .not('.red-text') to assert
tests ok on my instance

Please don't put issue numbers in your PR title. Change it to something more descriptive of the change being made. Also, please squash your commits.

this was messy, included another file edit somehow. i'm closing and trying again.


add join transformation

This adds a join transformation that behaves similarly to the drag-and-drop way of joining in CODAP.
CODAP allows you to perform a join by dragging one attribute from a context onto an attribute from another. In effect, it joins collections together by copying attributes from the dragged attribute's collection (what I've called the joining collection) into the dragged-onto attribute's collection (what I've called the base collection). It fills in case data by searching the joining collection for the first case for which the dragged-from and dragged-onto attributes' values match. If no such match is found, the case values for the new attributes are left missing.
CODAP's join effectively mutates the base collection, by literally adding attributes that then use a formula + lookup function to reference the data from another context/collection. Our join copies their functionality but produces a new table representing the join. See the following example:

Worth noting is that the base attribute and joining attribute ("State" and "State" here) are both present in the joined table. I figured this is useful since you can decide to delete one of them later if necessary, but they may be named different things and indeed you may want to use formulas/transformations that reference one or the other.

Data for good join example with birds:
State,State Bird
Nevada,Mountain bluebird
Oregon,Western meadowlark
Missouri,Bobwhite quail
Washington,Willow goldfinch
Virginia,Northern cardinal
Rhode Island,Rhode Island Red


Winget error handling

Sometimes the winget repo fails due to CDN issues (ex. https://www.bleepingcomputer.com/news/microsoft/microsoft-winget-package-manager-failing-due-to-cdn-issues/ - and that's not the first time)
For me it happened today after Optimizing and defragmenting my drives/games locally (and some unrelated registry cleaning tasks)
I got the same error:
Failed to open the predefined source
please report to winget maintainers.
An unexpected error occurred while executing the command:
0x80071130 : Fast Cache data not found

It appears when running winget list/winget upgrade

Some sort of error handling in WAU (Get-WingetOutdatedApps.ps1) would be nice so that the error doesn't go unnoticed!

The only way I spotted it was because the logs didn't look as usual:

A normal run for me includes listing of the blacklisted apps:

#245


docker handler

Added missing handler lines to restart docker when we make iptables.rule changes.

Shouldn't restart sshd be in this file too?

I'm pretty inexperienced with git and GitHub, but it looks like restart sshd is actually already there. Is the issue just that the interface doesn't show us lines 1 and 2 during the diff?
https://github.com/sophware/realworlddevopscourse/blob/docker3-handlers-patch/after-20-docker-03/ansible/security/handlers/main.yml

You are correct, my bad.
I'll merge and add this change to all future files too.
Thanks again for your effort!


fancontrol.py error after updating to v1.1b1

Stratux version: v1.1b1

Stratux config:
SDR

[ ] single
[X ] dual

GPS

[X ] yes
[ ] no
type: vk162

AHRS

[ ] yes
[ ] no

power source:
usb cable:

EFB app and version: (e.g., WingX Pro7 8.6.2)
EFB platform: (e.g., iOS 9.2)
EFB hardware: (e.g., iPad Mini 2)

Description of your issue:

Hi, I wanted to check out the gpsattitude so I uploaded the appropriate sh file. However, I noticed when I did that, my fan wouldn't turn on (I'm running the fancontrol.py script at boot). I tried manually running the script in terminal and I'm getting:
"File "fancontrol.py", line 12, in  from daemon import runner
ImportError: No module named daemon"
So I figured I'd try the beta update, and nothing changed. Any help is appreciated.
Thanks
If possible, enable "Replay Logs", reproduce the problem, and provide a copy of the logs in http://<IP_ADDRESS>/logs/stratux/ and http://<IP_ADDRESS>/logs/stratux.log.

check your reddit post for info
https://www.reddit.com/r/stratux/comments/5j0njz/updated_to_v11b1/

also
https://github.com/cyoung/stratux/issues/543

Little did I know that the fancontrol.py script was already added to run at boot, so I wasn't expecting it to run at all when I first booted. I was expecting to have to add it myself to crontab. However, I was looking at the history of fancontrol.py and noticed the called out GPIO pin was changed from 11 to 12 for some reason. Once I swapped pins, the fan started right up.

So you can close this issue then?


Add check for files listed in vendor.yml, fix location of Chart.min.js

At now the build system does not check whether vendor files listed in vendor.yml actually exist. This PR adds a check before populating src.jsVendor, src.cssVendor or src.angularVendor. If any of the dependencies is missing, the build fails.
Note that the vendor task is moved from the bottom of the gruntfile and placed before the project configuration. The real modification is that lines 64-68 are added.
This log shows the (errored) output when adding the check (first commit): https://travis-ci.org/1138-4EB/portainer/builds/303310565 (line 4887)
The second commit removes a duplicated incorrect location for Chart.min.js. See vendor.yml#L27. The build is correct: https://travis-ci.org/1138-4EB/portainer/builds/303315058
Close #1410

Done. Do you use any specific "beautification" tools/settings?

I'm using https://atom.io/packages/atom-beautify inside Atom.

Done and checked.

@1138-4EB mind opening an associated issue so we can link it?

Associated issue: #1410

Thanks @1138-4EB :+1: