feat(client): consolidate excessive amount of notes in a given tier

In Fedi's LN gateway deployement, we've hit a condition where LN gateway accumulated tons of tiny notes and at some point is trying to spend all of them at once, creating tx larger than consensus limit.
Fundamentally, when an client like LN gateway is handling a lot of tiny payements, it will keep dealing with and accumulating tiny note denominations. The change outputs note selection algorithm tries to create outputs so that user has at least certain amount of notes available at every tier, but does nothing to get rid of excessive ones.
At some point it has to consolidate them.
This PR hooks up into the tx finalization, to potentially add some excessive notes to each transaction.
There are multiple ways to approach this problem, some might be more elegant or smarter, but this solution is simple and should deal with the issue without changing normal use-cases.
Testing
Some automated test would be better, but RN I tested by manually lowering the tresholds and in devimint env running:
export RUST_LOG=fm::client::module::mint=debug,info
fedimint-cli info
fedimint-cli ln-invoice --amount 4msat
etc.

when the treshold of given denomination is reached:
2024-04-11T01:37:21.106950Z DEBUG fm::client::module::mint: Will consolidate excessive notes note_num=16 denominations_msats=[2, 2, 128, 128, 512, 512, 1024, 10
24, 4096, 4096, 32768, 32768, 524288, 524288, 1048576, 1048576]

Testing this is tricky, because it requires creating unbalanced notes, while the change selector keeps recreating notes to fill up the tiers.
Re #4913

After thinking about it more, it seems to me that making note selector spend excessive notes as a priority would be much better approach, but that seems like much more involved change, and I created #4923 on how I'd approach it.

I just can't help but keep refactoring...

I'm inclined to CONCEPT NACK this. I don't think "excess notes" was or will ever be an issue. The issue we had and will continue to have is a matter of a lack of notes (in that specific incident, a lack of mid-sized notes). So we should be talking about shortage of notes and filling these gaps. While filling these gaps we may or may not consume "excess notes". Probably we want spend high-tier notes to fill mid-tier ones.

I'm inclined to NACK this. I don't think "excess notes" was or will ever be an issue. The issue we had and will continue to have is a matter of a lack of notes (in that specific incident, a lack of mid-sized ones). So we should be talking about shortage of notes and filling these gaps. While filling these gaps we may or may not consume "excess notes". Probably we want spend high-tier notes to fill mid-tier ones.

"Lack of high-tier notes" is the same thing as "having too many low-tier notes". If we have 100s of low-tier notes, and consolidate them, we'll get the extra higher-tier notes we were lacking.

Probably we want spend high-tier notes to fill mid-tier ones.

If we lack a mid-tier note we can always spend a high-tier one and get a change. Also low-tier notes first consolidate into mid-tier, mid-tier into high-tier.
The goal is to generally have a flatter distribution altogether, so it's always possible to spend everything at once.
Though again - this is a quick provisional solution.

I'm inclined to NACK this. I don't think "excess notes" was or will ever be an issue. The issue we had and will continue to have is a matter of a lack of notes (in that specific incident, a lack of mid-sized ones). So we should be talking about shortage of notes and filling these gaps. While filling these gaps we may or may not consume "excess notes". Probably we want spend high-tier notes to fill mid-tier ones.

"Lack of high-tier notes" is the same thing as "having too many low-tier notes". If we have 100s of low-tier notes, and consolidate them, we'll get the extra higher-tier notes we were lacking.

lower tier notes won't give too much yield because of their small value. the way of solving lack of high tier notes spending even higher notes.

Probably we want spend high-tier notes to fill mid-tier ones.

If we lack a mid-tier note we can always spend a high-tier one and get a change. Also low-tier notes first consolidate into mid-tier, mid-tier into high-tier.

Lower tier notes are more "useful" (i.e they can always be used). Proactively spending them may be harmful. I think any consolidation algorithm should focus on spending higher tiers first.

The goal is to generally have a flatter distribution altogether, so it's always possible to spend everything at once.
A pyramid shape  is perfectly ok supposing we don't have missing tiers on the middle.

Though again - this is a quick provisional solution.
@douglaz 👍 ❓

Ok, this may help in some cases, so it's a valid quick-solution
My guess is that it would not be enough to avoid that particular incident and will create a slightly worse note distribution in some extraordinary circumstances.
Anyway it's probably better than we have today.

Lower tier notes won't give too much yield because of their small value. the way of solving the lack of high tier notes is by spending even higher notes.

The change consolidates all notes all the same. Notes will continuously consolidate into higher ones, higher into even higher.

Lower tier notes are more "useful" (i.e they can always be used if within the transaction limit). Proactively spending them may be harmful. I think any consolidation algorithm should focus on spending higher tiers first.

I don't understand. As implemented RN, consolidation only ever spends notes that you already have plenty of.
Maybe we can discuss the whole thing in a Deep Dive. Added to https://github.com/fedimint/fedimint/wiki/Deep-Dive-Topics

Commits with overlapping changes were pilling up, so just decided to squash it all.
Notably I changed the algo. so it first detects if any tier is above a MAX_TRIGGER and then consolidates all notes above MIN per tier. This should lead to less churn.
I need to test it again manually, and maybe think about a good test of some kinid, so will not merge just yet.

OK, seems to work.
I'd really love a test for that. Need to find some good way to test though.

Provide some initial funds to a client, then create transactions generating uneven notes (see how fedimint-load-test-tool does that) then check if a normal spending/reissue will rebalance the notes.

I would love a explicit "rebalance" function that I can run whenever client is online, so in future selection is better when offline

dev call: needs testing

Test should be ready.

 00:00:41 assertion failed: epochs_descriptors.contains(*utxo_descriptor)

Known problem, re-added to MQ.

Backport failed for releases/v0.3, because it was unable to cherry-pick the commit(s).
Please cherry-pick the changes locally and resolve any conflicts.
git fetch origin releases/v0.3
git worktree add -d .worktree/backport-4922-to-releases/v0.3 origin/releases/v0.3
cd .worktree/backport-4922-to-releases/v0.3
git switch --create backport-4922-to-releases/v0.3
git cherry-pick -x 966b8c4e79e871705e7f1da229d5b78cfa48088a c13c65cf22e7a91df9d70155497d0759837704a1 1b76381186ba595a1f3b4416948a78093ad7166e


feat: add reverse and toReversed methods to array/bool

Resolves: Subtask of #2304
Description

What is the purpose of this pull request?

This pull request:

Adds the typed array reverse and toReversed methods to prototype of BooleanArray

Related Issues

Does this pull request have any related issues?

This pull request:
Questions

Any questions for reviewers of this pull request?

No.
Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

No.
Checklist

Please ensure the following tasks are completed before submitting this pull request.

[x] Read, understood, and followed the contributing guidelines.

@stdlib-js/reviewers

Tests are currently failing due to a circular dependency.
@stdlib/array/bool =>
@stdlib/array/base/reverse =>
@stdlib/array/base/arraylike2object =>
@stdlib/array/dtype =>
@stdlib/array/bool

Looks like we need to revert the changes to use @stdlib/array/base/reverse and just perform manual iteration.

According to compat tables (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/reverse), typed arrays did not get a reverse method until Node.js v4. As we still support back to v0.10, we cannot rely on the built-in method, as it may not be present in older Node.js and browser versions. Hence, manual iteration is our only portable option.


supports double prompt

this is on my Mac. looks like the measuring maybe be off a few chars.

Linux isn't quite there yet:

here's the latest commit on Mac.

did some estimation instead of reading the cursor position directly: https://github.com/nushell/reedline/tree/double-prompt

Merged as part of #241


Develop AiiDAlab: AiiDAlab App Registry

App registry and all one needs to know about it.

the maintainer docs now explain how to point to a different registry.
I guess one can still move a couple of things from the README of the aiidalab-registry repo to the developer docs


How to handle soft-deletes

I'm not sure how to handle soft-deletes yet, so would love some ideas on this.

I think for soft-deletes it should handle as a regular delete. The item would be removed from lists on the page. But I'm not sure what we should do about the instances where the resource is on the screen (like showing a post and the post was deleted). Should we replace the post frame with a new render of the post partial? Or is that too specific for the library and should be handled in the app instead?

I think for soft-deletes it should handle as a regular delete. The item would be removed from lists on the page. But I'm not sure what we should do about the instances where the resource is on the screen (like showing a post and the post was deleted). Should we replace the post frame with a new render of the post partial? Or is that too specific for the library and should be handled in the app instead?

I think this concern is also for regular deletes. Should we replace the element with a new render of the frame for regular deletes and tell any list to remove it from there? I think this is a nice solution. 🤔

I think this concern is also for regular deletes. Should we replace the element with a new render of the frame for regular deletes and tell any list to remove it from there? I think this is a nice solution. 🤔

Yeah I would say there isn't too much difference between the two. It all depends on the use case. Sometimes, you want to 'grey-out' an item when it gets deleted, sometimes you want to completely remove it from the list.
Thus, one should have the option what to do on a (soft) delete: update or actually remove the item.

Yeah I would say there isn't too much difference between the two. It all depends on the use case. Sometimes, you want to 'grey-out' an item when it gets deleted, sometimes you want to completely remove it from the list.
Thus, one should have the option what to do on a (soft) delete: update or actually remove the item.

yeah, I was just thinking about what the default option should be. After sleeping on it a bit, I think we should handle it as a regular delete (remove from lists). If we need more control, we can override the "deleted_stream.blade.php" template, I guess.

yeah, I was just thinking about what the default option should be. After sleeping on it a bit, I think we should handle it as a regular delete (remove from lists). If we need more control, we can override the "deleted_stream.blade.php" template, I guess.

We'll handle soft-deletes as regular deletes. For applications that need more fidelity on soft-deletes, it's recommended to implement your customization on the turbo/deleted_stream.blade.php template.

Hi @tonysm,
Sorry to comment on this closed issue, but I don't think soft-deletes are handle as regular deletes right now.
According to this line: https://github.com/tonysm/turbo-laravel/blob/0.3.2/src/TurboStreamResponseMacro.php#L19
You check if the model exists, but a soft deleted model response to true on this exists attribute.
Maybe, you can do something like this?

        if (! $model->exists ||
            (method_exists($model, 'trashed') && $model->trashed())
        ) {
            return $this->renderModelDeletedStream($model);
        }

Have a good day,
Tortue Torche


(595) Set up HMPPS Audit for Refer

Context

https://trello.com/c/PfPffKbd/595-set-up-hmpps-audit-for-refer

As per this ADR, there's a requirement for services at HMPPS to send audit events of:

All updates to data
All views of personal data

And the following details:

User identifier - if in doubt, username or email address
Date and time - to at least the second level, ideally to millisecond
Data entity identifier - usually the most sensible ID for the Person record being viewed - CRN, Prison Number or something else
What is being viewed - not too fine-grained - “person risks” is appropriate, a list of each of the risks would not be. We think storing the page name might work, and if there’s a tab structure within pages, storing the tab name may also be appropriate. It is deliberate that we are talking about front-end interactions here - “what was the user doing?” rather than “what was the system doing to support the user’s request?”.

Changes in this PR
This adds auditing events to the routes in the Refer journey, sending them to HMPPS Audit alongside any important information like the prisonNumber of the person being searched for. This is handled by new audit middleware.
I've followed the approach taken by the CAS teams here - the code is fairly complex and could possibly simplified but in the interests of time I've kept things fairly as-is.
We need to set up some infrastructure on the Cloud Platform environment here to allow this to work.
Screenshots of UI changes
Examples of auditing logged in the terminal

Release checklist
Release process documentation
As part of our continuous deployment strategy we must ensure that this work is
ready to be released once merged.
Pre-merge

[ ] There are changes required to the HMPPS Cloud Platform environments repository for this change to work...

[ ] ... and they have been released to production already

Post-merge

[ ] This adds, extends or meaningfully modifies a feature...

[ ] ... and I have written or updated an end-to-end test for the happy path in the Accredited Programmes E2E repo

[ ] This makes new expectations of the API...

[ ] ... and I have notified the API developer(s) of changes to the contract tests (Pact), or the API is already compliant

[ ] Manually approve release to preprod
[ ] Manually approve release to prod

Closing for now until we're ready and in a position to pick this up again.


unique validation doesn't work

Hi there,
I have following User model:
module.exports = _.merge(_.cloneDeep(require('../base/Model')), {
attributes: {
	username: {
	  type: 'string',
	  unique: true
	},
	email: {
	  type: 'email',
	  unique: true
	},
	firstName: {
	  type: 'string',
	  required: true
	},
	lastName: {
	  type: 'string',
	  required: true
	},
	admin: {
	  type: 'boolean',
	  defaultsTo: false
	},

	// Below is all specification for relations to another models

	// Passport configurations
	passports: {
	  collection: 'Passport',
	  via: 'user'
	},
	// Login objects that are attached to user
	logins: {
	  collection: 'UserLogin',
	  via: 'user'
	},
	requestLogs: {
	  collection: 'RequestLog',
	  via: 'user'
	}
}
});

According to the above model declaration, I would expect when attempting to create a new user with already existing username and / or email to have exception returned. Instead I am able to successfully create as many users with identical usernames and passwords as I would like.
I am using MongoDB and  sails-mongo, if that matters.
Best Regards,
Angel

+1 @angel1st - I have exactly the same problem and it's preventing me from going live.

@gilesbutler - I could suggest you workaround solution I use by myself - you could define unique index within the database, so you could implement this constraint on DB level. Of course this is not the best solution, but it works at least.
BTW - I am quite confusing that during the last couple of weeks feedback from the Sails supporting team is almost missing... hope this is just and accident, and they will back on track again soon.

Thanks @angel1st - I tried this but the user doesn't get back an errors that are friendly so its a very bad user experience as the page just reloads and they don't know what's happened, maybe that's just my setup though.
There is an open issue for this in the sails-mongo repo anyway, hopefully they are working on a fix.
I'm not sure about the feedback from the Sails support team, maybe @tjwebb can shed some light on that one. I noticed the needs failing test label, @tjwebb is there anything we can do to help?

@gilesbutler - I would suggest another workaround - you can check for uniqueness before adding new record, using Model.find() or Model.findOne() and if you get result(s), you can process the exception in more user friendly manner... hope this helps.

@angel1st thanks for that, I thought that too. Is it a big performance hit to be hitting the DB twice though, once to check and then a second time to add a new record if none exists?

@gilesbutler - well it depends of DB size, record / document size, but in general since this is just a read request for specific DB column and since the lack of other alternatives, I would suggest you to go for it and re-work your code at later stage.

Thanks @angel1st - good to know and seems like the best solution for now! Thanks for your help.

Just dropping in to check if this issue is still being worked on?
This issue has occurred with multiple versions of waterline and sails. This should be added to the test list.

Same issue here!. Any news?

Guys, try this:
http://stackoverflow.com/questions/22868420/how-do-i-handle-a-unique-field-in-sails
It works like a charm now!


Implement performance test checking non-abi change in buildSrc

This PR adds performance test coverage for the use-case when the build sources change but the abi stays intact.

Why do we have this test run in the context of largeJavaMultiProject? IIUC this test only checks the build source invalidation performance, which seems independent of the project size. I think we could run the test only against smallJavaMultiProject, which would save some precious CI build time. WDYT? /cc @rieske @wolfs.


indilib: support all linux architectures

Motivation for this change
Support building indilib on non x86_64 architectures and make minor style/format changes to indilib.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 118696 run on x86_64-linux 1

5 packages built:

indi-3rdparty
indi-full
indilib
kstars
phd2

Result of nixpkgs-review pr 118696 at d18d31cb run on aarch64-linux 1

1 package marked as broken and skipped:

kstars

3 packages built successfully:

indi-3rdparty
indi-full
indilib

@ofborg eval


Using connection to Emulator for SAS token

The Powershell example connects to an Azure Storage container while this page focuses on the Azure Storage Emulator; updated the example to connect to the Local Storage Emulator.

@ppluijten : Thanks for your contribution! The author, @tamram, has been notified to review your proposed change.

@ppluijten - Thanks for making that update.
#sign-off


feat(pixels): add my new pixel (39, 31)

Checklist

[x] I ran npm test locally and it passed without errors.

[x] I only edited the _data/pixels.json file.

[x] I entered the username in the pixels.json that I'm also using to create this pull request.
[x] I acknowledge that all my contributions will be made under the project's license.

Request to please approve workflow and review.


Fix function/proc def regression in sql_modules view

Description
Previously, the function definition for plpgsql was modified so that it
uses tsql_get_functiondef() instead of pg_get_functiondef(). However,
this causes regression as all catalog procs have no definition. It is
preferred to have the plpgsql function definition over no definition at
all. This commit fixes the issue by displaying the definition of these catalog
procs and functions.
Additionally, this test also reverts the sys-modules side of #678 since the duplicated object id was fixed in #693
Task: BABELFISH-591
Signed-off-by: Favian (Ian) Samatha<EMAIL_ADDRESS>Issues Resolved
Task: BABELFISH-591
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

As mentioned in the PR description, this re-enables several tests that were disabled/commented out in https://github.com/babelfish-for-postgresql/babelfish_extensions/pull/678 since the duplicated object id was fixed in https://github.com/babelfish-for-postgresql/babelfish_extensions/pull/693.
This PR also deletes the sys-sql_modules, sys-system_sql_modules, and sys-all_sql_modules tests as they are replaced with the vu tests.


Disable windows CI tests for kogito-serverless-operator

https://github.com/apache/incubator-kie-issues/issues/1227

@tiagobento yey, it's green


Installing with postgresql & redis enabled with password is broken in the 2.1.0

Installing a release with postgresql & redis enabled that are not using pre-existing secrets causes nautobot pod to fail with Error: secret "release-name-nautobot-postgresql" not found .
Meaning I have a values file like:
postgresql:
  enabled: true
  auth:
    password: "nautobot"
redis:
  enabled: true
  auth:
    password: "nautobot"

but the Nautobot pods are failing that they cannot find those secrets.
The reason being that the names of the secrets and the ones that the deployment is configured to use are different.
Specifically, after some tshoot the pods are configured to use fullname-nautobot-postgresql & nautobot-redis, but the secrets have names fullname-postgresql & fullname-redis respectively.

Hi @gertzakis
I've just deployed a fresh installation onto an empty minikube cluster with your values and the current chart 2.1.0, which works just fine for me.
Can you send some more context on how to reproduce this?

Hey, yes the release should have a different name than nautobot. For example run helm install awesome-release nautobot/nautobot --set postgresql.auth.password="change-me" --set redis.auth.password="change-me" .
My mistake for not mentioning in the issue in the first place.

Good catch @gertzakis, totally missed that.
I've opened an additional PR that adresses this issue.


DLS-6527: added exemptions for small producers page

build is failing and please incorporate the changes suggested by bot above. Your branch is also behind main , please rebase it first. Let me know if you need any help

aproved


Import error in algorithms.py with 3.6.1

I have python 3.6.1. and jwt 0.5.1
I'm getting an import error. I don't know why.
File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 72, in get
   return request('get', url, params=params, **kwargs)
 File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 58, in request
   return session.request(method=method, url=url, **kwargs)
 File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 504, in request
   prep = self.prepare_request(req)
 File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 436, in prepare_request
   hooks=merge_hooks(request.hooks, self.hooks),
 File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 306, in prepare
   self.prepare_auth(auth, url)
 File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 543, in prepare_auth
   r = auth(self)
 File "/usr/local/lib/python3.6/dist-packages/requests_oauthlib/oauth1_auth.py", line 88, in __call__
   unicode(r.url), unicode(r.method), None, r.headers)
 File "/usr/local/lib/python3.6/dist-packages/oauthlib/oauth1/rfc5849/__init__.py", line 314, in sign
   ('oauth_signature', self.get_oauth_signature(request)))
 File "/usr/local/lib/python3.6/dist-packages/oauthlib/oauth1/rfc5849/__init__.py", line 151, in get_oauth_signature
   sig = self.SIGNATURE_METHODS[self.signature_method](base_string, self)
 File "/usr/local/lib/python3.6/dist-packages/oauthlib/oauth1/rfc5849/signature.py", line 505, in sign_rsa_sha1_with_client
   return sign_rsa_sha1(base_string, client.rsa_key)
 File "/usr/local/lib/python3.6/dist-packages/oauthlib/oauth1/rfc5849/signature.py", line 496, in sign_rsa_sha1
   alg = _jwt_rs1_signing_algorithm()
 File "/usr/local/lib/python3.6/dist-packages/oauthlib/oauth1/rfc5849/signature.py", line 473, in _jwt_rs1_signing_algorithm
   import jwt.algorithms as jwtalgo
 File "/usr/local/lib/python3.6/dist-packages/jwt/algorithms.py", line 5, in <module>
   from .exceptions import InvalidKeyError
ImportError: cannot import name 'InvalidKeyError'

This problem seems to be not of GehirnInc/jwt but of jpadilla/pyjwt.
According to the trace back you pasted above, jwt which was imported byoauthlib raised ImportError exception.  oauthlib does not depend on my jwt package, it depends on jpadilla/pyjwt instead. https://github.com/idan/oauthlib/blob/91152df142bdde134d84ed27963cda8e6b878416/setup.py#L20-L23
Would you provide a list of packages you've installed with pip freeze ?

I have solved this problem uninstalling "jwt" package and reinstalling "pyjwt" package. Both have the same name. Thanks @yosida95


Adding the support for literal strings (without multiline strings)

Will fix diktat issues in the next PR


visual studio code - insiders search not working due to not indexing files in project folder

Issue Type: Bug
all i've had to do is open a project folder in visual studio code insiders and try to search for a file using command T
at which point it just hangs despite knowing files with the search for key word exist
VS Code version: Code - Insiders 1.28.0-insider (b4d09d27509f2c023f63d4315aa966a5ec7581e9, 2018-09-27T14:37:17.724Z)
OS version: Darwin x64 16.7.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (8 x 2800)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
3, 3, 3

Memory (System)
16.00GB (0.30GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (18)

Extension
Author (truncated)
Version

npm-intellisense
chr
1.3.0

path-intellisense
chr
1.4.2

bracket-pair-colorizer
Coe
1.0.60

vscode-eslint
dba
1.6.0

vscode-npm-script
eg2
0.3.5

vscode-great-icons
emm
2.1.42

prettier-vscode
esb
1.6.1

auto-close-tag
for
0.5.6

path-autocomplete
ion
1.12.0

docthis
joe
0.7.1

vscode-subword-navigation
ow
1.2.0

material-icon-theme
PKi
3.6.0

vscode-icons
rob
7.27.0

stylelint
shi
0.43.0

code-spell-checker
str
1.6.10

vscode-todo-highlight
way
1.0.4

JavaScriptSnippets
xab
1.7.0

html-css-class-completion
Zig
1.17.1

(1 theme extensions excluded)

cmd+T search symbols by default. Are you searching symbols or files? And can you be more clear what you mean by "hangs"?


CVE-2022-38900 (npm/decode-uri-component) found on v14.x dependancy

@RafaelGSS
This public CVE is reported against decode-uri-component, which is a dependency of the npm in NodeJs 14.x.
Vulnerability ID: CVE-2022-38900
Vulnerability URL: https://nvd.nist.gov/vuln/detail/CVE-2022-38900
found in node-v14.21.2-linux-x64-musl.tar.xz (lib\node_modules\npm\node_modules\decode-uri-component)
src: node/deps/npm/node_modules/decode-uri-component/

@siemenstan do you know how/if it affects npm as that will depend on what APIs are used from decode-uri-component right?

@mhdawson No, my app doesn't use the decode-uri-component. It's just my company security scan system is picking up this public CVE from my app container image with the node 14.x image

@nodejs/npm could you check if that affect Node.js itself?

https://github.com/nodejs/node/pull/45936 addresses this

Believe this was addressed by recent security release, closing


144 Two step announcement

Pull Request Test Coverage Report for Build 477

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 91.638%

Totals

Change from base Build 470:
0.0%

Covered Lines:
374

Relevant Lines:
410

💛  - Coveralls

Thanks @NemTechCN for the solution! I have added a commit on top applying it to all the transaction-related commands and showing the complete payload.
@evias please review before merging 👍


Duplicated content

Hey.
I have configured the whole project, but when I run in my local "npm run dev", the index shows all the post published but duplicated, I mean I see the preview of the post duplicates, how could I fix that ?.
And when I add a new post, I can not see it in my index when I refresh the page, I can only see it when I restart the command "npm run dev".
Thanks in advance

It was just the double language configuration in the contentful app.


SIL.rst: use terms 'object type' and 'address type'

Use the terms used in compiler source when introducing these concepts in the documentation.

@swift-ci smoke test

⛵


Not able to create shared memory segment for PID

Some times it happens..
PHP Fatal error:  Uncaught exception 'Spork\Exception\ProcessControlException' with message 'Not able to create shared memory segment for PID: 20786' in /var/www/link.ru/vendor/kriswallsmith/spork/src/Spork/SharedMemory.php:91
Stack trace:
#0 /var/www/link.ru/vendor/kriswallsmith/spork/src/Spork/ProcessManager.php(119): Spork\SharedMemory->send(Object(Spork\Util\ExitMessage), false)
#1 [internal function]: Spork\ProcessManager->Spork\{closure}()
#2 {main}
  thrown in /var/www/link.ru/vendor/kriswallsmith/spork/src/Spork/SharedMemory.php on line 91
PHP Fatal error:  Uncaught exception 'Spork\Exception\ProcessControlException' with message 'Not able to create shared memory segment for PID: 20786' in /var/www/link.ru/vendor/kriswallsmith/spork/src/Spork/SharedMemory.php:91
Stack trace:
#0 /var/www/link.ru/vendor/kriswallsmith/spork/src/Spork/ProcessManager.php(119): Spork\SharedMemory->send(Object(Spork\Util\ExitMessage), false)
#1 [internal function]: Spork\ProcessManager->Spork\{closure}()
#2 {main}
  thrown in /var/www/link.ru/vendor/kriswallsmith/spork/src/Spork/SharedMemory.php on line 91

Related to this update
https://github.com/kriswallsmith/spork/pull/28

I'm getting this all the time. Any idea on what's causing this? I see shmop is enabled in phpinfo.

Might be related. Fixed similar shared memory errors on my fork.
https://github.com/TheLevti/spork/issues/7
https://github.com/TheLevti/spork/issues/9


Where to find the BXARM.deb ?

I am trying to follow up the tutorial but I cannot find where it is possible to download the IAR Build Tools installer packages .deb. How can I get it?

Hi @rertyStrike, Thanks for pointing the missing information. Our site has been updated recently so the sales contact link became broken. I just updated it now so you can navigate from the Requirements section on the tutorial and click on the IAR build tools for Arm 8.50.6 link from that list.


MySQL CDC supports parsing pt-osc generated schema changes

MySQL CDC supports parsing pt-osc generated schema changes

@yuxiqian I have completed the rest of the work on pt-osc. The main changes are as follows. Can you help review it?

There is no identifier like /* gh-ost*/ in the change record of pt-osc, so I modified some logic
Both pt-osc and gh-ost belong to the Online Schema Change tool, so I renamed some of them: XXXGhOstSchemaChanges -> XXXOnLineSchemaChanges

Thanks for helping on this!


Add alternative names for cursor icons

Many cursor themes in the wild don't adhere to the w3c standard, and only use legacy names.
The list in this PR is from a list in an older version of winit, keeping only the names that don't match the standard.
Also see: https://github.com/Smithay/client-toolkit/issues/425
cc @kchibisov

Could you check what Qt and GTK are using as well?

Could you check what Qt and GTK are using as well?

Good call. Qt especially had a lot of alternatives.
How close should we stay to the original cursor? Let's for example look at CursorIcon::Copy, which is described as "Indicates something is to be copied. Often rendered as an arrow with a small plus sign next to it.". My cursor theme does not have that exact icon, but it does have a drag&drop variant of it (referred to as dnd-copy) where instead of the arrow, there is a dragging hand. Should this be listed or should we stay as close to the original as possible? My current version opts to not list it.

How close should we stay to the original cursor? Let's for example look at CursorIcon::Copy, which is described as "Indicates something is to be copied. Often rendered as an arrow with a small plus sign next to it.". My cursor theme does not have that exact icon, but it does have a drag&drop variant of it (referred to as dnd-copy) where instead of the arrow, there is a dragging hand. Should this be listed or should we stay as close to the original as possible? My current version opts to not list it.

No, if it doesn't have it then it doesn't have it.

Thanks.


Broken Traces for ESM App

What happened?
Hello Everyone,
I am trying to instrument my JS app running on AWS Lambda, but it does not seem to work as expected since our app use ESM type of import and end up with broken traces with different traceIDs and not detecting X-RAY header from the lambda autodetect.
Steps to Reproduce
otel.js:
having the initialized Otel SDK configuration
app.mjs:
import "./otel.js"
Expected Result
3 spans with same traceid of X-RAY
Actual Result
2 traces with different context
Additional Details
OpenTelemetry Setup Code
// otel.js

import { SimpleSpanProcessor, ConsoleSpanExporter } from "@opentelemetry/sdk-trace-base";
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { AwsLambdaInstrumentation } from '@opentelemetry/instrumentation-aws-lambda';
import { registerInstrumentations } from '@opentelemetry/instrumentation';
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";

import { AWSXRayIdGenerator } from '@opentelemetry/id-generator-aws-xray';

  const provider = new NodeTracerProvider({idGenerator: new AWSXRayIdGenerator()});
  provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));

  // Initialize the provider
  provider.register();

  // Registering instrumentations / plugins
  registerInstrumentations({
    instrumentations: [
      getNodeAutoInstrumentations(),
      new AwsLambdaInstrumentation({
        disableAwsContextPropagation: false
      })
    ],
   });

package.json
{
  "name": "http-dynamo-tutorial",
  "version": "1.0.0",
  "description": "",
  "main": "app.mjs",
  "type": "module",
  "dependencies": {
    "@opentelemetry/api": "1.0.2",
    "@opentelemetry/instrumentation": "0.25.0",
    "@opentelemetry/auto-instrumentations-node": "0.25.0",
    "@opentelemetry/instrumentation-aws-lambda": "0.25.0",
    "@opentelemetry/instrumentation-http": "0.25.0",
    "@opentelemetry/sdk-trace-base": "0.25.0",
    "@opentelemetry/sdk-trace-node": "0.25.0",
    "@opentelemetry/id-generator-aws-xray": "1.1.1",
    "@opentelemetry/exporter-trace-otlp-http": "0.35.0",
    "@opentelemetry/sdk-node": "^0.33.0",
    "@opentelemetry/core": "1.9.0",
    "axios": "^0.21.1"
  },
  "devDependencies": {},
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
 }

Relevant log output
No response

Not a bug, ESM is not yet supported.
See https://github.com/open-telemetry/opentelemetry-js/issues/1946 for details


6317: Adjust colors of the Status on AP

https://dev.azure.com/avask-software/infiniti/_workitems/edit/6317

Closing, will go with primereact pt= customization in fabled-kit


Modernisation Platform Module Review

User Story
As a modernisation platform engineer,
I want to only maintain modules we offer that are useful, and actually in use, and edit / improve / remove ones which are not
So that we can focus our time on what customers need / want.
Value / Purpose
Currently our modules are either heavily used, or not at all. We maintain tests on modules which are never used, and keep them up to date, is this time wasted?
I think we need to review the modules that we have, work out if they are being used, and the ones that are not, do we keep, change, improve, remove etc.
Useful Contacts
Edd Proctor
Additional Information
No response
Proposal / Unknowns

Module / Repo
Link

Main Repo
~~https://github.com/ministryofjustice/modernisation-platform~~

Environments Repo
~~https://github.com/ministryofjustice/modernisation-platform-environments~~

Bastion Linux
https://github.com/ministryofjustice/modernisation-platform-terraform-bastion-linux

Configuration Management
https://github.com/ministryofjustice/modernisation-platform-configuration-management

Instance Scheduler
https://github.com/ministryofjustice/modernisation-platform-instance-scheduler

S3 Bucket
https://github.com/ministryofjustice/modernisation-platform-terraform-s3-bucket

AWS VM Import
https://github.com/ministryofjustice/modernisation-platform-terraform-aws-vm-import

SSM Patching
https://github.com/ministryofjustice/modernisation-platform-terraform-ssm-patching

EC2 Autoscaling
https://github.com/ministryofjustice/modernisation-platform-terraform-ec2-autoscaling-group

AMI Builds
https://github.com/ministryofjustice/modernisation-platform-ami-builds

Terraform Baselines
https://github.com/ministryofjustice/modernisation-platform-terraform-baselines

Cross Account Access
https://github.com/ministryofjustice/modernisation-platform-terraform-cross-account-access

DNS Certificates
https://github.com/ministryofjustice/modernisation-platform-terraform-dns-certificates

Load Balancer
https://github.com/ministryofjustice/modernisation-platform-terraform-loadbalancer

Github OIDC Role
https://github.com/ministryofjustice/modernisation-platform-github-oidc-role

Terraform Member VPC
https://github.com/ministryofjustice/modernisation-platform-terraform-member-vpc

Terraform IAM Superadmins
https://github.com/ministryofjustice/modernisation-platform-terraform-iam-superadmins

S3 Bucket Replication
https://github.com/ministryofjustice/modernisation-platform-terraform-s3-bucket-replication-role

EC2 Instance
https://github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance

Github OIDC Provider
https://github.com/ministryofjustice/modernisation-platform-github-oidc-provider

ECS Cluster
https://github.com/ministryofjustice/modernisation-platform-terraform-ecs-cluster

Pagerduty
https://github.com/ministryofjustice/modernisation-platform-terraform-pagerduty-integration

Terraform Module Template
https://github.com/ministryofjustice/modernisation-platform-terraform-module-template

Incident Response
https://github.com/ministryofjustice/modernisation-platform-incident-response

Trusted Advisor
https://github.com/ministryofjustice/modernisation-platform-terraform-trusted-advisor

Lambda Function
https://github.com/ministryofjustice/modernisation-platform-terraform-lambda-function

Definition of Done

[ ] Module usage has been checked.
[ ] Discussion surrounding unused modules.
[ ] Team agreement.
[ ] Modules are removed, or tickets raised to change them.

Terraform Module Template | https://github.com/ministryofjustice/modernisation-platform-terraform-module-template - removed this one


How to trigger custom NotificationHandler when user is offline ?

If I want to use third party push service like JPush or GeTui, I think I need to implement an custom NotificationHandler for Gateway, when user is online, the custom NotificationHandler code works well,  but I found that when user is offline, my custom NotificationHandler couldn't be triggered.
It seems that when user is offline, OutboundMessageService#forwardClientMessageByRecipientId get activeNodeIds is empty, and
return PublisherPool.emptySet(),  it doesn't forward message to GateWay Nodes, and my custom NotificationHandler couldn't be triggered, as follows:

But I am not sure if I have some misunderstandings about the code or it is a bug indeed.
Any help is appreciated

Your understanding of the code is correct, but you need to implement im.turms.service.infra.plugin.extension.RequestHandlerResultHandler instead of im.turms.gateway.infra.plugin.extension.NotificationHandler.
Turms has a  plugin named turms-plugin-push for push notifications, and you can reference it to implement your own plugin.

But I recommend you add support for JPush and GeTui in turms-plugin-push instead of developing a new one so that we can maintain them together and make them robust (e.g. save failed push notifications on Redis when the current node goes down, and other nodes can fetch these failed push notifications to help retry sending failed push notifications to users).
And if you also prefer this solution, you can reference im.turms.plugin.push.core.sender.fcm.FcmSender to implement your own JPushSender and GeTuiSender, and I will polish your code once you give a PR, so you don't need to worry too much on whether your code works fine with Turms.

Or you can also wait for me to implement them for you.
But I am currently busy with developing turms-client-cpp (a C++-based client SDK), I can only start developing them from scratch next week or the week after next.

Thank you for your explanation.
I will start working on understanding the code implementation in turms-plugin-push and  try to reference FcmSender to create the JPushSender implementation.
Since I am also studying turms in my spare time, I am not certain if I will be able to provide my implementation quickly. Moreover, this feature is not urgent for me. Therefore, please feel free to decide whether to implement this functionality based on your time schedule.
If I manage to have time to implement it before you do, I would be very happy to submit a pull request to contribute the code,
even though my implementation might not be professional enough in Reactive Programming.
Once again, thank you for your assistance.

Hello, I have started learning the code of turms-plugin-push, but I have some confusion.

I read the logic in the im.turms.plugin.push.NotificationPusher#afterNotify method. My thinking is that when a user goes offline, the corresponding user ID cached record in redis should be deleted.
However, before the im.turms.plugin.push.NotificationPusher#sendNotification method is invoked, the im.turms.server.common.domain.session.service.UserStatusService#fetchDeviceDetails method is called.
But the first parameter is offlineRecipientIds, query redis by passing offline user IDs as parameters, shouldn't the expected result be empty? If the result is empty, can the subsequent logic still be successfully triggered?

My thinking is that when a user goes offline, the corresponding user ID cached record in redis should be deleted.

turms-gateway store two entries for each user in Redis:

The current user session information and this data will be removed when the user goes offline.
The user device information that the client can send to turms-gateway by calling turmsClient.userService.login(deviceDetails={}, ...), and this data will not be removed when the user goes offline.
Another thing to note is turms-gateway only store device details passed from clients in Redis if im.turms.server.common.infra.property.env.gateway.session.DeviceDetailsProperties#items make these fields to store.
For example, if want to store the apns device details field sent by the client in Redis, you need to let Turms users add these properties, so that the plugin can fetch the device details data when im.turms.server.common.domain.session.service.UserStatusService#fetchDeviceDetails is called, and use the data to find push notification provider token to push notifications to offline users.

Thank you very much for your response.
It seems that I didn't have a deep enough understanding of the login code logic. Indeed, I did not pass the deviceDetails parameter at client side, thus I haven't observed a second cache entry for user in Redis. That's my mistake. I'll add the parameter in the business.js and try to study how the server handles deviceDetails.
I'll continue to understand this portion of the code logic, in order to implement the plugin correctly.

If I want to use third party push service like JPush or GeTui, I think I need to implement an custom NotificationHandler for Gateway, when user is online, the custom NotificationHandler code works well, but I found that when user is offline, my custom NotificationHandler couldn't be triggered.
It seems that when user is offline, OutboundMessageService#forwardClientMessageByRecipientId get activeNodeIds is empty, and return PublisherPool.emptySet(), it doesn't forward message to GateWay Nodes, and my custom NotificationHandler couldn't be triggered, as follows:
But I am not sure if I have some misunderstandings about the code or it is a bug indeed.
Any help is appreciated

Same question here:
 Set<String> activeNodeIds = sessionsStatus.getActiveNodeIds();
                    if (activeNodeIds.isEmpty()) {
                        notificationData.release();
                       // maybe this line should return the offline recipientId,
                       // otherwise  offline recipientIds is always empty, i`m not sure
                       return PublisherPool.<Long>emptySet();

                    }

Hi @Wujerry,
Thanks for pointing the bug out. The bug has been fixed and you can pull the latest code or docker images to have a try.

Hi @Wujerry, Thanks for pointing the bug out. The bug has been fixed and you can pull the latest code or docker images to have a try.

@JamesChenX It seems that there is indeed a bug here. Will this bug affect the logic of turms-plugin-push? I'm still unable to retrieve offline users correctly while implementing the JPushSender logic in turms-plugin-push, but I thought that is because I still misunderstand logic of Turms' code and I should re-study turms plugin logic first. So due to a lack of free time, I have not successfully implemented JPushSender yet.
But I will learn this bugfix later and continue to try implementing JPushSender in my spare time. Thanks for your bugfix, it will help me a lot.

In class ServiceRequestDispatcher, the code in Mono.defer never executed
return mono.map(offlineRecipientIds -> pluginManager.invokeExtensionPointsSequentially(
                RequestHandlerResultHandler.class,
                RESULT_AFTER_NOTIFY_METHOD,
                (cur, pre) -> pre.switchIfEmpty(Mono.defer(() -> cur
                        .afterNotify(result, requesterId, requesterDevice, offlineRecipientIds))))) // this line never executed
                .then();

and same in the plugin method NotificationPusher.afterNotify(), the code in Mono.deferContextual never executed.
I'm not familiar with the reactive style in java,  I just set a breakpoint in IDE and it never be hit.

Hi @Wujerry, the code in Mono.derfer(() -> ...) will only execute when:

there is more than one registered extension point implementing RequestHandlerResultHandler.
The return value of the first extension point isMonoEmpty (e.g. Mono.empty()), which means let the next handler handle the request handler result.

And it's an expected behavior.

Hi @chinamcafee, the bug will suppress some calls to methods of turms-plugin-push, so you should upgrade if you need to implement JPushSender.
And feel free to open a new issue to ask if you encounter any problems.

And find a bug in push plugin:
The default function of beforeNotify in push plugin return an empty mono, if u have not implement this method, the
 AfterNotify will never be reached.
    private Mono<Void> notifyRelatedUsersOfAction(
            @NotNull RequestHandlerResult result,
            @NotNull Long requesterId,
            @NotNull DeviceType requesterDevice) {
        Mono<RequestHandlerResult> notify =
                pluginManager.invokeExtensionPointsSequentially(RequestHandlerResultHandler.class,
                        RESULT_BEFORE_NOTIFY_METHOD,
                        result,
                        (resultNotifier, pre) -> pre.flatMap(handlerResult -> resultNotifier
                                .beforeNotify(handlerResult, requesterId, requesterDevice)));

  // the default function of ``beforeNotify`` in push plugin return an empty mono, if u have not implement this method, the
  //`` AfterNotify`` will never be reached.
        return notify
                .flatMap(handlerResult -> notifyRelatedUsersOfAction0(handlerResult,
                        requesterId,
                        requesterDevice))
                .then();
    }

    default Mono<RequestHandlerResult> beforeNotify(
            @NotNull RequestHandlerResult result,
            @NotNull Long requesterId,
            @NotNull DeviceType requesterDevice) {
        //return Mono.empty();
        return Mono.just(result);   // I changed to this but I don`t know if it is the best way
    }

@Wujerry

I have already loaded the push plugin, but Mono.defer never executed

If you load a plugin via HTTP, you need to start it to make it work.
And if there is a running extension, code in Mono.defer() will run.
I will write a doc for the plugin lifecycle this week. https://github.com/turms-im/turms/issues/1340

The default function of beforeNotify in push plugin return an empty mono, if u have not implement this method, the
AfterNotify will never be reached.

It is designed in this way.

If you load a plugin via HTTP, you need to startit to make it work.
And if there is a running extension, code in Mono.defer() will run.

That's weird. extension is running, invoker.invoke excuted and pre is empty mono.
I do some learning about Mono.defer and known why turms write this way, but it just did not worked as expected.
I put the turms-push-plugin jar into plugin dir as document said:

implement beforeNotify method, and set a breakpoint in IDE, it works.
offlineRecipientIds size > 0, but breakpoint in afterNotify can not hit.
then I remove the Mono.defer() wrapper,   afterNotify can be hit.

Then I changed code to this:
        return mono.map(offlineRecipientIds -> pluginManager.invokeExtensionPointsSequentially(
                RequestHandlerResultHandler.class,
                RESULT_AFTER_NOTIFY_METHOD,
                (cur, pre) -> Mono.defer(() -> {
                    System.out.println("----------------!");
                    return Mono.empty();
                })))
                .then();

Also can not  hit the breakpoint

default Mono<RequestHandlerResult> beforeNotify(
        @NotNull RequestHandlerResult result,
        @NotNull Long requesterId,
        @NotNull DeviceType requesterDevice) {
    //return Mono.empty();
    return Mono.just(result);   // I changed to this but I don`t know if it is the best way
}

@Wujerry  It seems that our confusion is the same. I also made such a modification, but I think it destroys Turms' original design, so I thought that my understanding of Turms' source code was not deep enough.
But it seems that you have encountered similar situation, and you modified return Mono.empty() to return Mono.empty() too.

any news?

@Wujerry I ran some quick tests but they don't reproduce the problems you have mentioned, and will have some full tests on the weekend.
btw: Personally, if someone asks multiple questions in one issue, I prefer to answer them in one comment after testing but it also means that people may need to wait for me more time because it's hard for me to take so much time to check multiple issues, especially after work. In other words, I tend to schedule to check them only on weekends when I have much available time.

@JamesChenX thanks a lot, I don`t know how to say very thank you in English but thank you.


[DOC]: Extending the Morpheus Stages Document

How would you describe the priority of this documentation request
High
Describe the future/missing documentation
Preprocessing stage can’t be changed (there was a request to proceed with some customizations through preprocess)
Any small change (like field name change) causes many problems and the solution is not clear and complex (VG: may be we need to provide some gradience for such possible changes as this scenario is very common)
Some of the teams wanted to use Cybert, didn’t know from where to start and how to use it
Where have you looked?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I have searched the open documentation issues and have found no duplicates for this bug report

Not an issue anymore.


Add ActiveCollab to other.yml

ActiveCollab is a project-management-type system.

ahem

Well then. :stuck_out_tongue:


Uart write change async

I believe I have the Asynchronous SerialX writes working, so I thought I would do a Pull Request, such that you can take a look and see what you think.
I updated the comments in the commits to hopefully have lots of the details.
I should mention that while doing this I updated the SerialX read buffers to have 128 bytes instead of 256, such that adding a 128 byte write buffer did not raise memory usage.
I would suggest that one or more of you take a close look at what I did and see what you think.  Likewise you might want to run additional tests.  Example I did not try out any bluetooth tests.  I did use Serial2, to output debug information, that I used a USB to Serial adapter to show the output in a PuTTy window.
Probably should test more Dynamixel Workbench programs.  Or let me know which ones you think should be run.
Let me know what you think.

@OpusK and others
Thought I would mention, that I have started playing with making similar changes to OpenCR code base.
I thought maybe it would be beneficial to discuss some of the things I looked at when doing the code for this PR and hopefully soon for OpenCR as for some explanation on why I did what I did...
I have noticed that looked like someone maybe had started doing some similar stuff awhile ago as there is a struct ring_buffer as part of the UARTClass and there is one defined for both RX and TX (Both 2K each), so currently 16K RAM memory used for Serial1-4...
Again I don't think any of the write code is used to work with this, only SerialX.flush() actually uses the ring buffer.   From the pieces I see, it was setup to use the function: HAL_UART_Transmit_IT
instead of HAL_UART_Transmit to transfer each character, and have the user interrupt, recall it with the next byte out of the ring_buffer...  You can see some of this in the commented out code in:  UARTClass::TxHandler.
Earlier that was my first approach for this PR, but I for myself did not like this approach as, I believe it forces the system to have to do three interrupts for each character output.  You first call the function HAL_UART_Transmit_IT with your one byte buffer, which sets the variables and then enables the TXE interrupt.  So then interrupt happens (UART_Transmit_IT), pulls the byte to transmit.  It then almost immediately will do another TXE interrupt as the byte was transferred to transmit shift register,  At this point the interrupt will see the transfer count is 0 and it will disable TXE and enable TC interrupt.  Then once the byte finishes being transferred and TC happens.  At that point I believe your TxHandler would be called to start up the process for the next byte...  And there will probably be a gap of time between bytes as you wait until the full transfer complete happens before you setup the next byte.  So the hardware double buffering is defeated.
I did not like that much overhead per byte, especially if you are trying to support 2Mbs or 3Mbs...
So I try to short circuit as much of this as possible.   So when you call the transfer, it instead checks to see if the queue is empty AND TXE is set and directly outputs the byte to TXE...   If you call again (Or use the buffer version of write), it will again check TXE and try to keep queue full.  If you call with buffer, I go ahead and transfer stuff into the queue, but before I return, I check to see if TXE is set again and go ahead and pump the next byte out, before I enable the TXE interrupt... So hopefully minimize interrupt overhead.
Note: One thing I know I have not tested much (if at all) with this code is handling when the ring buffer gets full and there is still more output to go.  The code is setup to hopefully enable the interrupts and spin waiting for space to be available to put the remaining bytes into the queue...
Again hope the above makes sense.  Let me know if there are things you would like changed.
Kurt

I personally think it would be good if the user can selectively use both IT and DMA methods.
And another point is to reduce the overhead as you said.
And OpenCR uses a lot of memory in the buffer, and if possible it would be best if the UART could communicate at high speed with minimal memory usage without overhead.
I would like to test it to see if it can handle the amount of data of about 3Mbps, 4.5Mbps and apply the most suitable method.

I agree that it would be great to support a high speed communications on the UARTS, with as minimal overhead as possible.   Which I hope to do!
The approach I have done so far for the CM (I will abbreviate the board names), is very similar to what I did several years ago, for the Arduino 8 bit AVR boards, that made it reasonably possible to support DXL outputs without having to write special code… This was stuff I did to hopefully allow Trossen Robotics to get off their own fork of Arduino (latest one is in the vintage of 1.0.4) and onto supporting mainline Arduino…  Luckily I was able to get Arduino to pick up the changes back in 1.5.3? (plus or minus a release…).
But by having the system try to first go direct to hardware before putting in queue and then rely on IT to move to hardware… So on CM board I did this.  With this I tried running the code in the PR at 2mbs and it showed some slight gaps between some bytes with my pseudo firmware app, but at 3mbs it showed more or less no gaps, as the timing and such was that it put the entire packet out using the TXE (DR register)  Did not try any faster.
As I mentioned I was thinking of trying DMA approach.  Problem with DMA TX is I believe  it is setup to output some fixed number of bytes.   So that Is why I thought about two buffers, which may work pretty well.  But wonder about a few things.
DMA:  There are a limited number of channels (or streams)?  What is unclear to me on the CR boards is suppose that the system is setup to want to use DMA for both SPI and UARTS.  Suppose I wish to use DMA1 Stream4 for UART4_TX but I also may want to use it for SPI2_TX Note: Not sure if both this UART/SPI are setup on this board… But as an example… Can both be active?  If so both want to implement: DMA1_Stream4_IRQHandler  If both can be active, I would assume you would need a combined handler which somehow detects which channel the interrupt is associated with…
DMA or IT: if you go to higher speeds. Like you mentioned 4.5Mbps the solution may depend.  That is with your current Arduino, all SerialX.write(?)  funnel down to SerialX.write(const uint8_t c),  Which again outputs one byte at a time, and current code goes through to Transfer Compete (TC) before returning… First change I put in was to add the virtual function override:  SerialX. write(const uint8_t *buffer, size_t size);
So if you call with buffer or string, it goes through system as one exchange.  At the higher baud rates and assuming sizeT is not overly large, this is about as efficient as it can be.  It loops through your buffer on TXE keeping the double buffering full, and then waits until TC to return for whole buffer.
So in this case for DXL from USB, may be as good as you can do as long as you remember to write as much as you can as one Arduino Serial call…  However if you are doing a large write, like maybe a large Sync Write, which does not send back data and suppose the output is to 25 servos and 8 bytes per servo (position/speed), then  would make sense to send buffer out with IT or DMA such that the sketch could continue…  What exactly the threshold is where this makes sense?  Good question.
As for allowing the user to chose IT or DMA.   Not sure.  With Arduino the idea is to keep thing simple for end users.  So not sure how to setup so one could choose.    But I know that most users now expect that when they do something like: Serial1.print(“Hello World”);, that the system does not wait for this to complete before continuing…   Where it bit me, was when I setup Serial1 on the CM to do debug outputs, as Serial is being used to receive DXL commands from a host)…
When I started to debug, I added things like: Serial1.println(“IR called”), and found it completely took over the timing…
Kurt

Hi @OpusK @robotpilot @chcbaram @routiful,
Thought I would update on the OpenCR updates to do like this.  I started merging this code in, but then decided to do a detour and try a more minimal change update to see if that works.
That is to go along the line of what appeared to be a WIP in the OpenCR code base which has large TX buffer built into UARTClass.   I put the current WIP up in a new fork/branch:
https://github.com/KurtE/OpenCR/tree/Async_Serial_minimal
Currently it is only doing Interrupt, not sure yet how hard to do yet.  I may issue PR over there soon to make it easier for all of you see which approach you might like.  Note: The approach currently here I believe can be more efficient as I try to minimize interrupts.
But the WIP OpenCR does already speed up some of the DXL Outputs, as it does not try to output one byte at a time which then waits until TC happens before starting next byte....
With current release code,  You see a gap between bytes.  On the LA output you will see gap between last bit and next start or the stop looks like a 2 stop depending on if byte... About 2.2us instead of 1us...

Current WIP code with ISRs  again at 1Mbps,  You can see these gaps are gone.

Note: Could get this part working even more minimal by simply doing the SerialX.write(buffer, cnt) to call the Transfer as one transfer...

Hi, @KurtE
Sorry for late reply you.
First of all thank you for your great contribution.
However, the code you have modified is the DYNAMIXEL SDK. Especially, in case of port_handler(like this) in double, it is common file, so it may cause problems when it is modified. Since the SDK is currently managed by a different team, it seems that some rules need to be followed.
However, the flush() function you added is a required feature when using DMA functions...
I think what we need to worry about, along with the things I mentioned below.
Among the examples that use Serial.print(), it can be assumed that all data is transmitted when the function is exited. Unfortunately, the examples do not call the flush() function further, which can be problematic.
And, adding the Flush function to the DMA function is also awkward.
I think we need to worry about this.

@KurtE ,
This PR is duplicated with #42, with the lib file conflict issue.
Can you end this PR and continue at #42?

As mentioned this one has been superseded by #42.


Azure AD Breaking change means first step does not work

Following a breaking change from Azure AD causes step 1 of the guide to fail.
https://docs.microsoft.com/en-us/azure/active-directory/develop/reference-breaking-changes#appid-uri-in-single-tenant-applications-will-require-use-of-default-scheme-or-verified-domains

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 55b2707d-7f51-554d-06dd-2cf71b906237
Version Independent ID: ab8a566c-adba-5c99-8a16-713f9c979774
Content: Azure RBAC for Azure Arc-enabled Kubernetes clusters - Azure Arc
Content Source: articles/azure-arc/kubernetes/azure-rbac.md
Service: azure-arc
Sub-service: azure-arc-kubernetes
GitHub Login: @shashankbarsin
Microsoft Alias: shasb

Thank you for your detailed feedback.  We are looking into the issue, and we will respond when we have more information.

As Azure Arc is still in preview, I've assigned this issue to the content author for further review.

Hello @dylreeves - first of all, I want to apologize for the extremely long time to respond. We've made many updates to this article since you left this feedback, and I believe all of the current content should be up to date and working properly. Because of this, I'll close this issue now, but please let us know if you still have questions or concerns. Again, I'm sorry for the long delay in responding, and thank you for taking the time to share your feedback with us.
#please-close


When, Oh When, Mustache 2.0?

Is there any hope Mustache 2.0 will make a break for it within the year?
If there is a hold up b/c of so many feature possibilities and undecided choices to make about them, may I suggest that this process be divided into 2.0 and a 3.0 tracks?
For 2.0 just adopt the things you are sure about and/or already have precedence in the field. For instance Hogan.js has already adopted template inheritance. I recommend taking that, adjusting it so the notation is more intuitive/useful (for instance, my recommendation), and go with it. Another possibility is Handlebar.js style helpers. And personally I'd like to see Liquid style filters. But I digress. The point is pick the things that are ready or near ready to go, and make 2.0 out of those and save anything that is still well up in the air for 3.0.
Personally I feel there is some urgency to get the move on, as new template engines are making the scene to draw people away from a common standard, which Mustache has succeeded in doing more than any other for logicless templates.

I'm closing this, partly because the discussion is long dead but mostly because having any particular version released should not be a goal in itself.


Adding gcp pub sub connector

Signed-off-by: Harsh Thakur<EMAIL_ADDRESS>Resolves #11
Cannot be merged until https://github.com/kedacore/keda/issues/1273 is resolved

https://github.com/fission/fission/issues/1864 needs to be resolved since  kedacore/keda#1273  is resolved in it.

@therahulbhati can you PTAL now?


/notice not being evaluated

So i am trying to get notifications sent on channel notices
Example would be /notice #channel test, these notices show up in the channel normally as a chat message (but marked as a notice)
When debug is on the notice never appears to be evaluated and as such i cant make it match any context or highlights
ideally id like /notices as a filter of their own, these are often used in some IRCs as "hey read this its important"

Glad to see this is planned to be part of the 2.0 release. Global notices are essentially invisible unless one is connected AND happens to look at the server tab.

Any movement on this? Would love to have /notice to a channel be sent to the push service.


Reuse generic parameters of class field

Search Terms
reuse generic parameters

Suggestion
I would like to have syntax sugar for reusing generic parameters of some class field. There is a way to do it, but it looks more like hack.

Use Cases
For example we have that code:
 class Foo {
      bar: Promise<string[]>;

      // a is the same type as type of promise in bar
      baz(a: ????): Foo['bar'] {
        // some code
        return Promise.resolve(a);
      }
    }

If we want to say that a has the same type as generic parameter of bar - we have to write something like that
(a: Foo['bar'] extends Promise<infer U> ? U : never)

of course we could create type or interface - but then it is not a reuse.
 type Baz = string[];
 class Foo {
      bar: Promise<Baz>;

      // a is the same type as type of promise in bar
      baz(a: Baz): Foo['bar'] {
        // some code
        return Promise.resolve(a);
      }
    }

Examples
It would be nice to have something like that
(a: <Foo['bar']>) // <Foo['bar']> is resolved to Promise<string[]>

So we can reuse generic types without creating a type or interface for that

Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

I would like to have syntax sugar for reusing generic parameters of some class field. There is a way to do it, but it looks more like hack.

We already have a way to do this, and it's not a hack.
type Unpromisify<T extends Promise<any>> = T extends Promise<infer R> ? R : never;

Unpromisify<Foo['bar']>

@MartinJohns
That why I wrote

syntax sugar

The Unpromisify is not optimal solution, We have to write the type all the time we have another Generics(Observables, Proxy etc.)
<Foo['bar']> is prettier than
type Unpromisify<T extends Promise<any>> = T extends Promise<infer R> ? R : never;
Unpromisify<Foo['bar']>

It's trivially solved with existing tools, and adding new syntax should always be the last resort. It will increase the likelihood of collisions with future JavaScript updates and it will increase the complexity of the compiler.
I really don't see the purpose for new syntax because it's a tiny bit prettier (for some people, not for me, it's very subjective).

Foo['bar']<> could work with both. For example Foo['bar']<0>
This solution have a major disadvantage - if I want to change Promise to some other class with same interface - I have to:

change field declaration
change 'Unpromisify' type
change 'Unpromisify' to 'Unsometype'
What I want - is to extract type from class by knowing it structure. And if this type changes - the extracted type changes automatically

Foo['bar']<>

Now you're in the domain of higher-kinded types.

This is already extremely possible with multiple avenues of solution. We'd only add new syntax here if this was a much more common scenario with more defined use cases that weren't being adequately addressed.
There are additional complications here -- if you wrote
type Foo<T> = Bar<string, T>;
class K {
  m: Foo<number>;
}

From a structural perspective, it's really not clear if the first type parameter of m is number because that's how it lexically appears (which should never matter!) or string because that's the first type parameter given to Bar - or maybe Bar itself references another generic type? Foo<T> and Bar<string, T> ought to behave identically and you're proposing making a mechanism that breaks that.

I've found recently a nice thing called Parameters. It takes a function and gives the tuple type back.
It would be nice to have that thing for generics.
For example:
class Foo {
      bar: Promise<string[]>;

      // a is the same type as type of promise in bar
      baz(a: Gemerics<Foo['bar']>[0]): Foo['bar'] {
        // some code
        return Promise.resolve(a);
      }
    }


Fix incorrect config path in --help

Fixes #3154.

I really feel like the definition of CONFIG_PATH should at least be near [...] if not unified in some way.

I don't think it's a good idea to move a constant to a different file, where it is never used.
Unification is also not possible because these are the primary default locations, not the absolute locations. Additionally, environment variables mean nothing to Alacritty when just present in a String.
So I think it's best to keep it simple and just move the constant to the top of the file where it is used. If we ever used it anywhere else, that would be a different story, but we don't, since this is not the actual config location, but just the documentation string for the first config location searched.

btw we do have the same problem for man page for macOS, what do you think about being a bit more verbose an just put every path into man page? It's a man page in the end of the day...

btw we do have the same problem for man page for macOS, what do you think about being a bit more verbose an just put every path into man page? It's a man page in the end of the day...

I've mentioned the manpage in the issue, but I haven't actually thought about the idea of just putting everything in it. I think that's a great idea, let me see if I can figure out how manpages work to extend that section a bit.

I've extended the manpage to include what is in our readme already. I think this is a great bit of additional information:


Testing code needs proper setup and tear down

We have failures paths in our test code that leave garbage on the test system. We need a proper clean up for each test alongside each test.

I believe the required fix for the issue has been pushed, so closing this now. Please reopen with the details if you think some work needs to be done.


Remove rules deletion on validation

Hi, @thedevsaddam .
It seems that fields of Options.Rule are deleted in some case, so Options.Rule cannot be used more than once. This behavior seems a bit inconvenient, so I'd like to fix it. What do you think?

Coverage increased (+0.07%) to 88.052% when pulling 263cfdd9eb9ea05792cd7e4b651f31cb2a9933e4 on bluele:remove-rules-deletion into 738a97e60a30ff7ac645e211bd0a003e65bd1945 on thedevsaddam:master.

Coverage increased (+0.07%) to 88.052% when pulling 263cfdd9eb9ea05792cd7e4b651f31cb2a9933e4 on bluele:remove-rules-deletion into 738a97e60a30ff7ac645e211bd0a003e65bd1945 on thedevsaddam:master.

Coverage increased (+0.07%) to 88.052% when pulling 263cfdd9eb9ea05792cd7e4b651f31cb2a9933e4 on bluele:remove-rules-deletion into 738a97e60a30ff7ac645e211bd0a003e65bd1945 on thedevsaddam:master.

Coverage increased (+0.07%) to 88.052% when pulling 263cfdd9eb9ea05792cd7e4b651f31cb2a9933e4 on bluele:remove-rules-deletion into 738a97e60a30ff7ac645e211bd0a003e65bd1945 on thedevsaddam:master.


Can't visit profile page using URL

When trying to visit http://<IP_ADDRESS>:8000/profile/USERNAME from the homepage, it doesn't work, but when clicking the profile button, it works, even though it's the same URL. So this issue might have something to do with the router and the location object passed to the ProfilePage.
Steps to reproduce

yarn build
python manage.py runserver
Visit <IP_ADDRESS>:8000
Log in
Visit http://<IP_ADDRESS>:8000/profile/USERNAME

I get a blank page here and the following error.

I've been trying to fix this and it seems that it's out of my hand.
Basically when viewing a profile, I need to use two parameters.

Current user id -> retrieve from local storage
The user id of this user profile -> get from the previous page where the user would click on something that has this person's id stored.
If viewing directly from the url, then there's no way i can get the id of this person. I checked the behaviour of linkedin, the user gets to define their unique profile page url.
Two possible solution here:
instead of using profile/username, use profile/id, say profile/https://cmput404-socialdistribution.herokuapp.com/author/1
pro: the user id can be retrieved from the url
con: url will be massive and looks weird
leave this for now since its not part of the requirements...

@ceciliaccwei Ok sure we can just leave this for now. I was thinking it would be an issue since we need the user to be able to access profile pages of other users but we can just hope that no one will type in the url and just use the UI to access it 😄

Once we finish the search function, the user should be able to see those users he wants to find and access their profile page from there. Hope this will be fine😄


Allow to select mode for FileUploader form widget

Hi there, thanks for a great plugin.
It would be nice if in the builder plugin inside the Models -> Forms -> fields.yaml file where we define all form widgets we could select the mode for the file uploader widget just like in the media finder widget.
Thanks again :)

@angay9 you should already be able to??? https://github.com/rainlab/builder-plugin/blob/master/classes/StandardControlsRegistry.php#L742-L753


Syntax error in forms.py

I get a syntax error when importing django remote forms
\django_remote_forms\forms.py", line 141
except Exception, e:
^
SyntaxError: invalid syntax
Is this a problem on my end or from the package?
thanks in advance

hey did you solve this problem ?

I had the same error. The solution for me was to move the library as normal django app and change the python 2 syntax to python 3. I do not expect support for this old python package.
except Exception, e: > except Exception as e:


SW-4503 Add the ability to inject a JSX element into the top bar buttons of the Table

Current dependencies on/for this PR:

main

PR #449   👈

This stack of pull requests is managed by Graphite.

publish rc

publish rc

Triggered a new build, vercel deployment had failed.


Code OK on V5.1 does not run on V7.0.0RC2

Name                           Value

PSVersion                      7.0.0-rc.2
PSEdition                      Core
GitCommitId                    7.0.0-rc.2
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0
The code in the attached script runs OK on V5 but generates the following error on V7:
MethodInvocationException: S:\PowerShellScripts\TestUserIngroup_1.ps1:5
Line |
5 |  … t -Process {$_.GetType().InvokeMember('Name', 'GetProperty', $null, $ …
|                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Exception calling "InvokeMember" with "5" argument(s): "Unknown name. (0x80020006 (DISP_E_UNKNOWNNAME))"
The script:
$user = $env:USERNAME
$group = 'Users'
$groupObj = [ADSI]"WinNT://./$group,group"
$membersObj = @($groupObj.psbase.Invoke('Members'))
$members = ($membersObj | ForEach-Object -Process {$_.GetType().InvokeMember('Name', 'GetProperty', $null, $_, $null)})

If ($members -contains $user)
{Write-Host -Object "$user exists in the group $group"}
Else
{Write-Host -Object "$user does not exists in the group $group"}

TestUserIngroup_1.zip

@SeeminglyScience Have you any thoughts?

I am somewhat new to GitHub so I am not sure what you are expecting when you say "Have you any thoughts" My only thought at this time would be expecting it to work on PSV7* or there to be a better/alternative way to achieve the same result?

@RG255 References like "@RG255" means contacting the owner of this name. So my question was for @SeeminglyScience who could share useful thoughts about the issue.

@iSazonov Looks like PowerShell isn't able to determine that IADsMembers (the collection returned by Members) is enumerable.  So instead of $membersObj enumerating into member objects like in Windows PowerShell, the attempt is made to call a Name property on the IADsMembers interface which doesn't exist.
You can kinda see this by attempting to get the property Count instead, which will throw not implemented instead of unknown name.

So playing with it a bit, my guess is that this happens because if you query IDispatch.GetIDsOfNames with the names _NewEnum and ppEnumerator then that exception is thrown.  However, if you manually invoke GetIDsOfNames with a preserved sig (no automatic HResult handling) you'll see the HResult above returned, but it'll also show DISPID_NEWENUM for _NewEnum.  If _NewEnum was really not found, it would be -1.
Not sure atm where this needs to be fixed though.

Tested with this code, click to expand
Add-Type -TypeDefinition '
    using System;
    using System.Runtime.InteropServices;
    using System.Runtime.InteropServices.ComTypes;

    namespace Testing
    {
        public class DispatchWrapper
        {
            private readonly IDispatch _real;

            internal DispatchWrapper(IDispatch real) => _real = real;

            public static DispatchWrapper Create(object obj) => new DispatchWrapper((IDispatch)obj);

            public int GetTypeInfoCount(out int info) => _real.GetTypeInfoCount(out info);

            public int GetTypeInfo(int iTInfo, int lcid, out ITypeInfo ppTInfo) => _real.GetTypeInfo(iTInfo, lcid, out ppTInfo);

            public int GetIDsOfNames(Guid iid, string[] rgszNames, int cNames, int lcid, int[] rgDispId)
                => _real.GetIDsOfNames(iid, rgszNames, cNames, lcid, rgDispId);

            public void Invoke(
                int dispIdMember,
                Guid iid,
                int lcid,
                INVOKEKIND wFlags,
                DISPPARAMS[] paramArray,
                out object pVarResult,
                out EXCEPINFO pExcepInfo,
                out uint puArgErr)
                => _real.Invoke(
                    dispIdMember,
                    iid,
                    lcid,
                    wFlags,
                    paramArray,
                    out pVarResult,
                    out pExcepInfo,
                    out puArgErr);
        }

        [Guid("00020400-0000-0000-c000-000000000046")]
        [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
        [ComImport]
        internal interface IDispatch
        {
            [PreserveSig]
            int GetTypeInfoCount(out int info);

            [PreserveSig]
            int GetTypeInfo(int iTInfo, int lcid, out ITypeInfo ppTInfo);

            [PreserveSig]
            int GetIDsOfNames(
                [MarshalAs(UnmanagedType.LPStruct)] Guid iid,
                [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPWStr)] string[] rgszNames,
                int cNames,
                int lcid,
                [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.I4)] [Out] int[] rgDispId);

            void Invoke(
                int dispIdMember,
                [MarshalAs(UnmanagedType.LPStruct)] Guid iid,
                int lcid,
                INVOKEKIND wFlags,
                [MarshalAs(UnmanagedType.LPArray)] [In] [Out] DISPPARAMS[] paramArray,
                out object pVarResult,
                out EXCEPINFO pExcepInfo,
                out uint puArgErr);
        }
    }'

$group = 'Users'
$groupObj = [ADSI]"WinNT://./$group,group"
$membersObj = @($groupObj.psbase.Invoke('Members'))
$disp = [Testing.DispatchWrapper]::Create($membersObj[0])
$nullGuid = [activator]::CreateInstance([guid])
$enUS = [cultureinfo]::CurrentCulture.LCID
$namesToQuery = '_NewEnum', 'ppEnumerator'
$dispIds = [int[]]::new($namesToQuery.Length)
$hr = $disp.GetIDsOfNames($nullGuid, $namesToQuery, $namesToQuery.Length, $enUS, $dispIds)

'HResult: 0x{0:X}' -f $hr
for ($i = 0; $i -lt $namesToQuery.Length; $i++) {
    '{0} ID: {1}' -f $namesToQuery[$i], $dispIds[$i]
}

Should return:
HResult: 0x80020006
DispIDs: -4, -1

Another update, here's the code that needs to change:
https://github.com/PowerShell/PowerShell/blob/e2f838e3c5baabffc4d14fa747902d18628044bc/src/System.Management.Automation/engine/COM/ComUtil.cs#L426-L439
It fails because IADsMembers implements IDispatch, but throws E_NOTIMPL when trying to get type info.  The other members work fine though.
Instead of (or in addition to) querying type info, it needs to directly invoke GetIDsOfNames with two names. The first has to be _NewEnum and the second can apparently be anything.  It needs two though, otherwise it returns unknown. If the first ID is DISPID_NEWENUM then _NewEnum can be invoked.
Example invoke using the above code in the details pane:
$results = $null
$excep = [Activator]::CreateInstance([Runtime.InteropServices.ComTypes.EXCEPINFO])
$argErr = 0u;
$disp.Invoke(-4, $nullGuid, $enUS, 'INVOKE_PROPERTYGET', @(), [ref] $res, [ref] $excep, [ref] $argErr)

# yield
$results

Which returns:
System.__ComObject
System.__ComObject
System.__ComObject
System.__ComObject
System.__ComObject
System.__ComObject

/cc @SteveL-MSFT I'm pretty sure I've seen quite a few folks on with similar issues on the PS slack.  This might be the first issue for it here, but it's potentially very impactful for folks still using COM objects frequently (and who aren't super likely to be vocal here). If possible, it may be worth resolving before 7.0 GA.

@SeeminglyScience Thanks for your investigations!
I think we need to have more tests for COM interactions.

Thanks @SeeminglyScience for analyzing the issue and pinpoint the cause ❤
The ComEnumerator was introduced back in early .NET Core 2.0 preview period of time (#4553), because GetEnumerator() didn't work on COM object even if the object can be cast to IEnumerable and .NET Core team said it was by design at the time (https://github.com/dotnet/runtime/issues/21690).
With .NET Core 3.1, GetEnumerator() works on the COM objects that can be cast to IEnumerable 🎉
however, for the COM object that can be cast to IEnumerator, exception is thrown when calling MoveNext() on it.
So we still need the ComEnumerator, but much simplified to just cover the case that the COM object implements COM.IEnumVARIANT interface.
PR was submitted: #11795


Failed to get extension list

Steps to reproduce
When you go to browse to download a new extension
Expected behavior
It suppose to display extension list i.e manhua sources
Actual behavior
Failed to get extension list
Crash logs
No response
Mihon version
0.16.5
Android version
Android 8.0
Device
Oppo A16k
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have gone through the FAQ and troubleshooting guide.
[X] I have updated the app to version 0.16.5.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

That screen really could use some sort of explanation along those lines instead of the standard kaomoji error message

Well the first issue to fix now the download link is not working

Mihon doesn't have, associate, or provide any extensions

So how is it been used

It doesn't provide any, never has. If you've been seeing a list there you must've provided your own at some point so just do it again ¯_(ツ)_/¯

Ok i will read that, also the download link is not working

Ok i will read that, also the download link is not working

What download link are you talking about? The one in the releases page of the GitHub?

In the release page and website

Check the download link from mobile

Please join our Discord GitHub is not the best option for troubleshooting issues


The amazon.ion.simple_ion.dumps method output doesn't work with DynamoDB import table

I am trying to generate ion data file manually using this library so that I can use it for DynamoDB import table, this is my DynamoDB item in python dictionary.
{
        "id": 1, # this is hash key
        "name": "Alice"
}

the amazon.ion.simple_ion.dumps method gives me: $ion_1_0 {Item:{id:1,name:"Alice"}}, note that there's no dot after number 1. Then the import_table API fails.
However, if I manually add the dot behind the number 1, making it to be $ion_1_0 {Item:{id:1.,name:"Alice"}}, then it works.
I also tried to export a manually createdDynamodb table and I found out that the export ION file has the dot after the integer number.
I also tried the loads method, I think the integer without dot is a valid value for deserialization. However, it doesn't work with DynamoDB table import.
How do I ensure that there's an dot after any integer in the text view of my data?

This isn't really a bug with ion-python.
In the Ion text format 1 is an Integer and 1. is a Decimal, see: https://amazon-ion.github.io/ion-docs/docs/spec.html
Per the DynamoDB Import Docs: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html#S3DataImport.Requesting.Formats.Ion
They import an Ion Decimal as a Dynamo DB Number. I do not know why they don't map an Ion Integer to a Dynamo DB Number, but they don't. That's a possible feature request for Dynamo DB.
Assuming that it's faster to change your code then get DynamoDB to change, and that your code block is your python code:
To serialize an Ion Decimal from Python you need to create a decimal.Decimal. That will emit in your Ion stream as an Ion Decimal.
See https://github.com/amazon-ion/ion-python/blob/master/amazon/ion/simpleion.py#L33

I would further advise that for your production code you serialize your Ion as Binary for the imports. Obviously the text format is great for debugging and developing, but the binary format has improved data density and will be faster to import.
Please check out the pydoc in simpleion and let us know how we can improve that if needed.

Thanks @rmarrowstone .
I believe it is still a bug, but not in amazon ION python, it is actually about DynamoDB Import.
The simpleion.dumps() method gives you the correct value $ion_1_0 {Item:{id:1,name:"Alice"}} (I expect the id to be integer). However, the DynamoDB import table feature doesn't recognize it. In my TableCreationParam, I defined the attribute type is N, however, DynamoDB import table feature raises an error for that.
@rmarrowstone another issue is that the DynamoDB import table  document didn't mention how to use ion binary format to prepare the data. And the document says that Items in an Ion file are delimited by newlines. Each line begins with an Ion version marker, followed by an item in Ion format., which implies that I should use text to code my data. Then how can I do this?
I would further advise that for your production code you serialize your Ion as Binary for the imports.

Sadly it does look like they only support the Text format, so that was some bad advice, sorry. It would be more optimal if they supported the binary format, but alas...


Assessment assignment table - Edit

As an analyst
I want to edit project information
So that I can assign assessments
Description:
Trying to recreate the project assignment spreadsheet.
Key is to display a lot of information clearly to enable users to scan it, make it editable
Acceptance Criteria:
Private Zenhub Image
Wireframe

[ ] Edit of the Assessment cells (PM, Tech, Permitting, GIS) is a Select of the List of Analysts

Implementation Details

Hey team! Please add your planning poker estimate with Zenhub @AntBush @marcellmueller @rafasdc

The dropdown of analysts in the table contains those marked Inactive. It should only include active analysts.

Also, would it be possible to change the styling of the dropdown to look more like a cell and less like a dropdown in a form? For example: [Uploading... Screenshot 2023-10-24 at 10.33.56 AM.png]

Okay I'll fix the inactive analysts issue. As for styling, make a styling ticket.

@marcellmueller @elliottSlee  I had created a bug ticket for the inactive users yesterday #2484


EventBridge targets are not identified by moto when parsing cloudformation file.

Hi,
I'm using a cloudformation file to create my test setup. The scenario is simple: put an event, use a rule to move that event to SQS, and use an InputTransformer to change the event a bit.
When loading the CFT, all is good, except that the target is not identified, and therefore the event is not present in the queue.
When I add a target using boto3 code (same target from the CFT), I can see the event in the queue (although it is not transformed - not sure if I did something wrong, or if it doesn't work as well)
using python 3.12.2
moto[all]==5.0.2
boto3==1.34.55
cfn-flip==1.3.0
pytest==8.0.2
(running on windows, I suspect it shouldn't matter, but just in case)
Full details below, but the important part is this:
Before I run events_client.put_targets
the result of events_client.list_targets_by_rule(EventBusName=event_bus_name,Rule=events_client.list_rules(EventBusName=event_bus_name)['Rules'][0]['Name'])  the target list is empty (and a message is not put in SQS).
After I run events_client.put_targets, using the same data parsed from the CFT (had to manually replace the ARN, as the function was not accepted by moto, which makes sense to me) - the target is present, and events are getting to SQS.
I'm using the following CFT:
Resources:
  MyEventBus:
    Type: AWS::Events::EventBus
    Properties:
      Name: "TestEventBusName"

  DestQueue:
    Type: AWS::SQS::Queue

  MyQueuePolicy:
    Type: AWS::SQS::QueuePolicy
    Properties:
      PolicyDocument:
      Version: 2012-10-17
      Statement:
        - Effect: Allow
          Principal:
          Service: events.amazonaws.com
          Action: sqs:SendMessage
          Resource: !GetAtt DestQueue.Arn
          Condition:
            ArnEquals:
              aws:SourceArn: !GetAtt MyRule.Arn
        - Effect: Allow
          Principal:
            Service: events.amazonaws.com
          Action: sqs:SendMessage
          Resource: !GetAtt DestQueue.Arn
          Queues:
            - !Ref DestQueue

  MyRule:
    Type: AWS::Events::Rule
    Properties:
      EventBusName: !GetAtt MyEventBus.Name
      EventPattern:
        source:
          - my.source
        detail-type:
          - Malicious
      State: ENABLED
      Targets:
        - Id: meh
          Arn: !GetAtt DestQueue.Arn
          InputTransformer:
            InputPathsMap:
              account: $.account
              detail: $.detail
              detail-type: $.detail-type
              id: $.id
              region: $.region
              resources: $.resources
              time: $.time
            InputTemplate: "{\"id\": \"<id>\",\"detail-type\": \"<detail-type>\",\"time\":\"<time>\",\"region\": \"<region>\",\"account\": \"<account>\",\"resources\":<resources>,\"http_method\": \"POST\",\"http_endpoint\":\"/test/endpoint\",\"detail\":<detail>}"

my test:
from cfn_tools import load_yaml
import boto3
from moto import mock_aws
from assertpy import assert_that

def format_message(Detail, DetailType, EventBusName, Resources: list[str], Source):
    entry = {
PutEvents call is used.
        'Source': Source,
        'Detail': json.dumps(Detail),
        'DetailType': 'Malicious',
        'Resources': Resources,
        'EventBusName': EventBusName
    }
    return entry

#This is the important stuff - I expected to not need this
def workaround_missing_target(events_client, q_url, sqs_client, template):
    q_arn = sqs_client.get_queue_attributes(QueueUrl=q_url, AttributeNames=['All'])['Attributes']['QueueArn']
    event_bus_name = next(filter(lambda x: x['Name'] != 'default', events_client.list_event_buses()['EventBuses']))[
        'Name']
    targets = template['Resources']['MyRule']['Properties']['Targets']
    targets[0]['Arn'] = q_arn
    events_client.put_targets(
        Rule=events_client.list_rules(EventBusName=event_bus_name)['Rules'][0]['Name'],
        EventBusName=event_bus_name,
        Targets=targets,
    )

@mock_aws
def test_moto():
    path = os.path.join(os.path.dirname(__file__), 'my_template.yml')
    with open(path) as f:
        template_str = f.read()
    template = load_yaml(template_str)
    cf_client = boto3.client('cloudformation', region_name='eu-west-1')
    sqs_client = boto3.client('sqs', region_name='eu-west-1')
    events_client = boto3.client('events', region_name='eu-west-1')
    cf_client.create_stack(StackName='my-stack', TemplateBody=template_str)
    q_url = sqs_client.list_queues()['QueueUrls'][0]

    workaround_missing_target(events_client, q_url, sqs_client, template)

    entry = format_message(Detail={'a': 'b'}, DetailType=None, EventBusName='TestEventBusName',
                           Resources=["path/to/file"], Source="my.source")
    events_client.put_events(Entries=[entry])

    m = sqs_client.receive_message(QueueUrl=q_url)
    assert_that(m['Messages'][0]['Body']).contains('http_method') # assertion fails, not sure if I did something wrong, or if this is another bug

Hi @amitw-di, thank you for raising this and for adding a reproducible test case!
Marking it as an enhancement to add this.