Register services in service registry during the service bootstrap phase

This pull request updates service bootstrap phase (st2common.service_setup.common_setup function) to register every StackStorm service in the service registry.
Background and Context
Registering each service in a service segistry will allow us to have a better overview of the active and running services. This will allow us to implement things such as more dynamic (utilization and capability based) action execution scheduling / routing and more.
The goal was so it also works outside of the Kubernetes context in a traditional HA environment. It's also worth noting that Kubernetes has it's own service registry concept, but that's based on a slightly higher level Kubernetes view of a service (two related concepts, but not exactly the same).
Implementation Details
Implementation utilizes group membership primitives from the OpenStack tooz library.
There were multiple reasons (as discussed before) for utilizing tooz library:

Consistency

We already use it in other places for distributed locking.

Support for multiple drivers / adapters

We will officially probably only support a single backend (e.g. ZooKeeper or etcd), but people will still be able to user other tooz supported backends at their own risk and this will also help with testing.
Keep in mind that only the following tooz drivers support group membership primitives which we utilize: zookeeper, redis, etcd3, etcd3gw, memcached.
To be able to implement it, I needed to change coordinator_setup and get_coordinator function signature so we can pass start_heartbeat argument to it. This way coordination service automatically starts periodic heartbeats which tell the backend that a particular service (member) is still alive.
In addition to that, I added tools/list_group_members.py script which allows user to list active services.
Terminology
OpenStack tooz library exposes a couple of different terms. Here I describe how they fit into our implementation.

Group

Each service is part of a single group. In our case, group represents a service type (e.g. api, actionrunner, scheduler, etc.). To view all the running services of a particular type, we just need to list all the members in a particular group.

Member

A service is represented as a member which is part of a particular group.

Capabilities

Optional metadata associated with a service (service name, listen host / port, service type, hostname, pid and other info). In the future, action runners will also utilize this to advertise their capabilities (e.g. linux, windows, etc.).
Example Output
Example output of tools/list_group_members.py script when all the services are running using launchdev.sh script (to test it locally, I used the redis backend).
$ python ./tools/list_group_members.py --config-file conf/st2.dev.conf
Available groups (13):
 - actionrunner
 - garbagecollector
 - workflow_engine
 - resultstracker
 - stream
 - timer_engine
 - auth
 - sensorcontainer
 - api
 - scheduler
 - rulesengine
 - notifier

Members in group "actionrunner" (1):
 - localhost_21837 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21837, 'name': 'actionrunner'})
Members in group "garbagecollector" (1):
 - localhost_21841 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21841, 'name': 'garbagecollector'})
Members in group "workflow_engine" (1):
 - localhost_21834 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21834, 'name': 'workflowengine'})
Members in group "resultstracker" (1):
 - localhost_21871 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21871, 'name': 'resultstracker'})
Members in group "stream" (1):
 - localhost_21930 (capabilities={'listen_host': '<IP_ADDRESS>', 'name': 'stream', 'listen_port': 9102, 'hostname': 'localhost', 'pid': 21930, 'type': 'active'})
Members in group "timer_engine" (1):
 - localhost_21853 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21853, 'name': 'timerengine'})
Members in group "auth" (1):
 - localhost_21927 (capabilities={'listen_host': '<IP_ADDRESS>', 'name': 'auth', 'listen_port': 9100, 'listen_ssl': False, 'hostname': 'localhost', 'pid': 21927, 'type': 'active'})
Members in group "sensorcontainer" (1):
 - localhost_21849 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21849, 'name': 'sensorcontainer'})
Members in group "api" (1):
 - localhost_21925 (capabilities={'listen_host': '<IP_ADDRESS>', 'name': 'api', 'listen_port': 9101, 'hostname': 'localhost', 'pid': 21925, 'type': 'active'})
Members in group "scheduler" (1):
 - localhost_21846 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21846, 'name': 'scheduler'})
Members in group "rulesengine" (1):
 - localhost_21850 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21850, 'name': 'rulesengine'})
Members in group "notifier" (1):
 - localhost_21870 (capabilities={'type': 'passive', 'hostname': 'localhost', 'pid': 21870, 'name': 'notifier'})

TODO

[x] Changelog entry
[x] Tests

Should the heartbeat be started for the coordinator/locking driver instances? I believe in some implementations that if locks don't get a heartbeat then they are released.

@nmlaudy I think you are correct, good catch :+1:
I will also make that change, it will likely require test changes though. I already have some issues with this change (tests are getting stuck, even though I correctly shut down the coordinator everywhere, could also be a bug in a test driver we use).

@m4dcoder This PR is now ready for a review.
I've added and fixed tests (there was an issue with tests hanging / timing out) and new CLI commands st2 service-registry group list, st2 service-registry member list [--group-id=<group id>]).
Adding CLI commands was a bit PITA since those commands don't follow common "pack resource" pattern and in general that CLI and http client related code is not the prettiest of things.

On a related note^ - I propose changing the following RBAC CLI commands in a separate PR in the future:

st2 role list -> st2 rbac role list
```st2 role-assignment list->st2 rbac role-assignment list``

This way it will be immediately obvious those commands refer to RBAC. In the past, we didn't have a pattern established yet for double nested commands so we went with a simple top-level approach which is somewhat confusing.

Just a heads up - I also pushed a change so we start heartbeat process also in places where we use coordination backend only for locking - 98b77d1.
This is a safer default since it means it will work correctly with backends which are timeout based.

Closes #4015

@m4dcoder

Practice wise, we don't usually put an underscore at the end of a variable. This is confusing and error prone. Can you rename group_id_ to something else? Maybe item just for this list comprehension.

That's a relatively common pattern in Python world. Also when variable names would clash with built-ins such as id, etc.


Model stuck in the forward pass

Hi,
When I ran disp0, disp1, disp2 = model(input1,input2) in the forward pass (training phase), I did't get any error nor get the loss. The model basically got stuck in the first epoch. I'm not sure if it's the compile error or the CUDA error. Has anyone run into similar problems? Thanks!
Update:
It seems that there's something wrong with the sync batchnorm, and the forward pass got stuck in the first batchnorm layer.

Could you please provide more details?
Stuck at the beginning? Or after some iterations? at which iteration?
Did you tried the prediction scripts? Does testing also have this problem?
And also system settings information?
CUDA version, NVIDIA-Driver version and also GPU version, Centos or Ubuntu?

Could you please provide more details?
Stuck at the beginning? Or after some iterations? at which iteration?
Did you tried the prediction scripts? Does testing also have this problem?
And also system settings information?
CUDA version, NVIDIA-Driver version and also GPU version, Centos or Ubuntu?

Thank you for your reply! The model got stuck at the very beginning in the batchnorm layer and I just get around this problem by downloading and importing a new sync batchnorm implementation from https://github.com/vacancy/Synchronized-BatchNorm-PyTorch. This code is runnable now. I'm not sure if this implementation can achieve the same performance as the one in your model. I'll update if there's new feedback.

Please let me know the results after training 10 epoches on Sceneflow dataset.
You can use 240x624 (better) or 240x 528 with batchsize as 8 or 16.
If the test EPE is among 0.75-0.80, that should be OK.

I've encountered the same problem before and solved it by training on only one GPU. I guess it is a problem about data parallel, tensors participated in one operation must be on the same GPU otherwise self-lock occurs,  such as tensor A multiplies tensor B, but tensor A is on GPU 0, and tensor B on GPU 1.

@SongKaixiang  Which pytorch version do you use currently? Also can you let me know some information about your cuda&system&GPU type/version? Could you also try to replace the sync_bn with another one like @sh2439 and then try the multi-gpu traning again.
Or just change the sync_bn to norm bn to run the training.
So I could know whether it's a problem of the sync_bn.
Thanks.

@feihuzhang I am using pytorch 1.1.0 because it can compile sga and lga layers succesfully, and the code runs on tesla M40 with cuda10.0. Now it can successfully run in parallel with pytorch build-in BatchNorm2d and BatchNorm3d.

It's a hidden bug for sync_bn. Here is a temporary possible solution:
Adding some useless layers sometimes helps avoid the training freezing problem.
For example, uncomment or add useless such a layer to SGABlock (about line 257 in "GANet_deep.py")
self.conv_refine = BasicConv(8, 8, is_3d=True, kernel_size=1, padding=1)

Could you please provide more details?
Stuck at the beginning? Or after some iterations? at which iteration?
Did you tried the prediction scripts? Does testing also have this problem?
And also system settings information?
CUDA version, NVIDIA-Driver version and also GPU version, Centos or Ubuntu?

Thank you for your reply! The model got stuck at the very beginning in the batchnorm layer and I just get around this problem by downloading and importing a new sync batchnorm implementation from https://github.com/vacancy/Synchronized-BatchNorm-PyTorch. This code is runnable now. I'm not sure if this implementation can achieve the same performance as the one in your model. I'll update if there's new feedback.

HI, is the performance comparable with sync_bn provided by GANet?

Solution is to change the sync-bn to another version.

Install NVIDIA-Apex package https://github.com/NVIDIA/apex
$ git clone https://github.com/NVIDIA/apex
$ cd apex
$ pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
Revise the "GANet_deep.py":
add import apex
change all BatchNorm2d and BatchNorm3d to apex.parallel.SyncBatchNorm


Add lint

Keep in mind:

[X] Documentation updates
[X] Changelog updates

The code for the dp lint command is present in this PR. However, it cannot be used yet because of arise of the dependency hell — SQLFluff uses newest version of the regex library (although for now, it does not specify it (just regex = *) so the incompatibility can be missed at the build time) while copier in the 5.1.0 version requires regex to be in 2020 version, causing SQLFluff to fail at runtime.
Right now in the PR we reference the private fork of the copier (https://github.com/swtwsk/copier/tree/regex-update) that has an updated reference. However, it is not a maintainable solution, hence the decision to stop working on dp lint for now. People responsible of copier are working on the new 6.0.0 version that will not have such an outdated dependency; on the other hand, it will be using Jinja2 > 3.0, which is unsupported by dbt for now. In order to introduce dp lint correctly, we shall wait for both dbt and copier updates that will eventually get compatible.


add Array::sum spec

Addresses Array::sum for #39
nothing to implement as sum is already provided by Enumerable and Array includes Enumerable.

While Array does include Enumerable, and Enumerable does provide a sum method, it seems that MRI implements its own Array#sum method:
irb(main):001:0> [1].method(:sum)
=> #<Method: Array#sum(*)>
irb(main):002:0> [1].method(:sum).owner
=> Array

Do you think it is helpful for this spec to be committed and exercising the wrong method? I'm kinda torn here. Passing tests are usually just fine and dandy, but in this case, I think the passing test does a disservice.

If that's the case then I'd rather not merge this just yet and work on the method, I don't like the idea of a passing test giving you the impression that things are working well or else this could turn into a nasty little bug we might forget about eventually. On the other hand it might make sense for the implementation of sum to still delegate to the Enumerable one, so it claims ownership but we avoid duplicating the logic, what do you think?

There is a pattern of delegating work from Array to Enumerable. It's not perfect, but it's good enough for now. If you want to utilize it in this case, I'd be fine with that.
There is an example of delegating to an Enumerable method here: https://github.com/seven1m/natalie/blob/56a8941e72c2bd5bb0688a892dd4d90caab04e32/src/array_value.cpp#L987-L992

Looks very good, since it looks like it's been repeated multiple times and could be a more generic mechanism to delegate to a module, would it make sense to make it a protected method for Value?

No, it's a broken pattern that I want to stick out like it does. We need an alternative, but I don't have one yet.

Done, anyhow if you have time I'd be curious about why this pattern doesn't quite work, thanks!

Sure, here is why it doesn't accurately represent what MRI does:
module Enumerable
  def one?
    false # overridden by the user
  end
end

p [1].one? # => prints true in MRI

In Natalie, the above will print false because we do not have two separate implementations.


Notebook Cell Output not generated

Steps to repro

Latest VS Code Insiders
Custom branch of Jupyter extension (se below)
PR in Jupyter extension https://github.com/microsoft/vscode-jupyter/pull/7100

Use the VSIX generated in the above PR (one of the assets in the CI is the Jupyter VSIX)
Or use the corresponding branch of the above PR

Please note:
As we're no longer awaiting on the promises, any errors thrown will be ignored completely.
We don't even await on <Execution Task>.end
Steps to repro
Run the following cell, the expected outupt is 999\n Done: xxxx
Sometimes, we don't get anything in the cell output.
If you are unable to repro, please change the counter from 1000 to 10000 or 3000 or other.
No idea what the magic number is, for some reason 1000 repros the bug, but 10,000 does not!
from timeit import default_timer as timer
from IPython import display

start = timer()

for i in range(1000):
    display.clear_output()
    print(i)

end = timer()
print(f"Done: {end - start}")

@roblourens /cc

First, the assertIndex check for an output item edit is not quite right because it's valid for an extension to be editing an output that was removed, when this has happened in the main process but is not synced back to the EH.
@rebornix I'm just making this not throw. It doesn't necessarily indicate a bug, but could still be useful for troubleshooting, do you think it should be logged somewhere?
Second, sometimes the EH process gets stuck processing lots of edits when it is supposed to send an update every 10ms. Then the renderer is stuck processing a huge update. Fixing to guarantee that an update will be sent from the EH every x ms.
Then the rendering is still slow on the frontend.

Realized that this is not what I thought it was. What @rebornix said this morning made me realize that this should happen much more often than it does. Debugged more and realized that the extension does have special handling for this case, which is working. But sometimes the message gets split where the newline comes in a second update.
Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = CleanOutput
Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = StreamMessage Cell Index 0, Stream 'stdout 659

Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = CleanOutput
Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = StreamMessage Cell Index 0, Stream 'stdout 660

Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = CleanOutput
Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = StreamMessage Cell Index 0, Stream 'stdout 661
Info Jupyter Extension: 2021-08-18 15:50:03: KernelMessage = StreamMessage Cell Index 0, Stream 'stdout

Info Jupyter Extension: 2021-08-18 15:50:09: KernelMessage = CleanOutput
Info Jupyter Extension: 2021-08-18 15:50:09: KernelMessage = StreamMessage Cell Index 0, Stream 'stdout 662

Info Jupyter Extension: 2021-08-18 15:50:09: KernelMessage = CleanOutput
Info Jupyter Extension: 2021-08-18 15:50:09: KernelMessage = StreamMessage Cell Index 0, Stream 'stdout 663

When that happens, the extension sends an appendOutput, then a replaceOutputItems. NotebookTextModel can't handle getting those two in the same batch because it looks for a cell with the given output id, before the append has been processed.

This morning I mentioned another issue with getting outputs with an undefined data. I can't repro that today.


Laravel 4.2 issue

I am using this package to solve the laravel issue 4573.
Unfortunately it's not working for me. After some time swiftmailer will start to throw "bad write retry" error and I've to keep restarting supervisord to temporarily solve the issue.
Full error:
fwrite(): SSL operation failed with code 1. OpenSSL Error messages: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry
I am calling artisan like this from supervisord worker:
php artisan queue:work --queue=my_emails_tube --daemon --env=production
Any hints? Thank you.

I'm not an expert in these - but from my understanding, a problem is in OpenSSL, not SwiftMailer? Guys here as suggesting to lower down amount of data being sent - that is, you may be sending too much of stuff at once.
In either case - it would be good to ask @xdecock (SwiftMailer author) how to solve it, if that's possible.
A different (possibly, better?) solution would be to close SMTP connection at the end of job processing - see Coding For Daemon Queue Workers, by calling $transport->stop() - in such case, you don't need my package at all. If you'll try that, and problem persists - I would definately search for a problem in server itself.

PHP version?
seems like a strange bug, do you have any backtrace to provide?
it seems like an openssl odd things, that force write retry to not account the bytes returned as sent.
Could be complex to avoid.
And, indeed, do you have any timeline? like the duration of the process and the "idle time" between mails?
Regards,
Xavier

Hi @xdecock , @YOzaz,
I am running PHP 5.5.9 on ubuntu server 14.04 LTS.
Here is the log:
https://gist.github.com/jithujose/1ea9543f1a10e81d3a12
Unfortunately I don't have the process duration but the idle time between mails is normally really long (1+ hours).
Anyway I will try to see if by closing the SMTP connection I can solve the issue.
Regards,
JJ

yes, the 1+ hour will cause ssl session renegotiation in the best case,
connection RST more probably, and is probably your issue.
On Thu, Sep 3, 2015 at 12:33 PM, Jithu Jose<EMAIL_ADDRESS>wrote:

Hi @xdecock https://github.com/xdecock , @YOzaz
https://github.com/YOzaz,
I am running PHP 5.5.9 on ubuntu server 14.04 LTS.
Here is the log:
https://gist.github.com/jithujose/1ea9543f1a10e81d3a12
Unfortunately I don't have the process duration but the idle time between
mails is normally really long (1+ hours).
Anyway I will try to see if by closing the SMTP connection I can solve the
issue.
Regards,
JJ
—
Reply to this email directly or view it on GitHub
https://github.com/YOzaz/Laravel-SwiftMailer/issues/1#issuecomment-137404214
.

--
Xavier De Cock
GPG Fingerprint: 93CA EE3F 9F57 5BE1 AE4A  794D 3C74 CA9E E7A5 0C1B
GPG Id: 0xE7A50C1B

Goes out of the package scope, specific to SSL certificate validation.


Add CopyConsistency test

Implemented a test for both buffer-to-buffer and texture-to-texture copies as mentioned in https://github.com/thatcosmonaut/SDL/issues/63. Wasn't sure what to call it, so suggestions are welcome for a better name!

Test looks good on AMD Windows drivers, merging now.


Show count of registrations on the table

Make a complex object for the FE

API done.

Relates to #28


btf: Add bindings for BTF_KIND_FLOAT

Fixes: #34
Signed-off-by: Dave Tucker<EMAIL_ADDRESS>

Thanks for this Dave! Planning to compile a newer and test later today.

@alessandrod I've expanded this PR a little.

Added some test cases for parsing the various BTF types using data from /sys/kernel/btf/vmlinux on my system.
Made Btf::parse issue an error if we run in to an Unknown btf type, as that was what caused the infinite loop.

That last commit got quite big since the BtfType::Unknown variant was then never constructed... and therefore a whole lot of tunctions that were returning Option didn't need to any more.

Good stuff! See the comments then it's good to go


⚠ Fortnite has degraded performance

In ba8836d, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1621 ms

Resolved: Fortnite performance has improved in 80f089b.


[1pt] Switch badges for Missions - Conditional components

Missions that the user has joined already should show a badge "Active Member" instead of the default "NOT A MEMBER" and a button "Leave Mission" instead of the "Join Mission" button (as per design).
Use the React conditional rendering syntax:
{rocket.reserved && (
    // render Cancel Rocket button
)}

https://github.com/mikemtzp/space-travelers-hub/pull/35


Add code attribution for ecdsa sig formatter

This PR adds proper code attribution for the ECDSA signature formatter and also fixes a small typo in there.
Probably fixes https://github.com/blockstack/jsontokens-js/issues/11.

Thanks for adding this @vsund


Wrong documentation link

first link in:
https://github.com/ActiveCampaign/postmark-rails/wiki/Email-sending
should link to:
https://postmarkapp.com/developer/user-guide/send-email-with-api
(or https://postmarkapp.com/developer/user-guide/send-email-with-smtp)
instead of https://postmarkapp.com/developer/developer-build.html#attachments

@PhilippGsell thanks a ton for pointing this out! 🙏
This has now been fixed. 🙌


Add a Unit Test Job to the Build Stage of the pipeline

As a developer
I need to add a Unit Test Job to the Build Stage of the pipeline
So that   it will run units tests with nosetests
Assumptions:

The unit tests will need to run successfully before the project can be deployed to subsequent stages

Acceptance Criteria:
Given the Unit Test Job has been added
When the project is deployed to the build stage
Then it will run units tests

@crb618 Based on what I'm seeing in IBM Cloud - this story is done as well. Thanks again for all the hard work. Will move over to done.


Is it possible to show FontAwesome icons in tab titles?

I tried <tab name="Kitchen" prefix="<font-awesome-icon icon='fa-duotone fa-cauldron' />">  but that did nothing (FontAwesome as such is installed and works at other places in the body)

Hello,
I've been giving this quite a bit of thought - the v-html directive, which is used for rendering the title, does not support rendering components.
The only way this is possible is to pass a native HTML element to the prefix or postfix, in this case perhaps an SVG as a prop:
<Tabs>
  <Tab name="Kitchen" prefix="<svg class='svg-inline--fa fa-house' aria-hidden='true' focusable='false' data-prefix='fas' data-icon='house' role='img' xmlns='http://www.w3.org/2000/svg' viewBox='0 0 576 512'><path class='' fill='currentColor' d='M575.8 255.5c0 18-15 32.1-32 32.1h-32l.7 160.2c0 2.7-.2 5.4-.5 8.1V472c0 22.1-17.9 40-40 40H456c-1.1 0-2.2 0-3.3-.1c-1.4 .1-2.8 .1-4.2 .1H416 392c-22.1 0-40-17.9-40-40V448 384c0-17.7-14.3-32-32-32H256c-17.7 0-32 14.3-32 32v64 24c0 22.1-17.9 40-40 40H160 128.1c-1.5 0-3-.1-4.5-.2c-1.2 .1-2.4 .2-3.6 .2H104c-22.1 0-40-17.9-40-40V360c0-.9 0-1.9 .1-2.8V287.6H32c-18 0-32-14-32-32.1c0-9 3-17 10-24L266.4 8c7-7 15-8 22-8s15 2 21 7L564.8 231.5c8 7 12 15 11 24z'></path></svg> ">
  </Tab>
</Tabs>

I understand this solution might not be ideal, however as far as I am aware, Vue 3 does not support rendering components programatically.

Thanks - that's at least a way forward which makes it achievable (with a bit of hacking) 👍


Improve content type identification

Now only application/json is considered to be JSON but for example text/html; charset=utf-8 is also valid JSON content type.

Done.


updated rv32i tests to support all registers (x31) with assertions

updated spec/TestFormatSpec.adoc example ISA test with new assertions

Is this a clean resubmission of https://github.com/riscv/riscv-compliance/pull/41
If so, this was approved on Wednesday and I can commit it.

Just seen your message on https://github.com/riscv/riscv-compliance/pull/41.
This was approved by TG on 13 February 2019.


Sets higher bound for RSS in file_to_* experiments

What does this PR do?
Adjusts limits for these experiments, they seem to fail on the current state of the Agent.
Motivation
https://dd.slack.com/archives/C03LY8MQ73R/p1728631585134969
Describe how to test/QA your changes
n/a
Possible Drawbacks / Trade-offs
Additional Notes

/merge


something went wrong when i add new content-type to rule 900220

Description
i added new content-type (application/scim+json) to rule 900220 and uncomment this rule. But when i use this content-type in my request and Modsecurity block me. I get this log:
ModSecurity: Warning. Matched "Operator Rx' with parameter ^application/scim+json|application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/soap+xml|application/x-amf|application/json|application/octet-stream|application/csp-r (70 characters omitted)' against variable TX:0' (Value: application/scim+json' ) [file "/home/bwaf/modsec/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "894"] [id "920420"] [rev ""] [msg "Request content type is not allowed by policy"] [data "application/scim+json"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS"] [tag "OWASP_CRS/POLICY/CONTENT_TYPE_NOT_ALLOWED"] [tag "WASCTC/WASC-20"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/EE2"] [tag "PCI/12.1"] [hostname "<IP_ADDRESS>"] [uri "/scim2/Me"] [unique_id "1631266210"] [ref "o0,21v437,21"]
my content-type was added . But Mod still block this request. Please help me. i have to turn off this rule in my system
Your Environment
i use:

Core Rule Set ver.3.2.0
nginx proxy (18.04)

Confirmation
[ ] I have removed any personal data (email addresses, IP addresses,
passwords, domain names) from any logs posted.

Hi @ChuongTran97,
thank you for your report.
Can you provide a curl request and your modification to emulate this issue?
I switched back in my test environment to CRS 3.2 (3.2.1), activated the rule 900220 and inserted your CT:
396,402c396,402
< #SecAction \
< # "id:900220,\
< #  phase:1,\
< #  nolog,\
< #  pass,\
< #  t:none,\
< #  setvar:'tx.allowed_request_content_type=application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/soap+xml|application/x-amf|application/json|application/octet-stream|application/csp-report|application/xss-auditor-report|text/plain'"
---
> SecAction \
>  "id:900220,\
>   phase:1,\
>   nolog,\
>   pass,\
>   t:none,\
>   setvar:'tx.allowed_request_content_type=application/scim\+json|application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/soap+xml|application/x-amf|application/json|application/octet-stream|application/csp-report|application/xss-auditor-report|text/plain'"

Please note, that I've used application/scim\+json instead of application/scim+json.
With my request, I couldn't reproduce your issue:
curl -v --request POST -d '{"username":"aaa","password":"bbb"}' -H "Content-Type: application/scim+json" http://localhost```

Please check your added extra CT - hope this helps your problem. If it does, please close this issue.

I used your configuration and it work. My problem was solved Thank you very much.


Add ability to hit enter to submit url.

This should be a quick change involving changing the way the button and text box are implemented. It should be similar to what you did for the thesaurus web app. Look at this JavaScript and HTML.

Added as a part of commit #16.


[Issue]: it seems like crash after building Mechanic Stationton is done

Description of the bug/crash
it seems like crash after building Mechanic Stationton is done
disable this mod and no crash
NullReferenceException: Object reference not set to an instance of an object

IgnoreFetch.IgnoreFetchUtils.isFetchAllowed (Pickupable pickup, System.String chore, System.Boolean isArm) (at <cb6863303c8d4497871395099ba6ae53>:0)
IgnoreFetch.FetchChorePatch+<>c.<.cctor>b__5_0 (Chore+Precondition+Context& context, System.Object data) (at <cb6863303c8d4497871395099ba6ae53>:0)
Chore+Precondition+Context.RunPreconditions () (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
Chore.CollectChores (ChoreConsumerState consumer_state, System.Collections.Generic.List`1[T] succeeded_contexts, System.Collections.Generic.List`1[T] failed_contexts, System.Boolean is_attempting_override) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
FetchChore.CollectChoresFromGlobalChoreProvider (ChoreConsumerState consumer_state, System.Collections.Generic.List`1[T] succeeded_contexts, System.Collections.Generic.List`1[T] failed_contexts, System.Boolean is_attempting_override) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
GlobalChoreProvider.CollectChores (ChoreConsumerState consumer_state, System.Collections.Generic.List`1[T] succeeded, System.Collections.Generic.List`1[T] failed_contexts) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
ChoreConsumer.FindNextChore (Chore+Precondition+Context& out_context) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
Brain.FindBetterChore (Chore+Precondition+Context& context) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
Brain.UpdateChores () (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
Brain.UpdateBrain () (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
MinionBrain.UpdateBrain () (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
BrainScheduler+BrainGroup.RenderEveryTick (System.Single dt, System.Boolean isAsyncPathProbeEnabled) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
BrainScheduler.RenderEveryTick (System.Single dt) (at <93e6d51e7c504f4483a782aaaf7248c3>:0)
SimAndRenderScheduler+RenderEveryTickUpdater.Update (IRenderEveryTick updater, System.Single dt) (at <0a61a0e7beaf44b6b425814476821935>:0)
UpdateBucketWithUpdater`1[DataType].Update (System.Single dt) (at <0a61a0e7beaf44b6b425814476821935>:0)
StateMachineUpdater+BucketGroup.AdvanceOneSubTick (System.Single dt) (at <0a61a0e7beaf44b6b425814476821935>:0)
StateMachineUpdater.RenderEveryTick (System.Single dt) (at <0a61a0e7beaf44b6b425814476821935>:0)
Game.LateUpdate () (at <93e6d51e7c504f4483a782aaaf7248c3>:0)

Build: U52-623711-SC

Steps To Reproduce
MOD LIST and Mod Preset Manager in JSON
군단 09.json

Mod Updater
Stock Bug Fix
Help, none of my mod options are working after the DLC update - fix
Mod Profile Manager
Mod Preset Manager [MPM]
Always Show Capacity
Skills Retainer
Eat every day
传送服 Teleport Suit
AI Improvements
Critter Shedding
No Critter Overcrowding
Efficient Supply
Extra Moulding Skins
Food Supply Tooltip
Stats Unlimited
Sweep By Type
Auto Light Tile(自亮砖)
Capsule 2.0
Deselect New Materials
Duplicant Stat Selector [DSS]
MegaBattery
Better Air Filters
[DLC] Radiation Rebalanced
Endless Science
No Manual Delivery
Demolish Neutronium
Blood
Sick Go To Med Bed
Block Decor Effects Behind Walls
Passive Gas Vent Input
Plastic Utilities
Better Deselect
Butcher & Fishing Stations
Falling Sand
Wounded Go To Med Bed
Steel Ladder
Rebalanced Plant Mutations
ConveyorTiles
No Drop // by @Ony
Cluster Generation Manager [CGM]
Plastic Mesh and Airflow Tiles
Happy Digging: Dwarf Edition (Fixed)
Enclosed Telescope Tune-Up
Space POI Additional Info
Gigawatt Wire
Movable Features 位移信标
Endless Telescope
Big Storage
Utility Glass
Smelter
Wall Pumps
Control your Robots
[Vanilla + DLC] Biobot Upgrades
MutationFarmer
Universal Electrolyzer
Advanced Flow Management
Automated Canisters Reloaded
Pick up Flopping Pacu
Shinebug Reactor
主动洗手 Active Hand Washing
Portal Network
Critter Traits
Wrangle & Carry
Bio-Inks: Rechargeable Printing Pods
Research Queue
Fixed Ore Scrubber
Aquatic Farm
Buildable Neural Vacillator
Scan & Relay Satellites
Configurable Timers
Dupes Engineering
Trash Cans
Move This Here
Daily Routine
Rocketry Expanded
Freezer
Better Info Cards
Better Farming Effects and Tweaks
Packaged Bleach Stone
More Plants
Space Radiator
Custom Reservoirs
Sealed and Insulated Containers
Queue For Sinks
Sealed Pitcher Pump
Planning Tool
Bubbles Bathtub
Plastic Door
Self-sealing Airlocks Revived
Workshop Profiles
True Tiles
Hysteresis Storage
Rest for the Weary
Insulated Radbolt Joint Plate
Weeb Dupes
Tiny Door
Clothing Locker
Squirrel Wheel
Dupes Refrigeration
Insulated Joint Plate
Insulated Door
Ribbed Fire Pole
Decor Pack I
Build Over Plants (식물 위에 건설)
Show Building Ranges
More Small Sculptures
More Marble Sculptures
Wild Planter Box
Giant Doors
传送端口 (Transfer Port)
Extended Building Width
Geysers Automation Extension 间歇泉自动化扩展
More Emotions
[Vanilla + DLC] Rooms Expanded
Mechanics Station
AI Controlled Rockets
自然种植盒（Natural Plant Box）
DuperyFixed
NSHS MOD For Dupery
MedicineExpanded
Pip Plant Overlay
Athletic Manual Generator
Any Ice Liquefier
Simpler Pip Plant Rule (Vanilla/DLC)
Build Priority Keys
Overjoy Chance Rebalanced
Element Capture Capsule
[Vanilla + DLC] 게임 진행 도우미+ V2.0
Reapy Fixed
Rotatable Radbolt Chamber Output
Waste Not, Want Not
Forbid Items
Rebuildable AETN
Stairs
No Mop Limit
The Golden Throne
Missing Element Textures
Asphalt Tile
Attribute-based Workshop Restrictions
Where's My Loot?
Meteor Drops
Radioactive Wheezeworts
Asphalt Stairs / 柏油台阶
Backwalls
SaltBox 盐盒
Germicidal Lamps
Double Bed
Scaffolds
Festive Halloween Spices
Storage Refrigerator Thresholds
Sweepn't
[Vanilla + DLC] 고급 발전기+ V2.0
Echo and the Chaos Crew
[Vanilla + DLC] More Logic Ports
[Vanilla + DLC] 친환경 화장실 & 농사+ V3.0
Auto Suit Delivery
Crabs Profit
Carousel
Auto Remove Baby
Grind Up Metals
Smart Pumps
Fluid Shipping
Only Tame
Artifacts in Care Packages
Vitals Display Rads
High Pressure Applications
Graveyard Keeper
RTGenerator
Automation Bypass
Debris Thermostat 碎片恒温器
Balloon Dispenser
Worn Suit doesn't drop Canister
Smart Coolers
Resource Sensor
Thermo Sensor Tile
Suit Recharger

Reproducing bug in save file
군단 클래식 02 Cycle 40.zip

build any room
there is two Rockcrucher in the room
build mechanic station in the room
For detail i attached sav file

Player.log
Player.log
Player.log

IgnoreFetch is not part of the Mechanic Station or any of my other mods.
Contact the author of the corresponding mod. Sweepn't, I suppose.

Thank you for reply


Create hava.java

Adding a language

[X] The code displays "Hello World" (tio.run may help for testing)
[X ] I have updated the readme to include the new language
[X] I have incremented the language count in the readme
[X] I have no association with the language
[X] There are no copyright issues with this code
[X] The language has not been added prior to this pull request

Link to programming language:  https://github.com/oycca/hello-world/blob/main/h/Hava.java

We already have Java


Enable runtime integer overflow check in release build

I believe this is the recommendation from external security audit.

I see that we plan to address this more holistically later. Closing.

0346268


implement new syntax for type declaration

This is my attempt at implementing the new syntax as proposed in https://github.com/ocaml-ppx/ppx_import/issues/62.
It only changes the type declaration and not the module type declaration as the latter is already defined as a regular context-free rule.  However, and for consistency purposes, I think that it would make sense to also change the syntax of the module type declaration. So
module type Hashable = [%import: (module Hashtbl.HashedType)]
would become
[%%import: module type Hashable = Hashtbl.HashedType] or module type%import Hashable = Hashtbl.HashedType
I've some local changes that partially address that but if you agree, I would prefer to address that in a separate PR as it seems trickier/requires more changes to the actual implementation.

Thanks a lot for this PR @tatchi! From a ppxlib perspective and, as far as I can tell, also from a ppx_import perspective, the new syntax is a great improvement!
@ejgallego, will you have time to review this? There's no rush! I'm just checking in case you were expecting me to review or similar.

I can review this, but of course any help would be very welcome!

While I get cycles to look at this, adding a CHANGES entry would help too.

While I get cycles to look at this, adding a CHANGES entry would help @tatchi

I added a change entry. We should probably also adapt the examples in the readme. What syntax do you want to emphasize? [%%import ... ] or type%import ...
`

What would be the plan for this, should we start a 2.0 branch?
If so, should we allow ppx_import 1.x to be coinstallable with 2.x ?

Thanks @tatchi , no idea either on what would be the best path for the 2.0 branch, I cc @gasche and @kit-ty-kate in case they'd like to weight in.

Sorry for the very long time without answering!

We should probably also adapt the examples in the readme.

Yes, I agree.

What syntax do you want to emphasize? [%%import ... ] or type%import ... ?

My opinion on this is that type%import is the nicer syntax. However, of course, also this is up to the maintainers!

should we allow ppx_import 1.x to be coinstallable with 2.x ?

@ejgallego, that's a good question, and I'm probably not the right one to answer this. My naive way of seeing this is that it's both simpler and better if they're not co-installable. Better because that way the ecosystem is likely going to switch faster to 2.x once the first packages have started doing so.

I also have another remark. @pitag-ha mentioned the improvement in performance, due to not having to do another pass.
Unfortunately, since ppx_import has to be run as a staged_pps, its pass won't be merged with other ppx (apart from the staged ones)...

Oh, right!

Thanks for the careful review @panglesd 🙏

A test for [%%import type t = Location.t] (without the :) should be added

I adapted one of the tests to this syntax, and also one with type%import ..... See https://github.com/ocaml-ppx/ppx_import/pull/68/commits/e51599245f9ce89d7682fdd4268ac7229c7b18bc

A suggestion for supporting type%import t = Location.t and t2 = Location.t2

Thanks, that definitely makes sense and seems easy to support 😁. I made the changes in https://github.com/ocaml-ppx/ppx_import/pull/68/commits/19b230e68527f2f2984dd3580095f4ac82d0c306

An unrelated suggestion for error reporting!

Interesting, I'll read the doc and see if I can avoid raising errors (that will be for another PR)

Thanks for all the work folks, so I guess it is time to create a 1.x branch and start thinking about merging this in master.
I agree that having a 2.0 release with the same package name is simpler, however the opam repos would need a pretty large patch to introduce a < 2.0 constraint in all of ppx_import rev deps.
Is this the standard operating procedure?

Is this the standard operating procedure?

yes, it is totally normal and should be merged rather quickly. In this case you can even use opam admin add-constraint ppx_import<2.0 unconditionally given (if i understand correctly) everything is going to break

Thanks for the feedback @kit-ty-kate !
So I'm happy to go ahead with that then, @tatchi what's the status of this PR from your part?

So I'm happy to go ahead with that then, @tatchi what's the status of this PR from your part?

I should have addressed all the points @panglesd mentioned in https://github.com/ocaml-ppx/ppx_import/pull/68#pullrequestreview-1088712206. Except for the improvement of the errors for which I opened a separate draft PR https://github.com/ocaml-ppx/ppx_import/pull/73
There's also the README that needs to be adapted, but I believe we can do it in a separate PR too.

Great! I think then this is ready to go, I'll do a last review pass and some internal testing ASAP.

I am not sure however if we should merge this PR without an update to the README; personally I'd prefer to keep the doc in sync at all times.

I updated the readme with the new syntax. Note that the module type declaration syntax hasn't been modified (because it's already written as a context-free rule).
IMO, it would make sense to update it to module type%import ... to stay consistent. Maybe something to keep in mind before releasing a 2.0 ?

IMO, it would make sense to update it to module type%import ... to stay consistent. Maybe something to keep in mind before releasing a 2.0 ?

Yes! If you think it is a good idea to keep track of this, please open an issue and assign the 2.0 milestone to it.

Thanks a lot for all the work, I've created the v1.x branch for patches going to the 1.x series.


"Finished dungeon search with no seeds."

I just tried a few dungeon searches (no end pillars due to a different seed being used for the end). The first couple produced seeds that did not equal the server's seed, and the last few I've tried have resulted in the error listed in the title of my issue post. I've tried a few different combinations of finders to no positive effect. Any suggestions?
The exploring was done entirely in 1.15 chunks. The border was only expanded for 1.15, so I stuck next to it until I got results.

Is the dungeon floor intact?

i have the same error


fix check pkg bug

Refer to #693 , require_transformers_version and require_tokenizers_version have been deprecated. This PR fixed the functions in models/auto.py
@Qubitium @ZX-ModelCloud

Closed with https://github.com/ModelCloud/GPTQModel/pull/695
This block of code actually need to removed entirely.


Issue Running BigCache store and ristretto

Hi,
i discovered your awesome library today and im trying to run the examples on the readme.
i get the below errors when i try to run them
1)
`
bigcacheClient, _ := bigcache.NewBigCache(bigcache.DefaultConfig(5 * time.Minute))
bigcacheStore := store.NewBigcache(bigcacheClient, nil)
cacheManager := cache.New(bigcacheStore)
err := cacheManager.Set("my-key", "my-value", nil)
if err != nil {
panic(err)
}
value, err := cacheManager.Get("my-key")
log.Println(value, err)

`
im getting an error marked at  err := cacheManager.Set("my-key", "my-value", nil)
panic: interface conversion: interface {} is string, not []uint8
goroutine 1 [running]:
github.com/eko/gocache/store.(*BigcacheStore).Set(0xc00019c660, 0x773a20, 0xc000032000, 0x773a20, 0x86e010, 0x0, 0xc00000e201, 0xc000032000)
D:/Dev/GOPATH/pkg/mod/github.com/eko/gocache@v1.0.0/store/bigcache.go:62 +0x19d
github.com/eko/gocache/codec.(*Codec).Set(0xc00019c680, 0x773a20, 0xc000032000, 0x773a20, 0x86e010, 0x0, 0x875001, 0xc000012140)
D:/Dev/GOPATH/pkg/mod/github.com/eko/gocache@v1.0.0/codec/codec.go:51 +0x70
github.com/eko/gocache/cache.(*Cache).Set(0xc00016bed8, 0x773a20, 0x86e000, 0x773a20, 0x86e010, 0x0, 0x875860, 0xb18120)
D:/Dev/GOPATH/pkg/mod/github.com/eko/gocache@v1.0.0/cache/cache.go:39 +0xa2
main.main()
D:/Dev/GOPATH/src/gokit/examples/cache/main.go:17 +0x28e
exit status 2
Alternatively when im using the example in the benchmark directly on the store, it works
`
client, _ := bigcache.NewBigCache(bigcache.DefaultConfig(5 * time.Minute))
store := store.NewBigcache(client, nil)
	key := "test"
	value := []byte("value")

	store.Set(key, value, nil)

	val, err := store.Get(key)
	ba := []byte{}
	for _, b := range val.([]uint8) {
		ba = append(ba, byte(b))
	}
	log.Println(string(ba), err)

`

`
import (
"log"
"github.com/dgraph-io/ristretto"
"github.com/eko/gocache/cache"
"github.com/eko/gocache/store"

)
func main() {
ristrettoCache, err := ristretto.NewCache(&ristretto.Config{
NumCounters: 1000,
MaxCost:     100,
BufferItems: 64,
})
if err != nil {
panic(err)
}
ristrettoStore := store.NewRistretto(ristrettoCache, nil)
cacheManager := cache.New(ristrettoStore)
err = cacheManager.Set("my-key", "my-value", &cache.Options{Cost: 2})
if err != nil {
	panic(err)
}

value, err := cacheManager.Get("my-key")
log.Println(value, err)

cacheManager.Delete("my-key")

}
`
err:
.\main.go:23:48: undefined: cache.Options
i noticed options struct is in store package which is imported.
Could you please help

bigcache.Set(string, []byte) the example code got some mistake

cache.Options is a mistake too, I have create an PR to fix this, you can see then in the new README.md.
https://github.com/eko/gocache/pull/49

shinchan saved me life.
also the get function is a trainwreck.
its returning a byte array which im having to build to generate my result value
	bigcacheClient, _ := bigcache.NewBigCache(bigcache.DefaultConfig(5 * time.Minute))
bigcacheStore := store.NewBigcache(bigcacheClient, nil) // No otions provided (as second argument)

cacheManager := cache.New(bigcacheStore)
err := cacheManager.Set("my-key", []byte("my-value"), nil)
if err != nil {
    panic(err)
}

value,err := cacheManager.Get("my-key")
ba := []byte{}
for _, b := range value.([]uint8) {
	ba = append(ba, byte(b))
}
log.Println(string(ba), err)

Also i find the examples to be wildly inconsistent.
ristrettoCache, err := ristretto.NewCache(&ristretto.Config{
		NumCounters: 1000,
		MaxCost:     100,
		BufferItems: 64,
	})
	if err != nil {
		panic(err)
	}
	ristrettoStore := store.NewRistretto(ristrettoCache, nil)

	cacheManager := cache.New(ristrettoStore)
	err = cacheManager.Set("my-key", "my-value", &store.Options{Cost: 2})
	if err != nil {
		panic(err)
	}

	value, err := cacheManager.Get("my-key")
	log.Println(value, err)

the above now gives me the following error
2020/06/24 14:23:45  Value not found in Ristretto store

For the first, you have two choice:
// method1, type converting
value, _ := cacheManager.Get("my-key")
src := string(value)

// method2, using marshaler to Set and Get
marshalMgr, err = marshaler.New(cacheManager)
marshalMgr.Set("mykey", "myvalue")

value, _ := marshalMgr.Get("mykey")
// now you get value is an interface with string type underlying

As for the second, i must to take a look of ristretto and try your case ~

the ristretto cache is fine for me now. adding a few nano seconds delay did the trick.
the big cache method 1 you specific is fundamentally wrong.
the interface returns []uint8 value type. as i demonstrated in my example.
also the setter expects a byte stream, if i want to store numbers or structs i have to convert them to a []byte representation and pass to set and convert back on get. Code-Link
you have to process it into a byte array and use.
Isn't the whole point of using a library to reduce all the work and let the library do the work for me
value,err := cacheManager.Get("my-key")
ba := []byte{}
for _, b := range value.([]uint8) {
	ba = append(ba, byte(b))
}

you misunderstand ~, all I want to express is that if you want to pass params to bigcache.Set, you should pass []byte, so you can Get with []byte. string can be easy to convert to []byte, but if you have a complicated variable or a not easy to convert type, you should think about Marshaler to help you finish converting, rather than convert by yourself, this is what you want ~

Closing this issue as it does not seems to have an issue with the library.
Feel free to reopen if you still have any issue.


[IMPROVMENTS] Increase Space Between Talents in Talent Tree

Hi ! :) I love your work, and i wanna say, a huge thanks to you, to deliver us this amazing system. Can i ask an improvment ? It could be nice to have a better visibility on nodes between talents on a character talent tree. The space between two talents is very small, and it's a bit hard to see, what is connect to. Is it possible to boost this visibility ? Thanks by advance.

Done, will be released in 1.805. Thanks for the suggestion!

(top spacer is current, second spacer is new)


Exams

used eslint and prettier for both tasks;
renamed key1 and key2 to keys1 and keys2 for merge.js


[Accessibility] Agenda: Voice over does not convey the name of the image under Agenda screen.

Target Platforms
iOS
SDK Version
App Version: Version 1.0 (2.3.1-beta.20210603.1)
Application Name
Visualizer
Problem Description
33767249
Voice over does not convey the name of the image. It is announcing only as 'Image'. Voice over should convey the name of the image or make it non interactive so that focus should not land on it.
Screenshots
No response
Card JSON
Agenda.json
Sample Code Language
No response
Sample Code
No response

doesn't repro anymore

issue is still repro'ing in latest build. activating this issue
Test Environment:
Device: iPhone X
iOS:15.5
App Version: Version 1.0 (2.4.0-beta.20220620.1)
Screen Reader: Voice Over
https://user-images.githubusercontent.com/95915837/174777494-74979329-abe3-45dd-99de-2e7ba3c8f10d.mov

@jwoo-msft - care to take a look?
might be this one lonely altTextless image:
https://github.com/microsoft/AdaptiveCards/blob/648d7392bcf54c8d82d5d6cf6b3c8d87042529ad/samples/v1.2/Scenarios/Agenda.json#L306-L311

Working fine on the latest build hence closing this issue.


NIFI-6047 Add DetectDuplicateRecord Processor

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[ ] Has your PR been rebased against the latest commit within the target branch (typically master)?

[ ] Is your initial contribution a single, squashed commit?

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[x] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Note: #3298 is an alternative implementation to this one.

@adamfisher

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (check-style) on project nifi-standard-processors: You have 7 Checkstyle violations. -> [Help 1]

That's why your build failed. To resolve this quickly run this from the standard-processors module:
mvn clean install -DskipTests=true -Pcontrib-check
The contrib-check profile will run RAT and tell you what the style violations are.  We enforce that to help keep things clean.

You will definitely need some unit tests for this before we could proceed with a real review. I did a preliminary look, and it covered a lot of ground. Seemed like a solid contribution so far. Probably best to take this to the user list and hash out the use case to make sure that we cover the bases here. When I raised the issue, I think I neglected to mention that my team was working with UUIDv5 IDs for records, so that (and UUIDv3) need to be hashed out.

@MikeThomsen if I am understanding your requirement, you have records containing UUIDs and wish to use them to determine uniqueness of your records. You should be able to assign a single user-defined property where the name is the record path of the field containing the UUID.

@adamfisher  @MikeThomsen I know you have different PRs/approaches here and a discuss thread but I think you two are the current parties of interest and effort here.  Can you two please find consensus and lets figure out what to do with these two PRs.
I haven't looked into this in great depth but my initial pass left me thinking the approach Adam took would serve as a nice option covering both precise detection and statistically 'good enough' for cases that allow that.

@adamfisher any ETA on the tests?

I don't expect this build iteration to pass. Still having issues running tests in IntelliJ. It complains about an unrelated area of NiFi RecordPath.java is missing dependencies when I attempt to run the tests I added. Mike I merged some of your tests in because you had good ideas and was a lot cleaner than the TestDetectDuplicate file.

@adamfisher
Sorry about not getting back sooner. Been very busy with a lot of work projects. Appears you missed a few files in your commit:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:testCompile (default-testCompile) on project nifi-standard-processors: Compilation failure: Compilation failure:
[ERROR] /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestDetectDuplicateRecord.java:[45,12] error: cannot find symbol
[ERROR]   symbol:   class MockCacheService
[ERROR]   location: class TestDetectDuplicateRecord
[ERROR] /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestDetectDuplicateRecord.java:[76,20] error: cannot find symbol
[ERROR]   symbol:   class MockCacheService
[ERROR]   location: class TestDetectDuplicateRecord
[ERROR] -> [Help 1]
[ERROR]

MockCacheService exists here:
https://github.com/apache/nifi/pull/3317/files#diff-cd27e5b5517d1b4636eb8184be641d9b
I'll try pushing again.

It is there, but it needs to be under the Groovy folder for Maven to consider it to be Groovy. It also appears that the code simply isn't compiling because the Groovy Eclipse Compiler plugin isn't handling it. I would recommend rewriting it as a Java class. Also, need to rebase against master because this branch is pulling 1.9.0-SNAPSHOT.

Also, when I made those changes locally, I got this running the unit tests:
[ERROR] Errors:
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"
[ERROR]   TestDetectDuplicateRecord.setup:89 » NumberFormat For input string: "0.10"

I'm not sure how a rebase works with a remote master I have no control over. I did invite @MikeThomsen and @joewitt to this branch. You should have received invites at https://github.com/adamfisher/nifi/invitations and your help is much appreciated.

@adamfisher

Checkout master locally.
Use git pull <remote> master to pull the apache master.
 You brought in about 100 odd commits from other people by merging master.

@MikeThomsen I tried following your steps for rebasing and I thought it all went ok but I seem to have a lot more commits now. Would you be able to advise? I'm not a git expert.

This is the general flow from your feature branch:

git checkout master
git pull apache master [Or whatever you call our remote]
git checkout NIFI-XYZ
git rebase master
git push origin NIFI-XYZ --force

Also, when you run git log you should see all of your commits cleanly placed at the top with all of the latest from master below them.

@mattyb149 @joewitt Been a while since I looked at this, but I think I was mostly +1 on this. We have a strong need for this, so I'll take over the PR and close the gaps.

Last I remember getting the tests running but it was really hard to get the whole framework to build. It's super close. Please let me know if you need anything. I did add commit permissions to this Branch if you want to work on this one. If I recall really it's just some test tweaking and then it's ready for merge.

@adamfisher A decent bit has changed in our git history since 6/2019, including us switching to main from master. I tried doing a rebase first to master and then to main yesterday and ran into some significant problems with a few commits you did in there that have "no message" as their subject line. The solution that worked for me was to squash your core commits (the top ones with good messages on them) and then cherry pick them to a branch based off of main.
After all that, the unit tests failed for various reasons including your handling of flowfile io streams. Not a big deal. I'll try to clean this up for you and push a new branch up that preserves your contribution so we can get past the Git issues and such. I'll tag you and you can just pull my copy if you want to make changes and then send updates to my branch instead of fighting with Git.

@adamfisher if you want to do this yourself, do the following:

Fetch the NiFi main branch.
Create a new branch.
Run git apply --check <PATCH_FILE>
Run git apply --stat <PATCH_FILE> (if the above shows no errors)

0001-Added-NiFi-DetectDuplicateRecord-standard-processor.patch.txt

I'm sorry but I don't know what to use for the patch file. Total newb to git when it comes to these more advanced scenarios.

@adamfisher
Alternatively, you can squash all of your commits and together (make sure to only do yours) and then run git cherry-pick <commit sha1> from a new branch based off of main.

@adamfisher I had to fix a few things locally. Not sure if it was due to API changes over the last year, but I have your contribution built and passing the tests locally. Overall, I'm +1 on your contribution. However, since we have two implementations of the same idea there needs to be some reconciliation of the two. I'm going to work on that, possibly under my branch/Jira ticket.
In the end, I think what we're going to do here is have three strategies:

Hash everything.
Bloom Filter
The UUID strategy I mentioned.

I'm going to talk to a colleague of mine who is working with me on a related use case and see what sort of implementation makes sense for #3.

It's been a while since we discussed this so I'm not familiar with the uuid strategy. if you are talking about the uuid on a flow file then that could be specified using the cache key identifier option already. User defined record path values could also be used to capture a uuid that appears in the record data.

@adamfisher I've been spending some more time getting back into your contribution, and there are some core issues that I have noticed:

It looks like the cache identifier cannot be set at the record level.
You're serializing the entire a very big value (either a bloom filter or a very large hashset)
Based on the logic that I see in how you're handling this, DetectDepublicateRecord as implemented here cannot filter across record sets.

#2 is a potentially big problem for HBase or Redis, as you could be stuffing several MB to dozens of MB of binary data into a single column/kv pair.
#3 is also a big issue because a large part of the use case revolves around detecting duplicates across an enterprise. For example, if two users submit a data set with a bunch of overlapping data it should be able to use the one data set to deduplicate the other.
I am leaning toward reopening my PR, but regardless I'd like to say this was a great first shot even with the aforementioned issues.

The cache key identifier could be used to identify the grouping of the data set and used as a key prefix. It sounds like it's just a matter of how we store record level hashes. The bloom filter stored in one cache record is necessary but the size is fixed regardless of the data set size. It's only driven by the capacity set on the bloom filter during the initial configuration. We could set that value to have a maximum to help protect the user from corruption. Storing individual hash sets for each record is obviously an option with this implementation already but it's good to be able to use Bloom filters because I think there will be scenarios for people don't care about exactness in unique records because it's primarily used to eliminate a lot of duplicate data processing.

Broadly speaking, we have two use cases that don't overlap that much: deduplication over one file vs over a data lake. Given the fact that NiFi follows a Unix-like philosophy of "simple tools that chain well together," I think the solution we're headed toward may be two processors.
I think this processor could work out if we pare it back to in-memory deduplication of a single record set. That way users won't have to turn a lot of knobs and dials to configure it. Then combined with my submission which would require a DMC as it is focused on the entire data lake I think we could hit both use cases with a targeted tool that is fairly intuitive.
Thoughts?

Yeah I like where you're headed with that. Are you thinking hashing across
data sets would be the hashing implementation? Then we would have a
separate detective wicked record processor for the bloom filter
implementation which would be an in memory one.
My time now is limited working on this since the use case has come and gone
for it. I was really hoping to get this pushed through last year before but
since git jujitsu tripped me up I was never able to get it into the main
line.
On Mon, Oct 26, 2020, 2:31 PM Mike<EMAIL_ADDRESS>wrote:

Broadly speaking, we have two use cases that don't overlap that much:
deduplication over one file vs over a data lake. Given the fact that NiFi
follows a Unix-like philosophy of "simple tools that chain well together,"
I think the solution we're headed toward may be two processors.
I think this processor could work out if we pare it back to in-memory
deduplication of a single record set. That way users won't have to turn a
lot of knobs and dials to configure it. Then combined with my submission
which would require a DMC as it is focused on the entire data lake I think
we could hit both use cases with a targeted tool that is fairly intuitive.
Thoughts?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/nifi/pull/3317#issuecomment-716743057, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAN75V2T2OBVX3JEIW27JFLSMW57TANCNFSM4GX5ED6A
.

I'm thinking that your processor would be limited to operations across a single record set (ie one flowfile). Mine would be slower, as it has to hit a DMC each pass, but it would be focused on deduplication across a whole data lake.

@joewitt @mattyb149 I have no problem taking over the cleanup for @adamfisher. Would one of you be up for a quick review once I have time to pare it back a bit and get it up on a new branch based off of main?

@adamfisher I've spent a few hours today playing around with this and integrating our two use cases into a single processor. Barring unforeseen circumstances, I should have a PR up by about tomorrow lunch time that has your contribution with my additions layered on top.


adding hudson high school of learning technologies

Hudson High School of Learning Technologies in NYC
hudsonhs.com

@marufaytekin First stage of verification passed, official domain is hudsonhs.com as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@philipto Any update on this request?

@marufaytekin Thank you for your patience while waiting for a review and processing your request. Pull request merged.


feat: Add Cisco ISE 3.0 driver

Adds the ability to create internal users in the ISE 3.0 API

wow names are unique? but many people have the same first and last name - what happens when we need to add a user with a same name as an existing user?

wow names are unique? but many people have the same first and last name - what happens when we need to add a user with a same name as an existing user?

The link below demonstrates how we generate an user name which is unique for each user.
https://github.com/PlaceOS/drivers/blob/07dfcf26d1390f31392f904dddf05577f9b3fed1/drivers/cisco/ise/models/internal_user.cr#L7
An example would be as such:
internalUser-4eed6c54fed4


[torch.compile] fast inductor

directly bypass aot-autograd and inductor, and load from cache

now, even if we compile for all sizes, the compilation time becomes negligible:
$ vllm serve meta-llama/Meta-Llama-3-8B --disable-log-requests -O "{'level': 3, 'candidate_compile_sizes': [$(seq -s, 1 1 256)]}"
INFO 12-11 14:04:25 backends.py:363] Dynamo bytecode transform time: 4.59 s
INFO 12-11 14:04:28 backends.py:155] Compiling a graph for general shape takes 2.77 s
INFO 12-11 14:04:32 backends.py:158] Compiling a graph for shape 256 takes 0.48 s
INFO 12-11 14:04:33 backends.py:158] Compiling a graph for shape 248 takes 0.48 s
INFO 12-11 14:04:34 backends.py:158] Compiling a graph for shape 240 takes 0.49 s
INFO 12-11 14:04:35 backends.py:158] Compiling a graph for shape 232 takes 0.37 s
INFO 12-11 14:04:36 backends.py:158] Compiling a graph for shape 224 takes 0.38 s
INFO 12-11 14:04:37 backends.py:158] Compiling a graph for shape 216 takes 0.47 s
INFO 12-11 14:04:37 backends.py:158] Compiling a graph for shape 208 takes 0.32 s
INFO 12-11 14:04:38 backends.py:158] Compiling a graph for shape 200 takes 0.47 s
INFO 12-11 14:04:39 backends.py:158] Compiling a graph for shape 192 takes 0.39 s
INFO 12-11 14:04:39 backends.py:158] Compiling a graph for shape 184 takes 0.34 s
INFO 12-11 14:04:40 backends.py:158] Compiling a graph for shape 176 takes 0.36 s
INFO 12-11 14:04:41 backends.py:158] Compiling a graph for shape 168 takes 0.49 s
INFO 12-11 14:04:42 backends.py:158] Compiling a graph for shape 160 takes 0.52 s
INFO 12-11 14:04:43 backends.py:158] Compiling a graph for shape 152 takes 0.47 s
INFO 12-11 14:04:44 backends.py:158] Compiling a graph for shape 144 takes 0.41 s
INFO 12-11 14:04:44 backends.py:158] Compiling a graph for shape 136 takes 0.33 s
INFO 12-11 14:04:45 backends.py:158] Compiling a graph for shape 128 takes 0.53 s
INFO 12-11 14:04:46 backends.py:158] Compiling a graph for shape 120 takes 0.33 s
INFO 12-11 14:04:47 backends.py:158] Compiling a graph for shape 112 takes 0.48 s
INFO 12-11 14:04:48 backends.py:158] Compiling a graph for shape 104 takes 0.51 s
INFO 12-11 14:04:48 backends.py:158] Compiling a graph for shape 96 takes 0.53 s
INFO 12-11 14:04:49 backends.py:158] Compiling a graph for shape 88 takes 0.54 s
INFO 12-11 14:04:50 backends.py:158] Compiling a graph for shape 80 takes 0.52 s
INFO 12-11 14:04:51 backends.py:158] Compiling a graph for shape 72 takes 0.59 s
INFO 12-11 14:04:52 backends.py:158] Compiling a graph for shape 64 takes 0.57 s
INFO 12-11 14:04:53 backends.py:158] Compiling a graph for shape 56 takes 0.51 s
INFO 12-11 14:04:54 backends.py:158] Compiling a graph for shape 48 takes 0.42 s
INFO 12-11 14:04:55 backends.py:158] Compiling a graph for shape 40 takes 0.56 s
INFO 12-11 14:04:56 backends.py:158] Compiling a graph for shape 32 takes 0.44 s
INFO 12-11 14:04:57 backends.py:158] Compiling a graph for shape 24 takes 0.47 s
INFO 12-11 14:04:57 backends.py:158] Compiling a graph for shape 16 takes 0.47 s
INFO 12-11 14:04:58 backends.py:158] Compiling a graph for shape 8 takes 0.47 s
INFO 12-11 14:04:59 backends.py:158] Compiling a graph for shape 4 takes 0.52 s
INFO 12-11 14:05:00 backends.py:158] Compiling a graph for shape 2 takes 0.51 s
INFO 12-11 14:05:02 backends.py:158] Compiling a graph for shape 1 takes 1.43 s
INFO 12-11 14:05:02 monitor.py:31] torch.compile takes 24.54 s in total

cc @ProExpertProg @bnellnm

throughput benchmark:
# run baseline first, to find out the number of scheduler steps to keep gpus busy
$ python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B
Throughput: 30.52 requests/s, 12620.07 total tokens/s, 6052.88 output tokens/s

$ python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B --num-scheduler-steps 8
Throughput: 43.57 requests/s, 18018.38 total tokens/s, 8642.04 output tokens/s

$ python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B --num-scheduler-steps 10
Throughput: 44.04 requests/s, 18212.79 total tokens/s, 8735.28 output tokens/s

$ python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B --num-scheduler-steps 12
Throughput: 44.65 requests/s, 18464.38 total tokens/s, 8855.95 output tokens/s

$ python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B --num-scheduler-steps 16
Throughput: 45.03 requests/s, 18622.54 total tokens/s, 8931.81 output tokens/s

$ python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B --num-scheduler-steps 20
Throughput: 44.18 requests/s, 18269.12 total tokens/s, 8762.30 output tokens/s

# the best number of scheduler step is 16, run this setting with `torch.compile`
python benchmarks/benchmark_throughput.py --dataset ShareGPT_V3_unfiltered_cleaned_split.json --model meta-llama/Meta-Llama-3-8B --num-scheduler-steps 16 -O "{'level': 3, 'candidate_compile_sizes': [$(seq -s, 1 1 256)]}"
Throughput: 46.93 requests/s, 19405.53 total tokens/s, 9307.35 output tokens/s

Now we don't need to profile the batchsize distribution anymore. We can just compile for all sizes.

I think there might be something wrong inside Inductor when I try llama 3 70B model (with tp 4). Even when I directly cache the graph hash, every shape takes 90 seconds to load the graph.
I will stop here as the results on llama 3 8B are quite good, compilation time is greatly reduced. The llama 3 70B case should be a bug to fix in the future.

I think InductorHashCache is a nice abstraction but it shouldn't live inside config (both file- and structure-wise).

@ProExpertProg I agree. I moved InductorHashCache to the backends.py now. For the structure-wise, I think we should add some VllmState in the future, instead of adding all fields in the config. But that needs a large refactor though, beyond the scope of this current PR.

I think I should addressed most comments now, let me know if you have any other concerns @ProExpertProg @tlrmchlsmth
One thing I'm not sure, is whether I should redirect inductor code cache and triton cache to our vllm cache directory or not.
Upside1: Sometimes (rarely) I find it is possible to have hash collision. Because we directly load fx graph from cache here, we don't have any check for hash collision. Redirecting inductor code cache and triton cache to our vllm cache directory is better for soundness.
Upside2: the vllm cache directory becomes self-contained, users only need to copy the whole directory for fast deployment.
Downside: different models cannot share max-autotune cache (mainly for gemm performance tuning). The code start time will be longer.

Downside: different models cannot share max-autotune cache (mainly for gemm performance tuning). The code start time will be longer.
Why can't models share the autotune cache anymore?

@ProExpertProg I plan to set TORCHINDUCTOR_CACHE_DIR to the cache directory we generate, this environment variable controls all of inductor cache.

we could hash everything in the configs instead of only hashing certain member variables. I think a lot of these could affect the compute graph in unexpected ways.

there's trade-off here. if we hash everything, then users need to recompile for some unnecessary changes, e.g. --num-scheduler-steps . I would still vote for only hashing certain variables we know they will affect the computation.

I think we should make the inductor cache opt-in for users.

If we hash lots of the variables, do you think it makes sense to make it on by default, and add a flag to disable it? I'm thinking about the end-user experience, compilation time is really important for user experience, and it would be great if we can turn it on by default.

I would still vote for only hashing certain variables we know they will affect the computation.

Maybe we can hash everything in the config classes by default, but exclude certain items that we know are safe?

If we hash lots of the variables, do you think it makes sense to make it on by default, and add a flag to disable it?

If we're confident that we're hashing everything relevant then, I agree it makes sense to be opt-out.

In the future, I plan to dump more information in the cache directory, including:

the transformed bytecode from Dynamo
the human-readable fx graph after every transform
the human-readable fx graph that inductor compiles
the human-readable executable python file inductor finally run

the goal is to make torch.compile crystally clear, we can know the exact correspondence of the compiled code and our original code, so that we can easily debug any potential correctness / performance issues.


Typo: schmd -> schomd

I think this is the cause of ZotPlus/zotero-citations#11

How the ^*@# did this slip by?! Thanks!


.github folder

Creating a .github folder should help new developers jumping in the project.
See: https://github.com/blog/2111-issue-and-pull-request-templates
I'll build it when i have time, but in the meantime, anyone can build it, if want.

[ ] CONTRIBUTING.md - To help new people join the project
[ ] ISSUE_TEMPLATE.md - To guide creation of new issues
[ ] PULL_REQUEST_TEMPLATE.md  - To guide creation of new pull requests


Why is the FindBugs plugin still built into Gradle?

The FindBugs tool hasn't been developed since 2015, and there is a newer tool SpotBugs which replaced it.  FindBugs doesn't support any Java version after Java 8.  Now that Java 11 is out, FindBugs is starting to look pretty antiquated.
It seems like Gradle should stop including the FindBugs plugin, and maybe consider to start including the SpotBugs plugin instead.  By including FindBugs in both the releases and the documentation, Gradle is spotlighting a tool that really no new project should even be considering.  Any new project starting up on Gradle could easily be led astray, assuming that because mighty Gradle builds in support for FindBugs that FindBugs must be a good tool to invest in (in 2018 and beyond)... wrong!  Why not give props to SpotBugs, or at least some other code inspection tool that is still being actively developed, and will have support for newer Java versions?

Its deprecated in the next release and recommends Spotbigs https://docs.gradle.org/nightly/userguide/findbugs_plugin.html
We can't rip it out immediately, so it'll be included until 6.0.

Related to #6652.


Mark Provider types as ineligible for convention mapping

Convention mapping does not work for Property-based properties
Convention mapping can work with Provider-based properties
Currently, this will only emit a deprecation warning

Fixes #8404
Context

Contributor Checklist

[ ] Review Contribution Guidelines
[ ] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[ ] Make sure all contributed code can be distributed under the terms of the Apache License 2.0, e.g. the code was written by yourself or the original code is licensed under a license compatible to Apache License 2.0.
[ ] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[ ] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[ ] Provide unit tests (under <subproject>/src/test) to verify logic
[ ] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[ ] Ensure that tests pass sanity check: ./gradlew sanityCheck
[ ] Ensure that tests pass locally: ./gradlew <changed-subproject>:quickTest

Gradle Core Team Checklist

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation
[ ] Recognize contributor in release notes

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@donat PTAL, I think this is good to go.
I updated the spec with some links to this PR. I found a couple of things that I didn't expect in the original spec:

The tests for Property/ListProperty/MapProperty ignore the value of the convention mapping. This appears to have been this way for awhile, which means that plugins can only really try to use convention mapping with Provider-typed properties.
The changes for AsmBackedClassGenerator couldn't go into the mixInConventionAware method because that isn't called for a lot of types with convention mapping. I had to call conventionMapping.ineligible(...) in the constructor for the class.


Token allocate

This PR adds functionality in the DAO creation process to allocate tokens to be minted to addresses.
It includes error handling to ensure valid Ethereum addresses have been entered, and that the sum of the token allocations is not greater than the specified token supply.

Please fill out the relevant metadata

Added PR Metadata

Please merge master into this branch, thank you

Merged master into this branch

@tbwebb22 only last thing here is to address the merge conflict


Zenodo Repository is not accessible

Hi,
I'm wondering if the Zenodo DOI:10.5281 is not made publicly available:

ERROR RESOLVING DOI
INVALID SSL CERTIFICATE
The origin web server does not have a valid SSL certificate.

Hi Yi-Ching,
I just checked the link and had a colleague do so as well and we could both access it, also it says there's been 14 downloads on the Zenodo. This may be a problem specific to your connection, have you tried using a different browser?
Best, Talal

It works now. Thank you!


ButtonToggle: V2 Development

Summary
What changed?
Introduced hair variant to ButtonToggle.
Why?
To bring web implementation in line with Figma specification.
Links

Jira
TDD
Figma

Checklist

[ Yes ] Added unit and Flow Tests
[ Yes ] Added documentation + accessibility tests
[ Yes ] Verified accessibility: keyboard & screen reader interaction
[ Yes ] Checked dark mode, responsiveness, and right-to-left support
[ Yes ] Checked stakeholder feedback (e.g. Gestalt designers, relevant feature teams)

I think we need to handle borders differently

Can we add here all the variants?

I think we need to handle borders differently

It shakes the surrounding ones, I believe because we are increasing the size of the wrapping borders

I tackled this with two changes:

The background color change made the border changes in most cases unecessary (no border in black selected states).
The only case with a variable size border remaining is the disabled button, and i added some css to offset the component size to compensate for the border size increase.


MemoryError: Unable to allocate array with shape (26577,) and data type <U1847338

base_extractor = Extractor(
File "/home/dragnet/dragnet/model_training.py", line 103, in train_model
train_html, train_labels, train_weights = extractor.get_html_labels_weights(training_data)
File "/home/dragnet/dragnet/extractor.py", line 124, in get_html_labels_weights
return np.array(all_html), np.array(all_labels), np.array(all_weights)
MemoryError: Unable to allocate array with shape (26577,) and data type <U1847338
i try with length data more than 20K and show memory error in numpy.array, is that problem in features engineering before fitting proccess?

maybe its too large str in all_html[0] more than 1000K characters


flutter build appbundle command

flutter_01.log

FileSystemException: FileSystemException: Cannot copy file to 'C:\Users\bened\Desktop\flutter_appkkk\ppp\build\app\outputs\bundle\app.aab', path = 'C:\Users\bened\Desktop\flutter_appkkk\ppp\build\app\outputs\bundle\app.aab' (OS Error: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
, errno = 32)
Is the app.aab used by another process?
You can quit all related processes and rerun the procedure to see if it helps?
@benedicte69

:) thanks, may be it was. I do not remember. I was trying to editing bundles in AS and I have been invited to send this error. So I sent it.
Envoyé depuis Yahoo Mail pour Android
Le mar., juin 25, 2019 à 9:02<EMAIL_ADDRESS>a écrit :
FileSystemException: FileSystemException: Cannot copy file to 'C:\Users\bened\Desktop\flutter_appkkk\ppp\build\app\outputs\bundle\app.aab', path = 'C:\Users\bened\Desktop\flutter_appkkk\ppp\build\app\outputs\bundle\app.aab' (OS Error: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
, errno = 32)
Is the app.aab used by another process?
You can quit all related processes and rerun the procedure to see if it helps?
@benedicte69
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

ok, then I think it's safe to close the issue. If you disagree or encounter it again, please comment and I will re-open it, or file a new issue.


Supported ESP-IDF versions

Hi! At the moment we support v4.4 and v5.1 in the template. Wanted to hear your opinion about two topics:

Shall we keep v4.4?

IIRC, there was some discussion of getting rid of v4.4 and its also EOL by July 2024
I don't have any hard preference on this topic

Update v5.1 to v5.2

Since v5.2 the latest stable, I think we should probably update to it

cc: @ivmarkov @Vollbrecht @esp-rs/espressif

My question
Does the version control of @esp-rs project just progressively follow Espressif's changes, or does it allow to keep legacy versions accessible? (LTS)
Assuming that version 5.1 has a relevant percentage of support and tools that use this version.
How would you maintain compatibility between versions when the upstream is in a much newer version with changes to the API?

How would you maintain compatibility between versions when the upstream is in a much newer version with changes to the API?

It depends =) In the first layer we have the direct dependency on esp-idf and the generated bindgen output. In the past we needed some rust specific patching that would not make sense to upstream. So we did it on the fly inside esp-idf-sys. This step is only because of "fundamental" problems.
Then there are general api changes and additions. The goal of our safe wrappers around the raw C api is not just to replicate the interface in most cases, but create a interface that makes more sense from a rust usage pattern. In most cases that leads to our public API to be general enough to survive underlying changes in esp-idf itself. We have lots of conditional compilation inside it to make it work, though yes its not perfect and the public API still leaks the underlying esp-idf version itself in some places.
When esp-idf introduces completely new drivers we just add it when its introduces and not backporting it. And esp-idf is stable enough that they just don't get rid of something before deprecating it. ( like the timer and rmt api's for example between v4 and v5)

Does the version control of https://github.com/esp-rs project just progressively follow Espressif's changes, or does it allow to keep legacy versions accessible? (LTS)

Currently we are testing only the latest v4 and v5 versions of esp-idf ( still transitioning to v5.2) and the git master. Technically we still could run esp-idf v4.3.x stuff but no one tested it in a long time and we marked it deprecated like have a year ago. We are not rigorous following the official release support schedule, but at least we more or less always stay on top of the latest releases. Since the esp-idf-* part of esp-rs is community driven there are also no grantees for anything here, we just do it for the fun =)


Submit PR for SwiftyJson change

Before we can submit, we need to confirm that we conform to IBM polices

We have moved this to the backlog to wait and see what happens when the automatic bridging of various Swift classes to Foundation classes is enabled on Linux.
We want to then understand the impact to the NSJSONSerialization class.

Closing - Reopen as needed.


Only show relevant graphs

This ensures that graphs covering multiple lines are only generated when there is more than one line to show and, more importantly, that quota graphs are not shown for lines without a quota - that in turn stops the plugin failing with an exception when presented with a line on a units tariff.

Many thanks for the contribution @tomhughes!


Failed to build - github CI action?

I can't get the project to build as you've described in the README.
Is there a supported range of gradle & java versions that need to be documented in the README?
is it possible to add a .github/workflows/gradle.yml build action? https://docs.gradle.org/current/userguide/github-actions.html
something that shows reproducibly how to build this extension?
gradle fatJar

FAILURE: Build failed with an exception.

* What went wrong:
Could not create service of type ScriptPluginFactory using BuildScopeServices.createScriptPluginFactory().
> Could not create service of type PluginResolutionStrategyInternal using BuildScopeServices.createPluginResolutionStrategy().

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 0s

gradle --version

------------------------------------------------------------
Gradle 4.4.1
------------------------------------------------------------

Build time:   2012-12-21 00:00:00 UTC
Revision:     none

Groovy:       2.4.17
Ant:          Apache Ant(TM) version 1.10.7 compiled on October 24 2019
JVM:          17.0.5 (Private Build 17.0.5+8-Ubuntu-2ubuntu120.04)
OS:           Linux 5.14.0-1057-oem amd64

java -version
openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment (build 17.0.5+8-Ubuntu-2ubuntu120.04)
OpenJDK 64-Bit Server VM (build 17.0.5+8-Ubuntu-2ubuntu120.04, mixed mode, sharing)

./gradlew build works fine, though.

Thanks for the suggestion!
I have added support for GitHub actions and artefacts building:

https://github.com/aress31/swurg/actions

this is awesome :D

That was a good suggestion. Don't hesitate to make more or/and to submit
PRs. 😀
On Tue, 4 Apr 2023, 20:11 Cameron Whiting, @.***> wrote:

this is awesome :D
—
Reply to this email directly, view it on GitHub
https://github.com/aress31/swurg/issues/95#issuecomment-1496393584, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYQNVWHBMCRI3XNRGY53KDW7RP4FANCNFSM6AAAAAAU3WP3NY
.
You are receiving this because you modified the open/close state.Message
ID: @.***>


big design problems

FIRST make a plan and then programm it!!

rewrite the program to make further development possible!!


Simplify pipelines

Removes ifs that are always true or false in public/internal pipelines.
Follow up on https://github.com/dotnet/razor/pull/10038.
Full main CI run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=635947&view=results
Official CI run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2425936&view=results

@dotnet/razor-compiler or @dotnet/razor-tooling for a simple PR review (just removing unused code), thanks


Fix the behavior of the resolvePropertyValue method for nested value as an array that a part of AllTreeNode

It fixes the situation when it needs to apply the only or except operation for a nested array of the nested property.
For example:
AlbumData::from(Album::first())->only('songs.artist.photo.*');

Here photo is an array property of artist.
Look at these lines:
https://github.com/spatie/laravel-data/blob/7f424af3dfb912845b8f0db48dd69e02fd636c57/src/Transformers/DataTransformer.php#L202-L204
As you can notice the result of $value will be [] always because $trees->only->getFields() will return ['*'] for AllTreeNode.
$value = Arr::only($value, $trees->only->getFields());

https://github.com/spatie/laravel-data/blob/7f424af3dfb912845b8f0db48dd69e02fd636c57/src/Support/TreeNodes/AllTreeNode.php#L27-L30
Unfortunately, this fix does not solve the problem in a case like this:
AlbumData::from(Album::first())->only('songs.artist.photo'); // Without .* at the end

because here on line 25 directive will be as photo without .* ending.
https://github.com/spatie/laravel-data/blob/7f424af3dfb912845b8f0db48dd69e02fd636c57/src/Support/PartialsParser.php#L25-L32
So it requires improving some logic maybe by rewriting execute() method of PartialsParser.

Closed due inactive


[Event Request] - OnBeforeOpenJnl in Page 207 "Resource Journal"

Hello, please consider adding the following event to Page 207 "Resource Journal"
    [IntegrationEvent(false, false)]
    local procedure OnBeforeOpenJnl(var ResJnlLine: Record "Res. Journal Line"; var CurrentJnlBatchName: Code[20])
    begin
    end;

and call it accordingly inside "OnOpenPage" trigger:
    trigger OnOpenPage()
    var
        ServerSetting: Codeunit "Server Setting";
        JnlSelected: Boolean;
    begin
        IsSaaSExcelAddinEnabled := ServerSetting.GetIsSaasExcelAddinEnabled();
        if ClientTypeManagement.GetCurrentClientType() = CLIENTTYPE::ODataV4 then
            exit;

        SetDimensionsVisibility();

        **OnBeforeOpenJnl(Rec, CurrentJnlBatchName);**

        if IsOpenedFromBatch() then begin
            CurrentJnlBatchName := "Journal Batch Name";
            ResJnlManagement.OpenJnl(CurrentJnlBatchName, Rec);
            SetControlAppearanceFromBatch();
            exit;
        end;
        ResJnlManagement.TemplateSelection(PAGE::"Resource Journal", false, Rec, JnlSelected);
        if not JnlSelected then
            Error('');
        ResJnlManagement.OpenJnl(CurrentJnlBatchName, Rec);
        SetControlAppearanceFromBatch();
    end;

Thank you!

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 52878.


openbmc pdbg can't access P9 cfam reg 0x2918

We need to set bits 28 and 29 in P9 CFAM reg 0x2918 to get P9 to boot properly.  However, pdbg can't seem to access that register on a read or write:
# pdbg -a -b fsi -d p9w getcfam 0x2918
 getaddr: Error reading register
 No valid targets found or specified. Try adding -p/-c/-t options to specify a target.
 Alternatively run pdbg -a probe to get a list of all valid targets

What I want to put in the scripts is: pdbg ... putcfam 0x2918 0x0000000C 0x0000000C
Alistair has already been investigating this, but I was asked to open this issue so people can track it.

This should have been fixed by the below commit, so closing.
commit 4ce90bfe6ddfd4f7c8f76298a10a6f31a6e9cf7a
Author: Dinesh Chinari<EMAIL_ADDRESS>Date:   Wed Feb 22 14:12:11 2017 -0600
Fix CFAM address computing for OpenFSI.


On Progress

@CodiumAI-Agent /review

PR Review

 ⏱ Estimated effort to review [1-5]
3, because the PR involves changes across multiple files including CSS and JSX components. The changes are moderate in size and involve both style and functional adjustments. Understanding the impact of these changes on the overall layout and behavior of the application requires a careful review.

 🧪 Relevant tests
No

 🔍 Possible issues
Possible Bug: The CSS property rotate: -6deg; in .secondBody might cause layout issues or unexpected behavior in some browsers. Testing across different browsers is recommended to ensure compatibility.

 🔒 Security concerns
No

 Code feedback:
relevant fileclient/frontEnd/src/Styles/home.css
suggestion

Consider using CSS variables for colors and font settings to improve maintainability and consistency across your stylesheets. For example, instead of hardcoding #F9D949, define a variable at the top of your CSS file like --highlight-color: #F9D949; and use var(--highlight-color) wherever needed. [important]

relevant linecolor: #F9D949;
relevant fileclient/frontEnd/src/Styles/home.css
suggestion

The rotate: -6deg; property might not be supported or might behave unexpectedly in older browsers. Consider adding browser-specific prefixes or using a fallback layout if the rotation is not supported. [medium]

relevant linerotate: -6deg;
relevant fileclient/frontEnd/src/App.jsx
suggestion

Ensure that the re-introduction of <Navbar/> in the /build route does not introduce any visual or functional regressions, especially if <Navbar/> was previously commented out for a specific reason. [important]

relevant line
relevant fileclient/frontEnd/src/Components/Home.jsx
suggestion

The removal of dynamic translation based on scrollY in .BrandName might affect the intended animation or visual effect. If this change was intentional, ensure it aligns with the design requirements. If not, consider reintroducing the dynamic behavior. [important]

relevant lineForge Savant

 ✨ Review tool usage guide:
Overview:
The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.
The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.


How do you stop deploying?

Seriously, I've almost broken my keyboard and trackpad because I'm seriously getting crazy as I can't find a way to stop deploying my Node-Red workflow. Once you pressed the 'deploy' button there is no way back. Yes, by adjusting anything. Which sometimes I don't want to.
Is it really that strange that someone wants to stop deploying their workflow without a change?

Hi, normally we prefer general questions on the mailing list.
Currently you can't stop a flow from the editor without deploying a blank flow.
Though you can ctrl-break in the console.- the browser can still then carry on editing - as long as you don't try to reload, and obviously you have to restart to then save/deploy the edited flow... so not recommended....
and yes... normally we find people want flows to keep running as long as possible without breaking :-)

Hi @Blackvibes,  as @dceejay says, questions like this are best suited on the mailing list. Given this isn't an 'issue' for us to fix, I'm going to close.
If you do have any follow ups, please do post them on the list.
Cheers, Nick

I just started using node red and I love it! Really! Great work... but I was looking for a way to stop node red from executing my workflow, too. Something like a pause or a undeploy button : )
Maybe, this is something only node red first time users want, but it was exactly what I was looking for , too.

I also find it a bit stressful when I cant stop the flow from executing on a loop. Especially when prototyping and testing many different configurations for a custom project on Arduino.
I end up needing to ctrl+c and then delete the file and start again.
Would really appreciate the options to stop the flow, it also pollutes the console log/debugger.
Please take this into the consideration, I don't think it should be hard to implement. Would defiantly stop from going crazy. Also rather annoying that it start the flow on "node-red" command. Would have been good if it loads the flow, but requires the command to start.

+1

Stopping node-red is an issue which many people seem to want, and the workarounds are non-obvious and, arguably, have undesirable side-effects. If I were interacting with the physical world I would always want to be able to stop my devices.
Since publishing a blank page "stops" the deployment, it would seem simple to have a stop button which automatically publishes a blank page. Not perhaps the sexiest solution, but a simple fix which could live on the deploy button, below the other deployment options.

@dceejay, @knolleary, could this issue be reopened?

This issue won't be reopened. If you'd like to discuss the requirement in general, please head over to the mailing list to discuss.

If you'd like to discuss the requirement in general, please head over to the mailing list to discuss.

I appreciate the offer, but signing up for a mailing list probably isn't the right thing for my level of involvement. Best of luck!

Hi @Vilscon , I don't know if I have found the solution to stop Node red from debugging or not. However, I just deactivated the "debug" node. Then it'll stop sending messages.
Hope it helps.

Серьезно, я чуть не сломал клавиатуру и трекпад, потому что серьезно схожу с ума, поскольку не могу найти способ остановить развертывание рабочего процесса Node-Red. После того, как вы нажали кнопку «развернуть», пути назад нет. Да, подправив что-нибудь. Иногда я не хочу этого делать.
Неужели это действительно так странно, что кто-то хочет прекратить развертывание своего рабочего процесса без изменений?

node-red --safe

Kubark answered is bit ... undesirable.
Where as githubwei's solution is appreciated.