JDA-NAS Toggle

Makes JDA-NAS togglable.
With in server property jdanas false or true. Depends on #489, also optional. And can be added without that PR. Set to true to mimic default behavior.
Not sure if there's any other place to document this change.

Why would someone want to disable UDP-Queue?
Note that JDA-NAS is no longer used.

On a further thought, there may not be much point to disabling it if we control what natives get loaded. Probably would have made more sense if it was no check for this PR but even then probably shouldn't be left to user to decide. Unless otherwise I will close this in a few.


NIFI-12478 Return Message Type as body for JMS Object Messages

Summary
NIFI-12478 Updates supporting utilities for the ConsumeJMS Processor to return the Message Type as the body when receiving JMS ObjectMessage types. This approach avoids various problems with attempting to serialize a wrapped opaque Object from a JMS ObjectMessage. Java Object serialization is not suitable for inter-operation with subsequent Processors, and the wrapped Object may reference classes that are not available on a given Java Virtual Machine. Rather than writing the message as an error, this approach allows NiFi flows to continue receiving ObjectMessage types and handling the associated metadata, while using a standard value of ObjectMessage as the bytes for the message body.
Additional changes include correcting expected values in a JMS integration test that previously resulted in failures.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[X] Build completed using mvn clean install -P contrib-check

[X] JDK 21

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

Thanks @exceptionfactory I agree it's dangerous to enable using an ObjectMessage and serializing the bytes like that. I'm not really sure what exactly else to do with such a message. This approach at least ensures that the message doesn't disappear, as it passes something on with metadata, etc. Will merge to main.


Unhandled Runtime Error

I don't know what might be causing the error but I think it is the missing .env file.

@Suryanshu1005
I did below procedure. not cause the above error.
I would be happy to help you!

run git clone https://github.com/WebDevSimplified/next-13-todo-list.git
run npm i
create file .env to project root directory
write "Connect information" to .env
(I referred to this repository for reference.)

DATABASE_URL="file:./data.db?connection_limit=1"

run npx prisma migrate dev --name init

terminal output

run npm run dev

Thanks man!!


Crop to top 30 apps for bar charts.

Fixes #43
Screenshot (zoomed out):

And a commit from @raboof makes this PR a combo effort. :D

Thanks for the submission!


<EMAIL_ADDRESS>

@ColinKerr any ideas?

Add remarks: target file bis\domains\GeotechnicalModel.ecschema.md could not be located

@ColinKerr any thoughts on docs build failure?

Add remarks: target file bis\domains\GeotechnicalModel.ecschema.md could not be located

It looks like the GeologicalModel schema remarks.md file had the wrong schema name in it (GeotechincalModel) so it was looking for the wrong generated schema markdown file.
Here is a pr to fix: https://github.com/iTwin/bis-schemas/pull/200

Working on fix for circular dep.


Improve testers

Codecov Report
Patch coverage: 100.00% and no project coverage change.

Comparison is base (5fb6f44) 100.00% compared to head (2108645) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #199   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           20        20
  Lines          988       987    -1
  Branches       197       197
=========================================
- Hits           988       987    -1

Impacted Files
Coverage Δ

src/coola/testers/allclose.py
100.00% <100.00%> (ø)

src/coola/testers/equality.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


⚠ PrivateHD has degraded performance

In 698fcbd, PrivateHD ($PHD) experienced degraded performance:

HTTP code: 200
Response time: 3371 ms

Resolved: PrivateHD performance has improved in 29460d9.


bugfix: type checking logic for awaitable and coroutine

Pull Request Test Coverage Report for Build 355

6 of 9 (66.67%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.3%) to 99.725%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pedantic/type_checking_logic/check_types.py
6
9
66.67%

Totals

Change from base Build 353:
-0.3%

Covered Lines:
1484

Relevant Lines:
1487

💛  - Coveralls


Fix update-release-branch-fix.py

This change ensures that the the script can handle
commits with no committer in them. This will happen
for some commits that are auto-generated during
PRs.
Merge / deployment checklist

[x] Confirm this change is backwards compatible with existing workflows.
[x] Confirm the readme has been updated if necessary.
[x] Confirm the changelog has been updated if necessary.

Darn...failing test-packaging-javascript workflow is unrelated, but my fault. I removed the special release of the CLI that this workflow requires. I'll fix.

OK...Still failing, even with the new CLI. The reason is different this time. With this new CLI, query and pack resolution is happening more correctly and the pack is failing to find it's library dependency codeql-javascript. In the previous version, the library path lookup was more lenient and was able to resolve the qlpack reference to the well-known codeql-javascript pack. So, I need to figure this out. I'll keep this PR open for now.

OK. This is now fixed. I had to recreate the codeql-pack1 and codeql-pack2 packs. Before, they were not properly created with their compile-time dependencies. Now they are.

Still failing on this branch when I run the Update release branch workflow. I'll have to inspect more.

Working now.


[Bug] Version 3.2.2 breaking interraction with everything on the ship.

Hey :) As tittle said, new version is breaking all interractions on the ship. In log it says something about Bigger Lungs constant error.

Update :)
Version 3.2.3 seems to fix that issue. Will monitor it further tho.
Thanks!

Another update
Version 3.2.3 made monsters immune to shovel damage.
Reverted back to 3.1.1 and everything works fine.

I probably know why. I will take a look at it.

Aight, thanks for info :)


fix: Do not create the event as published

MISP doesn't allow to create empty events anymore.

@gcrahay and team, this is a PR from the MISP team which I hope you will treat as a trusted source :) yet there's been no action for over a year, not even a comment.  Is otx_misp abandoned?


[AUTOMATED] Update Version Pinning for Terraform to support 0.13

What

Update Version Pinning for Terraform to support 0.13

Why

This is a relatively minor update that the CloudPosse module already likely supports.
This allows module consumers to not individually update our Terraform module to support Terraform 0.13.

/test all

ALB module dependent on this: https://github.com/cloudposse/terraform-aws-alb/pull/48


[BUG] Application Crash when using onChange view modifier

When CameraView is presented, if the onChange(of:) view modifier of the presenting view is triggered, two types of crashes occur as shown in the image below.

I have also checked Issue #43, and the problem persists even when using a manager via dependency injection. Additionally, the same crash occurs in the demo project of this library.
Here's a simple example in code:
struct SomeView: View {

let timer = Timer.publish(every: 0.2, on: RunLoop.main, in: .common)
        .autoconnect()

@State private var isCameraPresented = false
@State private var count = 0

var body: some View {
        VStack {
            Button(action: {
                isCameraPresented = true
            }, label: {
                Text("Present Camera")
            }
        }
        .onRecieve(timer) { _ in
            count += 1
        }
        .onChange(of: count) { _, newValue in
            print(count)
        }
        .fullScreenCover(isCameraPresented) {
            CustomMijickCameraView()
        }
    }
}

I think I have resolved the issue. Declaring the manager as a @StateObject instead of an @ObservableObject fixed it.


Redis cluster never removes hosts that are PENDING_DYNAMIC_REMOVAL

Description:
When a host is removed from a Redis cluster, and health checks are enabled, then the host will remain in PENDING_DYNAMIC_REMOVAL even after it fails health checks.
A test for this bug can be found (commented out) in this commit: https://github.com/envoyproxy/envoy/pull/14483/commits/cc8729f9bcd219bc8f016f21d90f086e63e02b39
It seems likely that this is also true of the strict DNS cluster type too.
Repro steps:
Create a Redis cluster with health checks enabled.
Remove a host from the cluster in a way that causes the health check to still pass.
Check that the host has moved to pending dynamic removal.
Fail the health check.
Expected result:
The host is removed from the cluster.
Actual result:
The host remains in pending dynamic removal state.

I'm suspect this failed healthcheck will cause envoy to crash if the CDS is using healthcheck
Ref: https://github.com/envoyproxy/envoy/issues/12043

I'm suspect this failed healthcheck will cause envoy to crash if the CDS is using healthcheck
Ref: https://github.com/envoyproxy/envoy/issues/12043


OD_RAM.x1001_errorRegister resets

Hello.
I am using the CANopenNode stack on my B-G431B-ESC1. The basic communication works already.
Now I want to send information about errors to other devices. When setting a bit of OD_RAM.x1001_errorRegister in my program, it gets reset back to 0 again. Is there any reason for this? I don't have this problem with other OD-Objects.
Thanks for your help.

you need to use the functions provided by 301/CO_Emergency.h, e.g. CO_error and macros

Thanks for the advice, I will check it out.

Hi.
I am still having trouble using the error functions. Can somebody provide an example on how to use them correctly?
Thanks!

Please check https://github.com/CANopenNode/CANopenDemo for some tutorials.


Which presto is supported, prestosql or prestodb or both?

In doc of query cost estimation, it states

Here, “version” should be the version of your Presto cluster. Support for this functionality was introduced in Presto 0.319.

Which I assume referring to PrestoSQL 0.319 version, because it matches PrestoSQL changelog and PrestoDB only released 0.237.1 to this date. But on doc index page, the Presto support is linked to PrestoDB.
I know it's a headache. But I hope there's some clarification on this. Maybe it would be nice to also clarify this situation in the doc?

Forks break my heart 💔
I'm assuming both and Athena as well. No one has reported issues on either side AFAICT

Thank you for clarifying this.


Feature Request: Add createdAt timestamp in documents

Feature Request: Add createdAt timestamp in documents
This fields will help in keeping collection clean with TTL index on createdAt.
createdAt can be option and set only if set in MongoAdapterOptions

@manishmotiani I came here looking for the same feature, but I just learned that capped collections don't support TTL indices.  It seems like the performance advantage of capped collections is pretty integral to this library, so it might not be possible to make this change.

@manishmotiani hi! Could you please explain your use case?
If I recall correctly, capped collections are more efficient than TTL indexes in that case, but I don't mind adding an option to cover both use cases.
See also: https://stackoverflow.com/questions/34181516/mongo-ttl-vs-capped-collections-for-efficiency

@darrachequesne socket data in mongodb is useful for say 6 hours. To keep things clean, I want it to auto remove document based on TTL which work on date/timestamp field index.
what should ideal size for capped collection if we don't want to use too much or too less space?

@manishmotiani well, that's the tricky part, since it depends on your own usage:

how many Socket.IO servers?
how many broadcast per second?
how big each packet is?

I'd say you only need a few minutes of historic data, to handle temporary disconnections to the MongoDB cluster. Having both a max size (size option) and a max number of documents (max option) might help a bit.
Reference: https://www.mongodb.com/docs/manual/core/capped-collections/#create-a-capped-collection

@darrachequesne Thanks this helps. Currently there is not much server load and packet size is few kbs. Limiting with max number of document will help achieve the purpose.
Now only use case for adding timestamp is for analytics to generate how many messages going through in a specific time period across all or specific server.
If you think that is good use case, then we can proceed with this enhancement or probably add in future timeline.

Thanks


Brain tumor Classification

I would like to add a project for brain tumor analysis. Here, I would use datasets containing images of the brain, from which I would detect whether a person is having brain tumor or not. And, then also classify the type of tumor.

@sudipg4112001, go ahead.


README docker example does not work

Following README to build and run docker version does not work.
Program fails with error: OSError: Cannot find ecCodes definition path

Can be fixed of course with
ECCODES_DEFINITION_PATH=/opt/eccodes/share/eccodes/definitions
but maybe add this to Dockerfile


Header retained in the flow from previous run

fix for #191

Rather than check the length of headers would a better check be to compare header choice vs stacktypeid - if its not a direct match then just recreate the stack using the headers rather than attempting to modify?

@ajbalogh At the time of testing I encounter some more issue in our existing Remove() mechanism.  (say Test1 : Ether, IP, Tcp and Test2 : pfcPause, Udp Result: pfcPause, IP, Tcp) It looks like IxNetwork is doing some serialization for newly created stack. Therefore I am not getting predictive result.  So I have recheck newly created stacks and remove accordingly. I know it will good if we did entire operation in one go. Could you please review and suggest any better solution.


DataStream.slice throws 'write after end' error

With this file, test.ndjson:
{ "abc": 1 }
{ "abc": 2 }
{ "abc": 3 }
{ "abc": 4 }

And this code:
const FS = require('fs')
const Scramjet = require('scramjet')

Scramjet.StringStream.from(FS.createReadStream('test.ndjson')).JSONParse().slice(0, 1).toArray()

Throws this error:
UnhandledPromiseRejectionWarning: Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at DataStream.Writable.write (_stream_writable.js:292:11)
    at StringStream.ondata (_stream_readable.js:719:22)
    at StringStream.emit (events.js:315:20)
    at StringStream.EventEmitter.emit (domain.js:548:15)
    at addChunk (_stream_readable.js:309:12)
    at readableAddChunk (_stream_readable.js:284:9)
    at StringStream.Readable.push (_stream_readable.js:223:10)
    at StringStream.Transform.push (_stream_transform.js:166:32)
    at /Users/maxharlow/Projects/test/node_modules/scramjet-core/lib/util/mk-transform.js:71:38
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Bizarrely, if you remove one line from the file (any line), you don't get the error.
Node v14.14, Scramjet v4.35.6

Hey,
Thanks for the info - can you downgrade to 4.34 and 4.35.5 and see if any of these has an effect.
I'll add the above case to tests but it will take a bit to reproduce, so you may solve the issue this way.
M.

Thanks for the quick response! I still get the same error on v4.34, but I tried going back to v4.19 and that seems to work

Ok. That's a good lead - hopefully we haven't added some specific function in between 4.19 and now. I can't remember any prod-grade vulnerable deps either so if that works for you, as a dependency it is safe to use it for now.
I think I know what is the reason so it shouldn't be long until I manage to fix it, rather days than weeks.

It turned out to be hours... I've got a fix - release tomorrow.

Excellent news, thank you!

@maxharlow - check<EMAIL_ADDRESS>...and your LinkedIn BTW ;)


dimr_config for parallel run is incorrect

Describe the bug
When generating a dimr_config.xml with hydrolib for an MPI run, we need to pass the process and mpiCommunicator arguments. The expected dtype for process is int, but this should not be directly passed but converted to a space-separated range. For 4 cores, the entry in the dimr should be <process>0 1 2 3</process>, but instead it reads <process>4</process>.
To Reproduce
The file resulting from the code below contains an invalid <process> entry, which makes the model run crash:
from hydrolib.core.dimr.models import DIMR, FMComponent
fm_comp = FMComponent(name="test", workingDir='.', inputfile='test.mdu',
                      process=4, mpiCommunicator="DFM_COMM_DFMWORLD")
dimr = DIMR(component=fm_comp)
dimr.save(filepath='dimr_config.xml')

Expected behavior
<process>0 1 2 3</process> in the dimr_config.xml. It might be necessary to align processor/core numbers between different components of the DIMR file, but that is currently unsure. This must also be fixed for reading the xml file
Version info (please complete the following information):

OS: Windows
Version 0.5.2

Additional request
Since the mpiCommunicator is always the same ("DFM_COMM_DFMWORLD"), we could also consider to always write it to the dimr_config if the amount of processes is >1. One argument less, always nice.

So, just to be sure. If the processes = 5, we should write: <process>0 1 2 3 4</process>?
Should we still support the files that we have written incorrectly? In other words, should we still be able to parse: <process>4</process>?
@veenstrajelmer & @rhutten is this correct?

Currently I have assumed the following:
This change is only for FM and not RR.
When 0 is input for process the dimr_config.xml will not have  added.
When 1 is input for process the dimr_config.xml will have 0 added.
open questions:
What should happen if an user inputs something different than a int?
e.g.
    component = FMComponent(
        name="test",
        workingDir=".",
        inputfile="test.mdu",
        process="Four",
        mpiCommunicator="DFM_COMM_DFMWORLD",
    )

or
    component = FMComponent(
        name="test",
        workingDir=".",
        inputfile="test.mdu",
        process=4.0,
        mpiCommunicator="DFM_COMM_DFMWORLD",
    )

Do we want to throw an exception or do we want to (silently) leave the process out?
open question for a followup issue:
Currently the abstract class component has process implemented.
For the FM component the changes have been applied in the specified branch.
For the RR component process can be given, but is not expected.
This will  still result in an output in the .xml file, should we handle this is some way?
    component = RRComponent(
        name="test",
        workingDir=".",
        inputfile="test.mdu",
        process=4,
        mpiCommunicator="DFM_COMM_DFMWORLD",
    )

If process = 0 or 1 is selected, no process keyword should be written to dimr_config.xml

Discussed with @rhutten:
Should we still support the files that we have written incorrectly? In other words, should we still be able to parse: <process>4</process>? --> No, but a warning should be given.
When 0 or 1 is input for process the dimr_config.xml will not have added --> No process will be added, a message will be given to inform te user "Note that for process 0 or 1 no sequential error will be given."

Correction on the last message, for process only 1 can be added. Please give the warning specified above. For 0-process is not possible and an error should be given. Error message: the keyword process can not be 0, please specify values 1 or greater.

Discussed with @Tim & @Jelmer, conclusion:
acceptance criteria:

component.process is an int.

when import the dimr_config.xml a string list is expected e.g. "0 1 2 3".
this should be converted/parsed to an int on the model.component.process, for the example 4.
when parsing from int to the string list, it will take the int and make the list up to (not including) the model.component.process number starting from 0.

when export the dimr_config.xml a string list should be written, e.g. component has 4.
the output file will have "0 1 2 3"
when parsing from string list to the int, it will take the the amount of ints in the list and set it to model.component.process.

keep it simple

input from file is validated on import list e.g. "0 1 2 3", (validation on converter/parser) --> input not parsable to list of ints, throw value error.

input from user of none or 1 can leave the process out, no message needed, since this is what the user intends (implemented on converter/parser)

input from user of > 2 will write the string list to the file, no message needed. (implemented on converter/parser)

input from user is validated on model.component e.g. int (fmcomponent, validator on field process)

input from user of 0 or negative, throws an error. (fmcomponent, validator on field process)

One more addition, according to the manual this is also supported:
<process>0:4</process>
Might be good to also support for this notation in hydrolib-core, maybe as part of a separate issue if not convenient now.

@veenstrajelmer, currently added the support for 0:4, the assumption is made that this is the same as 0 1 2 3 4.
The last value is used to determine the process value.
when writing it will thus be written as 0 1 2 3 4.
This means, 2:4 will be assumed as process = 5, and will be written as 0 1 2 3 4.
Further support for this can be done in a seperate issue.
Acceptance criteria for followup-issue is to be refined.
I will need to discuss the current additional support with @rhutten.

@MRVermeulenDeltares: <process>2:4</process> would  (I think) translate to <process>2 3 4</process> and therfore as FMComponent(process=3), when writing it again, it will result in  <process>0 1 2</process> I guess.

Discussed with @rhutten, I will create a follow-up issue.
We can keep the above described as is and determine the acceptance criteria for the full support in the follow-up issue.
since fmcomponent expects an int for process, a start and end are currently not supported, thus reading/writing of a semicolon input can only be partialy supported.
In the follow-up issue we will need to discuss what how the user should be able to apply start and end value for process.
This way we can also determine how to add support to read and write the semicolon value for process.
The acceptance criteria for the follow-up can be further refined with input from "D-Flow_FM-User_manual 17.1.3 Input DIMR"
Maybe we can also request more information on what the numbers actualy mean, e.g. 16:31, would this mean run on core 16 to 31?
@veenstrajelmer, If in your opinion the behaviour you have described is more desirable, I can have a look to implement that in this issue.
Any other update to support semicolon input and output should be handled in the follow-up issue.

@MRVermeulenDeltares thanks a lot for implementing this. This linked branch was tested and the code now indeed behaves as it should.


Nested img within a tag in Markdown causes Vite Error

vitepress-plugin-inline-link-preview

This will cause Vite Error : [plugin:vite:vue] Invalid end tag.

Hi, welcome to Nolebase community!
Thanks for reporting, fixed on #248, please wait our CI/CD to quickly rollout a fix on that.

Hi, welcome to Nolebase community!
Thanks for reporting, fixed on #248, please wait our CI/CD to quickly rollout a fix on that.

Also Thx for your attention. ^ ^

No problem! My pleasure.
And the fix was already released, please upgrade to v2.2.2 to give it a try.


OpenShift deployment with anyuid / arbitrary user and raw TCP

This pull request is an update to pull request #233 (see also a copy of the notes below)
For the use case that Kong needs to run in raw tcp mode I have added some more code - it does work on my test environment but maybe it does not look very elegant.

I create within the Dockerfile (while still being root) two copies of Nginx, one with setcap run against it, one without
In docker-entrypoint I create a link to either the setcap ("transparent") version of Nginx or the non-setcap one depending on startup variables
kong prepare is now after the check for raw tcp as we need to create the Nginx link first

So when raw tcp is needed the Nginx executable has the needed permissions for raw tcp.
Oh, and I have readded the "exec" again but without the su-exec as we are not root
*** from #233
This pull request changes the Dockerfile and entrypoint.sh in a way that the image can be run without root privileges and as an arbitrary user (anyuid). When those changes have been applied Kong can be run on a default OpenShift installation without changing any default settings regarding root user or arbitrary user.
See https://docs.openshift.com/enterprise/3.0/creating_images/guidelines.html for reference about the needed changes.
The Dockerfile changes in detail:
kong user is now member of the root group
the chown command which has been located in the entrypoint.sh until now is part of the Dockerfile (as we only have the permissions here to execute the command)
chgrp/chmod commands added so /usr/local/kong belongs to user kong's root group
USER kong added so image is run as kong from the start
The entrypoint.sh changes in detail:
chown and chgrp commands removed / relocated to Dockerfile (we are user kong in entrypoint.sh now so no permissions to do so anymore)
"exec su-exec kong" removed as we are king anyway

This looks promising -- the centos container worked in a trasparent proxy setup for me the alpine image is a little unhappy though
cd alpine
docker build -t temp .
docker run -it temp --rm temp /bin/sh
standard_init_linux.go:207: exec user process caused "permission denied"

@svenwal could you also try on Alpine?

Tried a small in change in docker run syntax and I can execute it:
docker run --rm -it temp  /bin/sh
/ $ whoami
kong

Note that when run in this way the nginx binary is not available (as it only will be copied into position by the if statement above). We could think to have a default binary in this location and overwrite it instead of creating the link.

This should be resolved now with https://github.com/Kong/docker-kong/pull/243 being merged (haven't updated the official images yet)


Implement Codable template

With introduction of native encoding/decoding with Swift 4 it might seem for some users that Sourcery is loosing its relevance in this domain. Though it is true for simplest cases for more complex cases requiring keys renaming and manual encoding/decoding it is still very relevant. This template should show how Sourcery still can be used to eliminate boilerplate around that.

Came here looking for this :) I have been working with swift 4 for a while now, and for 95% of the objects I have to write CodingKeys by hand to override one or 2 of the keys, and about 50% I need to write initFromDecoder by hand since some verification/processing of data is needed. I don't think Codable will kill Sourcery's usability for JSON parsing, I think it will probably be 10 times more useful! Can't wait for this feature, I really enjoy using Sourcery. I tried to implement this myself, but I ran into a problem. I wrote a basic stencil that creates an initFrom(decoder:) in the most naive way, but then I need to write some code to customise the method itself after it is generated. How would I go about mixing Sourcery generated code with hand written code?

Any updates on this? Especially Codable for enums.
The Codable Enums from documentation doesn't show how to use Sourcery to implement it as it claims, it only shows how to manually make an enum Codable.

Well you can't make enum Codable by default right now. And yeah, it would've been great not to write decode and encode for each enum, especially when you have a lot of them in hierarchy. Or at least to generate them.

Just implemented a basic version for my own project, it doesn't handle a lot of edge cases / is pretty rough around the edges, nonetheless for anyone that may be interested...
Template
import Foundation

// https://github.com/SwiftGen/StencilSwiftKit/blob/master/Documentation/tag-set.md
// https://github.com/kylef/Stencil
// https://cdn.rawgit.com/krzysztofzablocki/Sourcery/master/docs/index.html

{% macro modelName name %}{{ name | replace:"Mappable","Model" }}{% endmacro %}

{% macro immutableVariables variables %}
  {% for var in variables %}
  let {{ var.name }}: {{ var.typeName }}
  {% endfor %}
{% endmacro %}

{% macro realmConvert model %}
  {% set rmodel %}{{ model|replace:"Model","" }}{% endset %}
  public override func asRealm() -> {{ rmodel }} {
    return {{ rmodel }}(value: asDictionary())
  }
{% endmacro %}

{% macro codingKeys variables %}
  private enum CodingKeys: String, CodingKey {
    {% for var in variables %}
    case {{ var.name }} {% if var.annotations.coding %} = "{{ var.annotations.coding }}" {% endif %}
    {% endfor %}
  }
{% endmacro %}

{% macro decodeVar var %}
  {% set itype %}{{ var.annotations.incomingType }}{% endset %}
  {% if var.typeName.name == "Bool" and itype == "Int" %} {# its a 0 or 1 #}
    {{ var.name }} = try container.decodeIfPresent(Int.self, forKey: .{{ var.name }}) == 1
  {% elif var.typeName.name == "Date" and itype == "String" %} {# iso date string #}
    if
      let {{ var.name }} = try container.decodeIfPresent(String.self, forKey: .{{ var.name }}),
      let date = {{ var.name }}.dateFromISO8601 {
      self.{{ var.name }} = date
    } else {
      self.{{ var.name }} = Date()
    }
  {% endif %}
{% endmacro %}

{# cant get spacing right without doing it like this, better way when possible #}
{% macro decodeWithDefault var %}
  {% if var.type.name == "String" %}
    {{ var.name }} = try container.decodeIfPresent({{ var.typeName }}.self, .{{ var.name }}) ?? ""
  {% elif var.type.name == "Bool" %}
    {{ var.name }} = try container.decodeIfPresent({{ var.typeName }}.self, .{{ var.name }}) ?? false
  {% elif var.type.name == "Int" %}
    {{ var.name }} = try container.decodeIfPresent({{ var.typeName }}.self, .{{ var.name }}) ?? 0
  {% elif var.type.name == "Date" %}
    {{ var.name }} = try container.decodeIfPresent({{ var.typeName }}.self, .{{ var.name }}) ?? Date()
  {% endif %}
{% endmacro %}

{% macro kvoSubWithDefault var %}
  {% if var.type.name == "String" %}
    {{ var.name }} = object["{{ var.name }}"] as? {{ var.typeName.name }} ?? ""
  {% elif var.type.name == "Bool" %}
    {{ var.name }} = object["{{ var.name }}"] as? {{ var.typeName.name }} ?? false
  {% elif var.type.name == "Int" or var.type.name == "Double" or var.type.name == "Float" %}
    {{ var.name }} = object["{{ var.name }}"] as? {{ var.typeName.name }} ?? 0
  {% elif var.type.name == "Date" %}
    {{ var.name }} = object["{{ var.name }}"] as? {{ var.typeName.name }} ?? Date()
  {% endif %}
{% endmacro %}

{% macro decodableInitializer variables %}
  public required init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    {% for var in variables where var|annotated:"incomingType" %}
    {% call decodeVar var %}
    {% endfor %}
    {% for var in variables where var|!annotated:"incomingType" %}
    {% call decodeWithDefault var %}
    {% endfor %}
    try super.init(from: decoder)
  }
{% endmacro %}

{% macro convertVarForObject var %}
  {% set itype %}{{ var.annotations.incomingType }}{% endset %}
  {% if var.typeName.name == "Bool" and itype == "Int" %} {# its a 0 or 1 #}
    {{ var.name }} = (object["{{ var.name }}"] as? Int) == 1
  {% elif var.typeName.name == "Date" and itype == "String" %} {# iso date string #}
    if let {{ var.name }} = object["{{ var.name }}"] as? NSNumber {
      self.{{ var.name }} = Date.fromTimestamp(Timestamp({{ var.name }}.int64Value))
    } else {
      self.{{ var.name }} = Date()
    }
  {% endif %}
{% endmacro %}

{% macro objectInitializer variables %}
  public required init(with object: INRBaseObject) {
    {% for var in variables where var|annotated:"incomingType" %}
    {% call convertVarForObject var %}
    {% endfor %}
    {% for var in variables where var|!annotated:"incomingType" %}
    {% call kvoSubWithDefault var %}
    {% endfor %}
    super.init(with: object)
  }
{% endmacro %}

{% macro convertVarForEncoder var %}
  {% set itype %}{{ var.annotations.incomingType }}{% endset %}
  {% if var.typeName.name == "Bool" and itype == "Int" %} {# its a 0 or 1 #}
    try container.encode({{ var.name }} ? 1 : 0, forKey: .{{ var.name }})
  {% elif var.typeName.name == "Date" and itype == "String" %} {# iso date string #}
    try container.encode({{ var.name }}.iso8601, forKey: .{{ var.name }})
  {% endif %}
{% endmacro %}

{% macro encode variables %}
  public override func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)
    {% for var in variables where var|annotated:"incomingType" %}
    {% call convertVarForEncoder var %}
    {% endfor %}
    {% for var in variables where var|!annotated:"incomingType" %}
    try container.encode({{ var.name }}, forKey: .{{ var.name }})
    {% endfor %}
    try super.encode(to: encoder)
  }
{% endmacro %}

{% macro asDictionary variables %}
  public override func asDictionary() -> [String: Any] {
    var dictionary = super.asDictionary()
    {% for var in variables %}
    dictionary["{{ var.name }}"] = {{ var.name }}
    {% endfor %}
    return dictionary
  }
{% endmacro %}

{% for type in types.based.ModelMappable where type|protocol and not type.name == "ModelMappable" %}
{% set model %}{% call modelName type.name %}{% endset %}

// MARK: {{ model }} Mappable
public final class {{ model }}: BaseModel, {{ type.name }} {
  {% call immutableVariables type.allVariables %}

  // MARK: {{ model }} CodingKeys
  {% call codingKeys type.allVariables %}

  // MARK: {{ model }} Decodable
  {% call decodableInitializer type.allVariables %}

  // MARK: {{ model }} Object Init
  {% call objectInitializer type.allVariables %}

  // MARK: {{ model }} Encodable
  {% call encode type.allVariables %}

  // MARK: {{ model }} Dictionary
  {% call asDictionary type.allVariables %}

  // MARK: {{ model }} Realm Convert
  {% call realmConvert model %}
}

{% endfor %}

public protocol ModelMappable { }

public protocol ExampleMappable: ModelMappable {
  // sourcery: coding="some_variable_named_something_else"
  var someVariable: String { get set }
  // sourcery: coding="some_variable_id", incomingType="Int"
  var someBoolThatsAnIntFromTheServer: Bool { get set }
}

Produces
// MARK: ExampleModel Mappable
public final class ExampleModel: BaseModel, ExampleMappable {
  let someVariable: String
  let someBoolThatsAnIntFromTheServer: Bool

  // MARK: ExampleModel CodingKeys
  private enum CodingKeys: String, CodingKey {
    case someVariable  = "some_variable_named_something_else"
    case someBoolThatsAnIntFromTheServer  = "some_variable_id"
  }

  // MARK: ExampleModel Decodable
  public required init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    someBoolThatsAnIntFromTheServer = try container.decodeIfPresent(Int.self, forKey: .someBoolThatsAnIntFromTheServer) == 1
    someVariable = try container.decodeIfPresent(String.self, .someVariable) ?? ""
    try super.init(from: decoder)
  }

  // MARK: ExampleModel Object Init
  public required init(with object: INRBaseObject) {
    someBoolThatsAnIntFromTheServer = (object["someBoolThatsAnIntFromTheServer"] as? Int) == 1
    someVariable = object["someVariable"] as? String ?? ""
    super.init(with: object)
  }

  // MARK: ExampleModel Encodable
  public override func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)
    try container.encode(someBoolThatsAnIntFromTheServer ? 1 : 0, forKey: .someBoolThatsAnIntFromTheServer)
    try container.encode(someVariable, forKey: .someVariable)
    try super.encode(to: encoder)
  }

  // MARK: ExampleModel Dictionary
  public override func asDictionary() -> [String: Any] {
    var dictionary = super.asDictionary()
    dictionary["someVariable"] = someVariable
    dictionary["someBoolThatsAnIntFromTheServer"] = someBoolThatsAnIntFromTheServer
    return dictionary
  }

  // MARK: ExampleModel Realm Convert
  public override func asRealm() -> Example {
    return Example(value: asDictionary())
  }
}

I just found this project yesterday and it is really awesome by the way! @krzysztofzablocki What do you think about moving this repo to an organization and then creating a sort of community templates project to accompany it?

@thevest thanks for sharing the template, we discussed organization before but we don't want to manage multiple projects as its PITA, we already share common startup templates in this repo and have tests for them that run each time Sourcery is changed which helps with maintaining them
@thevest would you mind adding it to our bundled templates along with some basic tests? even if it doesn't handle all cases it would be good start for anyone needing Codable support :)

Resolved with #599


Refactor validate_data_dict and improved testing

This is refactor PR with no change in functionality.
data_model.validate_data._validate_data_dict is too complex and we split here the function into several. Also address the issue of a missing test for the call of _validate_data_dict_using_json_schema.
Closes #1193

@orelgueta - thanks for the review!
Let me know if it is ok to keep the if-statements. I really don't care, but have a slight preference to less lines of code.


Allow annotation keys to be Cedar reserved words

Description of changes
Allows annotation keys to be Cedar reserved words.  Tests that this works both for human-format and JSON-format (EST) policies.  Fixes #623.
Issue #, if available
#623
Checklist for requesting a review
The change in this PR is (choose one, and delete the other options):

[x] A bug fix or other functionality change requiring a patch to cedar-policy.

I confirm that this PR (choose one, and delete the other options):

[x] Updates the "Unreleased" section of the CHANGELOG with a description of my change (required for major/minor version bumps).

I confirm that cedar-spec (choose one, and delete the other options):

[x] Does not require updates because my change does not impact the Cedar Dafny model or DRT infrastructure.

(is that right?)
Disclaimer
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Opened #635 for the follow-up.  Unless @john-h-kastner-aws thinks this is important to address now (before merging)?


Fix crash when pinned SoftBody point is out of range

The master version of #53380.

The SoftBodyBullet modifications are the same as the 3.x PR.

It's not tested on master because the class seems not compiled and I don't know how to activate it.

Adds a >= 0 check for indices in SoftBody3DSW.

Fixed a typo when backporting the 3.x changes :P

Thanks!


Hard to see social icons

Social icons are very hard to see on your contact section.

Social icons are very hard to see on your contact section.

Thank you for your suggestion ❤ , I will be updating my portfolio after I've finished my other tasks.


cicirello/jacoco-badge-generator 추가

Codecov Report
Base: 29.89% // Head: 29.89% // No change to project coverage :thumbsup:

Coverage data is based on head (ecb0139) compared to base (0db5465).
Patch has no changes to coverable lines.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@                Coverage Diff                @@
##             jacoco-setting       #6   +/-   ##
=================================================
  Coverage             29.89%   29.89%
  Complexity              100      100
=================================================
  Files                    36       36
  Lines                  1154     1154
  Branches                158      158
=================================================
  Hits                    345      345
  Misses                  779      779
  Partials                 30       30

Flag
Coverage Δ

unittests
29.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Using Unicreds in AWS Lambda

Can we use Unicers for the functions that are deployed in AWS Lambda?

I haven't done this with unicreds - that said a lambda function should be able to run an arbitrary command.
This AWS blog may help
https://aws.amazon.com/blogs/compute/running-executables-in-aws-lambda/

Thanks for pointing out the relevant blog post. I will give it a try.

Yes, it is perfectly fine to use unicreds in a lambda function, one of the nice things about Go executables.. This is not really an applicable GH issue and should be closed.


[BUG] Page Header: Collapsed breadcrumb tab disappears

What package(s) are you using?

Carbon for IBM Products v 2.15.1

Detailed description
Interact with the page actions in page header when is collapsed make the breadcrumb tab disappears.

Steps to reproduce the issue
Unfortunately in the storybook the actions doesn't really works, but you can reproduce the behavior anyway

Step one
Go to: https://carbon-for-ibm-products.netlify.app/?path=/story/ibm-products-components-page-header-pageheader--with-actions-toolbar

Open the developer panel (so we can reproduce the behavior later).

Step two

Scroll down until the header is collapsed like this

Step three

Increase or reduce the size of the developer tools panel.
If you did everything correctly you shouldn't see the collapsed header anymore.

Note
The meaning of this steps is only to show the bug.
In my project the behavior occurs by scroll down the page and click on any action present on the page header.
Thank you everybody for your time!

Verified that this is a v2 specific issue.

Acceptance criteria
Breadcrumbs should not disappear on viewport resize.


Identifiers validation failed for dataset La vegetación y flora de las Lagunas de Catazajá-Emiliano Zapata

Identifier validation failed for the dataset La vegetación y flora de las Lagunas de Catazajá-Emiliano Zapata:

Crawler attempt: 286
Publishing organization: Comisión nacional para el conocimiento y uso de la biodiversidad
Publishing country: MEXICO
Installation: IPTCONABIO
Number of occurrences indexed for this dataset at the time of the issue: 240
Cause: GBIF IDs hit maximum allowed - 50%, duplicates - 100%, total records count 240, absent records count 240
Pipelines execution steps: http://api.gbif.org/v1/pipelines/history/execution/3073510/step

Publisher email
Hello,
I am contacting you from the GBIF Secretariat about a dataset published by the La vegetación y flora de las Lagunas de Catazajá-Emiliano Zapata : https://doi.org/10.15468/xdnj12.
We noticed that the occurrenceIDs were changed. We have temporarily paused the ingestions of this dataset.
As you might already know, when an occurrence record has a new occurrenceID for a given dataset, our system considers it to be a new occurrence.
This means that it will be given a new gbifid and a new occurrence URL (like this one: https://www.gbif.org/occurrence/1252968762) and the old gbifid and URL will be deprecated.
In this case, this means that the occurrence URLs would be deprecated when ingesting the newest versions of these datasets.
We would like to check with you if those changes were intentional. Do you know if this is the case?
Please let us know, thanks! We are happy to resume the dataset ingestion.
Note that some users rely on those occurrence URLs and gbifids (like https://bionomia.net for example). In an attempt to improve
the stability of the occurrence URLs and gbifids, we have implemented a warning system to detect these type of changes in datasets (see this news item).
If the data publisher can provide us with a list of old and new occurrenceIDs per record, we can avoid the identifier and URL changes. Could that be an option?
Please let us know if you have any question. Thanks!
All the best,

You can skip/fix identifier validation using the registry UI.

Identifier validation still  failing for the dataset La vegetación y flora de las Lagunas de Catazajá-Emiliano Zapata:

Crawler attempt: 289
Publishing organization: Comisión nacional para el conocimiento y uso de la biodiversidad
Publishing country: MEXICO
Installation: IPTCONABIO
Number of occurrences indexed for this dataset at the time of the issue: 240
Cause: GBIF IDs hit maximum allowed - 50%, duplicates - 100%, total records count 240, absent records count 240
Pipelines execution steps: http://api.gbif.org/v1/pipelines/history/execution/3094376/step

Publisher email
Hello,
I am contacting you from the GBIF Secretariat about a dataset published by the La vegetación y flora de las Lagunas de Catazajá-Emiliano Zapata : https://doi.org/10.15468/xdnj12.
We noticed that the occurrenceIDs were changed. We have temporarily paused the ingestions of this dataset.
As you might already know, when an occurrence record has a new occurrenceID for a given dataset, our system considers it to be a new occurrence.
This means that it will be given a new gbifid and a new occurrence URL (like this one: https://www.gbif.org/occurrence/1252968762) and the old gbifid and URL will be deprecated.
In this case, this means that the occurrence URLs would be deprecated when ingesting the newest versions of these datasets.
We would like to check with you if those changes were intentional. Do you know if this is the case?
Please let us know, thanks! We are happy to resume the dataset ingestion.
Note that some users rely on those occurrence URLs and gbifids (like https://bionomia.net for example). In an attempt to improve
the stability of the occurrence URLs and gbifids, we have implemented a warning system to detect these type of changes in datasets (see this news item).
If the data publisher can provide us with a list of old and new occurrenceIDs per record, we can avoid the identifier and URL changes. Could that be an option?
Please let us know if you have any question. Thanks!
All the best,

You can skip/fix identifier validation using the registry UI.


Add txid + nout (WIP)

This is the rebased pull request for adding txid + nout into lbrynet.
Requires lbryum pull request :
https://github.com/lbryio/lbryum/pull/34

There is a lot here!  Hopefully I didn't lose too much focus near the end.
We don't have a good test framework yet (its improving) but some tests, especially in the db migration would be comforting.  I'll leave it up to you to decide if there is other high risk parts of the change that warrant testing.

Made changes based on job's review. One thing I did not do is make a new data structure to represent txid + nout as that is going to take a bit of time ( will try to do either by end of this week or for next release cycle).
Found some bugs, and fixed it.
Reverted jack's changes to _ResolveNameHelper as it does not seem relevant to this patch.

Rebased to master

https://github.com/lbryio/lbry/pull/256 should be merged to master first , and this pull request rebased

regarding a ClaimLocation/Outpoint data structure - do you want to make this part of this PR or save it for future work?

Since this didn't make it into this week's release cycle, I can make ClaimLocation/Outpoint data structure part of this PR

@kaykurokawa when can this get merged in. its been open for a long time. whats the next step?

I finished the ClaimOutpoint data structure. I'm rebasing to master now and testing, will be ready to review by end of today.

Finished adding the ClaimOutpoint data structure, which is used internally in the Wallet class as a way to encapsulate txid and nout.
Needs final review for merging.

lgtm

Rebased to master and squashed to a single commit.
Merging to master.


sample: sample for limitToLast() query

Sample for limitToLast Query.

@AVaksman Please Take a look.


Replace id with name in forwarded messages

@RAZRULETEL, thank you for this PR, this was indeed a very desired a feature.
I've decided to return to a slightly earlier version of viktig, which is easier to deploy and maintain, so I rebased this PR onto the new main branch. Most of the code from the PR was unaffected though, so I will merge now. Thanks!


Update README.md

Add a portuguese youtube channel about Directus.

Thank you @hermesalvesbr for your contribution! However, I don't think we can add this under "Official Resources" but under a new section like "Community Resources" or something similar... What do you think @rijkvanzanten?
Edit: I think something like "Tutorials" would be a better fit. (Actually, a word which sums up articles, blogs, videos, etc. would be even better, but none comes to mind right now...)

Sorry, it's not a official resources. I will change it.


New version: RandomEngy.VidCoder version 8.26

Pull request has been created with WinGet Automation :rocket:
Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/116431&drop=dogfoodAlpha

Automatic Validation ended with:

/AzurePipelines run


Investigate PropTypes support

https://reactjs.org/docs/typechecking-with-proptypes.html
This is a react thing, but are interesting elements to it.

type checking for prop values passed to components
setting defaultProps via a simplified syntax

In our case we'll have to skip out on defaultProps, but the idea of a defaults overlay is something we could look in to implementing in a different way.
Type checking is interesting because... well, it'll help us find bugs.
The added code could be wrapped in an ifdef block.
// @ifdef DEBUG
only_do_this_in_debug_build();
// @endif

This way there's no overhead added to the runtime app, and we get all the benefits of type checking.

This is indirectly resolved by #1998 introducing relaxed Typescript checking within VSCode. That said that exploration continues here #2065.


Fixed Image and Moving Image must have the same spacing

First off - thank you so much for continuing to respond to issues thus far. I'm trying to use your tool to stitch together a 3x2 microscopy image using the Nuclear stain channel, and following your "SimpleMontage.py" but I am running into a very strange error detailed below:
Error:

RuntimeError                              Traceback (most recent call last)
Cell In [4], line 9
6     montage.SetInputTile(t, greyscale_images[t])
8 print("Computing tile registration transforms")
----> 9 montage.Update()

RuntimeError: D:\a\im\include\itkPhaseCorrelationImageRegistrationMethod.hxx:545:
ITK ERROR: PhaseCorrelationImageRegistrationMethod(000001C3C38771D0): Fixed image and moving image must have the same spacing!
Fixed spacing: [0.100488, 0.100488, 0.967742]
Moving spacing: [0.100488, 0.100488, 0.967742]

Code Block
I follow your SimpleMontage.py tutorial script exactly, but the exact error happens here:

montage = itk.TileMontage[type(greyscale_images[0]), itk.F].New()
montage.SetMontageSize(stage_tiles.GetAxisSizes())
for t in range(stage_tiles.LinearSize()):
montage.SetInputTile(t, greyscale_images[t])
print("Computing tile registration transforms")
montage.Update()

Files
I can make the exact files available, but they are large, so not sure if it is appropriate to send over github. These files first split by channel and converted from .ims images (a bioformats compatable format generated by Andor Dragonfly Confocal Microscopes) to .nrrd via ImageJ/Fiji to try to better match the tutorial.
Please let me know if there's any other information I can send over. And thanks in advance for your help.

spacing: [0.100488, 0.100488, 0.967742] is the spacing to 6 decimal digits. Could you check your files and see whether the spacing differs in some less significant decimal place (tenth, 15th)?

So the images have spacing to 8 digits, but it is identical across all images. Does ITK have a 10^-6 limit in precision?

for file in files:
if '405.nrrd' in file:
readData, header = nrrd.read('/Datasets/Montage NRRDs/'+file)
print(np.sum(header['space directions'], axis = 0))
[0.10048828 0.10048828 0.96774194]
[0.10048828 0.10048828 0.96774194]
[0.10048828 0.10048828 0.96774194]
[0.10048828 0.10048828 0.96774194]
[0.10048828 0.10048828 0.96774194]
[0.10048828 0.10048828 0.96774194]

(for note, spacing is strictly orthogonal in my files, so only the column sum is just a simplification of the following:

array([[0.10048828, 0.        , 0.        ],
[0.        , 0.10048828, 0.        ],
[0.        , 0.        , 0.96774194]])

Internally, ITK uses double for spacing, origin and direction information. It is just that by default, libraries print out less decimal digits. double is usually fully represented by 16-17 decimal places.
The code makes a comparison using all recorded decimals:
https://github.com/InsightSoftwareConsortium/ITKMontage/blob/21f8eb99ae350f4c0d271da149fea2adeda7d6e1/include/itkPhaseCorrelationImageRegistrationMethod.hxx#L542
That might be strict, but when dealing with homogeneous tiles, any deviation in metadata usually indicates an error.
This module is currently not actively developed, so if you want a less strict comparison, please submit a PR.

FWIW I went through and rounded the spacing to 6 decimal places using pynrrd, saved the files, and the montage command seems to work, so this seems like a good workaround given that all my tiles had identical spacing, but was still throwing the identical spacing error check.


Fixed syntax error in an example

Summary
Fixed syntax error in an example on the readme

Coverage remained the same at 88.624% when pulling 3a4f65c245b5ce736cb4662c410ccb0252bb59d2 on ThatTomPerson:master into 7ed434fb30c556b8d291de72d41082f56815a38b on algolia:master.

Thanks a lot!


RFC: Add a "chosen node" for the RT soc series that gives the ability to change the load address

This PR adds a "chosen node" for the RT series that would give the ability to specify in devicetree a different load address than the logical program address (which is the address of the flash chosen node). This functionality allows users to take advantage of the RT boot ROM capability to copy the image from a boot device to a different location.
The PR is marked as RFC in case there is any discussion about this type of functionality being valuable to other vendors as well and needing a more generic and mature solution in zephyr.
Depends on #55233
todo:

document the rt,boot-device chosen node
create example board overlay to show how to use rt,boot-device
add the rt,boot-device chosen node to the rt 3 digit series

I'm not keen on such a devicetree property. Why isn't zephyr,flash sufficient?

The zephyr,flash node currently will decide both the load address of the image and the logical address as the image sees itself (where it will execute from). The RT bootloader specifically has separation of these concepts and has functionality to boot from a different region than where the code will be run from. I am trying to support that functionality of the RT devices with this addition.


Doesn't see the tool's initiation UI on Alfresco Community (Build: 201707)

I doesn't see the tool's initiation UI on Alfresco Community (Build: 201707).  When access to http://<IP_ADDRESS>:8080/alfresco/s/bulk/import only get
`
Web Script Status 404 - Not Found
The Web Script /alfresco/s/bulk/import has responded with a status of 404 - Not Found.
404 Description: | Requested resource is not available.
Message:	01060019 Script url /bulk/import does not map to a Web Script.
Server:	Community v5.2.0 (re21f2be5-b22) schema 10.057
Time:	06-feb-2018 13:39:25
`

That means the AMP file wasn't installed correctly.  Take a look at the installation docs and if you still have issues please post to the mailing list.


Hoverover dropdown "owner" disappears on mouse move

I cannot select the owner I want to, because the dropdown with organisations in the owner field of the create repository form disappears when I try to move my mouse to it.
I'm at 0.5.13, fresh installation from aur, Arch Linux.

Thanks for the report!
But I think it's duplicated to https://github.com/gogits/gogs/issues/882 .

You're right, I missed that one while searching.

@ElessarWebb no worries, this happens often :stuck_out_tongue_closed_eyes:


EIP Author's Manual

First draft of the EIP authors manual. Looking for suggestions for improvement, things that can be added, removed, re-done etc.

Helpful document. I wonder if we should have it as an EIP? cc: @SamWilsn @lightclient

@poojaranjan That's a good idea. If EELS reference implementations become mandatory, we'll have to move this to EIP-1.
Let's leave it here for now, with the intent to move it later?


Add ability to set custom git blame parameters.

This will help with feature like https://git-scm.com/docs/git-blame#Documentation/git-blame.txt---ignore-revs-fileltfilegt
which allows to ignore some revisions in the history.
gitLens already offers such feature. Would be nice to have gitblame do the same.

Hello Jean-Yves!
Thank you for the feature request!
You can set this option by setting the file(s) you want to use as the value to the git configuration blame.ignoreRevsFile.
For more general parameters it is very easy to add something that changes enough of the output that it is either not parsable anymore or it would require extra code to appear in the usable tokens. For parameters that would require new tokens I would rather add them as specific features (like the whitespace option)
I try to keep gitblame as scaled down as possible within reason and if more complex features is your thing GitLens is a great alternative.


Failed to build oneMKL under Ubuntu 22.04

My ENV

➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04 LTS
Release:	22.04
Codename:	jammy
➜  ~ uname -r
5.15.0-37-generic
➜  ~ gcc --version
gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

➜  ~ clang --version
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
➜  ~

CMake ERROR messages:

 CMAKE_BUILD_TYPE: Release
 C compiler: icx was not found in PATH, using clang instead
 The CXX compiler identification is unknown
 CMake Error at CMakeLists.txt:135 (project):
   The CMAKE_CXX_COMPILER:

     dpcpp

   is not a full path and was not found in the PATH.

   Tell CMake where to find the compiler by setting either the environment
   variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
   to the compiler, or to the compiler name if it is in the PATH.

 Configuring incomplete, errors occurred!
 See also "....../oneMKL/build/CMakeFiles/CMakeOutput.log".
 See also "....../oneMKL/build/CMakeFiles/CMakeError.log".

My question: how to build oneMKL successfully from source under Ubuntu 22.04?

quick question: Did you install Intel oneAPI base tool kit from here?
https://www.intel.com/content/www/us/en/developer/tools/oneapi/dpc-compiler.html#gs.3r7dij

@mmeterel
Nope.... I actually built everything from github...
Is there any difference between oneAPI MKL and the oneAPI at Official website ?
Cheers

@jiapei-nexera You need DPC++ compiler to be able to build oneMKL interfaces. Where did you get that from?

@mmeterel
oh.... Okay.... I didn't realise that is a must...
So, DPC++ is able to handle both Intel and AMD chips, right???
Now, I'm downloading DPC++ ...

@mmeterel
Installed... But under ~/intel. How can I specify my own installation directory ? For instance, /opt/intel ?

@mmeterel
I still got some ERROR message while building oneMKL. It looks oneMKL is looking for MKL for building??
CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
   Could NOT find MKL (missing: mkl_sycl_file)
 Call Stack (most recent call first):
   /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
   cmake/FindMKL.cmake:54 (find_package_handle_standard_args)
   src/blas/backends/mklcpu/CMakeLists.txt:23 (find_package)

Hi @jiapei-nexera ,
You can find the cmake command to build the oneMKL interface here.
Note that you need to source in your current terminal the file named setenv.sh in your intel root directory (according to your previous comment, this is ~/intel or /opt/intel).
So just run source /opt/intel/oneapi/setenv.sh. It will set all the environment variables for you.
I hope this helps,

@mmeterel
I still got some ERROR message while building oneMKL. It looks oneMKL is looking for MKL for building??
CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
   Could NOT find MKL (missing: mkl_sycl_file)
 Call Stack (most recent call first):
   /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
   cmake/FindMKL.cmake:54 (find_package_handle_standard_args)
   src/blas/backends/mklcpu/CMakeLists.txt:23 (find_package)

Here, we are trying to build oneMKL interfaces. And by default, it is using Intel oneMKL backends (mklcpu/mklgpu). (oneMKL interfaces and Intel oneMKL are two different things, please see FAQ).
If you don't want to use mklcpu/mklgpu backends, they can be controlled using CMAKE variables as @Sholde pointed out.

So, you are actually saying that:
Intel oneAPI->MKL is the backend, but this github oneMKL is ONLY the interface of that backend?
Is my understanding correct now????

@jiapei-nexera That is true. We use name oneMKL interfaces for this repo and Intel oneMKL to refer the binary product shipped with oneAPI base tool kit. If you can use those names in the future, it would be helpful. :)

@mmeterel
Thank you so much...
Now, everything built....
The ONLY tough thing is now: Open3D.
Are you from the same Intel team???

@jiapei-nexera Anytime!
Unfortunately I am not familiar with it. Do you mind sharing how you are planning to use oneMKL interfaces?

@mmeterel
To tell the truth, I just need to compile Intel oneMKL with OpenCV and some 3rdparty libraries.
I used to use liblapack and libblas directly, for SVD Decomposition. That reminds of my ancient time... I ALSO used jama .... Really, long long long long ago...
Anyway, thank you @mmeterel
You cleared most of my questions....
Thank you ...
Nice dream...


Added appveyor ci build fix #77

PR against develop branch as requested (see #199)

Thank you again! We all owe you a huge beer!


Updating topics-on-dockershim-and-cri-compatible-runtimes page formatting

Followup to https://github.com/kubernetes/website/pull/31528, updating page formatting.

Deploy preview: https://deploy-preview-31661--kubernetes-io-main-staging.netlify.app/docs/reference/node/topics-on-dockershim-and-cri-compatible-runtimes/
Sorry, should have caught the formatting earlier.

/cc @sftim @rolfedh

/lgtm

Related PR: #31662

Closing in favour of https://github.com/kubernetes/website/pull/31662, Tim has a better solution 🙂
/close


[H001060] Rep. Robert Hurt

Rep. Robert Hurt:
http://hurt.house.gov
Likely contact form URL to start with:
http://hurt.house.gov/index.cfm/email-me

YMAL file not found. I am working on it.


ISSUE: Mail and Packaging not starting due to TypeError

Describe the bug
Mail and Packaging Integration not starting correctly.
Environment (please complete the following information):

OS: Docker
Type of device: NUC
Home Assistant version: 0.111.0
Docker: Core
Component version: 0.2.1-b4

Logs
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 178, in _async_setup_platform
    await asyncio.wait_for(asyncio.shield(task), SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/config/custom_components/mail_and_packages/sensor.py", line 134, in async_setup_entry
    sensors.append(PackagesSensor(data, variable))
  File "/config/custom_components/mail_and_packages/sensor.py", line 263, in __init__
    self.update()
  File "/config/custom_components/mail_and_packages/sensor.py", line 311, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper
    result = method(*args, **kwargs)
  File "/config/custom_components/mail_and_packages/sensor.py", line 192, in update
    count[sensor] = get_items(account, "count")
  File "/config/custom_components/mail_and_packages/sensor.py", line 703, in get_items
    if "will arrive:" in email_msg:
TypeError: a bytes-like object is required, not 'str'

Additional context
I rolled back to HA 0.110.6 and the issue still appears to be present.  I also tried to roll M&P back to the last non-beta release - both did not resolve the issue.
Thanks,
DeadEnd

Strange, do you have a Amazon email in your mailbox currently?

Yes I do.

Ok as a quick fix you can change line 703 from:
if "will arrive:" in email_msg:

to
if b"will arrive:" in email_msg:

Doesn't make sense because we convert to str at  email_msg = email_msg.decode("utf-8"), could you turn debugging on for the component and see what it's outputting?

I think I see the problem - here is the last line of the debug where it errors out.
2020-06-11 10:24:13 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Error attempting prase Amazon email: 'utf-8' codec can't decode byte 0xd4 in position 409: invalid continuation byte
2020-06-11 10:24:13 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up mail_and_packages platform for sensor
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 178, in _async_setup_platform
    await asyncio.wait_for(asyncio.shield(task), SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/config/custom_components/mail_and_packages/sensor.py", line 134, in async_setup_entry
    sensors.append(PackagesSensor(data, variable))
  File "/config/custom_components/mail_and_packages/sensor.py", line 263, in __init__
    self.update()
  File "/config/custom_components/mail_and_packages/sensor.py", line 311, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper
    result = method(*args, **kwargs)
  File "/config/custom_components/mail_and_packages/sensor.py", line 192, in update
    count[sensor] = get_items(account, "count")
  File "/config/custom_components/mail_and_packages/sensor.py", line 703, in get_items
    if "will arrive:" in email_msg:
TypeError: a bytes-like object is required, not 'str'

That's helpful, I'll see what I can turn up, likely have something this afternoon to fix it

Well here's a work around sensor.py that's working how it should work when an error is encountered. Give it a shot, it should just bypass any emails that we are not expecting.

Thanks, pulled the linked sensor.py and it is back to working.
Great stuff!


Low image quality even after 300k steps?

Hello,
first - thank you for sharing this repo, I've been looking for something like this for a long time. I run your code for 300k steps, but the quality is still somewhat different from what you shared at step 250k (see the attached picture from step 300k). I am using the GRIT dataset, all 20 parquet files, preprocessed with your script. I run with the same params you shared, but the loss kinda stagnates. I use 12 layers transformer, embed_dim 768, batch size 256 and lr 3e-4, the rest are the default params you have in the repo.  I asuume something is wrong either with the dataset or hyperparams.
Could you please share more details about how you trained your model on 250k steps? Which dataset did you use?
Thank you very much, really appreciate it.

Hey @aabzaliev - interesting results. Agreed the results aren't great (but kind of interesting too). I think it could be a few things.

The noise distribution - the defaults beta_a and beta_b aren't the best - I just changed them - https://github.com/apapiu/transformer_latent_diffusion/commit/2352f0051c40340bacd67920a56c7af91bfab93d - 1 and 2.5 lead to a more skewed distribution that has the model see less noisy data that in my training led to better results.

The data - this is a big one - the full GRIT data might contain a lot of low quality images and/or prompts. Most of the data I used was either synthetic or filtered by CLIP aesthetic score. Try the mj_latents.npy and mj_text_emb.npy from here https://huggingface.co/apapiu/small_ldt/tree/main - this is higher quality synthetic data - I think about 600k examples if I remember correctly.

Model architecture - can you point to what commit you are using or what the architecture looks like? I made some small changes since training the model that could affect training dynamics. Also I used a lr warmup for the first 1k iterations or so that is not in the current training code.

Feel free to try either and let me know what results you get. You can apply both 1 and 2 on top of an already trained model that's no issue.

awesome, thank you so much for such a detailed reponse! I am gonna try to train with the clean data now, and better noise scheduler. I run the experiments last week, cannot check which commit exactly it was, but I am just going to pull the latest from main. I will let you know how it goes!


Contributions toward goals should not reflect fees

If you set up a tier with a goal, and then give money to it, the progress bar only reflects the amount after fees instead of the full contribution.
For example someone gave $50 and it appears like this:

This design choice is a disappointing experience for users wanting to see their amount progressing the progress bar. It's also counter to how every other crowdfunding platform works: if you have a $1000 goal on your kickstarter and their fee is 5%, if two people give $500 then you meet the goal - you don't have to wait for another $50 to come in. Fees are calculated after the conclusion of a campaign, and it should work that way for goals.

H

Problem, some people are expecting the opposite. It's based on their feedback that we had to make it the way it is today.

Problem, some people are expecting the opposite. It's based on their feedback that we had to make it the way it is today.

Hmmm. I guess I can see it both ways. But the Collective admins know much better the situation with fees, so they can put the goal at $1100 if they need $1000 after fees, but financial contributors have no context for that, and shouldn't need to understand how fees work. They just want to give $50 and see $50 show up.
It's probably not important enough to prioritise now, but someday if we are improving goals we should think about it.

Hmmm. I guess I can see it both ways. But the Collective admins know much better the situation with fees, so they can put the goal at $1100 if they need $1000 after fees, but financial contributors have no context for that, and shouldn't need to understand how fees work. They just want to give $50 and see $50 show up.
It's probably not important enough to prioritise now, but someday if we are improving goals we should think about it.

Removing priority.


Swell fixes

Fixed "use-add-item" hook: removed the checkoutId from the request variables. We can't pass the checkoutId to swell because they already use the current session to determine the checkoutId (see use-update-item and use-remove-item).

 {
    "code": "permission_error",
    "message": "You are not allowed to update `items.checkoutId`",
    "param": "items.checkoutId"
  }

Fixed search sort: remove the underscore from the sort values ("price_asc", "price_desc"). Swell expect the format ${field} ${direction}.

Fixed "use-login" hook:

Unhandled Runtime Error. TypeError: Cannot read properties of undefined (reading '0').

Receiving this error when an account doesn't exist.

Moved the product variants normalization to normalizeProduct: For my use case, I had to expand the "use-search" hook to add product variants too. The normalize function didn't work because it was expecting the variants to be normalized before (see the "get-product operations"). It makes more sense to normalize everything inside the normalize function.

Add includeSubCategories to "use-search" hook: In swell, you can define a category tree structure. For example

Category A
	Category B
		Product B1
		Product B2
Category C
	Product C1
	Category D
		Product D1
		Product D2

In the search page, if you filter by Category D, you would get product D1 and D2. But, if you filter C, you would only get product C1, or even worse, if you filter by Category A, the response is "No product found!"

Amazing! Thank you. Taking a look 👀

Hey there! Thank you for your contribution. We have decided to take Next.js Commerce in a new direction and will be closing out current PRs and issues due to this change. Please see this PR for more details: https://github.com/vercel/commerce/pull/966


Small point about the coding

Great work, thank you!
Here is my questions. In the script 'network_architectures.py', many 'pool3's are defined but 'pool2's are used instead. Like this

Maybe it's not very good even though it doesn't matter.
Another question is that I found the 'Self-Ensembling' algorithm had been integrated into the library 'salad' (https://domainadaptation.org/). I'm wondering if this is consistent with your algorithm and if I can use 'salad' directly.
Thank you again for your great job!

Woops! Thanks for spotting that!
Thanks for pointing me to Salad. I had seen it a while back and lost the link, so thanks! They seem to get the results that we would expect, although I don't think they use the specific augmentation necessary to crack MNIST -> SVHN. I haven't checked their code yet.

Hi, I just pushed a new version that fixes the coding issues and is now more compliant with PyTorch v1.0.


[PORT] Add missing documentation to all visible classes, methods, properties and parameters in Adaptive-Expressions

Port this change from botbuilder-dotnet/master branch:
https://github.com/microsoft/botbuilder-dotnet/pull/4308

fixes: #4283
Add documentation in Adaptive-Expressions, for visible classes, methods, properties and parameters, including

Built-in Functions
FuntionUtils
ExpressionProperty
TriggerTree

Changed projects

AdaptiveExpressions

this is a part of #2602

this is a part of #2602


View page source is broken

When clicking on the 'view page source' button, it does not go to the correct github page.  Config issue?  Docsy seems to be looking for a branch named main, and the docs are on master.

Resolved in d58ee8d.


Adding a few basic comments to give GoDoc some content

Codecov Report

Merging #10 into master will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff          @@
##           master   #10   +/-   ##
====================================
  Coverage      26%   26%
====================================
  Files          16    16
  Lines         314   314
====================================
  Hits           82    82
  Misses        228   228
  Partials        4     4

Impacted Files
Coverage Δ

developer_account_balance.go
0% <ø> (ø)
:arrow_up:

sms.go
100% <ø> (ø)
:arrow_up:

call_base.go
71% <ø> (ø)
:arrow_up:

verify.go
0% <ø> (ø)
:arrow_up:

sms_base.go
71% <ø> (ø)
:arrow_up:

auth.go
33% <ø> (ø)
:arrow_up:

insight.go
62% <ø> (ø)
:arrow_up:

developer_base.go
71% <ø> (ø)
:arrow_up:

calls.go
0% <ø> (ø)
:arrow_up:

insight_base.go
71% <ø> (ø)
:arrow_up:

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7871712...ec5ad8a. Read the comment docs.


How can I run a job which will start exactly at 12:00 a.m. of everyday and run once each day?

@register_job(scheduler, "interval", days=1)
What should we be providing in the decorator so as to achieve this task?

After a bit of going through the codes of APScheduler and django-apscheduler I understood that it can be achieved by using the APScheduler expressions.
@register_job(scheduler, "cron", hour=0)
This makes us achieve the task.
Thanks


[Feature Request] remove from tracking site as well as library

Why/User Benefit/User Problem
It makes managing library and tracker easier as we can keep them in sync with what we want. We can remove series we dont like from both library and tracking site.
What/Requirements
A small button to remove from tracking sevice after removing tracking from series

Duplicate #92


Complete passwordless+surrogation scenarios (Puppeteer)

There are three passwordless scenarios (Puppeteer):

token authn: passwordlesss-login
password authn: passwordless-login-with-password
delegation authn: passwordless-delegated-login.

I have updated/created the equivalent scenarios with impersonation:

surrogation + token authn: passwordlesss-login-surrogate
surrogation + password authn: passwordless-login-with-password-surrogate
surrogation + delegation authn: passwordless-delegated-login-surrogate.

For all scenarios, I check the authn attributes to see if there is surrogateEnabled, surrogateUser and surrogatePrincipal and their appropriate values depending on the use cases.
From the tests, surrogation + password authn does not work and surrogation + delegation authn partially works.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks you Jérôme. I'll review this tomorrow.

Excellent! Thank you very much. I will certainly check that in January after my holidays (starting tonight :-)


Marvin toolkit container

This is an attempt to make the Marvin attack analysis more reproducible for RustCrypto/RSA. The Dockerfile and entrypoint script are both rough proofs of concept. It would be great if the maintainers can suggest how we can handle the outputs of the analysis, whether to test rsa1024 and rsa4096, whether this should be part of CI/CD, among other things.

This looks fine to me. Will try it out locally.

@tarcieri @dignifiedquire
This PR is ready for review. The main feature of this PR is a container that packages the Marvin toolkit. See the added README.md for how to build and run the container. Some notable features include:

CLI allows specifying -s <RSA_bits> -n <repeat>
Custom Cargo.toml for the test harness if custom version of RustCrypto/RSA and/or RustCrypto/crypto-bigint is needed
Preserve the generated keys, ciphertexts, and analysis results by copying them into a location that can be mounted into the host file system

Cool, I'll try to test it out soon


Reproduction Failure

Attempting to reproduce the effect of 1.6b step1 SFT using the default single_node script configuration resulted in slow training on 4 V100 32G GPUs. It took 6 hours to complete, which is much slower than expected.

Hi @JingerAI - I'm a little confused. There is on 1.6b model. Did you intend to write 7.6b or 1.3b?

I think @JingerAI want to say 1.3B, i also trained the demo 1.3B model slow.
Maybe there are some missing setting issue.


Added elasticsearch7 to elasticsearch docker

Status
In Progress (Reason for hold)
Related Content Pull Request
Related PR: link to the PR at demisto/content
Related Issues
Related: CIAC-10581
Description
A few sentences describing the overall goals of the pull request's commits.

Continued the work on this PR - pull/32947.


Fix unresponsive radial button

Overview
This PR fixes the problem of radial buttons in the radial set prefab are sometimes unresponsive. Investigation of the matter showed that in the pressable button script (Press distance - release distance delta) must be larger than start push distance or the button will not work properly.
Changes

Fixes: #8517.

Verification

Add the RadialSet prefab to your scene
Press each toggle once via near interaction
Try to press the toggles again and notice they still respond properly

Also validated on a HoloLens 2 device.

/azp run


Maintain collection of last image layers to speed-up get_latest_image method use in LayerMap::search

It will be nie to test it at captest to check if  compaction still uses 100% of CPU.

At this point, I think we should revert commit f3073a4db93e2d4e39e2bbef03ed6b742ef3afa0. The old linear search wasn't fast, but I don't have much confidence in the new code. We need to do our homework properly, and add unit tests and benchmarks for all the operations.

cc @yeputons

Frankly speaking I do not think that it it good idea to revert back R-Tree and replace it back with linear search.
rstar seems to be quite stable package and R-Tree - well known and widely used data structure.
Using i256 type for representing our 144 bit key may be considered as overkill and expensive, but this operation are not visible in profile. So I am against replacing R-Tree with dummy liner search. Even for free tier with 10Gb limit, number of image layers c an be up to 400. And taken i account one week history we can easily get several thousands layers where liner search will be very inefficient.
Even now we see problems  with 100% CPU usage in compaction at staging and captest. I am not sure that just adding latest age image map will solve this problem (I can check but do not see much sense i it).
I have performed some benchmarking using my test_layer_map.py test (slightly modified).
I just measure time of inserting data, vacuuming and traversing it.
Also I measured time of several last compaction iterations (which actually does nothing but consuming CPU).
Results are below:
latest_image_map:
test_layer_map.insert: 130.927 s
test_layer_map.vacuum: 105.797 s
test_layer_map.select: 432.148 s
total: 800.49s
compaction: min=1, max=834, avg=203.26

main:
test_layer_map.insert: 183.649 s
test_layer_map.vacuum: 114.857 s
test_layer_map.select: 530.850 s
total: 961.11s
Compaction: min=2.23147, max=3100.94, avg=775.338

main w/out R-Tree (reverted https://github.com/neondatabase/neon/commit/f3073a4db93e2d4e39e2bbef03ed6b742ef3afa0):
test_layer_map.insert: 169.829 s
test_layer_map.vacuum: 865.514 s
test_layer_map.select: 5,157.619 s
total: 6321.52s
Compaction: min=1, max=169255, avg=42535.8

So, you can see that this PR has not so much influence on insert/search speed, but allows to reduce compaction time about 3 times (actually it is not compaction itself, but preparation for compaction).
And implementation without R-Tree is extreeeeemly slow.
I do not want to say that R-Tree is most optimal data structure for this task. If @yeputons implementation of segment tree will be faster and simpler, then we should definitely use it. But just to be able to compare it with R-Tree we need to preserve current implementation.

So, looks like (thanks to @yeputons investigation) consumption of CPU at captest is not cause by LayerMap inefficiency.
And noticeable benefits of R-Tree in performance are obtained with several thousands layers.
For example, my results above were obtained with number of layers = ~50000. With size of layer 256Mb, it means database size ~10Tb. Nit sure that we are ready to handle such for such volume right ow. ut no doubt that in future we will have to deal with it.
The question is whether R-Tree can cause some slowdown for small number of layers, with our default setting and typeicall database size. In this case linear search is definitely faster than any other lookup method.
I have changed my test_ayer_ap example to use smaller database size  and default parameters. So there is just one layer.
Results:
latest_image_map:
test_layer_map.insert: 1.108 s
test_layer_map.vacuum: 0.613 s
test_layer_map.select: 1.791 s

main w/out R-Tree:
test_layer_map.insert: 1.081 s
test_layer_map.vacuum: 0.639 s
test_layer_map.select: 1.912 s

So there is almost no difference.
This is why I think there is completely no reasons to revert R-Tree implementation.

As discussed moving to draft


"Invalid bounds string" with the example setting from the help

MethylDackel extract --help says:
--OT INT,INT,INT,INT Inclusion bounds for methylation calls from reads/pairs
                  originating from the original top strand. Suggested values can
                  be obtained from the MBias program. Each integer represents a
                  1-based position on a read. For example --OT A,B,C,D
                  translates to, "Include calls at positions from A through B
                  on read #1 and C through D on read #2". If a 0 is used a any
                  position then that is translated to mean start/end of the
                  alignment, as appropriate. For example, --OT 5,0,0,0 would
                  include all but the first 4 bases on read #1. Users are
                  strongly advised to consult a methylation bias plot, for
                  example by using the MBias program.

But when I try to use --OT 5,0,0,0 I get:
$> MethylDackel extract -q 1 -p 20 -D 250 --OT 5,0,0,0 --cytosine_report hg38_full_gatk_HPV_HBV_HCV_spike-ins.fa test.bam
Invalid bounds string, 5,0,0,0

Any idea what's going on there?

Make sure those are regular commas and not getting converted to some weird comma-like unicode characters. That's my only guess as to what's going wrong there. Otherwise that error message should only appear if the values are negative or if there aren't 4 of them.

nope, they are definitely true commas. I reverted to 4.0 and it works, back to 5.1 or current HEAD, and it doesn't

The 0s are not correctly handled somehow. With 4.0, this worked --OT 10,0,0,140, but in 5.1, it throws said error. However, this works: --OT 10,150,1,140.
In this context: I don't really understand the --nOT parameter. How would I tell MethylDackel to trim 10bp from the beginning of each read, whether aligned to the forward or reverse strand, and irrespective of the length of the read (they are quality trimmed so uneven length)? I thought it would be --nOT 10,0,10,0, but it seems that --OT and -nOT syntax counts the reverse read in the orientation of the ref. Annoyingly, that means I have to know the length of the reverse read to trim the "beginning" of the read?

There can be up to 4 reverse reads in BS-seq, and --nOT 10,0,10,0 will trim off the first 10 bases of those arising from the original top strand regardless of their length. I assume the confusion is that not all reverse orientation reads arose from the same strand (everyone finds this confusing at first). You never need to know the length of a read to trim its beginning; the beginning of a read is it's 5'-most base in the orientation of its alignment (i.e., the first untrimmed base that the sequencer produced).

I think there are two confusions here :-D I should not have mixed up two issues. One is that there seems to be a bug in the parsing of 0s in the pattern string since version 0.5. The other "issue" is simply my poor understanding of the --nOT pattern syntax, which shouldn't be here in this tread, sorry 🙈

I'll definitely look into the parsing bug, thanks for reporting it :)