Modifying pact-consumer-js-dsl to work with both Nodejs and Web projects

Changed a few things around the build process which I think is needed to maintain the codebase, adding nodejs example, updating readme to make it easier for people to start using pact-mock-service.  Updated bower file to ignore certain files that aren't needed.  Registered pact-consumer-js-dsl with Bower, can view information with bower info pact-consumer-js-dsl.

Forgot to mention, version number has been incremented to 0.0.5 and would be nice to create a new release after the pull request is done.

@bethesque @BenSayers @markdalgleish
I've updated the build to include a UMD wrapper, effectively fixing all our issues with the js consumer, for all our use cases :)

Thanks Michael. I'm going to merge https://github.com/DiUS/pact-consumer-js-dsl/pull/21 first when I have a moment, because it's a bigger change, and I don't want a potential conflict to block it.

Sure, but be aware that there are some conflicts between our two pulls.  However, async http requests are good :)

Sorry, I know I've made your life harder now by merging in the node stuff, but it's for the best! Can you have a go at merging that into your branch, and then ping me when it's good to go? Thank you kindly!
Btw, I'm a bit confused about the browserify stuff. What's the difference between this approach and using browserify (apart from not having the extra dependency)?

@bethesque Browserify is essentially trying to do the same thing I'm doing with UMD.  The end goal is about having one source that works for all use cases (web, node, etc).  I like UMD in this case since this library is fairly small and doesn't have any extra libraries except for xmlhttprequest in nodejs, which I've fixed in this fork.
It's a matter of preference and in my opinion, Browserify is a tad overkill for what we're doing here since we don't really have any other requirements needed.  Unless we have a lot of them in a future, I can't recommend this at this very moment.
I'm merging in your changes right now and try to sort it all out.  Shouldn't be too hard.

@bethesque Well, I'm not entirely sure if it went missing, but it's not part of the release on the github page.  It used to be there if I recall, but somebody seems to have removed it?

@bethesque I've merged master into my PR.  There's still some conflicts of course, but you should be able to simply merge my changes and hard reset (removing all merge conflicts) to my latest commit.  I've tested everything and it works fine.
However, I've noticed that the 'clean' function is now missing from the mockService, and that when creating it, I must give a done function in the options.  I'm not too sure if this is your doing, but I can't say that I'm a fan of it.  Having the extra functions there for workflow is fine, but changing my control over how I want to use pact for my use case is not.
If this is not one of your changes (something added from the PR), then I can modify them.

The clean function is going to become irrelevant soon, because there is a new PUT /interactions endpoint that will take care of DELETE /interactions and POST /interactions. There's no point to setting up interactions sequentially in the Javascript use case - it was just a legacy of the way the Ruby code uses the service, and it makes the code HORRIBLE!  I will modify the code to use the new endpoint as soon as I have a moment.
The "done" function is required now that we have gone fully async. There needs to be a way for the mock service verify function to fail a test after the pact run has completed, if the expected interactions do not match the actual interactions. I need to document this properly in the README.

As you mentioned, there are still conflicts. This means that master has not been correctly merged into your branch. Can you resolve the conflicts, and then ping me when you've finished? Hard resetting is not a PR merging option!

Thanks for the browserify explanation. I also prefer a smaller dependency.

Alright, fixing merge conflicts right now.  I already did it earlier, but I think it didn't "register" that the merges were fixed for some reason.

@bethesque Fixed :)

@bethesque  I'm not sure I agree completely with the direction you're going.  So, you're saying that there can only be one 'setup' of the interactions.  Fine for us since we're not using it like that, but I'm not positive that it would be the use case for everyone.  It could be that some interactions needs to be added along the way in the tests (you know, nested tests potentially).
I'd also like to have the option to write out the schema without verification.  This of course shouldn't be used for production code, but while we're setting up the project I've created a lot of the interactions that we will be using on the front-end, but aren't using right away (still have to create the tests).  However, I still want the backend to test on something as a first version until we lock everything down.  This is a bit of an edge case, but I can see where it can get useful for many people starting.
I've also noticed that with version 0.2.3 of pact-mock-service, when I call write, it doesn't write all interactions, only the ones that's been hit, making my use case to give the backend guys something to start on completely useless.  ie. The backend can't be even built before the front-end interactions are complete, which is a big problem since we expect that the backend will in fact be ahead of the front-end since we can't deploy the front-end before the api is out.
This is not a showstopper exactly, but it is a bit annoying and would like for the pact service to consider our use case.

@bethesque I'd also like to say that the current way that the consumer has been coded won't work for our use case at all.  I feel like the library has been coded with only one use case in mind, which is unit testing, which is fine, but we need flexibility to allow e2e tests as well, which we do in our case.
We have one setup and one teardown (like the example) in which, with the changes done to the mock service, isn't possible since we can't verify and write without it being in the same file, which the reference to the callback is lost between them.

So, you're saying that there can only be one 'setup' of the interactions.

The normal flow would involved one set up per test.

Before test: Set up interactions for only this test (in the pre 0.3.0 world this would involve a DELETE /interactions and multiple POST /interactions, but in 0.3.0 you can just do PUT /interactions)
Test: execute
After test: Verify interactions and write updated pact

I've also noticed that with version 0.2.3 of pact-mock-service, when I call write, it doesn't write all interactions, only the ones that's been hit, making my use case to give the backend guys something to start on completely useless.

From my memory, this has always been the case, from the beginning of Pact time. I thought I'd double check my memory, so I wrote a quick test here with version 0.2.2, and it performs as I expected. What version of the pact-mock_service did you observe this behaviour in?
So, to clarify, you want keep public set up and verify methods, so that you can call them independently of the run method? I'm sure we can work something out, but what's confusing me is that I thought node was supposed to be all asynchronous? It confuses me that there's code in the setup and tear down example that uses exception throwing and is synchronous, but the previous PR was supposed to be supporting node by making it all asynchronous.

I'm just wondering... maybe we're trying to do the wrong thing by trying to support asynchronous use and synchronous use in the one codebase. Maybe they should be split off. Don't know yet, just pondering.

@bethesque we don't support synchronous use anymore.

@bethesque Don't take my current example as word for word.  The synchronous part is only there because that's how the past pact-consumer-js-dsl was working.  It definitely can be asynchronous and probably should.  Frankly, this is not my issue with it, it's the use case where we have to use the library exactly the same way; have to be unit tests, have to setup and teardown within the same javascript file.
The way I envision pact working for us, is to be both mock server while developing, and the same interactions can be used as the contract.  This makes a lot of sense for us since we wouldn't have to duplicate our efforts for both pact and mocking.  We're also using pact with protractor tests (interaction tests) which are very thorough on how we want to use our application in the real world.
In our case, some of the design concepts for the consumer dsl simply won't work.
Maybe we should get together and nail this out more.  I feel that we're close to getting something that'll work for everyone and I will admit that I'm not a Ruby expert (or even a user), so I'm not sure what is the standard in that world, but I'm sure we can create something that can work for everyone.

@mboudreau what you are asking for sounds very similar to something we are pondering on as well.
In essence Pact right now is very oriented toward unit testing consumer abstractions over provider apis. And these tests are geared toward single interactions. However obviously that on its own is not enough to give you confidence. So in addition functional/integration tests are needed, which can test the consumer application as a whole with all the provider api's stubbed out. I say stubbed instead of mocked intentionally here as in these tests it is important that the stub behaves the same way as the provider api when it is called, but we do not care how many times it was called or if it was even called at all.
So far it seems the recommendation is to not use Pact to write api/integration tests, but instead create a separate stub that shares data used in the Pact tests. The concern with this is it relies on developers not making a mistake when setting up the stub. The stub may have endpoints added to it that do not have any corresponding Pact tests for example. These sort of mistakes will go unnoticed as the stub is not validated against the real provider like the Pact tests are.
I've been thinking that Pact is in a good position to solve this problem and have wondered what a solution that generated the stub server off the Pact file would look like. This seems to me to be half of what you are asking for @mboudreau. I think the other half of what you are asking for is in a world where Pact is solving the stub server problem for you, how do you write the integration/api tests before you write the Pact unit tests (if you are following a BDD style of development for example).
@bethesque what are your thoughts on this subject?
This whole conversation is a bit off topic from the PR - does it belong somewhere else?

@BenSayers Thanks for that, that clears up a lot of things.  That is essentially what I'm trying to accomplish with our project.  I agree that the overall discussion should be taken elsewhere, but it is relevant to this PR somewhat since I'm trying to change the consumer dsl to work for both our use cases.
This is where @bethesque comes in to say if she likes my idea so I can go ahead and change my master on my fork.  I wouldn't want to spend time fixing the consumer dsl if it's not going to get merged into the main project.
I would definitely love to be continually part of the direction of pact.  I think it shows great potential for solving a lot of issues in the software lifecycle.

I don't see a problem with making the methods to set up and tear down interactions public so that they can be used independently of run method. That should allow @mboudreau to continue his integration testing approach.
The idea of setting up pact to work as a stub server from an existing pact file could work, but we'd need to work out an approach to a certain issue. One of pact's strengths is that you can test the same request, with different responses (eg. 200, 404, when x has a y, when x has no y etc...). This works because you set up the interactions on the mock server on a test-by-test basis - you only have one response mocked for a given request at a time, so it always knows which response to give.
If we used the generated pact file to set up a stub server, it is inevitable that there would be more than one response found for a given request - we'd need to work out how to handle this situation. We can't just take the first response found, because the order of the unit tests determines the order in which the interactions are written to the file, so it would not be predictable.
The other subtlety is that in a unit test, we're at a fine grained level, and we generally want to check exact values of an outgoing request - that's what tells us that "orderId: 345" on our model is being converted to "{order_id: 345}" in our request body. If we use pact for stubbing for integration tests, we want to be much less strict about that - we don't care what the value of the order id is, we just want the mock server to respond to any PUT order request that has the right document structure. We already have this type of matching implemented, we'd just need a way to turn it on when in "stub" mode.
Sorry this is a bit long winded. What we need is:

A way to choose which of the interactions that we used in a unit test end up being used in a stub test.
A way to relax the matching so the integration tests don't become brittle.

I will give these some thought.

Ok, I've started a new issue on the mock server repo, let's take this discussion there, because it's more about the mock server than it is about the DSL. The DSL is just javascript syntax sugar over the mock server behaviour.

@bethesque Any way of getting my PR in?

Hi Michael, I am not happy to merge these changes as it now pulls too far towards the integration use case, and make it too easy for errors to be lost. I would like verification to be on by default (done must be a mandatory option) and if you want to turn the error checking off, you can pass in an empty function. I am going to have a try to do this myself, but if it gets too complex, I will pass it back to you.

Also, there are WAY too many changes in this PR! I'll add some instructions to the CONTRIBUTING.md file, but there are some basic PR guidelines for open source contributions which are:

Keep the change set focussed and as small as possible to achieve one goal.
Put each change set on a branch, and create a pull request from that branch (otherwise, every commit you make to your own master gets added to the PR)

Also, rule number 3: Always always submit tests for the code that you have changed or added.

Other than wrapping the library in a UMD (which is tested by actually running gulp), merging the changes from master (from your request as per Ben's PR) and making the write/verify public (which is already tested).  I guess I can add some tests around write and verify to be public, but their functionality should already be tested by the tests that were already there.
I tried to achieve one goal through this PR, but you asked me to wrap several things, so I did.

I had a go, but ran out of time today, can you pick this up? This is what I'd like:

Revert all the changes that made the done function optional - an empty function can be provided if someone does not want to check for errors, but I don't want to make it easy for errors to be swallowed.
Revert the changes to run() and let's make cleanAndSetup() public and call it instead, because that is all the run function is doing if you call it with no args. The name is a bit messy, how about just setup?

I appreciate all the work on the README and the examples, that looks good.

I don't get a notification when there are new commits to the PR, so I haven't been ignoring this, just didn't know the previous changes were ready for review!

@bethesque I've changed what you asked for.

Many thanks! I will be looking at this as soon as I have a spare moment.

@bethesque What's the status on this?  I saw that you did some merging from my branch yesterday.

Hi Michael,
I've got it on a branch while I add some unit tests, but I'm having some very strange problems. I've added unit tests, and now have a task called run-unit-tests. When I run run-unit-tests and run-browser-tests separately, they pass, but when I run them together as part of run-tests, then I get:
1) MockService run when there are no errors invokes the done callback with null
1.1) ReferenceError: Pact is not defined
1.2) TypeError: Cannot call method 'run' of undefined

You're more experienced with gulp than I am, if you have any idea what might be causing that, I'd appreciate a pointer! It's on this branch: https://github.com/DiUS/pact-consumer-js-dsl/tree/mboudreau-master

@bethesque Created a new pull request with what you're asking for.  It should fix every problem.  The major one being that the tests used to run in parallel and was competing for the same pact mock service.  There was also a few issues with the integration code that was doing web or node specific stuff (which it shouldn't).  Should just be a simple pull, npm install/update and running gulp :)
https://github.com/DiUS/pact-consumer-js-dsl/pull/29

It's not clear to me what this pull request is trying to achieve. I am under the impression the project already does support nodejs and web projects.
I'm also concerned about the way in which node-xhr2 has been introduced by this PR
 global.XMLHttpRequest = require('xhr2');

I understand it is desirable to stop the code from caring about if it is running inside a nodejs context or a browser based one and agree it is something we should refactor in the code. However this particular attempt at doing so creates a global that will be leaked into every javascript file in the node executable and is bad practice. I think using browserify we can solve this problem more cleanly.

@BenSayers This is a matter of preference in the end, and Beth and I agree that having one codebase and not 2 (one for node, one for web) is a lot easier to maintain.
Frankly,  I like Browserify, but I feel like you're trying to use a sledgehammer for a job needed for a regular hammer.  The Pact consumer DSL is really not that complex.  Browserify's main point of use is to reduce complexity with require.js and AMD modules.  We're not doing any of that.  Seems to me that it's just another added layer of abstraction/complexity for a project of this size.
If Pact gets bigger (and the consumer dsl), then we should maybe look into Browserify to reduce complexity, but I don't think that time is now.

Closing, this is being merged manually.


"TDA Armin failed!" when just recording

When performing configuration and recording in one step everything works fine
$ ./mmwave --capture-dir testdir1 --configure --record
STARTSTATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS    0 | DEV MAP:  1 | [MASTER] Power up successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Firmware successfully uploaded!
STATUS    0 | DEV MAP:  1 | [MASTER] CRC type has been set!
STATUS    0 | DEV MAP:  1 | [MASTER] RF successfully enabled!
STATUS    0 | DEV MAP:  1 | [MASTER] Channels successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] ADC output format successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] Init completed with sucess

STATUS    0 | DEV MAP:  2 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  4 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  8 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP: 14 | [SLAVE] Firmware successfully uploaded!
STATUS    0 | DEV MAP: 14 | [SLAVE] CRC type has been set!
STATUS    0 | DEV MAP: 14 | [SLAVE] RF successfully enabled!
STATUS    0 | DEV MAP: 14 | [SLAVE] Channels successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] ADC output format successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] Init completed with sucess

STATUS    0 | DEV MAP: 15 | [ALL] RF deivce configured!
STATUS    0 | DEV MAP: 15 | [ALL] LDO Bypass configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Data format configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Low Power Mode configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] RF successfully initialized!
STATUS    0 | DEV MAP: 15 | [ALL] Datapath configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Profile configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Chirp configuration successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Frame configuration completed!
STATUS    0 | DEV MAP: 14 | [SLAVE] Frame configuration completed!
STATUS    0 | DEV MAP: 15 | [MIMO] Configuration completed!

STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] Arming TDA
STATUS    0 | DEV MAP: 15 | [MMWCAS-RF] Framing ...
STATUS    0 | DEV MAP: 32 | [MMWCAS-RF] Stop recording

When I'm just recording (directly after the command above) I get the following error message:
$ ./mmwave --capture-dir testdir2 --record
STARTSTATUS   -8 | DEV MAP: 32 | [MMWCAS-DSP] TDA Arming failed!

Same thing happen when I do the configuration only and then try to record a frame:
./mmwave --configure
STARTSTATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS    0 | DEV MAP:  1 | [MASTER] Power up successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Firmware successfully uploaded!
STATUS    0 | DEV MAP:  1 | [MASTER] CRC type has been set!
STATUS    0 | DEV MAP:  1 | [MASTER] RF successfully enabled!
STATUS    0 | DEV MAP:  1 | [MASTER] Channels successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] ADC output format successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] Init completed with sucess

STATUS    0 | DEV MAP:  2 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  4 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  8 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP: 14 | [SLAVE] Firmware successfully uploaded!
STATUS    0 | DEV MAP: 14 | [SLAVE] CRC type has been set!
STATUS    0 | DEV MAP: 14 | [SLAVE] RF successfully enabled!
STATUS    0 | DEV MAP: 14 | [SLAVE] Channels successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] ADC output format successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] Init completed with sucess

STATUS    0 | DEV MAP: 15 | [ALL] RF deivce configured!
STATUS    0 | DEV MAP: 15 | [ALL] LDO Bypass configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Data format configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Low Power Mode configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] RF successfully initialized!
STATUS    0 | DEV MAP: 15 | [ALL] Datapath configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Profile configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Chirp configuration successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Frame configuration completed!
STATUS    0 | DEV MAP: 14 | [SLAVE] Frame configuration completed!
STATUS    0 | DEV MAP: 15 | [MIMO] Configuration completed!

$ ./mmwave --capture-dir testdir2 --record
STARTSTATUS   -8 | DEV MAP: 32 | [MMWCAS-DSP] TDA Arming failed!

Do you have any hint what the problem is there?
BR Christian

Hi @chuber1986 ,
Indeed, using the options --configure and --record separately does not work as expected. My initial guess would be that the MMWL_ArmingTDA function is failing either because of a closed connection with the board or "still in use". Not sure what's the cause yet.
I could investigate this a little further. Do you mind testing when I ready a branch for this?!
I've developed this with the hardware of the university but I've finished my studies already. I'm planning to acquire a board myself but, for now, I couldn't directly implement and test bug fixes or new features

Hi @azinke,
sure, I can test that! :)
Just let me know then the branch is ready.
BR Christian

Hi @azinke !
Sorry, for the late answer. In this commit are 2 errors:

In int MMWL_ConfigureDeviveMap(unsigned char deviceMap) {} you return retVal once which is not define. This should be status, I guess. For consistency I'd rename status to retVal.
In int MMWL_ConfigureDeviveMap(unsigned char deviceMap) {} the timeOutCnt valiable was not defined.

After fixing these it again compiles, but unfortunately it doesn't fix the issue. I still get
$ ./mmwave record --capture-dir testdir2 STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected! STATUS   -8 | DEV MAP: 32 | [MMWCAS-DSP] TDA Arming failed!
BR Christian

Hi @chuber1986 !
Thank you very much for the feedback and sorry for the late response. It also took me some time to spot the issue with the "arming" without the board. But I also received helpful debugging logs to investigate this issue.
I've added a new commit to the Pull Request. Could you please test it and let me know of the outcome?!
Thanks!

Hi @azinke!
Unfortunately, I still get the same error:
> ./mmwave --help
usage: mmwave [command] [option]

Configuration and control tool for TI MMWave Evaluation Modules

Arguments:
    configure                      Configure the MMWCAS-RF-EVM board
    record                         Trigger data recording. This assumes that configuration is completed.
      -d,  --capture-dir           Name of the director where to store recordings on the DSP board
      -p,  --port                  Port number the DSP board server app is listening on
      -i,  --ip                    IP Address of the MMWCAS DSP evaluation module
      -t,  --time                  Indicate how long the recording should last in minutes. Default: 1 min
      -f,  --cfg                   TOML Configuration file. Overwrite the default config when provided
      -h,  --help                  Print CLI option help and exit.
      -v,  --version               Print program version and exit.

> ./mmwave --version
mmwave 0.1, Copyright (C) 2022 - 2023

> ./mmwave configure
STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS    0 | DEV MAP: 15 | [MMWCAS-DSP] TDA Initialized!
STATUS    0 | DEV MAP:  1 | [MASTER] Power up successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Firmware successfully uploaded!
STATUS    0 | DEV MAP:  1 | [MASTER] CRC type has been set!
STATUS   -8 | DEV MAP:  1 | [MASTER] Error: Failed to enable master RF

> ./mmwave configure -f config/short-range-cfg.toml
STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS    0 | DEV MAP: 15 | [MMWCAS-DSP] TDA Initialized!
STATUS    0 | DEV MAP:  1 | [MASTER] Power up successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Firmware successfully uploaded!
STATUS    0 | DEV MAP:  1 | [MASTER] CRC type has been set!
STATUS   -8 | DEV MAP:  1 | [MASTER] Error: Failed to enable master RF

> ./mmwave configure -f config/very-short-range-cfg.toml
STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS    0 | DEV MAP: 15 | [MMWCAS-DSP] TDA Initialized!
STATUS    0 | DEV MAP:  1 | [MASTER] Power up successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Firmware successfully uploaded!
STATUS    0 | DEV MAP:  1 | [MASTER] CRC type has been set!
STATUS    0 | DEV MAP:  1 | [MASTER] RF successfully enabled!
STATUS    0 | DEV MAP:  1 | [MASTER] Channels successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] ADC output format successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] Init completed with sucess

STATUS    0 | DEV MAP:  2 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  4 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  8 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP: 14 | [SLAVE] Firmware successfully uploaded!
STATUS    0 | DEV MAP: 14 | [SLAVE] CRC type has been set!
STATUS    0 | DEV MAP: 14 | [SLAVE] RF successfully enabled!
STATUS    0 | DEV MAP: 14 | [SLAVE] Channels successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] ADC output format successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] Init completed with sucess

STATUS    0 | DEV MAP: 15 | [ALL] RF deivce configured!
STATUS    0 | DEV MAP: 15 | [ALL] LDO Bypass configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Data format configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Low Power Mode configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] RF successfully initialized!
STATUS    0 | DEV MAP: 15 | [ALL] Datapath configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Profile configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Chirp configuration successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Frame configuration completed!
STATUS    0 | DEV MAP: 14 | [SLAVE] Frame configuration completed!
STATUS    0 | DEV MAP: 15 | [MIMO] Configuration completed!

STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA successfully disconnected!

> ./mmwave record
STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS   -8 | DEV MAP: 32 | [MMWCAS-DSP] TDA Arming failed!

Hi @chuber1986 !
Have you maybe forgotten to fetch the new update before building the source? In the latest update, I've moved the "Frame Configuration" steps in the record command. And it seems to be appearing in configure in the logs. I mean, these lines
STATUS    0 | DEV MAP:  1 | [MASTER] Frame configuration completed!
STATUS    0 | DEV MAP: 14 | [SLAVE] Frame configuration completed!

With a custom configuration file, the commands would be as follows:
# Prepare the DSP board for recording
./mmwave configure -f config/short-range-cfg.toml

# Trigger recording
./mmwave record -f config/short-range-cfg.toml

But it's also possible to use the default config
# Prepare the DSP board for recording
./mmwave configure

# Trigger recording
./mmwave record

Morning @azinke ,
you are right. It seams that my git-pull failed without me recognizing it.
Now I get:
> ./mmwave configure -f config/v
ery-short-range-cfg.toml
STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS    0 | DEV MAP: 15 | [MMWCAS-DSP] TDA Initialized!
STATUS    0 | DEV MAP:  1 | [MASTER] Power up successful!
STATUS    0 | DEV MAP:  1 | [MASTER] Firmware successfully uploaded!
STATUS    0 | DEV MAP:  1 | [MASTER] CRC type has been set!
STATUS    0 | DEV MAP:  1 | [MASTER] RF successfully enabled!
STATUS    0 | DEV MAP:  1 | [MASTER] Channels successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] ADC output format successfully configured!
STATUS    0 | DEV MAP:  1 | [MASTER] Init completed with sucess

STATUS    0 | DEV MAP:  2 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  4 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP:  8 | [SLAVE] Power up successful!
STATUS    0 | DEV MAP: 14 | [SLAVE] Firmware successfully uploaded!
STATUS    0 | DEV MAP: 14 | [SLAVE] CRC type has been set!
STATUS    0 | DEV MAP: 14 | [SLAVE] RF successfully enabled!
STATUS    0 | DEV MAP: 14 | [SLAVE] Channels successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] ADC output format successfully configured!
STATUS    0 | DEV MAP: 14 | [SLAVE] Init completed with sucess

STATUS    0 | DEV MAP: 15 | [ALL] RF deivce configured!
STATUS    0 | DEV MAP: 15 | [ALL] LDO Bypass configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Data format configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Low Power Mode configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] RF successfully initialized!
STATUS    0 | DEV MAP: 15 | [ALL] Datapath configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Profile configuration successful!
STATUS    0 | DEV MAP: 15 | [ALL] Chirp configuration successful!
STATUS    0 | DEV MAP: 15 | [MIMO] Configuration completed!

STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA successfully disconnected!

> ./mmwave record -f config/very-short-range-cfg.toml
STATUS    0 | DEV MAP: 32 | [MMWCAS-DSP] TDA Connected!
STATUS   -2 | DEV MAP:  1 | [MASTER] Frame configuration failed!

I tried to do the PowerUp before recording (right after connecting to the TDA), but the same error occurs.
Any ideas?

Could you provide a log with debugging enabled?! It would allow us to spot where exactly the issue is coming from
# define DEBUG 1

The full debug output from the configuration:
conf_log.txt
and from the recording:
rec_log.txt
BR,
Christian

Hello, I am currently using both the MMWCAS-RF-EVM and MMWCAS-DSP-EVM boards as well. When trying to separate the configure and record functionalities using the fix-2-multi-record branch, I encountered the same issue.
Is there anything I can do to help move this issue forward?

Hello,
sorry, we didn't find a solution to this problem. Also a college of mine play with that radar and couldn't find a viable solution for large scale data collection. The support from TI is also not too helpful....

@chuber1986 Hello, I found that by simply duplicating the following section of code under mimo.c once more, two MMW recordings can be achieved.
    status = MMWL_ArmingTDA(tdaCfg);
    check(status,
      "[MMWCAS-DSP] Arming TDA",
      "[MMWCAS-DSP] TDA Arming failed!\n", 32, TRUE);

    msleep(2000);

    // Start framing
    for (int i = 3; i >=0; i--) {
      status += MMWL_StartFrame(1U << i);
    }
    check(status,
      "[MMWCAS-RF] Framing ...",
      "[MMWCAS-RF] Failed to initiate framing!\n", config.deviceMap, TRUE);

    msleep((unsigned long int)record_duration);

    // Stop framing
    for (int i = 3; i >= 0; i--) {
      status += MMWL_StopFrame(1U << i);
    }

    status += MMWL_DeArmingTDA();
    check(status,
      "[MMWCAS-RF] Stop recording",
      "[MMWCAS-RF] Failed to de-arm TDA board!\n", 32, TRUE);
    msleep(1000);

To work around this issue, I created a Python binding to this library. Using the code below, it’s possible to perform multiple recordings:
import time
import mmwcas

record_duration = 1*60
status = mmwcas.mmw_set_config(dict())

status = mmwcas.mmw_init()

# First recording
time.sleep(2)
status = mmwcas.mmw_arming_tda("outdoor0")
time.sleep(2)
status = mmwcas.mmw_start_frame()
time.sleep(record_duration)
status = mmwcas.mmw_stop_frame()
status = mmwcas.mmw_dearming_tda()
time.sleep(2)

# Second recording
status = mmwcas.mmw_arming_tda("outdoor1")
time.sleep(2)
status = mmwcas.mmw_start_frame()
time.sleep(record_duration)
status = mmwcas.mmw_stop_frame()
status = mmwcas.mmw_dearming_tda()

For more details, you can check https://github.com/touxiaoling/mmwave. To run the mimo.py file, you need to install the Python Cython library and then compile by running python setup.py build_ext --inplace. After that, simply run python mimo.py to view the results.
I hope this helps.

That great, thank you very much!


Map while using SLAM is getting Distorted traversing some distance

Robot Model
Turtlebot4 Standard
ROS distro
Humble
Networking Configuration
Discovery Server
OS
Ubuntu 22.04
Built from source or installed?
Installed
Package version
humble
Type of issue
Navigation (SLAM, Nav2 etc.)
Expected behaviour
Create a map in a proper way as shown in tutorials
Actual behaviour
While performing the SLAM and generating a map, the map is getting distorted as show below.

Error messages
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [rviz2-1]: process started with pid [32607]
[rviz2-1] Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
[rviz2-1] [INFO] [1718362029.594707625] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1718362029.594778064] [rviz2]: OpenGl version: 4.3 (GLSL 4.3)
[rviz2-1] [INFO] [1718362029.601496305] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1718362030.865796886] [rviz2]: Trying to create a map of size 155 x 164 using 1 swatches
[rviz2-1] [ERROR] [1718362030.897419377] [rviz2]: Vertex Program:rviz/glsl120/indexed_8bit_image.vert Fragment Program:rviz/glsl120/indexed_8bit_image.frag GLSL link result :
[rviz2-1] active samplers with a different type refer to the same texture image unit
[rviz2-1] [ERROR] [1718362031.091367957] [rviz2]: Lookup would require extrapolation into the future.  Requested time<PHONE_NUMBER>.330883 but the latest data is at time<PHONE_NUMBER>.286958, when looking up transform from frame [rplidar_link] to frame [map]
[rviz2-1] [INFO] [1718362031.635140799] [rviz2]: Trying to create a map of size 156 x 164 using 1 swatches

To Reproduce
ros2 launch turtlebot4_navigation slam.launch.py
ros2 launch turtlebot4_viz view_robot.launch.py
control the turtlebot4 with keyboard using the following command
ros2 run teleop_twist_keyboard teleop_twist_keyboard
Other notes
No response

Hi @Rahul007RR
Thanks for your patience we looked into this.
Can you please:

Set up time sync / NTP to ensure timesync is setup properly between RPi and C3. Detail here
Please repeat the above again and provide the full terminal output of both RVIZ and full terminal output from the SLAM Node terminal.

Thank you,
Rusty

Closing due to no activity


Error while uploading Excel without all_database_access

There is a bug in the code. Superset crashes when you are trying to open the Upload Excel form from a user account without all_database_access permissions.
Expected results
Upload Excel form opened and can be used for file upload.
Actual results
Superset crashes because of the absence of schemas_accessible_by_user() method in SupersetSecurityManager class. The error is in the superset/views/database/forms.py. Correct method name must be **get_**schemas_accessible_by_user()
Screenshots
Haven't made any, sorry:)
How to reproduce the bug

Create database connection with upload support
Create user with Gamma role + permission for Excel uploading + permission for database
Try to open the Upload Excel form from the menu
See the error

Environment

superset version: 1.1.0 (but I see the same problem in 1.2.0 and master too)
python version: 3.7.9
node.js version: not relevant

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[X] I have checked the superset logs for python stacktraces, but didn't include any, sorry again :)
[X] I have reproduced the issue with at least the latest released version of superset.
[X] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
I can solve the problem by myself and make a pull request later. I've created this issue as it's written in CONTRIBUTING.md:

Before creating a PR, create an issue.

Taking a look 🙏

This has been fixed by #16756


PanicException: Python API call failed

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
This was discovered here
https://github.com/vitalik/django-ninja/issues/1128
Example Code
How to reproduce
pip install django-ninja

^ this will install pydantic as well
Create two files

settings.py (keep empty)
main.py (paste the following code)

from typing import Annotated, Literal

from ninja import Schema
from pydantic import ConfigDict
from pydantic import Field

from pydantic import RootModel

class ExtraForbidSchema(Schema):
    model_config = ConfigDict(extra="forbid")

class AbstractProfileIn(ExtraForbidSchema):
    first_name: str | None = None
    last_name: str | None = None

class NewUserBOIn(ExtraForbidSchema):  # <==== If Schema it's good
    email: str
    role: str
    project_id: int | None = None

class AdminProfileIn(AbstractProfileIn):
    first_name: str
    last_name: str

class AdminBOIn(NewUserBOIn):
    site_ids: list[int] | None = Field(None, json_schema_extra={"deprecated": True})
    role: Literal["ADMIN"]
    profile: AdminProfileIn

class HCProfileIn(AbstractProfileIn):
    first_name: str
    last_name: str

class HCPBOIn(NewUserBOIn):
    role: Literal["HCP"]
    project_id: int
    profile: HCProfileIn

class NewUserBOUnionIn(Schema, RootModel):
    """
    Class to discriminate between different user roles with payload
    The solution : https://github.com/pydantic/pydantic/issues/3714#issuecomment-1051093100
    """

    root: Annotated[
        (HCPBOIn | AdminBOIn),
        Field(discriminator="role"),
    ]

    def __getattr__(self, attr):
        return getattr(self.root, attr)

    def __getitem__(self, item):
        return self.root[item]

user_data = {
    "email"<EMAIL_ADDRESS>    "project_id": "2",
    "role": "HCP",
    "profile": {"first_name": "FOO", "last_name": "BAR"},
}

user = NewUserBOUnionIn(**user_data)
print(user.email)

run with
RUST_BACKTRACE=1 DJANGO_SETTINGS_MODULE=settings python main.py

this gives exception
Traceback (most recent call last):
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/ninja/schema.py", line 66, in __getattr__
    value = getattr(self._obj, key)
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/ninja/schema.py", line 62, in __getattr__
    raise AttributeError(key)
AttributeError: __dict__. Did you mean: '__dir__'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/django/template/base.py", line 816, in __init__
    self.literal = int(var)
                   ^^^^^^^^
ValueError: invalid literal for int() with base 10: '__dict__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/django/template/base.py", line 827, in __init__
    self.literal = mark_safe(unescape_string_literal(var))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/django/utils/functional.py", line 231, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/django/utils/text.py", line 483, in unescape_string_literal
    raise ValueError("Not a string literal: %r" % s)
ValueError: Not a string literal: '__dict__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/ninja/schema.py", line 70, in __getattr__
    value = Variable(key).resolve(self._obj)
            ^^^^^^^^^^^^^
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/django/template/base.py", line 832, in __init__
    raise TemplateSyntaxError(
django.template.exceptions.TemplateSyntaxError: Variables and attributes may not begin with underscores: '__dict__'
thread '<unnamed>' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.21.1/src/err/mod.rs:1098:5:
Python API call failed
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
--- PyO3 is resuming a panic after fetching a PanicException from Python. ---
Python stack trace below:
Traceback (most recent call last):
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/ninja/schema.py", line 214, in _run_root_validator
    return handler(values)
           ^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: Python API call failed
--- PyO3 is resuming a panic after fetching a PanicException from Python. ---
Python stack trace below:
Traceback (most recent call last):
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/ninja/schema.py", line 214, in _run_root_validator
    return handler(values)
           ^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: Python API call failed
Traceback (most recent call last):
  File "/private/tmp/pyda27test/main.py", line 73, in <module>
    user = NewUserBOUnionIn(**user_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyda27test/.venv/lib/python3.12/site-packages/pydantic/root_model.py", line 71, in __init__
    self.__pydantic_validator__.validate_python(root, self_instance=self)
pyo3_runtime.PanicException: Python API call failed

### Python, Pydantic & OS Version

```Text
pydantic version: 2.7.0
        pydantic-core version: 2.18.1
          pydantic-core build: profile=release pgo=true
                 install path: /private/tmp/pyda27test/.venv/lib/python3.12/site-packages/pydantic
               python version: 3.12.1 (main, Dec 13 2023, 15:47:03) [Clang 15.0.0 (clang-15<IP_ADDRESS>.5)]
                     platform: macOS-13.5.2-arm64-arm-64bit
             related packages: typing_extensions-4.11.0
                       commit: unknown

@vitalik,
Thanks for reporting this. We'll look into a fix for our patch release!

@davidhewitt,
Hmm, looks like a pyo3 thing, maybe related to the recent upgrades?

I think there's two overlapping bugs here. The PyO3 error is https://github.com/PyO3/pyo3/pull/4100 but the root cause is that __dir__ is failing. However, I can't work out why we're calling __dir__ here.

It seems to be an interaction between django-ninja's Schema and RootModel. The following minimal repro also triggers the crash, but only when validating the full RootModel and not the inner type.
from typing import Literal

from ninja import Schema
from pydantic import ConfigDict

from pydantic import RootModel

class HCPBOIn(Schema):
    model_config = ConfigDict(extra="forbid")
    role: Literal["HCP"]

class NewUserBOUnionIn(RootModel, Schema):  # removing `Schema` from here also fixes the crash
    """
    Class to discriminate between different user roles with payload
    The solution : https://github.com/pydantic/pydantic/issues/3714#issuecomment-1051093100
    """

    root: HCPBOIn

user_data = {
    "role": "HCP",
}

user = HCPBOIn(**user_data)  # ok
print(user.role)

user = NewUserBOUnionIn(**user_data)  # crash
print(user.role)

When looking at the input getting validated, I see the input dict gets wrapped twice: <DjangoGetter: <DjangoGetter: {'role': 'HCP'}>>
@vitalik as you're familiar with the Schema class, does this give you any ideas what might be going wrong?

@davidhewitt hard to tell - the only hint I see is that DjangoGetter has '__dict__' in __slots__
maybe that somehow influences


Do not update vendor on every make

Not sure if this solves #24

@nhlfr should we add something like this: https://github.com/Mirantis/virtlet/blob/master/Makefile.am#L29-L32

@alban @schu @nhlfr
Can this be merged?


Performance regression in __spreadArray

Bug Report
🔎 Search Terms
__spreadArray performance benchmarking
🕗 Version & Regression Information

This changed between versions 4.1 and 4.2

⏯ Playground Link
CodeSandbox with benchmarks
Playground link with configuration example
💻 Code
__spreadArray(__spreadArray([], arr1), arr2)
// vs.
__spreadArrays(arr1, arr2)

🙁 Actual behavior
Starting from 4.2, occurences of __spreadArrays are replaced with multiple calls to __spreadArray. The change fixes side effects in the polyfill (#32959).

Function
Browser
Arrays
Elements per Array
Runs
Mean time in ms
Slowdown

__spreadArrays
Chromium 90
1
(numbers) 1000000
100
2.32

__spreadArray
Chromium 90
1
(numbers) 1000000
100
10.29
4.43

__spreadArrays
Chromium 90
2
(numbers) 1000000
100
4.35

__spreadArray
Chromium 90
2
(numbers) 1000000
100
25.4
5.83

__spreadArrays
Chromium 90
2
(objects) 1000000
100
4.82

__spreadArray
Chromium 90
2
(objects) 1000000
100
68.1
14.10

🙂 Expected behavior
No or moderate performance impact.

We changed from __spreadArrays to multiple calls to __spreadArray to match the ES2015 runtime semantics because the runtime behavior of __spreadArrays incorrectly captured side effects too early, as per the linked PR.
I compared what we do to what BabelJS does. Babel has the same error we used to have for this case:
const k = [1, 2];
const o = [3, ...k, k[0]++];
console.log(o.join(", "));
// out
var k = [1, 2];
var o = [3].concat(k, [k[0]++]);
console.log(o.join(", ")); // prints: 3, 2, 2, 1; should print: 3, 1, 2, 1

This is because Babel statically looks up k and sees it's an array, emitting the "faster" code that doesn't correctly preserve side-effects. If you're able to convince Babel that k isn't statically an array, then it emits a _toConsumableArray helper:
const k = (() => [1, 2])();
const o = [3, ...k, k[0]++];
console.log(o.join(", "));
// out
...
var k = function () {
  return [1, 2];
}();
var o = [3].concat(_toConsumableArray(k), [k[0]++]);
console.log(o.join(", "));

I ran this through your benchmark, and while its still slower than the old __spreadArrays helper, it is faster than __spreadArray.
What's expensive in __spreadArray is that we're rebuilding the array each time from the beginning. We can improve this by using concat and ensuring that to is always an array during emit, and that any from that is a spread is always a packed array:
var __spreadArray2 = function(to, from, pack /*optional*/) {
  if (pack) for (var i = 0, l = from.length, ar = Array(l); i < l; i++) ar[i] = from[i];
  return to.concat(pack ? ar : from);
};
var k = [1, 2];
var o = __spreadArray2(__spreadArray2([3], k, 1), [k[0]++]);
console.log(o.join(", "); // prints: 3, 1, 2, 1

It's important that we coerce from into a packed array, since that does matter to spread:

Unfortunately, this is something that we unintentionally regressed with __spreadArray, as the following results in incorrect output:
const k = [1, , 2];
const o = [3, , 4, ...k, , k[0]++];
console.log(1 in o); // expected: false; actual: false (ok)
console.log(4 in o); // expected: true; actual: true (ok)
console.log(6 in o); // expected: false, actual: true (wrong)
console.log(o.join(", ")); // [3, , 4, 1, , 2, , 1]

We always guarantee that the first argument to the innermost __spreadArray is always an array literal containing the leading non-spread arguments. This means it should generally be safe to call .concat on the first argument since it's always an array.
The new version of __spreadArray2 above is backwards compatible with __spreadArray (though with the same incorrect packing behavior for non-spread holes), has the correct packing behavior if we update our emit to pass a truthy value for the 3rd argument, and outperforms the current __spreadArray (with or without the 3rd argument) as well as Babel's toConsumableArray:

While still slower than the older __spreadArrays helper, I'm not sure we can fully recover that performance without sacrificing correctness with runtime semantics.
As a result, I propose the following changes:

Replace __spreadArray with the arguments and body of the __spreadArray2 variant.
Update EmitHelpers.createSpreadArrayHelper to take an extra pack argument to emit a truthy value (i.e., 1 for brevity or true for clarity).

Alternatively, we can deprecate __spreadArray and introduce a new __pack helper:
var __pack = function (from) {
  for (var i = 0, l = from.length, ar = Array(l); i < l; i++) ar[i] = from[i];
  return ar;
};

We could then use __pack along with `concat directly:
const k = [1, 2];
const o = [3, ...k, k[0]++];
console.log(o.join(", "));
// out
var k = [1, 2];
var o = [3].concat(__pack(k), [k[0]++]);
console.log(o.join(", ")); // 3, 1, 2, 1

const k = [1, , 2];
const o = [3, , ...k, , k[0]++];
console.log(o.join(", "));
// out
var k = [1, , 2];
var o = [3, ,].concat(__pack(k), [, k[0]++]);
console.log(1 in o); // expected: false; actual: false (ok)
console.log(4 in o); // expected: true; actual: true (ok)
console.log(6 in o); // expected: false, actual: false (ok)
console.log(o.join(", ")); // 3, , 1, , 2, , 1

This approach has slightly better performance than __spreadArray2, but has the downside of shipping yet another helper that replaces __spreadArrays, meaning we would need to ship all 3 variants to maintain backwards compatibility in perpetuity (or at least until we no longer support TS versions that emit those helpers).

Thank you so much @rbuckton and team for working on this issue. Looking forward to bumping the version :)


big_core - print 'end_of_test' at the end of sim test

Print 'end_of_test' at the end of the simulation run, as shown in the rvc_asap reference

Welcome to use the sc as a reference.
Use a function which can have multiple reasons to "eot" - timeout, assertion, ebreake

I think the current design has something simular.
But we should have a function/task that handles the EOD sequence.


Increase max depth of graphql queries

Coverage: 94.015%. Remained the same when pulling a36f03a3f9b847a9b829d15bf388cab0bcb46030 on cody-increase-graphql-max-depth into 8c6d4edb625bc39232253f4e7af653a566f85ca2 on main.


Fix the no convenient translation of return

The return statement in the code inside the description should not be translated.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Thanks!


OMBI V4 dark theme not working at all

Describe the bug
It appears the OMBI dark theme was meant for the old version of OMBI.  The theme colors look exactly the same after adding the subfilter.  Instead of a dark color theme similar to Petio, it's still the default ugly dark blue theme.   I tried using Petio's CSS; but, that didn't help much.
This should be very easy to reproduce.
Theme Install Method (Don't skip)
nginx subfilter:
https://theme-park.dev/CSS/themes/ombi/organizr-dark.css

The Ombi theme is for V3 afaik, and has not yet been updated for V4.
Ombi supports custom css through Settings -> Configuration -> Customization, and you can paste the CSS directly there.
Just made some quick and dirty style, until someone more skilled comes around and does it properly.
Plex-theme background, with white text.
/* Removing Application Name at top left */
#nav-applicationName{display:none;}

/* Visibility */
.top-search-block, .mat-flat-button, .mat-mini-fab, .mat-raised-button {color:white !important;}
.top-search-bar .mat-form-field-flex, .discover-filter-buttons-group{border: 1px solid white !important;}

/* Plex Background */
body, .discover-filter-buttons-group, .card-top-info, .mat-table, .mat-paginator, .sidenav-container, .mat-menu-panel, .mat-fab.mat-accent, .mat-flat-button, .mat-mini-fab, .mat-raised-button, .sidenav, .content-container, .top-bar-container {background: url("https://raw.githubusercontent.com/gilbN/theme.park/master/Resources/preset-dark2.png") center center/cover no-repeat fixed !important; }

Thank you so much for the quick and very help response.  It is significantly better.  There's only a weird issue... the status for each TV/movie is not displaying correctly.  Any idea how I might be able to resolve that?  Otherwise, it's very nice.
Original:  (correct)

CSS-Mod:  (status not displayed correctly)

The black overlay was not positioned perfectly after changing the background of the whole box, so I changed the overlay as well. Seems it broke it though.
Unable to find a quick way to fix the position, but I guess a few pixels off is better then broken.
Updated my previous post, and removed .card-top-info from background.

@Lanjelin thank you so much for taking the time to do this.  I was able to find the specific css element and properties to adjust .card-top-info using Chrome Developer tools.  However, just pasting the below code in OMBI's CSS customization gets ignored.
Any ideas why the CSS code below works in Developer Mode; but, not when adding it to OMBI's CSS customization?
.card-top-info[_ngcontent-nhx-c214] {
    position: absolute;
    top: 5px;
    width: 100%;
    background-color: rgba(15,23,31,.7);
    display: flex;
    justify-content: space-between;
    padding-top: 4.6px;
    padding-bottom: 6.3px;
    z-index: 2;
    letter-spacing: .2px;
}

Use !important to override already defined properties. https://www.w3schools.com/css/css_important.asp
.card-top-info {
    top: 5px !important;
    padding-top: 4.6px !important;
    padding-bottom: 6.3px !important;
}

Thank you!  It looks great now.  I moved this CSS mod to Nginx subfilter since the CSS customization feature displays the original CSS style/colors briefly before displaying the CSS mod.
Thanks for telling me about !important
Hopefully, someone much more experienced than me will update theme.park to officially support Ombi V4.

Thanks guys, you inspired me to update the css for v4. :) Check it out https://github.com/GilbN/theme.park/commit/ee227ff8c42ed810a550724325f3120c7ff5a67c

Thank you!  Ombi also just updated their Interface today too!  200% improvement from before!

Thank you! Ombi also just updated their Interface today too! 200% improvement from before!

Oh, What's new? Did something in the theme break?

It looks better than ever.  They fixed the sidebar size being too big.  Before it was too big.  Also, they fixed poster aspect ratio issue.
The only thing I noticed is that the app name for Ombi is showing on the top of the sidebar.  It looks exactly the same as without the theme.  I'm not sure if it's supposed to show or not with the theme.

It looks better than ever. They fixed the sidebar size being too big. Before it was too big. Also, they fixed poster aspect ratio issue.
The only thing I noticed is that the application name for Ombi looks exactly the same as without the theme. I really wish they would just replace that ugly light-blue plain-text application name "Ombi" with a nice custom logo instead.

/* Nav Bar */
.sidenav-container .sidenav .mat-toolbar.application-name {
content: url("linktoimage/logo.png");
}

/* Nav Bar */
.sidenav-container .sidenav .mat-toolbar.application-name {
content: url("linktoimage/logo.png");
}

Thank you so much!!  Awesome.

No worries mate.


Design Footer Component

Description: Develop a static footer with links to various app sections.
Components Involved: Footer.jsx

changed to a side menu bar


Add templates to be used with kube-compare

This is a temporary directory for a copy of the reference-crs to be used with kube-compare
/cc @MarSik @imiller0

/approve

There are a couple of improvements that have been suggested that we will take up as separate PRs which can be reviewed individually.


Performance issue in StringUtil

Hi, we're having performance issues with the jsoup lib. We are working on servers with very many cores and want to parse html highly parallel. Unfortunatelly there is a border where we can not increase throughput anymore by increasing threads.
I created a thread dump and nearly all threads are waiting here:
INFO   | jvm 1    | 2020/04/17 12:38:23 | "thread-55" ...  waiting for monitor entry INFO   | jvm 1    | 2020/04/17 12:38:23 |    java.lang.Thread.State: BLOCKED (on object monitor) INFO   | jvm 1    | 2020/04/17 12:38:23 |       at org.jsoup.internal.StringUtil.borrowBuilder(StringUtil.java:243) INFO   | jvm 1    | 2020/04/17 12:38:23 |       - waiting to lock <0x00007ef4a2361248> (a java.util.Stack) INFO   | jvm 1    | 2020/04/17 12:38:23 |       at org.jsoup.internal.StringUtil.normaliseWhitespace(StringUtil.java:146) INFO   | jvm 1    | 2020/04/17 12:38:23 |       at org.jsoup.nodes.TextNode.text(TextNode.java:46)

How did you check the thread usage? I mean what tools you used in your project?

After we dig into the StringUtil class, we found that after increasing the size of memory, the performance will increase while the number of threads increases.

Yes, that's true until a certain point it get's better. But as said, we have very big machines, in detail they have 80 cpu cores, 196GB Memory and a threadpool with the size of up to 200 was used. In the thread dump we could see that nearly all of them were waiting for the monitor as shown above.
As a workaround we now re-implemented the TextNode.text() funktion like this:
String wholeText = node.getWholeText();
StringBuilder sb = new StringBuilder();
StringUtil.appendNormalisedWhitespace(sb, wholeText, false);
return sb.toString();

To not use a StringBuilder from the pool and it clearly solved the performance problem and we don't see problems related to the many StringBuilders created.

Thanks for the report. This is fixed by #1402.


Run e2e tests on host

What type of PR is this?
/kind documentation
/kind feature

Any specific area of the project related to this PR?
/area tests

Does this PR require a change in the driver versions?

What this PR does / why we need it:
For some time now the e2e-tests have only been able to be run in a very specific system (coincidentally that system being the ubuntu-20.04 runner on GHA). With these changes, the tests should now be available to run on a system that can build sinsp-example and the drivers.
In order to achieve this, sinsp-example is now run directly as a subprocess of the tests and its logs are streamed in for assertion. The containerized way of running the tests is still available through the e2e-tests-container target and is the one that will be used by CI for the time being (until I or someone else get the time to start running the none containerized way I guess).
Which issue(s) this PR fixes:
Special notes for your reviewer:
The test_process.py test has been removed because ultimately it tested the same thing test_container.py tested, but in a much more difficult way to make compatible with these changes.
Does this PR introduce a user-facing change?:
NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Molter73
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Molter73]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Tagging some people that might be interested on this.
@incertum and @gnosek had some problems when trying to change the output of sinsp-example in #839, this probably could've helped.
@deepskyblue86 has also run into problems when running the tests before.
Additionally, anyone else that could run the tests on their system is more than welcomed, I've run them on a Fedora 37 and an Ubuntu 20.04 VMs and both pass consistently.

I will give it a go on my arch system and report back! Thank you very much for this big effort Mauro!

thank you very much @Molter73 I will give it a try!

Thanks a lot for this @Molter73 ❤
If you have time now or in a future PR a "How To Contribute" to the e2e test section would be appreciated, thank you!

Hi! I was able to build the e2e tests and run them; but there are lots of failures:

23 failed, 9 passed, 16 errors in 224.03s (0:03:44)

I am not sure whether i am missing anything (perhaps we need some binary in the host system?)

Hey @FedeDP! Have you checked the HTML report for any clues? You should be able to find it under build/report/.
If you can zip that directory and share it with me, we might be able to narrow down what's going on.

Yep, a bunch of:

AssertionError: Did not receive expected event: {'container.id': '2d7e468f4a73', 'evt.args': r'^res=\d+ exe=/tmp/falco-event-generator\d+/mysqld args=--loglevel.info.run.^helper.ExecLs$. tid=\d+(mysqld) pid=\d+(mysqld) ptid=\d+(event-generator) .* flags=\d+([|A-Z_]+) uid=0 gid=0 vtid=\d+ vpid=\d+', 'evt.category': 'process', 'evt.num': r'^\d+$', 'evt.time': r'^\d+$', 'evt.type': 'clone', 'proc.cmdline': 'mysqld --loglevel info run ^helper.ExecLs$', 'proc.exe': r'/tmp/falco-event-generator\d+/mysqld', 'proc.pid': r'^\d+$', 'proc.ppid': r'^\d+$'}

But i also see:

AssertionError: sinsp-example terminated with code -6
E           assert -6 == 0
E            +  where -6 = <Popen: returncode: -6 args: ['/home/federico/Work/libs/test/e2e/scripts/../...>.returncode

Perhaps sinsp-example is crashing badly :laughing:
Will investigate further!

Well, after a clean, non minimal, build, everything is green! :green_circle:
32 passed in 197.21s (0:03:17)

If you have time now or in a future PR a "How To Contribute" to the e2e test section would be appreciated, thank you!

Fully agree with Melissa, this would be useful for everyone!

Thanks @Molter73 for the adjustments! Just a minor clarification:
First of all entire test suite is passing :tada: nice work!!!, but perhaps we could think one more time about the sudo setup for localhost runs?
Definitely would not want to run make sinsp-example  etc as root, but now I have to run the new make commands as root, which also goes a bit against the style, other tests have make command run as non-root and then you have to specifically run some binary as root.
sudo make e2e-install-deps # this also gives pip warning as you shouldn't do it as root, but otherwise package is not available
sudo make e2e-tests

Would you have more ideas @Molter73?

Good to hear people are succeeding in running the tests!
As for @incertum's ask for the make targets to not be run as root, I fully agree that shouldn't be a requirement, but it could also be a tricky thing. The sinsp-example process runs as a subprocess of the tests and this is needed for it to insert the drivers in the system, we could run the subprocess as root, but that would require:

The test checks if it has root access.
If it doesn't have root access, needs to be aware on HOW to get that access (sudo, doas, su...)
Let's say the system has sudo, we would need to wire up the subprocess to prompt for the password, pass it in, etc...

Honestly, that sounds like a bad idea. Instead, you could run the script that runs the tests locally as root (there's instructions for it here).
As for the warnings on the sudo make e2e-install-deps, that's another thing virtual environments are for. Even if you install those as root, as long as you are in a venv, your system will not break after you deactivate it. I could move the install commands to a separate script if running the make target as root is the problem, that particular target does nothing more than installing dependencies.

As for @incertum's ask for the make targets to not be run as root, I fully agree that shouldn't be a requirement, but it could also be a tricky thing. The sinsp-example process runs as a subprocess of the tests and this is needed for it to insert the drivers in the system, we could run the subprocess as root, but that would require:

The test checks if it has root access.
If it doesn't have root access, needs to be aware on HOW to get that access (sudo, doas, su...)
Let's say the system has sudo, we would need to wire up the subprocess to prompt for the password, pass it in, etc...

Honestly, that sounds like a bad idea. Instead, you could run the script that runs the tests locally as root (there's instructions for it here).

I could move the install commands to a separate script if running the make target as root is the problem, that particular target does nothing more than installing dependencies.

Thanks Mauro and yes aware ... thanks for updating the readme, it's awesome!
Yes we could have a target just to install dependencies as non root. Another thought maybe sinsp-example, bpf and driver shouldn't be a target dependency and could just say run those before (normally as non root) in the instructions. Totally up to you @Molter73, I can work with the current version. Sometimes maybe just another explaining sentence or comment does all the magic as well.
After you decide if something still gets changed or not, we should go ahead and merge it 🎉 !

And @Molter73 actually wait you already updated the instructions, let me pull once more and do a fresh build, then will approve, thanks so much!!!

LGTM label has been added.  Git tree hash: 0b0cfd220d20cbd3674883cbb1515ee03258345b

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: incertum, Molter73
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Molter73,incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, incertum, Molter73
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,Molter73,incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Reverse proxy auth support

Changes

Add rudimentary support for a new header x-remote-user, to allow webserver-based authentication
Use /usr/bin/env bash instead of /bin/bash to support bash in non-standard locations

Tasks

[ ] Passed header should be configurable via an environment variable
[ ] Disable feature altogether if not configured
[ ] Fix: warn: Client Error ErrorWithCode: No view access

(relates to https://github.com/gristlabs/grist-core/issues/160)

Thanks for polishing this up and submitting it @MHOOO! It looks good overall.
I wonder if we should anticipate other information that might be available. For example, you derive a name for the user from their email; in other cases that might be available externally. So perhaps the header could be GRIST_AUTHENTICATED_USER_EMAIL, and later other headers could be added as GRIST_AUTHENTICATED_USER_*.

Hmm, I guess that's possible. Though as I tested it, the name for example can be changed in grist-core, at which point the name of the header-based profile no longer matches what grist-core uses. I fear this might be confusing, since it is not clear which values are always coming via the header and which are only used once to create the profile in grist-core but are then ignored.
On another note, maybe a JSON dictionary could be passed. This way, one header would be enough. Though then again, I'm really not sure if headers are the right place for additional data. From what I remember there are restrictions on length & allowed characters. What if the name contains, e.g., non ASCII characters (I'm thinking of the german "ö" for example).

Is it a problem for your use-case that Grist makes a session for the user, and will allow access via presentation of a cookie with the right secret in it even if the headers you are adding are not present? It might make logging out a little awkward.

Yes, signing out doesn't actually work. As it stands now, the user has to delete the cookies manually. This is an inconvenience, but it is not a deal breaker for me at the moment, since our users each have only one account anyway.

One thing missing is writing tests, but we haven't made that easy for you. I'll look at getting a batch of server tests released to use as a model (we have a big messy soup of tests internally that we've been cleaning up and releasing as needed). Are you blocked on getting this functionality merged or do you have time?

No, I'll have to roll my own docker images anyway, since yours are based on an older debian release and our IT department requires up to date debian images.


[BUG]

Not able to upload Divoc Realm
To Reproduce

Create Keycloak DB and execute following commands

Create user keycloak;
Create user keycloak with encrypted password ‘keycloak’;
grant all privileges on database keycloak to keycloak;

Create Registry DB and execute the following commands

create database registry
grant all privileges on database registry to postgres;
Setting Keycloak HTTPS value

\c registry
select  * from realm
update realm set ssl_required='NONE' where id='master'

Deploy Keycloak and registry with appropriate env values in kubernetes.

4 . Use REST API
Request 1
RESTful command: POST
URL: https://KEYCLOAKSERVER/auth/realms/master/protocol/openid-connect/token
Header:

Key
Value

Content-Type
application/x-www-form-urlencoded

Body:

Key
Value

grant_type
password

client_id
admin-cli

username
admin

password
admin

Request 2
RESTful command: POST
URL: https://KEYCLOAKSERVER/auth/admin/realms
Header:
Copy the access_token value and past the token value into the authorization.

Key
Value

Authorization
bearer access_token value

Content-Type
application/json

Body:
<@Divoc Realm as json>
raw 	JSON(application/json)

Direct from Keycloak Console.
Directly going into keycloak Pod in kubernetes.

this gives issue that  curl: (7) Failed to connect to localhost port 8596: Connection refused

curl --location --request POST 'http://localhost:8080/auth/realms/master/protocol/openid-connect/token' \
         --header 'Content-Type: application/x-www-form-urlencoded' \
         --data-urlencode 'grant_type=password' \
         --data-urlencode 'client_id=admin-cli' \
         --data-urlencode 'username=admin' \
        --data-urlencode 'password=admin'

curl --location --request POST 'http://localhost:8080/auth/admin/realms' \
--header 'Authorization: bearer  <Auth token>\
--header 'Content-Type: application/json' \
--data-raw '<realm-json>'

Expected behavior
Divoc realm should be update properly in keycloak.
Additional context
Keycloak YAML is taken from Divoc github

what is the port 8596? are you using default configuration from the repo?

Aahh its 8080, got it typo.

Have resolved this Issue by

Creating image from keycloak
Adding realm to the image
Executing Realm script with realm.
IMP: adding config.batch is optional

Though the issue is not resolved, this the way around i did.
Dockerfile
FROM dockerhub/keycloak:1.23.3-generic

COPY --chown=jboss ./config.batch /tmp/config.batch
COPY ./divoc.json /opt/jboss/keycloak/divoc.json

RUN cd $JBOSS_HOME \
  && ./bin/jboss-cli.sh --file=/tmp/config.batch \
  && rm -rf $JBOSS_HOME/standalone/configuration/standalone_xml_history \
  && rm -rf $JBOSS_HOME/standalone/data \
  && rm -rf $JBOSS_HOME/standalone/tmp \
  && rm -rf /tmp/config.batch

config.batch
embed-server --std-out=echo --server-config=standalone-ha.xml
batch
/system-property=jboss.as.management.blocking.timeout:add(value=900)
/subsystem=deployment-scanner/scanner=default:write-attribute(name=deployment-timeout,value=900)
/subsystem=transactions:write-attribute(name=default-timeout,value=900)
run-batch
stop-embedded-server

Then added env below to the keycloak deployment yaml
- name: KEYCLOAK_IMPORT
   value: "/opt/jboss/keycloak/divoc.json -Dkeycloak.migration.realmName='divoc' -Dkeycloak.profile.feature.upload_scripts=enabled"


Put JC materials under MIT license

I removed LICENSE.md from the main directory and added versions of this file that include MIT Licensing to the directories with content created by JC employees.
I'll have creators of other directories update with their licensing info.
@aviks @ViralBShah

The copyrights need to be Julia Computing only.

Yes, for the JC created sections. copyright should be JC only. We can credit the authors in a README.md in the directory.

I updated the copyrights as requested and put contributors I know of in README files. I'm not sure who put together the parallelism demos and advanced-julia, so someone else will need to update.


Constants

Codecov Report

Merging #24 (0677017) into main (15b9aee) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #24   +/-   ##
=======================================
  Coverage   99.65%   99.65%
=======================================
  Files          13       12    -1
  Lines         572      579    +7
=======================================
+ Hits          570      577    +7
  Misses          2        2

Impacted Files
Coverage Δ

src/aerodynamic_conductance.jl
100.00% <100.00%> (ø)

src/boundary_layer_conductance.jl
100.00% <100.00%> (ø)

src/evapotranspiration.jl
95.00% <100.00%> (ø)

src/meteorological_variables.jl
100.00% <100.00%> (ø)

src/potential_radiation.jl
100.00% <100.00%> (ø)

src/stability_correction.jl
100.00% <100.00%> (ø)

src/sun_position.jl
100.00% <100.00%> (ø)

src/surface_conductance.jl
100.00% <100.00%> (ø)

src/surface_roughness.jl
100.00% <100.00%> (ø)

src/unit_conversions.jl
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15b9aee...0677017. Read the comment docs.


Failed to install Operator strimzi-cluster-operator.v0.26.0 on OCP 4.9

The following warning is often seen on OCP 4.9:
Failed to install Operator strimzi-cluster-operator.v0.26.0 version 0.26.0. Reason-NoOperatorGroup

We need to investigate if the creation of operator is valid

I'm going to close it as this doesn't seem relevant anymore


Generates nothing

No output CSV created
the program instantly exits
➜  hdfc-cc-parser-rs git:(master) ✗ RUST_LOG=info ./target/release/hdfc-cc-parser-rs --name=",,,," --dir ./statements/ [2024-03-18T16:50:29Z INFO  hdfc_cc_parser_rs] starting main ["./statements/0001015720000521082_01222024_02112024.PDF", "./statements/0001015720000521082_02222024_03132024.PDF", "./statements/0001015720000140487_09222023_10122023.PDF", "./statements/0001015720000521082_11222023_12122023.PDF", "./statements/0001015720000521082_12222023_01112024.PDF", "./statements/0001015720000521082_10222023_11112023.PDF"] ➜  hdfc-cc-parser-rs git:(master) ✗ ls -latr
I am not sure hwo this file is generating any file -
let mut wtr = csv::Writer::from_writer(io::stdout());

csv::Writer::from_writer(io::stdout())
It writes the parsed records to stdout like you can see. If you aren't getting any parsed records, it is likely that pdf format is slightly different that what is expected.
Sample result

I am downloading infinia statements and using them without password (they
are unlocked)
Let me give it one more try
Thanks,
Ved
On Tue, 19 Mar 2024 at 9:27 AM, Joe Paul @.***> wrote:

csv::Writer::from_writer(io::stdout())
It writes the parsed records to stdout like you can see. If you aren't
getting any parsed records, it is likely that pdf format is slightly
different than what is expected.
Sample result
image.png (view on web)
https://github.com/joeirimpan/hdfc-cc-parser-rs/assets/11286580/512e7f6e-364c-48f6-b967-9d631c1d4802
—
Reply to this email directly, view it on GitHub
https://github.com/joeirimpan/hdfc-cc-parser-rs/issues/14#issuecomment-2005708719,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACY6XURVURFTAT2MCBHSOTYY6ZSHAVCNFSM6AAAAABE37MP32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBVG4YDQNZRHE
.
You are receiving this because you authored the thread.Message ID:
@.***>


Display human readable peer name for smartphone connections.

The current BLE library does not provide an easy way to get the smartphone name during connection, we use the bluetooth MAC address at the moment.
However, there are changes coming that will make this possible.
Thus, rather than being asked to connect to unreadable "AA:BB:CC:....", we should be able to ask to connect to "My smartphone".

This requires changes that are only available in an unstable release of NimBLE-Arduino. As of writing, the latest release is 1.4.2.

This is the commit that serves our needs:
https://github.com/h2zero/NimBLE-Arduino/commit/44a69ed80518127e86302115bc6f985c285d5d2f


Avoid problems in str.format() with char accents

If you input a char like "é" or "ã" with terminalAdapter, in log method str.format(), if no unicode .format(() raise a error.
To reproduce, just clone master, and run with IN/OUT using TerminalAdapter.
Thanks

Hi @davizucon, sorry about the delay in reviewing this. I was able to confirm the issue, but I'm going to have to make a small change to the pull request so that it doesn't break Python 3 compatibility.
Python 3 has an improved way of dealing with strings which eliminates issues with unicode characters, however it breaks backwards compatibility with the string decode method.
Python 3:
'é'.decode('utf-8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute 'decode'

Python 2.7:
'é'.decode('utf-8')

>>> u'\xe9'

This has been closed by https://github.com/gunthercox/ChatterBot/commit/35760762280d082f777545dc11ca2be0355ecfaf


First-run wizard for setting the destination

This seems pretty important as part of a first release.


Incorrect Example for Get-SPOContentType

This following explanation is not correct, it looks like it may have been copied from Set-SPOContentType.
PS:> Get-SPOContentType -Identity "Project Document"
This will add an existing content type to a list and sets it as the default content type

Thanks for reporting this one!
Is was already fixed in the dev branch: https://github.com/OfficeDev/PnP/blob/dev/Solutions/PowerShell.Commands/Documentation/GetSPOContentType.md
Closing the issue


Fixes issue with img dragging on FireFox.

Fixes issue with <img/> dragging on FireFox.
Tested with version: 98.0.2 (64-bit)
Issue: https://github.com/Blazor-Diagrams/Blazor.Diagrams/issues/165

Hi, thanks for the PR!


Intra-document validation for OSCAL artifacts.

Issue description / feature objectives
The current pydantic models do not describe sufficiently the constrains required for a OSCAL schema.
NIST defines two sets of IDs

UUIDs: UUIDs must be globally unique within their object type. An acceptable solution is to test whether
a) UUIDs conform to required schema
b) UUIDs are unique (and error identifying the conflict if it exists)

Note: Many UUIDs are optional - I think we should have an option to populate more UUIDs as required.

NIST also defines ID fields. ID fields are scoped within the referenced document e.g. two catalogs can have colliding ID fields
Suggest two tiers of validation:
Compulsory: For an given object type (e.g. control) no id fields collide
Best-practice: For a given schema document where ID's are defined no ID's collide

Note: (1) and (2) explicitly point to defining UUIDs / ID's and not references to UUIDs.
The above functionality should be generic such that it can operate on any OSCAL object, however, presuming uuid and id are special fields that exist.
Completion Criteria

Completed functionality
Unit test coverage across all new code
tested for catalog, profile, target, and SSP at a minimum.
This functionality should work for both assembled and 'distributed' artifacts.
validation should be able to operate over all files in a repository.

trestle validate -f catalog.json --type uuid
- All UUID are in correct format
- All UUID is unique in the file


Libsql query multi-statements string

Build on top of #847 to fix #807

Lets get #847 merged then I will review this one.


[AUTOPATCHER-CORE] Upgrade python-sphinx to 7.2.6 3.0 upgrade

[AUTOPATCHER-CORE] Upgrade python-sphinx to 7.2.6 3.0 upgrade
Upgrade pipeline run -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=509733&view=results
buddy build -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=509735&view=results

Closing to re-run the auto-upgrade pipeline with new source url


 Sometimes I am unable to use the command myth -h because it just hangs for some reason.
How to Reproduce
myth -h
myth analyze [contract].sol
Both of these give no output for a very long period of time.
-->
Expected behavior
Give results stating whether a contract is vulnerable or not
Screenshots
<!-- This section is optional.
![image](https://github.com/Consensys/mythril/assets/52522679/9d9f63c2-cddc-4530-a882-dc95faccea79)

## Environment

- Mythril version: output from  `0.24.7`
- Solidity compiler and version: `0.8.23`
- Python version: `3.10.12`
- OS and Version: [Ubuntu WSL 22.04.3]

##Additional Information
Also, I installed mythril using pip3 install mythril.

Hi @JacobYiu, the default timeout for symbolic execution is usually set high.
You can adjust execution timeout using --execution-timeout <timeout> which is in seconds.
Usually techniques such as symbolic execution and Fuzzing are meant to be run for long.


Allow passing of class names

Right now, if you add a className to ReactCardFlipper it does not accept it. Should be adjusted to allow passing of class names.
<ReactCardFlipper width="100%" height="300px" levitate={true} className="your-class-name">

PR #7 should do the trick, will test and merge in soon.

Merged and released on 0.3.0


Unable to start program with LiveShare session started

Describe what happened:
I get error 'Unable to start program...' when trying to start Debug (F5) while started LiveShare session.
The same behaviour is noticed for my collegue.
Recommendation from the error window does not help.
Product and Version [VS/VSCode]: VS 2019 16.10.3
OS Version[macOS/Windows]: Windows 7
Live Share Extension Version: 1.0.4535.0
Target Platform or Language [e.g. Node.js]: C++
Steps to Reproduce / Scenario:

Open project
Start LiveShare session
Try to run project with any configuration
Get the above error

Please attach logs to this issue:
ActivityLog.zip
Screenshots

Running into the exact same issue when trying to debug a C# console application with a Live Share session active.
Visual Studio 2019 version 16.11.9
Live Share version 1.0.5185.0
Windows 8.1

Apologies for the delay here. As is, this issue doesn’t have enough information to be reproduced. Please reopen if the issue persists with additional details. Thank you.


Add Recipe Guide for Migrating to Java 17

We need a recipe guide for migrating to Java 17

The docs have been updated with a guide for migrating to Java 17.
https://docs.openrewrite.org/running-recipes/popular-recipe-guides/migrate-to-java-17


Adding color to Icon

Is your feature request related to a problem? Please describe.
Icon color can be styled with class, but it would be nice if it could also use the color parameter to use the primary/rose/etc colors like more other objects.
Describe the solution you'd like
Adding a color parameter to Icon.
Describe alternatives you've considered
As stated you can already color with class, but matching the exact tailwind color, or changing the colors with a theme would make it easier to use the same format on all components.

To use primary color with my icons, I use <UIcon     name="i-mdi-star-outline"     class="text-primary-500"   />
the same way it is advised to do with text in the doc:


Behat: Wait until CMS loading overlay is removed

This makes tests more robust at the expense of an additional
check with every step. The "ajax steps" logic works similarly,
but does not respect the loading indicator. While this logic
waits for a short time after ajax requests are finished,
in some cases that isn't enough time for the browser to
process the response and remove the loading indicator.
Since the "ajax steps" logic applies to operations outside of the CMS
as well, we can't add the loading indicator check there.
Note: It would be cleaner to create a JS API for checking the loading
state of the CMS, but I'd consider that scope creep here ;)

@tractorcow Any idea why the branch alias isn't picked up here? Looks like travis-support creates it

Are you referring to the framework appearing as 3.x-dev instead of 3.2.x-dev? I think that could be a feature of the framework repo being the one that is actually tested. That being said, is there any harm in how it's working currently? :D

@tractorcow Re-ran tests, and they're still passing. I think the branch alias problem resolved itself in the meantime. Wanna merge?

Small bugfix for cms errors at https://github.com/silverstripe/silverstripe-framework/pull/5370


make install fails due to leveldb submodule

Running make install on this project will fail due to the fact that the Makefile that gets generated, will run make install for each submodule that is included in this project. The problem is that the leveldb submodule does not have an install target. To work around it, you can simply add an empty install: stanza to the leveldb makefile.

Hello ! I still have this problem problem on master, is that normal ?


Revert "Fix 25807: implement move for sliver multibox widget (#29188)"

This reverts commit 77ab0b83214792e8e6117295982c9397e03193a2.
Description
Reverted because of test failures rolling into the Google tree.

Do any issues need to be reopened for this?  Is there an issue opened ot track relanding this?

Do any issues need to be reopened for this? Is there an issue opened ot track relanding this?

yes I will reopen the issue https://github.com/flutter/flutter/issues/25807


bug: The icon for the indented code block is missing

Neovim version (nvim -v)
0.10.0
Operating system
Windows11
Terminal emulator / GUI
WezTerm
Describe the bug
Unindented code blocks behave normally, but the icon for indented code blocks is missing, like:

source code is:

Expected behavior
The icon for indented code blocks should exist.
Healthcheck output (:checkhealth render-markdown)
render-markdown: require("render-markdown.health").check()

markdown.nvim [nvim-treesitter] ~
- OK installed
- OK markdown: parser installed
- OK markdown_inline: parser installed
- OK highlights enabled

markdown.nvim [executables] ~
- OK none to check

markdown.nvim [configuration] ~
- OK valid

Additional information
No response

Can you provide the raw markdown for your example?
I am seeing the same when you put a slightly indented code block under another code block, but it actually works if I put it under a list. Am wondering if it has to do with how far indented things are.
There's also a separate issue of indented_code_block, which this plugin does not support.

I found something strange. If the indent is 2, the icon will not be missing:

source code is:

Yeah, I found the same thing.
I think I figured out the cause, not even sure why I even tried this, but it seems to be an interaction with indent-blankline.nvim.
If I disable indent blank line then everything starts working as expected, no random cutoff of language.
I'll try to see what's going on there.

You are right, after I disabled indent-blankline.nvim, I can also see the icon when indent is 4.

Ah it was a combination of indent blank lines overlaying the leading spaces as well as the code fence capturing the leading space. So when things got concealed they were left shifted then overlayed.
Handling it is pretty straightforward pushed a fix here: https://github.com/MeanderingProgrammer/markdown.nvim/commit/48083f81aa1100293b92755a081764f61dce2f1f.
LMK if it works for you as well!

After taking a quick look at some of my notes, this should be able to display the icon normally.

Sweet, closing the issue, feel free to re-open if you come across something.


Add request argument to opt-out of source file path information

Bug/issue #, if applicable: rdar://107075861
Summary
This adds an argument to ConvertRequest to allow opting out of source file URI information for requests that don't need or want that information.
Dependencies
None
Testing
Describe how a reviewer can test the functionality of your PR. Provide test content to test with if
applicable.
Steps:

Provide setup instructions.
Explain in detail how the functionality can be tested.

Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.

[x] Added tests
[x] Ran the ./bin/test script and it succeeded
[x] Updated documentation if necessary

@swift-ci please test


module 'Crypto.Cipher.AES' has no attribute 'AESCipher'

Hi  There,
I'm running this on Python  3.9.2 and I'm getting the package ff-3.py
\lib\site-packages\ff3\ff3.py", line 90, in init  self.aesBlock = AES.AESCipher(reverse_string(self.key))
Code from ff-3.py line 90
aes.NewCipher automatically returns the correct block based on the length of the key
    # Always use the reversed key since Encrypt and Decrypt call ciph expecting that

    self.aesBlock = AES.AESCipher(reverse_string(self.key))

I've imported the package crypto. cipher but I couldn't find the AESCipher but it seems to be under crypto, let me know what do you think?

I have updated ff3.py to use pycryptodome instead of pycrypto.  Check that you have the latest version of the file ff3.py and that you have pycryptodome installed.
Pycrypto is no longer maintained and should be uninstalled prior to installing pycryptodome.


FlowAdapters splits the org.reactivestreams package

It is impossible to consume the reactive streams flow adapters in JDK9 as a module since it splits the org.reactivestreams package across itself and the org.reactivestreams module (which it depends on). Here's an example error message from javac when you try to use it:
error: module org.reactivestreams.tck reads package org.reactivestreams from both org.reactivestreams.flowadapters and org.reactivestreams

JDK9 does not allow packages split between modules. FlowAdapters must be moved to its own package if its to be in its own artifact.

But moving something to a new pqckahe is not backwards compatible.
Is there any tooling that we should have been using to verify this problem?
What are our options here?
--
Cheers,
√
On Feb 21, 2018 03:21, "James Roper"<EMAIL_ADDRESS>wrote:

It is impossible to consume the reactive streams flow adapters in JDK9 as
a module since it splits the org.reactivestreams package across itself
and the org.reactivestreams module (which it depends on). Here's an
example error message from javac when you try to use it:
error: module org.reactivestreams.tck reads package org.reactivestreams from both org.reactivestreams.flowadapters and org.reactivestreams
JDK9 does not allow packages split between modules. FlowAdapters must be
moved to its own package if its to be in its own artifact.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/reactive-streams/reactive-streams-jvm/issues/424, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAqdyxgifRjB2z2F0RGSzLPIXOo7VBsks5tW322gaJpZM4SM-sS
.

I guess we can work around it by having the latest version of the flow-adapters jar but the non-automatic-module-name  version of the API-jar? Or?

@reactive-streams/contributors Any thoughts or suggestions here?

No clue. I have no experience with Java 9 modules but fixing this sounds like we have to break the structure of library code already released.

We could just include the adapters in the main reactive streams jar. It won't cause any problems for users of JDK8 and earlier, since they just won't use them.
We could also use MR jars to make org.reactivestreams.* extend juc.Flow.*, however the JDK maintainers have explicitly stated that that's not an intended use case for MR jars, so we'd be on shaky ground, particularly when it comes to whether other tooling (IDEs, compilers for other JVM languages) supported it.

We could just include the adapters in the main reactive streams jar. It won't cause any problems for users of JDK8 and earlier, since they just won't use them.

Yes, this works;
Akka has been doing this since JDK9 was released: https://github.com/akka/akka/commit/349a5f749ae401ca815482233af3b8d4d2bf67cd The existence of such JDK9-refering classes in a JAR does not cause problems for JDK8 users.
The classes can remain "where they are" package-wise then, so the change is source as well as binary compatible.
It also means deprecating the adapter artifact as it would not be intended to be used. It may happen that someone ends up with both artifacts on the classpath... "new" RS that contains adapters, and "old RS adapters", however since the classes are to be exactly the same that's managable risk... and should not really happen one would hope (though stating the fact that that's the risk/tradeoff we pick when we do this)

MR jars

Yeah it's not really their purpose. I would leave them out of the game -- we did investigate this quite deeply when doing the decision for the adapters in Akka Streams, deciding that it'd be abusing the mechanism, so we decided to stick to "plain boring but works" including of the JDK9 classes in artifact as usual. This makes the build a bit of a pain, but it is doable.

Is there any tooling that we should have been using to verify this problem?

Nope; no tools exist for this. The module dance 💃  is something library maintainers have to plan rather much ahead and precise... Too bad we missed it here, but it's not not-fixable thankfully this time around.

There will be tooling some day, but it's far too soon, particularly as no one seems to have any interest in upgrading JDKs until JDK11 at a minimum. Even then, the module system is incompatible with much of the Java ecosystem, due to problems like this, and isn't even compatible with current practices for unit testing (where you put unit tests in the same package as the code you're testing), without major tooling to work around that (you can do it by passing several hundred characters worth of command line options to both the compiler and java itself, these command line options allow you to patch a module with classes from another module, but it's very verbose and not at all practical to maintain these command line arguments manually, since you have to do it not just in your build tool, but also in your IDE if you want to run/debug tests in the IDE). So there's little incentive for anyone to create tooling to verify that modules work correctly any time in the near future.

include the adapters in the main reactive streams jar

Technically possible, but some developers like to dig around and will attempt to use them from Java 6..8 anyways, then complain about it doesn't work.

I find that counter argument not convincing -- "someone complains" is weaker than "it is not possible to use" which we are now facing due to the module situation :)

I think a viable solution is to move the adapters to the main jar, and then clearly JavaDoc that they require JDK9+ This wouldn't break any app since duplicates are ignored (if old adapters jar is still on the classpath), and we can just decide not to release the adapters jar anymore?

How will anyone try to use them from Java 6..8? Before they try to use them, they'll need to have a use case to use them. A use case to use them means they need some other API that accepts/produces juc.Flow that they want to use. But, since they are on Java 6..8, that other API also won't work, they won't be able to compile against it, let alone load it at runtime, since juc.Flow doesn't exist in Java 6..8.
And if after explaining that, they still find a reason to complain anyway, what's wrong with that? They're complaining about something that anyone with the smallest amount of common sense would understand is logically impossible, it's not a matter of opinion about why they won't work and why we can't make them work on Java 6..8, it's a fact, and a refusal to accept that is irrational. I don't think we should optimise our APIs to ensure that irrational people won't complain about them.


update index.html

Description

...
...
...

Related issue(s)

Thanks for contributing @akghosh111  :)
Keep contributing to open-source.