Optimize reading FixedSizeBinary arrays from parquet

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We have anecdotal evidence in DataFusion (see @samuelcolvin 's ticket https://github.com/apache/datafusion/issues/11170) that reading 16 byte UUID values from Decimal128 is much faster than FixedSizeBinary, despite seemingly very little difference between the two
@appletreeisyellow  found that most of the time is spent reading parquet: https://github.com/apache/datafusion/issues/11170#issuecomment-2276692323
@etseidl also noted the slowness when working with FixedSizeBinary here https://github.com/apache/arrow-rs/pull/6159#discussion_r1702333489
Describe the solution you'd like
Look into improving the parquet reader so reading FixedSizeBinary was faster
Describe alternatives you've considered

Additional context

I think there might be some confusion here, https://github.com/apache/datafusion/issues/11170#issuecomment-2276692323 appears to be misreading of a profile.
Whereas https://github.com/apache/arrow-rs/pull/6159#discussion_r1702333489 concerns the non-arrow codepaths which are not optimised and perform an allocation for each value

I could definitely be confused -- is there any low hanging fruit left for reading FixedSizeBinary in the ArrowReader that you know of?

Not that immediately springs to mind, but it has been almost 2 years so I could just have forgotten

It appears @etseidl  is on the case https://github.com/apache/arrow-rs/pull/6220 https://github.com/apache/arrow-rs/pull/6222 🕵

Unless I'm mistaken those are PRs for the non-arrow reader and some newly added unreleased functionality

Unless I'm mistaken those are PRs for the non-arrow reader and some newly added unreleased functionality

Correct, but #6222 is at least in the same module as this issue 😅. While I'm working on that I can take a look at the other decoders and see if there's any low hanging fruit (although tbh I'm not seeing any at the moment).

It seems like the majority of time is spent converting FixedSizeBinary to other arrow types (most of the magic happens in collect). @tustvold left a TODO https://github.com/apache/arrow-rs/blob/5c5a94a11f01a286dd03b18af0f11c327a9accc6/parquet/src/arrow/array_reader/fixed_len_byte_array.rs#L165 for a possible improvement, but the conversion has to happen eventually, so I don't know if there's any more room for improvement.

I did a quick test of FIXED_LEN_BYTE_ARRAY(16)/Decima128 vs unannotated FIXED_LEN_BYTE_ARRAY(16), and the latter was much faster.
arrow_array_reader/FIXED_LENGTH_BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs
                        time:   [1.0326 ms 1.0456 ms 1.0595 ms]

arrow_array_reader/FIXED_LENGTH_BYTE_ARRAY/FixedLen(16)Array/plain encoded, mandatory, no NULLs
                        time:   [110.30 µs 110.48 µs 110.67 µs]

I don't think FLBA decoding in the arrow reader is the culprit.

Pardon a little more spam on this, but as I dig deeper into the FixedSizeBinaryArray->PrimitiveArray<T> transformation, one thing I noticed is that we're creating an iterator of Option<NativeType>, and passing that to from_iter https://github.com/apache/arrow-rs/blob/a693f0f9c37567b2b121e261fc0a4587776d5ca4/arrow-array/src/array/primitive_array.rs#L1318 which iterates over the values, collecting them into a Buffer, while creating a null buffer as it goes. But in FixedLenByteArrayReader::consume_batch, we already have a null buffer in binary https://github.com/apache/arrow-rs/blob/a693f0f9c37567b2b121e261fc0a4587776d5ca4/parquet/src/arrow/array_reader/fixed_len_byte_array.rs#L163 so it seems a waste to create one again.
I'm wondering if it would make sense here to (in the cases where we're converting from FixedSizeBinaryArray to PrimitiveArray<>) take the null buffer from binary, modify the iterator to return the native type rather than an Option,  and create the PrimitiveArray from the iterator and the stolen null buffer. I've implemented a PoC that knocks off a good bit of time (1.0ms -> 740us for the FLBA/Decimal128 bench).
            ArrowType::Decimal128(p, s) => {
                let nb = binary.take_nulls();
                let decimal = binary
                    .iter()
                    .map(|o| match o {
                        Some(b) => i128::from_be_bytes(sign_extend_be(b)),
                        None => i128::default(),
                    });
                let decimal = Decimal128Array::from_iter_values_with_nulls(decimal, nb)
                    .with_precision_and_scale(*p, *s)?;
                Arc::new(decimal)
            }

Am I missing something subtle (not out of the question...this is my 5th attempt or so) that would break this? In particular, is it safe to assume the null buffer in binary would be the same as computed in from_iter()?
@tustvold  @alamb

It sounds plausible, but I've not spent much time with this code beyond what was necessary to allow ripping out the legacy ComplexObjectArrayReader. There is almost certainly some low hanging fruit when it comes to reading Decimal128Array and IntervalArray, that is part of why I was quite so confused to see this issue which suggests the opposite.

Thanks. I'll clean up what I have and submit a PR in the next day or two then. I agree that the initial report was likely a misunderstanding.

I think we can plausibly claim that we have accomplished this ticket: Looking into optimizing the reads. So marking it as closed. Let's open other tickets as we find other ways to improve things

label_issue.py automatically added labels {'arrow'} from #6244


bluetooth module visible if no bluetooth controller found

Issue #1707 still occurs in v0.9.17. I wasn't sure if I should tag you so I opened a new issue.
When the module has some padding or margin set, it is visible as an empty field.

When you click on it, it appears as if bluetooth was enabled (my computer doesn't have bluetooth):

I was about to open basically the same issue instead I want to add something to this one. I propose that the module should show either the "format" or "format-disabled" entry when no bluetooth controller has been found.


Fixed typo (clas)

@hubalazs : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Broker: Specified group generation id is not valid

Description
I have a streams application running which uses 2 ForeachAsync() loops for processing. I am seeing an error/exception being logged by the Streamiz Library:
external-stream-thread[] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:
{
"error.class": "Confluent.Kafka.KafkaException",
"error.message": "Broker: Specified group generation id is not valid",
"error.stack": "\t   at Confluent.Kafka.Impl.SafeKafkaHandle.Commit(IEnumerable1 offsets) \n\t   at Confluent.Kafka.Consumer2.Commit(IEnumerable`1 offsets) \n\t   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.CommitOffsets(Boolean clearBuffer) \n\t   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.Commit() \n\t   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.<>c__DisplayClass44_1.b__2() \n\t   at Streamiz.Kafka.Net.Crosscutting.ActionHelper.MeasureLatency(Action action) \n\t   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.Run()",
"level": "ERROR",
"message": "external-stream-thread[] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:",
}
Also there is warnings being logged like this :
stream-task[] Error with a non-fatal error during committing offset (ignore this, and try to commit during next time): Broker: Specified group generation id is not valid
I notice issues with rebalance when this happens and all consumers do not join the consumer group until a manual restart happens. Although not fatal, I see lag build up and overall slow processing.

Can you please review and let me know if the above is a critical error from Streamiz?
Does it affect re-balance?
Also are there any config settings that I need to tweak in the client to overcome this error?

How to reproduce
Not sure if this can be easily reproduced. Having a topology that uses 2 ForeachAsync() loops (which does some work processing the message) and adding consumers to an existing group to see if the re-balance happens correctly.
Config :
AutoOffsetReset = earliest
AutoRegisterSchemas = false,
UseLatestVersion = true,
MaxPollIntervalMs = 300000
Checklist
Please provide the following information:

[ ] Streamiz.Kafka.Net nuget version. - 1.5.1
[ ] Apache Kafka version. - 3.6
[ ] Client configuration. - attached above
[ ] Operating system. - linux
[ ] Provide logs (with in debug mode (log4net and StreamConfig.Debug) as necessary in configuration). - logs provided above
[ ] Critical issue. - low

Hi @rao-mayur ,
Thanks for your ticket.
This kind of issue happens when you take too much time to process messages, so the consumer instance of the group is evicted. Remember in Kafka, you have to poll data regularly depending your configuration MaxPollIntervalMs.
So in your application, the external stream thread failed because when the application has tried to commit the last offset, the instance has evicted from the group.
You can increase the MaxPollIntervalMs, and if you would like to continue to process messages in this case, you can override the InnerExceptionHandler to return CONTINUE. In this case, the consumer will stop, recreate another instance, but keep in mind that you can have duplicated in the downstream processors.

Hi @LGouellec
Thanks for the response. I have a few follow up :

So more time to process messages mean consumer instance is assumed dead/evicted and hence the generation id not available error is thrown by the broker right?
I see two clients when I startup my app, one just  and other -external (which I think is for the foreachasync), so if time is spent in the foreachasync loop are consumers under -external evicted/dead? What happens to ones under ?
RE InnerExceptionHandler the consumer will stop, recreate another instance : meaning a new stream thread is created (entire topology is built again)? Like a restart of the app?
I see another aption FAIL for InnerExceptionHandler, what are the cases to use this? Should I try it for solving this error?

@rao-mayur
1- Yes exactly, there are lot of blog post on that. Example : https://medium.com/codex/dealing-with-long-running-jobs-using-apache-kafka-192f053e1691. Streamiz has internally some mecanishm with a back pressure pattern, but in some cases that doesn't always avoid being kicked out of the group
2- Yes exactly, all the async processing is made with a dedicated consumer-group (ConsumerGroupName'-external). So doesn't have impact of the initial  consumer group.
3- Not totally, if an internal exception is raised (Consumer group evicted for instance), the external consumer is closed properly, and another instance of the consumer group ConsumerGroupName'-external will be created. See : https://github.com/LGouellec/kafka-streams-dotnet/blob/3e77bb5b3ae670a9e261258faa9adf8c5cd44ca2/core/Processors/ExternalStreamThread.cs#L406.
4- This is the default behavior, FAIL means the thread will die and nothing else more. So no more external processing in your current application instance.

Ok thank you @LGouellec
I will look into the config setting or InnerExceptionHandler and see if it helps with the error.

Hi @LGouellec
I set the InnerExcetionHandler to continue in the stream config setting. Got the warning Detected that the thread throw an inner exception. Your configuration manager has decided to continue running stream processing. So will close out all assigned tasks and rejoin the consumer group so I know the config setting is being applied.
Even after making the config update the issue I continue to see is : all the consumers in a group (I have 6 consumers running) do not join the group as part of the rebalance. This is for the -external group (ForEaschAsync() processor). This causes processing to go down and builds up lag. Manually re-starting the processors seem to help and more consumers join the group.
Do you have any advise for this? How to be sure that all consumers re join the group after InnerExceptionHandler returns Continue?
Thanks,
Mayur.

Hi @rao-mayur ,
Can I share your last logs please ?
Best regards,

Hey @rao-mayur ,
I don't understand your logs and usecase. So you have 6 pods, and so 1 external stream thread per pods.
The log shared is a concatenation of your 6 pods ? or just 1 pod ? Why the 6 threads should restart after the Continue, it's just the thread impacted by the error.
Can you share in 6 differents files the logs of each container please ? It will be easier to troubleshoot.
Btw,  I'm currently conducting a satisfaction survey to understand how I can better serve and I would love to get your feedback on the product.
Your insights are invaluable and will help us shape the future of our product to better meet your needs. The survey will only take a few minutes, and your responses will be completely confidential.
Survey
Thank you for your time and feedback!

I see this error but not because it took too long to process messages; rather the state transitioned BACK from RUNNING to PARTITIONS_ASSIGNED. Here is the timeline:
INFORMATION
Sep 24, 2024 01:41:03.062
stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-stream-thread-0] State transition from PARTITIONS_ASSIGNED to RUNNING

INFORMATION
Sep 24, 2024 01:41:06.176
stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-stream-thread-0] State transition from PARTITIONS_ASSIGNED to RUNNING

INFORMATION
Sep 24, 2024 01:42:00.076
external-stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-external-stream-thread] Processed 62 total records in 60000ms

As you can see, the thread processed 62 messages in a minute; which is less than the default 5 min MaxPollInterval.
Now these state transitions occur but I am not sure why:

INFORMATION
Sep 24, 2024 01:42:06.317
stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-stream-thread-0] State transition from RUNNING to PARTITIONS_ASSIGNED

INFORMATION
Sep 24, 2024 01:42:06.718
stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-stream-thread-0] State transition from PARTITIONS_ASSIGNED to RUNNING

This causes the error:
ERROR
Sep 24, 2024 01:42:06.763
external-stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-external-stream-thread] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:
error.message:Broker: Specified group generation id is not valid
error.stack:	   at Confluent.Kafka.Impl.SafeKafkaHandle.Commit(IEnumerable`1 offsets)
	   at Confluent.Kafka.Consumer`2.Commit(IEnumerable`1 offsets)
	   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.CommitOffsets(Boolean clearBuffer)
	   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.Commit()
	   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.<>c__DisplayClass44_1.<Run>b__2()
	   at Streamiz.Kafka.Net.Crosscutting.ActionHelper.MeasureLatency(Action action)
	   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.Run()

INFORMATION
Sep 24, 2024 01:42:06.792
external-stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-external-stream-thread] Shutting down

ERROR
Sep 24, 2024 01:42:06.811
external-stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-external-stream-thread] Failed to close external stream thread due to the following error:
error.message:Broker: Specified group generation id is not valid
error.stack:	   at Confluent.Kafka.Impl.SafeKafkaHandle.Commit(IEnumerable`1 offsets)
	   at Confluent.Kafka.Consumer`2.Commit(IEnumerable`1 offsets)
	   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.CommitOffsets(Boolean clearBuffer)
	   at Streamiz.Kafka.Net.Processors.ExternalStreamThread.CompleteShutdown()

Is the expectation here to CONTINUE on inner exceptions like you proposed earlier? I don't think increasing MaxPollIntervalMs helps in this particular case.
Moreover, HandleInnerExceptions() makes a call to CommitOffsets(true). Won't this itself throw the same exception as above? I don't think this gets caught anywhere. Please confirm.

Another case: On startup, task restoration takes > 5 min, which is the default for MaxPollIntervalMs. As a result, Consume() fails:
      stream-task[3|1] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
      Restoration took 382079ms for all tasks 5-2,5-0,6-1,6-0,3-0,2-1,2-0,5-3,5-1,3-1,6-3,6-2,5-4,6-5,6-4,5-5

fail: Streamiz.Kafka.Net.Processors.StreamThread[0]
      stream-thread[digestmanager-363a729f-8fe7-4883-9e04-9795aa3625aa-stream-thread-0] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:
      Confluent.Kafka.ConsumeException: Application maximum poll interval (300000ms) exceeded by 441ms
         at Confluent.Kafka.Consumer`2.Consume(Int32 millisecondsTimeout)
         at Confluent.Kafka.Consumer`2.Consume(TimeSpan timeout)
         at Streamiz.Kafka.Net.Crosscutting.KafkaExtensions.ConsumeRecords[K,V](IConsumer`2 consumer, TimeSpan timeout, Int64 maxRecords)
         at Streamiz.Kafka.Net.Processors.StreamThread.<>c__DisplayClass65_0.<Run>b__0()
         at Streamiz.Kafka.Net.Crosscutting.ActionHelper.MeasureLatency(Action action)
         at Streamiz.Kafka.Net.Processors.StreamThread.Run()

We are able to workaround this by changing InnerExceptionHandler to CONTINUE and by increasing MaxPollIntervalMs. Is a better alternative to create Consumer instance in StreamThread after the restoration is complete? Is that feasible?

external-stream-thread[sessionmanager-ce34557c-bc83-48cf-a569-a801ee06db33-external-stream-thread] Processed 62 total records in 60000ms

Thanks for your input, for your information, I will complete refactor the external stream thread processor in the 1.8.0 release.
Let me check your logs and troubleshoot the origin

Another case: On startup, task restoration takes > 5 min, which is the default for MaxPollIntervalMs. As a result, Consume() fails:
      stream-task[3|1] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
      Restoration took 382079ms for all tasks 5-2,5-0,6-1,6-0,3-0,2-1,2-0,5-3,5-1,3-1,6-3,6-2,5-4,6-5,6-4,5-5

fail: Streamiz.Kafka.Net.Processors.StreamThread[0]
      stream-thread[digestmanager-363a729f-8fe7-4883-9e04-9795aa3625aa-stream-thread-0] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors:
      Confluent.Kafka.ConsumeException: Application maximum poll interval (300000ms) exceeded by 441ms
         at Confluent.Kafka.Consumer`2.Consume(Int32 millisecondsTimeout)
         at Confluent.Kafka.Consumer`2.Consume(TimeSpan timeout)
         at Streamiz.Kafka.Net.Crosscutting.KafkaExtensions.ConsumeRecords[K,V](IConsumer`2 consumer, TimeSpan timeout, Int64 maxRecords)
         at Streamiz.Kafka.Net.Processors.StreamThread.<>c__DisplayClass65_0.<Run>b__0()
         at Streamiz.Kafka.Net.Crosscutting.ActionHelper.MeasureLatency(Action action)
         at Streamiz.Kafka.Net.Processors.StreamThread.Run()

We are able to workaround this by changing InnerExceptionHandler to CONTINUE and by increasing MaxPollIntervalMs. Is a better alternative to create Consumer instance in StreamThread after the restoration is complete? Is that feasible?

This is a common error yes, when the task restoration is bigger than 5 minutes per default, the consumer fail due to an inactive consumer.
Workaround :  InnerExceptionHandler to CONTINUE and by increasing MaxPollIntervalMs.

Is a better alternative to create Consumer instance in StreamThread after the restoration is complete? Is that feasible?
No better alternative.

For your information, I will work to improve the restoration time soon (plan for 1.8.0) to leverage bulk upsert into RocksDb instead of unitary upsert.


[BUG] Random CTD

Describe the bug
I consistantly get CTDs on new saves and existing saves.
To Reproduce
Steps to reproduce the behavior:
There doesn't seem to be a rhyme or reason to it, but it seems to prolong it when I change areas, but it happens anywhere.
Expected behavior
No crashes
Desktop (please complete the following information):

OS: [Win11 Pro]

** FSMP**

Tried 1.50.9rc1
CUDA:  Both No_CUDA or CUDA
AVX: No_AVX and AVX.
SKSE
2.0.20

Skyrim

1.5.97

crash-2023-08-23-01-43-29.log
crash-2023-08-23-01-15-00.log
crash-2023-08-23-01-04-37.log
hdtSMP64.log
Skyrim.zip

2nd and 3rd crashes aren't related to FSMP, see FSMPU.
1st crash happens on the parallelization code. I would bet a crash happened elsewhere in the parallelized threads and it caused this crash.


Add filtering to checkbox select menu

Want the ability to filter items in a checkbox select component. This is useful when the list of potential options is very large.
Here is the target design:

This takes the existing checkbox select and adds a type-ahead filter and an optional Clear button. The clear button, when present, will clear any existing selections. If nothing selected, the clear button should be hidden.

@bmignano I've opened this issue following conversation with Zac and @jcaianirh working on a new search page for Openshift. The request was to add filtering to a checkbox select to quickly select resources from a potentially long list of options as shown in this design: http://openshift.github.io/openshift-origin-design/admin-perspective/future-openshift/search/search
Can you confirm that this is what you would expect? Wasn't sure if the Clear button is needed for your application, but I know that this is something that was discussed earlier with you and @serenamarie125 .

@mcarrano thanks for the fast turnaround. this is awesome!

@mcarrano Thanks, this looks great to me! I'm not sure that a Clear button is needed in this context. I'd expect the textbox filter to behave like any other textbox filter in a toolbar, which at least in OpenShift do not have a clear functionality. Also, since it's a dynamic filter (i.e., it searches the list as you type), it seems unnecessary to me. Open to hearing other opinions though!

:tada: This issue has been resolved in version 2.63.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


clangd jump to .h file , and don't know where is the implementation of a function

source code is neovim
git clone https://github.com/neovim/neovim
cd neovim && make CMAKE_BUILD_TYPE=Release

editor is NVIM v0.9.2
Build type: Release
LuaJIT 2.1.0-beta3
a function named lua_tolstring （and some other function like this)
1、shortcut gd jump to .h file
2、shortcut gr find all the references
but all this two steps never goes to the implementation of this function
telescope.nvim also can't find the right position
It seems compile_commands.json  or cmakelists.txt  is not  fully parsed by neovim and clangd .

Where in the project's source code is the definition of lua_tolstring that you would like clangd to navigate to?

Sorry guys
The implementation of lua_tolstring is not a part of neovim .  The C file named lj_api.c is  a part of .deps/ , which means external  package or library .

Right, so .deps/builds/src/luajit/src/lj_api.c is not part of clangd's index, because it's not listed in the project's compile_commands.json. As a result, clangd does not know about this file and never opens or processes it.

I write a new issue in xmake project

Programmer  would read a source code package which can't be compiled . Make tools like xmake should provide a simple way to create a fake compile_commands.json ，which make neovim and lsp happy .


Add option to place Vertical Tabs' sidebar at the top of the Files sidebar, similar to vscode

Visual Studio Code's Explorer sidebar which lists all open tabs and the open folder (in this case EXAMPLE).
I find how vscode displays open tabs and the open folder good for how I work. I would like to have an option in the plugin to, instead of having a separate sidebar for Vertical Tabs, add that content at the top of the Files sidebar.

Being able to glance at the file structure in the sidebar without having to switching sidebar is helpful when I want to open a file I can't quite remember the name of.
When I want to browse for a file, if the Vertical Tabs and Files sidebars were a single sidebar, it would save me the click to switch to the Files sidebar.

Hi @redoak ! This functionality is already supported by Obsidian. You can simply drag the horizontal tab of File Browser to the bottom, or drag the Vertical Tabs to the bottom. You can make splits in sidebars, just as in the main areas. If you encounter any problems, please let me know. Thank you.


drivers: timer: stm32 LPTIM  input clock with a prescaler

This PR adds a prescaler to the LPTIM input clock to increase the max timeout (0xFFFF lptim counter unit).
With this prescaler, the mcu is able to sleep up to 256 sec w/o wakeup.
With the LSE clock as input (32768Hz),
prescaler = 1 (no division)   one lptim count is  of  0.03 ms and max timeout is 2 seconds
prescaler = 2  one lptim count is  of  0.06 ms and max timeout is 4 seconds
prescaler = 4   -->   one lptim count is  of  0.12 ms and max timeout is 8 seconds
prescaler = 8    -->  one lptim count is  of  0.24 ms and max timeout is 16 seconds
prescaler = 16  -->  one lptim count is  of  0.48 ms and max timeout is 32 seconds
prescaler = 32  -->   one lptim count is  of  0.97 ms and max timeout is 64 seconds
prescaler = 64  -->  one lptim count is  of  1.95 ms and max timeout is 128 seconds
prescaler = 128   one lptim count is  of  3.9 ms and max timeout is 256 seconds
Signed-off-by: Francois Ramu<EMAIL_ADDRESS>

POSITION_VAL  macro is not defined for stm32l0xx so use __CLZ(__RBIT(value)) instead

Will use the LPTIM1 device only and define the clock  freq as a constant of this DTS NODE from the clock source divided by the prescaler

add the configuration so that nucleo_l476rg can run the tests/subsys/pm/power_mgmt_soc/
--> PM is one of the tags in the testcase.yaml : this is done through the LPTIM as kernel timer

with comment in the samples/boards/stm32/power mgmt/blinky/   nucleo_wb55rg.overlay

with comment in the samples/boards/stm32/power mgmt/blinky/ nucleo_wb55rg.overlay

@FRASTM I am not sure what you mean by this, but it sounds like you may want to add some more comments to this file before merging. Is that right?
Also, please rebase.

rebased on c6b05afc92550eea26fb38ac92cfac114f5e5581

rebase on 801fe1fa06bd6f8cc2a55079411fd4feac55eff6

Codecov Report

Merging #44266 (801fe1f) into main (801fe1f) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 801fe1f differs from pull request most recent head 40b1fda. Consider uploading reports for the commit 40b1fda to get more accurate results

@@           Coverage Diff           @@
##             main   #44266   +/-   ##
=======================================
  Coverage   50.05%   50.05%
=======================================
  Files         648      648
  Lines       81299    81299
  Branches    19101    19101
=======================================
  Hits        40697    40697
  Misses      33869    33869
  Partials     6733     6733

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 801fe1f...40b1fda. Read the comment docs.

That's a fair point, though IMHO auto-computing this from the tick rate seems a little clumsy.  Certainly it's not worth a big bikeshed over this.
Most app authors tend to be told things (by me, as often as not) like "The tick rate is just an arbitrary number, you can set it as high as you like as long as your hardware can support the precision and you aren't subject to rollover bugs with long delays".  Routine systems tend to use over-precise values (e.g. 10 kHz is pretty standard, 50 kHz on intel_adsp!) because of this.  If that setting now has power impact, it's going to be surprising to app authors.
My core point is this: kconfig is trivially easy for applications to use, devicetree is very hard for apps.  If apps need to set something, we should strongly prefer kconfig.

My core point is this: kconfig is trivially easy for applications to use, devicetree is very hard for apps. If apps need to set something, we should strongly prefer kconfig.

Ok, got it.
About:

If it that setting now has power impact, it's going to be surprising to app authors.

Unfortunately, it's already what we are doing, but we don't have much choices on STM32 currently. For low power operations, we have two constraints:
A] In Zephyr kernel atm, it's not possible to switch tick source between modes (same tick source required in running and stop modes)
B] counters are available in STOP mode:

RTC, allows long sleep duration but it's not meant to be used in runtime (it's actually a time counter, we need to perform conversions all the time)
LPTIM, using 32KHz clock: easy to use but it's a 16 bits counter (on 32KHz: Max timeout 2 seconds)

So, today for STM32 users on Zephyr:

in non PM use cases, Cortex SysTick is used (with default tick rate applied default 10000 if TICKLESS_KERNEL)
in PM use cases, we're using LPTIM as tick source, but tick rate has to be adapted (to 4096) since counter clock is 32KHz

What we're trying to do here is just to extend the LPTIM timeout (because which is legit for PM use cases), playing with its prescaler, but since it has direct impact on tick count, it requires to play with the tick rate (to an extent acceptable by the app, to be fine tuned by the user > we document this in the binding).
Of course, if it were possible to switch tick sources at change of operation mode, we would be able to use the RTC for long sleep periods, be back on Cortex Systick at run time, and we wouldn't need all this.

My core point is this: kconfig is trivially easy for applications to use, devicetree is very hard for apps. If apps need to set something, we should strongly prefer kconfig.

Wait, what? No. Hard disagree on this, and frankly I am sick of this sentiment.

rebase and move one change in commit f6445c1a8fa11432695ee353efa7893738bcc813

With high prescaler values (especially above 32) the Tick per sec might be adapted to the lptim clock unit (when running tests/kernel/timer with  CONFIG_PM=y )

To be rebased after merge of https://github.com/zephyrproject-rtos/zephyr/pull/48771

rebase on 3217d9737530bddbd1a109612ec9cdb1765e3ff1
Choose one lptim instances with zephyr,pm-timer = &lptim1

to come after #49357 is merged

rebased on 2e04bfdfe0f8fda3ec5df1365f8f71c744cf5c9f

rebase

Rebase on c4b10874eb9bd46b9b824d16aa25545dc2f788e6
Reduce the lptim_clock_freq by LPTIM_CLOCK_RATIO during the init.

@FRASTM please rebase

rebase on 6c1363e34dfc121fed3346c55241fcfd87b2070a

Not happy about the Kconfig option and I think it could be moved elsewhere, but accepted for now since I don't have a concrete suggestion.
@kartben this is an example of where it would be useful to have an enumeration of device drivers and associated docs.


Add Loongarch64 support

Loongarch is a new RISC architecture.
Loongarch doc: https://github.com/loongson-community/LoongArch-Documentation/

Thanks! Is there an upstream rust target specification for this target?

Yes, the definition of this target is based on the rust target specification.
And regarding rust's support for loongarch64 architecture, we are under development.

Hm I don't see the target here? https://github.com/rust-lang/rust/tree/master/compiler/rustc_target/src/spec

Sorry, we are still  testing  and  developing rust-lang/rust locally. After the testing is completed, we will submit it to the upstream community.

Ah ok sounds good, would you be ok if I hold off on merging this until your target is closer to being merged into rust-lang/rust?

Of course.
However, the integration of rust-lang/rust and its dependent library libc into the community may be a long-term work, and it will take a while.

If cc-rs can be incorporated into the community, it will be friendly to users of the new architecture, because they do not need to manually adapt it locally every time they use it.

Yes that makes sense @zhaixiaojuan, but I prefer to have targets landed in rust-lang/rust before they're added here.

@alexcrichton The loongarch64-unknown-linux-gnu was added as a tier 3 target, could you review this patch again please?


Log the changed timeout

Log the changed timeout to gather data for a default timeout value in nordpol, as described in #76

stale


DOC: remove empty html to reduce size of the artifact

Looking at one of the output log of the artifact upload, one can notice that there are lot of entries for lapack and blas. But these pages do not bring any value since they are just wrappers to low level functions. See for example https://docs.scipy.org/doc/scipy-1.8.0/html-scipyorg/reference/generated/scipy.linalg.lapack.zungrq.html
I checked locally and removing these would save:

Lapack 580 files: 11Mo
Blas 141 files: 2.7 Mo

On 125Mo, it's 10% for an easy change with no impact for the user. Considering the volume of builds we have, this is not negligible IMO. This would reduce the upload time and our network consumption.
The next contestant for sliming things would be MathJax. Here I don't know so would leave this out for now.
Apart from that, there is not much we can do with upload speed. CircleCI is storing the artifacts on S3 so we cannot upload an archive and unarchive it there. Going further I think we would need to use something like Netlify which is more appropriate for websites.

Would be nice to actually add some docstrings to BLAS/LAPACK functions. That would be quite a job though: starting from netlib sources, one would need to go through all our wrappers and manually adjust signatures for hidden low-level arguments (leading dimensions etc). So maybe not worth the effort, the ROI is too low.

LAPACK funcs actually have some use but as you said it can be smaller if we can find an alternative representations. Because our wrappers are not just wrappers we change the API substantially. Most importantly, chars become ints or bools, e.g., any job='L' or job='N' becomes lower=1 or eigv=0 and so on. Not many customers but still some

Ok these are good precisions. Not as simple as removing then, but maybe a single html with the correct signature would do instead (so the current listing pages but without links to individual functions).

Not many customers but still some .

I guess that's the dilemma. There aren't many users, but they are public API functions and there is real content here:

It comes from the f2py signature files. If we throw that all away, then anyone who needs info should then just reads the raw f2py signature files.

Not as simple as removing then, but maybe a single html with the correct signature would do instead (so the current listing pages but without links to individual functions).

That is still throwing away all this information? The size of the html docs is a pain, but this doesn't seem justified. Further optimizing the Sphinx theme seems like a better way to go I'd think.

as you said it can be smaller if we can find an alternative representations

There is little superfluous info on each page that can be removed. The size issue simply comes from the number of pages. We could reduce it by having multiple functions on a single page, but that'd be a fairly large amount of work I'd think in our Sphinx machinery or numpydoc.

Yes, that would make sense. The best example to look at is probably https://github.com/scipy/scipy/blob/main/doc/source/scipyoptdoc.py, which does something custom for the different optimize.minimize methods. It's not all that easy to deal with unfortunately.


[stable/redis] Allow templated secret name

Signed-off-by: Josh Bryans<EMAIL_ADDRESS>
What this PR does / why we need it:
This allows the secret name to include templating.
For example, in a case where there is an umbrella chart with the sub charts redis and myApp. If the redis chart let me use templating in the secret name. I could at the umbrella level create my own secret, name that secret {{.Release.Name}}-myapp-redis-secret, and I could also pass {{.Release.Name}}-myapp-redis-secret into the redis.existingSecret value.
Currently, if I want to create a secret in the umbrella chart that is used by the redis chart it needs to be a static name.
Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

LGTM.
@carrodher  Cloud you please merge it.

/ok-to-test
/lgtm


Not able to build the play-scala-websocket example

play-scala-websocket-example git:(2.7.x) ./gradlew clean build

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'play-scala-websocket-example'.
> Exception thrown while executing model rule: PlayApplicationPlugin.Rules#createBinaries(ModelMap<PlayApplicationBinarySpec>, PlayApplicationSpecInternal, PlatformResolvers, PlayToolChainInternal, PlayPluginConfigurations, File, ProjectIdentifier) > create(binary)
   > Not a supported Play version: 2.7.0. This plugin is compatible with: [PLAY_2_2_X, PLAY_2_3_X, PLAY_2_4_X, PLAY_2_5_X, PLAY_2_6_X].

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 0s

Ah.  So Gradle doesn't support Play 2.7 (yet?).  I removed the test-gradle CI was using, but I see I didn't remove the other references to Gradle.  I'll do that, too.

Thanks for the quick reply. Meanwhile it works if I just downgrade to 2.6.0:
def playVersion = "2.6.0"

There's also the 2.6.x branch which maintains these samples working on Play 2.6.


Set background of the cell

Hi
Is there's anyway to set (dynamic) background to the cells (td's)?
Thanks

Yes, this is native DataTables functionality - see https://datatables.net/reference/option/rowCallback


Added new test + fixed tracer

added the second test set where the aim is to recover the most significant coefficients.
fixed tracer
added inline doc for bpdual

@mpf  -- if you are happy then I'd like to suggest that this can be registered as 0.0.1 in the General registry.

Great experiment! Thanks for adding that, @tinatorabi. Homotopy seems to behave exactly as it should.
@cortner — I'll next setup the registration bot.


The cardiomechanic pump kills

So, my problem is that the cardiomechanic pump kills me because i cant pit it with the heart and  i dont find a way to get the "card. pump" and survive, and i dont find something like a mechanic heart

You're powering it, right?

As with most cyberware, you'll need a power source of some kind.


'SigFieldSpec' Object is not iterable

According to documentation, I am trying to add a signature field to a PDF document, so:
This is my code:
  `from  pyhanko.sign.fields import append_signature_fields,SigFieldSpec
   from pyhanko.pdf_utils.incremental_writer import IncrementalPdfFileWriter
   with open('output-signed.pdf', 'rb+') as doc:
                w = IncrementalPdfFileWriter(doc)
                append_signature_fields(w, SigFieldSpec(sig_field_name='Sig'))
                w.write_in_place()`

I had to change"append_signature_field" for "append_signature_fields", but no matter the parameters I add to SigFieldSpec, that I have always the same result which is  'SigFieldSpec' object is not iterable.
But my CLI version works perfecly
Any help will be grateful
Thanks in advance

Hi, thanks for posting!
That bit of API changed in commit a3ffbdb7, and that's what the documentation reflects. The point is that add_signature_field used to be called add_signature_fields (note the plural), and take a list of SigFieldSpec objects as input. I decided that that was counterintuitive, so I tweaked the API a little.
I think you might be using an older version of the codebase.
Can you try pulling master and/or with the 0.1.0 alpha release, and see if your issue persists?
PyHanko's signing API isn't fully stable yet, and probably won't be until 1.0.0 is released. Sorry about that! Anyway, even before we reach that point, all future minor releases will come with a changelog, which will definitely document any high-level API breakage.

Hi, thanks for posting!
That bit of API changed in commit a3ffbdb7, and that's what the documentation reflects. The point is that add_signature_field used to be called add_signature_fields (note the plural), and take a list of SigFieldSpec objects as input. I decided that that was counterintuitive, so I tweaked the API a little.
I think you might be using an older version of the codebase.
Can you try pulling master and/or with the 0.1.0 alpha release, and see if your issue persists?
PyHanko's signing API isn't fully stable yet, and probably won't be until 1.0.0 is released. Sorry about that! Anyway, even before we reach that point, all future minor releases will come with a changelog, which will definitely document any high-level API breakage.

Hi Matthias:
Thanks for your reply. I have upgraded to the latest release and now works
like a charm. Now I am trying to sign with a PKCS11 device (Spanish eID
card), which seems to be a little bit messy.
Nevertheless, congratulations for your code and hope you will be successful
in future releases. Once again, thank you very much.
Best regards
Fernando
El mié, 6 ene 2021 a las 2:00, Matthias Valvekens<EMAIL_ADDRESS>escribió:

Hi, thanks for posting!
That bit of API changed in commit a3ffbdb
https://github.com/MatthiasValvekens/pyHanko/commit/a3ffbdb7065b7289a6647642430827a033d33b89,
and that's what the documentation reflects. The point is that
add_signature_field used to be called add_signature_fields (note the
plural), and take a list of SigFieldSpec objects as input. I decided that
that was counterintuitive, so I tweaked the API a little.
I think you might be using an older version of the codebase.
Can you try pulling master and/or with the 0.1.0 alpha release, and see if
your issue persists?
PyHanko's signing API isn't fully stable yet, and probably won't be until
1.0.0 is released. Sorry about that! Anyway, even before we reach that
point, all future minor releases will come with a changelog, which will
definitely document any high-level API breakage.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MatthiasValvekens/pyHanko/issues/2#issuecomment-754996335,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIHEHGXC6WYR5GCBEE77GM3SYOYZHANCNFSM4VV2GIEQ
.

Hi Matthias:
Thanks for your reply. I have upgraded to the latest release and now works
like a charm. Now I am trying to sign with a PKCS11 device (Spanish eID
card), which seems to be a little bit messy.
Nevertheless, congratulations for your code and hope you will be successful
in future releases. Once again, thank you very much.
Best regards
Fernando
El mié, 6 ene 2021 a las 2:00, Matthias Valvekens<EMAIL_ADDRESS>escribió:

Hi, thanks for posting!
That bit of API changed in commit a3ffbdb
https://github.com/MatthiasValvekens/pyHanko/commit/a3ffbdb7065b7289a6647642430827a033d33b89,
and that's what the documentation reflects. The point is that
add_signature_field used to be called add_signature_fields (note the
plural), and take a list of SigFieldSpec objects as input. I decided that
that was counterintuitive, so I tweaked the API a little.
I think you might be using an older version of the codebase.
Can you try pulling master and/or with the 0.1.0 alpha release, and see if
your issue persists?
PyHanko's signing API isn't fully stable yet, and probably won't be until
1.0.0 is released. Sorry about that! Anyway, even before we reach that
point, all future minor releases will come with a changelog, which will
definitely document any high-level API breakage.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MatthiasValvekens/pyHanko/issues/2#issuecomment-754996335,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIHEHGXC6WYR5GCBEE77GM3SYOYZHANCNFSM4VV2GIEQ
.

Thanks a lot!
My experience with PKCS#11 is pretty limited, so I only fully implemented an adapter for the Belgian eID (that's all I had to test with).
Provided that the Spanish cards also use RSA-based keys, it shouldn't be too hard to adapt the BeID implementation to your needs.
If you manage to figure out how to get the Spanish one to work, I'd hugely appreciate it if you could report your findings (either by opening another issue and/or pull request). The more input I get, the easier it'll be to get support for general PKCS#11 in the future. :)

Thanks a lot!
My experience with PKCS#11 is pretty limited, so I only fully implemented an adapter for the Belgian eID (that's all I had to test with).
Provided that the Spanish cards also use RSA-based keys, it shouldn't be too hard to adapt the BeID implementation to your needs.
If you manage to figure out how to get the Spanish one to work, I'd hugely appreciate it if you could report your findings (either by opening another issue and/or pull request). The more input I get, the easier it'll be to get support for general PKCS#11 in the future. :)

Hi Mathias again. Sorry for the disturbance. I have upgraded my system and
python has upgraded from 3.8 to 3.9. On version 3.8 everything worked fine,
but now I have got this error which I do not know how to solve. Error
occurs when I get a p12 file  certificate to sign, when is verifying the
certificate. This is the error:
File
"/root/Descargas/programacion/firma_digital/pades_sign/firma_final.py",
line 307, in certificadodig
out = signers.PdfSigner(signature_meta=signers.PdfSignatureMetadata(
File "/usr/local/lib/python3.9/site-packages/pyhanko/sign/signers.py",
line 2198, in sign_pdf
signer_cert_validation_path = validator.validate_usage(
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/init.py",
line 211, in validate_usage
self._validate_path()
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/init.py",
line 139, in _validate_path
validate_path(self._context, candidate_path, self._params)
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/validate.py",
line 70, in validate_path
return _validate_path(validation_context, path, parameters=parameters)
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/validate.py",
line 661, in _validate_path
qualified_policies = _finish_policy_processing(
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/validate.py",
line 741, in _finish_policy_processing
qualified_policies = frozenset(_enum_policies())
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/validate.py",
line 732, in _enum_policies
user_domain_policy_id=next(
File
"/usr/local/lib/python3.9/site-packages/pyhanko_certvalidator/validate.py",
line 735, in
if ancestor.parent.valid_policy == 'any_policy'
AttributeError: 'PolicyTreeRoot' object has no attribute 'valid_policy'
Is it something related to the new python version??
Thanks in advance
Fernando

Can you open a new issue for this, and post your package versions for pyhanko-certvalidator and pyHanko there? The former has seen some changes recently. See the release notes for pyHanko 0.6.0 for more information on how this affects existing pyHanko code. :)
I run CI tests against Python 3.7, 3.8 and 3.9 for both of these packages, so the change in Python versions is probably not the root cause here.


Adding filtering

Adds ability to filter by date range and status. One note, I was getting zero results nationwide for recalls in short time ranges (i.e. last 7 Days as show in mock), so opted to go 6 months, 1 year, and 3 years as options. This can be changed, but seemed like a better experience than consistently 0 results for 7 days.
Closes: https://github.com/esridc/openfda/issues/49
Closes: https://github.com/esridc/openfda/issues/48

Agreed - if the data was on a more live cycle then this might make more sense - but they are only updating once or month or so....if that.


Split generated code into separate files

The output for the code generator can get quite large, which also makes it harder to find things in.
I think it would be nicer to write to separate files for the different GraphQL types.

Any news on this?

In case anyone wanted to see this in action:
https://github.com/Avicus/graphql_java_gen

Why didn't you open a PR?

Implemented in https://github.com/Shopify/graphql_java_gen/pull/35


parse timestamp

not familiar w streaming data but i believe the lossless values are in seconds? someone in the comments made a good point that if you have the timestamp as 15:10 you'll have to calculate that manually. it would be cool to parse 15:10 as 15 * 60 + 10 seconds.
a little regex to get someone started: ^[0-5][0-9]?\d:[0-5][0-9]\d$

const timeString = "1:33:25";
const seconds = +(timeString.split(':').reduce((acc,time) => (60 * acc) + +time));

With checks:
function parse(value: string): number | undefined {
  // if there are no colons, assume its seconds
  if(/^\d+$/.test(value)) return parseInt(value, 0);

  // Supports HH:MM:SS, MM:SS, SS
  // If it's not in the format, return undefined
  if (!/^([0-5]?[0-9]:){0,2}[0-5][0-9]$/.test(value)) return undefined;

  return value
    .split(":")
    .reduce((acc, cur) => (acc = acc * 60 + parseInt(cur, 0)), 0);
}


Fixed Migrations for Default Currency

I want to merge this change because the "default currency" is hard coded in the migrations which will fail if one want to migrate with a different currency which is described here:
https://github.com/mirumee/saleor/issues/3235
Pull Request Checklist

[ ] Privileged views and APIs are guarded by proper permission checks.
[ ] All visible strings are translated with proper context.
[ ] All data-formatting is locale-aware (dates, numbers, and so on).
[ ] Database queries are optimized and the number of queries is constant.
[ ] Database migration files are up to date.
[ ] The changes are tested.
[ ] The code is documented (docstrings, project documentation).
[ ] GraphQL schema and type definitions are up to date.
[ ] Changes are mentioned in the changelog.

@bykof Would you mind updating the CHANGELOG.md file?

@maarcingebala done

Thanks! Congrats on your first contribution to Saleor! 🎉


Error -32 EPIPE broken pipe

Getting such an error after first request
warning : [Microsoft.AspNet.Server.Kestrel] Connection processing ended abnormally
Microsoft.AspNet.Server.Kestrel.Networking.UvException: Error -32 EPIPE broken pipe
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.Microsoft.AspNet.Server.Kestrel.Http.ISocketOutput.Write(ArraySegment1 buffer, Boolean immediate)
at Microsoft.AspNet.Server.Kestrel.Http.Frame.WriteChunkedResponseSuffix()
at Microsoft.AspNet.Server.Kestrel.Http.Frame.ProduceEnd(Exception ex)
at Microsoft.AspNet.Server.Kestrel.Http.Frame.d__77.MoveNext()
running on OS X
1.0.0-beta8          coreclr x64          darwin

@dopare Does this still repro with RC1/RC2 bits?   Also, what client are you using?

@muratg cannot reproduce on RC1, thanks, closing the issue.

Thanks @dopare!


Automator: update<EMAIL_ADDRESS>in<EMAIL_ADDRESS>

Generated by Automator - 2023-08-24T02:16:20+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-basic-arm64_istio_release-1.17
06fe82fa08d533faa7791aa5cfae9c5493d695c3
link
true
/test integ-basic-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-basic-arm64


[FEATURE] Add getErrors and setErrors for fields in formCtx

Add getErrors and setErrors for fields in formCtx:
 <Form
            onSubmit={(values, form) => {
                form.setErrors("field", [])
            alert("Form was submitted with: " + JSON.stringify(values));
        }}>
        ```

This is already implemented:
       <Form
            onSubmit={(values, form) => {
                form.getFieldValue('password')!.setErrors(["This is an error"]);
            alert("Form was submitted with: " + JSON.stringify(values));
        }}>


Add Hampton Roads Planning District Commission

Add Flash Briefing for the Hampton Roads Planning District Commission
https://www.hrpdcva.gov/events

Completed by @alanhagerman


OpenSeadragon moving images unexpectedly with angles over 90 degreees

I'm using OpenSeadragon in a project where we are trying to align images with each other. The images have arbitrary rotations, and I've noticed a problem whenever we have images over 90 degrees. The docs state that any angle outside the 0 to 90 range is mapped back into that, but is there a reason for this? When I try to align two images (which involves finding a translation, rotation, and scale) where one of them is rotated more than 90 degrees, the image shifts unexpectedly. The amount it shifts seems to be related to the aspect ratio of the image (square images don't seem to exhibit this behaviour). Unfortunately I can't seem to easily recreate the behavior, it only seems to happen on a few samples in the site I'm creating (All my artificial samples that I create don't seem to trigger it).
Is there a way to access the position more directly in OpenSeadragon? I know it rotates around the centre of the image, but can I get it to rotate around the top left corner always? It would simplify the maths a lot for me, and I think it would fix this issue. Otherwise, is there a formula or something that I can use to counteract this shift when rotating by more than 90 degrees?

@avandecreme @nelsonn3c Any thoughts?
@peterreeves Thank you for reporting this! I'm actually not sure the best way to proceed, but isolating what causes it does seem like the next step; I'm sorry to hear you're having trouble doing that. As for where this is happening, you might look in TiledImage.drawTiles (for calls to getRotation).
Are you doing anything else that might complicate the drawing, like using opacity or clipping or drawing with PNG tiles?
One thing to try (to isolate the issue) might be to set up an animation that slowly rotates through all of the rotations to see if anything pops out.

hum... It's awkward. When possible, please try to better reproduce your study case @peterreeves


core: Support path expansion (#87)

std::fs::canonicalize does most of the work, but it doesn't expand ~. std::env::home_dir is deprecated. So I added the dirs dependency, which is relatively light and only has one dependency.
Features:

. and .. are resolved
~ is resolved to the current users home directory (should work on all major OSs; see the documentation for dirs::home_dir)
symbolic links are resolved (see std::fs::canonicalize)

Limitations:

expansion of other users home directories is not supported (~user/)
expansion only works for files which exist on the filesystem (see std::fs::canonicalize)

Closes #87

Thanks!


Tools instead of deprecated functions

First of all, thank you for your clarification and your kind words!
Regarding your code review, I've applied all your suggestions locally 👍, and will push them later (see rest of the comment)
Now, regarding this:

Our primary branch for development at the moment is api-v2. We plan to merge this into main once v2 is complete (we keep main around because ansari.chat uses main, but beta.ansari.chat for v2 until it is ready).

Great, so I'll close this pull request and create another one for this target branch: "api-v2"

By using tools, I meant the OpenAI tools. https://platform.openai.com/docs/guides/function-calling. Note that this also requires passing back tool results and maintaining the ideas.

Ok, I'll see what other code modifications I can do based on the hyperlink you mentioned, and then push all my local changes to the api-v2 branch ISA 🙌


[US2703] support list snapshots capability in ISTGT

Changes :
Adding support for the command to fetch snapshot list from ISTGT controller.
Command :
istgtcontrol snaplist volume_name <wait time>
ISTGT will wait for 30 seconds to get a response from replica for snaplist command. One can specify the timeout (in seconds) in ISTGTCONTROL command. (istgtcontrol snaplist vol1 60 // wait for 60 seconds).
This command will list snapshot and its properties. Snapshot details will be fetched from all the replicas. The output will be in JSON format as listed below:
{
    “snapshot list” :  [
        {    //replica1 snapshot
            “replica” : REPLICA GUID,
            “snapshot”: [
                {
                    “name” : “snapshot1 name”,
                    “properties”  : {
                        “prop1”:”val1”,
                        “prop2” : “val2”,
                        “prop3” : “val3”
                    },
                },
                {
                    “name” : “snapshot2 name”,
                    “properties”  : {
                        “prop1”:”val1”,
                        “prop2” : “val2”,
                        “prop3” : “val3”
                    },
                },
            ]
        },
        {replica2 snapshot},
        {replica3 snapshot}
    ]
}

Sample output:
{
  "snapshot list": [
    {
      "replica": "13043472030227184458",
      "snapshot": [
        {
          "name": "test_snap",
          "properties": {
            "io.openebs:targetip": "<IP_ADDRESS>:6060",
            "refcompressratio": "100",
            "logicalreferenced": "6144",
            "compressratio": "100",
            "used": "6144",
            "available": "5057968128",
            "referenced": "6144",
            "creation": "1542384844",
            "createtxg": "59",
            "refquota": "0",
            "refreservation": "0",
            "guid": "16536607678276942459",
            "unique": "6144",
            "objsetid": "91",
            "userrefs": "0",
            "defer_destroy": "0",
            "written": "6144",
            "type": "3",
            "useraccounting": "0",
            "volsize": "104857600",
            "volblocksize": "8192"
          }
        }
      ]
    }
  ]
}

fixes: https://github.com/openebs/openebs/issues/2936
Signed-off-by: mayank<EMAIL_ADDRESS>

@mynktl please look at my last two comments..
@pawanpraka1 can you please review this PR?

@pawanpraka1 @vishnuitta I have addressed all the comments and also added test case. PTAL.

Closing this since not needed.


how to add poastcal code into nonce

i got error like
Postal code is required for nonces used by merchants in IN and i want to add postal code by manually using php api .
Can you please help us ?
Thanks

Hey @ipehimanshu, to add the postal code for nonce generation, you'll need to do it in the In-App Payments SDK or Square Payment Form; it cannot be added with one of the backend Connect APIs.
I'm closing this, feel free to reopen if you need further info.


Can't compile devel with CppADCodeGen

Hi,
If I try to compile the current devel branch on 16.04 with clang (both 3.8 & 6.0), CppAD 20180000.0 and CppADCodeGen 2.3.0, I get the following errors:
cd /local/users/gsaurel/stack-of-tasks/pinocchio/build/unittest && /usr/bin/clang++-6.0   -DPINOCCHIO_URDFDOM_TYPEDEF_SHARED_PTR -DPINOCCHIO_WITH_CPPADCG_SUPPORT -DPINOCCHIO_WITH_CPPAD_SUPPORT -DPINOCCHIO_WITH_HPP_FCL -DPINOCCHIO_WITH_URDFDOM -I/local/users/gsaurel/stack-of-tasks/pinocchio/build -I/local/users/gsaurel/stack-of-tasks/pinocchio/build/include -I/local/users/gsaurel/stack-of-tasks/pinocchio/include -isystem /usr/include/eigen3 -isystem /opt/openrobots/include -isystem /usr/include/assimp -I/local/users/gsaurel/stack-of-tasks/pinocchio/unittest  -pedantic -Wno-long-long -Wall -Wextra -Wcast-align -Wcast-qual -Wformat -Wwrite-strings -Wconversion     -DBOOST_TEST_DYN_LINK -DBOOST_TEST_MODULE=kinematicsTest -DFCL_USE_ASSIMP_UNIFIED_HEADER_NAMES -DFCL_HAVE_OCTOMAP= -o CMakeFiles/kinematics.dir/kinematics.cpp.o -c /local/users/gsaurel/stack-of-tasks/pinocchio/unittest/kinematics.cpp
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/unittest/aba-derivatives.cpp:5:
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/multibody/model.hpp:9:
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/spatial/fwd.hpp:9:
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/fwd.hpp:17:
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/math/cppad.hpp:18:
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/math/cppadcg.hpp:9:
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: error: explicit specialization of non-template struct 'ScalarBinaryOpTraits'
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
       ^                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:177:8: error: too many template arguments for class template 'ScalarBinaryOpTraits'
struct ScalarBinaryOpTraits<Base, CppAD::AD<CppAD::cg::CG<Base> >, BinOp> {
       ^                                                           ~~~~~~
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: note: template is declared here
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
       ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:182:8: error: too many template arguments for class template 'ScalarBinaryOpTraits'
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, CppAD::cg::CG<Base>, BinOp> {
       ^                                                                          ~~~~~~
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: note: template is declared here
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
       ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:187:8: error: too many template arguments for class template 'ScalarBinaryOpTraits'
struct ScalarBinaryOpTraits<CppAD::cg::CG<Base>, CppAD::AD<CppAD::cg::CG<Base> >, BinOp> {
       ^                                                                          ~~~~~~
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: note: template is declared here
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
       ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:197:8: error: too many template arguments for class template 'ScalarBinaryOpTraits'
struct ScalarBinaryOpTraits<CppAD::cg::CG<Base>, Base, BinOp> {
       ^                                               ~~~~~~
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: note: template is declared here
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
       ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:202:8: error: too many template arguments for class template 'ScalarBinaryOpTraits'
struct ScalarBinaryOpTraits<Base, CppAD::cg::CG<Base>, BinOp> {
       ^                                               ~~~~~~
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: note: template is declared here
struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
       ^

And with GCC 5.4.0:
cd /local/users/gsaurel/stack-of-tasks/pinocchio/build/unittest && /usr/bin/ccache  g++  -DPINOCCHIO_URDFDOM_TYPEDEF_SHARED_PTR -DPINOCCHIO_WITH_CPPADCG_SUPPORT -DPINOCCHIO_WITH_CPPAD_SUPPORT -DPINOCCHIO_WITH_HPP_FCL -DPINOCCHIO_WITH_URDFDOM -I/local/users/gsaurel/stack-of-tasks/pinocchio/build -I/local/users/gsaurel/stack-of-tasks/pinocchio/build/include -I/local/users/gsaurel/stack-of-tasks/pinocchio/include -isystem /usr/include/eigen3 -isystem /opt/openrobots/include -isystem /usr/include/assimp -I/local/users/gsaurel/stack-of-tasks/pinocchio/unittest  -pedantic -Wno-long-long -Wall -Wextra -Wcast-align -Wcast-qual -Wformat -Wwrite-strings -Wconversion     -DBOOST_TEST_DYN_LINK -DBOOST_TEST_MODULE=cppadcg_algoTest -DFCL_USE_ASSIMP_UNIFIED_HEADER_NAMES -DFCL_HAVE_OCTOMAP= -std=gnu++11 -o CMakeFiles/cppadcg-algo.dir/cppadcg-algo.cpp.o -c /local/users/gsaurel/stack-of-tasks/pinocchio/unittest/cppadcg-algo.cpp
In file included from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/math/cppadcg.hpp:9:0,
                 from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/math/cppad.hpp:18,
                 from /local/users/gsaurel/stack-of-tasks/pinocchio/build/include/pinocchio/fwd.hpp:17,
                 from /local/users/gsaurel/stack-of-tasks/pinocchio/unittest/cppadcg-algo.cpp:7:
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:8: error: ‘ScalarBinaryOpTraits’ is not a class template
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
        ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:177:73: error: wrong number of template arguments (3, should be 2)
 struct ScalarBinaryOpTraits<Base, CppAD::AD<CppAD::cg::CG<Base> >, BinOp> {
                                                                         ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:73: note: provided for ‘template<class Base, class BinOp> struct Eigen::ScalarBinaryOpTraits’
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
                                                                         ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:182:88: error: wrong number of template arguments (3, should be 2)
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, CppAD::cg::CG<Base>, BinOp> {
                                                                                ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:73: note: provided for ‘template<class Base, class BinOp> struct Eigen::ScalarBinaryOpTraits’
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
                                                                         ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:187:88: error: wrong number of template arguments (3, should be 2)
 struct ScalarBinaryOpTraits<CppAD::cg::CG<Base>, CppAD::AD<CppAD::cg::CG<Base> >, BinOp> {
                                                                                ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:73: note: provided for ‘template<class Base, class BinOp> struct Eigen::ScalarBinaryOpTraits’
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
                                                                         ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:197:61: error: wrong number of template arguments (3, should be 2)
 struct ScalarBinaryOpTraits<CppAD::cg::CG<Base>, Base, BinOp> {
                                                             ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:73: note: provided for ‘template<class Base, class BinOp> struct Eigen::ScalarBinaryOpTraits’
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
                                                                         ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:202:61: error: wrong number of template arguments (3, should be 2)
 struct ScalarBinaryOpTraits<Base, CppAD::cg::CG<Base>, BinOp> {
                                                             ^
/opt/openrobots/include/cppad/cg/support/cppadcg_eigen.hpp:172:73: note: provided for ‘template<class Base, class BinOp> struct Eigen::ScalarBinaryOpTraits’
 struct ScalarBinaryOpTraits<CppAD::AD<CppAD::cg::CG<Base> >, Base, BinOp> {
                                                                         ^

I think your problem is coming from your version of Eigen. Can you check with Eigen 3.3.5, which is working fine for me?

I was having issues with 3.2.92 (16.04 default version).
With 3.3.5, I can confirm that I can (almost, cf #618) build Pinocchio, and that tests regarding CppADCodeGen are passing.

The bug is then coming from the CppAD itself.
I will propose a fix there.
I close this issue. Feel free to re-open if needed.

Yes, the version 2016 is too old.
I have made some modifications inside CppAD and CppADCodeGen that I have been merged by their maintainers.
We should use these specific versions to make our framework working well.

In this case, we should use ADD_OPTIONAL_DEPENDENCY( "cppad >= 20180000")


add requestTriggerHotKey

Great works! Thanks for your completing this function and add an example for it.
To make README page clear, I will only leave the minimum example. So I will move the example into docs later :D


Q4 deep dive

We need fast (& high quality) 4 bit quantisation.
You can see the kernel I have written here: https://github.com/FL33TW00D/wgpu-mm/blob/master/shaders/qgemm/qgemm_int4.wgsl
This is very naive, and also very slow.

References for Q4K:

https://github.com/ggerganov/llama.cpp/issues/1240
https://github.com/ggerganov/ggml/blob/cf1acc512432a969cee18947330fd26df67fb456/src/ggml-metal.metal#L5066-L5098


Single model for customer and address

IBM Cloud toolchain: Delivery Pipeline deployed customers to nyu-devops, including commit 5e97cd6c9c2c3b089c27ce0be8c2ea5e36c43d91


Bug: Optimizer rule 'common_sub_expression_eliminate' failed

Description
I'm hitting this error as part of a dbt run in GlareDB Cloud
Database Error in model nyc_sales_jan_2019_feb_2019 (models/demo/nyc_sales_jan_2019_feb_2019.sql)
  Optimizer rule 'common_sub_expression_eliminate' failed
  caused by
  Schema error

Steps to reproduce:

Create initial table:

 CREATE TABLE IF NOT EXISTS dbt_nyc_sales AS
    SELECT * FROM
read_parquet('https://github.com/GlareDB/tutorial_data/raw/main/nyc_sales/sale_year=2019/sale_month=1/f55363e2587849bcb25c057be706c69d-0.parquet')"

Create secondary model selecting from that table:

create table staging_nyc_sales_with_prehook
as
select
    borough,
    neighborhood,
    building_class_category,
    residential_units,
    commercial_units,
    total_units,
    land_square_feet,
    gross_square_feet,
    year_built,
    sale_price,
    sale_date,
    latitude,
    longitude,
    bin,
    bbl
from dbt_nyc_sales

Create a table unioning the previous table with another parquet file:

  create  table "rough_glitter"."public"."nyc_sales_jan_2019_feb_2019__dbt_tmp"
    as
  (
SELECT * FROM "rough_glitter"."public"."staging_nyc_sales_with_prehook"
UNION
SELECT
    borough,
    neighborhood,
    building_class_category,
    residential_units,
    commercial_units,
    total_units,
    land_square_feet,
    gross_square_feet,
    year_built,
    sale_price,
    sale_date,
    latitude,
    longitude,
    bin,
    bbl
FROM read_parquet('https://github.com/GlareDB/tutorial_data/raw/main/nyc_sales/sale_year=2019/sale_month=2/f55363e2587849bcb25c057be706c69d-0.parquet')
  );

Oddly enough, in that last query, unioning the SELECT * FROM "rough_glitter"."public"."staging_nyc_sales_with_prehook" works fine. It also works fine if you create that as a view instead of a table.

Should have followed up on this:

does this still happen?
is there a workaround?
Schema error makes me think that somehow the types in delta as we write them, or the case of the field names has changed. is there a case difference issue in our table vs the upstream type?


Improve SDK integration

Closes #3
Closes #5
Closes #112
Closes #114
Closes #115
Steps:

[x] implementation
[x] double export to export util functions
[x] tests
[x] types
[x] docs
[x] benchmarks

In this PR we are:

removing the filtering with allowedStatusCodes in favour of using a function
adding more functions to configure the extracting of metadata to attach to the sentry event
removing the onErrorFactory in favour of register your own error handler manually
adding a hook to customise the error response in the default error handler
exporting a /utils path to expose out internal functions to handle event metadata

I think I will start to introduce support of the Sentry Performance API in this PR. The code is so intertwined with the work done so far that it makes sense to give it a go now.

Can we schedule a pair review tomorrow since it's a bit old and it'll be easier :)

Can we schedule a pair review tomorrow since it's a bit old and it'll be easier :)

:tada: This PR is included in version 5.0.0-next.1 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 5.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


fixing sorting of ports in bundle routing

Currently, both input and output ports are sorted independently. This causes the logical connections to get jumbled if the input and output ports do not actually have consistent ordering in the cartesian space. This should be fixed somehow, either by throwing an error or sorting only by one set or the other, as proposed in this MR.

thank you Troy ^^


Unable to Import Existing Records Into Algolia

LOCATION=us-central1
PROJECT_ID=nexiev
ALGOLIA_APP_ID=
ALGOLIA_API_KEY=
ALGOLIA_INDEX_NAME=ev
COLLECTION_PATH=ev
GOOGLE_APPLICATION_CREDENTIALS=nestra-firebase-adminsdk-jufxu-6f8cfa91bb.json
npx firestore-algolia-search
I ran the above command in Powershell. But I got the following error and existing records were not imported.
WARNING: The backfill process will index your entire collection which will impact your Search Operation Quota.  Please visit https://www.algolia.com/doc/faq/accounts-billing/how-algolia-count-records-and-operation/ for more details.  Do you want to continue? (y/N): y
{"severity":"ERROR","message":"Error when performing Algolia index TypeError: Cannot read property 'split' of undefined\n    at retrieveDataFromFirestore (C:\Users\Afzal\AppData\Roaming\npm-cache\_npx\7620\node_modules\firestore-algolia-search\lib\import\index.js:72:65)\n    at C:\Users\Afzal\AppData\Roaming\npm-cache\_npx\7620\node_modules\firestore-algolia-search\lib\import\index.js:80:9\n    at Interface._onLine (readline.js:327:5)\n    at Interface._line (readline.js:658:8)\n    at Interface._ttyWrite (readline.js:999:14)\n    at ReadStream.onkeypress (readline.js:205:10)\n    at ReadStream.emit (events.js:315:20)\n    at emitKeys (internal/readline/utils.js:335:14)\n    at emitKeys.next ()\n    at ReadStream.onData (readline.js:1133:36)"}

Hello, having the same issue with WSL (ubuntu) , any luck on this @imakevirtual ?

There's no stack trace, but do you have collectionFields set? https://github.com/algolia/firestore-algolia-search/blob/dc264bec3ba2b2f9b8b7263a96686707d42928de/functions/src/import/index.ts#L80

The way I ran the command was wrong. What I did to resolve the problem is,

Move the service-account.json file to Desktop.
Created a file named Algolia.sh and paste the command inside of it.
After that, I ran that file through Git Bash from Desktop.

Hello, having the same issue with WSL (ubuntu) , any luck on this @imakevirtual ?

@Lucas-L
The way I ran the command was wrong. What I did to resolve the problem is,

Move the service-account.json file to Desktop.
Created a file named Algolia.sh and paste the command inside of it.
After that, I ran Algolia.sh file through Git Bash Terminal from Desktop.

Same issue on M1 Mac
Solution doesn't work for me

Okay it works when used in a single line


Administrator can add a new Action

An Administrator should be able to click an  button to add a new Action.
Adding an Action
They will see an “Add Action” page which requests the following information:

Action name (text field)

Action description (multi-line text field)

Action date (three drop-downs: day (dd), month (mm), and year (yyyy))

User must provide action year as minimum

If user does not provide value for dd/mm, default values will be 01/01

Action type (drop-down)

Location of publication - drop down displaying list of geographies

Keywords - the user may enter 1 or more keywords that will be associated with the Action.

Adding Documents to Action
The user should be able to add one or more documents to the Action.
Proposed Add Action page

Proposed Add Document section

Policy team notes

Location of publication

column 1 = drop-down of supranational/national/sub-national

column 2 if national selected in column 1 = drop-down of country names

column 2 if sub-national selected in column 1 = drop-down of country names

column 3 when country selected in sub-national column 2 = drop-down of ‘highest’ (i.e. state/regional) subnational jurisdiction (e.g. states in USA, provinces in China, départements in France, devolved administrations in UK etc)

column 4 when subnational jurisdiction selected in column 3 = drop-down of ‘next-highest’ (i.e. intermediate) subnational jurisdiction (e.g. counties in US states, counties in UK devolved administrations, prefectures in China etc)

column 5 when subnational jurisdiction selected in column 4 = drop-down of ‘next-highest (i.e. intermediate) subnational jurisdiction (e.g. counties in China etc)

column 6 when subnational jurisdiction selected in column 5= drop-down of next-highest (i.e. municipal) subnational jurisdiction (e.g. townships in China etc)

**Note: **the number of columns required will vary by country, depending on its hierarchy of subnational jurisdictions - e.g. China requires all 6, where USA requires 5).
As illustrated below:

Publishing authority

column 1 = drop-down of international organisation/negotiating coalition/national government/local government/NGO/corporation

column 2 if international organisation selected in column 1= free text box for organisation name with autocomplete according to recognised entities already stored in database

column 2 if negotiating coalition selected in column 1 = free text box for coalition name with autocomplete according to recognised entities already stored in database

column 2 if national government selected in column 1 = free text box for central government/government department/ministry name with autocomplete according to recognised entities already stored in database

column 2 if NGO selected in column 1 = free text box for NGO name with autocomplete according to recognised entities already stored in database

column 2 if corporation selected in column 1 = free text box for corporation name with autocomplete according to recognised entities already stored in database

As illustrated below:

Instrument

Instrument values:

**All instrument and sub-instrument values **CCLWCPD

Barrier removal

Barrier removal: grid access and priority for renewables

Barrier removal: net metering

Barrier removal: removal of fossil fuel subsidies

Barrier removal: removal of split incentives (landlord tenant problem)

Building codes

Capacity-building - general

Climate fund

Climate strategy

Climate strategy: coordinating body for climate strategy

Climate strategy: formal and legally binding climate strategy

Climate strategy: political and non-binding climate strategy

Creating bodies and institutions

Designing processes

Developing plans and strategies

Disclosure obligations

Economic instruments

Economic instruments: direct investment

Economic instruments: fiscal or financial incentives

Economic instruments: market-based instruments

Ecosystem restoration and nature-based solutions

Education and training

Governance and planning

Incentives

Information and education

Information and education: advice or aid in implementation

Information and education: information provision

Information and education: performance label

Information and education: professional training and qualification

International cooperation

Insurance

Knowledge generation

Knowledge sharing and dissemination

Monitoring and evaluation

Monitoring, reporting, verification

Multi-level governance

Policy support

Policy support: institutional creation

Policy support: strategic planning

Provision of climate finance

Public goods - early warning systems

Public goods - other

Regulation

Regulatory instruments

Regulatory instruments: codes and standards

Regulatory instruments: monitoring

Regulatory instruments: obligation schemes

Regulatory instruments: other mandatory requirements

Research & development and deployment (RD&D)

Research & development and deployment (RD&D): demonstration project

Research & development and deployment (RD&D): research programme

Research and development

Social safety nets

Standards and obligations

Subsidies

Target

Target: energy efficiency target

Target: GHG reduction target

Target: renewable energy target

Taxes

Voluntary approaches

Voluntary approaches: negotiated agreements (public-private sector)

Voluntary approaches: public voluntary schemes

Voluntary approaches: unilateral commitments (private sector)

Zoning and spatial planning

Sector

Sector values:

| All sectors & sub-sectors CCLW CPD |
| ---- |
| Adaptation |
| Agriculture |
| Agriculture and forestry: agricultural CH4 |
| Agriculture and forestry: agricultural CO2 |
| Agriculture and forestry: agricultural N2O |
| Agriculture and forestry: forestry |
| Buildings |
| Buildings: appliances |
| Buildings: construction |
| Buildings: heating and cooling |
| Buildings: hot water and cooking |
| Coastal zones |
| Disaster Risk Management |
| Electricity and heat: CCS |
| Electricity and heat: coal |
| Electricity and heat: gas |
| Electricity and heat: nuclear |
| Electricity and heat: oil |
| Electricity and heat: renewables |
| Energy |
| Environment |
| Finance |
| Health |
| Industry |
| Industry: fluorinated gases |
| Industry: fossil fuel exploration and production |
| Industry: industrial energy |
| Industry: industrial N2O |
| Industry: industrial processes CO2 |
| Industry: negative emissions |
| Industry: waste CH4 |
| LULUCF |
| Residential and commercial |
| Rural |
| Social development |
| Tourism |
| Transport |
| Transport: air |
| Transport: heavy-duty vehicles |
| Transport: light-duty: vehicles |
| Transport: low-emissions mobility |
| Transport: rail |
| Transport: shipping |
| Urban |
| Waste |
| Water |

Theme

Based on CCLW & CPD, these are: adaptation, DRM, loss and damage, mitigation, DRM/DRR, air pollution, economic development, energy access, energy security, food security, land use, water

Further themes could be added from other sources, such as UNFCCC - transparency, finance...

Discussed with @danny-waite-cpr and @marcusodavies:

Publication location - this will be a country level geography lookup
Publication authority - this will be out of scope for the alpha release.


docs: add readme English and Chinese version

Signed-off-by: songtao98<EMAIL_ADDRESS>To add simple readme file introduction in both English and Chinese version.
More details should be added continuously.

/lgtm
/approve


isparta no longer compatible

Looks like current isparta (3.5.x) is no longer compatible with the coverage gulp task.
Travis was thus failing
I forced v3.0.x
It's now passing
I did not investigate further though...
HTH

Fixed in https://github.com/babel/generator-babel-boilerplate/pull/289
Will release later today!


POC: Request Batching

Description
In many places we have lists where each item needs to make some individual requests for more details. For example, the FlowRunListItem makes an individual request for the flow, work pool, work queue, and deployment. Thats 4 additional GET requests per flow run.
We compensate for this already in two ways

Subscriptions make sure we're not fetching duplicate data. If all the flow runs are from the same flow only 1 flow GET request will be made
Visibility observers ensure that we're not fetching data the user can't even see

Proposal
Batching is the concept of taking multiple individual actions and combining them into a single action. By batching multiple individual GET requests into a single request (using the filter endpoint) we can be much more efficient while also worrying less about how many resources we are requesting.
Implementation
This PR introduces a new service called Batcher (don't love the name, open to suggestions). You can give it a callback for how to handle a batch and then call batcher.batch(value). It will automatically queue batch calls and then flush the queue into the provided callback.
The callback can return either a Map that is keyed by the value or a lookup method.
Working Example
This PR includes an example of using the batcher to batch getFlow requests.
Without the batcher

With the batcher

Additional Notes
This can be used for more than api requests. We could use it for any repetitive task that can happen many times quickly. For example, when we implement websockets we might use this to batch reacting to individual events to we don't process events to quickly.

I'll avoid a full review for now (unless you're looking for one) but this feature is great

I'd love a full review on the batcher itself and the syntax for implementing it.

@znicholasbrown I made some updates based on your comments.


Nominating Patrick Hindmarsh for commit access and Covid Green TSC

I'd like to nominate Patrick Hindmarsh (@phindmarsh) from the New Zealand Health Ministry for commit access and for the covid green TSC ... /cc @colmharte @aspiringarc
New Zealand is leveraging components of covid green in their application.

I'm happy to support Patricks nomination

LGTM

Org invite sent!


Deploying Jar File On Azure Container(Linux-Based)

Been looking through azure documentations of deployment. Might still be confused about certain concepts of container instances.
But is there a specific guide on how I can deploy jar files in an Azure container that is linux based?
I do know that there is a similar documentation about uploading jar file to Azure App Services but I couldnt find something similar to uploading jar file to Azure Kubernetes Container (Linux Based). Most documentations I found are about uploading Spring Boot Applications to containers.
my findings:
https://docs.microsoft.com/en-in/java/azure/spring-framework/deploy-spring-boot-java-app-on-kubernetes?view=azure-java-stable
https://docs.microsoft.com/en-in/java/azure/spring-framework/deploy-spring-boot-java-app-on-linux?view=azure-java-stable

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb1bc58a-332c-ccc9-edff-47b175858a85
Version Independent ID: 57c881ee-265d-0c63-e39e-afb40a241bd3
Content: Deploy a Spring Boot App on Kubernetes in Azure Kubernetes Service
Content Source: docs-ref-conceptual/spring-framework/deploy-spring-boot-java-app-on-kubernetes.md
Service: multiple
GitHub Login: @rmcmurray
Microsoft Alias: robmcm

@rmcmurray Did you get a chance to take a look at this customer's question.
@iainfoulds Is this something you can share your insights on? Thanks :)

@DarrenCzen I have assigned the issue to the content author to investigate further and update the document as appropriate.

@DarrenCzen I don't think we have anything specific to this scenario, but I can add the suggestion to our docs backlog.
At a high level, you'd build a container image that contains your Java application, and that container image is then used for deployments into a Kubernetes cluster. It's not necessarily anything AKS-specific. The container image could be stored in a container registry such as Azure Container Registry (ACR) or Docker Hub. There's not really a provision for taking a generic container image in Kubernetes and then injecting your Java application. That's not really how Kubernetes handles deployments.
You could also use Azure Dev Spaces to build and deploy your Java app.
@Karishma-Tiwari-MSFT With a doc suggestion added to backlog but no ETA on when that may be published, #please-close

@DarrenCzen Thanks for bringing this to our attention. We will now close this issue. If there are further questions regarding this matter, please tag me in a comment. I will reopen it and we will gladly continue the discussion.


Compatibility with version 2.8.x broken?

Now that the message types have changed with the recent commits, we would expect compatibility with version 2.8 to be at least partially broken, right?
We might need to adjust the README, or point people to a compatibility tag/branch that still have an older version.

I haven't tested it with <IP_ADDRESS> again, but when I added the new messages, I was trying to be careful to break as little as possible. :)
The default configuration should still be working on <IP_ADDRESS>. The only thing that doesn't work any more is BMSData, but that isn't published by the mir_driver by default (without commenting in a line) anyway.


fix(graphql): correct searchable instance types

This change adds the OpenSearch instance types allowed by cloudformation.
This reverts the incorrect instance types added in PR https://github.com/aws-amplify/amplify-cli/pull/9834.
This is a short term solution to unblock the customers.
Long term solution is tracked in https://github.com/aws-amplify/amplify-cli/issues/10075

Fixes https://github.com/aws-amplify/amplify-cli/issues/9956

Why not just remove the allowedValues altogether? Just let CFN define those. That way no one has to maintain the list and people that need to run an older version of the CLI are not stuck with an old list.

If we remove the allowed values list and someone misspell an instance type, the error would appear at much later time on push resulting in poor DX. We are thinking about using SDK to retrieve the allowed instance types.
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configuration-api.html#configuration-api-actions-listinstancetypedetails

Codecov Report

Merging #9973 (699db62) into master (2f2229c) will decrease coverage by 0.00%.
The diff coverage is 62.50%.

@@            Coverage Diff             @@
##           master    #9973      +/-   ##
==========================================
- Coverage   54.09%   54.08%   -0.01%
==========================================
  Files         837      837
  Lines       46374    46376       +2
  Branches     9890     9891       +1
==========================================
  Hits        25084    25084
  Misses      19295    19295
- Partials     1995     1997       +2

Impacted Files
Coverage Δ

...rc/graphql-transformer/transform-graphql-schema.ts
15.41% <25.00%> (-0.12%)
:arrow_down:

...chable-transformer/src/cdk/create-cfnParameters.ts
100.00% <100.00%> (ø)

...graphql-elasticsearch-transformer/src/resources.ts
100.00% <100.00%> (ø)

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@sundersc why were all the X.Y.search instance types removed, and only the X.Y.elasticsearch instance types kept?
I believe .elasticsearch instance types are deprecated/outdated, and .search instances are now the preferred ones. E.g. see this document describing the OpenSearch summary of changes that occurred on September 8th 2021:

Instance types in Amazon OpenSearch Service are now in the format <type>.<size>.search—for example, m6g.large.search rather than m6g.large.elasticsearch

That document goes on to say that

"Existing domains will start automatically referring to the new instance types within the API and in the Billing and Cost Management console"

BUT it seems that if this is going to happen, it hasn't yet. .elasticsearch instance types appear to result in outdated deployments which behave different from the latest ones, and which the AWS OpenSearch console prompts you to upgrade.


Undocumented behaviour: Configuration secondary variants inherit the primary's attributes

Current Behaviour
Configuration secondary variants inherit the primary's attributes. This behaviour is unexplained and undocumented.
Expected Behaviour
Either:

The behaviour is a bug, and Configuration secondary variants should not inherit attributes.

The behaviour is intentional, and should be documented.
(Note: It would be much appreciated if the current behaviour should not just be written down, but also explained to give context. For example, by answering these questions: Why should attributes be automatically inherited? Why would this be advantageous? In what situations is this relevant? Apparently, the inherited attributes can be overridden - when should this be done, if at all?)

Context
Related: https://github.com/gradle/gradle/issues/28149
Steps to Reproduce
// build.gradle.kts

group = "x.y.z"
version = "1.2.3"

val myFooAttribute = Attribute.of("FooAttribute", String::class.java)
val myFooVariantAttribute = Attribute.of("FooVariantAttribute", String::class.java)

val fooFiles: Configuration by configurations.creating {
  isCanBeDeclared = false
  isCanBeResolved = false
  isCanBeConsumed = true

  attributes {
    attribute(myFooAttribute, "main")
  }
}

val fooFilesVariant1: ConfigurationVariant by fooFiles.outgoing.variants.creating {
  attributes {
    attribute(myFooVariantAttribute, "variant1")
  }
}

val fooFilesVariant2: ConfigurationVariant by fooFiles.outgoing.variants.creating {
  attributes {
    attribute(myFooAttribute, "secondary")
    attribute(myFooVariantAttribute, "variant2")
  }
}

Run gradle outgoingVariants -q
--------------------------------------------------
Variant fooFiles
--------------------------------------------------

Capabilities
    - x.y.z:proj:1.2.3 (default capability)
Attributes
    - FooAttribute = main

Secondary Variants (*)

    --------------------------------------------------
    Secondary Variant fooFilesVariant1
    --------------------------------------------------

    Attributes
        - FooAttribute        = main
        - FooVariantAttribute = variant1

    --------------------------------------------------
    Secondary Variant fooFilesVariant2
    --------------------------------------------------

    Attributes
        - FooAttribute        = secondary
        - FooVariantAttribute = variant2

(*) Secondary variants are variants created via the Configuration#getOutgoing(): ConfigurationPublications API which also participate in selection, in addition to the configuration itself.

Observations:

fooFiles only has FooAttribute = main, which is expected.
fooFilesVariant1 contains FooAttribute = main. This is unexpected behaviour, because it's not documented.
fooFilesVariant2 overrides FooAttribute to be FooAttribute = secondary. Because of the automatic inheritance, I'm now confused. Is overriding the primary's attributes supposed to be allowed? Is it problematic?

Gradle version
8.7
Build scan URL (optional)
No response
Your Environment (optional)
No response

This is most likely just a documentation issue.
Closing this as a duplicate of:

https://github.com/gradle/gradle/issues/12126

After discussing this again, this could be considered a separate documentation issue.
The role of secondary variants is hinted here
And this is a very specific part of cross-project publication that is beyond the general scope of understanding variant and attributes. Although it builds on it.


Can we create Visual Graph with this

It will be great to create a visual graph for prompt game logic :)

gpt-3.5 doesn't know how to program with Visual Scripting.


The type or namespace name 'Web' does not exist in the namespace 'MVCGrid' (are you missing an assembly reference?)

I Was configure MVCGrid to my mvc application using the tutorial.
application building successfully,
but when run the application there is a error message show as below.
The type or namespace name 'Web' does not exist in the namespace 'MVCGrid' (are you missing an assembly reference?)
do you have any idea to fix it ?
Thank you.

Hi @iamchamith,
What version of MVC are you using?
Thanks,
Joe


Show git status

Can we rely on an output of git status?
$ touch x
$ renga run touch y
Error: The repository is dirty. Please use the "git" command to clean it.

On branch master
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        x

nothing added to commit but untracked files present (use "git add" to track)

addresses https://github.com/SwissDataScienceCenter/renga/issues/186

@jirikuncar I think in addition to showing the message from git we need to also follow with something like
Once you have added the untracked files, commit them with "git commit"


Feature request (Reports): Coloured bars

Sorry, this is really for the online webapp...I don't know how to make a feature request for this.
Would like to request stacked bars for the summary page with colours matching the pie chart. This would help you to see when you worked on what.

This is not web app repo. Please write to support for feature requests about web app.


Too many requests on signup page

Hi! I just found this project and love it so far! I keep having an issue with the signup page serving a "too many requests" error after 3-4 page loads after startup; simply restarting the container fixes the issue, but then after 3-4 more page loads it happens again.
System information

Instance: https://w.dunkirk.sh/signup
Wakapi version: 2.11.2
Operating system: Docker with the latest tag from ghcr
Database: Postgres hosted locally on the machine

Here's my docker compose; it's super simple and basically the same as the README sample minus the Postgres container.
services:
  wakapi:
    image: ghcr.io/muety/wakapi:latest
    ports:
      - 35555:3000
    restart: always
    environment:
      WAKAPI_DB_TYPE: "postgres"
      WAKAPI_DB_NAME: "usr_wakapi"
      WAKAPI_DB_USER: "usrname"
      WAKAPI_DB_PASSWORD: "xxxx"
      WAKAPI_DB_HOST: "hackclub.app"
      WAKAPI_DB_PORT: "5432"
      ENVIRONMENT: "prod"

Docker container logs:
wakapi-1  | 2024-09-08T18:03:46.329908551Z [INFO ] [request] status=429, method=GET, uri=/signup, duration=47.026µs, bytes=18, addr=<IP_ADDRESS>:41306, user=-

Found the issue lol: security.signup_max_rate / WAKAPI_SIGNUP_MAX_RATE was set to the default of 5/1h. I changed it to 1/1s. I don't think that just loading the page should count against you though.

Hi @kcoderhtml, thanks for bringing this up! I had a look at it and turns out that GET /signup is rate-limited, while POST /signup actually isn't. That's definitely a bug. Will fix it.

Thanks!