Deploy transactions take a long time

Compared to invoke transactions (but also in absolute terms), deploy transactions take a very long time (many seconds).
With the sieve (non-deterministic) consensus protocol, we need to execute all transactions (and wait for their result) before the protocol can move on.  However, to prevent against a malicious leader, the protocol also needs to use timeouts to ensure progress.
This timeout needs to accommodate the duration of deploy transactions (or at least the variation of the duration).  Sizing this timeout conservatively means that faulty leaders will be detected more slowly.
Alternatively, we could split deploy transactions into a hint, which will initiate the deployment, and a final transaction, which will record the deployment on the ledger.

@corecode @muralisrini is this still an issue or can it be closed?

Still an issue.

Moved to Jira https://jira.hyperledger.org/browse/FAB-120 closing


Request: Add Slack app to repo

I was wondering if the Slack app could be added to this project so folks can subscribe to issue/release/comment activity via the Github app in their own Slack workspaces?
Relevant links:
https://slack.github.com/
https://github.com/apps/slack/installations/new

Thanks for the request. I've added it to the Helm meeting agenda so we can discuss.

via the Github app in their own Slack workspaces?

As I understand it, the Slack app needs to be installed into each workspace. We'd need administrative access from the Kubernetes Slack maintainers to allow access there, for example.

Perhaps I may be mis-reading the documentation. It looks as though it's just an install link and users can then subscribe to events from that app. I guess it's just a matter of what permissions we're comfortable delegating to Slack.
https://slack.com/intl/en-ca/help/articles/232289568-GitHub-for-Slack

users from arbitrary workspaces can subscribe to issues/events. I guess it's just a matter of what permissions we're comfortable providing to Slack.

It is this, yes. Currently we have the Github app installed in our Slack workspace, but I am unable to subscribe to Github events for helm/helm because this project doesn't have the Slack App installed, which would be done here: https://github.com/apps/slack/installations/new by a maintainer


shown.bs.collapsible event is not working after an Ajax call

I am doing an AJAX call in jquery to dynamically generate html and binding it to my accordian. Before the ajax call, the shown.bs.collapsible event is triggered. But after ajax call, this event is not being trigger. Please help me.

Please post a JS Bin or other live example which demonstrates the problem, per the issue template.

Bootstrap 3 is no longer being officially developed or supported.
All work has moved onto our next major release, v4. As such, this issue or pull request is being closed as a "won't fix." For additional help and support, we recommend utilizing our community resources. Thanks for your understanding, and see you on the other side of v4!
<3,
@mdo and team


Show errors for assets

Given there is an asset 'span`' that also has a method associated with it in one of two ways:

Directly saved in the backend as a MethodCodeObjectId
Discoverable by the plugin (similar to the 'target' icon)

And that method has error stack frames associated with it.
When I navigate to that span in any way, including:

Through the assets tab
Using the code button
Using a lens
Using an insight link

I should see in the errors tab all of the errors associated with the method associated with this span
implementation details
asset scope should contains the following details:  (just an example)
    span_code_object_id [optional] (more details can be found https://github.com/digma-ai/digma-ui/issues/454)
    method_code_object_id  [optional]
    display_name

Any scope change in the UI should follow these steps:
call assets/display_info API to get asset display information
if the response contains method_code_object_id (related to the current asset)
    set the scope with both span_code_object_id and method_code_object_id
else
    try to discover the method_code_object_id for that span_code_object_id
    if method found:
         set the scope with both span_code_object_id and method_code_object_id found by the discovery
   else
         set the scope with both span_code_object_id with no method_code_object_id
after the scope is changed all tabs should be refreshed according to the scope

**errortab**
if scope.method_code_object_id  is not null:
     fetch errors from the backend by using  scope.method_code_object_id
     if num of errors > 0:
        show errors + red icon in the errors tab
    else:
       no errors empty state

https://www.figma.com/file/MxA95ZHK5NqGX4ptFQljRJ/⭐-Design-System-3.0?type=design&node-id=2452-46463&mode=design&t=Z4NAQFTz8hrIgEEw-11 - red dot design link

Completed in #432


Update the first call to an authentication flow to be named start

There's a 2nd commit that isn't showing up here...I wonder what happened to it. I'd updated all the README/samples/docs.


.repos dependencies are outdated

After cloning this repo and running
vcs import < dependency_repos.repos
I get the following:
=== ./mocap (git) ===
Could not checkout ref 'main': fatal: invalid reference: main
=== ./mocap_msgs (git) ===
Could not checkout ref 'master': fatal: invalid reference: master

Hi @gritelli
Please, check again after merging #8


More effective mission statement

I asked Bing, "Is 'Helping Teams Work on the Right Thing at the Right Time' an effective mission statement?"
It replied:

A mission statement is a concise statement that describes the purpose of an organization. It should be clear, concise, and memorable. The mission statement “Helping Teams Work on the Right Thing at the Right Time” is effective because it is clear and concise. It communicates the purpose of the organization and what it aims to achieve. However, it could be more specific about what kind of teams it is referring to and what “the right thing” means in this context.

I asked, "Is this better, 'Helping education technology teams work on the right thing at the right time to deliver student success.'?"
It replied

Yes, that’s better. It is more specific about the kind of teams it is referring to and what “the right thing” means in this context. It also communicates the purpose of the organization and what it aims to achieve.


Clicking on MenuButton triggers form submit

MenuButton should have type='button' by default. It could accept  button default props (type)

Hi @shahalihridoy , thanks for reporting the issue.
Will fix it in the next release.

Issue was fixed in v2.1.0


Integrate jemalloc

We should integrate jemalloc, especially now that we are working on multithreading support.
https://github.com/jemalloc/jemalloc

Should we not first investigate if its worth it?

Yes, agreed, but I think it will be quite a bit better. There has been a paper written about this after all :) The system malloc is generally quite poor. In general the numbers I heard were around 10~20% on single-core workloads performance improvements, with higher percentages when dealing with NUMA nodes and multiple threads.

I have heard those numbers too but believe them to be coming more from transactional workloads. We shall see.

Ran TPC-H with and without jemalloc on Linux:

Query
System
Jemalloc
Speedup

Q01
0.470
0.469
0%

Q02
0.064
0.057
11%

Q03
0.157
0.155
1%

Q04
0.149
0.164
-8%

Q05
0.180
0.161
11%

Q06
0.055
0.052
4%

Q07
0.299
0.281
6%

Q08
0.190
0.170
11%

Q09
1.688
1.492
13%

Q10
0.320
0.315
1%

Q11
0.026
0.023
15%

Q12
0.188
0.187
0%

Q13
0.315
0.312
1%

Q14
0.074
0.066
12%

Q15
0.116
0.113
2%

Q16
0.091
0.080
14%

Q17
0.245
0.267
-8%

Q18
0.682
0.678
0%

Q19
0.421
0.372
13%

Q20
0.089
0.095
-5%

Q21
0.593
0.634
-6%

Q22
0.063
0.068
-7%

Not too impressive, and many queries got worse as well. I think the difference might be bigger in multi-threaded workloads especially when NUMA is considered. On Windows the system allocator could be worse than on Linux as well, but I'm not sure about that. Either way, not a priority right now.

Closing this for now then


Add minecraft:double_plant to overwritable blocks

Hi,
I'm enjoying Davinci's a lot, but when trying to land aircraft, I'm constantly running into double-tall grass blocks keeping them elevated an extra block when landing, making it hard to climb back inside.
I added minecraft:double_plant to the overwritable_blocks list in the config file, and it fixed it.
I would send a PR, but I don't know Java, and I don't know the correct Blocks.BLOCK-type object to add to the list in the code.
Thank you very much for your work on these mods!

It might also be good to add minecraft:red_flower and minecraft:yellow_flower, as these tend to prevent disassembly in plains.


Problem with image size and dilation rates

ValueError: input tensor's spatial dimensionality not not compatible, please tune the input window sizes:
lambda x: x % modulo == 0)
Getting the above error when reducing the image size to train with Dense Vnet, we require a size reduction as we recieve memory problems when training with the original dimensions of 144,144,144
is there a simple way to turn the dilation off? is this network co,patible with reduced dimensions of the ordr of 40,40,40 voxels
thanks

# Validate input dimension with dilation rates
modulo = 2 ** (len(hyper['dilation_rates']))
assert layer_util.check_spatial_dims(input_tensor, lambda x: x % modulo == 0)

I think the window size must be a multiple of 8.


Upload tag specifications to github

you know what to do....

https://docs.google.com/spreadsheets/d/1tk0NP7dpz1ZtAXIjQ4QUtfNnRx3I957ML1raFGggZac/edit#gid=0
מסמך תגיות ב sheet


DIV-2661: Change how save and close works after CYA

Added handling that checks if a session already exists or not in CCD before deciding to save the draft in draft store
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Tested yesterday on PFE preview and can verify it doesn’t save in draft store if already submitted to CCD

This is tested and working as expected hence going to merge the PR. There is dependent PR https://github.com/hmcts/div-petitioner-frontend/pull/195. This is going to delete by John once the merge successful.

Tested here : https://pr-195-div-pfe-preview.service.core-compute-preview.internal/index
Tested and working as expected. check the attached screen shots for more details. for Session data store i used the following to verify.
http://div-cps-aat.service.core-compute-aat.internal/swagger-ui.html#!/drafts-controller/retrieveDraftUsingGET.
All looking good. Merged the code successfully and in Prod hence marking the ticket to Done.

This has been reverted. The way to get it back in master is to revert the revert commit. Navigate to #55 and click "Revert"


Adding blog with a Hyper v3 post

For the launch of Hyper 3 we want to have a blog on the website to talk about the improvements. This PR uses mdx-js to have an easy way to write the posts.

Closing in favor of #120


Bump version to 0.4.1

Right after opening https://github.com/guidomb/carthage_cache/issues/19 I realised a PR might have been more appropriate.
The only thing left to do is rake release 😄

Yeah I was waiting in case anything else was added but I think is time to cut a new version.


problem when extract

i am sure i have the correct videos path,and have a opencv with video io support,but i still cannot get the extract

I have the same problem wtih #6
but i try all them solve way but noway

FATAL[default]check failed:[video_streanm.isOpened()]cannot openvideo stream "/home/xx/UCF-101/TrampolineJumping/v_TrampolineJumping_g04_c03.avi"for optical flow extaction.
WARN[default]Aborting appliation.Reason:Fatal log at[/home/xx/temporal-segment-networks/lib/dense_flow/src/dense_flow_gpu/cpp:15]

i only get a empty folder

my opencv can output
--   Video I/O:
--     DC1394 1.x:                  NO
--     DC1394 2.x:                  YES (ver 2.2.1)
--     FFMPEG:                      YES
--       codec:                     YES (ver 54.35.0)
--       format:                    YES (ver 54.20.4)
--       util:                      YES (ver 52.3.0)
--       swscale:                   YES (ver 2.1.1)
--       resample:                  NO
--       gentoo-style:              YE

Hi @cyani527 . Have you solved your problem?

yes,i have solved my problem,i find there are different between 32bit and 64 bit when use opencv.

Good to know. Thanks.

@cyani527  hello, I meet the same problem when I run bash scripts/extraction_optical_flow.sh SRC_FOLDER OUT_FOLDER NUM_WORKER ,the image extraction part does not get the images. can you give me some suggest about the image extraction? Thank you .


Error response from wallet: error code -2

Hi and thanks for the gui wallet work.
I've got my daemon for zcashd running:
Splash: zcashd already running...
Using ZCash utilities: zcashd: /home/user/Downloads/zcash/src/zcashd, zcash-cli: /home/user/Downloads/zcash/src/zcash-cli
[main] [Sun Dec 17 16:37:22 CST 2017] [ERROR] Unexpected error:
com.vaklinov.zcashui.ZCashClientCaller$WalletCallException: Error response from wallet: error code: -2
error message:
Safe mode: Your client is out of date and potentially vulnerable to blockchain disruption. Please update to the most recent version of Zcash and use -reindex to ensure integrity of your blockchain state. More info at: https://z.cash/support/security/
I've gone to the z.cash site and grabbed their latest version, but this still happens.
You can see that my zcash-cli and zcashd are all in the directory withthe wallet jar:
zcash-cli
zcash_cli-bitcoin-cli.o
zcashd
zcashd-bitcoind.o
zcash-gtest
ZCashSwingWalletUI.jar
zcash-tx
I also see: zcashd is not runing at the moment or has not started/synchronized 100% - showing splash...
[INFO] Current blockchain synchronization date isTue Nov 15 05:36:59 CST 2016
Do I just have to wait?
If I have to update zcash again do you have a link to the latest git repo?
Also why isn't the daemon running:
user@droid0:~/Downloads/zcash/src$ service zcashd status
● zcashd.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
user@droid0:~/Downloads/zcash/src$
user@droid0:~/Downloads/zcash/src$ ./zcashd -reindex
Error: Cannot obtain a lock on data directory /home/user/.zcash. Zcash is probably already running.
Right now I'm pretty sure issues are related to syncing the blockchain.  From the log:
date isWed Nov 16 05:34:49 CST 2016
[main] [Sun Dec 17 16:53:16 CST 2017] [INFO] Current blockchain synchronization date isFri Nov 18 07:36:05 CST 2016
[main] [Sun Dec 17 17:00:02 CST 2017] [INFO] Current blockchain synchronization date isSun Nov 20 05:11:08 CST 2016
[main] [Sun Dec 17 17:02:07 CST 2017] [INFO] Current blockchain synchronization date isMon Nov 21 09:40:47 CST 2016
[main] [Sun Dec 17 17:02:29 CST 2017] [INFO] Current blockchain synchronization date isMon Nov 21 15:24:17 CST 2016
You can see it advancing every time I try to launch the wallet.
I'll wait a few hours until synced and try again
I'm attempting the upgrade posted by ayleph here:
https://forum.z.cash/t/how-to-update-zcash-client-without-losing-my-wallet/14362/2
to version 1.0.11 with:
~ $ cd zcash
~/zcash $ git fetch && git checkout v1.0.6
~/zcash $ make clean
~/zcash $ ./zcutil/build.sh
some slight path modifications were necessary in my case.
That worked.
There is still an issue though:
Warning: This version will be deprecated at block
Apparently they're fixing this in yet another update.

The main problem described comes from running old/deprecated versions of ZCash. Maybe updating to the very latest one (1.0.13) makes sense. The user guide is:
https://github.com/zcash/zcash/wiki/1.0-User-Guide
You need to make sure you have only one version of zcashd/zcash-cli on your system that is started (manually or automatically by the GUI wallet).


[trino's compatibility issue] date related calculation

Enhancement
there are some difference between trino and starrocks when dealing with date calculation. here are two examples that need to be fixed.

select date_add('day', -1, date(concat(substr('2023-12-10', 1, 7), '-01')));
in StarRocks, when set SQL dialect to trino, the SQL returns: 2023-11-30 00:00:00
in trino, the SQL returns 2023-11-30

select date('2023-12-10')-interval '1' day;
in StarRocks, when set SQL dialect to trino, the SQL returns: 2023-12-09 00:00:00
in trino, the SQL returns 2023-12-09

a simple solution is to use date_trunc or cast as date. the first one can be fixed by adding transform in Trino2SRFunctionCallTransformer and the second one need to be fixed in AstBuilder

I could help finish this, please assign it to me if OK


Unit tests: Keybinding service NPE

I see this when running the unit tests with --debug:
Uncaught (in promise) TypeError: this._storageService.getBoolean is not a function
    at TestKeyboardMapperFactory.setActiveKeyMapping (keymapService.js:141)
    at _getBrowserKeyMapping.then.keyMap (keymapService.js:198)

No test fails, though.

Actually I see a lot more on my Linux box:

@joaomoreno I fixed the error message so now it displays warning as expected. The warning is used to help us troubleshoot when chromium's keymap result conflicts with the keyboard event, which also means it's a bug in chromium's implementation of the keymap API.

Makes sense... but should this really be running during our Unit Test suite?

I removed the code base as currently on Linux and Windows, the error is almost everywhere. Chromium already fixed it so we can have this turned on once they ship.


Use small-vec::VecLike over Vec to prevent std dependency where possible.

I'd love it if I could use these tools in a libstd free environment and currently the use of Vec is hard-coded for the backing storage.
Is there interest from making the storage type be a (static generic parameter that implements) VecLike trait where possible? The default type could be Vec so it'd be entirely backwards compatible I think.
Pros:

Can use this library in an environment without libstd,
Might allow more efficient use of this library by avoiding heap allocations.

Cons:

Complicates implementation,
Adds a generic type to the interface, complicating error messages and possibly use,
Non-std Rust is currently niche at best.

Alternatives:

A different trait instead of VecLike
Do nothing
Could implement the iterators in a macro, allowing them to be made generic over the storage without exposing it to the end-user.

Note that new in itertools 0.7 was that it finally does support std-free environments. But yes, all the things that need Vec or similar are then removed, without substitute.
The idea sounds good, but we need a high quality dependency for the collection traits, and that's a rather tricky requirement. I also don't know a way to supply a default in a way that is backwards compatible like you suggest.
(I haven't looked around what kind of interface we actually need to a "vec", per functionality, but it could be interesting to look at that for a single item.)


[FEATURE REQ] Azure Service Bus Enforce mandatory processor arguments on CreateProcessor

Right now I can do the following
await using var receiver = serviceBusClient.CreateProcessor(destination, optionalOptions);
receiver.ProcessMessageAsync += async processMessagesEventArgs =>
{
    // processing logic
};

but if I try to start this I get a runtime exception telling me
Unhandled exception. System.InvalidOperationException: Cannot begin processing without ProcessErrorAsync handler set.
   at Azure.Messaging.ServiceBus.ServiceBusProcessor.ValidateErrorHandler()
   at Azure.Messaging.ServiceBus.ServiceBusProcessor.StartProcessingAsync(CancellationToken cancellationToken)

given that only ever one handler can be assigned, which is enforced by
https://github.com/Azure/azure-sdk-for-net/blob/6e4c8ea877c94adf7279c0c8ca458dd69ccadee6/sdk/servicebus/Azure.Messaging.ServiceBus/src/Processor/ServiceBusProcessor.cs#L289
why not enforce the two required handler parameters as input to CreateProcessor? How useful is it that those two are exposed as "event handler delegate" that are not really event handler delegates (because normally you could have multiple, here you can't) and in all the cases you always have to set both?
I think a similar API to the following would be more intention revealing and avoid runtime errors:
public ServiceBusProcessor CreateProcessor(otherMandatoryParams..., Func<ProcessMessageEventArgs, Task> messageHandler, Func<ProcessErrorEventArgs> errorHandler, ServiceBusSessionReceiverOptions options = default)

Once you have that signature it becomes clear for me as a user of the API that I always have to specify both. Then it might also become an option to rename the parameter from *EventArgs so something different because really those handlers are not real multicast .NET events.

// cc @jongio @JoshLove-msft @ramya-rao-a @SeanFeldman

Track 2 had this as an overload. When supplied, it would be called. Otherwise, the exception would bubble up. While it was not a trivial signature it did reveal the intent. A mandatory error handler should be required at construction time if the processor has to have it.


Feature/player page

Add Player pages.
closes #20
Links to player pages added to player tables in dashboard, as well games table in games page and player page.
Sorry about the messy git history. I made a mistake with git commit --amend, but now there's two merge commits? If you care, I can go back and figure it out... but I think it will take a while.

Hrm yeah. I usually avoid explicit merges and sync with master using rebases, tends to keep the history cleaner.

I'm also thinking of putting AGA info (and future info) in its own little group. So, we'd have a little AGA pane, and then maybe a KGS or League Stats pane, etc. But! that can wait for another PR :)

Looks good! gonna wait to see if @njritter has input, then we can merge

Awesome!


"Sending GOAWAY failed"

Hello everyone,
I'm new to heroic and not sure how to approach this error.
I've configured the heroic API, using debian package built from git repo.
After the service starts I'm getting the following message every minute.
10:11:15.922 [heroic-scheduler#2] INFO  com.spotify.heroic.cluster.CoreClusterManager - [new] grpc://131.x.x.x:8100 10:11:15.929 [nioEventLoopGroup-2-6] INFO  com.spotify.heroic.cluster.CoreClusterManager - [refresh] no nodes discovered, including local node 10:11:15.929 [nioEventLoopGroup-2-6] INFO  com.spotify.heroic.cluster.CoreClusterManager - [update] [{}] shards: 1 result(s), 1 failure(s) 10:11:15.930 [nioEventLoopGroup-2-5] WARN  io.netty.handler.codec.http2.Http2ConnectionHandler - [id: 0x7ffeead0] Sending GOAWAY failed: lastStreamId '0', errorCode '2', debugData 'Connection refused: /131.x.x.x:8100'. Forcing shutdown of the connection. java.nio.channels.ClosedChannelException
Any ideas why ?

Hey, is the node at 131.x.x.x configured to listen on port 8100 on gRPC?
This should be apparent in the configuration.

Hi John,
Yes, this is my config
`# heroic.yaml
port: 8080
cluster:
protocols:
- type: grpc
discovery:
type: static
nodes:
- "grpc://131.x:8100"
consumers:

type: kafka
schema: com.spotify.heroic.consumer.schemas.Spotify100
topics:

"metrics"
config:
group.id: heroic-consumer
zookeeper.connect: kafkabaer1.x,kafkabaer2.x,cassandra-node1.x
auto.offset.reset: largest
auto.commit.enable: true

metrics:
backends:
- type: datastax
seeds:
- cassandra-node1.x
- cassandra-node2.x
- cassandra-node3.x
- cassandra-node4.x
- cassandra-node5.x
metadata:
backends:
- type: elasticsearch
connection:
seeds:
- "heroic-e.x"
suggest:
backends:
- type: elasticsearch
connection:
seeds:
- "heroic-e.x"
shellServer:
host: "131.x.x.x"
port: 9190
`

If you are only using one node, there is no need to have a cluster configuration.
But I believe the problem you are seeing is caused by the node in the discovery configuration using a different port than the one which is used by grpc by default: grpc://131.x:8100 instead of omitting the port, like grpc://131.x.

Have been having a time getting Heroic going and one note here.  It seems like if you don't define which host/port GRPC should run on it won't start at all and you'll get an error like the OP did.  I "fixed" this by doing:
cluster:
  protocols:
    - type: grpc
      host: 131.x
      port: 8100
  discovery:
    type: static
    nodes:
      - "grpc://131.x:8100"

Hope it helps someone :)

Yes, you are correct. We came to a similar conclusion in #139
Thanks for sticking with it!


Fuse some bycolumn loops

This PR fuses some bycolumn loops. This is the direction I see things moving for the gpu / improving thread scaling.
This improved threaded allocations inside additional_tendency! by ~50% and step! by ~19%. This does however slow down the unthreaded step! by ~10%. Some of that may be variation, but it's unclear.

bors r+


[kueuectl] Add list LocalQueue command.

What type of PR is this?
/kind feature

What this PR does / why we need it:
To get list LocalQueues that match the given criteria: point to a specific CQ, being active/inactive, belonging to the specified namespace or matching the label selector.
Which issue(s) this PR fixes:

Fixes #2115
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Add list LocalQueue command on kueuectl

/ok-to-test

/cc @alculquicondor @mwielgus

/assign @trasc

Created another PR #2157 to fix integration tests.

Also about active/inactive filter, we will add it on the next follow up after finished #2109.

@vladikkuzn
Kubectl get columns are defined in the api. Changing them would be a breaking change and for list we want a couple more + couple Kueue-specific flags.

/test pull-kueue-test-e2e-main-1-28

/assign @alculquicondor

/release-note-edit
CLI: Add list LocalQueue command

/release-note-edit
CLI: Add list LocalQueue command


io.popen flashes cmd.exe window

Linters, gitstatus etc. are very annoying on Windows, because every time there is io.popen invoked, it creates visible cmd.exe window.

This is something that should be solved by the recently-added system.exec function -- this function currently exists only on the master branch; when the next version of lite is released plugins can be reworked to use it instead which should solve this issue.


Prometheus metrics of dqlite

Summary
Note: I am unsure if this has been asked; I want to know if microk8s exposes Prometheus metrics of dqlite. As source dqlite, Does it expose Prometheus metrics?
When we install microk8s, by default, it is HA enabled, and we would have dqlite as storage; however, if at all we want to use etcd, we need to disable ha-cluster, then we would get etcd on port 12379, we can get the metrics of etcd by supplying TLS certificates.
So, in the same way, can we get the dqlite prometheus metrics?
Why is this important?
Every other K8s distro exposes metrics to understand better the things related to cluster storage.
Are you interested in contributing to this feature?
No

Hello @developer1622,
there are metrics collected on the k8s-dqlite layer (former kine layer that translates k8s requests to dqlite) or you can dive deeper into dqlite logs.
K8s-dqlite arguments
You can configure k8s-dqlite to collect metrics by passing arguments, their description and defaults are here: https://github.com/canonical/k8s-dqlite/blob/master/cmd/root.go#L179-L182.
The metrics endpoint allows you to view the metrics of the k8s-dqlite layer with Prometheus.
In MicroK8s 1.31 you will need to enable the metrics endpoint first before scraping the metrics.
Since MicroK8s 1.31, otel is our newest addition that allows you to gather insights via traces on queries to dqlite using a tool like Jaeger.
You can configure these flags by running sudo vim /var/snap/microk8s/current/args/k8s-dqlite and restarting k8s-dqlite: sudo snap restart microk8s.daemon-k8s-dqlite.
Dqlite logs
Currently, we don't have an endpoint to gather metrics on dqlite itself. If you need further insights on the underlying Raft protocol (node consensus) I would recommend you have a look at the logs as follows:
Optional: Add debug log to dqlite:
sudo vim /var/snap/microk8s/current/args/k8s-dqlite-env

uncomment LIBDQLITE_TRACE=1 and LIBRAFT_TRACE=1.
restart k8s-dqlite with the flags: sudo snap restart microk8s.daemon-k8s-dqlite.
Check dqlite logs: sudo journalctl -u snap.microk8s.daemon-k8s-dqlite -f.
These options will help you to gather further insights into dqlite. However, we encourage you to provide further feedback to help us improve our visibility into dqlite!


fix: update docker image location in readme

This PR updates the docker image location to use the official intersectmbo published images.

superseded by #5771


Fix ssl connect tests, re-generate expired certifications

The certifications used inside the blackbox ssl connection tests
expired, generated new ones.
Fixes test failures as seen e.g. at https://github.com/crate/crate/pull/11309#issuecomment-825599461.
Checklist

[ ] Added an entry in CHANGES.txt for user facing changes
[ ] Updated documentation & sql_features table for user facing changes
[ ] Touched code is covered by tests
[x] CLA is signed
[x] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in CHANGES.txt
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

looks good. By the way, would this be recurring every year? I wonder if this can be addressed.

New test certs are valid for 10years.


Question about memory scaling during training

Hi,
Thanks for building these models. I noticed that the training scripts for the MP pre-trained models use small batch sizes of 16. What was the reasoning for this choice?
My application requires training on graphs with hundreds to a few thousand nodes, and I was hoping that MACE's lack of explicit triplet angle computation (as in DimeNet or GemNet) would offer more favorable memory scaling. Any insights would be greatly appreciated.
Thanks,
Rees

Hi @rees-c,
Sorry for the long delay in reply, the MACE github would be a more suitable place for your question.
The batch size has both an effect on the memory consumption but also on the training dynamics.
MACE can fit during training about 1000 nodes on a single GPU, A100. However we rarely go over 64 of batch size per GPU because we see degradation of accuracy past that.


Make List.sort and List.shuffle static methods.

Every implementation of List needs to have a sort and shuffle method, but we only really need one implementation that works on all lists (except the ones that are unmodifiable, and they'll throw early anyway).
If we had non-virtual methods, those two would be perfect candidates. We don't, so we should just make them static helper functions on List.

Not planning to change the List API at this time.


(PUP-3638) Update the help text for puppet doc

This removes the now useless options, examples, and updates the
helptext with a reference to puppet-strings which should be used instead
of the deprecated puppet doc command.

@nfagerlund ping

Other than that comment, :+1:

CLA signed by all contributors.


Settings: Optimize publisher logos image sizes

Feature Description
On the editor settings screen we currently display the publisher logos in their full size, which can be quite large.
Some thoughts:

We should use srcset here for optimization.
(calculateSrcSet util might serve as inspiration)
The publisher logos are displayed in 100x100px (Logo component). The AMP format uses/recommends 96x96 - let's also use that size? Otherwise a 96x96 image will get resized a bit.
Let's set explicit width and height attributes on the img
Pick reasonable sizes limit so we don't unnecessarily load large images.

Alternatives Considered

Additional Context

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria

Implementation Brief

@swissspidy I'm not seeing any sizes on the logo image media. Are there smaller images generated from uploads somewhere? or a way to request smaller image sizes from WP through some url params or something?

If there are no sizes it means there are no smaller ones. Just using 96x96px consistently should be good enough.

@csossi  - all this ended up being for right now is updating the size of the publisher logos on the dashboard settings to 96x96px

Looks like it's slightly larger

@csossi That is OK, as you are just inspecting the div, not the actual img element inside it. The img is a bit smaller. I just verified that it's 96x96 so I'll merge this.


Deprecated features

Since a couple of months i noticed that HA writes some warnings in the log, when it starts, speaking about deprecated features.
I am actually using HA Core rel 2024.5.3 on Debian.
Here is the log:
2024-05-14 02:42:36.265 WARNING (MainThread) [homeassistant.helpers.frame] Detected that custom integration 'dahua' calls async_add_job, which is deprecated and will be removed in Home Assistant 2025.4; Please review https://developers.home-assistant.io/blog/2024/03/13/deprecate_add_run_job for replacement options at custom_components/dahua/init.py, line 85: hass.async_add_job(, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:41.992 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:41.993 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.137 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.138 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.141 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.141 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.142 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.142 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.142 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.142 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.143 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.143 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.144 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.144 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues
2024-05-14 02:42:42.144 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation
2024-05-14 02:42:42.144 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues

This should be resolved now. Please let me know if you still see this in the latest version


Swagger-ui does not validate parameter types correctly

For API methods that have an IRI
argument that is required, for example

http://grlc.io/api/peta-pico/nanopub-api/#/default/get_get_nanopub

http://grlc.io/api/peta-pico/nanopub-api/#/default/get_find_nanopubs_with_uri
the user interface complains about the required argument (marks it in
red) and doesn't allow me to execute the query even after entering a
fully valid IRI.

Seems like swagger data-types does not validate IRI's: https://swagger.io/specification/v2/
It might work to replace
type: iri

For something like:
type: string
format: iri
pattern: 'http[s]*://'

But then we need a regex pattern for validating iri's (or leaving it blank and allow anything). I would be inclined to do that and just make it the user's responsibility to provide a valid IRI.

Hi @c-martinez, this is a bit of a shot in the dark, but would this be of help?
https://github.com/CLARIAH/iribaker

Long shot indeed. iribaker is not directly what we need, but we could

Modify iribaker to have a validate function (it already does but we could modify it to be able to use it externally
Add iri validation to grlc (I believe we can change the swagger specification so that it uses a validation endpoint).

The question is: is it crucial for grlc to validate iri's? Is it a nice-to-have? Is it completely out of scope? @albertmeronyo -- what do you think?

"It's not directly what we need" -> provides enough reason not to use it :)

Thanks @c-martinez @rlzijdeman for your suggestions!
I agree that iribaker does not directly solve the problem, but might give us directions into how to solve it.
For the time being I would just

leave iris as strings as per @c-martinez suggestion, but I wouldn't add any additional validation requirements (we might end up implementing the whole RFC :-) ).
look into openAPI's documentation for common datatypes/iris, or find a real world openAPI spec that validates them correctly

Fixed this in dev.grlc.io as per @c-martinez suggestion. A parameter's type should never be 'iri' anyways (MUST be one of "string", "number", "integer", "boolean", "array" or "file" as per https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#parameterObject), so now SPARQL variables that should be replaced by IRIs are specified as parameters with
type: string format: iri
(format is an open value)
There's a bit of extra processing in the query rewriter that looks into format in order to properly replace literal and iri parameters when the query is executed


Improve security for SSS webtasks

 Issue by kbrandwijk
Sunday Jun 25, 2017 at 21:23 GMT
Originally opened as https://github.com/graphcool/api-bugs/issues/149

What is the current behavior?
Although it should not be possible for an end user to get hold of the actual URL of a SSS, I would at least expect that it needs some Authorization token to execute. This applies to inline SSS functions.
What is the expected behavior?
SSS cannot execute without a proper Authorization header (PAT?).

 Comment by sorenbs
Monday Jun 26, 2017 at 11:45 GMT

Thank you for raising this issue Kim.
Currently the security of inline functions is based on a secret in the url. We have some planned changes to our infrastructure that will enable us to disable public access to the endpoint of inline functions.
Inline functions are fully managed by Graphcool the developer should not be required to think about how to secure their functions.
If you use something like the Serverless framework to host your functions we encourage you to use headers to secure the endpoint. We will introduce a way to securely manage environment variables on Graphcool, which should make this much easier.

 Comment by kbrandwijk
Monday Jun 26, 2017 at 11:51 GMT

@sorenbs Thank you for your explanation. It touches on a few good points.
However, you mention that when using 'external' webhooks, you encourage the use of headers. Would that mean manually specifying a Authorization header in the definition of the RP or SSS, or would a Authorization header be passed automatically? In the first case, I have no way to identify my user from an RP or SSS webhook, in the second case, how can I validate that token?

 Comment by sorenbs
Monday Jun 26, 2017 at 12:23 GMT

To be super clear - a user token DOES NOT grant access to the system API. Please reach out to me or Nilan in slack so we can investigate what is happening for you.
Permanent Access Tokens currently do grant access to the system API. We will be adding more granular control to the PAT concept in the future.
To your previous point. There are two aspects to consider:

Ensuring that only the Graphcool infrastructure can call your functions. For inline functions we handle this concern and for webhooks you can use headers.

Access the execution context for the function in your code to for example perform different actions depending on data on the User node. We will extend the function event to include more context like this.

 Comment by kbrandwijk
Monday Jun 26, 2017 at 12:31 GMT

@sorenbs I deleted the comment about the user token, I accidentally checked the wrong Authorization header while testing. So I agree that is not the case! I apologize for jumping to that conclusion.
Regarding your points:

To be super clear, for inline functions you currently do not handle that concern. All inline functions for RP and SSS are publicly accessible. Also, they do not run under the context of the authenticated user.

Without Graphcool passing any Authorization headers to my functions, it is currently impossible to check for authorization in a webhook. I can ensure that only Graphcool can call my webhooks however by some arbitrary header.

As you see, both options are seriously flawed.

 Comment by sorenbs
Monday Jun 26, 2017 at 12:42 GMT

Thanks

Currently the security relies on a secret in the url. In the future we will block public access to all inline functions.

The syntax is not final, but our plan is to augment the function event like this:

currently a function receives:
{
  data: { some: "data" }
}

in the future a function will receive something like this:
{
  data: { some: "data" }
  context: {
    request: RequestContext
    graphcool: GraphcoolContext
    environment: Json
    auth: AuthContext
    sessionCache: Json
  }
}

type RequestContext {
  sourceIp: String
  headers: Json
  httpMethod: HttpMethod
}

type GraphcoolContext {
  projectId: String
  alias: String
}

type AuthContext {
  nodeId: String
  type: AvailableTypes
  token: String
}

I would love your take on 2.

 Comment by kbrandwijk
Monday Jun 26, 2017 at 13:07 GMT

By the way, although a user cannot access the system API using his own token, it is accessible using any PAT you define on your project. I still believe that shouldn't be the case. Because with a PAT, your now not only giving away full access to the frontend, but also to the backend of your app, and that is not right...
About AuthContext:
- Add a checkbox to RP and SSS in the Console 'Run in caller context'
- If so, pass the original (user/PAT) Authorization token to the function or webhook, and add a more convenient method to the API to validate a token (other that calling the user query and hoping for the best).
- If the box is not checked, pass a one-time PAT-like token (with expiry), and make sure that token can be validated from the webhook against the API.
- The AuthContext would (only) be useful if you don't run in user context, to get information about the user the request originated from.

Add a dropdown to RP and SSS with 'Pass caller context', 'Pass elevated context', 'Do not pass context'
Use the Authorization Header in the request to provide a token that the webhook can verify (see the Github webhook secret link below)
Use a X-Graphcool-Admin-Token header to pass a PAT, only when 'Run in elevated context' is selected
Use a X-Graphcool-Caller-Token header to pass the token of the caller user, except when 'Do not pass context is selected'
Do not put Authorization data in the body
What are the nodeId and type fields in AuthContext?
Allow the developer to reset the PAT that was being passed to the webhook as elevated context. This is a must, because as soon as you pass it outside the system, you need a way to revoke it!

About GraphcoolContext:

Why would you need the projectId and/or alias? The security context should be linked to the project, is there a use case where you actually need the projectId? Also, the ProjectId is already in the Authorization token...

About RequestContext:

I think headers should be passed as headers, not in the body
The HttpMethod is used to call the webhook in the first place, so that's already accessible without passing it
Would sourceIp be the IP of the client that triggered the RP or SSS webhook? If so, use the `X-Forwarded-For' header, it's the standard way to pass a client IP through any middle layer (for example a proxy).

About environment and SessionCache:
What would those be? I guess environment would be variables you want to pass to the webhook? Maybe it's better to allow the developer to specify how to pass the data somehow. Use query parameters, how to format the body data? This would help connecting to external webhooks directly, where you have to implement their API format for calling the webhook and sending the data.
--- Updated with new thoughts about Authorization
--- Added requirement to reset PAT

 Comment by kbrandwijk
Monday Jun 26, 2017 at 13:41 GMT

Addition: found this for how Github deals with authenticating itself against user defined webhooks (using a JWT token based on a predefined secret and a checksum). Sounds great, use this :) https://developer.github.com/webhooks/securing/


Add Merge-On-Read index files

Right now we handle deleted and moved rows poorly. For deleted rows, we have to read an individual fragments' deletion file. For moved rows, we consider them no longer part of the index, even though they theoretically should be query-able.
We can add a new file to better handle Merge-On-Read index modifications. This would store:

Deleted fragment_ids
Deleted row_ids
Remapped ids

The IO cost of deletions would be reduced because we just need to read this one file.
Additionally, we just need this file to perform "compaction" on indices.

cc @westonpace @chebbyChefNEQ What do you think of this?

Should we just re-write the index. as well as #1342

Should we just re-write the index. as well as #1342
@eddyxu

I think these are two different things.

Always be able to query the index, regardless of any changes that have been made since the index was created
"Update" an index to bring it up to the current dataset version

My original approach was to do both of these as the same operation but this ran into trouble because it's impossible to know the new row ids until the rewrite is committed (because that's the only point when we know the new fragment IDs).
@wjones127 How should a single unified merge-on-read file work with something like a multi-task compaction job?  This also means that rewrite and delete will conflict even if they don't modify the same fragments, but I don't think that's a big problem.

How should a single unified merge-on-read file work with something like a multi-task compaction job?

I was thinking it would be used for a separate update / compact index operation, so wouldn't relate to that.

This also means that rewrite and delete will conflict even if they don't modify the same fragments, but I don't think that's a big problem.

Yeah, plus the remap probably needs to know the fragment ids for the current commit, so I was thinking this would have to be written in the commit loop.


Config Documentation is Incorrect

Example Documentation for Config
https://github.com/cloudfoundry-incubator/bpm-release/blob/21958b5/docs/config.md#job-configuration
Shows that <job> and <worker> are keys. Looking at the source code it reveals that the processes is an array, and that the job name is part of name
It seems that this is a more accurate configuration, but it's hard to tell
processes:
- name: uaa
  executable: /var/vcap/jobs/uaa/bin/uaa
  env:
    CATALINA_BASE: /var/vcap/data/uaa/tomcat
    CATALINA_HOME: /var/vcap/data/uaa/tomcat
    CLOUD_FOUNDRY_CONFIG_PATH: /var/vcap/jobs/uaa/config
    CATALINA_OPTS: "-Xmx768m -XX:MaxMetaspaceSize=256m"
  args: []

There's a pending PR which should fix the doc issues https://github.com/cloudfoundry-incubator/bpm-release/pull/18

#18 has been merged now

@fhanik is it ok to close this based on the updates from #18 ?


[SPARK-23630][yarn] Allow user's hadoop conf customizations to take effect.

This change restores functionality that was inadvertently removed as part
of the fix for SPARK-22372.
Also modified an existing unit test to make sure the feature works as intended.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1410/
Test PASSed.

Test build #88104 has started for PR 20776 at commit 5abac2c.

Test build #88104 has finished for PR 20776 at commit 5abac2c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88104/
Test PASSed.

Merged build finished. Test PASSed.

LGTM.

Merging to master / 2.3.


Train model on property other than LogP and QED

Hi, thanks so much for developing this amazing tool! I would really appreciate it if you could help me with an issue when I tried to apply it. In the src/train.py, line 90, there are five choices provided for property model training, QED, SA, logP, GSK3B and JNK3. However, an error '' Invalid Property" raised when I assigned 'gsk3b' as the props parameter. I wonder how I could enable the model to train in this property, or if there is some possible way to assign customized property for the training process. Thanks so much!
error_props.txt

Hi, I think the following steps can help you train a model on a custom property. Suppose the property is GSK3B:

Implement get_gsk3b(molecule) in src/evaluation/utils.py. Here the argument molecule is a RDKit Molecule object, so this function calculates the gsk3b score given the molecule.
Register the function in eval_funcs_dict(), e.g. add an entry in the dict {'gsk3b': get_gsk3b}. This tells the framework to use the function get_gsk3b to calculate the score for the property "gsk3b".
Append the normlized score in get_normalized_property_score() and the restored score in restore_property_score(). The first function normalizes the property scores to approximately standard Gaussian (i.e. N(0,1)) which is better for regression. The second function restore the original scale from the predicted results. But if the score is already in [0, 1], I think you can just implement them as identity mapping (i.e. f(x) = x).
Add threshold in PROP_TH. When the scores of the generated molecules are above the thresholds, they are regarded as successful optimization. Add the property name in PROPS.
Add the property name to the choices in train.py.
Now you can start training!


Add option to disable certificate validation

Issue describing the changes in this PR
resolves #1351
Pull request checklist

[x] My changes do not require documentation changes

[ ] Otherwise: Documentation issue linked to PR

[ ] My changes should not be added to the release notes for the next release

[x] Otherwise: I've added my notes to release_notes.md

[x] My changes do not need to be backported to a previous version

[ ] Otherwise: Backport tracked by issue/PR #issue_or_pr

[x] I have added all required tests (Unit tests, E2E tests)

@microsoft-github-policy-service agree

Please also add this property to the RabbitMQOutputAttribute class to have consistency. Were you able to disable certificate validation in your function app after making this change?

Good point, I added the property to the output attribute as well. And yes, I was able to disable certificate validation by adding this property locally.

@JatinSanghvi thanks for your quick approval. Can you give me any perspective what the cycle time is before this change is included in a new stable nuget package? And is there any way we can help get this change there?

Can you give me any perspective what the cycle time is before this change is included in a new stable nuget package? And is there any way we can help get this change there?

@kshyju, can you give an ETA on when a NuGet package version with the change will get released?

Can you give me any perspective what the cycle time is before this change is included in a new stable nuget package? And is there any way we can help get this change there?

@kshyju, can you give an ETA on when a NuGet package version with the change will get released?

We need the version bumped for a new package version release.  I also noticed that Microsoft.Azure.WebJobs.Extensions.RabbitMQ 2.0.3 is also out  but the worker is still using 2.0.0
I think we could point to in 2.0.3 Worker.Extensions.RabbitMQ and include change from this PR and bump our version to 2.0.3 and publish that. Thoughts @JatinSanghvi ?

Sounds good @kshyju. I have created #1365.

Thanks for you quick response @JatinSanghvi. Your suggestion sounds great. Please let me know if I can help in any way.
The pipeline guarding this merge seems to be pending on a new permission. Who can approve this?

The pipeline guarding this merge seems to be pending on a new permission. Who can approve this?

@kshyju, can you merge the PR if it looks good?

Thanks @JatinSanghvi for merging #1365.
@ysbakker Could you rebase please?

@kshyju done!

Thanks for your contribution @ysbakker  🎉
Will share an update to this thread when the new version is published to nuget gallery.

Thanks for your contribution @ysbakker 🎉
Will share an update to this thread when the new version is published to nuget gallery.

Awesome @kshyju 😄
I was curious if there are any updates?

@v-shenoy Can you help with releasing this version?

https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.RabbitMQ/2.0.3 is out.


Migrate configurations storage from Firestore to Redis

Problem
Now we are using Firestore as backend storage for UI. This prevents us from running hosted UI due to dependency on external storage.
Solution
We need to migrate all the data requests from UI to our backend which will store configurations at Redis.

implement a configuration storage layer at UI backend (should be configurable to be able to use Firestore and Redis)
implement API to store and get data from the backend
migrate all existing configurations from Firestore to Redis

Implemented at this PR

Implemented at this PR


Going from keras model to dlib model

I know that this is not strictly dlib related question and maybe I will not get the answer, but still wanted to try, the audience is pretty big here :)
I am trying to migrate from keras to dlib, because of the superior documentation and ease of metric learning here. For the beginning I am working doing the loss_multiclass_log optimization based on dnn_introduction_ex.cpp, dnn_introduction2_ex.cpp and dnn_metric_learning_ex.cpp to check if I get comparable results with the same model before diving into metric learning. The average loss goes from 4.63 to 4.55 on the first epoch and afterwards it does not get any further. I even tried with increasing trainer.set_iterations_without_progress_threshold(10000) but it still does not get the loss lower than 4.53... Am I missing something here?
I am feeding the data in dlib as a std::vector<matrix<double, 0, 1>> where matrix comprises of 340 row vector.
The dlib network:
using net_type = loss_multiclass_log<
        fc<286,
        dropout<relu<fc<1500,
        dropout<relu<fc<1500,
        input<matrix<double, 0, 1>>>>>>>>>>;

layer<0>	loss_multiclass_log
layer<1>	fc	 (num_outputs=286) learning_rate_mult=1 weight_decay_mult=1 bias_learning_rate_mult=1 bias_weight_decay_mult=0
layer<2>	dropout	 (drop_rate=0.275)
layer<3>	relu
layer<4>	fc	 (num_outputs=1500) learning_rate_mult=1 weight_decay_mult=1 bias_learning_rate_mult=1 bias_weight_decay_mult=0
layer<5>	dropout	 (drop_rate=0.275)
layer<6>	relu
layer<7>	fc	 (num_outputs=1500) learning_rate_mult=1 weight_decay_mult=1 bias_learning_rate_mult=1 bias_weight_decay_mult=0
layer<8>	input<matrix>

Keras model:
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense_1 (Dense)              (None, 1500)              505500
_________________________________________________________________
activation_1 (Activation)    (None, 1500)              0
_________________________________________________________________
dropout_1 (Dropout)          (None, 1500)              0
_________________________________________________________________
dense_2 (Dense)              (None, 1500)              2251500
_________________________________________________________________
activation_2 (Activation)    (None, 1500)              0
_________________________________________________________________
dropout_2 (Dropout)          (None, 1500)              0
_________________________________________________________________
dense_3 (Dense)              (None, 286)               429286
_________________________________________________________________

The dlib output:
Epoch: 1     learning rate: 0.01  average loss: 4.6328       steps without apparent progress: 2410
Epoch: 2     learning rate: 0.01  average loss: 4.55244      steps without apparent progress: 5213
Epoch: 3     learning rate: 0.01  average loss: 4.5501       steps without apparent progress: 7901
Epoch: 4     learning rate: 0.01  average loss: 4.54932      steps without apparent progress: 9152
Epoch: 5     learning rate: 0.001  average loss: 4.54276      steps without apparent progress: 5283
Epoch: 6     learning rate: 0.001  average loss: 4.53794      steps without apparent progress: 6098
Epoch: 7     learning rate: 0.001  average loss: 4.53792      steps without apparent progress: 8740
Epoch: 8     learning rate: 0.0001  average loss: 4.53726      steps without apparent progress: 9715
Epoch: 9     learning rate: 1e-05  average loss: 4.5363       steps without apparent progress: 9730
Epoch: 10    learning rate: 1e-06  average loss: 4.53364      steps without apparent progress: 0
training num_right: 5835
training num_wrong: 91605
training accuracy:  0.059883
testing num_right: 1460
testing num_wrong: 22900
testing accuracy:  0.0599343
Keras output
epoch,acc,loss
0,0.6818555008210181,1.3333302831659568
1,0.8814347290640394,0.42806850175919203
2,0.9182881773399014,0.2863174603056095
3,0.9363505747126437,0.2230666387248142
4,0.9461720032840722,0.18629631130621205
5,0.952524630541872,0.1625511203678432
6,0.9577380952380953,0.14498288792539663
7,0.9617918719211823,0.12999034807233564
8,0.9647167487684729,0.12031887783768337
9,0.9670566502463054,0.11327744584053843
Accuracy: 9.9670567
If there is anything more I should provide I will happily do so.

It’s hard to tell based on what you posted. Maybe the learning rate should be higher. Maybe you set the weight decay to something way too high. Maybe you just aren’t loading the labels and it’s training on random labels.

weight_decay_mult is at 1 so I guess this is not a problem. I will try with the higher learning rate and confirm how it behaves, thank you!

I am not sure what was it, I assume some numerical instability, because when I removed some data columns in the samples to remove columns with very low variance and applied the distribution normalization the network is being trained properly. So success :)
But I had some problems with the distribution normalization:
here I wanted to apply the normalization as in example 'rank_features_ex.cpp':
 const sample_type m(mean(mat(samples)));  // compute a mean vector
 const sample_type sd(reciprocal(stddev(mat(samples)))); // compute a standard deviation vector
 // now normalize each sample
 for (unsigned long i = 0; i < samples.size(); ++i)
     samples[i] = pointwise_multiply(samples[i] - m, sd);

but sd vector had some values which were difficult to interpret, some infs and 0s where the shouldn't be.
So I applied it in a different way:
   const auto m(dlib::mean(dlib::mat(dlib_data)));  // compute a mean vector
   const auto sd(dlib::stddev(dlib::mat(dlib_data))); // compute a standard deviation vector
   // manually do reciprocal
   std::vector<double> sd_vector(sd.begin(), sd.end());
   std::for_each(sd_vector.begin(), sd_vector.end(), [](double& v){v = 1.0 / v;});
   const auto sd_reciprocal(dlib::mat(sd_vector));
    // now normalize each sample
    for (unsigned long i = 0; i < samples.size(); ++i)
        samples[i] = pointwise_multiply(samples[i] - m, sd_reciprocal);

by manually making reciprocal of each element. That was it what made the network train properly.
Why did this dlib::reciprocal not work properly? Is it the problem, that the data has size NC = 1 and NR = 0? I just create data structure as it was in 'dnn_metric_learning_ex.cpp': std::vector<matrix<double,0,1>> samples;

reciprocal does just what it sounds like it does, you must have a bug in the code.  Please post a complete and minimal program that reproduces the error if you think you found a bug in reciprocal.

Perfect, I'll try to prepare an example and post it in a separate issue, which description will better suit the case.


latest go-apk with consistent file ordering

@mattmoor had found that the file ordering from go-apk was not consistent, which led to differing sha256 sums of the layer.tgz. This fixes that.
In my tests, every time I built any image, I got the exact same contents in the tar, right down to sha256sum of each.
Note that the output tar itself is not consistent, despite the consistent contents. This could be for various reasons, and should be resolved separately.

Once this is in, we should rebase https://github.com/chainguard-dev/apko/pull/668 to it and see that it passes, then merge that in.

This has a race condition. Fix is in https://github.com/chainguard-dev/go-apk/pull/30 . Wait for that to be merged in, I will update this, then this should be clear.

Updated now that https://github.com/chainguard-dev/go-apk/pull/30 is in with that commit.
Also made sure that the test that was getting stuck, sails through. There might be more, but that is why we have CI!

awesome, I'll rebase my other pr 🙏


DIAM config for JAM RCC front end

Changes being made
Add JAM-RCC configuration for the front end.
Context
What is/are the context for these changes? For example: particular role needs to be shown in token, JSUTIN Claims etc ?. Reference JIRA Board task, if applicable.
JUSTIN Claims, similar to LEA
Quality Check

[ ] Client has Name and Description defined.
[ ] Full Scope Allowed is disabled.
[ ] Direct Access Grants Enabled is disabled.
[ ] Valid Redirect URIs are properly defined, or explanation for * (allow all) is provided.
[ ] Web Origins are set to + instead of * to restrict the CORS origins.
[ ] Client Scopes are not assigned to client, or explanation for doing so is provided. [^1]
[ ] Client module and all references are defined in clients.tf in realm root folder. Same rule applies to other resources, like groups and realm roles.
[ ] Terraform plan contains only my changes, or other developers are aware that their manual changes can be overridden. [^2]
[ ] DIAM JIRA  issue is created and/or updated, if applicable.
[ ] When updating composite roles (eg. Realm roles) and scope mapping resources, remember to re-run the apply. [^3]

[^1]: Data transparency. Does the client you are creating have the permissions to pass/access all the Client Scope attributes in the token? For example profile scope includes user birthdate, which is used by BCSC, but other applications shouldn't necessarily have access to it.
[^2]:
Keep in mind that sometimes Keycloak automatically adds properties to newly created resources. terraform plan will show them as changes made outside of Terraform. As long as those attributes are empty and do not interfere with existing configuration, they can be ignored. Here is example of one:

[^3]: Due to the terraform provider bug, updating/deleting one entry within the resource deletes all of them. Re-running the apply action will result in restoring the configuration to the desired state. Keep in mind, that composite role deletion will show up on the terraform plan output, on contrary to scope mapping.

@GeorgeWalker  you will have to create the PR from a Feature branch on the bcgov/ag-sso-terraform repo


css not working on local dev

while i ran bun dev or pnpm run dev i got this broken pages

hi! sorry for the delay.
You can check the developer tool output in your browser. It will tell you why css is not loading. Did you ran bun install or bun update successfully?


Does waterline mongodb supports $text search ? not using the native method

+1 for this feature. Would facilitate the "populate" on these cases, for instance.

+1


fix: remove incubator.

Proposed changes
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[x] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[x] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[x] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[x] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 33.6 seconds
load time: 452 seconds
storage size:<PHONE_NUMBER>5 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221110103229_clickbench_pr_43354.html


Added fuselage cross beam strut

Depends on #408
added cross beam strut geometry getter

looks good:

rebased on current cpacs_3 (and made sure the cpacs_3_debug code was not included). Are we good to merge?

Looks good to me! I will merge as soon as CI is through