Add support for auto-pilot throughput feature for cosmosdb

Is your feature request related to a problem? Please describe.
I manage all of my infrastructure as code, using az cli scripts.  When provisioning cosmosdb resources, I want to be able to configure the auto-pilot preview feature for throughput rather than just the numeric R/U option via --throughput flag.  While I understand this feature is in preview, this feature massively eases the billing burden of spiky loads and I recommend it to all of my colleagues and use it myself.  It really pains me to manage all my infrastructure as code, but have to manage cosmosdb by using the portal -- especially when recommending to others.
Describe the solution you'd like
Add options to az cosmosdb create/update to allow input for using auto-pilot throughput & auto-pilot throughput range (the portal displays these options).  E.g.
az cosmosdb database create ... --auto-pilot true --auto-pilot-max-throughput 1000 ...
Describe alternatives you've considered
You have to manage cosmosdb via the portal -- it has disrupted my workflow and GitOps pipelines, as I now have a manual step (which unfortunately is worth it for the auto-pilot benefit).
Additional context

Hi @thisisbrianstewart thank you for opening this issue. As this feature is still in preview, we are going to wait to add support via the CLI for now. We do not yet have a timeline for adding this feature, but we plan on adding it once we have more feedback. We are tracking this internally as work item 541736. I will update once we get closer to adding support.

move to S167

move to S168

move to S169

move to S170

move to S171

I believe @ravgill is working on adding support for this in the CLI now.

This was merged in the last release branch.


Element-wait plugin not retrying when element not found

I'm using the element-wait plugin with Robot Framework and Appium2 to automate mobile testing. I've created a configuration file .appiumrc.json with the following settings:
{
"server": {
"plugin": {
"element-wait": {
"timeout": 2000000,
"intervalBetweenAttempts": 200,
"excludeEnabledCheck": ["click", "false"]
}
},
"use-plugins": ["element-wait"]
}
}
When executing tests, the Appium server starts with the plugin, and the timeout and interval settings are correctly applied. However, when attempting to find an element using the AppiumDriver.findElements() method, if the element is not found on the first attempt, the test immediately fails without retrying.
I expected the plugin to retry finding the element based on the configured timeout and interval settings, similar to how the Wait Until Page Contains Element keyword works in Robot Framework. However, it seems that the plugin is not retrying the element search.
Steps to Reproduce:
Set up Appium server with the element-wait plugin configured as described above.
Write a test case to find an element that may not be immediately available.
Execute the test case and observe that the test fails without retrying the element search.
Environment:
Operating System: Windows
Appium Version: 2.5.1
Device Under Test: android emulator
Expected Behavior:
I expect the element-wait plugin to retry finding the element according to the configured timeout and interval settings, allowing for more robust and reliable element detection during test execution.

you can find log here :  https://gist.github.com/rimeak/8e7273e5520663942cdc3ee7aa7d8773

@rimeak  if the element is not found on the first attempt, the test immediately fails without retrying I don't see any failures in the logs.

The test begins by establishing a session through a POST request to the /session endpoint. Following this, it proceeds to navigate to a designated URL using a POST request to /session/b4b308f3-e53e-4ef6-b135-33f4cdb9410e/url, with the URL provided in the request body.
Upon reaching the specified URL, the test initiates the search for elements on the page by sending a POST request to /session/b4b308f3-e53e-4ef6-b135-33f4cdb9410e/elements, accompanied by the XPath query. Upon successful execution, the response typically contains the ID of the element in the "value" field :   Got response with status 200: {"sessionId":"403e3d02-7387-455e-9284-54cb0f01af9c","value":[{"ELEMENT":"00000000-0000-0023-ffff-ffff00000012","element-6066-11e4-a52e-4f735466cecf":"00000000-0000-0023-ffff-ffff00000012"}]}
During the execution flow, the test successfully locates and interacts with the second element identified by the XPath query //android.widget.Button[@resource-id="com.android.chrome:id/negative_button"], as the response contains the ID of the element.
However, when attempting to locate the third element using the XPath query //android.widget.Button[@resource-id="details-button"], the response returns with an empty "value" field. This implies that the element may not have been fully loaded on the page at the time of the request, preventing the test from clicking it. Consequently, the test fails, and a screenshot is captured.
I've ensured that the locators are correctly defined, and when utilizing the "wait until page contains element" keyword before each click action, the test successfully retries until the element is found. I anticipated that the plugin would exhibit similar retry behavior, persistently attempting to locate the element within the specified timeout of 300000 milliseconds and an interval between attempts of 60000 milliseconds. However, it appears that the plugin does not retry the element lookup as expected, the test fails on the first attempt if the element is not found within the timeout period.
see pic below for fail when executing test with element-wait plugin

@rimeak is it possible to give a sample project to reproduce this issue ?

Sure @saikrishna321, here is a sample project :
SampleProjetc.zip
here is what i got in my case :


fix: script to replace local ip in case of apple m1

Description
On M1 mac, ./start-https.sh has a different handling to use en0 interface ip.
Currently, it doesn't replace local ip in generated nginx.conf file which leads to docker startup failure.
Fixes #3563
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Locally by running and manually checking nginx.conf generated files and docker logs.

Configuration generated after this change

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Works for me, I'll just create a new PR so that tests are able to run.


Classloader clash with other plugins using newer (3.x+) version of Ehcache

The latest (4.0.0) version of this plugin has a dependency on net.sf.ehcache:ehcache:2.9.0. The Ehcache specific classes contained in that dependency are of course contained in the net.sf.ehcache package. That version also contains com.terracotta and org.terracotta packages w/ relevant classes.
Some other more recently updated Grails plugins (I'm looking specifically at org.grails.plugins:cache-ehcache:3.0.0 here) use a newer version of Ehcache (org.ehcache:ehcache:3.4.0). In that version, the Ehcache specific classes use a different package naming convention of org.ehcache, however the org.terracotta classes still exist but with subtly different implementations.
Therefore, when trying to use the latest version of both grails plugins, the following class loader error occurs at startup:
2020-07-12 12:09:33.673 ERROR --- [  restartedMain] o.s.b.d.LoggingFailureAnalysisReporter   :

***************************
APPLICATION FAILED TO START
***************************

Description:

An attempt was made to call a method that does not exist. The attempt was made from the following location:

    org.ehcache.impl.internal.store.basic.BaseStore.registerStatistic(BaseStore.java:82)

The following method did not exist:

    'void org.terracotta.statistics.StatisticsManager.createPassThroughStatistic(java.lang.Object, java.lang.String, java.util.Set, org.terracotta.statistics.StatisticType, java.util.function.Supplier)'

The method's class, org.terracotta.statistics.StatisticsManager, is available from the following locations:

    jar:file:/Users/jamesdh/.gradle/caches/modules-2/files-2.1/net.sf.ehcache/ehcache/2.10.6/38076997b10f5c762a210ea8b9f6099da35ac5f/ehcache-2.10.6.jar!/org/terracotta/statistics/StatisticsManager.class
    jar:file:/Users/jamesdh/.gradle/caches/modules-2/files-2.1/org.ehcache/ehcache/3.6.3/d8da79858f4016e8f3092453cd6b1a4ece6f6d27/ehcache-3.6.3.jar!/org/terracotta/statistics/StatisticsManager.class

It was loaded from the following location:

    file:/Users/jamesdh/.gradle/caches/modules-2/files-2.1/net.sf.ehcache/ehcache/2.10.6/38076997b10f5c762a210ea8b9f6099da35ac5f/ehcache-2.10.6.jar

Action:

Correct the classpath of your application so that it contains a single, compatible version of org.terracotta.statistics.StatisticsManager

2020-07-12 12:09:33.692  WARN --- [  restartedMain] o.s.b.f.support.DisposableBeanAdapter    : Invocation of destroy method failed on bean with name 'tomcatMetricsBinder': java.lang.NullPointerException

Task List

[x] Steps to reproduce provided
[x] Stacktrace (if present) provided
[x] Example that reproduces the problem uploaded to Github
[x] Full description of the issue provided (see below)

Steps to Reproduce

Add compile "org.grails.plugins:cache-ehcache:3.0.0" as a dependency
Add compile "org.grails.plugins:spring-security-core:4.0.0" as a dependency

Expected Behaviour
The application will start normally
Actual Behaviour
The application fails to start due to a classloader collision w/ the same classname existing in multiple different dependencies, each with slightly different implementation.
Environment Information

Operating System: MacOS 10.15.4
GORM Version: 7.0.4.RELEASE
Grails Version (if using Grails): 4.0.3
JDK Version: 11.0.4

Example Application

https://github.com/jamesdh/grails-ehcache-class-collision

Filed this here because it feels like the most likely place that we can do something about this. I guess ideally Ehcache would have published a 3.x+ version with different classpaths for the org.terracotta classes as well, but that is beyond our control.
I'm guessing that the easiest solution to this is to just upgrade Grails Spring Security Core to use the latest Ehcache version.

Yes, we should update the Grails Spring Security Core plugin to the latest Ehcache version. Would you be interested to send a PR for the same?

@puneetbehl it appears the plugin defines certain spring beans based on Spring Security Core classes which in turn have a hard dependency on the old net.sf.ehcache package e.g. EhCacheBasedUserCache.java.
So it's not just a matter of upgrading the dependency in the build.gradle but appears it may be quite a bit more involved than what I'd feel comfortable taking a jab at given the widespread use of this plugin.

And just for reference, it appears the latest version of the spring-security dependencies used all still rely on the old net.sf.ehcache namespace/API and have not been upgraded to be compatible with the newer org.ehcache namespace/API.

I am running into the same issue. It seems like this is going to be a common issue for folks upgrading to grails 4. Did you find a workaround?

@reecega my fix was I stopped using Grails, unfortunately.


[card-cache] optimizations to avoid deadlocks

[card-cache] optimizations to avoid deadlocks

deadlocks happened when cleanups coincided with heavy load
tests simulating a cleanup (disk + shared-objects) and heavy load together were successfully able to reproduce the deadlock situation
Locking at the context level:

Each time we want to clean up we lock to create a new context
All directories/processes are written within that new context.
Switching this context which ensure that all new processes get created differently and the cleanup process can safely remove everything.
Context also sets the read/write directory for the cache object used in the API endpoint.
All locking on the API side now always time-bound. The code time's out if it can't acquire a lock.
The method ensure that all operations won't in-definately hold the lock.

Changed defaults for minimum amount to time to wait for cards in the cache process to 20 seconds (helps make things snappier)
Added timings dict in card cache to optimize loading cycles (Ensured that it is set based on a per-card basis)

#442 supersedes this PR


Add fold_count parser equivalent for count

This is already possible with fold_many_m_n. I dont think adding a completely new parser just for m == n is worth it.

This is already possible with fold_many_m_n. I dont think adding a completely new parser just for m == n is worth it.

You could say the same about count.

You could say the same about count.

There are some convenience parsers that allow you to avoid using fold_* (such as many_*). But I think thats a bit different from adding another fold_* parser.
My current problems with this:

We already have 3 fold_* functions and adding another one for a use case thats already covered by another (and is trivial to use for such cases) is just unnecessary .
Its name doesnt fit in with the other functions of the fold_* family. If anything it should be called fold_many_m.

At least name it fold_many_n

At least name it fold_many_n

fold_many_m. The n is the second parameter which doesnt exist here.

At least name it fold_many_n

I think you mean fold_many_m. The n is the second parameter which doesn't exist here.

in math, n come first then you use m, plus n is the max in fold_many_m_n.

in math, n come first then you use m, .

I dont think thats a set rule. n is often used to denote a number of something. However in math one also often uses x or even a. It usually comes down to preference and context.
In this case I would prefer to stay in line with the other function names and make it fold_many_m. Alternatively we could drop it altogether and just make it fold_many, with the possibility to later expand the parameter to also take ranges as proposed in #1393.

plus n is the max in fold_many_m_n

Irrelevant. This parser is about the case of m == n.

Irrelevant. This parser is about the case of m == n.

how irrelevant, that stupid, obviously this is for max only so take n is totally relevant.

Alternatively we could drop it altogether and just make it fold_many

yes why not

how irrelevant, that stupid, obviously this is for max only so take n is totally relevant.

The only obvious thing is that n and m are the same in this parser. So they can be used interchangeably. Whether you use the minimum or the maximum is irrelevant if they are the same.

how irrelevant, that stupid, obviously this is for max only so take n is totally relevant.

The only obvious thing is that n and m are the same in this parser. So they can be used interchangeably. Whether you use the minimum or the maximum is irrelevant if they are the same.

I will repeat so, no that different you don't understand. if you take m this would mean only minimum, so take_many_m would be read as "take at least m" cause IS the minimum in fold_many_m_n, n is more logical ( n come before m), match the max from already exist function and is often used as "the number n".

This case is also handled by #1402 which in my opinion is a much more elegant solution compared to adding yet another specialized parser.

This should be solved by #1608, which allows fold(n, parser, init, fold).


sending attributes and parameters injected in scope controller

sending attributes and parameters injected in scope controller

What's the rationale for changing close to onClose?  because it's a promise?

@brunoporto: My concern about pulling this code in is one of lifecycle.
At the moment, the code is set up so that the modal controller gets a clean scope and decorates it with the appropriate data. With the init functionality, we now have the scope having extra data set from outside the controller. This may be a convenience at times but does it not confuse responsibilities? I think the controller should be the thing to set up the scope.
Thoughts? Also changing the name of the close promise and injected function may make it more readable but will break the API, which I'd rather avoid until I have to.


Add "top queried domains" to Query Library

Inspired by the community inquiry tracked in https://github.com/brimsec/zq/issues/2092, it seems we could benefit from adding something similar to the default Query Library. Right now the only query we have tagged as dns is "Unique DNS Queries" (_path=="dns" | count() by query | sort -r). This other one seems like it would be handy to give a similar summary at the domain level:
_path=="dns" | count() by domain:=join(split(query,".")[-2:],".") | sort -r

An example output with test data https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz:

Alas, while a useful table, it reveals another example of where you can't successfully "Pivot to logs" from this table to isolate the intended results, hence making another case for brimdata/brim#1420. But we might want to add it regardless just because the summary is useful by itself.

The queries.json that can be downloaded from the Brimcap repo now includes this as the "Top Domains" query. The following screenshot shows example output of it when run against logs generated from a wrccdc pcap. Note that at the moment a "from" operator/pin must be added after clicking such a saved query, and we're actively working on improving this.


[BUG] train error

hi,
I try to use timm to train my own dataset, I make them as the imagenet struct, but I found the train loss don't decrease, I don't know if I do something wrong, my train script is like this
sh ./distributed_train.sh 4 ./data --model swin_small_patch4_window7_224 -b 54 --class-map ./data/class_map.txt --sched cosine_lr --epochs 50  --opt adamw  -j 16  --weight-decay 0.05  --lr .001 --reprob 0.25 --remode pixel --aa rand-m9-mstd0.5-inc1 --drop-path 0.1 --pretrained --num-class ${class_num}
it runs normally, but i find the train loss is not decrease, and at first, it warned that
Reducer buckets have been rebuilt in this iteration.
I use timm to train last year, it work perfectly, but I pull the newest version, and run as the past params, it can't train well

well,
I check the params, it seems like the loss function cause this problem, maybe my dataset classes is too many, cross entropy loss is not suit for this


Update RELEASE.md to use bundle*Aar task

Fixes #186
You can check this by doing the following:
First run clean bundleRelease.
Checkout the thirtyinch/build/outputs/ dir. There is no aar  directory which contains the aar.
Now run the bundleReleaseAar task and check the same directory.
The aar directory is available and the aar file

Rebased the current master (and forced push).
Otherwise the task wasn't available.

Now it works correctly.


Attendees cannot raiseHand

When attendees click on "Raise Hand" we have a socket error:
index.tsx:143 Error sending Live Event message.
index.tsx:171 DOMException: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state.

Attendees and moderators are connected to the same socket URL:
wss://xxxx.execute-api.us-east-1.amazonaws.com/Prod?LiveEventId=LiveEvent-xxxx&AttendeeId=
wss://xxxx.execute-api.us-east-1.amazonaws.com/Prod?LiveEventId=LiveEvent-xxxx&AttendeeId=

Any solution ?
thank you

Hi @scredii, thanks for bringing this up - we found this bug on our end yesterday also. I'll take a look and see if I can find out more. Thanks!

Hi @michhyun1.
Thanks for your quick reply.
Do you have any update ?
thanks

Hi,
Facing same issue, Attendees not able to join also and not able to raise hand. Can you please update.

@scredii @vijayamin83 @msgoo7 thank you for your patience. I've been able to reproduce this, I see the following:

I do notice that the console log says that the attendee is not authenticated. Can you confirm this is the same for you?

Hi @michhyun1 ,Thanks for your response.
I am getting the same error.
There is no issues in broadcast, moderator and talent. But when we login to attendee page via the url from s3 bucket, got a similar response which you attached in the image.
Due to which when attendees raised hand , no notification is there to the moderator and also when the moderator broadcast the live session, attendees is also not able to see the same and a constant screen saying no one is streaming appears on the attendee page.

I am getting below error while trying to raise hand.

@michhyun1 it show attendee unauthenticated, even if I use the authenticated attendee for that particular events.
Where as other users like moderator, talent and broadcaster.
I have cross verified in the dynamo table also and it contain the valid data. But still getting an error.

@vijayamin83 @Ashrma2021 @msgoo7 @scredii
Thanks all for your patience - it seems like there was a bug introduced when we updated our dependency to the SDK to 2.0.0.
I've gone ahead and deployed a version using the git commit hash (30c8f9e3ad66825a7088e2f4843cebdf33c51ca4) - This is the commit right before the update to the 2.0.0 version of the SDK. You may want to do the same, as the raise hand feature will work on this commit.
Sorry for the delay - hope this unblocks you for the time being until we can make the necessary fixes. Thanks.

Hi, We are still having issue. This time error is different.
https://dpszoeh59ynwr.cloudfront.net/attendee?aId=0790f465-8fab-4a39-b71b-32a54d4da111&eId=LiveEvent-9YUI&name=Mohit Gupta 1

@Ashrma2021 Please check that you've create a completely new stack and bucket before trying again.

I'm able to see the raised hand in the moderator's view using the https://github.com/aws-samples/amazon-chime-live-events/commit/30c8f9e3ad66825a7088e2f4843cebdf33c51ca4 commit.

Also, You should make sure to enable the CORs extension.

I have tried attendee view with downgrading chime sdk as "30c8f9e", but still I am not able see the attendee view.

@vijayamin83 it seems as if the attendee URL works, but there is no video feed. In order to enable the video feed on the Attendee view, you need to follow a few more steps.
Make sure you have completed all of the steps here.
One more thing you may need to do is follow this guide.

@michhyun1 I have followed the all steps as you mentioned above. Also started the channel ID in MediaLive before broad casting. But still, I am not able to get the attendee video working.
When I check the related MediaPackage HLS stream it also not working. Please see the screen shot.

I have also checked the "30c8f9e" commit. But still getting the error.

You may want to double check the transcoding start lambda test event. You'll need to stop the transcoding and then start it again with the test event being:
{
  "action": "start",
  "meetingURL": "<your-meeting-url-to-be-broadcast>", <-- use the broadcast meeting URL
  "rtmpEndpoint": "<your-rtmp-endpoint-url>" <-- this should be part of your outputs
}

Ok I am able to run this broadcast lambda function, But stilll I am not getting the attendee view's video.
Also, can you explain the process of the connect/disconnect and raising hand process by attendee?

@michhyun1 Yes, I have found the flow of the raise hand.
But I am still don't get the attendee video when I test transcoding lambda function.
So, Let me explain you. I put my broadcast url in the test event and I also run the broad cast url in my local browser. So, is this something that will be broadcasted to RTMP endpoint?
Or Should I have to use the OBS software to stream the RTMP EndPoint separately?

I have confirmed that I am using the correct Media Live Channel ID. Also set the proper RTMP endpoint. But still getting the error.

@vijayamin83 Ah I see. You may not be seeing the video because you are joining with the broadcast URL. You shouldn't need to open the broadcast URL at all.

@michhyun1 I am not opening the broadcast URL in my system or any where. But still I can't see the video on the Attendee Video.
Another question, If broadcast url open in the EC2 on the fly, then which video/audio it will be broadcast?

@vijayamin83 that depends on which URL you use when you start the transcoding lambda:
{   "action": "start",   "meetingURL": "<your-meeting-url-to-be-broadcast>", <-- use the broadcast meeting URL   "rtmpEndpoint": "<your-rtmp-endpoint-url>" <-- this should be part of your outputs }
The meeting URL field should be the broadcast URL that should be part of the CloudFormation Outputs when you deploy your live events stack. Similarly, the rtmp endpoint should be the MediaLivePrimaryEndpoint in your CloudFormation output

@vijayamin83 Can you please create a new issue for the attendee video not showing - it is unrelated to the raise hand feature

creating the new ticket https://github.com/aws-samples/amazon-chime-live-events/issues/48


Split the HGGT dataset into several parts for downloading

Hi, thanks for your great job! Since the HGGT dataset is huge and Google Drive downloading always suspend, could you please split the HGGT training set into several parts?  This would help me a lot, thanks again!

PS. I think 5 to 10 parts would be enough.

So where are you from? Any other Cloud Stroage such as Aliyun Drive from the Chinese Mainland could you use? Since I have no more Google Drive space, it seems difficult to upload another 100GB data onto it.

I'm from  Chinese Mainland, and yes I can use Aliyun Drive. Thanks for your reply!

I think Aliyun/Baidu Drive would be more convenient for me. Could you please share the Aliyun/Baidu Drive download link if possible?

I think Aliyun/Baidu Drive would be more convenient for me. Could you please share the Aliyun/Baidu Drive download link if possible?

https://www.aliyundrive.com/t/bZMATXjtlrbAWUGJJS7f
阿里云快传链接。注意，这个链接有效期是1天，因为阿里云不允许我分享经过压缩后的压缩包，所以不能设置分享功能。因此，我基本都不会使用国内的各种网盘来分享数据，一个是分享功能受限，另外一个是传输速度过慢。百度云更不可能使用的。

I think Aliyun/Baidu Drive would be more convenient for me. Could you please share the Aliyun/Baidu Drive download link if possible?

https://www.aliyundrive.com/t/bZMATXjtlrbAWUGJJS7f 阿里云快传链接。注意，这个链接有效期是1天，因为阿里云不允许我分享经过压缩后的压缩包，所以不能设置分享功能。因此，我基本都不会使用国内的各种网盘来分享数据，一个是分享功能受限，另外一个是传输速度过慢。百度云更不可能使用的。

能再发一个吗

I think Aliyun/Baidu Drive would be more convenient for me. Could you please share the Aliyun/Baidu Drive download link if possible?

https://www.aliyundrive.com/t/bZMATXjtlrbAWUGJJS7f 阿里云快传链接。注意，这个链接有效期是1天，因为阿里云不允许我分享经过压缩后的压缩包，所以不能设置分享功能。因此，我基本都不会使用国内的各种网盘来分享数据，一个是分享功能受限，另外一个是传输速度过慢。百度云更不可能使用的。

能再发一个吗

我后面回大陆了想办法上传一下百度云，阿里云的快传链接限制太多了

I think Aliyun/Baidu Drive would be more convenient for me. Could you please share the Aliyun/Baidu Drive download link if possible?

https://www.aliyundrive.com/t/bZMATXjtlrbAWUGJJS7f 阿里云快传链接。注意，这个链接有效期是1天，因为阿里云不允许我分享经过压缩后的压缩包，所以不能设置分享功能。因此，我基本都不会使用国内的各种网盘来分享数据，一个是分享功能受限，另外一个是传输速度过慢。百度云更不可能使用的。

能再发一个吗

训练集目前已经上传百度云，请查看链接 https://pan.baidu.com/s/1Z5pO-gwV12tK_yoKK6vg6A?pwd=3uai


refactor: rewrite Dockerfile to pass SSH key in a legacy way

JIRA
Jira Ticket
Change Description
Page Screenshot
Work Checklist

[ ] Tests added where applicable
[ ] Test coverage of new code meets target of 80%
[ ] UI changes look good on mobile
[ ] UI changes meet accessibility criteria

Merge Instructions
We are committed to keeping commit history clean, consistent and linear.  A BREAKING CHANGE can be part of commits of any type,
types other than fix: and feat: are allowed, for example build:, chore:, ci:, docs:, style:, refactor:, perf:, test:, and others,
footers other than BREAKING CHANGE: <description> may be provided.

Closing as it was achieved through base image.


FIX - broken link in Doc ReadMe

Fixes https://github.com/Microsoft/Quantum/issues/8

Fixed in #3.


[TxRxResult] Port is in use! error working with XC330-M288T on ROS2 Humble

System Info

ROS2 Humble
XC330-M288T x1 in Position Control Mode
With Joint Trajectory Controller
Using branch humble
Using ros-humble-dynamixel-workbench*
Using ros-humble-dynamixel-sdk

I am encountering the following errors in my log
[INFO] [launch]: All log files can be found below /home/container_user/.ros/log/2024-05-11-14-00-18-474799-E14-7727
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [ros2_control_node-1]: process started with pid [7736]
[INFO] [robot_state_publisher-2]: process started with pid [7738]
[INFO] [spawner-3]: process started with pid [7740]
[ros2_control_node-1] [INFO] [1715416218.928573690] [robot.controller_manager]: Subscribing to '~/robot_description' topic for robot description file.
[ros2_control_node-1] [INFO] [1715416218.929964564] [robot.controller_manager]: update rate is 200 Hz
[ros2_control_node-1] [INFO] [1715416218.930277923] [robot.controller_manager]: RT kernel is recommended for better performance
[robot_state_publisher-2] [INFO] [1715416218.978141700] [robot.robot_state_publisher]: got segment wheel_link
[robot_state_publisher-2] [INFO] [1715416218.978346253] [robot.robot_state_publisher]: got segment world
[ros2_control_node-1] [INFO] [1715416219.144636252] [robot.controller_manager]: Received robot description file.
[ros2_control_node-1] [INFO] [1715416219.146690928] [resource_manager]: Loading hardware 'Wheel'
[ros2_control_node-1] [INFO] [1715416219.148623249] [resource_manager]: Initialize hardware 'Wheel'
[ros2_control_node-1] [INFO] [1715416219.149287259] [DynamixelHardware]: joint_id 0: 101
[ros2_control_node-1] [INFO] [1715416219.149321060] [DynamixelHardware]: usb_port: /dev/ttyUSB0
[ros2_control_node-1] [INFO] [1715416219.149333352] [DynamixelHardware]: baud_rate: 57600
[ros2_control_node-1] [INFO] [1715416219.213230954] [DynamixelHardware]: Position control
[ros2_control_node-1] [INFO] [1715416219.229221121] [DynamixelHardware]: Torque enabled
[ros2_control_node-1] [INFO] [1715416219.229490831] [resource_manager]: Successful initialization of hardware 'Wheel'
[ros2_control_node-1] [INFO] [1715416219.229742245] [resource_manager]: 'configure' hardware 'Wheel'
[ros2_control_node-1] [INFO] [1715416219.236202030] [resource_manager]: Successful 'configure' of hardware 'Wheel'
[ros2_control_node-1] [INFO] [1715416219.236272636] [resource_manager]: 'activate' hardware 'Wheel'
[ros2_control_node-1] [ERROR] [1715416219.240448750] [DynamixelHardware]: [TxRxResult] Port is in use!
[ros2_control_node-1] [ERROR] [1715416219.240558395] [DynamixelHardware]: groupSyncRead getdata failed
[ros2_control_node-1] [ERROR] [1715416219.240573689] [DynamixelHardware]: groupSyncRead getdata failed
[ros2_control_node-1] [ERROR] [1715416219.240585910] [DynamixelHardware]: groupSyncRead getdata failed
[ros2_control_node-1] [ERROR] [1715416219.240726423] [DynamixelHardware]: [TxRxResult] Port is in use!
[ros2_control_node-1] [ERROR] [1715416219.245845826] [DynamixelHardware]: groupSyncWrite addparam failed

I am however able to control the motor using Dynamixel Wizard 2.0

I seem to have solved this issue, it was a combination of below

Too high controller frequency

Was running at 200hz, the errors went away when brought down to 10hz

Latency Setting not setup according to u2d2 settings

Once the latency was set to 1ms, the max frequency jumped to around 40-50hz

I believe one of the reasons for this is that the read and write functions are called once init fucntion is done, if we can delay till after on activate function, this problems go away at 200hz+ on my system, will put a pr for the same

@soham2560 Did you ever get this to run fast at 50hz+


Update sql-server-on-azure-vm-iaas-what-is-overview.md

The Azure Portal GUI allows SQL Server 2014 SP3 SQL Server on Azure VM to be deployed, but the documentation says SP2 (see https://dev.azure.com/msft-skilling/Content/_workitems/edit/118749)

thanks @v-savila ! #sign-off

@v-savila @MashaMSFT - can you address the merge conflicts? we'll need those resolved before we can review and merge. thanks.

@MashaMSFT - after the merge conflicts are resolved, can you sign off again?
#hold-off

cc @MicrosoftDocs/public-repo-pr-review-team

Hi @v-savila thanks for your submission! I'm going to pull your changes into the private repository to address these issues :) Thanks!


An error occurred while trying to automatically install the required extension 'delta'

Now that DuckDB v0.10.3 has been released, I am trying to use this extension, but I receive the following error:
DuckDB Error

duckdb error: An error occurred while trying to automatically install the required extension 'delta':
Failed to download extension "delta" at URL "http://extensions.duckdb.org/v0.10.3/linux_arm64/delta.duckdb_extension.gz"

Candidate extensions: "md", "inet"

This may be triggered by:
Issue 1002 - The database returned an unexpected error.

I am on an Apple M1. Trying to run this through Superset which is running inside a Docker container.

I also tried "plain" duckdb from the CLI:
Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'delta':
Failed to download extension "delta" at URL "http://extensions.duckdb.org/v0.10.3/osx_arm64/delta.duckdb_extension.gz"

Candidate extensions: "md", "inet"

Same issue, just a different architecture-version.

@mvilrokx The extension was not yet uploaded, it should be available now!

I am still seeing the issue on linux_arm64:
DuckDB Error

duckdb error: An error occurred while trying to automatically install the required extension 'delta':
Failed to download extension "delta" at URL "http://extensions.duckdb.org/v0.10.3/linux_arm64/delta.duckdb_extension.gz"

Candidate extensions: "md", "inet"

ah my bad i did not fully read your error message, the extension is not yet supported on linux_arm64 as stated in the README. I've opened an issue for this to track progress: https://github.com/duckdb/duckdb_delta/issues/10

arm binaries are available from nightly now: force install delta from core_nightly


exception when using searchPattern

searchPattern: "/(?<=[\\w\\d ])\\n(?=[\\w\\d])/g"
results in:
search-replace Custom rule [This rule threw an exception: Property 'range' of onError parameter is incorrect.]
what's interesting is that the rule actually seems to work, it's just a problem with the output.  this rule is meant to catch linebreaks within a sentence.  e.g.
The following line
break should not exist.

@ctcpip From a plugin, we can't report error to markdownlint which spans multiple lines. The error range can't cross line ending.
As a workaround, when the plugin finds a multi line match it is split on \n and a separate error is reported for each line. So the match needs to have at least one character on both sides of \n. With following custom rule definition you can achieve what you want:
{
  "name": "no-hardwraps",
  "message": "Do not use hard wraps",
  "searchPattern": "/[\\w\\d ]\\n[\\w\\d]/gm",
  "searchScope": "text"
}

Note: Due to the limitation this will only report errors and will not be able to perform auto fixes.


Simple blue - Icon pack submission

Name of the icon pack
Simple blue
Version
1.0.0
Author of the icon pack
MidoMatrix
Description of the icon pack
A simple pack of blue icons, has media controls, on/off states for some and general use on/off switches.
Icon pack file
simple-blue-1.0.0.iconpack.zip

Hi, have you created the icons yourself?

yep, aside from the discord logo in the discord icons for mute and deafen, i created them all myself


more red icons

Problem:
High demand for red icons means we ran out of some of them.
Fix:
Make some more, upload them. Like so:

Merging since assets only.


fix: use host+pathPrefix indices

This fixes 2 issues:

Keda does not allow backslash in metrics' names. Replacing those with an underscore
The interceptor should have 'host+pathPrefix' values as the queue keys. This is because we now allow having multiple HttpscaledObjects having the same .Spec.Host value. Both Scaler and Interceptor treat these as required

Please review for https://github.com/kedacore/http-add-on/pull/677

Found it: https://github.com/kubernetes/kubernetes/blob/7f8c4315dce1b515a182409290a61692ee5602c6/staging/src/k8s.io/apimachinery/pkg/api/validation/path/name.go#L24-L46C2
Related: https://github.com/kedacore/keda/pull/1172
It seems we just need to use kedautil.NormalizeString. I'm also adding a prefix to it so we follow the pattern other scalers are using.

Actually, the NormalizeString implementation is not safe and it does result in collisions.


[perf][cluster-test] Set params for 100 node experiment to get 1000+ TPS

Summary

Set consensus max block size to 1000

In the future, allow setting this from experiment rather than hardcoding it

Increase ulimit for docker run from the default of 4096
Update EmitJobRequest::for_instances to set different params based on experiment type

@dmitri-perelman what do you think about setting default block size to 1000?
Should we try to re-run this experiment for 30 nodes cluster first and try to see what is optimal block size for this setup? 1000 txn per block seem a bit high, I am not sure if we want this as default

While a 1000 size block may or may not be ideal for benchmarking the 30 node cluster, the real limitation currently is that we don't have support for setting validator configuration from cluster test. Once we can support setting that from the tests, we can set this back to the original 100 number, which should be a trivial change.

To be honest I don't have preferences for the block size in the cluster test (you guys have run more tests with more combinations so you should now have a better intuition :))

Well thats the thing - this PR sets block size everywhere, not just for cluster test

Oh I see, got it. Let's try the large blocks: it seems to work fine for cluster test, and the block size seems to not matter for testnet today anyway (because there is not enough load to fully utilize these blocks).

Ok, thats what I would propose - @ankushagarwal would you be ok with merging this PR without block size change, and then find what is optimal block size for cluster-test-30 setup? (full nodes + 30 validators)?
I just don't want to set giant default block size for configuration that we are effectively retyring

Why do you think 1000 is a high number for max_block_size? We have data to support that this gives much higher performance on our current setup and we don't have enough data which illustrates that this max block size degrades the performance of a 30 node cluster.
While I am happy to run the experiments and get back to you, I don't see a reason to block this PR

Yeah, lets get some data for 30 nodes cluster - if 1000 is not ideal block size for that, I would prefer to have defaults in master that are optimal for 30 nodes cluster
@bors-libra delegate+

:v: @ankushagarwal can now approve this pull request

@bors-libra r=andll
I'll try to get the results by tomorrow on the 30 node cluster. My prediction is that a higher block size would generally lead to a higher throughput till we reach a breaking point at which latency spikes a lot. I think that sweet spot will probably not be lower than 1000 for a 30 node cluster.

:pushpin: Commit ccc21ff has been approved by andll

:hourglass: Testing commit ccc21ffec4c5867261e9b5c42249e17d9abe2bd5 with merge 65b9c572da820515d89d0eedf0fc70f627d49032...

:sunny: Test successful - checks-circle_commit_workflow
Approved by: andll
Pushing 65b9c572da820515d89d0eedf0fc70f627d49032 to master...


test: Handle closed event stream in TestDomainMigration

This was previously panicking if the events channel closed.
I've also added some debug messages and refactored a little bit.
/cc @jvatic

LGTM

:shipit:


feat: 🎸 Replace number with BigNumber

BREAKING CHANGE: 🧨 Updated all instances of number to BigNumber

:tada: This PR is included in version 13.0.0-alpha.24 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 14.0.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 14.0.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


package qtdeclarative 6.3 is failing to build

qtdeclarative 6.3.1 is failing to build

OS: Windows 10
Compiler: Visual Studio 2022

To Reproduce
Steps to reproduce the behavior:

./vcpkg install qtdeclariative:x64-Windows
See error
error: building qtdeclarative:x64-windows failed with: BUILD_FAILED Please ensure you're using the latest port files with git pullandvcpkg update`.
Then check for known issues at:
https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+qtdeclarative
You can submit a new issue at:
https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[qtdeclarative]+Build+error
Include '[qtdeclarative] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above.
vcpkg-tool version: 2022-09-01-dfb82802c8cc562ce3b665a904a65b22314de724
vcpkg-scripts version: 927006b24 2022-09-02 (6 days ago)

You can also use the prefilled template from C:\Users\ArshamA\vcpkg\installed\vcpkg\issue_body.md.`

no useable logs..... I would guess C:\Users\ArshamA\vcpkg is too long of a root dir for it to work.

log is attached.
install-x64-windows-dbg-out.log

The latest qtdeclarative version is 6.3.1.  I'm not able to repro this issue, could you please git pull the latest vcpkg and try again?

@Adela0814 this is a known issue if the build path is too long. see e.g. https://github.com/microsoft/vcpkg/issues/23714
C:\Users\ArshamA\vcpkg\buildtrees\qtdeclarative\x64-windows-dbg\include\QtQuickDialogs2QuickImpl\6.3.1\QtQuickDialogs2QuickImpl\private/qtquickdialogs2quickimplforeign_p.h(1): fatal error C1083: Cannot open include file: '../../../../../../src/here-src-6-b9307f657e.clean/src/quickdialogs2/quickdialogs2quickimpl/qtquickdialogs2quickimplforeign_p.h': No such file or directory

We hope your question was answered to your satisfaction; if it wasn't, you can reopen with more info.


🆕 Software Suggestion | Kleeppr

Basic Information
Name: Kleeppr
Category: Email, Newsletter
URL: https://kleeppr.com
Description
Kleeppr lets users subscribe to email newsletters without using personal or work email ids. On signup it creates random email and read-only inbox for a user. It provides a simple interface to browse, read newsletter feed and manage subscriptions.
Why I am making the suggestion
As newsletter scene exploded and became one of the primary sources of content, I grew more and more hesitant while giving away my personal email address to random mailing lists. So I created Kleeppr to solve my use case. Its simple concept and works well for newsletters and marketing emails (remember email coupon popups on virtually every e-commerce site).
My connection with the software
I am the author.

[x] I will keep the issue up-to-date if something I have said changes or I remember a connection with the software.

This website runs in debug mode of Django. This is a security issue.

@ph00lt0 Debug mode is off .. did you see something?..pls provide more info so that i can check

Kleeppr lets you subscribe to email newsletters anonymously.

Where can I find the Privacy policy?

@rusty-snake yes thats the link .. glad you found it!

@ph00lt0 Debug mode is off .. did you see something?..pls provide more info so that i can check

Previously when I tried searching it gave me the debug=true screen. I just tried again and now I get a normal Django CSRF bug page.
As @rusty-snake points out your website contains many trackers/tracking cdn's which make it inconsiderable for PTIO.

@ph00lt0 Debug mode is off .. did you see something?..pls provide more info so that i can check

Previously when I tried searching it gave me the debug=true screen. I just tried again and now I get a normal Django CSRF bug page.
As @rusty-snake points out your website contains many trackers/tracking cdn's which make it inconsiderable for PTIO.

Debug mode was never on.
Info @rusty-snake provided is from uBlock Origin, and apart from google analytics it did not flag anything.
Kleeppr helps you interact with mailing lists without your personal email id. Let's first focus on that use case and its utility.

@sandeepbele Calm down a bit, I have seen them in other issues and they are not doing anything in bad faith.
IMO besides G Analalytics you also have connections to G Tag Manager and the rest of the connectios for them to work (G Static and google.com), plus the connection to CloudFlare might also be a concern for some, even if the rest of the project is awesome. SimpleLogin started the same way and now they don't have prorpietary dependancies anymore.

Info @rusty-snake provided is from uBlock Origin, and apart from google analytics it did not flag anything.

@sandeepbele I can't review your setup man, I have definitely seen it and unfortunately can't provide you more info.
I do consider bootstrapcdn, gstatic, netdna, and fontawesome as trackers as well.

@ph00lt0 @rusty-snake @LongJohn-Silver Thank you for taking time and commenting. I will consider your feedback. Appreciate pointer to SimpleLogin, sounds like amazing service in similar spirit.

@sandeepbele It's nothing! If you want I can recommend some alternatives to those dependencies that are privacy respecting.

@LongJohn-Silver Sure, that will be great!


add build profile

add build profile

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

merge


[DOC] Fix wrong value and typo in model zoo doc

Description

Fix a wrong value in bert/index.rst
Fix typo in language_mode/index.rst

Checklist
Essentials

[ ] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage
[ ] Code is well-documented

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

I changed the target branch to master branch.
We can backport after merge.
Did you verify all other numbers are correct? During previous review I only checked on a sample and missed this one.

@leezu Yes, I double checked and there are good.

Job PR-1150/7 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1150/7/index.html


[]/.md open []/.md: The filename, directory name, or volume label

https://github.com/Snowflake-Labs/sfquickstarts, trying to run through windows
npm run serve

serve
gulp serve

[17:13:04] Using gulpfile ~\Desktop\sfquickstarts\site\gulpfile.js
[17:13:04] Starting 'serve'...
[17:13:04] Starting 'build'...
[17:13:04] Starting 'clean'...
[17:13:04] Starting 'clean:codelabs'...
[17:13:04] Starting 'clean:build'...
[17:13:04] Starting 'clean:dist'...
[17:13:04] Finished 'clean:codelabs' after 7.05 ms
[17:13:04] Finished 'clean:build' after 7.35 ms
[17:13:04] Finished 'clean:dist' after 7.31 ms
[17:13:04] Finished 'clean' after 8.17 ms
[17:13:04] Starting 'build:codelabs'...
[17:13:04] Starting 'export:codelabs'...
err     []/.md open []/.md: The filename, directory name, or volume label syntax is incorrect.
[17:13:04] 'export:codelabs' errored after 65 ms
[17:13:04] Error: 1
at ChildProcess. (C:\Users\rajesh.pasupuleti\Desktop\sfquickstarts\site\tasks\helpers\claat.js:21:13)
at ChildProcess.emit (events.js:315:20)
at ChildProcess.EventEmitter.emit (domain.js:529:15)
at maybeClose (internal/child_process.js:1048:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
at Process.callbackTrampoline (internal/async_hooks.js:131:14)
[17:13:04] 'build:codelabs' errored after 66 ms
[17:13:04] 'build' errored after 75 ms
[17:13:04] 'serve' errored after 76 ms

Hi @rajeshpleti
Have you found a solution/workaround ?
thanks


Auto suggestion and Error Management

I don't know about others but i am feeling lag its not that smooth sometime and their should be something for error management

Could it be that you are experiencing the initial, first time only lag when auto complete/suggestion is updating?
Other than that Fisherman should always outperform other systems, is just as fast as having nothing at all, but the first time you try to auto complete, there will be a small lag as fish updates the auto complete so that you can do things like:
fisher install <tab> [ packages you can install show up here ]
fisher update <tab> [ packages you can update show up here ]
fisher uninstall <tab> [ packages you can uninstall show up here ]

Notice that is not as trivial as Wahoo or Oh My Fish! because Fisherman generates completions for all plugins, those in the Index as well as those you install via a custom URL.

Major speed improvement to land in 0.3.0. Thanks for suggesting this @namandistro.


tapdb: Federation DB fixes

Fixes multiple issues with sync config load and store.
The first issue was the federation DB handle not returning the correct global config when the global config is modified for one proof type - this is resolved by loading the default config, and keying configs by proof type. Configs read from the DB then replace the default config with the matching proof type.
The second issue affected grouped assets, and led to group member asset IDs not being shown on the CLI.
The third issue was the UNIQUE constraint on universe sync configs not being enforced, since NULL values are distinct in UNIQUE constraints by default for SQLite and Postgres. This could lead to a bad configuration state with multiple entries for the same universe. This behavior is not configurable for SQLite, so extra columns were added that have a default value when one field of the universe ID is NULL. This should fix the DB constraint.

Needs a rebase.

We can also add a migration to DROP any contents in this table. Since no deployment should have a local Universe config set, including the Universe servers we run, this should not be an issue.

I think this could work. Drop the old, then add on new.

Verified locally on mainnet setup with the original issue that old local configs were dropped and upsert now works via CLI.


 See my pocket feed compared below

and

Let the edges of the images align with the text rather than indented.

If this doesn't work very well I think we could nick a bit from the margin space to make the images bigger.
Larger images would not only look better, but the boxes would also not need to be quite so deep because the text would   have more width too...
Clearly this isn't urgent but I think it's worth a little play around at some point. I still like the front page a lot....

the edges of the images align with the text rather than indented.

I don't understand that bit.
If we scale the images they're all going to look a bit blurry.  They are currently 300x185 pixels and are saved at that resolution.  Should we start making images that are bigger?

Ah Ok in that case not to worry. otherwise we would need to regenerate all the images.


Try diffusion models

Since the current VAEs and GANs don't look satisfactory, you might want to try some diffusion models. Hugging Face's diffusers will be a good starting point.

I tried the following tiny model from diffusers, but the optimization step takes ~1s per batch of 64 images on my Apple M1 PC. Perhaps you might want to define a lightweight architecture on your own.
from diffusers import UNet2DModel

UNet2DModel(
    sample_size=64,
    in_channels=1,
    out_channels=1,
    layers_per_block=1,
    block_out_channels=[32, 32],
    down_block_types=["ResnetDownsampleBlock2D", "ResnetDownsampleBlock2D"],
    up_block_types=["ResnetUpsampleBlock2D", "ResnetUpsampleBlock2D"],
)


Add rerun-if-changed to libgit2-sys build script.

pkg-config 0.3.18 changed the default of env_metadata to true (see https://github.com/rust-lang/pkg-config-rs/pull/105), which means it is now emitting rerun-if-env-changed directives, which means the default Cargo behavior of scanning the entire directory is no longer working. This means that changing anything in libgit2 will not trigger a build.
This fixes it by adding rerun-if-changed directives for libgit2.

@alexcrichton I've always wondered why if you pass a directory to cargo:rerun-if-changed, why it doesn't interpret that as "anything within this directory"?  Would there be any reason not to change that behavior?  It would allow avoiding needing the recursive function which I added here.

We debated it a bit when it was first implemented in Cargo but figured it was conservative not to do it. I don't think there's any reasonable portable interpretation though other than "everything in here", so I think it would be fine to implement nowadays in Cargo.
Thanks for taking the time to do this!


Translated a couple of french words, removed unecessary colons

Working on the french translation. I've also removed the unecessary colons in the translations already done, and I've added two notes on translations on which I'm not sure of the accuracy.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


ETag header must be enclosed in double quotes

(according to https://tools.ietf.org/html/rfc7232#section-2.3)
This closes #2053

Pull Request Test Coverage Report for Build 5000

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 54.405%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

bundle/src/main/java/com/adobe/acs/commons/etag/impl/EtagMessageDigestServletFilter.java
0
1
0.0%

Totals

Change from base Build 4984:
0.0%

Covered Lines:
14375

Relevant Lines:
26422

💛  - Coveralls


As a developer, I want to implement serving incoming RPC requests and test for incoming bootstrap, so our node can bootstrap peers

Status 18.03.
Convert internal request/response into libp2p (wire) format. Implement missing reducer/effects. Fixed few bugs in RPC protocol implementation.
Still not working, probably Yamux incoming stream is buggy.
TODO: Remove BTreeMap of streams, fix security issue that a malicious peer could create an arbitrary number of streams and overflow memory.

List of fixes:

Do not select multiplexing protocol if already have selected.
Fixed error in unoptimized noise incoming handshake.
Fixed enabling conditions for RPC bootstrap.
Fixed message counting.

TODO: test. There is an issue with the testing framework.

Fixed debugger to work with newest openmina.
Fixed ledger service in test.
Merged the PR (bootstrap incoming peer).
One problem left: wrong MAC during testing. This can be a serious problem that can happen in production (not just testing).


Make GetStatisticsStartDate in Table:Intrastat Jnl. Batch External

The function:GetStatisticsStartDate of Table:Intrastat Jnl. Batch is used in Report:594.
For our App we need to change this report. So made a copy :( of this report but using GetStatisticsStartDate isn`t allowed because it isn't external.

I will expost that function for you. Thanks for reporting!


ExpensiMark: allow trailing whitespace in blockquote in edit mode

Required by #27977.
Problem
Currently, ExpensiMark trims all trailing spaces and newlines in blockquote. However, when typing a new message, we want to persist trailing whitespace characters so that the user is able to write new words and proceed to the next line.
Solution
Add a flag named shouldKeepWhitespace (disabled by default) to replace method and persist trailing whitespace when the flag is enabled.
Test cases

Trailing space

 Input: "> Hello world "
-Current output: "
Hello world
"
+Desired output: "
Hello world
"

Trailing newline

 Input: "> Hello world\n"
-Current output: "
Hello world
"
+Desired output: "
Hello world

"

https://github.com/Expensify/App/assets/20516055/48ea3919-342a-450c-991d-849aec063df4

Hi I’m Michał from Software Mansion, an expert agency, and I’d like to work on this issue

Posted a thought about potentially reassigning this: https://expensify.slack.com/archives/C04878MDF34/p1703718815187329

@roryabraham, this issue was addressed by https://github.com/Expensify/expensify-common/pull/602. I can't track the following PR for expensify/App, but based on the last version bump in package.json (DEC 15th), I think we can assume that the change from https://github.com/Expensify/expensify-common/pull/602 is already deployed.


Update nix psql_exe path so that it's aware of per-user profiles.

Summary
I made the path in scripts/psql-wrapper match the change that happened here
Setup to Run Your Code
Terminal 1
Run database setup
make db_dev_reset

Verification Steps for Reviewers
Does this still work for single user installs of Nix?

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against dbb87391e12f57bd3fe01d3454787a993f077194


Referenced schema could not be loaded when defined in external schema file

Hello,
I'm consuming an API from a system which define components in multiple schemas/files. The problem is when one type from a schema used for C# client code generation reference type from another schema and this type reference third type which is in second schema. I made simple.
Schema with consuming API
{
    "openapi": "3.0.1",
    "info": {
        "title": "Simple API",
        "version": "1.0"
    },
    "paths": {
        "/api/test": {
            "get": {
                "tags": [],
                "operationId": "GetWeatherForecast",
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/Foo"
                                }
                            }
                        }
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "Foo": {
                "type": "object",
                "properties": {
                    "bar": {
                        "$ref": "./referencedSchema.json#/components/schemas/Bar"
                    }
                },
                "additionalProperties": false
            }
        }
    }
}

Second schema saved in same directory in file referencedSchema.json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Common API",
        "version": "1.0"
    },
    "paths": {
       //.......
    },
    "components": {
        "schemas": {
            "Bar": {
                "type": "object",
                "properties": {
                    "language": {
                        "$ref": "#/components/schemas/Language"
                    }
                },
                "additionalProperties": false
            },
            "Language": {
                "type": "object",
                "properties": {
                    "code": {
                        "type": "string",
                        "nullable": true
                    }
                },
                "additionalProperties": false
            }
        }
    }
}

An exception throw from Nswag is

System.InvalidOperationException: Error while rendering Liquid template CSharp/Class:
The schema reference path '#/components/schemas/Language' has not been resolved.
---> System.InvalidOperationException: The schema reference path '#/components/schemas/Language' has not been resolved.

How to instruct NSwag/NJsonSchema to use all types from all referenced schemas?
Thank you.

I'm currently facing the exact same issue. Any work-a-rounds known for this issue?


Sugarizer Server Sisters

A number of other projects very similar to Sugarizer Server exist.
It would be great for them to be integrated with Sugarizer Server in some way.... Here's the 2 I'm aware of so far:

http://solarspell.org
https://learningequality.org/ka-lite/

I know KA Lite, I used it to develop the KA View activity currently included in Sugarizer.
But I find KA Lite too complex both for users and to deploy on a server. My wish was only to have a browser to explorer Khan Academy videos and had a way to just copy videos somewhere on a basic HTTP Server. I had to deconstruct KA Lite to finally do that and just point on a server where video are deployed (see https://github.com/llaske/kaexplore for more).
BTW Sugarizer Server is not today a content manager. For example, you could deploy Khan Academy videos on a Sugarizer Server using an Apache root but it's not managed directly by Sugarizer Server. Transforming Sugarizer Server into a deployment server where teachers could deploy content to give it access to learners is interesting but it's not in the feature lists for the moment.
Bastien in CC https://github.com/bzg

Transforming Sugarizer Server into a deployment server

Hmm. That's not quite what I had in mind... but that phrasing gives me another idea :)
www.Sandstorm.io is a deployment server system, a good one.
Perhaps Sugarizer Server should be packaged as a Sandstorm.io package (SPK)?
Then if KA Lite and SolarSpell are also packaged as SPKs, it would be easy for schools to run them all.

Deployment server was not the right word that I had in mind. I'm thinking more about a "content server".
I'm not a big fan of the approach of mix tools together without building a real bridge between them.
Sure it could be easy with tools like www.Sandstorm.io but it's easy to deploy, not easy to use !
If a teacher have plenty of tools to admin  (Sugarizer Server, KA Lite, Solar Spell, ...) with each time a different UI, a different users administration, a different set features ... it will be very complex for him to use it.

Sure. Many schools will also be running Moodle or something like that too, I guess.
How many schools are using Sugarizer Server today? :)

Hmmm.
I guess: none :-)

hehe :) Are the stats for server.sugarizer.org available?

Not yet. It was precisely what I've described in the GSoC project "Sugarizer Dashboard Server".
But because we don't have a token for that this year, @mikklfr tell me that it could work on it ;-)

Okay cool :) Are any stats being collected for later analysis?

Very few. Hope that it will be improved with the start of the Dashboard project.

Okay cool :)


golang extension: fix loaded_ may be overwritten unexpectedly.

Commit Message:
fix loaded_ may be overwritten unexpectedly.
fix #25757
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/review @wangfakang

/assign @wbpcode
Could you please take a look? Thanks!


Virtual environment (Venv)

[ x] I'm sure this issue is not a duplicate.

[x ] I'm submitting an icon request:

Type: python, extension
Icon Name: venv
Sample original Icon:
Language ids: python

Language Extensions:

More info: This is commonly used and can just be a folder with the python logo at the bottom, or whatever you guys can come up with

.venv folder icon is already implemented


Re-add setup.py

As per the discussion on #334 here is a pull request for re-adding setup.py. Sorry for the different accounts.

Fixed by Allan. So closing


Shader compiler traverser doesn't handle conditionals with uniforms for D3D

If you have a shader with something like:
uniform bool isFoo;
...
vec4 outColor = isFoo ? foo : bar;
// or
vec4 outColor;
if (isFoo)
    outColor = foo;
else
    outColor = bar;

This error gets thrown:
https://github.com/BabylonJS/BabylonNative/blob/2227c1ce4f197c585505f555595274b206d51bad/Plugins/NativeEngine/Source/ShaderCompilerTraversers.cpp#L70
This is because the traverser hits a parent node of type TInterimSelection, which has a condition that contains the uniform that needs to be replaced with a struct member.
Possible solutions include finding a way to properly replace the symbol in makeReplacements (by adding something like else if (auto* branch = parent->getAsSelectionNode())), or potentially adding another special case in visitSymbol.

Ping @syntheticmagus and @Drigax

Ping @syntheticmagus and @Drigax

Cool, thanks for the info. Looking into this now, I have a test scene here: https://playground.babylonjs.com/#QPA82A that I'll be using to try to hit these paths in the transpiler.


Bump Hive 2.3.10

:mag: Description
Hive 2.3.10 is available now, and it will be the latest version of the 2.3 serial.
Types of changes :bookmark:

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪
Pass GA.

Checklist 📝

[x] This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 58.39%. Comparing base (90491fc) to head (73d4fdc).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #6328      +/-   ##
============================================
- Coverage     58.42%   58.39%   -0.04%
  Complexity       24       24
============================================
  Files           653      653
  Lines         39834    39834
  Branches       5477     5477
============================================
- Hits          23274    23260      -14
- Misses        14073    14086      +13
- Partials       2487     2488       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Merged to master/1.9


Add customised 404 page

I want to add customised 404 page with well designed UI and with graphics or illustration and with some appropriate information like we should inform the user that he has jumped or redirected to wrong page or the page he is trying to find is not available.
Please assign this issue to me under GSSOC '22.
Thanks!

Go ahead @rsinghcodes


Title can not be displayed

Title can not be displayed on android

Can you please elaborate your problem?

The title prop for the View component never gets displayed.


Include tests

Description
There are currently no tests, as a user of the package I would like the package to be well and fully tested. Mainly to ensure the code consistently works and no unexpected breaking changes are pushed.
Requirements

[ ] Add tests to the codebase
[ ] Reach 100% test coverage
[ ] Include workflow to automatically check all tests are passing

Further context
This package is currently being recommend as part of a Dart Frog tutorial. It would be ideal if this package had test to guarantee a certain level of quality.

Thanks for the package, would you be willing to accept external contributions that add tests to the codebase?

Hey @alestiago !
I didn't know that Dart Frog recommended this package, very glad to hear it!
External contributions are always welcome! I'll try to set up a some tests by the end of the week and after that you'll be able to add your own tests to reach 100% of test coverage.
I've already started a workflow to auto-deploy the package, at the same time I'll make one more to run the tests.

@alestiago
I wanted to let you know that I've just set up the first tests. These are still basic tests that only test the 2 main functions sign & verify with all available algorithms but I may add more soon, you're also free to add your own.
In the meantime, I've created 2 GH actions:
one that will analyze the code (formatting & litering) and run tests for each new PR and PR change,
and another that will publish the package on the new tags (also running the CI analyze and test beforehand).


noms sync to roll back 1 commit when there are may is slow

to repro:
noms sync http://demo.noms.io/ahl::bug2213 /var/tmp/test::biggie
(wait for a long time)
noms sync /var/tmp/test::o38hugtf3l1e8rqtj89mijj1dq57eh4m /var/tmp/test::biggie
This should be fast, but is a lot slower than this:
noms sync /var/tmp/test::o38hugtf3l1e8rqtj89mijj1dq57eh4m /var/tmp/test::entirelynew

@rafael-atticlabs
After looking at this, I realized that we actually currently explicitly disallow this! Noms only allows fast-forwards, and this is actually a rewind. That said, we probably want noms sync to allow this sort of behavior.
You can currently work around this by doing
noms ds -d /var/tmp/test::biggie
noms sync /var/tmp/test::o38hugtf3l1e8rqtj89mijj1dq57eh4m /var/tmp/test::biggie

Essentially, delete the dataset and then recreate it with the Commit you want at the head. I am going to work on making this case not take so damned long, though, and figure out how noms sync should handle this.


Animations with panan configs

ACCESS-NRI is looking to have some spinup projects to develop capacity in producing animations.
Our suggestion here is that we could attempt to re-make the first half of the Circulation in the Southern Ocean movie using output from higher resolution panan simulations -- ultimately the 1/40° case. This work will be done by Owen Kaluza (@OKaluza) who has recently joined the ACCESS-NRI team.
I'm proposing this issue to kick off the discussion. The process I would be thinking of would be:

[ ] Get used to the data format using existing snapshots or time averages.
[ ] Learn how to plot density interface on top of bathymetry
[ ] @willaguiar save some data showing speed on an isopycnal surface to produce still images, and plot that
[ ] Start to save high-frequency output for a year
[ ] Produce a full animation.

Suggestions on the viability of this, or where to start, are very welcome.

Very interested in seeing how these movies would look like in the 1/40th. We would use the branched last year simulation of panan0025-ryf with high frequency output speed (u,v) for these movies, right?

Actually, I think you might actually be able to write speed itself on the isopcynal. But we can start by playing around with just showing isopycnal surfaces in 3D?

In addition to the speed on density surfaces, another cool animation would be to take a single snapshot of the zonal velocity and animate movement through longitude around Antarctica. That is, to show meridional-depth sections of u(y,z) with x-animated. Or, one might be able to do a 3d perspective with different shading to depict the current in all of its jets and filaments.
The structure of the ACC is is fascinating, with huge jet-like currents extending from the top to the bottom.  I can imagine that seeing it in high detail would produce "wow" responses, at least from oceanographers who study the current.
Indeed, there is interesting science related to the ACC jets, which can be both eastward and westward, along with the Antarctic Slope Current.  I am genuinely curious to see what these jets look like in this model.

Here is some plots with possible density layers we could output in high frequency for the ACCESS-NRI animations and/or video.

Ross Sea:
From top to bottom:  top: Monthly transects of density along -190 West ($\sigma_2=37.14\ Kg\ m^{-3}$ , gold line), bottom: Bottom Depth of the $\sigma_2=37.14\ Kg\ m^{-3}$ surface, in June 2000 of panan0025, superimposed over bathymetry surface (Shades of black/grey). I could only find an isopycnal that connects the shelf to deeper layers in June ($\sigma_2=37.14\ Kg\ m^{-3}$ , gold line), because if we go much lighter than that, the isopycnal just don't go deep enough offshore. This is probably an issue of the monthly mean, as I suspected we would be able to see denser isopycnal along the slope in outputs with higher frequency.

Weddell Sea:
Plots are the same as the ones done to the Ross Sea. Here a deeper isopycnal seem to have no trouble representing the overflow along the slope(  $\sigma_2=37.17\ Kg\ m^{-3}$).

Based on that, I think we could try to output maybe ~10 layers between 37.14 and 37.175 $Kg\ m^{-3}$. What do you all think? Let me know if you think we could explore additional plots/locations

Just to recap on today's discussion:

We think that 37.18 is probably the right isopycnal (based on looking at a super-slow ncview version of daily $\sigma_2$ volumes).
Just in case, we will also save 37.16 and 37.2 as well.
Plan is to save layer thickness and (if possible) speed on the isosurface at 3-hourly temporal resolution.
Also, will save speed at ~30m depth, SSH, SST(?) and SIC.
Is that all??
Initially, we'll just try a 5-day segment.

@OKaluza  - Once this done this will give you something to start playing with.

Update....
I have ran the model, with 3 hourly output for about a month now, and was able to output 10 days at a time, with all variables above. It would take about a week to run a full model year. I am stopping the run now to save some resources, but as the week goes by if we have too many MSU's left on oz91 or x77 before the end of the quarter, I will continue the run.
A few additional info in case you want to plot something while I am running...
1- There was no output for speed itself in the model, so I saved U and V in the $\sigma_2$ levels we mentioned above, and in the following z levels 25.6, 29.4, 33.6 m.
files should be in /scratch/oz91/wf4500/panan_Hifreq_formovies/wf4500/mom6/archive/mom6-panan/output{108:110}. Later outputs (>110) will have the tiles collated as I run, so I recommend to only look at these rn, while I continue the run.
or, few free to use the database below (I will update with the collated files as I run)
database:/home/156/wf4500/databases/access/panan_HF.db , experiment name mom6-panan

Nice - thanks @willaguiar.
One wrinkle that has me worried. We go up to 1037.2 as our lowest layer, but we don't know how thick the layer below that is. Likewise for the upper layer. So, I'm worried that we won't have the diagnostics saved to re-reate the total height of the interface (relative to either the surface or the bottom).
What do you think?

Make sense. I looked up for other diagnostics like mean depth of isopycnal but couldn't find any for MOM6.  I guess the alternative is to save all layer thicknesses till the maximum $\sigma_2$ (21 levels), or save daily $\sigma_2(z)$ for conversion (Which would likely use much more memory)?
Ill run a test to check if we can save 21 $\sigma_2$  levels on the frequency we need without hitting the memory ceiling

I think if you just save a very deep isopycnal (like 1038) then you will get all the depths below the deepest interface -- you can then add this to topo to get total height.

update: I added a deeper 1038 layer as suggested, and it seems to work ( shows a thickness everywhere in the model domain, from 1037.25 until 1038).

Update @OKaluza
I just finished running ~4 months of the 3 hourly data. The data should be in /scratch/oz91/wf4500/panan_Hifreq_formovies/wf4500/mom6/archive/mom6-panan/ , or if you prefer to access through a Cosima database, you can use /home/156/wf4500/databases/access/panan0025_HF.db, experiment mom6-panan.
Let me know if you have any trouble accessing the files


algorithms generator

Can use this framwork for algorithm app generator ?

I need more input data to answer your question


Configure: tolerate slash-style pass-through options.

Even though Visual Studio compiler accepts dash-style command-line
options, slash-style are likely to appear more natural to user. In
addition recognize /link option that collects following slash-style
options to link command line. Even though dash-style would work even
here, we adhere to slash-style in order to preclude slashes with
other systems.

+1

so @dot-asm you can merge this now if you want or address @levitte's issue.

I find that I am feeling quite strongly about my issue.
Enough to implement it, come to think of it...

Sorry about delay.

I find that I am feeling quite strongly about my issue. Enough to implement it, come to think of it...

You mean as separate request? I'd still argue that it makes sense to tolerate slash-style the way it's suggested here, just for consistency/fairness sake. I mean if it's possible to pass dash-style, then slash-style should work too. As for /link taking special treatment. Yes, one can argue that it's ugly, but at the same time it should appear quite natural to a VS user, which make it reasonable compromise...

There's more than that.

Why single out the linker?
Why make it so that only one tool can use this at all (i.e. eating up the rest of the config command line)

Quite plainly, I can't see this as a complete implementation. It raises many questions and answers few.

Why single out the linker?

As already mentioned, because it would appear more natural to VSC user. I mean since we allow users to intermix Configure arguments with target compiler ones, we kind of suggest to use their prior knowledge and intuition, right? And VSC users would(*) recognize cl /compiler-option file.c /link /linker-option pattern...

Why make it so that only one tool can use this at all (i.e. eating up the rest of the config command line)

It doesn't eat the rest of the config command line, only those that start with slash, so that you can do for example perl Configure VC-WIN32 /arch:SSE /link /subsystem:console,5.1 --debug...

I can't see this as a complete implementation.

It's not. One calling to intuition never is.
(*) it might be "should" instead of "would", because after all most typical VCS user doesn't work with command line.

Oh! Here is a crazy idea. What if each target could define own options? I mean we have bunch of --this and --that options that are common for all targets. Why don't allow individual target define target-specific --something-else? Well, we probably know answer to that question, and it's "pass additional options on Configure command line", i.e. "use your intuition"... :-)

Mm, ok, thanks for pointing out the things I missed...  I'm still skeptical of this change, because it's becoming confusing (actually already is, leading to tickets like this).
I still like the demand of explicitness more, and would really like to replace our current "intuitive" approach with a more explicit one, rather than adding on to the "intuitive" approach.
But at the very least, would you please document this?

(actually already is, leading to tickets like [https://rt.openssl.org/Ticket/Display.html?id=4664]).

Not quite... I would argue that that ticket was specifically about making life easier for specific kind of users, package builders/maintainers. They want to better harmonize our build infrastructure with the way they build packages. They would rather have way to override specifically release or debug flags in their entirety than to add something to them. In other words they want line of distinction between flags we require and flags they have complete control over. While what we are talking about here is custom tailoring for specific usage cases, not packages that are used by whole bunch of different users.

Ping @dot-asm, @levitte. What are we doing with this change? Closing it?

I'm working on the alternative proposal with full support for arguments like --cflags, --ldflags et al (with our current model retained, so it's going to be a mix)


Export IRC Publications content into CSV

In preparation for exporting the old IRC content into the new Hub:

Please indicate whether resources on either of the following pages can be exported into a CSV file (If not, these will have to be imported into the Hub manually.):

Publications on Campus Sustainability - http://www.aashe.org/resources/publications-campus-sustainability
Publications from AASHE - http://www.aashe.org/resources/publications-aashe

Please create a CSV or Excel file showing exactly how the new Hub fields should be displayed in the csv that will be used to be imported into the Hub. Please use the current Hub content for Publications  (http://aashe-hub-dev.herokuapp.com/browse/types/publication/) as an example to generate this file.

Considerations:

How to display multiple organizations, topics, keywords, etc. - single column or separate column for each - if single column, what kind of separator?
Need to ensure that major changes to submission forms are done before completing step 2 above.
Is it possible to automatically migrate over the uploaded files? If so, we need to change the file pathways (change http://www.aashe.org/files/resources/student-research/2009/nac_leed_brochure.pdf to http://www.aashe.org/files/hub/case-study/2009/nac_leed_brochure.pdf)
If possible, include the date and time the IRC resource was submitted and/or approved.

@charris2015  this looks like just a parse of the html on http://www.aashe.org/resources/publications-campus-sustainability
With the others from the second link (http://www.aashe.org/resources/publications-aashe), find those that aren't simply html pages stored in drupal. those that are flat HTML we don't really make sense to parse.
Converting from HTML to CSV works sometimes, but if the HTML is variable, then it doesn't make sense and we'll have to do things manually.

hey @monika-aashe I've sent a spreadsheet of all the publications content from http://www.aashe.org/resources/publications-campus-sustainability organized by category. still working on the export for http://www.aashe.org/resources/publications-aashe but will be sending those shortly.
-Chris

Hey @monika-aashe ,
I've sent an export containing the additional exports. The content is broken up by title, so each different publication fills it's own column.
-Chris

Hey @charris2015
I just realized we also need to fold all of the student research resources into the publications database. I will also need an export of the following - there are 198 total resource entries: http://www.aashe.org/resources/student-research
Please include all of the information provided in these pages, including publication date, uploaded files, images, etc. Thanks in advance!

Hey @charris2015 - Any updates on this? Here is the list of fields needed for this export:

Make sure there are 198 total entries
Title
Type of Paper
Institution
Other Organizations
Disciplines
Admin Depts.
Keywords
Date (year and month)
Abstract
File upload
Supplemental Materials 1
Supplemental Materials 2
Supplemental Materials 3...
First Author (first name, last name)
Second Author (first name, last name)
Third Author (first name, last name)
Additional Authors (text field)


[Feature] Service instance dependency

Please answer these questions before submitting pull request

Why submit this pull request?

[ ] Bug fix

[x] New feature provided

[ ] Improve performance

Related issues
#3829

New feature or improvement

Storage ServiceInstanceRelationServerSideMetrics and ServiceInstanceRelationClientSideMetrics
Add and update  ServiceInstanceInventory#mappingServiceInstanceId
Add getServiceInstanceTopology(clientServiceId: ID!, serverServiceId: ID!, duration: Duration!): Topology  to TopologyQuery

getServiceInstanceTopology(serviceIds: [ID!]!, duration: Duration!): Topology to TopologyQuery

This is not right. For service instance relationship, it should not be defined as topology. The backend should allow query instance relationships when given two specific service IDs. Don't consider it as same as the old topology for service. They are different.

When both serviceA -> serviceB and serviceB -> serviceA calls exist in the topology,  how do I handle it?

works like this:

I will review this today.

I verified es6 and es7 and found no problem.

Do you really have two instances of projectB?

Do you really have two instances of projectB?

Yes.

First, codes look good enough, but I also prefer others could review too. @kezhenxu94 Do you have time to do this?
@arugal For you, I think you need to update a document about storage extension. https://github.com/apache/skywalking/blob/master/docs/en/guides/storage-extention.md. I think you add new APIs, could you run and check whether we miss other APIs?

@arugal For you, I think you need to update a document about storage extension. https://github.com/apache/skywalking/blob/master/docs/en/guides/storage-extention.md. I think you add new APIs, could you run and check whether we miss other APIs?

I will check.

@wu-sheng @kezhenxu94  Need to update the e2e test?

Need to update the e2e test?

Yes. We should. But maybe after @kezhenxu94 basic code review?


Re-enable GroupIntoBatchesTest.testWithShardedKeyInGlobalWindow

Autosharding support for Java pipelines is now fixed on Dataflow. Un-sickbays GroupIntoBatches tests, fixes #25675.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @damccorm @kennknowles

Run Java Dataflow V2 ValidatesRunner Streaming

Run Java Dataflow V2 ValidatesRunner Streaming

(trying this again since on the first run we ran into lots of quota issues)

This test is still failing after re-enabled: https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/
Are there additional change need to be made? @danielarn

Still permared after test re-enabled: https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/
error message:
java.lang.IllegalArgumentException: Runner determined sharding not available in Dataflow for GroupIntoBatches for non-Streaming-Engine jobs. In order to use runner determined sharding, please use --streaming --enable_streaming_engine
	at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:141)
	at org.apache.beam.runners.dataflow.DataflowRunner.maybeRecordPCollectionWithAutoSharding(DataflowRunner.java:1644)
	at org.apache.beam.runners.dataflow.GroupIntoBatchesOverride$StreamingGroupIntoBatchesWithShardedKey.expand(GroupIntoBatchesOverride.java:335)
	at org.apache.beam.runners.dataflow.GroupIntoBatchesOverride$StreamingGroupIntoBatchesWithShardedKey.expand(GroupIntoBatchesOverride.java:314)
	at org.apache.beam.sdk.Pipeline.applyReplacement(Pipeline.java:576)

shall we lift the check argument at the exception path?

The failures are non-v2, so this seems different...

The change is made for V1 validate runner test task "validatesRunnerStreaming". If it itended to re-enable v2 test then should change here: https://github.com/apache/beam/blob/7a060c5c95f2fbff07bdf455a86705cd6b5e8ee7/runners/google-cloud-dataflow-java/build.gradle#L532

Yep, that's the issue - thanks @Abacn

https://github.com/apache/beam/pull/26415 should fix this

closed in favor of #26416 which Yi put up at the same time


Linking to backend container.

Running docker run -u $(id -u) --rm -v "$PWD":/app trion/ng-cli-e2e ng e2e works as expected.
This container seems to run the angular application on port 49152
I have a backend container running on the network on port 3000
Any idea how I can make sure the database is connected. perhaps via link?
Getting this error docker: Error response from daemon: Cannot link to /kilterly_server_1, as it does not belong to the default network. when trying to run : docker run --link $(SERVER_CONTAINER) -....

I recommend to not use the (legacy) container links but to use a custom network instead. In fact running your backend on a different network than the default network seems to be the cause that your container link does not work.
If you want to have your angular container conntected to multiple networks, you need to separate the container creation and startup phases. So you would use separate docker create and docker start commands.
If you automate this process keep in mind that your backend might take longer to startup than the frontend, so you need to either implement some way to wait for the backend or use a heuristic sleep().
It might be more convenient to use docker-compose to orchestrate multiple containers.
Let me know if this solves your issue.

So I am using compose and linking via the link option.
server:
ports
- 3000:5000
links:
-mongodb
ng:
ports:
- 3001:4200
links:
- server
when starting docker run -u $(id -u) --rm -v "$PWD":/app trion/ng-cli-e2e ng e2e I tried using the --net option while looking up the running network on the server.
The problem is that my angular application is making requests to port 3000. compose links work fine, but I can't figure out how to link your running container to my server.

Please use custom networks as links are deprecated. See the warning at https://docs.docker.com/compose/compose-file/#links
You need to use http://server:3000/ as target for requests from the frontend inside the e2e container. (When using custom networks DNS entries for the services will be created as with the links)

All resolved! Thank you, great advise on the networking with compose.


[Discussion] Gitlab Pipeline

Has anyone managed to get the Gitlab pipeline to work?
It fails when cypress try to get http://localhost:8080/__cypress__/csrf_token (not found)
script:
    - ... other steps ...
    - yarn cypress:run (fails here when try to get /__cypress__/csrf_token)

So, manually I have tried to  get with some urls
script:
    - curl http://localhost:8080/posts  (works)
    - curl http://localhost:8080/users (works)
    - curl http://localhost:8080/__cypress__/csrf_token (fail)

I suppose you're installing composer dev-dependencies, and not setting the environment to production when running this (otherwise, the routes from this package are not registered)?

@Krisell  My bad! When env is production for sure this route __cypress__ won't  work.


JK Independent Project

This project is to create a dialog system with rdf and SPARQL. This would be a good exercise to explore the 2 technologies within my majors style.

Chart to show the basics for the project.


Correct Existing Natural Keys 3 of 3 [Advanced] - Boundary Uniqueness

Blocked by: #2900
Proposed Changes
This is where we need to assert Tenant relationships (or if not Tenant, a specific relationship that serves a similar purpose as a  namespace/boundary) as a first-tier uniqueness boundary and these changes will be highly invasive and require more extensive testing.
Tenancy vs. Global

Duplicate objects can only be in Tenants
No tenant = "global"
Tenant relationships moved from some "child" objects up to their parents, and inherited by the children (e.g. IPAddress.tenant instead inherited from IPAddress.parent(Prefix).tenant, etc.)

IPAM Duplicates

Use a VRF with enforce_unique=False
Tenant boundaries still enforced
IPAM "Namespaces" should still be a thing, so you could have multiple namespaces per Tenant
IPAddress shouldn’t have a relationship to Tenant. It should be inherited from the Prefix.

Enumerate these bullet items per app/model:
Circuits
Provider

Similar to what we’ve discussed with Prefix and IPAddress, the same uniqueness “boundaries” for other relations should be considered here, e.g:

tenant
role

IPAM

Tenant/name might not be good for L2 objects (such as VRF, VLAN)
VLAN/location might make sense in this context, but not Tenant?

We might want name/location or vid/location as an example for VLAN?

VLAN

Constraints:

        unique_together = [
            # 2.0 TODO: since group is nullable and NULL != NULL, we can have multiple non-group VLANs with
            # the same vid and name. We should probably fix this with a custom validate_unique() function.
            ["group", "vid"],
            ["group", "name"],
        ]

Similar to what we’ve discussed with Prefix and IPAddress, the same uniqueness “boundaries” for other relations should be considered here, e.g:

tenant
role

name is not unique and IMO should absolutely be unique, unless we consider the unique_together along with other boundaries described above

name, vid may be a great idea for unique_together

VRF

rd is unique, but not name?

What is the use-case for duplicate name values with unique rd?

Similar to what we’ve discussed with Prefix and IPAddress, the same uniqueness “boundaries” for other relations should be considered here, e.g:

tenant

We might want to re-use rd across tenants, so maybe unique together rd/tenant?

Users
ObjectPermission

No uniqueness constraints whatsoever

This means the same exact object permission could be implemented and assigned with no reasonable way to differentiate them

At a minimum I think that name should be made to be unique and be the natural key

Several m2m relations

object_types → ContentType
groups → Group
users → User

Shall we fix this or just replace this with Django Guardian? (See: https://github.com/nautobot/nautobot/issues/2247)

Do we want unique permissions per tenant, or at least to have it be an optional constraint of some sort?

Virtualization
Cluster

name is unique, yet there are several FKs that might need to be considered
FKs

group → ClusterGroup
tenant - Tenant
site → Site
location → Location

Tenancy vs. Global
* Duplicate objects can only be in Tenants

Should this read "in different Tenants"?

IPAM Duplicates
* Use a VRF with `enforce_unique=False`

Personally I'd like to get rid of this as even being an option. Can we instead enumerate the cases where "duplicate" entries are needed and figure out how to design the data model and uniqueness constraints accordingly?

* IPAM "Namespaces" should still be a thing, so you could have multiple namespaces per Tenant

Is this being proposed as a new/distinct model from VRF?

* IPAddress shouldn’t have a relationship to Tenant. It should be inherited from the Prefix.

Not sure I understand why we want to remove this as a feature?

Circuits
Provider
* Similar to what we’ve discussed with `Prefix` and `IPAddress`, the same uniqueness “boundaries” for other relations should be considered here, e.g:

  * `tenant`
  * `role`

Neither of these are currently fields on a Provider. I'm also not clear on why role would be a uniqueness boundary?

IPAM
* Tenant/name might not be good for L2 objects (such as VRF, VLAN)

* VLAN/location might make sense in this context, but not Tenant?

  * We might want `name/location` or `vid/location` as an example for VLAN?

I'm honestly not entirely clear what's being proposed for any of these three.

VLAN
* Constraints:

        unique_together = [
            # 2.0 TODO: since group is nullable and NULL != NULL, we can have multiple non-group VLANs with
            # the same vid and name. We should probably fix this with a custom validate_unique() function.
            ["group", "vid"],
            ["group", "name"],
        ]

* Similar to what we’ve discussed with `Prefix` and `IPAddress`, the same uniqueness “boundaries” for other relations should be considered here, e.g:

  * `tenant`
  * `role`

Same here, I'm not clear why we'd want to permit two VLANs with the same name/group/vid/tenant but differing Roles?

* `name` is not unique and IMO should absolutely be unique, unless we consider the `unique_together` along with other boundaries described above

VLAN "voice" at Location Alpha may have a different VID than VLAN "voice" at Location Beta. Pretty sure we want to still keep this possible to model.

  * `name`, `vid` may be a great idea for `unique_together`

As opposed to the current (vlan_group, vid) and (vlan_group, name) constraints? What's the benefit?

Users
ObjectPermission
* No uniqueness constraints whatsoever

* This means the same exact object permission could be implemented and assigned with no reasonable way to differentiate them

  * At a minimum I think that name should be made to be unique and be the natural key

This seems reasonable to me.

* Several m2m relations

  * `object_types` → `ContentType`
  * `groups` → `Group`
  * `users` → `User`

Yes? These all seem necessary and appropriate to me.

* Do we want unique permissions per tenant, or at least to have it be an optional constraint of some sort?

Not sure how we'd model that or what the benefit would be. Can you clarify?

Virtualization
Cluster
* name is unique, yet there are several FKs that might need to be considered

* FKs

  * `group` → `ClusterGroup`
  * `tenant` - `Tenant`
  * `site` →` Site`
  * `locatio`n → `Location`

If you're proposing making name non-globally-unique for Cluster, I think that's an idea worth considering.

Carved into #3850, #3851


Reverse swap

"Anybody can swap a child token he holds for the same amount of its parent token."
Why that ?
Once a carbon credit is retired, ti cannot be activated again. An M+ should not be able to go back to M-.

First M- is not the parent of M+. So, yes, it cannot be "activated again".
This feature is useful for two reasons:

Netherlands non-retired carbon tokens would be child of European non-retired carbon tokens. If we don't allow to swap European tokens back to Netherlands tokens, it would somehow devalue the European tokens (they have less usages and therefore are cheaper), thus creating an economical incentive for greater emissions. (This is explained in the README.md file.)

My system could be used not only for accounting carbon but for other public goods, too. For example a token for free Web servers would be a child a token for free software. It is useful if anyone having been rewarded for a free Web server would be able to purchase something for the token of free software, as it is a special case of the free software. For carbon accounting, it is indirectly useful as increases adoption of our system for different usages, so adverting it, what benefits all our subprojects including the carbon subproject.