[BUG] {OAK-D-PRO-W POE Stereo Depth Estimation Is Not Correct}

First, let me explain my setup. I am using OAK-D-PRO-W POE to generate depth based on stereo. Here is the related part (I modified stereo_inertial_publisher.cpp in depthai-ros)
    // MonoCamera
    monoLeft->setResolution(monoResolution); // mono resolution is 720p
    monoLeft->setBoardSocket(dai::CameraBoardSocket::LEFT);
    monoLeft->setFps(stereo_fps);  // Stereo fps is 40
    monoRight->setResolution(monoResolution);
    monoRight->setBoardSocket(dai::CameraBoardSocket::RIGHT);
    monoRight->setFps(stereo_fps);

    // StereoDepth
    stereo->setDefaultProfilePreset(dai::node::StereoDepth::PresetMode::HIGH_ACCURACY);
    stereo->initialConfig.setMedianFilter(dai::MedianFilter::KERNEL_7x7);
    stereo->initialConfig.setConfidenceThreshold(confidence);        // Known to be best // confidence is 90
    stereo->setRectifyEdgeFillColor(0);                              // black, to better see the cutout
    stereo->initialConfig.setLeftRightCheckThreshold(LRchecktresh);  // Known to be best
    stereo->setLeftRightCheck(lrcheck); //lr check is 1
    stereo->setExtendedDisparity(extended); //false
    stereo->setSubpixel(subpixel); // true

I am generating point clouds based on depth using rtabmap_ros/point_cloud_xyz nodelet in ROS
    <arg name="gen_cloud"                   default="true"/> <!-- Generate Point Cloud -->
    <arg name="gen_queue_size"              default="100"/>
    <arg name="gen_approx_sync"             default="false"/> >
    <arg name="gen_decimation"              default="4"/>
    <arg name="gen_voxel_size"              default="0.00"/>
    <arg name="gen_min_depth"               default="0.0"/>
    <arg name="gen_max_depth"               default="5.0"/>
    <arg name="gen_noise_filter_radius"     default="0.1"/> <!-- play with noise filtering more-->
    <arg name="gen_noise_filter_min_neighbors" default="10"/>
    <arg name="gen_normal_k"                default="0"/> <!-- check normal filtering-->
    <arg name="gen_normal_radius"           default="0.0"/>
    <arg name="gen_filter_nans"             default="true"/>
    <arg name="gen_roi_ratios"              default="0.0 0.0 0.0 0.0"/> <!-- this helped to get rid of noise part close to camera at top-->

    <!-- Convert Compressed Depth to Depth (Only for rosbags)-->
    <param if="$(arg use_rosbag)" name="/use_sim_time" value="true"/>
    <node if="$(arg use_rosbag)" pkg="image_transport" type="republish" name="republish_depth" args="compressedDepth in:=/stereo_inertial_publisher/stereo/image raw out:=/stereo_inertial_publisher/stereo/image"/>

    <node pkg="nodelet" type="nodelet" name="oak_nodelet"  args="manager"/>

    <!-- Create Point Cloud from Depth -->
    <node if="$(arg gen_cloud)" pkg="nodelet" type="nodelet" name="gen_cloud_from_depth" args="load rtabmap_ros/point_cloud_xyz oak_nodelet">
        <remap from="depth/image"       to="/stereo_inertial_publisher/stereo/image"/>
        <remap from="depth/camera_info" to="/stereo_inertial_publisher/stereo/camera_info"/>
        <remap from="cloud"             to="/oak/point_cloud" />

        <param name="queue_size"                type="int"    value="$(arg gen_queue_size)"/>
        <param name="approx_sync"               type="bool"   value="$(arg gen_approx_sync)"/>
        <param name="decimation"                type="double" value="$(arg gen_decimation)"/>
        <param name="voxel_size"                type="double" value="$(arg gen_voxel_size)"/>
        <param name="min_depth"                 type="double" value="$(arg gen_min_depth)"/>
        <param name="max_depth"                 type="double" value="$(arg gen_max_depth)"/>
        <param name="noise_filter_radius"       type="double" value="$(arg gen_noise_filter_radius)"/>
        <param name="noise_filter_min_radius"   type="int" value="$(arg gen_noise_filter_min_neighbors)"/>
        <param name="normal_k"                  type="int" value="$(arg gen_normal_k)"/>
        <param name="normal_radius"             type="double" value="$(arg gen_normal_radius)"/>
        <param name="filter_nans"               type="bool" value="$(arg gen_filter_nans)"/>
        <param name="roi_ratios"                type="string" value="$(arg gen_roi_ratios)"/>
    </node>

Here is the problem. I see points in the cloud that should not be there. Here are some screenshots.
Shot 1: I do not know why it is seeing stuff significantly above the ground, but bear with me this is the least weird one.

Shot 2: See the points on the right of the axes? What is it? There is nothing there :(

Shot 3: Again there are some stuff on the right that should not be there at all. (Do not worry about yellow points they are also part of point cloud. They are just clustered as obstacles (no need to worry for this bug. Think them as white too)

Can someone help me out what is the issue here? I feel like there is a problem with the camera extrinsic. Do I need to calibrate it? Thanks a lot for the support! :)

@samialperen
There are multiple issues.

Stereo depth estimation highly depends on the quality of input images.
In your case the images are overexposed, which leads to a loss of detail, thus bad depth estimation.
OAK-D-PRO-W camera sensors are without IR-filter, because it comes with an active illuminator (dot-projector). This is not suitable for outdoor use.
The buildings you are trying to measure are far away (20+ meters from what it looks like). OAK-D-PRO-W-POE is suitable for up to 10 meters, but ideally 5-6. See accuracy evaluation. This is due to 7.5 cm baseline and high FOV. The higher the FOV, the shorter the maximum perceived distance by stereo depth.

If you want depth for longer distances you need narrower FOV, higher baseline and IR-filter (for outdoor use).

Hi @szabi-luxonis,
Thanks a lot for the response.
1-) Yes, naturally. I was wondering whether there is a way to improve image quality, too, so that we can have a better depth.
2-) Is there a version that supports outdoor use? We selected this version since we want to use it for outdoor during the night time and IR flood light coming with this version seemed appealing to us. At the time of taking these screenshots, both flood light and dot projector were on with 400mA.
3-) Actually, I am not trying to measure any of those buildings. The main case for my application is to use camera for obstacle avoidance. At the moment, I am generating point clouds up to 5 meters and I am fine with that. Those buildings are just in the background during the screenshot, but the points I am getting have nothing to do with those buildings since I am not getting any points that are far away than 5 meters.
At the moment, I can not change the camera, so I am trying to make the best out of it. What do you suggest? Let's say one for night time outdoor, and the other one for indoor outdoor environments.
Thanks a lot again for the support! :)

@samialperen
This is a physical limitation. There is too much light reaching the sensor, which results in oversaturation. Wide-angle lenses will let in more light, compared to narrow ones.
You can use a third-party IR light filter and mount it on the front of the lens to reduce ambient light.

I see! Thanks a lot for all the information. Highly appreciated. I will share more testing results for night time to see how it performs during the night. If it is better than day light, I will accept the physical limitation and try to find a work around for that (like buying an IR filter as you suggested)

One of the easiest solutions I have tried is enabling manual exposure following the recent commit to depthai-core: https://github.com/luxonis/depthai-ros/commit/9ac79711a7ae247588a86c75911ddb306e03617f
Auto Exposure:

Manual exposure with exposure time 10 and ISO 100

Does anyone know what else I can do to make it better using CameraControl node: https://docs.luxonis.com/projects/api/en/latest/components/messages/camera_control/

@szabi-luxonis Do you know if an external IR filter will affect performance for the night time? How can I order a filter that will not interfere with IR flood light for night vision?

@szabi-luxonis
Although I am using HIGH_DENSITY in a situation without over exposure, still the depth cloud is not great at all. You can see the picture below. It is hard to tell whether there is a person from the depth cloud.

these are the stereo settings I used at that time

Here is a different perspective


Button Group incorrect on dark backgrounds

The color of the active button is incorrect on dark backgrounds:

https://www.figma.com/file/xZ0IW0MJO0vnFicmrHiKaY/Components-Post?type=design&node-id=51-59&mode=design&t=IYvLqgFWoBWJjcJM-0


#8636 Request a Copy's "helpdesk" strategy requires authentication t…

Dspace-angular related pull request is required to test this.
References

Fixes #8636
Related to and required: DSpace/dspace-angular/pull/2199

Description
Allow grant access without authentication
Instructions for Reviewers
List of changes in this PR:

Bitstream can be added to grant email without authorized logged user.

To test:

Activate request a copy with Helpdesk Strategy: https://wiki.lyrasis.org/display/DSDOC7x/Request+a+Copy#RequestaCopy-(Optional)RequestingacopywithHelpDeskworkflow
As anonymous user, look at an embargoed item, click on the file to display the request a copy form.
Complete and send the request a copy form
As helpdesk staff, look at the email inbox
Open the Request a copy email and click on the link to respond to it.
Approve  the request (without login)

Checklist

[X] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & integration tests). Exceptions may be made if previously agreed upon.
[X] My PR passes Checkstyle validation based on the Code Style Guide.
[X] My PR includes Javadoc for all new (or modified) public methods and classes. It also includes Javadoc for large or complex private methods.
[X] My PR passes all tests and includes new/updated Unit or Integration Tests based on the Code Testing Guide.
[X] If my PR includes new libraries/dependencies (in any pom.xml), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
[X] If my PR modifies REST API endpoints, I've opened a separate REST Contract PR related to this change.
[X] If my PR includes new configurations, I've provided basic technical documentation in the PR itself.
[X] If my PR fixes an issue ticket, I've linked them together.

I agree:  the requestor has no business ever seeing the token, which is meant solely for the person(s) who will grant or deny the request.

@mwoodiupui or @aroman-arvo : Would one of you be willing to change the POST /api/tools/itemrequests to return a 201 with an empty body? Currently it returns a JSON response as documented here: https://github.com/DSpace/RestContract/blob/main/item-requests.md#creating-a-request
If that change is made, then this PR would be acceptable as-is.  However, we cannot move this PR forward until that change is made.  That change could be made within this PR, or it could be done as a separate PR.
I feel this fix an important change to make for 7.6 if possible.  If neither of you have time, I'll see if I can find time to make the change.  The current behavior is very obviously flawed & should be fixed as soon as possible.

I think its OK now

Thanks @aroman-arvo .  I'll give this a fresh review now and see if it looks good


Tabline needs to be styled

I'd love to take this on! I'm trying to soften up my company workstation and this has been the biggest issue for me


refactor/class c to enum

There are many uses of the class C that holds constants that can be represented by enums instead.
For example the RTCSessionState can be converted to this:
class C {
  // RTCSession states.
  static const int STATUS_NULL = 0;
  static const int STATUS_INVITE_SENT = 1;
  static const int STATUS_1XX_RECEIVED = 2;
  static const int STATUS_INVITE_RECEIVED = 3;
  static const int STATUS_WAITING_FOR_ANSWER = 4;
  static const int STATUS_ANSWERED = 5;
  static const int STATUS_WAITING_FOR_ACK = 6;
  static const int STATUS_CANCELED = 7;
  static const int STATUS_TERMINATED = 8;
  static const int STATUS_CONFIRMED = 9;
}

to this
enum RtcSessionState {
  none, // STATUS_NULL
  inviteSent, // STATUS_INVITE_SENT
  provisionalResponse, // STATUS_1XX_RECEIVED
  inviteReceived, // STATUS_INVITE_RECEIVED
  waitingForAnswer, // STATUS_WAITING_FOR_ANSWER
  answered, // STATUS_ANSWERED
  waitingForAck, // STATUS_WAITING_FOR_ACK
  canceled, // STATUS_CANCELED
  terminated, // STATUS_TERMINATED
  confirmed, // STATUS_CONFIRMED
}

@victortive  love this, rebase and fix up the conflicts and we'll be good to go.

@mikaelwills What conflicts are you referring to?

@victortive Just below our comments "this branch cannot be rebased due to conflict"

@victortive  merged in and it passed all the build checks, looking good!
Im sure you've done some already but will give it an integration test with two clients to make sure all is well.


[efsw] Update to 1.3.1.

Fixes https://github.com/microsoft/vcpkg/issues/29188.
Update efsw to 1.3.1, no feature needs to be tested.

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download
[ ] The "supports" clause reflects platforms that may be fixed by this new version
[ ] Any fixed CI baseline entries are removed from that file.
[ ] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

Thank you!


Ignore email JS pixel events

Task/Issue URL: https://app.asana.com/0/414235014887631/1203393655574807/f
Tech Design URL:
CC:
Description:
This PR updates the iOS app to account for email JS pixel events. The iOS app sends its own email autofill pixels, so those coming from the JS layer should be ignored.

Steps to test this PR:

Test the email autofill feature and verify that you don't see double pixels being fired

Copy Testing:

[ ] Use of correct apostrophes in new copy, ie ’ rather than '

Orientation Testing:

[ ] Portrait
[ ] Landscape

Device Testing:

[ ] iPhone SE (1st Gen)
[ ] iPhone 8
[ ] iPhone X
[ ] iPhone 14 Pro
[ ] iPad

OS Testing:

[ ] iOS 14
[ ] iOS 15
[ ] iOS 16

Theme Testing:

[ ] Light theme
[ ] Dark theme

Internal references:
Software Engineering Expectations
Technical Design Template
When ready for review, remember to post the PR in MM

Marked as Do Not Merge as this is pending upstream changes being merged


[WIP] Implemented Epoch based part, using SR25519 as a VRF and also replace ambiguous macro with procedural-macro

@keyvank , I divided the code into different parts according to different functions and connect them with interface. So there maybe some ambiguous generics which would implement in some days. I write this message because I just found this one may confuse u.


cache_stats endpoint orderby=value returns Internal Server Error

I'm submitting a ...

[X] bug report
[ ] new feature / enhancement request
[ ] improvement request (usability, performance, tech debt, etc.)
[ ] other

Traffic Control components affected ...

[ ] CDN in a Box
[ ] Documentation
[ ] Grove
[ ] Traffic Control Client
[ ] Traffic Monitor
[X] Traffic Ops
[ ] Traffic Ops ORT
[ ] Traffic Portal
[ ] Traffic Router
[X] Traffic Stats
[ ] Traffic Vault
[ ] unknown

Current behavior:

When making a GET Request to the cache_stats endpoint with orderby=value results in an 500: Internal Server Error.
Expected / new behavior:

Response should have correct ordered values.
Minimal reproduction of the problem with instructions:

GET
https://{{TO_BASE_URL}}/api/{{api_version}}/cache_stats?cdnName=Kabletown2.0&endDate=2019-10-31T10:44:00-06:00&metricType=bandwidth&startDate=2019-10-30T10:44:00-06:00&orderby=value
Anything else:

@ZugNZwang - looks like orderby was never supported for this endpoint:
https://github.com/apache/trafficcontrol/blob/master/traffic_ops/app/lib/Extensions/TrafficStats/API/CacheStats.pm#L32-L62
however, it's use before would not return a 500 so i guess this is a regression.


[SPARK-42552][SQL] Correct the two-stage parsing strategy of antlr parser

What changes were proposed in this pull request?

This PR follows the https://github.com/antlr/antlr4/issues/192#issuecomment-15238595 to correct the current implementation of the two-stage parsing strategy in AbstractSqlParser.
Why are the changes needed?

This should be a long-standing issue, before SPARK-38385, Spark uses DefaultErrorStrategy, and after SPARK-38385 Spark uses class SparkParserErrorStrategy() extends DefaultErrorStrategy. It is not a correct implementation of the "two-stage parsing strategy"
As mentioned in https://github.com/antlr/antlr4/issues/192#issuecomment-15238595

You can save a great deal of time on correct inputs by using a two-stage parsing strategy.

Attempt to parse the input using BailErrorStrategy and PredictionMode.SLL.
If no exception is thrown, you know the answer is correct.
If a ParseCancellationException is thrown, retry the parse using the default
settings (DefaultErrorStrategy and PredictionMode.LL).

Does this PR introduce any user-facing change?

Yes, the Spark SQL parser becomes more powerful, SQL like SELECT 1 UNION SELECT 2 parse succeeded after this change.
How was this patch tested?

New UT is added.

CI passed w/ small changes in UT, kindly ping @cloud-fan

thanks, merging to master!


Should switch to MySQLi or PDO

The system we're currently using is deprecated.  Either of the two new systems allow for 'prepared statements' which protect us from SQL injection attacks.

Switched to using PDO and Parameterized Queries


RRateLimiter trySetRate and expire method not an atomic method

When implementing rate limiting for user requests or third-party calls, we often limit based on the user's ID. However, the two operations—setting the rate limiter and setting the expiration time—aren't atomic. If the rate limiter is successfully set, but the expiration time fails to be set, it can lead to significant memory leaks. Ideally, there should be a way to ensure these operations are atomic to prevent such issues.
While I could implement this through Lua scripts, it would be much more desirable to have this functionality natively supported, as many developers encounter this scenario and would benefit from a streamlined, built-in solution. It would not only simplify development but also minimize the potential for errors.
In short, a native, atomic method for setting both the rate limiter and its expiration time would be a game-changer for managing rate limiting more efficiently and safely.

When the rate limiter is configured with an expiration time of 3 minutes and a token bucket that allows 5 requests per minute, the following scenario can lead to uneven traffic distribution: If there is one request at the 1st second, followed by 4 requests at the 2nd minute and 59th second, and then another 4 requests right after the 3rd minute mark (i.e., at the 3rd minute and 3rd second), this pattern could cause the rate limiter to improperly distribute the allowed requests, potentially leading to bursts of traffic that exceed the intended rate limit.
My preferred lua code is:
"redis.call('hsetnx', KEYS[1], 'rate', ARGV[1]);"
+ "redis.call('hsetnx', KEYS[1], 'interval', ARGV[2]);"
+ "local res = redis.call('hsetnx', KEYS[1], 'type', ARGV[3]);"
+ "if tonumber(ARGV[4]) > 0 then "
+ "redis.call('pexpire', KEYS[1], ARGV[4]); "
+ "end; "
+ "return res;"

Since I just want to keep atomicity, hsetnx will fail when the key is present, so I can change it this way to keep atomicity and refresh time. The above problem does not occur.
My business code is used like this⬇


The behavior of "tenant access add" command is confusing

Currently the "tenant access add" command can be run any number of times to grant a datastore access for the same tenant. This may cause weird behavior which is counterintuitive. For example:
I have granted the following permissions for my tenant:
[root@localhost:/usr/lib/vmware/vmdkops/bin] vmdkops_admin tenant access ls --name MyTenant
Datastore   Create_volume  Delete_volume  Mount_volume  Max_volume_size  Total_size
----------  -------------  -------------  ------------  ---------------  ----------
datastore1  1              1              0             128.00MB         1.00GB

Now I want to add the missing "mount" privilege. As the implementor, we know all the commands and we expect user to run "tenant access set" to modify the privileges. But a new user who are not familiar with all the command options might (incorrectly) do this instead:
[root@localhost:/usr/lib/vmware/vmdkops/bin] vmdkops_admin tenant access add --name MyTenant --datastore datastore1 --right mount
{'mount_volume': 1, 'global_visibility': 0, 'max_volume_size': 0, 'create_volume': 0, 'delete_volume': 0, 'datastore': 'datastore1', 'usage_quota': 0}
tenant access add succeeded

Now what happened? All the existing privilege settings are gone:
[root@localhost:/usr/lib/vmware/vmdkops/bin] vmdkops_admin tenant access ls --name MyTenant
Datastore   Create_volume  Delete_volume  Mount_volume  Max_volume_size  Total_size
----------  -------------  -------------  ------------  ---------------  ----------
datastore1  0              0              1             0B               0B

This is not a correctness issue (we can argue this is a user error). But I think we can improve the usability a little bit and make the program more robust to prevent this wrong operation happening.
Regarding the solution, I think one option is to prevent user to run "tenant access add" twice on the same tenant, and we can print a tip message to remind user to use "tenant access set" to modify the permissions.
Any other suggestions are welcome.

we can print a tip message to remind user to use "tenant access set" to modify the permissions.

I just tried a different scenario:
First I revoked all access to a datastore for my tenant. Then I tried to run "tenant access set" directly - here's what I got:
[root@localhost:/usr/lib/vmware/vmdkops/bin] vmdkops_admin tenant access set --name MyTenant --datastore datastore1 --add-right create,delete,mount
No privileges exist for (MyTenant, datastore1)
So, for the current issue of running "tenant access add" multiple times, instead of reminding user to use "tenant access set", we can simply print a similar message, such as "Privileges already exist for (Tenant, Datastore)". This way the user experience will be more consistent.

This issue has already been fixed in the recent change:　https://github.com/vmware/docker-volume-vsphere/pull/859
Now if we try to add the same privilege (same datastore, but other parameters are different), we will get the following error:
privilege for (TENANT, datastore) already exists.


Proposal to add rank_dropout scale

Hi @KohakuBlueleaf! Thank you very much for your awesome library!
I was reading through your adapters code and was wondering - why there is no scaling when doing rank_dropout?
According to torch docs nn.Dropout performs scaling of output values using the provided dropout probability to retain the overall normalization the same.

I am not sure whether your implementation does have this scale intentionally or not (probably the further group normalization neutralizes the lack of this scale), but I've made several experiments, and adding this scaling may help training a bit faster if dropout is enabled.

I just copied the dropout mechanism from kohya
I think what you pointed out is a good question
I will dig into this

@kovalexal 2 things
1, Plz open a new PR to dev branch
2, Can you make this as an optional setting? so we will not change the default behavior (If you want, you can add some warning for this)

@KohakuBlueleaf sure, I'll add this as an optional parameter in the following commits to this PR.

@KohakuBlueleaf I added rank_dropout_scale parameter as an optional argument to all adapters.
I've also tried to run kohya_ss with this updated version, but from what I can tell, there is no ability to pass rank_dropout and module_dropout in kohya_ss (at least in their gui) according to this script.
In GUI there is an option "additional parameters" in the advanced tab, but when I write something like --network_args "rank_dropout=0.3" it overwrites the remaining  --network_args parameters in this line, so by default it switches training adapter to LoRA.
So there is a chance that we need to create PR to kohya_ss to be able to pass these parameters to your code.
I've also run several experiments with kohya_ss gui (lokr-sd15 preset) and rank_dropout passed as 0.3 and from what I can tell, the resulting adapter does not produce meaningful results after training without the proposed scaling (see the following results). Is there a chance that we should enable this scale by default?
Scaling disabled (lokr-sd15, weight: 1.0):

Scaling enabled (lokr-sd15, weight: 1.0, rank_dropout: 0.3):

For reference, sd-script already scales by default.
https://github.com/kohya-ss/sd-scripts/blob/2d87bb648f30adab00ceb38a0da786cd548d5ce7/networks/lora.py#L109


exec with image id

1.Describe what this PR did
exec support image id
2.Does this pull request fix one issue?
fixes #374
3.Describe how you did it
check image's name/id has existed before request HTTP server
4.Describe how to verify it
5.Special notes for reviews

Thanks a lot for your contribution. @jomenxiao 🥂

check image's name/id has existed before request HTTP server

I am afraid in your code, you use an ImageList request torwards HTTP server(pouch daemon) to get image list from server then check the existence of user's input image.
So, you still requested the daemon. To be honest, maybe it is not the way it should be.
Feel free to tell us if you need any help.

ok,
when the image not existed,  the pouch daemon tries to pull it.
I miss it.
and what's your idea and suggestion?

My suggestion is that, pouch cli could have ability to pass image id, and it is the pouch daemon side that deals with this image id and try to find the actual image with reference(image:tag).
So, I would suggest that daemon side is a good way. Maybe more input from @0x04C2 😄

Now the daemon side is supported to create container with image's id.

I think we need to add more details of all aspects which support image id in swagger.yml and this will help to sync information among us. @skoo87


Problems with moment 2.19.0

Description of the Issue and Steps to Reproduce:
Did you search for duplicate issue? yes
Today was the relase of moment 2.19.0. There are some build issues with this angular2-moment and moment 2.19.0. The output is:
Module not found: Error: Can't resolve './locale' in 'C:\angular-app\node_modules\angular2-moment\node_modules\moment\src\lib\locale'
resolve './locale' in 'C:\angular-app\node_modules\angular2-moment\node_modules\moment\src\lib\locale'
  using description file: C:\angular-app\node_modules\angular2-moment\node_modules\moment\package.json (relative path: ./src/lib/locale)
    Field 'browser' doesn't contain a valid alias configuration
  after using description file: C:\angular-app\node_modules\angular2-moment\node_modules\moment\package.json (relative path: ./src/lib/locale)
    using description file: C:\angular-app\node_modules\angular2-moment\node_modules\moment\package.json (relative path: ./src/lib/locale/locale)
      as directory
        C:\angular-app\node_modules\angular2-moment\node_modules\moment\src\lib\locale\locale doesn't exist
[C:\angular-app\node_modules\angular2-moment\node_modules\moment\src\lib\locale\locale]
 @ ./node_modules/angular2-moment/node_modules/moment/src/lib/locale/locales.js 65:16-60
 @ ./node_modules/angular2-moment/node_modules/moment/src/lib/locale/locale.js
 @ ./node_modules/angular2-moment/node_modules/moment/src/moment.js
 @ ./node_modules/angular2-moment/locale.pipe.js
 @ ./node_modules/angular2-moment/index.js
 @ ./src/app/master-data/master-data.module.ts
 @ ./src/$$_gendir lazy
 @ ./node_modules/<EMAIL_ADDRESS> @ ./src/main.ts
 @ multi webpack-dev-server/client?http://localhost:4200 ./src/main.ts

Please answer the following questions:

Angular version? 4.4.4
moment version? 2.19.0
Are you using moment-timezone? no
Are you using the angular-cli? yes

I am also getting the following error
WARNING in ./~/moment/src/lib/locale/locales.js Module not found: Error: Can't resolve './locale' in 'C:\Workspaces\node_modules\moment\src\lib\locale'  @ ./~/moment/src/lib/locale/locales.js 65:16-60  @ ./~/moment/src/lib/locale/locale.js  @ ./~/moment/src/moment.js  @ ./~/angular2-moment/date-format.pipe.js  @ ./~/angular2-moment/index.js  @ ./src/app/app.module.ts  @ ./src/main.ts  @ multi webpack-dev-server/client?http://localhost:4200 ./src/main.ts
Angular versions -
@angular/cli: 1.2.6
@angular/animations: 4.4.4
@angular/common: 4.4.4
@angular/compiler: 4.4.4
@angular/compiler-cli: 4.4.4
@angular/core: 4.4.4
@angular/forms: 4.4.4
@angular/http: 4.4.4
@angular/language-service: 4.4.4
@angular/platform-browser: 4.4.4
@angular/platform-browser-dynamic: 4.4.4
@angular/router: 4.4.4
@angular/cli: 1.2.6

For a quick fix, locking the angular2-moment and moment versions worked for us:
    "angular2-moment": "1.3.3",
    "moment": "2.18.1",

@chapulina your solution is not working here. We are working with yarn instead of npm and yarn adds moment (2.19.0) in a node_modules subfolder of angular2-moment. As a result i have two versions of moment in my project.

add:
  "resolutions": {     "moment": "2.18.1"   }
To your package.json and then install angular2-moment.

@lotoad thanks! that worked for me.

It works fine with latest moment 2.19.1 just in case


Feature/orthographic UI

Problem
This closes #279 / #320 and #419.
Orthographic UI (#419) was to be implemented in tandem with button styling. In hindsight: review would be easier if this were two PRs, one for styling, one for implementing the camera. Luckily it's not a huge merge.
Solution
Camera:
Add UI buttons and props to call setCameraType. (see note about dev build below)
Styling:
Updated colors, size, spacing, and borders of camera and playback controls. Did some cleanup adding .container wrappers on CSS rules and renaming a few things.

New feature (non-breaking change which adds functionality)

Steps to Verify:
This branch will not run with the current viewer, as we have not done a release that includes the orthographic camera yet.
If you would like to see it in action:
From the main branch of the viewer create a dev build (npm run build, npm run copy-build, if having trouble delete node modules and package lock)
From the feature branch of the website: npm i [PATH TO YOUR BUILD]
Screenshots (optional):

the tool tips look backwards to me on these screen shots


WIP adding support for python3

WIP on porting magenta to python3, posting to get early comments
bazel test --test_output=streamed --verbose_failures //magenta/... --host_force_python=py3

Isn't that fixed? It looks like it is working but I haven't tested all the models yet. See #429

Yes, we now fully support python 3. This PR is out of date, so I'll close it.


Frames not in sync when using appsrc

I am using an custom appsrc(which sends camera frames on the "need data" signal)
as the video source for this example.
The browser is showing video frames reasonably(about 1 sec delay) however some of the frames are out of order or the video freezes.  When running the appsrc to autovideosink the frames are smooth.
Is this more likely to be appsrc related or webrtc
Changes to example
pipe1 =
      gst_parse_launch ("webrtcbin bundle-policy=max-bundle name=sendrecv " STUN_SERVER
      " appsrc name=zedCamera ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! "
      "queue !  " RTP_CAPS_OPUS "97 ! sendrecv. ",
      &error);

  if (error) {
    g_printerr ("Failed to parse launch: %s\n", error->message);
    g_error_free (error);
    goto err;
  }

   appsrc	= gst_bin_get_by_name( GST_BIN( pipe1), "zedCamera" );
  /*Set caps for appsrc*/
  g_object_set (G_OBJECT (appsrc), "caps",
        gst_caps_new_simple ("video/x-raw",
                     "format", G_TYPE_STRING, "BGRA",
                     "width", G_TYPE_INT, 1920,
                     "height", G_TYPE_INT, 1080,
                     "framerate", GST_TYPE_FRACTION, 30, 1,
                     NULL), NULL);
  /* setup appsrc */
  g_object_set (G_OBJECT (appsrc),
        "is-live", 0,
        "format", GST_FORMAT_TIME, NULL);
  g_signal_connect (appsrc, "need-data", G_CALLBACK (cb_need_data), NULL);

custom appsrc
//sends frame to appsrc
extern "C" GstFlowReturn push_frame(GstElement *appsrc)
{

  static GstClockTime timestamp = 0;
  GstBuffer *buffer;
  GstFlowReturn ret;
  Mat img;
  guint size = 1920 * 1080 * 2;

  // A new image is available if grab() returns SUCCESS
  if (zed.grab() == SUCCESS) {

      // Get the left image
      zed.retrieveImage(img, VIEW_LEFT);

      //Determine Buffer size
      gsize sizeInBytes = (img.getResolution().area() * img.getPixelBytes());
      //Rertive pointer to image data from sl MAT
      gpointer image = img.getPtr<sl::uchar1>(MEM_CPU);

      //Wrap the data in GstBuffer
      buffer = gst_buffer_new_wrapped_full((GstMemoryFlags)0,
      image, sizeInBytes, 0, sizeInBytes, NULL, (GDestroyNotify)buffer_destroy);
    }
    else{//A frame is dropped
          g_print("frame was dropped \n");
          buffer = gst_buffer_new_allocate (NULL, size, NULL);
          /* this makes the image black*/
          gst_buffer_memset (buffer, 0, 0x0 , size);

    }
      //allocate the timestamp
      GST_BUFFER_OFFSET(buffer) = 0;
    	GST_BUFFER_PTS(buffer) = timestamp;
    	GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 30);
      timestamp += GST_BUFFER_DURATION (buffer);

      //send frame
      g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
      gst_buffer_unref (buffer);

      return ret;

}

This is not a support forum.  Please use the GStreamer mailing list or IRC channel for support.


Added FAQs

This PR...
Changes

Fixes

How to test it

Screenshots

Checklist

[ ] tested locally
[ ] added new dependencies
[x] updated the docs
[ ] added a test

looks like you need to resolve conflicts before we can merge!


[NEW CONTENT TYPE REQUEST] Visual C project

What type of file would you like magika to detect?
Visual C project (.vcxproj)
What software can create/open these files?
Visual Studio, CMake
Where can these files be found?
C++ app source code.
If possible, please provide a specification for this file type.
Additional context

Closing, see #152 for context. Thanks!


Emitting .d.ts files while type checking

In our project we are working with TypeScript, Webpack and Babel 7.
To deal with the TypeScript files we were using the awesome-typescript-loader but there were happening a lot of problems when the generated .js files were taken by Babel. Then, we decided not to use the awesome-typescript-loader anymore, but instead stick to Babel and, since Babel does not do type checking for TypeScript, we decided to do it by using your plug-in fork-ts-checker-webpack-plugin, and we are enjoying a lot the experience.
The only problem now is that, for our application, we need to generate the .d.ts files so that the imports from one Project to another will be done properly.
We observed that your project is missing only the "emit" step for fixing the problem above.
We want to make a contribution to your project by following the steps bellow and your opinion would be very welcome.
In service.ts, inside the "run" function we will do the "getDiagnostics" for the type checking as it is done now but we will also "emit" to generate the d.ts files.
In order to not affect the other users the "emit" phase should be optional. Thus we are creating a new field in the plug-ins Options interface called "ignoreEmit" with default value of "true".

I'm really glad you're enjoying using the plugin!
I'm afraid I have got very mixed feelings about what you're suggesting though.  The fork-ts-checker-webpack-plugin has always been intended to be a checker - not something that generates TS code in any way.
Classically code generation has been the responsibility of ts-loader which is typically what people use alongside the fork-ts-checker-webpack-plugin. (There are examples of how you can use them together: https://github.com/TypeStrong/ts-loader/tree/master/examples/fork-ts-checker-webpack-plugin - which reminds me; I must update them now fork-ts-checker-webpack-plugin has 1 point oh-ed)
I see you're using Babel - that's cool. (It's only relatively recently that this has become possible - super cool!)
If you wanted to switch to use ts-loader and try and add the functionality there then I'd be happy to help make that happen. The switch should be relatively simple. I think there's a need for what you're trying to do.
Alternatively if you wanted to roll your own plugin which plays well alongside the fork-ts-checker-webpack-plugin that's a great idea too. There's already examples of this such as the https://github.com/dividab/tsconfig-paths-webpack-plugin
What do you think?

I'm closing this issue as there no further comments :)


[Docs] misleading key-binding "when" resource

VSCode Version: 1.44.0
OS Version: Windows 10 19041.207

Steps to Reproduce:

Specify a command to appear only when resource =~ /some regex/

From the documentation for "resource" Resource context:

The full Uri of the Explorer or editor file

I would expect that I should be able to filter by a full path, such as file:///C:/foo/src/bar.cs or C:/foo/src/bar.cs, but instead the path used to filter starts at the workspace root (i.e. if VS Code has opened folder "foo", the path used is src/bar.cs). Is this expected? Is there a way to filter when a command appears by the actual full URI, not the full workspace URI?

Does this issue occur when all extensions are disabled?: Yes/No

Please provide the full sample code for how to set this up until the issue comes up.

I'll close this as I've discovered a solution to my problem.
It seems you can filter commands with resource =~ /src[...]/ or resource =~ /folderRoot/src[...], which allows you to determine which workspace root the resource originates from


Spring Boot 3 support

Hello,
The actual Spring Boot autoconfiguration registration method will not be supported anymore in Spring Boot 3.
The current milestone M5 have alreday removed the support of the "spring.factories" file in flavor of the "META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports" file.
This new method have been introduced in Spring Boot 2.7.
It's a blocker since the Picocli Spring Boot Starter can't be used with the upcoming release of Spring Boot 3 (and current milestone) .
See comments here;
https://github.com/spring-projects/spring-boot/issues/32566
https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.0.0-M5-Release-Notes#auto-configuration-registration

Hello,
Adding a new file "/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports" with this content works:
picocli.spring.boot.autoconfigure.PicocliAutoConfiguration

Hi @brunomace I believe this was completed in https://github.com/remkop/picocli/pull/1778

Duplicate


AUR package

Howdy (always good to encounter another Rust-writing Wesley). This issue is just to mention that I created an AUR package for jfmt. Sometime folks mention this in the README, but that can be the road to madness (E.g. see the giant list of packages ripgrep has accumulated), so up to you. 👍
https://aur.archlinux.org/packages/jfmt

I'll drop this in the README, and mention that it's maintained by someone else.

Cool

Want me to link you in the README or no?

Nah, it lists me as maintainer on the AUR page and it's not all that uncommon for my AUR packages to be picked up by an Arch trusted user and get promoted to the community repo (at which point I'm no longer the maintainer).

Added in da68338

I guess it adds value to add repology badge to include the packaging status: https://repology.org/project/jfmt/badges
I have added it this to nixpkgs recently (https://github.com/NixOS/nixpkgs/pull/159164) and I believe in some days the repology will get updated with the appropriate entry for nixpkgs-unstable.


Make the popup "switch to ALSA" dialogue a bit nicer

Change the popup dialogue text to be framed as "if you have problems, switch to alsa" rather than "if you want problems, stay with pulseaudio". Also tidy up the dialogue box and format it properly (and fix a bug where it was missing a backslash in the yad command line), and frame responses to skew towards "do what I'm doing now", and word in terms of "Don't ask again" which is familiar from other dialogue types. Make the "always use PulseAudio" file be visible, so that if a user wants to undo their decision but can't remember how, they stand a chance of finding the file by poking around in snap/APPNAME.
I have not been able to test that this actually does the right thing, because doing so requires building and installing a whole snap, so that will need checking still of course.

Many Thanks, This LGTM :-)


add frontend validation for package creation

Purpose
Fixes https://github.com/LSFLK/MedicinesforLK/issues/123
Goals

Approach

Screenshots

Related PRs

Learning

@jayasanka-sack Thanks for taking a look. I'll fix this up.


Delete node-modules

Node-modules will be automatically recreated by running "npm i" due to the package.json file. Deleting node-modules will decrease the file size drastically

Fixed in this commit
Please, @JoshnaksPNG & @johnnydevelops, one of you should close this issue


Passing extra query parameters to the request

What's the best way to pass custom query parameters to the Storage methods? For example, slack supports sending a team= query parameter for narrowing down oauth2 flows. I'd love to do something similar (with param 'channel'), but getting errors when trying to pass any extra query params:
...     query=Query(**query_params),
TypeError: __init__() got an unexpected keyword argument 'channel'

I understand the Query dataclass doesn't have support for it today, but is there any other good way to handle this? Or am I overcomplicating things?
p.s. – Thanks for the great work on this lib btw! 🙏

@pedropregueiro you can inherit from Query and add your own fields, like this:

from aioauth.requests import Post, Query as _Query, Request
from dataclasses import dataclass

@dataclass
class Query(_Query):
    team: str | None = None

async def to_oauth2_request(
    request: Request, settings: Settings = Settings()
) -> OAuth2Request:
    """Converts :py:class:`fastapi.Request` instance to :py:class:`aioauth.requests.Request` instance"""
    form = await request.form()

    post = dict(form)
    query_params = dict(request.query_params)
    method = request.method
    headers = HTTPHeaderDict(**request.headers)
    url = str(request.url)

    user = None

    if request.user.is_authenticated:
        user = request.user

    return OAuth2Request(
        settings=settings,
        method=RequestMethod[method],
        headers=headers,
        post=Post(**post),
        query=Query(**query_params),
        url=url,
        user=user,
    )

Perfect, this works!

@aliev using custom dataclasses is creating some type validation issues (using mypy).
Here's an example with custom Post and Request classes:
from dataclasses import dataclass
from typing import Optional

from aioauth.models import AuthorizationCode
from aioauth.requests import Post as _Post
from aioauth.requests import Request as _Request
from aioauth.storage import BaseStorage
from aioauth.types import RequestMethod

@dataclass
class CustomPost(_Post):
    my_field: str = ""

@dataclass
class CustomRequest(_Request):
    post: CustomPost = CustomPost()

class Storage(BaseStorage):
    async def create_authorization_code(
        self,
        request: CustomRequest,
        client_id: str,
        scope: str,
        response_type: str,
        redirect_uri: str,
        code_challenge_method: Optional[str],
        code_challenge: Optional[str],
        code: str,
    ) -> AuthorizationCode:
        pass

oauth2_request = CustomRequest(
    method=RequestMethod.POST,
    post=CustomPost(**{"grant_type": "authorization_code", "my_field": "test"}),
)

When running mypy I get the following errors:
test.py:22: error: Argument 1 of "create_authorization_code" is incompatible with supertype "BaseStorage"; supertype defines the argument type as "Request"
test.py:22: note: This violates the Liskov substitution principle
test.py:22: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
test.py:38: error: Argument 1 to "CustomPost" has incompatible type "**Dict[str, str]"; expected "Optional[GrantType]"
Found 2 errors in 1 file (checked 1 source file)

Tbh, I haven't used dataclasses that often so not sure what's the best way to handle type inheritance.

hmm that seems to be expected. I think I can find possible solutions, give me some time.

@pedropregueiro Created PR. not sure if it won't break anything further. needs to be tested

thanks @aliev! that solves the error with the Storage methods, but unfortunately not the custom Query or Post one:
test.py:38: error: Argument 1 to "CustomPost" has incompatible type "**Dict[str, str]"; expected "Optional[GrantType]"

I tried to fix this locally with the same idea you applied (using generic types) but bumped into an issue when trying to do:
@dataclass
class Request(Generic[TQuery, TPost]):
  ...
  query: TQuery = Query()
  post: TPost = Post()
  ...

mypy didn't like having a different type as the default value. thoughts?

well, something like this should work:
TQuery = TypeVar("TQuery", bound=Query)
TPost = TypeVar("TPost", bound=Post)

@dataclass
class BaseRequest(Generic[TQuery, TPost]):
    method: RequestMethod
    query: TQuery
    post: TPost
    headers: HTTPHeaderDict = HTTPHeaderDict()
    url: str = ""
    user: Optional[Any] = None
    settings: Settings = Settings()

@dataclass
class Request(BaseRequest[Query, Post]):
    """Object that contains a client's complete request."""
    headers: HTTPHeaderDict = HTTPHeaderDict()
    query: Query = Query()
    post: Post = Post()
    url: str = ""
    user: Optional[Any] = None
    settings: Settings = Settings()
    method: RequestMethod

I'm checking now.

@pedropregueiro could you check what i just pushed?

sorry for the delay, this seems to be working as expected now, mypy is happy!
I had to change the CustomPost initialization, but that seems to be a diff issue. here's the latest working sample code for reference:
from dataclasses import dataclass
from typing import Any, Optional

from aioauth.requests import Query, Post, BaseRequest
from aioauth.types import RequestMethod
from starlette.datastructures import ImmutableMultiDict

@dataclass
class MyPost(Post):
    channel: Optional[str] = None

@dataclass
class MyRequest(BaseRequest[Query, MyPost, Any]):
    post: MyPost = MyPost()
    query: Query = Query()

test_form_dict = ImmutableMultiDict(
    {"grant_type": "authorization_code", "channel": "123"}
)
post: MyPost = MyPost(**test_form_dict)

oauth2_request = MyRequest(
    method=RequestMethod.POST,
    post=post,
)

print("request", oauth2_request)
print("channel", oauth2_request.post.channel)

result
request MyRequest(method=<RequestMethod.POST: 'POST'>, query=Query(client_id=None, redirect_uri='', response_type=None, state='', scope='', nonce=None, code_challenge_method=None, code_challenge=None, response_mode=None), post=MyPost(grant_type='authorization_code', client_id=None, client_secret=None, redirect_uri=None, scope='', username=None, password=None, refresh_token=None, code=None, token=None, token_type_hint=None, code_verifier=None, channel='123'), headers={}, url='', user=None, settings=Settings(TOKEN_EXPIRES_IN=86400, REFRESH_TOKEN_EXPIRES_IN=172800, AUTHORIZATION_CODE_EXPIRES_IN=300, INSECURE_TRANSPORT=False, ERROR_URI='', AVAILABLE=True))
channel 123

I'm using the ImmutableMultiDict because that's the type I'll get from FastAPI requests form data. That said, if I try to instantiate the MyPost() just using a normal dictionary, mypy will keep failing with the error I described before: error: Argument 1 to "MyPost" has incompatible type "**Dict[str, str]"; expected "Optional[GrantType]"

try like this?

from aioauth.types import GrantType

test_form_dict = ImmutableMultiDict(
    {"grant_type": GrantType.TYPE_AUTHORIZATION_CODE, "channel": "123"}
)

yeah, of course, but I meant when parsing a framework's Request (eg: FastAPI) into aioauth's structure. I was using the to_oauth2_request method from your fastapi-extension as inspiration, but that's where the type issues were coming from:
async def to_oauth2_request(
    request: Request, settings: Settings = Settings()
) -> MyRequest:
  form = await request.form()  # type: ImmutableMultiDict
  post = dict(form)  # type: dict
  # ... some more code hidden
  return MyRequest(
    post=MyPost(**post),  # this gives the incompatible type error
  )

A simple change, fixed this for me:
async def to_oauth2_request(
    request: Request, settings: Settings = Settings()
) -> MyRequest:
  form = await request.form()
  post: MyPost = MyPost(**form)
  # ... some more code hidden
  return MyRequest(
    post=post,  # ok types
  )

Anyway, this feels unrelated now. Thanks for fixing the custom types!

maybe we should use Literals instead of enums. I'll take a look later

I can add these changes to the current PR, but I'm not sure how soon I can merge it, since the documentation needs to be edited, a lot has changed.

Can maybe open a diff issue for the literal work, if that's what you think will delay a release?
The way I have it now works well and is valid with mypy, not in a rush to change the enums.

@pedropregueiro I think that I will make a release on the weekend, as this requires updating the documentation as well. give me some time

Of course, no problems!

@pedropregueiro v1.5.0 has been released, including the fixed with literals (enums)

With the latest changes, I have to either maintain my own enums or have some hardcoded strings in the code:
if request.post.grant_type == "authorization_code":

Rather than what was possible to do before:
if request.post.grant_type == GrantType.TYPE_AUTHORIZATION_CODE:

Is there a better way to do this?

I see several issues with Enum that I come up in this project:

they must be explicitly converted to a string, otherwise mypy will complain

duplicates in the code. how does GrantType.TYPE_AUTHORIZATION_CODE actually differ from the string literal "authorization_code"? Actually, there is no difference. It only adds complication in the code and unnecessary imports.

string literals are typed, so the following code will throw a mypy error:
from enum import Enum
from typing import Literal

GrantType = Literal["authorization_code"]

def check_grant_type(grant_type: GrantType):
    ...

check_grant_type("refresh_token")

replacing check_grant_type("refresh_token") with check_grant_type("authorization_code") will fix the error.
so string literals are very useful, they add typing, and at the same time you can enjoy all the benefits of strings without additional type casting.
in your case, I think the first option is quite appropriate.

duplicates in the code. how does GrantType.TYPE_AUTHORIZATION_CODE actually differ from the string literal "authorization_code"? Actually, there is no difference. It only adds complication in the code and unnecessary imports.

I disagree with this. It differs because explicitly having to write the string (e.g.) "authorization_code" might generate odd errors on the developer end. Someone might write "authorisation_code" or a myriad of other typos. Even further, someone might confuse grant_type with reponse_type and just use the word "code" wrongfully.
In both cases, unless the dev is using a type checker (and continuously re-checking their code on each change), the errors will go unnoticed as the interpreter won't raise them on start. This will lead to unnecessary time trying to figure out why things are failing down the line. Especially with OAuth2, where errors tend to be slightly generalized for obfuscation purposes.

Someone might write "authorisation_code" or a myriad of other typos. Even further, someone might confuse grant_type with reponse_type and just use the word "code" wrongfully.

Literal won't let you do that, mypy will complain if the passed string doesn't match what it expects:
(.venv) ➜  ~ cat test_enum.py
from enum import Enum
from typing import Literal

GrantType = Literal["authorization_code"]

def check_grant_type(grant_type: GrantType):
    ...

check_grant_type("authorisation_code")
(.venv) ➜  ~ mypy test_enum.py
test_enum.py:12: error: Argument 1 to "check_grant_type" has incompatible type "Literal['authorisation_code']"; expected "Literal['authorization_code']"
Found 1 error in 1 file (checked 1 source file)
(.venv) ➜  ~

mypy will complain if the passed string doesn't match what it expects

Indeed, but if you take the same example and do python test_enum.py you won't get any errors calling check_grant_type("authorisation_code")

that's why we have to rely on CI with mypy and tests :)

100% agree with having mypy and tests (+more) on CI, and I also use pre-commit locally as an additional check as you do on aioauth. That said, I think it's risky to assume that all devs using the library will be using mypy or any form of type checking. Still early days for typed Python after all :)
In any case, I made the changes to make this work for me, so mostly discussing this for the future of the library and upcoming users hehe


[api] Address.model: always return lower case addresses across all resources

Background: We should try to use Checksum casing when dealing with user-entered addresses, or providing addresses to the user for copy-paste etc. The checksum provides an extra layer of validation.
However for the api, returning addresses with the checksum case creates a ton of boilerplate .toLowerCase() and introduces a lot of bugs.
It would be incredibly convenient to know that an address is always lowercase and comparable, if it came from the API.
Optional stretch goal: we can use a JsonSchema Regex and create a reuseable Address model to validate that all the addresses we return are valid and are lower-case.

Hey @Atmosfearful have you thought about something like string manipulation types?
https://www.typescriptlang.org/docs/handbook/2/template-literal-types.html#lowercasestringtype
Maybe we could even use this for project ids


Encrypted root

I didn’t see any option to use an encrypted root during install but I may have overlooked it.
I think since FreeBSD 13 is coming soon it’s best to wait for that since then the OpenZFS native encryption feature can be used. Rather than to make it use geli as one would need with 12.x I mean.

+1
https://github.com/helloSystem/hello/issues/60 (apologies for overlap)

… OpenZFS native encryption …

https://github.com/helloSystem/ISO/issues/135 there's packaged OpenZFS with FreeBSD 12.2-RELEASE-p3

… FreeBSD 13 … best to wait …

Fair enough, if (say) fbsd13 will become the default branch before too long …
https://github.com/helloSystem/ISO/issues/124 re: branch names.
Keywords privacy and security
https://github.com/helloSystem/hello/wiki/Brainstorming/_compare/05df4f8222660ccd72e5bed2a229dd249ebeb6c5#diff-81a2ecfe1fa5cdadf7732f754d891e0eb1c0a063a73d9c653375da399681dd32R104
https://github.com/helloSystem/hello/releases/tag/draft
https://github.com/helloSystem/Utilities/blob/8d8e2c6daa23f55e78252be224bc270fc24d35ae/System/Welcome.app/Resources/content.en.md#L10 and https://github.com/helloSystem/Utilities/blob/8d8e2c6daa23f55e78252be224bc270fc24d35ae/System/Welcome.app/Resources/content.en.md#L14
… and so on 👍


The link in the description is dead

http://people.cs.uchicago.edu/~rchugh/djs doesn't appear to be an existing site.

Oops, sorry. I have that old page saved, but there's nothing interesting: it just contains links to things that can be found on my webpage (POPL 2012 and OOPSLA 2012 papers and a SoCal 2011 poster) and links to a few commits in this repo.


error when connecting to socket

when i run gos from command line successfully, then try to connect to channel from the browser the connection is terminated, and i get this exception in the terminal, any idea ?
[Gos\Bundle\WebSocketBundle\Client\Exception\StorageException]
Driver Gos\Bundle\WebSocketBundle\Client\ClientStorage failed
[Predis\Connection\ConnectionException]
Connection refused [tcp://<IP_ADDRESS>:6379]

connection refused means the address is not reachable nor the server reject you. Have you try to connect to your redis server from outside the websocket context (like in a conttroller, just hardcode the connection to try)

thanks ,, it was because of redis, solved now

no problem :)


Update route-registrar to use go-nats

[#152164959]

@ushachar @yossigo can you please review? (no one else has writing permission to this repo)


add reql pro to third party administration tools

Once upon a time Daniel say .

ReQLPro provides a data browser for RethinkDB
It also allows you to update values in-place by double-clicking them in the table view.

I like update the third-party list, with add reqlpro section
cc @csndra0504, @grailian

Closed in #1216

Thanks! @brutalcrozt 🙏


Add russian language

Hi everyone!
Thanks for creating and supporting this project. I'm used it for creating own competition. I needed to support Russian language and I added new localize file.
Now I want to contribute this language file in the project.

Hi again @shipko
I really appropriate all the work that must of you did for this PR and would immediately merge this under normal circumstances. But with Putins current war on Ukraine I don't feel comfortable adding a russian translation to a security related project, especially as we don't have a ukranian translation yet.


Lorem Ipsum Generator

Is your feature request related to a problem? Please describe.
A web App that would generate Lorem Ipsum statements which can be used to just have demo on the website we make.
Describe the solution you'd like.
N/a
Describe alternatives you've considered.
N/a
Add any other context or screenshots about the feature request here.
No response

Assign @swapnilsparsh


Blending result of supervised GP-GAN

Hi,
Thanks for sharing your work.
I have run the latest code and got the test result (supervised GP-GAN) like the following image:

The result looks like it does not work properly.
Is this an issue?

I haven't tested the latest code. It shouldn't look like this.
Can you run the code in commit 9213b41005259a19c5cb9c56e598b78f1f0b9123?

I haven't tested the latest code. It shouldn't look like this.
Can you run the code in commit 9213b41005259a19c5cb9c56e598b78f1f0b9123?

Just tested with commit "9213b41005259a19c5cb9c56e598b78f1f0b9123" with bad luck, got the same result.
I wonder if it is the issue of migrating to newer Chainer version.
Here is my development environment:
OS: Ubuntu 18.04
Cuda: 10.0
Chainer: 6.3

In 9213b41005259a19c5cb9c56e598b78f1f0b9123 , chainer==1.24
What's your command to get the result?

1. In `9213b41005259a19c5cb9c56e598b78f1f0b9123` , chainer==1.24

2. What's your command to get the result?

I used the command described in the README.md file.
Command:
python run_gp_gan.py --src_image images/test_images/src.jpg --dst_image images/test_images/dst.jpg --mask_image images/test_images/mask.png --blended_image images/test_images/result.png

Then the only issue is the version of chainer. I will work on this problem next week.

Now the model can be run in test mode correctly. See here for the instructions.

Now the model can be run in test mode correctly. See here for the instructions.

@wuhuikai
Thank you very much~
By the way, does the problem cause by the wrong Chainer's configure flag?

Yes, Chainer's config doesn't have the attr test but train.


Update drugWarnings annotation in the Platform

As part of ChEMBL32, the team has revised the drug warnings annotation to extract -when possible-  the specific adverse outcome associated with the drug warning.
Out of 854 drugs with some sort of safety warning, 219 have specific annotation about the reported adverse outcome and their mapping to EFO. We need to update the Drug Warnings datasets and widget to display this data.
Tasks
Process the drugWarnings dataset to include the new fields:
efo_term: label of the curated EFO term that represents the adverse outcome
efo_id: ID of the curated EFO term that represents the adverse outcome
efo_id_for_warning_class: ID of the curated EFO term that represents the high level warning class

[x] Update PIS to pull the new fields
[x] Update the ETL to integrate the new fields
[x] Add the new fields to ES/Clickhouse and GraphQL
[x] Update the Drug Warnings widget

Context
An example of the new annotation is Carisopropodol, where the high level class is just "misuse" (OT page):

and now we have the specific AOs that led to this drug being withdrawn (chembl page):

More documentation about this changes here

@ireneisdoomed could you review and close if appropriate?

Looks good to me, 2 small details that'd be nice to have before going live:

I'd like the link to point to the latest OLS, since OLS3 is no longer updated:

Instead of https://www.ebi.ac.uk/ols/ontologies/efo/terms?short_form=, use https://www.ebi.ac.uk/ols4/ontologies/efo/terms?short_form=

In the tooltip, remove the extra whitespace between in Description : ...


<IP_ADDRESS>

@bf-cy hello.  these pull requests don't make much sense. what are you trying to do exactly?


connection Test() doesn't really mean if a connection is authenticated and ready

The problem
Seems like the Connection.Test() is broken. It doesn't really test if the connection is authenticated and can be used.
For long running processes, which want to periodically invoke api commands, we want to use a cached connection. There is no need to recreate the connection over and over again, and to recreate sessions.
On a case of a session termination, for any reason, we need to supply a way to clients the understand that, and create a new connection. Part of the problem is that ovirt doesn't support service-accounts which don't have a session nature. This itself deserve an RFE to ovirt separate issue)
For now I would like this to work reliably:
response, err := connection.Test()

How to reproduce:

create a client code which creates a connection and wait for engine restart signal:

package main

import (
        "fmt"
        "os"
        "time"
        "github.com/ovirt/go-ovirt"
)

func main() {
        c, err := ovirtsdk.NewConnectionBuilder().
                URL(url).
                Username(username).
                Password(password).
                CAFile(cafile).
                Insecure(insecure).
                Build()

        if err != nil {
                panic(err)
        }
        fmt.Printf("system service %v", c.SystemService().Get().MustSend())
        for {
                time.Sleep(5 * time.Second)
                f := "/tmp/engine.restart.done"
                fmt.Printf("waiting for egine restart - watchin file %s \n", f)
                fileInfo, err := os.Stat(f)
                if err != nil || fileInfo == nil {
                        fmt.Printf("file %s doesn't exists yet\n", f)
                        continue
                }
                break
        }

        fmt.Println(c.Test())

remove the session from the DB and restart ovirt-engine, to simulate session timeout
to remove the session from the db , clear the session table:    delete from engine_sessions;

now touch /tmp/engine.restart.done

Expected behavior: [What you expect to happen]
Test() should return false when the connection isn't authenticated
Actual behavior: [What actually happens]
Test() returns nil leading the client to think the connection is valid
Reproduces how often: [What percentage of the time does it reproduce?]
100%

@rgolangh The idea of reusable connection and lightweight session is great to me. Looking forward to the RFE and I could also get involved in that job.
Regarding to Test() for connection, my original purpose was to check if the attributes are able to correctly establish a connection. So connection.Test() will always try to redo the authentication if no existed sshToken, and lead to the issue mentioned  above. I like your opinion to make Test() more reliable. Many thanks to you for this.

Another thing that bothers me, is that when the session isn't authenticated, the sdk code returns an error to process an html, which is what the api returns when redirecting the connection if its not autheticated.

@imjoey this is kind of urgent to me - I can take it, and would just need a review from you.

here is a working prototype so far:
func (c *Connection) Test() error {
	options, err := http.NewRequest(http.MethodOptions, "", nil)
	if err != nil {
		// shouldn't fail to construct a request, but report anyway.
		return err
	}
	// a simple http OPTIONS request is the lightest method to test auth.
	_, err = c.client.Do(options)
	if err != nil {
		// failed, then clear state.
		c.ssoToken = ""
	}
	_, err = c.authenticate()
	return err
}

Another thing that bothers me, is that when the session isn't authenticated, the sdk code returns an error to process an html, which is what the api returns when redirecting the connection if its not autheticated.

Hi @rgolangh , Apologize for being late as no chunk of time to have a in-depth consideration about this. Hope not block your work.
Regarding the html error, shall we check the status code of the response in advance before calling json.Unmarshal(body, &instance-of-ssoResponseJSON)?

thanks for looking, yes that would probably be the way to go.
I would expect an error code of 401 . In fact any > 300 should be not marshalled?
We don't have something to handle response code in the SDK?

The response codes of the APIs sent to engine are handled, but not for the one of the APIs to access SSO. See the codes here at https://github.com/oVirt/ovirt-engine-sdk-go/blob/8e4607561f1a518a20bde242d18639984fee33aa/sdk/ovirtsdk/connection.go#L285-L290. I could fix this by return an error when response code > 300. What's your insights? Please let me know. Thanks.

Yes I think we should handle that for all codes which are >= 300
On Thu, 6 Feb 2020 at 17:01, Joey<EMAIL_ADDRESS>wrote:

The response codes of the APIs sent to engine are handled, but not for the
one of the APIs to access SSO. See the codes here at
https://github.com/oVirt/ovirt-engine-sdk-go/blob/8e4607561f1a518a20bde242d18639984fee33aa/sdk/ovirtsdk/connection.go#L285-L290.
I could fix this by return an error when response code > 300. What's your
insights? Please let me know. Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/oVirt/ovirt-engine-sdk-go/issues/193?email_source=notifications&email_token=ABGBYHCJTSFFUJ74NUQMJEDRBQQ4XA5CNFSM4KNDHQL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK7QYUA#issuecomment-582945872,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABGBYHFQBGUS3DPFWFX7GETRBQQ4XANCNFSM4KNDHQLQ
.

That's right. 300 should be also included. Thanks for the correction.

You had a chance to review #197?
On Thu, 6 Feb 2020 at 18:06, Joey<EMAIL_ADDRESS>wrote:

That's right. 300 should be also included. Thanks for the correction.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/oVirt/ovirt-engine-sdk-go/issues/193?email_source=notifications&email_token=ABGBYHG66ZWK4CFIUIA2T5DRBQYO5A5CNFSM4KNDHQL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK7YOOI#issuecomment-582977337,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABGBYHENF5KUTY3YT65IAZDRBQYO5ANCNFSM4KNDHQLQ
.

You had a chance to review #197?

You had a chance to review #197?

Yep, it's merged and thank you for that. Do you think a rapid new release is required for your current work?

Not urgemt - I would currently bump ovirt/cluster-api-provider go module to use that.


FT: Adds jenkins declarative yaml pipepline with CI stage

Adds jenkins declarative yaml pipepline with CI stage for server.
Adds continuous-integration docker image.

Test

Test

Test


wrapLabel: check for case when value is set without an attribute.

This handles a scenario when using certain frameworks that handle data-binding outside of the value attribute.  For example in Angular an input will look something like
Small change will check in 'wrapLabel' method to see if the element has a value even if a value attribute is not present.
<input id="firstName" type="text" [(ngModel)]="person.firstName">

Wouldn't it be simpler to just change it from this:
if( el.hasAttribute( 'value' ) && el.value.length ) {
    wrapper.classList.add( this.prefixed( 'is-active' ));
}

To this:
if( el.value !== undefined && el.value.length ) {
    wrapper.classList.add( this.prefixed( 'is-active' ));
}

?

Yes, it would.


pull request for e2e test at 01/22/2017 20:26:16

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


44 defence filtering

There seems to be an issue when adding a string which is just numbers e.g. '999'


Update Synapse contacts

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

@wonner  please help  to confirm, thanks!


Update version to 2.0.0

@yuit can you take a look.

:+1:

@mhegazy  @DanielRosenwasser  just install the nightly. Install successfully!


Card class

I deviated a bit from the diagram and removed the match() function, since I ended up using dataClass wich handdles all the __eq__ function stuff.
Only the color and the value are used in the constructor since the image src can be generated from that information so in the __post_init__ we generate it usign the values.
Also apply some PEP8.

Match method of card has the intention of compare if the receiving card is the same color OR the same value, so it will be necessary regardless the dataclass.

Oh ok I made a mistake while reading the docs. Here it its


[FEATURE REQ] Managed Identity Support for Azure Synapse spark notebooks/jobs

Is your feature request related to a problem? Please describe.
The Azure Cosmos Spark library lacks support for Spark jobs running in Azure Synapse, although Synapse supports linked service creation for Azure Cosmos DB.
Describe the solution you'd like
Need a solution that can leverage the linked service created for a cosmos db account in the synapse and make use of it in the connector to connect to cosmos db in spark notebooks and jobs
 "spark.cosmos.auth.type" -> "ManagedIdentity"
  "spark.cosmos.auth.linkedService" -> "My linked Service for comos db"

Describe alternatives you've considered
Additional context

provided the managed identity rbac needed and created a linked service successfuly
opened a synapse notebook with azure cosmos spark library installed
added the spark.cosmos.auth.type as ManagedIdentity and provided the manged identity object id in clientid

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[*] Description Added
[*] Expected solution specified

This is a limitation in Azure Synapse - not the Cosmos Spark connector - Neither the Synapse Token Library / LSR allows retrieving managed identity tokens for Cosmos DB Linked Services nor does Azure Synapse provide the workspace's managed identity to Spark Executors (like Azure Databricks does). So, until Azure Synapse addresses this limitation the only options are to use ServicePrincipal or switch to Azure Databricks (which supports authentication with Managed Identity).


How to read selected option from dropdown in Golem v2?

Hi,
Is there a way, in Golem, to get the currently selection option in a dropdown (read the value)?
In bare selenium one can use the SelectedOption property of the SelectElement class.
I have HTML that looks like:
`<select.... >
another option
displayed text for option
`
thank you!

There isn't a property or method that exposes this, but you can always use the actual WebElement, which is stored in the element property.
var someElement = new Element(By.Id("SomeId"));
var webelement = someElement.WrappedElement;  //finds the element on this step
var select = new SelectElement(webelement);
var text = select.SelectedOption.Text;
I will post an update soon with some methods in the element class to handle this.

Thank you!


unittests

Hi @Khushi369 ,
Could you explain to me this PR? I've checked few files in it and their content is the same as in test directory but file names have been changed? Second thing, I don't know if I explained it correctly here: https://github.com/DataverseLabs/pyinterpolate/issues/165 but the core branch for those updates is v0.3, not main. I can redirect this PR into v0.3 but I must know the reasoning behind it.
Thanks,


I want to help you, but need some clarification

Hi!
I want to learn Rust and Bevy and improve your battle-city project at the same time.
I noticed that your implementation of the battle-city game is different that the original one from Nintendo:

(and to compare with a screenshot from your implementation)

For example, the area is 27x18 in your game instead of 13x13 in the original. I think it is not an issue and it is easy to change, but  was it made by intend? How about to create an original 13x13 map just for development sake to make sure it matches the original game?
There are also a lot of other differences.
The most important are: the size of the battle tank is much less than the area of a cell (one cell of 27), but in the original the size of the tank is almost the same as the cell

Is it a bug, can I fix that, or you want it to be like that?
I noticed you game is missing some original features, like partial walls damage, and two bullets crash.

Can I try to implement those features after fixing the size of the battle tanks?
Thank you for a great opportunity to learn bevy on your project.

Here is a list of similar projects to get inspiration from:
https://github.com/dogballs/cattle-bity - the most implemented version, it is almost full clone of the original Battle City by Namco (1985). It is written on TypeScript, unfortunately.
We can grab a transparent full sprite 32x32 from there, by the way https://github.com/dogballs/cattle-bity/blob/master/data/graphics/sprite.png
https://github.com/newagebegins/BattleCity - a decent javascript implementation.
https://github.com/olegpolyakov/battlecity - it is a javascript version too, but the cool thing is that the author made 10 full-length video about how he made it. Unfortunatlely for anybody (not me), it's on Russian language, so you can use subtiteles or just skip it. https://www.youtube.com/playlist?list=PLqHlAwsJRxAPZ5LQpGfZRQjLeWIEIfX2-
We can grab some ideas about how to use particles system from there.

I created this project for learning bevy and Ldtk, not for creating a clone of original game, so just developed it based on my memory. But if you can help developing it to make sure it matches the original game, that will be really cool and I'm very appreciated!

Thank you very much for the answers!
How do you think, when will be a proper time to move this project to Bevy 0.13? I noticed there is lack of documentation for 0.13 (no TextureAtlas::from_grid for example, but it definitely exists in the migration guide https://bevyengine.org/learn/migration-guides/0-12-to-0-13/), so I think it is wise to wait till it be more mature, right? What do you think?


SSH on panes and run commands in the new shell

Hi, I have the below itermocil template and it works partially. When I execute it, it opens 3 panes in the right layout and it runs an ssh command (from a variable) and logs in successfully. However the second and third panes do not run ps faux and top even after sleeping a few seconds. I noticed that once I exit those ssh sessions, the ps and top commands will run but I want them to run in the ssh'ed session. Is there a way to do this?
 windows:
   - name: quichssh
     root: ~
     layout: main-vertical
     panes:
       - commands:
         - ${ssh}
       - commands:
         - ${ssh}
         - sleep 2 && ps faux
       - commands:
         - ${ssh}
         - sleep 2 && top

I saw a prevoius post  about adding -t flag to the ssh command but lets' assume i can't change ${ssh} as it's coming from another automated variable. As a last resource, I can try to parse and change ${ssh} and append -t as an additional argument if there is nothing else to do. But my priority is to see if there is an easier way. Thanks in advanced!

While not useful for the exact issue, when the ssh command is known this can be used:
ssh <server> -t "bash -ic '<commands here, separated by semicolons>; bash -l'"
# e.g.: create ssh tunnel, load python env, then start jupyter server
ssh <server> -L 8810:cn50:8888 -t "bash -ic 'lpy; jupyter lab --ip <IP_ADDRESS> --no-browser; bash -l'"

It allows to run commands after loading .bashrc, then keeps the interactive shell alive.


Split ci

Motivation

Progress on #30.
Description of the Changes

Split all integration modules and tests into their respective directories.
Added a workflow file for each integration module.
Removed tests.yml and tests-api.yml.

Let me resolve conflicts with myself.


Error when trying to open .sol

Error
Error reading file:
NetworkError: Failed to load worker script at "http://cdn.coursevector.com/minerva/4.1.2/js/lib/ByteArray.js"
(http://apps.coursevector.com/minerva/js/parsers/SOLReaderWorker.js:1)
Not sure what this means really but it prevents editting saves.

Try again? Not sure what to tell you. Maybe it's a caching issue, clear your browser cache. I've been using it regularly the past few days without issue.
Looks like you're using a really old version of it though, I'm on 4.3.1 and it's looking for 4.1.2 which is long gone.
Don't know what browser you're using, but clear out any application caching from the service worker might help.

The web version of .minerva does work offline if you use a modern browser.
As for the legacy version (Adobe AIR), the issue was that Adobe requires you to package the file with a signed certificate that typically expired every 2 years. Which meant I had to repackage it regardless if there were any changes or it would fail to install. Maybe a month or so ago I found out how to repackage it with a certificate that lasts for any number of years. So i have a repackaged version with a 50 year certificate. I updated the link in the About window on .minerva but forgot to update the webpage. Here is the link for the re-signed version: https://mariani-cdn.s3.amazonaws.com/download/Minerva-3-5-0-resigned.air


Links are unclicakble

Links are rendered and appear in blue yet unclickable.
Example:
@startuml
Bob -> Alice : [[https://example.com{Optional} Hello]]
@enduml

Link to the example

Check the SVG version: https://plantuml.github.io/plantuml-core/svg.html?SyfFKj2rKt3CoKnELR1Ik3xl-XI5wrE0rBBh4-qLegCpIaeAYgtqzLCh4dCBSbBralDpg_qBIZBpyn9pQXKyKdDoycDZ0G00

It works on the SVG version, now the glyphs appear as Unicode placeholders in my linked example.
Thanks.


Form guidance imagery

Finish guidance imagery for 'Forms' so it can be put into the website.

Done


Bug in edit ticket

I want that fields 'Subject' and 'Content' are disables in edit ticket, but when i remove 'required' and add 'disabled' to fields 'Subject' and 'Content' appears the error below.

If i remove 'disabled', works!
Can someone help me?
Thanks!

What about making them hidden?

But let me mention that making them hidden or disabling them doesn't stop users from submitting them, you need to edit TicketsController upload function to the appropriate validation.

How to hidden field?
Disabled not works?

Won't work because of the validation rules as it makes subject and content fields required fields, and disabled fields won't be posted.


Scheduled task generates unhandled exceptions in the site log

Describe the bug
Scheduled task generates unhandled exceptions in the site log
To Reproduce
Steps to reproduce the behavior:
We just need to run the scheduler in debug mode and step through the code to see the unhandled exceptions. I think it relates to changes to time utilities in the Dnn APIs.

That was on Dnn 9.1.1 instance.

I cannot reproduce this in my current Events 7.0.3 (under development).
Since I corrected a whole lot of string comparisons, it might be solved by that.
Closing issue now. If the error resurfaces, reopen the issue


Intelligent Dispatching Sensor no longer available

Describe the bug
After update to v8.4.0 the binary_sensor.octopus_energy_intelligent_dispatching is no longer available
I've read the notes around the new events triggers but I can't see anything that says this has been removed.
Reproduction steps
Need to be on Intelligent Octopus
Simply upgraded to v8.4.0 and it is no longer provided
Expected behaviour
The binary sensor would be available and turns true when the IO low tariff is active, it's attributes provide the dispatch schedules.
Tariff Code
Octopus Intelligent
Integration Version
v8.4.0
Home Assistant Version
2023.10.1
Fresh Install?
After upgrading
Home Assistant Logs
2023-10-07 12:09:23.322 ERROR (MainThread) [homeassistant.components.binary_sensor] Error adding entities for domain binary_sensor with platform octopus_energy
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
state, attr = self._async_generate_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/binary_sensor/init.py", line 218, in state
if (is_on := self.is_on) is None:
^^^^^^^^^^
File "/config/custom_components/octopus_energy/intelligent/dispatching.py", line 72, in is_on
rates = self.coordinator.data.rates if self.coordinator is not None and self.coordinator.data is not None else None
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'rates'
2023-10-07 12:09:23.334 ERROR (MainThread) [homeassistant.components.binary_sensor] Error while setting up octopus_energy platform for binary_sensor
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 367, in _async_setup_platform
await asyncio.gather(*pending)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
state, attr = self._async_generate_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/binary_sensor/init.py", line 218, in state
if (is_on := self.is_on) is None:
^^^^^^^^^^
File "/config/custom_components/octopus_energy/intelligent/dispatching.py", line 72, in is_on
rates = self.coordinator.data.rates if self.coordinator is not None and self.coordinator.data is not None else None
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'rates'

Reverted to v8.3.0 - working ok.

Sorry about that. This should be fixed in v8.4.1

no problem, thanks for all your great work 👍

Since the update the intelligent dispatch sensor is now available ok but the standard low tariff rate is not being set correctly.
Rather than switching on at 11:30pm and off at 5:30am, it switches on erratically somedays at 11:59 and off at 5:59, or on others at 12:08 and off at 6:08am (but never at 11:30 and 5:30)
It seems to have lost it's precision around the 'standard' low tariff times - I haven't had any out of hours slots since then so can't confirm the accuracy of those but i'll try and get one today and see what it shows.
First 2 screen shots show before the update, last 2 after the update

@FozzieUK As this is a separate bug from the original, it would be good in future to raise a separate ticket. This helps me work out what is outstanding and what isn't at a glance, as well as helps others looking for similar issues that may or may not still be an issue. I've moved your issue into https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/issues/466, and will therefore reclose this issue.


How to add EFA to instances (or custom resources requests)

Description
How can the docs be improved?
Despite seeing support for EFA in the code, it is unclear how to make this happen for good.
AFAICT, the EFA interfaces are enabled when the NodeClaim resources requests any number of vpc.amazonaws.com/efa devices, but since NodeClaim objects are created by Karpenter itself, it's unclear how to express that in either the Nodepool or the EC2NodeClass definition.
Nodepool used to have a spec.template.spec.resources that looked promising, but it's been removed in 0.33 (maybe that was a red herring?).
Any guidance to support this would be appreciated, and docs would benefit from providing an example how to achieve this.
Thanks!

NodeClaim requests are based on the resource requests made by pods the NodeClaim is created for. Do you have a use case where you want to configure EFA when no pods are requesting it?

Thank you @jmdeal, I was able to figure this out later yesterday but forgot to update the issue.
Indeed, creating a pod with a resources.requests containing vpc.amazonaws.com/efa: x will pick the right instance and create it 🎉

Do you have a use case where you want to configure EFA when no pods are requesting it?

Actually, yes, but because of a correlation of issues, namely related to Karpenter being unaware of hugepages on target nodes. I'm aware that this is currently being discussed, but would gladly appreciate some guidance on this.
For the context: we're running GPU based nodes where we need support for EFA (as much as the node supports), but also hugepages, for GPU RDMA.
In order to create the node, I can simply create a "fake/balloon" pod that requests all EFA interfaces that the underlying instance type supports and expect to see the instance coming up properly configured.
Now, the issue, is that once nodes are created, I would need to stop my fake/balloon pod to release the EFA devices for the real business pod to be scheduled. This makes orchestration harder to achieve, especially in the context of automatic replacement of nodes (think immutable infra, security compliance, ...)
 The fake balloon pods would allow for creating the nodes (no hugepages request, no EFA), and as soon as nodes would come up, the business pods would be scheduled.

Also, having EFA unconditionally for nodes of a nodepool would help with labelling the nodes, so that aws-efa-k8s-device-plugin targets those nodes.
Right now, there's no way to differentiate (label-wise) 2 nodes coming from the same nodepool, where one would have EFA and the other wouldn't. This leads to the plugin on one node (no EFA) while successfully working on the other.

Got it, so it sounds like this is mainly a workaround for Karpenter not supporting hugepages. You can't use the business pods directly, so you use balloon pods which don't request hugepages and the business pods can schedule once the nodes are created and the resources are registered. Out of curiosity, what additional orchestration would you need to do if the balloon pods requested EFA resources? Are they not currently being preempted by the business pods?
Either way, I don't personally see any reason Karpenter shouldn't support this. IMO a reasonable semantic would be if vpc.amazonaws.com/efa is specified in requirements, instances would always be configured for EFA. Otherwise, the current dynamic behavior would be used. I'm not sure how much work this would entail, but I'm going to go ahead and change this to a feature request.

Out of curiosity, what additional orchestration would you need to do if the balloon pods requested EFA resources? Are they not currently being preempted by the business pods?

The balloon pods would be preempted indeed, but they would end up in pending state, further triggering Karpenter again, creating more nodes.
Obviously, that's based on the assumption that those pods would be driven by a deployment. Turning this into a job-based solution would solve that problem, but it would prevent using those same balloon pods again for automatic node replacement.

Either way, I don't personally see any reason Karpenter shouldn't support this. IMO a reasonable semantic would be if vpc.amazonaws.com/efa is specified in requirements, instances would always be configured for EFA. Otherwise, the current dynamic behavior would be used. I'm not sure how much work this would entail, but I'm going to go ahead and change this to a feature request.

Thank you!
To add more thoughts into this, it seems really important for Karpenter to support and understand EFA for nodepools because EFA usually requires a dedicated security group, as well as a custom label for the aws-efa-k8s-device-plugin to target those nodes only.
It makes a lot of sense for this setup to be opinionated and encoded into configuration (nodepool + ec2nodeclass) for the rest of the infra to support this properly.


Onboard MongoCluster for first beta in June

https://github.com/Azure/azure-rest-api-specs/pull/29642

https://github.com/Azure/azure-rest-api-specs/pull/29442


fix: remove legacy span creation

We were creating spans in a deprecated way.

Could be related: https://github.com/rocket-connect/graphql-debugger


Skip adding preventDefault to onChange events in React Native

Bug
When using onChange in React Native, the users are faced with a couple possible error messages, as seen in #3153 and #3238.
This is due to the fact that it doesn't seem to be possible to properly extends React Native's event ( https://github.com/facebook/react-native/issues/5507). My proposal is to skip adding .preventDefault to TextInput's onChange's event.
Tests
No test was broken
Eslint
Linting was respected
Closes #3153, closes #3238

Codecov Report

Merging #3460 into master will decrease coverage by 0.06%.
The diff coverage is 80%.

@@            Coverage Diff             @@
##           master    #3460      +/-   ##
==========================================
- Coverage     100%   99.93%   -0.07%
==========================================
  Files          68       68
  Lines        1531     1533       +2
==========================================
+ Hits         1531     1532       +1
- Misses          0        1       +1

Impacted Files
Coverage Δ

src/ConnectedField.js
98.94% <80%> (-1.06%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0837539...5136938. Read the comment docs.

@gustavohenke Yep, I've already tried it, and with it I only achieved to delay the error: instead of appearing at the first change, it would appear at the second.
Also, I've tried the "lazy" way, described here, but calling event.preventDefault() from the app side wouldn't do anything (double checked with proper console.logs)

As per the tests, you're actually right, here's my plan.
The test should assert that in a RN environment the following snippet would throw:
onChange(
  {
    ...event,
    preventDefault: () => {
      defaultPrevented = true
      return eventPreventDefault(event)
    }
  },
  newValue,
  previousValue
)

My issue is that I have no idea on how to emulate RN's TextInput's event. Would you mind helping me with the test?

@erikras this PR is used in production by me with no issues whatsoever, but unfortunately I wasn't able to create a test for this PR (to avoid codecov alerts).
Would you please know how to simulate a React Native Event class for a unit test?

Hmm... I'm seeing 100% code coverage when I run it locally. I'm going to merge and then fix any codecov problems later.

Thanks @erikras!
@AlimovSV the PR has been finally merged!

Thanks @erikras, @jkomyno!

Published in v7.1.0.


dnsproxy: 0.34.1 -> 0.35.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/AdguardTeam/dnsproxy/releases.
meta.description for dnsproxy is: "Simple DNS proxy with DoH, DoT, and DNSCrypt support"
meta.homepage for dnsproxy is: "https://github.com/AdguardTeam/dnsproxy"
meta.changelog for dnsproxy is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

directory tree listing: https://gist.github.com/f66f96f08c2513a15d771667d03181e8

du listing: https://gist.github.com/2b032297d88143201b578d40e0ec8674

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
dnsproxy

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5yrf2qjxcqyg282262wbfyhvrzz0wn29-dnsproxy-0.35.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A dnsproxy https://github.com/r-ryantm/nixpkgs/archive/c75fd03b7a2ccdaec4e395a18552659c30950c96.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5yrf2qjxcqyg282262wbfyhvrzz0wn29-dnsproxy-0.35.1
ls -la /nix/store/5yrf2qjxcqyg282262wbfyhvrzz0wn29-dnsproxy-0.35.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

dnsproxy

Maintainer pings
cc @contrun @kalbasit for testing.

Result of nixpkgs-review pr 114638 at c75fd03b run on aarch64-linux 1

1 package built:

dnsproxy

1 suggestion:

warning: unclear-gpl
gpl3 is a deprecated license, check if project uses gpl3Plus or gpl3Only and change meta.license accordingly.
Near pkgs/tools/networking/dnsproxy/default.nix:21:5:
   |
21 |     license = licenses.gpl3;
   |     ^


Error handling for Stress Test

Hook into warning/error events on Container, ensure we have unhandled promise rejections on, etc.

Bonus: Also handle errors better in e2e tests

My plan here is to throw together some simple fault injection mechanism in a branch and run the pipeline against that branch to generate various interesting errors, and see which ones appear in telemetry v. failing the test run, and open issues based on those repro cases.

There are some obvious cases to cover too:

unhandled exceptions/rejections via Node's unhandledExceptionMonitor event
Container warning event
Container closing with error

There are a few other parts of the system that emit error/warning events so I may look over them too.

Moving to April, while also breaking out into smaller items, some of which may happen in March. This will hopefully just close before April actually happens.

Closing this.  I opened issues for logging uncaught exceptions/rejections, but I don't think there are other proactive steps to take.
We are getting more and more real data from the tests running - especially as the stress tests get more interesting with fault injection and leave/rejoin - and as we work through those error logs we'll find cases to fix up in terms of how the test behaves relative to what was logged.


CIS CentOS 7 - CIS-1.5.1

Per the CIS documentation https://benchmarks.cisecurity.org/tools2/linux/CIS_CentOS_Linux_7_Benchmark_v1.1.0.pdf
/boot/grub2/grub.cfg should be updated, but in centos-7-level-1-scored-v1.yaml it is looking for /etc/grub2/grub.cfg this is causing this check to fail

I'm going to add this to #288 to keep things cleaner.


Update package list with grpcurl

Now that it's actually there. Let's update the list.

@LarsBingBong i'm waiting on alpine 3.18 to be published to docker hub before i can merge this PR.


Added MahmoodyGroupBetweenness,GTest and updated BucketPQ

This commit contains the MahmoodyGroupBetweenness-feature and updates BucketPQ with a getKey-Method which is used by MahmoodyGroupBetweenness. GTests for MahmoodyGroupBetweenness are also included.

I already did this but Travis does not pick up existing PRs. It should work for new ones though.

What about creating a new pull request? It is safer in case something breaks.