rs-record frame drops?

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
L515

Firmware Version
<IP_ADDRESS>

Operating System & Version
Linux 18.04

Kernel Version (Linux Only)
4.9.253-tegra

Platform
Jetson Nano

SDK Version
2.49.0

Language
{C++}

Segment
Perception

Issue Description
<Describe your issue / question / feature request / etc..>
Howdy Realsense Group!
My main question is: Is it considered a "frame drop" if the number of frames counted in the callback function is not the same as what is recorded into the rosbag when using the rs-record tool? For one 10sec test, I recorded 184 depth/infrared/colorframes, 253 accel frames and 251 gyro frames. However, when opening the rosbag (using matlab) I get 130 depth, 153 color, 254 accel, 252 gyro frames (im ignoring infrared for now since I dont need it).
Followup Question: Is the discrepancy based off of latency due to the time it takes to write the data, and is there something I can do (i.e. increase the frame_queue size, something that usually seems to be dicouraged) to be able to record every frame that comes through the pipeline?
Followup question for fun: are the depth and camera frames automatically aligned,  can they be aligned before placed into the rosbag, or do I need to perform some alignment based on frame capture time to get the closest frames and then use the align tool in some post-processing code?
Some extra context:
I've been working with my L515 for some time and while I've learned a lot from the examples/tools and other online sources, I can't seem to find the right answer to a question about frame drops. Specifically, I've been learning some of the different methods to gather data using callbacks from examples such as rs-callback, rs-pose-and-image, rs-record-playback etc.
From the rs-callback example, my understanding is that it shows a count for EVERY frame/frameset that the device captures. Now, looking at the rs-post-processing example, it uses frame queues to process the depth data in the callback, enqueue it to the proper queue, then display the frame during the main while loop. I've tried to use both of these principles to essentially use the callback to align the depth and camera frames and place them in their own queues. Then in the main while loop I poll for frames and display both images using opencv and the frame_to_mat method. I also count how many frames are displayed in the main while loop vs. the callback and there's also a discrepancy there...I suspect due to the extra processing needed to display the frames.
Thank y'all so much for your help!
C. Peck
Edit: Fixed some formatting

Hi @Cpeck71 At https://github.com/IntelRealSense/librealsense/issues/7488#issuecomment-704124850 a RealSense team member provides an explanation about how frame drops are calculated in the RealSense SDK.
If a rosbag does not contain all of the frames that were generated during recording then they could be considered to be 'dropped frames'.  Loss of frames from a recording may be caused by a range of factors, such as a slow hard drive not being able to keep up with the rate that frames are being delivered and causing a bottleneck in the recording process, or a CPU's usage percentage nearing or reaching 100%.
If you are using more than one stream type simultaneously then you could consider increasing the frame queue size from its default of '1' to a value of '2'.  This introduces more latency into the stream, which can help to reduce the likelihood of dropped frames.  There is the risk though that using a custom frame queue size may unintentionally break the streams, so '2' would be a safe value to use.  This is described in the Frame Queue Management documentation at the link below.
https://github.com/IntelRealSense/librealsense/wiki/Frame-Buffering-Management-in-RealSense-SDK-2.0#latency-vs-performance
Even then though, it may not guarantee that every frame will be recorded.  An approach that you could explore is using the save_single_frameset instruction to save every frameset into its own tiny individual bag file, as described for C++ in https://github.com/IntelRealSense/librealsense/issues/3671#issuecomment-479767187
Depth and color frames are not automatically aligned.  Although they can be aligned, the aligned frames cannot be saved into a rosbag.  A rosbag acts like a video recording that contains the data of individual stream types.  Once a bag file is loaded into memory then the retrieved streams can have alignment applied to them in real-time.
Regarding your final question about the workings of rs-callback, I do not personally have sufficient programming knowledge about callbacks to answer that particular question, unfortunately.

Thank you @MartyG-RealSense for the quick reply!
I suspect the dropped frames is due to a bottleneck somewhere, considering I'm running on a nano. Thanks for the info on the alignment, I suspect I'll probably do a post-record process to align them.
I've looked at the queue management documentation a few times. I think i have some confusion on whether the pipeline needs a larger frame queue or my user-defined queues need to be larger. Since the documentation suggests the pipeline has its own queue to sync the frames. (I'm kind of thinking out loud at this point) The callback is counting each frame properly, and it's only when other processes try to extract the frames from the queue that there is a drop. So would increasing the size of the user-defined frames act as a buffer in the midst of the assumed hardware bottleneck? Right now my understanding is that with a queue size of '1', if there is currently a frame in the queue, and a new frame comes along and is 'encueued' the prior frame is dropped if it didnt get 'polled' before the new frame arrived? So increasing the queue size would allow for more than one frame to be sitting in the queue ready to be polled?
Currently, my main purpose for all of this is to post-process some real-time data of the sensor. Since I see that X number of frames comes through the sensor, but only Y of the frames are able to be saved (probably due to hardware), I'm trying to see if theres a way to hold those frames until the 'save' process can be performed such that all X frames eventually get saved.
Cheers

A RealSense team member in https://github.com/IntelRealSense/librealsense/issues/5041#issuecomment-542241323 explains that RS2_OPTION_FRAMES_QUEUE_SIZE limits the total number of frames in circulation.  A frame queue size of '1' provides minimum latency, favouring performance but with an increased risk of frame drops.
You can use an SDK instruction called Keep() to store frames in memory instead of writing them to storage during streaming.  Once the pipeline is closed then you can perform batch processing operations on all of the frames at the same time (such as post-processing and alignment) and then save all the frames to file in a single action.
Because Keep() stores frames in memory, it is only suited to short recording durations though because memory capacity is progressively consumed as time passes.  For a device with modest memory size, 10 seconds would be an appropriate maximum, whilst computers with higher memory capacity could record for 30 seconds.
A couple of C++ examples of scripts that use Keep() are at https://github.com/IntelRealSense/librealsense/issues/1942#issuecomment-400031901 and https://github.com/IntelRealSense/librealsense/issues/2223

Looking at #1942, further comments mention either frame or frameset could be 'kept' using keep(), right? So i could 'keep' a whole frameset and use rs::align to align the depth and color frames? Rather than keeping both depth and color, and then running some script to align the two?
After reviewing the #5041 coment and some others, I think I still have some confusions about the frame size. When you say 'in circulation', is that in ciruclation of the pipeline? As in, if I do a pipe.wait_for_frame with a RS2_OPTION_FRAMES_QUEUE_SIZE of '1', I'm only going to get one frame or frameset? (typically i see framesets, which AFAIK collects all of the frames from each sensor) If the queue_size was, say, '2' would that mean that, if the first frameset was not polled yet, a second frameset would be buffered into the queue rather than dropped? I think my goal was to use the callback to extract frames/framesets from the pipeline (without messing with it's queue frame) and throw them into separate queues that can have a large capacity, and then save frames from those queues as fast as possible. I'm not sure that would really work if the record_to_file happens in the pipeline though.
thanks so much for your time/patience!

I will refer you to a complete C++ Keep() script at https://github.com/IntelRealSense/librealsense/issues/6865#issuecomment-661024268 shared by a RealSense user that incorporates alignment and multi-threading.  This should hopefully be a useful reference for you.
Yes, rs2::align would be used to perform the depth-color alignment on the frames that have been collected using Keep().  The above script link demonstrates that principle.
In regard to the frame queue, by default the queue can have 16 frames of each type in it at any one time, and old frames get pushed out of the queue by newly arriving frames, like a continuously moving conveyor belt where items fall off the end.  This is referred to at https://github.com/IntelRealSense/librealsense/issues/2711#issuecomment-437994827 which advises increasing the frame queue size and calling Keep() for every frame you enqueue if you want to keep more frames in the queue.

Is there a function to view the current volume of a queue? My understanding from the documentation is that the queue.size() returns the capacity of the queue but not necessarily how many frames are currently in the queue.

The SDK has a C++ example tool called rs-data-collect for profiling the streams using callbacks and logging the result to file.
https://github.com/IntelRealSense/librealsense/tree/master/tools/data-collect
The tool uses the SDK's Low-Level API which directly accesses the camera hardware in order to minimize software-imposed latencies.  RealSense applications typically run in the High-Level API.
https://dev.intelrealsense.com/docs/api-architecture#section-low-level-device-api

Thanks so much @MartyG-RealSense for your guidance! You've helped explain a lot and point me to some additional resources. I'll close the issue for now as I continue to work through and learn!

You are very welcome, @Cpeck71 - thanks very much for the update, and good luck with your work!


Address Issue #37

Three very small things

Explicitly export check_sys_mapshaper
add !! to fid call in par_split_cat
Change default CRS from 5070 to "EPSG:5070" in case command line interface is used (reference catchment work for example)

An old (don't know why it never sent!) addition is also caught here that stemmed from the reference catchment work. MULTIPOLYGONS are isolated before processing to speed up geometry workflows.

I need to look into these failing tests  -- just updated my environment and found some things failing.


map-based example app

Create a simple HTML/CSS/JS map-based example web app. This app will provide a simple map-based web-ui (for example using leaflet.js & openstreetmap )to allow users to explore BBK entity data in the context of a data-sharing policy by providing an associated key.

Done and deployed at https://demo.bit-broker.io/apps/map/


[TC-EWATERHTR-2.1] Python Script has to be aligned with Test plan for Test step 5, 6 and 7

Description:
TC-EWATERHTR-2.1
In TC-EWATERHTR-2.1, Test Steps 5, 6, and 7 utilize the PIXIT.EWATERHTR.TP and PIXIT.EWATERHTR.EM values in the Python script. However, the test plan does not reference these PIXIT values. Therefore, the script should be updated to align with the test plan.
Test step that Require Review:

Test Steps 5, 6, and 7

Python Script References:

https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/TC_EWATERHTR_2_1.py

Test plan References:

https://github.com/CHIP-Specifications/chip-test-plans/blob/master/src/cluster/WaterHeaterManagement.adoc

GitHub hash of the SDK that was being used
708558881bf6e5b0dac3c09489124907927b6cd2

I think the debug message should be changed to PICS instead of PIXIT and that should resolve this?


fix: use calculated fee for wallet connect transaction as well

This is fundamently not incorrect the issue is that the target is the ckETH contract which requires higher fee than 21_000 and currently the Oisy Wallet do not estimate the fee of the targeted contract.
Contract: https://etherscan.io/address/0x7574eb42ca208a4f6960eccafdf186d627dcc175#code


Add OSX 10.11 deployment target in podspec file.

This PR adds the OSX deployment target in the podspec file, to use CircuitBreaker with CocoaPods dependency management on MacOS. This is required if the Swift Package Manager is not used.

The failure of the bot doesn't seem to be related to the code I've pushed. Does anyone has an insight on how I can fix it? Thanks.


Blindly create namespace and watch, don't assume get privileges

https://github.com/rancher/rancher/issues/29282

related to: https://github.com/rancher/rancher/issues/29282


Select Physics Body Node Instead of Mesh Node

Operating system or device - Godot version:
Any OS, any Godot version
Issue description (what happened, and what was expected):
When selecting objects in the editor window to reposition, it selects the mesh node instead of the physics body it's attached to. What you get is the mesh moving and the collision shape node and (I think) physics body (rigidbody, static, etc) node staying behind. The only way to work around this is to select the node in the node list.
Steps to reproduce:
Link to minimal example project (optional but very welcome):

That's when you would normally use subscenes.

Ah, I see! I think I figured it out last night but then forgot. Thanks.

Not being able to select instances from 3d viewport is a whole other story, fix is coming though. As a quick workaround you can use this for now https://github.com/TheoXD/godot/commit/6bd73baf1c4397e6bf0e9e123feecf9c807bb41f  if you know how to build from source.

Thanks!

As there was no answer to the above request, we're closing this issue as inactive.
If you can still reproduce it, please open a new issue against a currently supported release of Godot.


reapp run: Uncaught TypeError: Cannot read property 'self' of undefined

I recently start with reapp. I'm using Linux so first install sudo npm install -g reapp later do
$ reapp create my-app
$ cd my-app
$ reapp run

open my application on http:localhost:3010 but I try to change for example the title on Home.jsx and i get the following error

Same error to me in my Yosemite.

Tested out install today and fixed up a few things. Let me know if this is still broken.

@natew same error using 0.8.23

Did you update your reapp-kit/reapp-ui versions?

Yes I ran sudo npm install -g reapp and later in the project npm update
<EMAIL_ADDRESS>node_modules/reapp-kit
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>└──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

This may have to do with: facebook/react#3404 (fixed in facebook/react#3494). This should get into 0.13.2: facebook/react#3629.
For the record, I just tried making a new app and running it and it doesn't give me these problems, hot reloading seems to work just fine. Try updating your React to master perhaps as a temporary fix, I will see if I can patch the error as well internally so it doesn't break.

@natew I just create a new app and have the same error :(. I try to install the master version of react using this dependencies

but not works!

Don't have much time today but tried something, let me know if that works. If not can you show me the Styled file where it errors?

Here https://github.com/reapp/reapp-ui/blob/master/src/mixins/Styled.js#L49

This should be fixed if you update to latest reapp and react 0.13.2. Let me know if that's fixed.

I'll try tomorrow, thanks for the info

Closing unless still an issue with new React.


Training Swin Transformer?

I see that there are changes in comparison with the official implementation of Swin Transformer. Also is the Swin transformer getting trained while training Inspyrenet?

Hi, we follow the official implementation of Swin Transformer from their Semantic Segmentation version. Although we did not mention in the paper, there are some minor changes to provide intermediate feature maps.
We train both backbone while training our model, so if you use our method with swin transformer (InSPyReNet_SwinB), then every parameter in the network is participating to the training.
Thank you and feel free to ask more question 😄

Thank you.


Feat/add technologies

Add processing functions for Xenium and MERSCOPE (already included in the notebooks) to the pipeline scripts.
Add the optional prefix parameter to the data parameters file, so output filenames are prefixed {project}-{dataset}-{prefix}-file.ext.
Add data type raw_image_data to generate/pre-process raw images.
Needed for MERSCOPE output raw images are they are split by channel to different tif images. merscope_raw function in process_merscope.py then handles concatenating the channels to a single tif image that can then be processed by Process_images.
Add support for multiple image output from raw_image_data and label_image_data.
If data produces multiple images, processing scripts should append an additional prefix so images are not overwritten.
Process that converts images to zarr directly uses images filenames if they come from raw_image_data or label_image_data as to not loose any prefixes.

Codecov Report
Base: 49.65% // Head: 44.94% // Decreases project coverage by -4.72% :warning:

Coverage data is based on head (48c3a59) compared to base (1764504).
Patch coverage: 40.93% of modified lines in pull request are covered.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##              dev      #61      +/-   ##
==========================================
- Coverage   49.65%   44.94%   -4.72%
==========================================
  Files          10       12       +2
  Lines         439      574     +135
==========================================
+ Hits          218      258      +40
- Misses        221      316      +95

Impacted Files
Coverage Δ

bin/build_config.py
0.00% <ø> (ø)

bin/process_merscope.py
23.68% <23.68%> (ø)

bin/process_xenium.py
34.78% <34.78%> (ø)

bin/consolidate_md.py
92.85% <50.00%> (ø)

bin/generate_image.py
58.33% <58.33%> (ø)

bin/router.py
83.33% <62.50%> (-5.56%)
:arrow_down:

bin/process_spaceranger.py
58.62% <67.56%> (+25.28%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Notes on docker image:

previous image based on bioformats2raw:0.4.0 is based on ubuntu 18 which caused issues when using pyvips to save a tif image with page-height (used when processing MERSCOPE raw image data) as it required libvips > 8.10 which has no builds for that ubuntu version
changed image to be based on python:3.9 (debian 11). Building image requires platform to be amd64 for java libraries for bioformats2raw to work


[WIP] Embed Sinatra

This branch is an ongoing effort to get the following code to parse and return a Rack app:
require 'sinatra/base'

class MyApp < Sinatra::Base
  set :sessions, true
  set :foo, 'bar'

  get '/' do
    'Hello world!'
  end
end

This branch is a top-down approach to implementing core and the standard library in contrast to the bottom up approach I've been taking with Regexp.
The supplied main function in the ditty crate will return MrbError until the code parses and compiles. Address errors as they come up.
Some things I've had to do so far:

Import Ruby sources for standard library components from Ruby 2.6.3.
Modify Ruby standard library components for the mruby VM (e.g. Thread, Monitor, Mutex).
Patch gem sources to work around mruby limitations.

I anticipate needing to implement more of core and standard library in Rust.

solid

uri depends on String#scan, which depends on Regexp#match.
Once I get a working Regexp matcher, I can implement scan in pure Ruby, using a string pointer. Here is the prototype:
https://github.com/lopopolo/ferrocarril/blob/37034d455d75d39eae4be5ffdd458de98713e42b/mruby-sys/vendor/mruby-c078758/mrblib/string.rb#L108-L116

GH-89 and GH-92 were extracted from this PR and added Thread, Mutex, and Monitor implementations.

As of 6368f9ff3e4ff2beae90b42ef4f204426bb94ec5, Sinatra is able to successfully load the mustermann gem. This requires a stub implementation of StringScanner. Implementing the required methods on StringScanner is tracked in GH-91.

As of this most recent push, the ditty crate successfully initializes a Sinatra::Base Rack application and returns a 404 page.

(Showing the 404 page is a bug, but expected given the amount of stubbing going on.)

leak tests fail locally for me but seem to pass in CI. Will dig into this later.

Got an echo server running that outputs a JSON response containing the Rack request.
http://localhost:8000/boom succeeds
http://localhost:8000/boom?foo=bar blows up

Can't tell if this was GH-124 or GH-125, but the root page now renders and query string parsing works 🎉

It works!


Fatal error: cuda.h: No such file or directory

I am building the project with "make build" command.
The following error occurred.
Should I modify the Makefile or any environment parameter?
[ 20%] Building CXX object lwis/CMakeFiles/lwis.dir/src/lwis.cpp.o
In file included from /home/aluo/inference_results_v0.5/closed/NVIDIA/code/harness/lwis/src/lwis.cpp:17:0:
/home/aluo/inference_results_v0.5/closed/NVIDIA/code/harness/lwis/include/lwis.hpp:31:10: fatal error: cuda.h: No such file or directory
#include <cuda.h>
^~~~~~~~
compilation terminated.
lwis/CMakeFiles/lwis.dir/build.make:62: recipe for target 'lwis/CMakeFiles/lwis.dir/src/lwis.cpp.o' failed
make[4]: *** [lwis/CMakeFiles/lwis.dir/src/lwis.cpp.o] Error 1
make[4]: Leaving directory '/home/aluo/inference_results_v0.5/closed/NVIDIA/build/harness'
CMakeFiles/Makefile2:122: recipe for target 'lwis/CMakeFiles/lwis.dir/all' failed
make[3]: *** [lwis/CMakeFiles/lwis.dir/all] Error 2
make[3]: Leaving directory '/home/aluo/inference_results_v0.5/closed/NVIDIA/build/harness'
Makefile:83: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/home/aluo/inference_results_v0.5/closed/NVIDIA/build/harness'
Makefile:280: recipe for target 'build_harness' failed
make[1]: *** [build_harness] Error 2
make[1]: Leaving directory '/home/aluo/inference_results_v0.5/closed/NVIDIA'
Makefile:228: recipe for target 'build' failed
make: *** [build] Error 2

@andyluo7 Did you run make build inside the container? If so, could you try nvidia-smi and  ls /usr/local/cuda/include | grep cuda.h to see if nvidia-docker is running correctly? Thanks

@nvpohanh , no, i did not run it in the container. I try to run the inference on AGX Xavier.
Should I run it in container?

Ah, sorry, didn't know that this was on Xavier. No, you don't need docker on Xavier.
Which JetPack did you install? We were using JetPack 4.3 DP.

@nvpohanh, I used the latest Jetpack 4.4 DP. Is there any difference?
There are multiple cuda.h files in the system.
I wonder if I should change the Makefile to add the include path.

Yes, maybe they changed the path. Could you add that to /usr/local/cuda/include/cuda.h?

I have to hard coded #include </usr/local/cuda/include/cuda.h> in the .h and .c files in /inference_results_v0.5/closed/NVIDIA/code/harness/lwis to make it work.


[Bug] [Python] Task python class description error in Python API

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
Error in description of task python class in Python API
What you expected to happen
just class description error
How to reproduce
just class description error
Anything else
No response
Version
dev
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Looking forward your contribution 👍

@xdu-chenrj May I ask which PR closes this issue? Seems PR not linked.

@xdu-chenrj May I ask which PR closes this issue? Seems PR not linked.

Sorry, I didn't follow the process. I didn't create an issue and chatted privately with the administrator. The problem has been solved

@xdu-chenrj May I ask which PR closes this issue? Seems PR not linked.

Sorry, I didn't follow the process. I didn't create an issue and chatted privately with the administrator. The problem has been solved

I'm preparing for the 3.0.2 release, and this issue is tagged with 3.0.2 milestone, which means I need to cherry-pick the related PR. Could u plz link the PR to this issue, or just simply give me the PR number?


test [NO MERGE] [DO NOT RESTART THE FAILED TEST]

problem understood

problem understood

What's the root cause?

@joshfree BufferReader code is passing large struct by value:

Good: This turned out to be a good stress test for the runtime that uncovered a bug: https://github.com/dotnet/coreclr/issues/16044 has more details and highly reliable repro.
Bad: This is very inefficient. https://github.com/dotnet/corefxlab/pull/2097 has discussion about fixing it. It does not seem straightforward.


Fix decorator emit regression for static fields

This fixes an over-aggressive transformation for static fields when a class is decorated.
Fixes #44931

thanks！


Line is not drawing during zoom in sometimes.

Hello Harley,
Please review attached video, I am facing problem when zoomed and drawing very sort line(draw line in " v" shape), then some times got only tap gesture instead of pan gesture.
It's working normal zoom mode, getting issues when zoom in only
DrawingDemo Issues video.mov.zip
.
Please help me.
Thanks and Regards
Rahul

zoom and scale have been reconstructed in the Metal Version, this issue should have been resolved

@rahulglemaing try the new Metal Version


PlatformView: Add Offset method

Added Offset function to PlatformVIew to support [webview_flutter_tizen] Fix youtube player on TV_PROFILE .

Is there any way to test it? "offset" is still not called. (I tested with webview_flutter example)

[webview_flutter_tizen] Fix youtube player on TV_PROFILE
Copy flutter_platform_view.h to /flutter-tizen/flutter/bin/cache/artifacts/engine/tizen-common/public and apply PR on the link above, and you can test it.
@JSUYA


Quantile or Jenks classification

Good morning,
First thanks for your library which produces amazing maps.
I have an issue with some recent data for which the current classification does not render it properly in one of my chloropeth map (few records with extreme values).
I think it is related to the fact that the current classification I am using with DVF is of equal interval or defined interval.
Would it be possible to choose a different classification such as Jenks or quantile to properly render my data and maximize the variance between my classes?
Thanks in advance for your answer
Sylvain

That sounds like a great idea.  I'll give it some thought.  If you have any suggestions on how to best implement this, let me know.  I've been checking out TurfJS recently, which seems like a good fit here possibly.

Thanks Scott,
I have used this piece of javascript code: https://gist.github.com/tmcw/4977508 from the creator of turf to create jenks classes. It works great.
I have now the boundaries of my classes,
For examples, i have 3 classes with the following breaks :[1712, 2021, 8855, 20421]
How can i inject these values into the display options of my map to have relevant color palette and legend?
Thanks for your answer.
Sylvain

Sylvain,
Currently, you can use an L.PiecewiseFunction instance to specify the color function.  To use L.PiecewiseFunction, you pass in an array of LinearFunction instances (e.g. color functions), like so:
var colorFunction = new L.PiecewiseFunction([
   new L.HSLHueFunction([1712,  0], [2021, 30]),
   new L.HSLHueFunction([2021, 30], [8855, 60]),
   new L.HSLHueFunction([8855, 60], [20421, 90])
]);

This approach gives you complete control over color output, but can be a little involved for this general use case.  I need to make some tweaks to the library to make this easier.  I'm thinking of at least adding a class that takes an array of breaks and an array of colors and does this for you behind the scenes.
In terms of the legend, I also need to make some updates.  Currently, the legend will display some fixed number of boxes (controllable via the legendOptions.numSegments option), where each box has a fixed width (legendOptions.width / legendOptions.numSegments).  The default width is 100px and the default numSegments is 10, so each box is 10px wide.  The legend will work for this use case, but it's not optimal.  Based on other legends I've seen, my feeling is that it needs to be updated to only a display a box for each class.  For now, you can get closer to this effect by removing the border-style: solid CSS style from each legend element - this should at least give the legend more of a gradient appearance where it isn't so obvious that there's not a box for each class.
I'll let you know when I get in enhancements to address this, but I'm hoping in the meantime the current approach will work for your use case.  Let me know what you think and if I can help further.

Thanks a lot Scott,
This is what I was thinking, i tried to create a "L.CustomJenksColorFunction" which was an extension of PiecewiseFunction and which was taking breaks and color as parameters but with no success. I will be using your code in the meantime and then switch to the next version when ready.
Thanks again for all
Sylvain

Sylvain,
I just pushed a branch called classification that has a new function called L.ColorClassFunction (I'm open to changing the name).  You should be able to pass this an array of breaks along with an array of colors (per your example).  Let me know if this works for you, and I'll merge it into master.  Here's the code for the new class:
L.ColorClassFunction = L.PiecewiseFunction.extend({
	options: {
		interpolate: true
	},

	initialize: function (classBreaks, colors, options) {
		var functions = [];
		var colorFunction;

		L.Util.setOptions(this, options);

		for (var i = 0; i < classBreaks.length - 1; ++i) {
			var start = classBreaks[i],
				end = classBreaks[i + 1],
				startColor = colors[i],
				endColor = this.options.interpolate ? colors[Math.min(colors.length -1, i + 1)] : colors[i];

			colorFunction = new L.RGBColorBlendFunction(start, end, startColor, endColor);

			functions.push(colorFunction);
		}

		L.PiecewiseFunction.prototype.initialize.call(this, functions);
	}
});

Scott

Thank you so much Scott,
I just tried it and it looks promising.
'''
var breaks = getJenks(dataJSON.grid[0], 'A', 4);
        var fillColor = new L.ColorClassFunction(breaks, L.ColorBrewer.Diverging[selectedPalette][4].slice(0).reverse(), {
            interpolate: false
        });

'''
I just see an issue of color between the legend and what is shown on the map:
Please see the difference below between normal L.CustomColorFunction (equal intervals) and the new L.ColorClassFunction (Jenks)

In my case, i am dealing with 4 classes, the Jenks classes are [166, 789, 1161, 1685, 2540] but when I am mousing over the legend to display boundaries of each color class, it shows values corresponding to equal value classification ([166, 759.5, 1353, 1946.5, 2540] with an interval of 593.5).
Moreover, the green color shown in the map is not displayed in the legend and the red color is over 2 classes which is a bit confusing.
Hope it make sense and thanks again
Sylvain

That makes sense, and I definitely see the issue.  I'll make some updates to the legend building code that will hopefully solve this issue.

I think I have a fix for the legend, but I just need to do some more testing/code cleanup before I push it.  I'll push it to the classification branch at some point today once I get some time to work on it.

Went ahead and pushed it to the classification branch.  I didn't do a ton of testing, so let me know if you encounter any issues.  In order for the legend to work with breaks, you'll need to supply the breaks as part of the legendOptions you pass in to the DataLayer like so:
var dataLayer = new L.ChoroplethDataLayer(data, {
    ...
    legendOptions: {
        breaks: <array of break values>
    }
});

Just realized that this solution won't work for all cases but should work for your use case.  Need to make some more tweaks before it's ready for release.

Ok one more update, now pass breaks with your displayOptions field like so:
var dataLayer = new L.ChoroplethDataLayer(data, {
    ...
    displayOptions: {
        'field1': {
            ...
            breaks: <array of break values>
        }
    }
});

This is great Scott.
This is my current code using your last update:
var breaks = getJenks(dataJSON.grid[0], MyField, parseInt(MyClass, 10));

            var fillColor = new L.ColorClassFunction(breaks, L.ColorBrewer.Diverging[MyPalette][MyClass].slice(0).reverse(), {
                interpolate: false
            });

            var color = new L.ColorClassFunction(breaks, L.ColorBrewer.Diverging[MyPalette][MyClass].slice(0).reverse(), {
                postProcess: function (y) {
                    var newColor = new L.RGBColor(y);
                    newColor.l(0.2);
                    return newColor.toRGBString();
                }
            });

            displayOptions[MyField] = {
                displayName: $('#field-select option:selected').text(),
                fillColor: fillColor,
                color: color,
                breaks: breaks
            }

MyLeafletLayer.setDisplayOptions(displayOptions);

I think we are almost there, the legend is correctly generated in term of boundaries values and class width. I still have this issue of color being duplicated. As you can see below, the green color has disappeared from the legend and the red has been duplicated which makes the legend wrong as the green color still exists in the map.
I tried to explore the _getLegendElement element  in your code where the color is allocated to each legend class with no success

Thanks for everything
Sylvain

Thanks Sylvain.  Looking into this now.  What are the break values for your example above?

Sure. My break values with 5 classes are [166, 729, 929, 1241, 1713, 2540]
Thanks again
S.

Sylvain, give the latest a try when you get a chance.  I believe it should work, but appreciate your testing it.  Thanks for your patience.  I definitely need to spend some time rethinking/refactoring the legend code.

Thanks Scott,
Everything works perfectly for my break classification and the legend display now.
The last issue to be corrected, which is still there in the master version, is the fact that the chloropeth map colors do not match the default legend.
Let me explained: In the case of a default legend (equal intervals) with 5 classes for example (boundaries: 166, 640, 1115, 1590, 2065, 2540 with colors: blue, green, yellow, orange, red), a square with the value 2300 will be shown in orange while it should be red.
Please see below an example for which the squares of the grid appears in yellow while they should be in orange since their values is between 1590.4 and 2065.2:

When the classification chosen is Jenks with breaks, the legend and colored grid matched perfectly.
Hope it makes sense and thanks again for the push concerning the breaks classification.
Sylvain

Almost there.  Thanks for sticking with this.  I'll look into it and hopefully have a fix in soon.

Does the latest commit fix this?  I noticed/fixed an issue with the legend code.  If not, does it work when you pass in the equal interval breaks yourself (you might already be doing this?):
var breaks = [166, 640, 1115, 1590, 2065, 2540];
var colors = L.ColorBrewer.Diverging['Spectral'][5].slice(0).reverse();
var colorFunction = new L.ColorClassFunction(breaks, colors);
var layer = new L.DataLayer(data, {
     ...
     displayOptions: {
         <field>: {
              fillColor: colorFunction,
              breaks: breaks
         }
     }
});

Thanks Scott,
The breaks classification works fine since your previous release and yes, I am already passing the interval breaks myself as you code is showing it.
It is only the default classification which is not working properly. Now the colors are spread over the legend with 2 classes with the same color and the red color missing (see below). It also seems that there is an inconsistency between the color given in the map and the range values indicated in the legend.

Sorry about that
Sylvain

Darn.  Would you be willing to share your code (or an equivalent example when classification is set to equal intervals) and some example data?  I understand if you don't want to.  Just want to try and recreate this issue and make sure my test case matches your case.   I've been testing updates against the earthquake example (updated to use the ColorClassFunction and breaks) and it seems to be working fine in the default and jenks case.

I'll add a new example to the classification branch that matches what I think your use case is, and maybe you can let me know if it's correct or not.  Are you passing in legendOptions.numSegments?  I see an issue when I pass in legendOptions.numSegments and don't include the breaks under a field in displayOptions.  The legend is definitely off in that case.

Thnaks Scott for all of this,
i am effectively using the numSegments to define th number of classes.
Please see below part of my code:
This part is generating the default map based on equal intervals classification
    selectedClass = 9;

    selectedBiomass = 'a';
    selectedPalette = 'Spectral';
    selectedYear = 0;
    selectedType = 'E';
    var range = $.map(dataJSON.grid[0]["_" + selectedBiomass].split('-'), Number);

    //Diverging
	var fillColor = new L.CustomColorFunction(range[0], range[1], L.ColorBrewer.Diverging[selectedPalette][selectedClass].slice(0).reverse(), {
	    interpolate: false
	});

    //Diverging
	var color = new L.CustomColorFunction(range[0], range[1], L.ColorBrewer.Diverging[selectedPalette][selectedClass].slice(0).reverse(), {
	    postProcess: function (y) {
	        var newColor = new L.RGBColor(y);
	        newColor.l(0.2);
	        return newColor.toRGBString();
	    }
	});

	var options = {
	    locationMode: L.LocationModes.LOOKUP,
	    recordsField: null,
	    codeField: 'id',		//field in the data Json identifying uniquely a feature to be linked with an item in the GeoJSON
	    locationLookup: gridJSON, // A GeoJSON FeatureCollection that will be used to lookup boundaries/location
	    locationTextField: 'xy', //field in the data Json to be displayed in the mouse hover popup
	    locationIndexField: 'ID', //Property in GeoJSON to identify each feature
	    includeBoundary: true, // Whether or not to include a background boundary so people know what boundary each marker is associated with
	    layerOptions: {
	        fillOpacity: 0.9,
	        opacity: 0.5,
	        weight: 0.5
	    },
	    tooltipOptions: {
	        iconSize: new L.Point(130, 60),
	        iconAnchor: new L.Point(-5, 60)
	    },
	    legendOptions: {},
	    displayOptions: {}
	};

	options.displayOptions[selectedBiomass] = {
	    displayName: $('#biomass-select option:selected').text(),
	    fillColor: fillColor,
	    color: color
	};

	options.legendOptions[selectedBiomass] = {
	    numSegments: selectedClass,
	    gradient: false
	};

	biomassLayer = new L.ChoroplethDataLayer(dataJSON.grid[0].items, options);

	layerControl.addOverlay(biomassLayer, 'Biomass');

	map.addLayer(biomassLayer);

Then the user has the possibility to change the palette, the classification type (Jenks or equals interval, number of classes), see below the generic function which handle that with the function when number of classes are changed :
'''
$classSelect.on('change', function (e) {
    selectedClass = $classSelect.val();
    var displayOptions = updateColor();

    //https://github.com/humangeo/leaflet-dvf/issues/26
    biomassLayer.options.legendOptions.numSegments = selectedClass;
    legendControl.redrawLayer(biomassLayer);

    biomassLayer.setDisplayOptions(displayOptions);

});

function updateColor() {
var displayOptions = {};
    if (selectedType == 'E') {
        //--------------------------------------------   Equal intervals
        var range = $.map(dataJSON.grid[selectedYear]["_" + selectedBiomass].split('-'), Number);

        var fillColor = new L.CustomColorFunction(range[0], range[1], L.ColorBrewer.Diverging[selectedPalette][selectedClass].slice(0).reverse(), {
            interpolate: false
        });

        var color = new L.CustomColorFunction(range[0], range[1], L.ColorBrewer.Diverging[selectedPalette][selectedClass].slice(0).reverse(), {
            postProcess: function (y) {
                var newColor = new L.RGBColor(y);
                newColor.l(0.2);
                return newColor.toRGBString();
            }
        });

        displayOptions[selectedBiomass] = {
            displayName: $('#biomass-select option:selected').text(),
            fillColor: fillColor,
            color: color
        }

    } else {
        //--------------------------------------------   Natural breaks (Jenks)
        var breaks = getJenks(dataJSON.grid[selectedYear], selectedBiomass, parseInt(selectedClass, 10));
        alert(JSON.stringify(breaks));

        var fillColor = new L.ColorClassFunction(breaks, L.ColorBrewer.Diverging[selectedPalette][selectedClass].slice(0).reverse(), {
            interpolate: false
        });

        var color = new L.ColorClassFunction(breaks, L.ColorBrewer.Diverging[selectedPalette][selectedClass].slice(0).reverse(), {
            postProcess: function (y) {
                var newColor = new L.RGBColor(y);
                newColor.l(0.2);
                return newColor.toRGBString();
            }
        });

        displayOptions[selectedBiomass] = {
            displayName: $('#biomass-select option:selected').text(),
            fillColor: fillColor,
            color: color,
            breaks: breaks
        }
    }
    return displayOptions;
}

'''
Again everything seems fine for the breaks classification, it is really the default classsifcation which is giving me a hard time.
Hope it is detailed enough. If you give me your contact details, i could even give you  the credentials to access my application and see the issue in live.
Thanks again
Sylvain

Thanks for sharing.  I think I understand what the problem is.  There's an issue with the CustomColorFunction.  Should have a fix in soon.  Hopefully this actually fixes things.

Scott,
I just tried the last version of the classification branch you pushed lately and it works beautifully.
Thank you very much.
I have just added a rounding to the min and max values of the legend boundaries in the _getLegendElement function as I had numbers with extreme decimal<PHONE_NUMBER>00003) as shown in one of my previous post.
It is perfect now.
Thanks again for your precious help and your reactivity
Sylvain


Consolidate js-parity issues list

https://github.com/maplibre/maplibre-native/labels/js-parity

The tag


No boot on raspberry pi 2

To enable us to help you with the issue, please provide as much information as possible about your device and the problem. At a minimum, provide:
Step 1: Are you in the right place?
This repository is for Farmbot OS problems.
Step 2: Describe your environment

Device: raspberry pi2
Farmbot OS version: farmbot-rpi2-3.0.8.img
Arduino Firmware version: not sure

Step 3: Describe the problem:
Steps to reproduce:

its just doesnt bootup the raspberry pi

Observed Results:

What happened?  This could be a description, log output, etc.

Expected Results:

What did you expect to happen?

I believe you are the first one to test Raspberry pi 2, so thank you!
Just a note but you will NOT see a wireless ssid on any other device except raspberry pi 3 making it difficult to configure. Expected behavior should be that your raspberry pi 2 is to be plugged in via Ethernet and it will automatically get an ip address, you can then configure your bot by navigating to its ip address in a web browser.
so to troubleshoot:

What image did you flash onto your device?
What do the LED's do when you power the board up?
You have dhcp network access right?

Just one last note:
Support for devices other than Raspberry Pi 3 is minimal, we strongly suggest picking up a Pi3 as it is much easier to configure, and runs 4 times faster.

[image: Inline image 1]
What do the LED's do when you power the board up?
For LED it just flash and turn off the whole system
You have dhcp network access right?
and yes i do have dhcp network
On Tue, Apr 11, 2017 at 11:38 AM, Connor Rigby<EMAIL_ADDRESS>wrote:

I believe you are the first one to test Raspberry pi 2, so thank you!
Just a note but you will NOT see a wireless ssid on any other device
except raspberry pi 3 making it difficult to configure. Expected behavior
should be that your raspberry pi 2 is to be plugged in via Ethernet and it
will automatically get an ip address, you can then configure your bot by
navigating to its ip address in a web browser.
so to troubleshoot:

What image did you flash onto your device?
What do the LED's do when you power the board up?
You have dhcp network access right?

Just one last note:
Support for devices other than Raspberry Pi 3 is minimal, we strongly
suggest picking up a Pi3 as it is much easier to configure, and runs 4
times faster.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/FarmBot/farmbot_os/issues/287#issuecomment-293303664,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVCYynh89vyzDyLfZrSu4Uoq1iQuAZAvks5ru56JgaJpZM4M5ezl
.

Because we are focusing on developing new features right now, we are temporarily not supporting other platforms such as rpi2, 1, or 0. We anticipate bringing support back in the future and welcome pull requests, but in the meantime we encourage people to get a pi3.


Fishbowl should correctly set copyright dates

Currently the copyright year is set to 2022 in the templates. It should instead use the actual date to create correct copyright dates.

It also makes code which causes this PEP8 change to be required.


function should be in merkle proof and imported for cleaner code

https://github.com/123R3N321/hmk1/blob/ff9c39514de4b8fa7193c1b0c228a5ab5c78c8ca/main.py#L185

done.


[pr] add ability to open Pull Request in the browser

add the ability to open a PR page in browser (also copy & URL print to screen).
ability to open the forked PR by pass the head flag.

in the example above hub is a fork of github/hub.

@mislav in the above scenario, if origin has PR with feature has the head, we query should origin and as a "fallback" check upsteam for a PR with owner:feature as head, do you agree?

@mislav is this cover the scenario you wrote above?

I'm not so happy with the amount of if \ else in the function code, but I can't think how to support all the possible combinations, so maybe it is better not to support the head flag, what do you think?

@ashemesh I've cleaned up code & tests, added more tests, and added the hub pr show <NUMBER> functionality as per @whatsadebugger. Please review!

@mislav sorry only saw it now! thank you!

@ashemesh No worries; thank you for your contribution! 🎉


refactor(package.json & contributing.md): add pnpm link.dist script

What is it?

[x] Feature / enhancement
[ ] Bug
[x] Docs / tests

Description
First PR :nerd_face:
Checklist:

[x] My code follows the developer guidelines of this project
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[ ] Added new tests to cover the fix / functionality

I have read the CLA Document and I hereby sign the CLA

recheck

Looks good to me! Merging.
The CI error seems to be a problem with GitHub and the "danger bot" for first time contributors.

@all-contributors please add @maiieul for code, ideas, maintenance


Does not support TypeScript style JSDoc, which were added in TS2.3

Example here https://github.com/GoogleChrome/puppeteer/pull/1170

corresponding issue in TS https://github.com/Microsoft/TypeScript/issues/7546


caddy: 2.4.5 -> 2.4.6

Motivation for this change
upstream update https://github.com/caddyserver/caddy/releases/tag/v2.4.6
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[X] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 145120 run on x86_64-linux 1

2 packages built:

caddy
styx


Customer-reported issue with NNBD migration tool: couldn't load dart file /E:/Jaybhai/Testing%20App/Adanisaksham-master/lib/Screens/AboutUs/about_us.dart

couldn't load dart file /E:/Jaybhai/Testing%20App/Adanisaksham-master/lib/Screens/AboutUs/about_us.dart
Error: Error reaching migration preview server
Please fill in the following:
Name of package being migrated (if public):
What I was doing when this issue occurred:
Is it possible to work around this issue:
Has this issue happened before, and if so, how often:
Dart SDK version: 2.17.1
Additional details:
Thanks for filing!
Stacktrace: auto populated by migration preview tool.
Error reaching migration preview server:
This usually happens because the migration preview server has exited.  For
example it may have been aborted with Ctrl-C, or you may have completed this
migration, or an exception may have occurred.  Please check the console where
you invoked `dart migrate` to verify that the preview server is still running.

    at Object.b (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:294:3)
    at http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:2945:32
    at Gs.a (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:1401:63)
    at Gs.$2 (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:4677:14)
    at SX.$2 (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:4674:16)
    at Ji.rp (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:4943:25)
    at Fe.Kw (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:4734:17)
    at RW.$0 (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:4878:36)
    at Object.HZ (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:1449:76)
    at vs.ZL (http://<IP_ADDRESS>:5391/E:/Jaybhai/Testing%20App/Adanisaksham-master?authToken=pTcKexV4KbY%3D:4796:3)

As of 1c7fe7196dd7a542f7f91cd6ea2ac81e1a13146c, the null safety migration tool has been removed from active development and retired. No further work on the tool is planned.
If you still need help, or you believe this issue has been closed in error, please feel free to reopen.


Cannot chaining panel 128x32

Hi mrfaptastic,
I used smartmatrix pinout and the display runs good using single panel.
However, the displays is always cloned when I used :
 #define MATRIX_WIDTH 128   #define MATRIX_HEIGHT 32
I  run  example sketches and no modification other than MATRIX WIDTH 128.
Did I just go wrong?

Hello. Impossible for me to say, the library works with chaining, so double check the wiring and that it's actually compiling with the amended library header .h file.

Thank you.
So I also have to change MATRIX WIDTH value that lies in ESP32-RGB-64-32-MATRIX-PANEL-I2S-DMA.h and 2 chaining panels work well.
This is not an issue at all.
But can we just add #define MATRIX_WIDTH in sketch file and leave the .h library file untouched?

@arinurcahyo you can't use #defines in sketch to control library options since it a compile-time flag.
Although if you use platformio instead of Arduino IDE you can define "build_flags =  -DMATRIX_WIDTH=128"

Thank you all.
This is due to my lack of knowledge.


WIP: Guide for getting started with supercomputing

Proposed changes
The purpose of this feature branch is to write a guide for getting started with supercomputing. It should successfully outline a user path that describes which tools users should expect to utilize at different points of their development as HPC practitioners.
Preview pages:

Getting started with supercomputing at CSC
Available batch job partitions
What's new

Checklist before requesting a review

[ ] I have followed the instructions in the Contributing and Styleguide documents.
[ ] My pull request passes the automated tests. If not, visit pull requests at Travis CI and have a look at the job log.
[ ] I have included a link to the appropriate preview page (select your branch from the list).

Great work Mitja! I think this will help a lot of first time users, if they find the page :)
Added a few comments, please feel free to disregard/adapt if you think it does not fit.
For visibility, this page should then probably be linked from docs starting page?

Should the Puhti/Mahti getting started pages be removed in this feature branch? Or should I link to them from the new page?

Should the Puhti/Mahti getting started pages be removed in this feature branch? Or should I link to them from the new page?

I think they could be removed. However, we should ensure that no important links are lost, i.e. this page should provide similar pointers where to find more information. I think some links can be replace by others, though, e.g. no need to link to compiling pages, probably installing overview is enough.

Puhti and Mahti quick start pages are now removed from this branch and links to them have been replaced with links to the HPC quick start.

I now read the page first time rendered, which revealed a few different things. Putting them in separate comments for easier discussion if necessary:

(This may be a bigger topic to discuss and nothing for here, but we could at least address the current state) This page appears when users click on computing in docs navigation, which then includes mostly HPC, but also quantum stuff but not cloud computing. In the long run, maybe quantum could get its own navigation page, and this section could be called "supercomputing", or so, for now it may be enough to start this page with a few words on ALL CSC computing services and links to their index page and what to expect from the rest of this page (focus on supercomputing) ?

Scalability section seems a bit out of place under "connecting"; agree that it should come early though so suggestion to move it with the first paragraphs on the page and add a few leading words on resource request and speedup?

I wonder if rather than as a tutorial, the hpc_quick page could become the new landing page for "computing" ?
A few words could be added in the beginning that the remainder of the page is related to HPC and links to quantum and cloud index pages could be linked.

I'll add a few comments still asap, sorry for the delay! Putting as WIP meanwhile to avoid premature merging.


italian translation final

after tried the nigthly build, and checked all the menu items, some minor changes are required to fit display available space and better translation terms understanding.

There are some mistakes in these changes, I'll mark them later when I'm at home.

I made them using the git web editor.
please let me know what's wrong in order to avoid them in the future

please let me know if I can proceed with the suggested modifications (a new pull request will appear) or if you could change the two sentences directly by yourself


Error Out of Bounds

I get this from time to time running my tests on saucelabs, mostly IE  9 - Edge
Output of gemini --version:
2.1.1
...
Contents of .gemini.yml file:
rootUrl: http://beerfridayautotest.cms.dealer.com
tolerance: 3
screenshotsDir: './screens'
windowSize: 1600x1080
sessionsPerBrowser: 5
suitesPerSession: 10
calibrate: true

browsers:
  IE-10:
    screenshotsDir: './screens/ie10-screens'
    desiredCapabilities:
      version: "10.0"
      browserName: 'internet explorer'
      platform: "Windows 8"
      screenResolution: "1280x1024"
      name: "IE10 BeerFriday"

  IE-9:
    screenshotsDir: './screens/ie-screens'
    desiredCapabilities:
      version: "9.0"
      browserName: 'internet explorer'
      platform: "Windows 7"
      screenResolution: "1280x1024"
      name: "IE9 BeerFriday"

  EdgeWin10:
    screenshotsDir: './screens/edge'
    desiredCapabilities:
      version: "20.10240"
      browserName: 'MicrosoftEdge'
      platform: "Windows 10"
      screenResolution: "1280x1024"
      name: "EdgeWin10 BeerFriday"

system:
  plugins:
    sauce:
      username:
      accessKey:

Test source code:
    gemini.suite('beerfridayautotest Inventory Search', function (el) {
        el.setCaptureElements('div.ddc-container.tabbed-widgets-group.tabbed-widgets-group-1.tabs-slide-direction-left')
        .capture('Inventory Search', function (actions, el){
            actions.waitForElementToShow('div.ddc-container.tabbed-widgets-group.tabbed-widgets-group-1.tabs-slide-direction-left', 5000)
        });
    });

Command used to run the test:
gemini test ./test/ -c sauce.yml --reporter html

Publically accessible URL of the page you are testing against:
https://beerfridayautotest.cms.dealer.com/
Result:
Error: Out of the bounds
at Error (native)
at module.exports.inherit.fill (/Users/van.patton/test-framework-gemini/node_modules/png-img/index.js:66:19)
at module.exports.inherit.fill (/Users/van.patton/test-framework-gemini/node_modules/png-img/index.js:56:25)
at clear (/Users/van.patton/test-framework-gemini/node_modules/gemini/lib/image.js:31:19)
at /Users/van.patton/test-framework-gemini/node_modules/gemini/lib/capture-session.js:74:25
at Array.forEach (native)
at module.exports.inherit._cropImage (/Users/van.patton/test-framework-gemini/node_modules/gemini/lib/capture-session.js:73:33)
at wrapper (/Users/van.patton/test-framework-gemini/node_modules/gemini/node_modules/lodash/index.js:3533:19)
at _fulfilled (/Users/van.patton/test-framework-gemini/node_modules/q/q.js:834:54)
at self.promiseDispatch.done (/Users/van.patton/test-framework-gemini/node_modules/q/q.js:863:30)
...
Expected behaviour:
Hopefully a screenshot
...

@vpatton hi, sorry for delayed response.
From log you provided it seems that you set some ignore selectors in your tests. Are them really set and if so what are they?

gemini.suite('autotest About page', function (suite) {     suite.setUrl('/dealership/about.htm?_disablePixall=true')         .before(function (actions, find) {             actions.wait(3000)         })         .setCaptureElements('div.ddc-content.header-value-statement')         .ignoreElements('.cycle.ddc-list-unstyled')         .capture('header');     gemini.suite('autotest Navigation Menu', function (el) {         el.setCaptureElements('div[data-widget-id="template-navigation1"]')         .capture('Navigation Menu', function (actions, el){             actions.waitForElementToShow('div[data-widget-name="navigation-default"]', 5000)         });     }); });

I've seen the same issue happen, when the ignored element is not (fully) contained within the bounds of the captured element.

so you want to ignore element, but it positioned outside of the page?

Not outside the page, but outside the captured element. Partly outside is enough too.

These three images explain our problem. The black box hides a span containing some text and an animated svg that sticks a couple of pixels above the selection. We would like to hide just the animated svg, but can't due to this issue. Now it also makes the tests to fail sometimes, as the animation is slightly different above the captured area.

@hannesj you can add wrap for your selected block that will be with pudding. So your svg never will be outside of the box

I have the same problem. I ignore elements which are not seen at some dimensions, and so get an error. Why should the test fall in this case? The element I would like to ignore is not seen, so it should be OK.

@varya https://github.com/gemini-testing/gemini/issues/543

The element I would like to ignore is not seen, so it should be OK.

why should you ignore element if it doesn't seen on the page?

I am having the same issue trying to ignore a dynamic element that changes with every page load. The css element is not actually present in the UI but loads outside and I'm getting the Out of the bounds errors trying to ignore that element. Any way around it?

I'm also seeing this, were the ignored element is on the page but not part of the first screenshot of the page

Fixed in 5.4.1
Proof


Redundant context in genesis document

In the genesis document, the @context array includes the https://w3id.org/security/multikey/v1 context which does not seems to be needed since currently only the id field is set.
https://github.com/bcgov/trustdidweb-py/blob/d730195c7c5afd1ac3536e72b69b62f094452390/did_tdw/provision.py#L109-L119

Thanks, this is fixed now.


Update dependencies

Should not be merged until https://github.com/input-output-hk/cardano-ledger-specs/pull/2068 is done.

bors merge

bors merge


Add "estimate time", "last modified" to blog posts

in automated way, ideally

Done in https://github.com/mjyc/mjyc.github.io/commit/3a334ae7548262c21c3b0c9a5dc932105a1bf926


Update TaskStatus picklist

Mantis #0845394

Validated that new picklists items "Open" and "Re-Opened" is added to picklist TaskStatus

Approved


Remove development and test files from the gem package

There are a bunch of files in the gem package that aren't useful for downstream projects. Removing these reduces the gem package size from 12K to 9.5K.
< .github/workflows/ci.yml
< .gitignore
< Appraisals
  CHANGES.md
< Gemfile
  MIT-LICENSE
  README.md
< Rakefile
< gemfiles/rspec_rails_4_0.gemfile
< gemfiles/rspec_rails_5_0.gemfile
< gemfiles/rspec_rails_6_0.gemfile
  lib/generators/rspec/cell_generator.rb
  lib/generators/rspec/templates/cell_spec.erb
  lib/rspec-cells.rb
  lib/rspec/cells.rb
  lib/rspec/cells/caching.rb
  lib/rspec/cells/example_group.rb
  lib/rspec/cells/tasks.rake
  lib/rspec/cells/version.rb
  rspec-cells.gemspec
< spec/cells/caching_spec.rb
< spec/cells/cell_generator_spec.rb
< spec/cells/cell_spec_spec.rb
< spec/spec_helper.rb

@yogeshjain999 Do you mind reviewing this PR? 🙇

Thanks @apotonick!


Out of range ASCII

When an ASCII flag is active and the output is out of the ASCII range (e.g. negative) a nasty ruby error is thrown.  Ideally this will change but the question is how.  We could create a custom Brain-Flak error or we could implement a wraparound (i.e. mod 256 the output before conversion).

What if we extended ASCII mode to unicode characters? That still brings up the point of what to do with negative characters and characters above 65535. It seems like the most sensible thing would be to just take modulo 65536.
I suppose the other option would be to just silently ignore characters outside of ASCII.

I think extending ASCII mode to unicode would be a good idea since Brain-Flak can already handle unicode strings as input and I agree that modulo at the end would be probably most sensible as well.

I believe that we should drop support for unicode as input and have ASCII only for both input and output.

I like the idea of ASCII only being the default mode (error on non-ascii input), but having a flag to enable unicode support. My only remaining question is do we want to use ascii which is 7-bit (128 characters) or do we want to use one of the many 8 bit extensions? The only reason I would consider the 8 bit extensions is because I like 256 more than 128.


cannot import name 'Conformer'

Hi when I tried to import conformer, I got this issue
>>> from conformer import Conformer Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "/workspace/bert/conformer/conformer.py", line 3, in <module>     from conformer import Conformer ImportError: cannot import name 'Conformer' from partially initialized module 'conformer' (most likely due to a circular import) (/workspace/bert/conformer/conformer.py)
I did as the installation instruction. Would you please see where I might be wrong? Thanks.

Maybe your environmental problem.
How about python3 setup.py install?


Cannot read property 'MemberList' of undefined  still exisit in the latest docker version

Run npm run doctor or wechaty run doctor(for docker user), paste output here
the latest docker version:
><EMAIL_ADDRESS>check-node-version /wechaty
> check-node-version --node ">= 7"

node: 8.9.1

Logs
#984 said it has fixed, but the latest one still has the following error
07:35:48 ERR Room contactGetter(@@a69aa054039bf45da752c504983142c01d47e3d8825f30723fd744d2fe9cf0c0) exception: Cannot read property 'MemberList' of undefined

My fault...
I update docker image in a wrong way...
I thought docker pull zixia/wechaty will always pull the latest image version, but actually, it didn't,  it is an old docker latest version, until I run docker pull zixia/wechaty:0.12
Anyway, this is not a good issue = =


gopherjs: update tags to support gopherjs build

Fixes the build with "gopherjs build -v"

cc @marten-seemann
cc @nevkontakte

GopherJS already sets purego tag on all builds by default: https://github.com/gopherjs/gopherjs/blob/0b2280d3ff9688cca64f090ccd7aeeed06ab7a97/build/context.go#L256, so I am surprised this change makes a difference.

@nevkontakte You're right, it seems this is no longer needed with recent GopherJS builds (but it was with older ones).


Creates a score generator file command

Description
Creates a score.yaml file from the CLI
What does this PR

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New chore (expected functionality to be implemented)

Checklist:

[ ] My change requires a change to the documentation.

[ ] I have updated the documentation accordingly.

[ ] I've signed off with an email address that matches the commit author.

Closing this as won't do - this functionality is already present in the Humanitec CLI.


Fix tests autoloading path

The autoloading for test classes did not work correctly on case-sensitive filesystems. Changing the autoloading config for composer to be the same case as the actual path fixes this.
Previously, tests failed with "Class not found" error on case-sensitive file systems:
There was 1 error:

1) Tests\E2E\Services\Teams\FunctionsTest::testCreateProject
Error: Class 'Tests\E2E\Client' not found

Pushed a second commit installing required dependencies to run the tests on travis. Unfortunately, the tests still fail, but that seems to be rather a logic error and is unrelated.

THANK YOU! All changes merged 🥳
Please reach out to me on our Discord server if you would like me to send you some cool Appwrite stickers and magnets! Please note that shipping might take longer times because of COVID19.
As a way of saying thank you, we have just invited you to join the Appwrite organization on GitHub. You can accept the invite by visiting https://github.com/orgs/appwrite/invitation. By joining our team, you will officially be an Appwrite maintainer on GitHub.
You can change your membership visibility settings, so your new Appwrite team membership badge will show up on your personal GitHub profile.


penta elo and penta sprt

https://elocalculator.netlify.app/

Merged as https://github.com/raklaptudirm/arbiter/commit/8ab9d7697ba7c53758ddbb3b7587948c05d51584. Thanks for your contribution!


Local previewing in container is broken

This is a Bug Report

Problem:
You cannot run make container-serve to get a local website preview, nor can you run make container-image to build the container locally.
Proposed Solution:
Fix local previewing by ensuring that make container-image builds a working container image

Additional Information:
This is relevant to contributor experience.

Optionally, we could run CI tests whenever there is a change to hugo.toml, netlify.toml or Makefile, and verify that we can:

build a container image
run Hugo inside that image (smoke test)

I am unable to produce this because being blocked to gcr.io, the command never worked for me (in China). I'm curious though what kind of errors we have encountered and why an upgrade of Go is required.

@tengqm you can read more on this thread https://kubernetes.slack.com/archives/C1J0BPD2M/p1724428225643559?thread_ts=1724226266.705639&cid=C1J0BPD2M
TLDR; the error was:
#10 2.275 /root/src/hugo-0.133.0/go.mod:172: invalid go version '1.22.6': must match format 1.23

Due to the latest Hugo upgrade that requires Golang 1.23 when building the image.
The fix works locally:

@sftim +1 to running a test in the CI.
Would that be something completely new or a pre-existing pipeline to enhance? Asking because I don't see any other GitHub workflow aside from netlify-periodic-build.yml.
Note: I'm still getting accustomed to the codebase so this question might sound silly.

Got it. A change to Go version is necessary.

I am unable to produce this because being blocked to gcr.io, the command never worked for me (in China). I'm curious though what kind of errors we have encountered and why an upgrade of Go is required.

To test the container image build locally, run make container-image. You need access to the Docker registry; you don't need access to push to Google.


[feature]: parse filter as JSON5 in iroha_client_cli

Description
There is inconsistency of what iroha_client_cli accepts:

JSON5 input for iroha_client_cli json command
JSON input for filters, e.g.iroha_client_cli domains list filter '{"Identifiable": {"Is": "wonderland"}}'

This chore PR adds support for JSON5 for filters:
iroha_client_cli domains list filter '{Identifiable: {Is: "wonderland"}}'

Context
It makes examples of iroha_client_cli usage a little more clear and consistent.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
6162 unchanged lines in 104 files lost coverage.
Overall coverage decreased (-1.1%) to 58.339%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

client_cli/src/main.rs
0
1
0.0%

Files with Coverage Reduction
New Missed Lines
%

cli/src/main.rs
1
0.0%

cli/src/torii/pagination.rs
1
98.9%

config/base/src/runtime_upgrades.rs
1
51.72%

config/src/torii.rs
1
96.67%

core/src/smartcontracts/isi/block.rs
1
87.5%

crypto/src/merkle.rs
1
96.23%

data_model/src/domain.rs
1
47.37%

logger/src/lib.rs
1
95.15%

config/src/lib.rs
2
0.0%

logger/src/layer.rs
2
48.05%

Totals

Change from base Build<PHONE_NUMBER>:
-1.1%

Covered Lines:
21449

Relevant Lines:
36766

💛  - Coveralls


PSAlignAssignmentStatement: Ignore hashtables with a single key-value pair

PR Summary
Hashtables with a single key-value pair should be ignored by PSAlignAssignmentStatement as there is nothing to align.
There are other rules that handle whitespace around operators etc which can handle formatting according to the users preference.
Currently:
Invoke-Formatter -ScriptDefinition "@{'Key'='Value'}" -Settings @{
    Rules = @{
        PSAlignAssignmentStatement = @{
            Enable = $true
            CheckHashtable = $true
        }
    }
}

Results in:
@{'Key' ='Value'}

Note the space added after the hashtable key
Resolves #1979
PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Change is not breaking
[x] Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
[x] Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

@liamjpeters what was the issue or reason for closing this PR? Happy to support/help where needed

@bergmeister - I'm not the most familiar with git. I used the main branch of my fork for this PR but wanted to use a different branch. Didn't know how to do that so got things organised locally, closed this PR and opened a new one. #1986
Hopefully I've not created any issues by doing so

Gotcha, no worries


Double Status Bar Support

When in-call or tethering, the margins aren't quite correct for both shyNavBar states. For my UITableView-embedded UIViewControllers the hidden state remains 20px, and isn't pushed behind the 40px status bar. In addition, after revealing the shyNavBar, the navigation bar grows to 62px (the 20px appendage on the double status bar remains below it). When the content in my scroll view is not pushaway-scrollable (smaller than the frame), the small contraction when bouncing it offsets the navigation bar behind the status bar, as well.
I'm not certain if every one of these issues is completely attributable to TLYShyNavBar, but I'm certain at least the first and mutations of the rest are likely to exist in other apps. If these results are intentional, please let me know if there's something else I should do with the view controller to make the presentation less intrusive.

Hi @koenbud, was wondering if you worked it out. I'm struggling with this now.

Hi,
I created pull request with fix for this bug:
https://github.com/telly/TLYShyNavBar/pull/62

Merged. I hope this one is finally fixed. Cocoapods version bumped, as well.
Thanks again @Remki


Getting Dependency Errors (404) in Application Insights, while using Cosmos DB as State Store in .Net based Bot.

Bot Info

SDK Platform: .NET
SDK Version: <IP_ADDRESS>
Active Channels: Direct Line
Deployment Environment: Azure Bot Service, Azure App Service

Issue Description
Getting Dependency Errors (404) in Application Insights, while using Cosmos DB as State Store for the Bot.
The Bot SDK is probably unable to fetch the documents from the Cosmos DB and recording 404, while I am able to see the documents in the DB.
Code Example
N/A
Reproduction Steps

Look into other bots having Cosmos DB as data store and DB provisioned in one region only.

Expected Behavior
Ideally, there should be no 404 errors while reading data from Cosmos DB. I am not sure how to fix it as it is being done through SDK.
Actual Results
Application Insights shows 404 errors while getting documents which probably are accessed by the Bot SDK. The url that are shown as failure have the format:
http://[CosmosDBServiceName]-[Location].documents.azure.com/
I have provisioned only one Cosmos Instance in West Europe Region. In fact the Application Map under App Insights shows two Azure Document DB, I am not sure know why. I have experimented with both ETagBaseConsistency and LastWriteWins. Let me know which one to use.
Below is compiled picture to speak more about the error, some names are removed. Please help me how to fix this issue.

Which version of BotBuilder-Azure are you using?
404's when querying DocumentDb/CosmosDb are a known issue: https://github.com/Azure/azure-documentdb-dotnet/issues/219
You can see in the source for DocumentDbBotDataStore.cs that HttpStatusCode.NotFound exceptions are caught, and handled accordingly: https://github.com/Microsoft/BotBuilder-Azure/blob/master/CSharp/Library/Microsoft.Bot.Builder.Azure/DocumentDbBotDataStore.cs

Eric, thanks for responding, I am using BotBuilder-Azure 3.2.5.
I am not directly querying the documents from State Log, I believe SDK is doing so but failing with 404's which are getting recorded in AppInsights.
I don't see the source code of BotBuilder-Azure v3.2.5 on github
I checked the code is handling the NotFound exception, but it seems to be a newer version.

Please advice..

The point i was trying to make is that the 404's are not real errors.  https://github.com/Azure/azure-documentdb-dotnet/issues/219  The underlying library used by BotBuilder-Azure produces 404 exceptions when attempting to retrieve a document that does not exist.  This is a known issue with querying DocumentDb/CosmosDb.


[W5.11][W12-2]Lim Zhi Hui Alden

This could possibly increase the overall security of our program.  Nice!

This could possibly increase the security of our program, but we might have to implement additional features to ensure that the underlying location where the passwords are stored are secure too.  Overall nice work!


Exception error when using Microchip Pack

I encountered a problem when using the Microchip PACK for the atsamd51 series MCU's.
The first issue appears to be that pyOCD cannot determine the RAM layout.
The second problem causes an exception.
I tried with two different probes.  A JLink and a Particle Debugger (CMSIS-DAP) and the errors are similar.
Here is the output when using the JLink:
0010018:WARNING:cmsis_pack:Overlapping memory regions in file <zipfile.ZipFile filename='/Users/stevenslupsky/Documents/source/CMSIS-Packs/Microchip.SAMD51_DFP.3.3.76.atpack' mode='r'>, deleting outer region. Further warnings will be suppressed for this file.
0010054:WARNING:cmsis_pack:CMSIS-Pack device ATSAMD51J20A has no default RAM defined, cannot program flash
0010057:INFO:board:Target type is atsamd51j20a
0010612:INFO:dap:DP IDR = 0x2ba01477 (v1 rev2)
0010616:ERROR:discovery:Exception while probing AP#2: Unspecified error.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/jlink_probe.py", line 318, in read_ap
    value = self._link.coresight_read((addr & self.A32) // 4, ap=True)
  File "/usr/local/lib/python3.9/site-packages/pylink/jlink.py", line 204, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pylink/jlink.py", line 3361, in coresight_read
    raise errors.JLinkException(res)
pylink.errors.JLinkException: Unspecified error.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/discovery.py", line 128, in _find_aps
    isValid = AccessPort.probe(self.dp, apsel)
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/ap.py", line 304, in probe
    idr = dp.read_ap((ap_num << APSEL_SHIFT) | AP_IDR)
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/dap.py", line 703, in read_ap
    result_cb = self.probe.read_ap(addr, now=False)
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/jlink_probe.py", line 320, in read_ap
    six.raise_from(self._convert_exception(exc), exc)
  File "<string>", line 3, in raise_from
pyocd.core.exceptions.ProbeError: Unspecified error.
0010619:INFO:ap:AHB-AP#0 IDR = 0x24770011 (AHB-AP var1 rev2)
0010625:INFO:ap:AHB-AP#1 IDR = 0x74770001 (AHB-AP var0 rev7)
0010647:INFO:rom_table:AHB-AP#0 Class 0x1 ROM table #0 @ 0x41003000 (designer=01f part=cd0)
0010659:INFO:rom_table:[0]<-1ff01000:ROM class=1 designer=43b part=4c4>
0010659:INFO:rom_table:  AHB-AP#0 Class 0x1 ROM table #1 @ 0x-1ff01000 (designer=43b part=4c4)
0010678:INFO:rom_table:  [0]<-1fff2000:SCS-M4 class=14 designer=43b part=00c>
0010686:INFO:rom_table:  [1]<-1ffff000:DWT class=14 designer=43b part=002>
0010695:INFO:rom_table:  [2]<-1fffe000:FPB class=14 designer=43b part=003>
0010706:INFO:rom_table:  [3]<-20000000:ITM class=14 designer=43b part=001>
0010716:INFO:rom_table:  [4]<-1ffc0000:TPIU-M4 class=9 designer=43b part=9a1 devtype=11 archid=0000 devid=ca1:0:0>
0010724:INFO:rom_table:  [5]<-1ffbf000:ETM-M4 class=9 designer=43b part=925 devtype=13 archid=0000 devid=0:0:0>
0010734:INFO:rom_table:  [6]<-1ffbe000:ETB class=9 designer=43b part=907 devtype=21 archid=0000 devid=0:0:0>
0010736:INFO:cortex_m:CPU core #0 is Cortex-M4 r0p1
0010743:INFO:cortex_m:FPU present: FPv4-SP-D16-M
0010746:INFO:dwt:4 hardware watchpoints
0010751:INFO:fpb:6 hardware breakpoints, 4 literal comparators
0010772:INFO:server:Semihost server started on port 4444 (core 0)
0010776:INFO:gdbserver:GDB server started on port 3333 (core 0)

Here is the output when using the Particle Debugger:
0001084:WARNING:cmsis_pack:Overlapping memory regions in file <zipfile.ZipFile filename='/Users/stevenslupsky/Documents/source/CMSIS-Packs/Microchip.SAMD51_DFP.3.3.76.atpack' mode='r'>, deleting outer region. Further warnings will be suppressed for this file.
0001120:WARNING:cmsis_pack:CMSIS-Pack device ATSAMD51J20A has no default RAM defined, cannot program flash
0001123:INFO:board:Target type is atsamd51j20a
0001272:INFO:dap:DP IDR = 0x2ba01477 (v1 rev2)
0001310:ERROR:discovery:Exception while probing AP#2: Memory transfer fault
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/cmsis_dap_probe.py", line 286, in read_ap_result_callback
    return result()
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 847, in read_reg_cb
    res = transfer.get_result()
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 144, in get_result
    self.daplink.flush()
  File "/usr/local/lib/python3.9/site-packages/pyocd/utility/concurrency.py", line 28, in _locking
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 672, in flush
    self._read_packet()
  File "/usr/local/lib/python3.9/site-packages/pyocd/utility/concurrency.py", line 28, in _locking
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 932, in _read_packet
    decoded_data = cmd.decode_data(raw_data)
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 452, in decode_data
    data = self._decode_transfer_data(data)
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 359, in _decode_transfer_data
    self._check_response(data[2])
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/pydapaccess/dap_access_cmsis_dap.py", line 338, in _check_response
    raise DAPAccessIntf.TransferFaultError()
pyocd.probe.pydapaccess.dap_access_api.DAPAccessIntf.TransferFaultError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/discovery.py", line 128, in _find_aps
    isValid = AccessPort.probe(self.dp, apsel)
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/ap.py", line 304, in probe
    idr = dp.read_ap((ap_num << APSEL_SHIFT) | AP_IDR)
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/dap.py", line 729, in read_ap
    return read_ap_cb()
  File "/usr/local/lib/python3.9/site-packages/pyocd/coresight/dap.py", line 717, in read_ap_cb
    result = result_cb()
  File "/usr/local/lib/python3.9/site-packages/pyocd/probe/cmsis_dap_probe.py", line 288, in read_ap_result_callback
    six.raise_from(self._convert_exception(error), error)
  File "<string>", line 3, in raise_from
pyocd.core.exceptions.TransferFaultError: Memory transfer fault
0001345:INFO:ap:AHB-AP#0 IDR = 0x24770011 (AHB-AP var1 rev2)
0001355:INFO:ap:AHB-AP#1 IDR = 0x74770001 (AHB-AP var0 rev7)
0001375:INFO:rom_table:AHB-AP#0 Class 0x1 ROM table #0 @ 0x41003000 (designer=01f part=cd0)
0001382:INFO:rom_table:[0]<-1ff01000:ROM class=1 designer=43b part=4c4>
0001382:INFO:rom_table:  AHB-AP#0 Class 0x1 ROM table #1 @ 0x-1ff01000 (designer=43b part=4c4)
0001389:INFO:rom_table:  [0]<-1fff2000:SCS-M4 class=14 designer=43b part=00c>
0001394:INFO:rom_table:  [1]<-1ffff000:DWT class=14 designer=43b part=002>
0001399:INFO:rom_table:  [2]<-1fffe000:FPB class=14 designer=43b part=003>
0001404:INFO:rom_table:  [3]<-20000000:ITM class=14 designer=43b part=001>
0001409:INFO:rom_table:  [4]<-1ffc0000:TPIU-M4 class=9 designer=43b part=9a1 devtype=11 archid=0000 devid=ca1:0:0>
0001414:INFO:rom_table:  [5]<-1ffbf000:ETM-M4 class=9 designer=43b part=925 devtype=13 archid=0000 devid=0:0:0>
0001419:INFO:rom_table:  [6]<-1ffbe000:ETB class=9 designer=43b part=907 devtype=21 archid=0000 devid=0:0:0>
0001422:INFO:cortex_m:CPU core #0 is Cortex-M4 r0p1
0001430:INFO:cortex_m:FPU present: FPv4-SP-D16-M
0001435:INFO:dwt:4 hardware watchpoints
0001438:INFO:fpb:6 hardware breakpoints, 4 literal comparators
0001457:INFO:server:Semihost server started on port 4444 (core 0)
0001460:INFO:gdbserver:GDB server started on port 3333 (core 0)

This is the contents of pyocd.yaml:
pack:
  - /Users/stevenslupsky/Documents/source/CMSIS-Packs/Microchip.SAMD51_DFP.3.3.76.atpack
  - /Users/stevenslupsky/Documents/source/CMSIS-Packs/Microchip.SAMD21_DFP.3.3.98.atpack

Thanks for the report. I'll have to look at the reference manual for the SAMD51 parts. The fact that it has a second AHB-AP is kind of surprising since it's a single core device.
I'll also take a look at the CMSIS Pack memory region definitions. Given the warnings from pyocd, it seems like there are problems that Microchip will have to fix.
The J-Link "unspecified error" is really the same transfer fault, J-Link just doesn't return a meaningful error. I've created #1063 to convert it to a transfer fault, about the best that can be done.

Thanks for the report. I'll have to look at the reference manual for the SAMD51 parts. The fact that it has a second AHB-AP is kind of surprising since it's a single core device.
I'll also take a look at the CMSIS Pack memory region definitions. Given the warnings from pyocd, it seems like there are problems that Microchip will have to fix.
The J-Link "unspecified error" is really the same transfer fault, J-Link just doesn't return a meaningful error. I've created #1063 to convert it to a transfer fault, about the best that can be done.

The exception says it crashed while probing AP#2.  The numbering in the dump suggests they start at 0 so AP#2 would be the third port?  The info dump lists AP#0 and AP#1 but there is no rom_table info for AP#1.
Looking at the datasheet I cannot really tell if it has more than one AHB-AP.  It seems to me there is only one but there is a TPIU port and a SWD port so perhaps somehow they enumerate as separate AP's?

The exception says it crashed while probing AP#2.  The numbering in the dump suggests they start at 0 so AP#2 would be the third port?  The info dump lists AP#0 and AP#1 but there is no rom_table info for AP#1.
Looking at the datasheet I cannot really tell if it has more than one AHB-AP.  It seems to me there is only one but there is a TPIU port and a SWD port so perhaps somehow they enumerate as separate AP's?

Oh, I didn't even catch that about probing AP#2! That means that is got a fault result from attempting to read the ID register of AP#2. This is against the Arm ADI specification, which requires that reads of nonexistent AP registers return 0 (RAZWI in Arm terminology).

Oh, I didn't even catch that about probing AP#2! That means that is got a fault result from attempting to read the ID register of AP#2. This is against the Arm ADI specification, which requires that reads of nonexistent AP registers return 0 (RAZWI in Arm terminology).

I get the same error with mcu ATSAME54P20A which is on the microchip devkit for same54 family. Is there any work around using the microchip pack? I am trying to setup a toolchain using pyocd as the chip handling layer but this seems to be a blocking problem doesn't it?


Fix/refs passing

Currently Github::Client::GitData::References.list(user, repo, :ref => ref) with ref = "/refs/heads/lleger-refactor" is not working.
Tests were passing but request mocking was on .../refs/refs/heads/lleger-refactor and not .../refs/heads/lleger-refactor.

Coverage decreased (-0.0004%) to 95.401% when pulling 6d1503046da65b1489b5a138b94fa86de58331f4 on nicolasleger:fix/refs_passing into da252ecfe7c5b692cf61ff6be40f67d888fb2953 on piotrmurach:master.

Thanks ❤


Error unmarshal string

I'm having problems discovering queues and some other data.
When I send the command in order to debug then not supported message, I received:
[t|json: cannot unmarshal string into Go struct field QueueInfo.consumer_utilisation of type float64]
Please anyone can help me?
Thanks

Hey @ericllistar. Could you please provide a little bit more information. What version of the extension do you use? What version of RabbitMQ? What command leads to the error message?

Sure! Sorry days ago this description had sense in my mind, but now.... reading again.... I don't understand anything hahaha.
I've installed that extension for Zabbix agent, and import the template.
And then the item "Template App Rabbitmq Service: RabbitMQ stats" give me that output:
Cannot evaluate expression: "Cannot evaluate function "<IP_ADDRESS>:rabbitmq.stats[{$ZABBIX_SERVER_IP},{$RABBITMQ_MGMT_ADDRESS},{$RABBITMQ_USER},{$RABBITMQ_PASSWORD},{$RABBITMQ_CA_PATH},{HOST.HOST}].regexp(OK)": item is not supported.".
So, in order to debug that error I check it directly on target machine (rabbitmq machine):
COMMAND --> zabbix_agentd -t rabbitmq.stats[<IP_ADDRESS>,http://<IP_ADDRESS>:15672,user,password,/,<IP_ADDRESS>]
And that's giving me the following error:
ERROR --> [t|json: cannot unmarshal string into Go struct field QueueInfo.consumer_utilisation of type float64]
I'm very newbie in go and don't know how to solve that
I'm using Zabbix Server 4.2, Zabbix Agent 4.2 also. The rabbit version is 3.3.0
Thanks for your patience

Hmm, it seems to the RabbitMQ has a too old version (5 y) and management API doesn't support several fields.
This exception happens when extension trying to unmarshalling JSON data into Go struct QueueInfo
Could you please ask API directly via curl and try to find consumer_utilisation field?
URL for checking - curl -u user:password -H "content-type:application/json" http://RABBITMQ_ADDR:15672/api/queues
For lastest version of RabbitMQ (3.6, 3.7) these field looks like
    "memory": 198460,
    "consumer_utilisation": 0.8916439525099574,
    "consumers": 1,
    "exclusive_consumer_tag": null,
    "effective_policy_definition": {
      "ha-mode": "all",
      "dead-letter-routing-key": "api.events.dead",
      "dead-letter-exchange": "events-firehose"
    },
    "operator_policy": null,
    "policy": "dlx-api.events"
  },


Error: 401, 'Failed to verify the HTTP(S) Signature'

Describe the bug
File "/home/ubuntu/oracle-freetier-instance-creation/main.py", line 310, in execute_oci_command
response = getattr(client, method)(*args, **kwargs)
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/oci/identity/identity_client.py", line 8285, in get_user
return retry_strategy.make_retrying_call(
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/oci/retry/retry.py", line 308, in make_retrying_call
response = func_ref(*func_args, **func_kwargs)
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/oci/base_client.py", line 535, in call_api
response = self.request(request, allow_control_chars, operation_name, api_reference_link)
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/circuitbreaker.py", line 159, in wrapper
return call(function, *args, **kwargs)
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/circuitbreaker.py", line 170, in call
return func(*args, **kwargs)
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/oci/base_client.py", line 726, in request
self.raise_service_error(request, response, service_code, message, operation_name, api_reference_link, target_service, request_endpoint, client_version, timestamp, deserialized_data)
File "/home/ubuntu/oracle-freetier-instance-creation/.venv/lib/python3.10/site-packages/oci/base_client.py", line 891, in raise_service_error
raise exceptions.ServiceError(
oci.exceptions.ServiceError: {'target_service': 'identity', 'status': 401, 'code': 'NotAuthenticated', 'opc-request-id': '394A6BBD79A24FD7BDF0F791110943B9/F96194D1D64B5396A5BDC553A6491C3E/33F6DED5A9BC2698251D8F393C1F31DF', 'message': 'Failed to verify the HTTP(S) Signature', 'operation_name': 'get_user', 'timestamp': '2024-07-30T03:15:30.628592+00:00', 'client_version': 'Oracle-PythonSDK/2.129.4', 'request_endpoint': 'GET https://identity.us-ashburn-1.oci.oraclecloud.com/20160918/users/ocid1.user.oc1..aaaaaaaart3iegzqs6dyasojlff4gbpt4ly6qcjml2hgn33pemvx35fmbw6a', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.', 'troubleshooting_tips': "See https://docs.oracle.com/iaas/Content/API/References/apierrors.htm#apierrors_401__401_notauthenticated for more information about resolving this error. Also see https://docs.oracle.com/iaas/api/#/en/identity/20160918/User/GetUser for details on this operation's requirements. If you are unable to resolve this identity issue, please contact Oracle support and provide them this full error message."}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/oracle-freetier-instance-creation/main.py", line 464, in
launch_instance()
File "/home/ubuntu/oracle-freetier-instance-creation/main.py", line 364, in launch_instance
user_info = execute_oci_command(iam_client, "get_user", OCI_USER_ID)
File "/home/ubuntu/oracle-freetier-instance-creation/main.py", line 317, in execute_oci_command
handle_errors(args, data, logging_step5)
File "/home/ubuntu/oracle-freetier-instance-creation/main.py", line 290, in handle_errors
raise Exception("Error: %s" % data)
Exception: Error: {'status': 401, 'code': 'NotAuthenticated', 'message': 'Failed to verify the HTTP(S) Signature'}
Expected behavior
I get the above output when running main.py before the process closes.
Instance Type:

ARM or Micro: ARM
Micro Second Instance: no
Custom Image or default:

Environment:

Micro Instance
OS: Ubuntu
Version: 20.4

check if your key path is proper in config file

Seemingly not, here's my path to the key:

And here's how I have it in the oci_config file:

@willjvv  Can you try to change the key file permissions:
chmod go-rwx  /home/ubuntu/oracle-freetier-instance-creation/oci_api_private_key.pem

@willjvv Can you try to change the key file permissions: chmod go-rwx /home/ubuntu/oracle-freetier-instance-creation/oci_api_private_key.pem

Same problem:
setup_init.sh: 21: source: not found
setup_init.sh: 25: pip: not found
setup_init.sh: 26: pip: not found
setup_init.sh: 27: pip: not found


Share smtp/webUI with different subdomain

Hello
I'm not sure how to formulate my question then sorry for this first opening.
We would propose on a shared hosting a mailpit instance to each different users.
Actually we could start different instance and set a different ports to each account. But is not convenient as we need to open many ports and provide different proxy configuration to match instance with user.
If I've underestood correctly, some element are yet present to manage different tenant with same instance. In this case our shared user hosting could be identified as different tenant.
Could be possible to provide to difference service as smtp/web ui an environnement value to associate tenant value ?
Thanks a lot

Hello @camlafit. I think I understand what you are asking.
Firstly the SMTP protocol does not support subdomains in the same manner HTTP does, so that is not possible (unless subdomains point to different unique IP addresses).
Secondly, Mailpit does not support multiple users. Mailpit was just not designed for this, and to support Multiple users (and mailboxes) would mean a complete rewrite (which is not going to happen, sorry).
The tenancy support you are thinking of is a database tenancy whereby different instances of Mailpit can use a single hosted rqlite database, so this is not what you are looking for either.
The only solution (for what you are trying to do) in in fact either to separate different Mailpit instances (one for each user) on different ports, or on different (virtual?) IP addresses. I don't know how many users you are wanting to support, nor what your network is like, but I see no other option.

Hello
Thanks for your reply.
As we are with shared hosting, we can have 1 to hundreds accounts, but I think we have to manage less than 10 users requiring an mailpit service.
We can manage it with different ports , we must be a bit intelligent to enable/disable service on the fly. I'll reflect about this and search an approach to manage these instances.
Thanks a lot

I think, we can close in this case :)
thanks

I'm sorry I can't help :)