Automation fails when longitude is slightly negative

Some jobs submitted with the following inputs failed (email redacted):
site_id=np,uc,hf
start_date=20210423
end_date=20210731
lat=51.426,51.523,51.608
lon=-0.345,-0.132,0.175
email=******
mod_fmt=none
vmr_fmt=none
map_fmt=text

The error was:
Traceback (most recent call last):
  File "/home/tccon/AutoModRust/python/ginput/run_ginput.py", line 61, in <module>
    main()
  File "/home/tccon/AutoModRust/python/ginput/run_ginput.py", line 57, in main
    driver(**args)
  File "/home/tccon/AutoModRust/python/ginput/ginput/priors/automation.py", line 255, in job_driver
    _make_map_files(all_args)
  File "/home/tccon/AutoModRust/python/ginput/ginput/priors/automation.py", line 172, in _make_map_files
    vmrf = vmr_files[key]
KeyError: '2021073000_000W_51N'

Looking at the files produced for uc (the slightly west site):
$ find fpit/uc/ -type f
fpit/uc/vertical/FPIT_2021073000Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073003Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073006Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073009Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073012Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073015Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073018Z_52N_000W.mod
fpit/uc/vertical/FPIT_2021073021Z_52N_000W.mod
fpit/uc/vmrs-vertical/JL1_2021073000Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073003Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073006Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073009Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073012Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073015Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073018Z_52N_000E.vmr
fpit/uc/vmrs-vertical/JL1_2021073021Z_52N_000E.vmr

the .vmr files are named "000E" instead of the "000W" that the automation is looking for, so there is a mismatch in how rounding around 0 is handled.

Looking into this more thoroughly, I see that the .mod and .vmr files actually have different names: the .mod files use "000W" and the .vmr files use "000E". The .mod files generate the file name with mod_utils.mod_file_name and the .vmr files do so with mod_utils.vmr_file_name.
The issue is that we know the longitude when creating the .mod file, so it can correctly compute east vs. west. The .vmr file needs to use just the coordinate in the file name, which since it is 0, it assumes that the direction should be "east". The .mod files correctly match gsetup in that the coordinate should be west if the obs. longitude is at all < 0.
My fix for this was to change how ginput determines the output path for .vmr files; if it's given paths to .mod files, it uses the .mod file name directly, rather than recalculating it from the lat/lon. It falls back to that approach if given a data dictionary instead. This is implemented in commit 14b6fbe for v1.0.6 and 8aa57ff for the main branch.


Optionally get rid of intermediary boxes in visualize, and add more labels

[x] Tests added / passed
[x] Passes black dask / flake8 dask

This PR adds the collapse and verbose options to visualize. Collapse gets rid of all non-connected non-task boxes as well as any boxes that are just the output of a task that feeds into other tasks. Verbose adds a label to all the boxes even if the label is a duplicate.
This came up in #5928. Here are some images of what the new graphs from that page would look like:

collapse=False
collapse=True

verbose=False

verbose=True

@quasiben since we were talking about this at the dask meeting.

CI failure is just https://github.com/dask/dask/issues/5972, which I haven't had any luck with yet.

I added an unconnected nodes example to the first comment.

Great, thanks.
I pushed a commit merging master, which should make the CI failure go away.

Thanks!

I'll redo #5928 once this is in :)

Thanks @jsignell!


Adds Schema Mapping

I think that we can use SchemaStore to import Schemas and then used it to offer autocompletion.
It can save a lot of time writing code.

duplicate of https://github.com/microsoft/vscode/issues/26289


Ft 104 multi file sync handle set deletions

@swordEdge are we able to update the test coverage to pass?

@swordEdge are we able to update the test coverage to pass?

Modified test coverage for multi-file sync. I think it's still failing because of another unit test.

@swordEdge it's stil failing because src/storage/GithubTokenStorage.ts has a branch coverage diff of -1, maybe you can check which lines can be covered additionally?


ITMS-90338: Non-public API usage Frameworks/Flutter.framework/Flutter: _ptrace

Hello everyone,I upload my flutter app to AppleStore,but gets an "Frameworks/Flutter.framework/Flutter: _ptrace" error,how to resolve it?

flutter build ios --release


Training process not stable

Durintg training, the loc_loss ocassionly became a very large value (As shown below). I checked that the calculation of loc_loss would not encounter the denominator=0 situation. I wonder why.

counter:[21000], loss:9098183573504.00000000, loc_loss:9098183573504.00000000, conf_loss:0.00971856, conf_pos_loss:0.00420601, conf_neg_loss:0.00551255, reg_loss:32.57967377

counter:[22000], loss:285012888911872.00000000, loc_loss:285012888911872.00000000, conf_loss:0.01419698, conf_pos_loss:0.00380920, conf_neg_loss:0.01038779, reg_loss:34.89893341

Durintg training, the loc_loss ocassionly became a very large value (As shown below). I checked that the calculation of loc_loss would not encounter the denominator=0 situation. I wonder why.

counter:[21000], loss:9098183573504.00000000, loc_loss:9098183573504.00000000, conf_loss:0.00971856, conf_pos_loss:0.00420601, conf_neg_loss:0.00551255, reg_loss:32.57967377

counter:[22000], loss:285012888911872.00000000, loc_loss:285012888911872.00000000, conf_loss:0.01419698, conf_pos_loss:0.00380920, conf_neg_loss:0.01038779, reg_loss:34.89893341

Since everyone using this code could encounter his/her specific issues, completely relying on us to solving these issues is unpractical, in part because the information is not enough in part because we have not so much time or effort, so we refer the users to the Q&A part in README, especially the second Q&A for your own data.


267 error in your code

i tried all configurations
always 267 error in your code when try to built
tried all possible configurations, still giving 267 errors while compiling !!!

Hey, you can add a screenshot of error. I can try to help you.

By the sounds of it you didn't fill out anything in the GameDefines/PiecesOfCode files, this repository is not a place for help regarding how Unreal Engine 3 works.


fixes #21

Does not switch a color depending on theme

@hnvn any comments, thoughts?

Great. I miss your request


"tldr --update" shows message "Local data is older than two weeks, use --update to update it."

Reproduction Rate
everytime
Steps to Reproduce
Install tldr.
Wait more than 2 weeks
perform Local data is older than two weeks, perform "tldr --update"
Result
Shows error "Local data is older than two weeks, use --update to update it."
Expected Result
Why is it showing me an error to update, when I just passed the update flag? Shouldnt show this error message.
Additional Information
~ $tldr --version
tldr v1.3.0 (v1.3.0)
Copyright (C) 2016 Arvid Gerstmann
Source available at https://github.com/tldr-pages/tldr-cpp-client

Still having this redundant warning, @Leandros could you please release a new stable version?
~ » tldr --version
tldr v1.3.0 (v1.3.0)
Copyright (C) 2016 Arvid Gerstmann
Source available at https://github.com/tldr-pages/tldr-cpp-client

~ » tldr --update
Local data is older than two weeks, use --update to update it.

Still having this redundant warning, @Leandros could you please release a new stable version?
~ » tldr --version
tldr v1.3.0 (v1.3.0)
Copyright (C) 2016 Arvid Gerstmann
Source available at https://github.com/tldr-pages/tldr-cpp-client

~ » tldr --update
Local data is older than two weeks, use --update to update it.

@waldyrious / @sbrl, could we release a new version?

@waldyrious / @sbrl, could we release a new version?

@bl-ue what precisely do you mean by release a new version? I'm happy to add a new GitHub release, but I would assume there would be some package managers that would also need to be updated.
Is there a "making a release" guide anywhere?

@bl-ue what precisely do you mean by release a new version? I'm happy to add a new GitHub release, but I would assume there would be some package managers that would also need to be updated.
Is there a "making a release" guide anywhere?

@sbrl I believe the workflow in @owenvoke's #55 will react when a new release is tagged in this repo, and automatically create a PR to update the version of tldr in the Homebrew package manager (which is the only one mentioned in the README, so I assume no other updates would be needed). Can you merge that PR so we could test that flow?

@sbrl I believe the workflow in @owenvoke's #55 will react when a new release is tagged in this repo, and automatically create a PR to update the version of tldr in the Homebrew package manager (which is the only one mentioned in the README, so I assume no other updates would be needed). Can you merge that PR so we could test that flow?

Oops! This must have been drowned in my GH notifications.
PR merged, @waldyrious!

No worries @sbrl :) thanks for merging the PR!


[PE-D][Tester D] editta not working and showing unrelated error message

Similar to the editstu bug, I am getting a unrelated error message for editta, when I have inputted the index and the parameter to be changed.

Labels: severity.High type.FunctionalityBug
original: shogun187/ped#5

duplicate problem with #245


Normalize network in docker-compose.yaml

See https://github.com/Senzing/docker-python-demo/issues/85

Pull requests:

https://github.com/Senzing/configurator/pull/76


Automatic documentation generation for web-sys bindings

It'd be awesome if we could automatically hook up something like Element::class_name to the MDN docuementation for the method!

Were you thinking about just linking to the MDN page or including part of it as rust comments?

Oh nothing as fancy as scraping words, just mdn URLs!

Some improvements on what I just submitted:

[ ] Mock arguments to setters like input.accept = "val"; based on their type.
[ ] Explain arguments to methods like: document.querySelector('val') or document.querySelector(selectors)
[ ] Scrape code examples from MDN, use ESTree to translate them to Rust.
[ ] Provide a mode that checks for 404's on links and omits them if they don't exist. (a fair few of the HTML*Interface.getter don't exist)
[ ] Place Rust code examples in a file and combine them, this would provide the best code examples and also provide compile time checks for the documentation against webidl changes. (It may make sense to have some IDL overrides in a file anyway: #458)


Release Candidate 0.4.0-rc2

This patch marks release candidate 0.4.0-rc2.

Current coverage is 30.89% (diff: 100%)

Merging #368 into release/0.4.0 will not change coverage

@@           release/0.4.0       #368   diff @@
===============================================
  Files                 29         29
  Lines               1719       1719
  Methods                0          0
  Messages               0          0
  Branches               0          0
===============================================
  Hits                 531        531
  Misses              1130       1130
  Partials              58         58

Powered by Codecov. Last update d2ea674...dd612d4


Error running Uchiwa: undefined: http.MethodHead && undefined: http.MethodGet

When I run uchiwa I get this:
godep go run uchiwa.go
# github.com/sensu/uchiwa/uchiwa/authorization
/usr/lib/golang/src/github.com/sensu/uchiwa/uchiwa/authorization/authorization.go:34: undefined: http.MethodHead
/usr/lib/golang/src/github.com/sensu/uchiwa/uchiwa/authorization/authorization.go:34: undefined: http.MethodGet
godep: go exit status 2

I am running uchiwa 0.17.1.  I upgraded our sensu - tbh I don't remember which version.
Not sure where to go from here?

Looks like https://github.com/sensu/uchiwa/pull/528 may have changed the requirements from go>=1.3 to >= 1.6

Yep - that is correct - just kind of figured that out by digging around and IRC.
Documentation still has Recent version of go (>= 1.3) (guide).

YEP update GO to 1.6 or higher. CLOSING

@EZtheOG you should probably keep this open and change it to be a request ot update the documentation to state the correct depdenency


Improve UG clarity

This PR does the following:

Change the term 'parameters' to the more layman-friendly 'fields'
Fix broken links
Add a short paragraph regarding the purpose of the UG

Fixes #165, #166, and #167

Codecov Report

Merging #216 (8c0ccde) into master (5913f6d) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #216   +/-   ##
=========================================
  Coverage     51.12%   51.12%
  Complexity      616      616
=========================================
  Files           118      118
  Lines          2707     2707
  Branches        322      322
=========================================
  Hits           1384     1384
  Misses         1224     1224
  Partials         99       99

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5913f6d...8c0ccde. Read the comment docs.


Does the vocabulary size impact the amount of data required for training?

Checks

[X] This template is only for question, not feature requests or bug reports.
[X] I have thoroughly reviewed the project documentation and read the related paper(s).
[X] I have searched for existing issues, including closed ones, no similar questions.
[X] I confirm that I am using English to submit this report in order to facilitate communication.

Question details
I've been experimenting with this recently, there seems to be some great potential here.  It does clearly take a lot of data to work well.  I was wondering then, if it would train more quickly or with less data if the vocab.txt file is smaller?  Has anyone tried this out?  I realize it would reduce the multi-language capability of the resultant model.

no idea,
thought the hard part is to learn speech-text alignment but word embed seems no conflict with that
and if we use big vocab size but just use part of it, the gradient will just go for the used ones, thus the only bad thing is wasted params and sparse weights (low utilization rate)?

That's clear, thank you!  Seems like it wouldn't make much of a difference in practice.


Use RTCPeerConnection defined on the window

... to prevent reference error in a browser where it's not defined globally. In Safari, this was throwing a ReferenceError: Can't find variable: RTCPeerConnection and also happened to be a linter error when I opened up the file.

 Looks like we need a safari test :-(

Fixed in #63


Navigation doesn't work in onOpened callback when app in background

Hi all!
I'm creating a simple video chat app. When the "accept" button on the notification is clicked, the video screen should open up.
I'm using the standard react-navigation package.
Here's an abridged copy of my code:
export default class HomeScreen extends Component {

  constructor(props) {
    super(props);
    this.onOneSignalIds = this.onOneSignalIds.bind(this);
    this.onOpened = this.onOpened.bind(this);
  }

  componentDidMount() {
    OneSignal.configure({});
    OneSignal.removeEventListener('ids', this.onOneSignalIds);
    OneSignal.removeEventListener('opened', this.onOpened);
    OneSignal.addEventListener('ids', this.onOneSignalIds);
    OneSignal.addEventListener('opened', this.onOpened);
  }

  ...

  onOpened(openResult) {

    if (openResult.action.actionID == "accept") {
      let roomId = openResult.notification.payload.additionalData.roomId;

      const { navigate } = this.props.navigation;
      navigate('VideoScreen', { 'roomId' : roomId });
    }
  }

  ...

}

When the app is in the foreground, this works great. However, when the app is in the background, HomeScreen opens up, not VideoScreen.
Any ideas how I can fix this?
Thanks everyone!!! :)
PS - Why do I have OneSignal.removeEventListener() in componentDidMount() instead of componentWillUnmount()? I found that for whatever reason, the event listeners weren't always getting removed and the handlers were getting called multiple times.

I think this might be related to this issue here:
https://github.com/react-community/react-navigation/issues/476
HomeScreen is actually getting loaded multiple times! Check this out (I put some logs in):
HomeScreen::constructor()
HomeScreen::componentDidMount()
HomeScreen::onOpened()
HomeScreen::constructor()
HomeScreen::componentDidMount()
HomeScreen::componentWillUnmount()
HomeScreen::constructor()
HomeScreen::componentDidMount()

What I noticed is that VideoScreen is getting loaded (constructor() + componentDidMount() are called), however it's not visible. It seems like HomeScreen is getting loaded right back on top of it!
As a temporary workaround, I've had to declare a variable (roomToLoad) outside the scope of HomeScreen. I then check this in componentDidMount(), and navigate from there, which works.
// roomToLoad is a variable at file scope
if (roomToLoad) {
  const { navigate } = this.props.navigation;
  navigate('VideoScreen', { 'roomId' : roomToLoad });
  roomToLoad = null;
}

Someone tell me what the difference if I register event on parent App component or child component. Because I using react-navigation thanks.


Nix: split overlays further, improve documentation

Splits the overlays such that there's one overlay per language, and no package definitions in flake.nix. Also, updates nix/README.md and adds a lot of comments to Nix expressions.

!ci-build-me


Fix nextui node tooltip without image field

For non container nodes, image filed is not set.
Without this, the nx graph node tooltip image field in value colume will be skipped. Cause the filed name and value mismatch.
I set the image field to  N/A to indicate no image property to fix this problem.
Before fix:

After fix:

thanks @gamerslouis ! It is amazing to see you fixing JS code of the graph engine as I am myself has very little knowledge of JS


OCPBUGS-9960: Add uninstall message annotation to csv

/lgtm

/cc @xingxingxia
for qe-approved
/cc @davemulford
for px-approved

/label px-approved

Will do post-merge test.
/label qe-approved

/hold

/hold cancel

/retest-required

/lgtm

/label docs-approved
adding this label on my own as @xenolinux is OOO
(do doc changes would be required for this PR except for what is already in https://github.com/openshift/openshift-docs/pull/59057)


Calculate DOXY_ADJUSTED_ERROR after filling DOXY_ADJUSTED

Requires time, error in gain and its time of calculation, oxygen values. Will detail further while referencing the QC cookbook (Bittig et al. 2018)

Going to consider this issue to be effectively the same as #11, so will close it now as not to have a duplicate.


Two-way aggregation binding does not work

When a UI control is bound to an oData model (two-way data binding), it is not possible to insert new aggregations to the model.
Could you please fix that?
Thanks.

Hi,
could you please provide a JSBin example or something similar where we can reproduce the problem?
Kind regards,
Janina

Hi guys,
This is an old feature request, I have no idea whether the feature has been implemented. Here we accept mainly bug reports, therefore I am closing the issue.
BR,
Shtiliyan


cmd/vet: false positive for user-defined verbs

What version of Go are you using (go version)?

$ go version
go version go1.13.6 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/user/bin"
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/user"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/user/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/user/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/user/src/github.com/kortschak/loopy/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build878536771=/tmp/go-build -gno-record-gcc-switches"

What did you do?

~/src/github.com/kortschak/loopy/cmd/bundle [master*]$ go test
# github.com/kortschak/loopy/cmd/bundle
./bundle.go:64:3: Fprintf format %60a has unknown verb a
~/src/github.com/kortschak/loopy/cmd/bundle [master*]$ echo $?
2

What did you expect to see?
No complaint.
What did you see instead?
Vet failure causing test failure (well, it would be a test failure if there were a test).
Additional information
The %a verb is non-standard verb, an addition to the biological sequence types provided by bíogo through their fmt.Formatter satisfaction. However, they have been in existence since 2013 and when they were included (as is still true), there was no documentation stating that only verbs listed in the fmt package for the built-in types should be used.

Can I help with this?

@tiriplicamihai go for it

I will try to investigate tomorrow evening (EET here).

Hey @kortschak vet seems to properly work. I wrote this dummy example and the command runs without any issues:
package main

import (
    "fmt"
    "os"
)

type A struct{}

func (_ *A)  Format(f fmt.State, c rune) {}

func main() {
     out, _ := os.Create("cucu")
     defer out.Close()
     a := &A{}
     fmt.Fprintf(out, "%a", a)
}

I looked a bit over your code and it seems the dependency  does not actually implement the Formatter interface: https://github.com/biogo/biogo/blob/master/seq/seq.go (at least not in the version I found on GH).
My 2c here is that you either change '%a' to an allowed verb or submit a request to have sequnece implement the Formatter interface.
@josharian I think this issue can be closed.

I’ll let @kortschak confirm and close if appropriate.

It's not correct that the type doesn't implement fmt.Formatter. The interface doesn't, but the concrete type does.

It does however explain what the problem is. Clearly vet is not considering whether the arg to fmt.Printf is an interface type when it's looking at the method set. Probably it should not try to look at interface types since the concrete value held in the interface may satisfy the fmt.Formatter interface.
You can see this kind of behaviour in vet for the common verbs here
package main

import "fmt"

func main() {
	var i interface{} = 1
	fmt.Printf("%s", i)
}

This does not cause a vet warning, but the equivalent without the interface {} type decl does. The vet check on fmt.Formatter should follow this behaviour.
In src/cmd/vendor/golang.org/x/tools/go/analysis/passes/printf/printf.go, isFormatter should optimistically return true if the type is an interface, something along the lines of (although named types may need to be unwrapped)...
func isFormatter(typ types.Type) bool {
        if _, ok := typ.(*types.Interface); ok {
                return true
        }
        obj, _, _ := types.LookupFieldOrMethod(typ, false, nil, "Format")
        fn, ok := obj.(*types.Func)
        if !ok {
                return false
        }
        sig := fn.Type().(*types.Signature)
        return sig.Params().Len() == 2 &&
                sig.Results().Len() == 0 &&
                isNamed(sig.Params().At(0).Type(), "fmt", "State") &&
                types.Identical(sig.Params().At(1).Type(), types.Typ[types.Rune])
}

If that's acceptable, I can send a CL. Presumably this goes to x/tools and will then get vendored in to the project source from there.


ruff fix on UP032 breaks applied code when f-string is too long

Hi, I'm not sure where I should post this error but I have a weird chain reaction of errors that result in ruff breaking my code.
The following code results in a UP032 Error:
from django.utils.translation import gettext

long = 'long'
split_to = 'split_to'
gettext(
    'some super {} and complicated string so that the error code '
    'E501 Triggers when this is not {} multi-line'.format(
        long, split_to)
)

when I then run ruff --fix on it I get
from django.utils.translation import gettext

long = 'long'
split_to = 'split_to'
gettext(
    f'some super {long} and complicated string so that the error code '
    f'E501 Triggers when this is not {split_to} multi-line'
)

This would be perfect, I love the style of code, but there is a problem, this code does not work.
When I then go ahead and run makemessages as usual, the text generated from this is
some super {long} and complicated string so that the error code  and the second line is lost
Should I report this upstream at Django or at python? I would prefer if this would work but sadly this is a rare edge case where ruff --fix breaks code.
this happens on ruff 0.0.290

It looks like gettext does not yet support f-strings, per the Django docs:

We can avoid suggesting this in Ruff, for gettext calls...


TAKEANDEAT -Incorrect county in the job listings on "peviitor.ro"

ENVIRONMENT: production
URL: https://scrapers.peviitor.ro/
Browser: Chrome
Device: Laptop
OS: Windows 10
STEPS TO REPRODUCE:

Open the url
Type "takeandeat" in search field
Press on search button
Click on "Run Scraper" button
Click on "See jobs"
Check the county of the jobs

EXPECTED RESULTS:
In the job listings from "peviitor.ro", for the city "Iași" we expected to see the county "Iași"
ACTUAL RESULTS:
In the job listings from "peviitor.ro" we see the village "Iași" and the county "Brașov"
ATTACHMENTS:

fixed can you retest

Retested, the county is correct.


Multiple currencies v3

This is based on #76, with few changes and bugfixes. Tested myself various scenarios, it works.

You can specify BTC in addition to fiat currencies in a supported currency list. In that case also sats will be added as an option automatically.
base_currency config option is kept, it will specify default selection for a list of supported currencies. Also made supported_currencies default to just USD, so that for people who had base_currency = USD default previously, this will work as before, without requirement to even add supported_currencies to config.
Some config sanity checks.
Removed rewrite of payments/price_feed.py with PriceFeed class. It's likely good idea, just out of scope for this PR. Separate refactor PR can be opened for that, but better when author will have time to test it himself (hint - SatSale works with signet too, which is kinda lightweight and syncs fast, you can just install bitcoin-qt on your development computer).
Some HTML / CSS changes.

Note - when user select BTC or sats in donation page, this value will be in db field fiat_currency. In this regard maybe we should rename it to specified_currency or something, but maybe it's not a big deal for now, in that case we should also rename fiat_amount column.
Co-authored-by: @eddiepease
Fixes #18.

works so well! And perfectly setup to make sats the default currency


Truncate long file names in file browser

File browser is messed up when having long file names

This isssue also causes a Download failed when trying to download the file


Install HWRaid packages as part of rpc_support

Add HWRaid repo and install default packages megaraid and lsiutil. The RAID
tools installed can be overridden with the variable hwraid_apt_packages.
Partial-Fix: #186
Signed-off-by: Matthew Thode<EMAIL_ADDRESS>

recheck - failed misc tempest test

recheck - 22:18:39 TASK: [os_tempest | Ensure cirros image] **************************************

:+1:

Straightforward backport


feat: added item freeze functionality to draggable list

Description
This PR introduces disabling the drag of an item in the draggable list.
Fixes PR Issue
Type of change

Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?
Manual
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Issues noticed:

When two or more draggable list items are frozen either on top or at the bottom, the gap between the items vanishes and they stick together.
The drag experience is not smooth. Dragging an item towards the bottom pusher it above the item before it.

Please refer the below video for reference:
https://www.loom.com/share/cdb396d49a394254b41f93bb6ee864f8

Issues noticed:

When two or more draggable list items are frozen either on top or at the bottom, the gap between the items vanishes and they stick together.
The drag experience is not smooth. Dragging an item towards the bottom pusher it above the item before it.
For frozen items we see the same cursor icon that we see for unfrozen items. This can be confusing for the users.

Please refer the below video for reference: https://www.loom.com/share/cdb396d49a394254b41f93bb6ee864f8

@shadabbuchh issue 1 is resolved. Issue 2 already exists in the prod.
For issue 3, it is a widget-specific issue i.e. the cursor is managed by the widget that is using it. So I will make changes to the main PR specific to the table widget.

All issues are fixed.

@keyurparalkar the sticky property now does not shift the list item to the top/bottom based on what value we have given.
https://www.loom.com/share/9a0ec3a404254533a203aa030bf12aea

All issue are fixed. The new isDragDisabled property is working as expected.


sidekiq_status integration

Hello!
I'm trying to use sidekiq_superworker with sidekiq_status. The problem is that i'm not able to use a core method of sidekiq_status, which is SidekiqStatus::Container.load. It gives me SidekiqStatus::Container::StatusNotFound: <jid here>.
What i'm doing is pulling some job with Sidekiq::Superworker::Subjob.find(x) and trying to use the returned JID param with SidekiqStatus method.
According to comments in SidekiqStatus code, that error means, that:

Exception raised if SidekiqStatus job being loaded is not found in Redis

https://github.com/cryo28/sidekiq_status/blob/master/lib/sidekiq_status/container.rb#L9
I'm not sure if that's a fault of sidekiq_status or sidekiq_superworker but i decided to start my small investigation here. Do you have any ideas or advices about something i have no idea about and what can be causing such issue on sidekiq_superworker side?
Btw. thanks for this awesome gem! I'm going to use it in quite complex video encoder and i'm really excited i've just found it :)
Regards,
m.

:+1:


Day46: 'Binary Tree Maximum Path Sum' solution

Binary Tree Maximum Path Sum
https://leetcode.com/problems/binary-tree-maximum-path-sum/
A path in a binary tree is a sequence of nodes where each pair of adjacent nodes in the sequence has an edge connecting them. A node can only appear in the sequence at most once. Note that the path does not need to pass through the root.
The path sum of a path is the sum of the node's values in the path.
Given the root of a binary tree, return the maximum path sum of any non-empty path.

Example 1:
Input: root = [1,2,3]
Output: 6
Explanation: The optimal path is 2 -> 1 -> 3 with a path sum of 2 + 1 + 3 = 6.
Example 2:
Input: root = [-10,9,20,null,null,15,7]
Output: 42
Explanation: The optimal path is 15 -> 20 -> 7 with a path sum of 15 + 20 + 7 = 42.


Add denormalize_relations configuration

Q
A

Bug fix?
no

New feature?
yes

BC breaks?
no

Deprecations?
no

Tests pass?
yes

Fixed tickets
-

License
MIT

Doc PR
-

Far from perfect:

I didn't try with circular references, not sure it will be handled correctly.
There is no tests. It's linked to #306 I guess but I have no idea of how to run a behat tests with another configuration. :/
Only 1 configuration entry for both normalization / denormalization, is it a good idea?

At least, it works for my use case, that's better than nothing. :)

Could you add tests?

First attempt to add tests.
I still need to create test dedicated to this new option but I would like to have your opinion on this implementation first.

Here you can see the diff between `features/suites/denormalize_relations/hydra/doc.feature` & `features/suites/normalize_relations/hydra/doc.feature`

What is the interest of adding this configuration option instead of using the (already supported) Serializer groups?

Cause I don't want to add a group on all properties of all my entities... :/
Do you see any other way to have this results without adding a new option which will probably not be used a lot?

I think you can use a custom Normalizer class decorating bundle normalizers and fitting your project needs.


Rotate texture in shader according to underlaying geometry

Hello, I am trying to accomplish texture rotation in Tangram according to geometry rotation as is done in Mapnik, for example this is OpenTopoMap styling for pitch texture:

I am trying to achieve this in Tangram style, but I am not sure whether it is possible to rotate texture based on geometry orientation (in this case landuse polygon) and then render it correctly in Shader. Currently I only get this:

This is styling I use:
texture-scale-pitch:
    base: polygons
    blend: overlay
    blend_order: -1
    texcoords: true
    shaders:
        uniforms:
            u_customtexture: images/otm-symbols-modified/sports-soccer.svg
        blocks:
            color: |
                color = texture2D(u_customtexture, v_texcoord);

Is it possible in Tangram styling to achieve what can be achieved in Mapnik?

I always wondered how Mapnik rendered the sports pitches!
@bcamper @tallytalwar @matteblair Does Tangram support data driven texture rotation and fitting?

Unfortunately you can't really do this with Tangram right now. The supported texture UV mapping types are here: https://tangrams.readthedocs.io/en/latest/Overviews/Materials-Overview/#mapping
I've previously been interested in adding an Oriented Bounding Box mapping type that would help with this case. However, ultimately I think it would still be of limited values, without a way to account for polygons broken up over tiles, e.g. without explicit hints in the feature data, the client has no way to know to adjust the UVs to correctly span a pitch or similar polygon that crosses tile boundaries.

Thank you for your answer. I thought that it will be impossible to achieve, but I had a little hope. So I will close this issue.


Remove all Pusher/Puller boxing using GATs

Currently the Allocate::allocate method is hardcoded to return a (Vec<Box<dyn Push<Message<T>>>>, Box<dyn Pull<Message<T>>>) pair. Boxing was necessary because the generic T parameter was on the method.
Generic associated types is the missing piece to allow the concrete type to be spelled out and since they are right around the corner for stable Rust I went ahead and implemented it.
This PR removes every single use of a Box<dyn Puller/Pusher> instance in the codebase. It is not mergeable yet because it requires the nightly feature flag but we could discuss it and merge it once GATs are released.

This PR now compiles on stable!


Bootstrap5 -- Should .ts-wrapper be prepended rather than appended?

Here is a video demonstration of what happens when changing the order of elements: https://files.catbox.moe/owx8q3.mp4
If a Bootstrap5 theme uses a border radius, it appears to be lost when using TomSelect. But only because of a first-child selector... Since the .ts-wrapper element also uses the .form-select class, and the real select is hidden... Perhaps it would make sense to have .ts-wrapper as the first-child, rather than the hidden select...
Here is the CSS from a Bootstrap 5 theme (from Bootswatch)... Without Tom Select initialized, the border radius displays... but initializing Tom Select causes the element to get pushed down from its position in the DOM:
.input-group > :not(:first-child):not(.dropdown-menu):not(.valid-tooltip):not(.valid-feedback):not(.invalid-tooltip):not(.invalid-feedback) {
  margin-left: -1px;
  border-top-left-radius: 0;
  border-bottom-left-radius: 0;
}

I believe this is the relevant spot where the appending happens: https://github.com/orchidjs/tom-select/blob/7a7744226ac9b6e12fc428b9cfedba126d48af67/src/tom-select.ts#L172
There does not currently appear to be a prepend function in utils.ts... I wonder if it's worth a toggle option of some kind?

Thanks for the bug report. Can you include a jsfiddle that demonstrates the problem? You might want to use one of these examples as a starter: https://tom-select.js.org/examples/styling/


New banner and winget badge logo changed to Windows 11 one

See how the new banner in readme.md looks here: https://github.com/Void48/ame.readme.improvements
Also here's the static version if you want to change the GitHub Social preview banner too

closing because we kinda already did this


[Event Request] table 5406 "Prod. Order Line" OnShowDimensionsOnAfterEditDimensionSet

Describe the request
Can you add event OnShowDimensionsOnAfterEditDimensionSet
    procedure ShowDimensions()
    var
        OldDimSetID: Integer;
    begin
        OldDimSetID := "Dimension Set ID";
        "Dimension Set ID" :=
          DimMgt.EditDimensionSet(
            Rec, "Dimension Set ID", StrSubstNo('%1 %2 %3', Status, "Prod. Order No.", "Line No."),
            "Shortcut Dimension 1 Code", "Shortcut Dimension 2 Code");
// >>>>>>>>>>>>>>>>
OnShowDimensionsOnAfterEditDimensionSet(Rec, OldDimSetID); // Rec as reference
// <<<<<<<<<<<<<<<<
        if OldDimSetID <> "Dimension Set ID" then begin
            Modify();
            if ProdOrderCompExist() then
                UpdateProdOrderCompDim("Dimension Set ID", OldDimSetID);
        end;
    end;

Additional context
To be able to check and update other dimensions.
Internal work item: AB#537651

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 21842.

Availability update: We will publish a fix for this issue in the next update for release 24.
Build ID to track: 21929.


add create member modes in driver

@<reviewer_id>
What issues does this address?
Fixes #
WIP #
...
What's this change do?
Where should the reviewer start?
Any background context?

Can we also set this argument in delete_bulk()?

Yes, you are right. Done in the delete_bulk while we still need to implement the batch delete in agent later ...

@jikui Please cherry-pick this change to h3c-pike and pike-rj branch if possible


Added a cool installer uninstaller for windows, linux and maxos

Hi, I have added few scripts to install/uninstall every thing in a completely automated manner

Thanks a lot @ParisNeo for providing the installers.
I will test them asap.

I have generated the LLAMA image using Stable diffusion then cnverted it to ASCII art.
I think it is a good idea to have such artworks in the installer :).
Don't hesitate to test it.
Thank you.

@ParisNeo, The linux installer looks good, I don't have access to others OSs so i didn't test the other installers though.
could you please add a few words to the readme on how to use them, so I can merge the PR.
Thanks!


Unable to connect to a gRPC Server hosted in Kestral as HTTPS from a gRPC C++ Client using default certificates (Windows 10)

What version of gRPC and what language are you using?
gRPC v1.38.1
C++ Client, C#(ASP.Net Core) Server
What operating system (Linux, Windows,...) and version?
Windows 10
What runtime / compiler are you using (e.g. python version or version of gcc)
Visual Studio 2019 C++ Compiler
What did you do?
I am using ASP.Net Core (Grpc.Net) for creating a HTTPS gRPC server hosted in Kestral. The communication between C# Client to the server (HTTPS) works fine with out adding any certificate.
It looks like they are using the default certificates for communication.
Now I have a C++ gRPC Client in Windows 10 and trying to connect to same server from client, the endpoint is https://localhost:50051.
This is my Kestral configuration @server.
webBuilder.ConfigureKestrel(serverOptions =>
                    {
                        serverOptions.Listen(IPAddress.Any, 50051, listenOptions =>
                       {
                           listenOptions.Protocols = Microsoft.AspNetCore.Server.Kestrel.Core.HttpProtocols.Http2;
                           listenOptions.UseHttps();

                       });
                    }).UseStartup<Startup>();

As you could see I am not using certificates in the server (wanted to use default certificates) the same way I used C# gRPC Client.
i.e. Connection to server works using C# Client
var channel = GrpcChannel.ForAddress("https://localhost:50051");
ecgDataClient = new Data.DataClient(channel);

But when I tried to connect using a C++ gRPC Client it fails.
What did you expect to see?
Able to connect to Kestral gRPC Server using default certificates.
What did you see instead?
But with C++ gRPC Client Client I am unable to connect (tried both InSecure & SslCredentials),
auto channel_creds = grpc::SslCredentials(grpc::SslCredentialsOptions());
DataGrpcClient grpcClient( grpc::CreateChannel("localhost:50051", channel_creds));

With grpc::SslCredentials(grpc::SslCredentialsOptions()) it gives below error,
E0709 19:46:20.488000000  6724 ssl_utils.cc:570] load_file: {"created":"@1625840180.488000000","description":"Failed to load file","file":"D:\DEV\vcpkg\buildtrees\grpc\src\17cc203898-db2679e7f2.clean\src\core\lib\iomgr\load_file.cc","file_line":72,"filename":"/usr/share/grpc/roots.pem","referenced_errors":[{"created":"@1625840180.488000000","description":"No such file or directory","errno":2,"file":"D:\DEV\vcpkg\buildtrees\grpc\src\17cc203898-db2679e7f2.clean\src\core\lib\iomgr\load_file.cc","file_line":45,"os_error":"No such file or directory","syscall":"fopen"}]}
E0709 19:46:20.509000000  6724 ssl_security_connector.cc:413] Could not get default pem root certs.
E0709 19:46:20.512000000  6724 secure_channel_create.cc:108] Failed to create secure subchannel for secure name 'localhost:50051'
E0709 19:46:20.517000000  6724 secure_channel_create.cc:50] Failed to create channel args during subchannel creation.
E0709 19:46:20.521000000  6724 ssl_security_connector.cc:413] Could not get default pem root certs.
E0709 19:46:20.525000000  6724 secure_channel_create.cc:108] Failed to create secure subchannel for secure name 'localhost:50051'
E0709 19:46:20.529000000  6724 secure_channel_create.cc:50] Failed to create channel args during subchannel creation.

Anything else we should know about your project / environment?
I am trying localhost in Windows 10.

Yeah, in gRPC C++, the default location will be pointed to a place defaults to POSIX systems, meaning it doesn't automatically work on the Windows Systems. To solve that, you will need to manually load the trust bundles from that Windows location, and specify them through SslCredentialsOptions.
The reason why it works for ASP.Net is probably because it does some customizations and just specify the default Windows location on top of the core. It makes sense since I guess most of .Net users will use gRPC on Windows systems? It might be helpful to have some folks working on .Net to confirm that(@markdroth do you know any of them)?
You might also find the following useful:
[1] https://github.com/grpc/grpc/issues/26106
[2] https://github.com/grpc/grpc.io/pull/763

@msbasanth did the suggestion solve your problem?

For anyone caught on this, someone added a fix for this with full code:
https://stackoverflow.com/questions/71580681/ssl-grpc-client-works-fine-in-c-but-fails-with-unavailable-empty-update-in-c


HTTPS Memory and CPU Issue

I discovered a bug with memory and CPU usage when Node-RED is using HTTPS.
Here is the original thread:
https://groups.google.com/forum/#!msg/node-red/q6P6fTgYWR0/i7cZHa2EVWAJ
Environment:
ARMv5
v0.10.38 or v0.8.28 of NodeJS
Node-RED v0.10.4
Linux kernel 3.12.27
Here is the flow file:
[{"id":"f37069bc.0c8f98","type":"debug","name":"debug-output","active":true,"console":"true","complete":"true","x":760.2000122070312,"y":222.00001525878906,"z":"4e726af0.b18d94","wires":[]},{"id":"ba391381.45c6f","type":"http in","name":"","url":"/test","method":"get","x":304.2000274658203,"y":259.0000305175781,"z":"4e726af0.b18d94","wires":[["f37069bc.0c8f98"]]}]
Here is the curl command which causes the memory leak/CPU bug:
curl -k https://<IP_ADDRESS>:1880/test?bar=foo
Once curl is run, the memory on the system is rapidly consumed and the processor is maxed out.  Eventually, Node-RED crashes if it is given less than 128MB of memory to start with.  Otherwise, it can run for tens of minutes with high CPU and memory usage with >128MB memory.

Can you give some details as to how you have HTTPS configured?
We rely entirely on the underlying node.js libraries for this - there isn't any special code in node-red that would be causing this.

I generated a key for Node-RED similar to this:
openssl req -new -subj "/CN=example.com/O=Example Company Name/C=US" -newkey rsa:2048 -days 3650 -nodes -x509 -keyout key -out cert
Here is close to what we use for the settings.js file:
var fs = require("fs");
adminAuth: require("/opt/node-red/user-authentication.js");
module.exports = {
adminAuth: require("/opt/node-red/user-authentication.js"),
https: {key: fs.readFileSync("/opt/key"), cert: fs.readFileSync("/opt/cert")},
flowFile: "/var/config/app/install/development/flows.json",
uiPort: 1880,
userDir: "/var/config/app/install/development",
mqttReconnectTime: 15000,
serialReconnectTime: 15000,
debugMaxLength: 1000,
functionGlobalContext: {},
logging: {console: {level: "info",metrics: false}}}

Once I removed the following from the settings file, the issue goes away:
https: {key: fs.readFileSync("/opt/key"), cert: fs.readFileSync("/opt/cert")}

Need to see if this is still the case on more recent versions of node

I have the same problem on the 0.18.4: Maintenance Release


Reworked SPI class with direct register access to fix #28 for R_SPI

The discussion for issue #28 has a plethora of information what this is.
In essence, a buffer of 231 bytes with a SPI clock of 8MHz is transferred in 252µs with this new version of SPI.cpp while the FSP based class needs 652µs to do the same, so this new version needs 21µs overhead while the FSP based class needs 421µs overhead.
For single byte transfers this new version has a delay between two bytes of 1.22µs when using 8MHz while the FSP based class has a delay of 10.9µs.
Another bug that I just discovered and that is fixed with this new version of SPI.cpp is that the FSP based version needs 652µs to transfer a buffer of 231 bytes with 6, 8, 12 and 24 MHz, somehow changing the SPI clock works but the time needed overall is the same.

Hi @RudolphRiedel :coffee: :wave:
Thank you for this PR. Let's start the work of getting it merged in ...

I was not aware that this is for the Portenta C33 as well which uses a RA6M5 and I do not have one to test this.
I just compared the datasheets though and I am quite positive that this works for the Portenta C33.

I was not aware that this is for the Portenta C33 as well which uses a RA6M5 and I do not have one to test this.

I'll handle the tests for C33 :+1:

And I just found a "goto" in the code I did not touch.
Well, checking the repository, I guess one more will not hurt - but are there really no coding guidelines for Arduino?

Testing was performed at 1MHz, 10MHz and 24MHz SPI clock rate using BMP388-Basic from 107-Arduino-BMP388. The example exercises SPI both from interrupt and normal execution context.

Uno R4 Minima: SPI :heavy_check_mark:

I tested with the Minima as this is the one I could get.
I checked the schematics and the user manual, so the Minima is using R_SPI1 and the WiFi is using R_SPI0.
It compiles fine for the WiFi variant.
As a hardware engineer I am inclined to add more comments, but this would be off-topic.

Well, I've put in the request for the Uno WiFi R4. I'll test once it arrives on my table and then we can proceed merging this PR.

Just tested the changes on a R4 WiFi board:

MPU-9250 on SPI with CS pin 10 and worked no issues
@greiman SDFat Bench sketch:
a.  Failed if I used pin 10 but worked once using pin 4 but then would not run again
b. switched to pin 5 and it ran:

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
601.32,4294967133,836,842
603.50,4294967133,836,840

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
583.98,4294967162,865,868
584.11,4294967162,865,868

Hit reset and it ran again no issues.  Cycled power and would not run sketch but after hitting reset it ran the sketch no issue?

@mjs513
Here are the only place CS is accessed in SdFat.  This wrapper is to accommodate users with port expanders.
#if SD_CHIP_SELECT_MODE == 0
//------------------------------------------------------------------------------
void sdCsInit(SdCsPin_t pin) { pinMode(pin, OUTPUT); }
//------------------------------------------------------------------------------
void sdCsWrite(SdCsPin_t pin, bool level) { digitalWrite(pin, level); }
#elif SD_CHIP_SELECT_MODE == 1
//------------------------------------------------------------------------------
__attribute__((weak)) void sdCsInit(SdCsPin_t pin) { pinMode(pin, OUTPUT); }
//------------------------------------------------------------------------------
__attribute__((weak)) void sdCsWrite(SdCsPin_t pin, bool level) {
  digitalWrite(pin, level);
}
#endif  // SD_CHIP_SELECT_MODE == 0

The default value for SD_CHIP_SELECT_MODE  is zero in SdFatConfig.h
/**
 * SD_CHIP_SELECT_MODE defines how the functions
 * void sdCsInit(SdCsPin_t pin) {pinMode(pin, OUTPUT);}
 * and
 * void sdCsWrite(SdCsPin_t pin, bool level) {digitalWrite(pin, level);}
 * are defined.
 *
 * 0 - Internal definition is a strong symbol and can't be replaced.
 *
 * 1 - Internal definition is a weak symbol and can be replaced.
 *
 * 2 - No internal definition and must be defined in the application.
 */
#ifndef SD_CHIP_SELECT_MODE
#define SD_CHIP_SELECT_MODE 0
#endif  // SD_CHIP_SELECT_MODE
/** Type for card chip select pin. */
typedef uint8_t SdCsPin_t;

I have been testing on a R4 WiFi but only with CS == 10.  I have been using a Adafruit SD shield with level shifters on SPI pins since R4 boards are 5V.  I don't know what 5V means in Arduino speak.  The Adafruit shield probably uses CS to enable level shifters.
What is your setup.

@mjs513
I believe the MPU-9250 has some configuration jumpers for power level.  A set of R4 GPIO ports might be affected if this is not correct.  The GPIO on R4 is more current limited than AVR.

@mjs513
One more question.  Have you edited SdFatConfig.h?  It appears you are using the default SPI settings since you get about 600 KB/sec.
The default settings only uses the uint8_t  transfer(uint8_t data).

Btw., I am testing with my library for EVE chips: https://github.com/RudolphRiedel/FT800-FT813
The UNO R4 configuration and a couple of inline functions is here:
https://github.com/RudolphRiedel/FT800-FT813/blob/5.x/EVE_target/EVE_target_Arduino_UNO_R4.h
EVE_start_dma_transfer() is currently still using a blocking SPI.transfer(buf, count).
It uses digitalWrite() and EVE_CS is wired to 10.
I am using a 5" TFT with 800x480 and BT817 chip right now and have it connected using one of these boards:
https://github.com/RudolphRiedel/EVE_display-adapter/tree/master/L-D5019-08
So this is with a 3.3V target board and a level-shifter.
My application does three simple transfers of 6, 6 and 5 bytes to read two bytes and 1 byte every 5ms.
The buffer is transferred every 20ms.

@mjs513
I looked at more MPU-9250 boards.  They often have a 1kΩ resistor on CS for 5V operation.
Do you have problems if the MPU-9250 is not connected?

@greiman
In answer to your questions
I ran your bench sketch (default settings) using 2 different adafruit boards: https://www.adafruit.com/product/254 (Onboard 5v->3v regulator)
The issue with pin 10 was that I didn't hit the reset button just did a power cycle.  Didn't try reset until I switched the CS pins.
Just now I tried it with the adafruit: https://www.adafruit.com/product/4682.  With this card and using pin 10, on upload I had to hit the reset button to the sketch to run but did not have to hit the reset button or cycle power.  So it could have been the 5v card reader.
The answer to your second question is yes was using the default settings.  Using your array transfer settings in SDConfig.h I get:

Write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1629.73,4294966601,304,303
1613.94,4294966601,304,306

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1711.16,4294966586,289,288
1712.33,4294966585,289,289

@greiman
Sorry Bill think you misunderstood - there were no issues with the MPU-9250 using the updated library.  Ran out of the box after I uploaded the sketch.
Was late so probably did not make myself clear on that one.

MPU-9250 using the updated library. Ran out of the box after I uploaded the sketch.

The MPU-9250 may run but unless it's board is rated for 5V it may drag down voltage on other pins.
The Adafruit module for 5V is good.  Lot of people use it at 5V.  Please disconnect the MPU-9250 and try the SD for CS pins pins 4 and 5.
Don't use the 3V only breakout socket.  It's a nice product and I use it on 3.3V ARM since it has card detect.
The R4 boards are powered by 5V to be Uno R3 compatible.

The MPU-9250 may run but unless it's board is rated for 5V it may drag down voltage on other pins.

Version I am using is rated for 5v with an on-board level shifters and testing was done stand alone, i.e., only the MPU-9250 or the the SD Card reader attached.

The Adafruit module for 5V is good. Lot of people use it at 5V. Please disconnect the MPU-9250 and try the SD for CS pins pins 4 and 5.

Ok retested with pins 4, 5 and 10 but this time I changed your delay(1000) to delay(5000) in setup (just so I had a bit more time to open up the Serial Monitor in IDE2.1.1.
This time sketch ran and outputted data with having to do anything with reset button, same output for all three pins.  Not sure what happened last night but its seems to working now.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1614.47,4294966601,304,306
1622.06,4294966601,304,305

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1711.74,4294966586,289,289
1712.33,4294966586,289,289

@mjs513
Good to see it works for you.  I was just testing on R4 WiFi with the AdaFruit module you are using.  Guess I don't need to tell you it works for me.

I changed your delay(1000) to delay(5000) in setup.

Yes the while (!Serial) {} doesn't stop in 2.1.1.  The initial message gets lost.

Good to see it works for you. I was just testing on R4 WiFi with the AdaFruit module you are using. Guess I don't need to tell you it works for me.

Not sure what the issue was last night. The only difference between last night and this morning was a different breadboard.  But glad its working.  Hopefully will get incorporated soonish.

@mjs513
Your R4 WiFi now is faster than a $113.90 Portenta H7 with array transfer:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1346.98,402,374,378
1346.62,403,375,378
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1363.88,381,368,374
1364.26,381,368,373

With single byte transfer the Portenta H7 is about half as fast as a Uno R3.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
352.51,1476,1443,1451
352.46,1474,1443,1451
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
351.72,1468,1446,1454
351.17,1469,1449,1456

Your R4 WiFi now is faster than a $113.90 Portenta H7 with array transfer:

Well, that runs on ArduinoCore-mbed which uses libmbed.a for which I can not find the specific source code.
And SPI.cpp is only a wrapper for the MBED SPI class.
However, ArduinoCore-mbed includes the STM32 cube library:
cores\arduino\mbed\targets\TARGET_STM\TARGET_STM32H7\STM32Cube_FW\STM32H7xx_HAL_Driver
stm32h7xx_ll_spi.h
stm32h7xx_hal_spi.h
So it is possible to write a driver around the SPI class without even using registers and for the Nucleo F446RE I already did that - and it uses DMA.

I just tried it to see what it would do now.  At first H7 STM32Cube SPI would not run over a few MHz if at all.
Arduino on top of mbed with STM32Cube is more than I can put up with.  mbed depends on ST to setup STM32Cube . so you must deal with three big companies.  Also Portenta H7 has lots of other problems.
mbed is an RTOS I really dislike so I probably won't fight with SPI.
I would really want SDIO on STMH7 but the STM32 SDIO controller is too archaic.
I wrote a SDIO driver for teensy 3.x and 4.x.  It does over 22 MB/sec SD read/write.  That's about the limit since Teensy only supports 50 MHz SDIO.

@greiman
just incorporated the micros fix in the latest PR and reran you bench - just as a fyi
Serial number: 0X882B3DA3
Manufacturing date: 7/2017

FILE_SIZE_MB = 10
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1664.72,30693,299,303
1664.72,30681,299,303

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1769.91,288,284,285
1770.22,288,284,285

@mjs513
The write transfer max latency is interesting, over 30 ms.  What card are you using?
I need to investigate.
Could be the card since the spec allows cards to have occasional long write latency.  Most cards have shorter max latency since they multi-task erase and wear leveling.
Could be a problem still in micros().
I suspect SPI is not the problem.  Could be another device like USB.

@greiman

What card are you using?
Using a SanDisk Ultra 32GB has HC I marking on it.

Guess it is the card.  I tried several Ultra 32GB cards on an Arduino Due and got longer write latency.  Most were 15-20 ms but in one test I got 102ms.
My memory was that Ultra cards had shorter max latency.  Guess I was wrong.
Here is a 32GB Samsung Pro on a Due board.  This is outstanding.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4537.21,128,110,111
4537.21,127,110,111
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4516.71,113,110,111
4528.99,113,110,111

Here is a older 32GB Samsung Evo, a lower end card.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4524.89,3653,110,111
4524.89,3667,110,111

The R4MA1 processor is not bad at $3.2 in volume on DigiKey.  As for the software, too bad about FSP.
I looked at the RA6M5 in the C33 board and decided to order one.  It has a nice SD/MMC Host Interface (SDHI).  too bad the pins are not more accessible.
The SPI controller has more buffering and probably goes faster than 24 MHz.  I will play with mods to use this structure.

Good Morning @RudolphRiedel and @greiman :coffee: :wave:
I see that you've been enjoying yourselves :wink: As for my part I've not yet received yet my Uno R4 WiFi. I'll try and get someone else to test this PR against Uno R4 WiFi.

Please consider activating the Github discussions feature for at least this repository.

Please consider activating the Github discussions feature for at least this repository.

Sorry, that's not my decision to make. Please consider that :octocat: discussions are open both for arduino/language and arduino/ArduinoCore-API.

Discussions opened

Here is a discussion about the SPI API I opened in May.

Sorry, that's not my decision to make. Please consider that :octocat: discussions are open both for arduino/language and arduino/ArduinoCore-API.

These two topics are not really not appropriate for this pull request.   @RudolphRiedel didn't extend the SPI API.
What is wrong with this pull request?
Is the issue the Arduino policy for Renesas FSP?   Is the way this implementation interacts with FSP unacceptable?
The AVR Core only depends on vendor software at the lowest level. It is amazing how many new boards that depend on vendor software have lower SPI performance than Uno R3.
Single byte SPI transfer on Portenta H7 is about half as fast as Uno R3.

These two topics are not really not appropriate for this pull request. @RudolphRiedel didn't extend the SPI API.

These two repositories are highly relevant for discussing extension of the Arduino API 🤷. At any rate @facchinm - who has in fact decision making power to open GitHub discussions for this repository - has already decided to do so.

Is the issue the Arduino policy for Renesas FSP? Is the way this implementation interacts with FSP unacceptable?

There's no issue with this pull request. As soon as I have a greenlight from @Rocketct or complete testing with my own R4 WiFi (delivery pending) I'll merge this PR.

These two repositories are highly relevant for discussing extension of the Arduino API 🤷. At any rate @facchinm - who has in fact decision making power to open GitHub discussions for this repository - has already decided to do so.

You mean I may actually get a response to my post of May 16?

You mean I may actually get a response to my post of May 16?

I hope you will :+1: . Please understand that there are many competing priorities (you would not believe the amount of Emails I and our team get from GitHub alone each day) and although I am inclined to help you I have no decision making role. I can understand your frustration but you've got to understand that my role is limited to pointing out the previous designated areas for discussions concerning API extensions :shrug: .
You may continue venting your frustration at the post you linked but please limit discussion in this PR thread to the immediate content of this PR :pray: :bow: .

Note: About this pull request.
I suspected not being pro Renesas FSP was a bad thing.  I had a friend check Arduino with DNB.  I bet tossing FSP is not an option.

June 2022, Arduino raised $32.0M in Series B funding led by Robert Bosch Venture Capital, joined by Renesas, Anzu Partners, and Arm.
As part of the round, Renesas is investing $10 million. This investment is part of Renesas’ strategy to grow in the mass market and reach new customers within Arduino’s community of 30 million developers worldwide.

Out of curiosity I tried an adafruit product: https://www.adafruit.com/product/4899.  Basically a NAND chip that acts like an SD card with @greiman bench sketch - it worked using a wifi board:
FreeStack: 27756
Type is FAT16
Card size: 0.52 GB (GB = 1E9 bytes)

Manufacturer ID: 0XB
OEM ID: XT
Product: XTSDA
Revision: 1.1
Serial number: 0X4FAE0070
Manufacturing date: 9/2020

FILE_SIZE_MB = 10
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1511.49,95620,299,334
1522.07,95025,299,332

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1769.29,292,284,285
1769.91,291,284,285

Tested this on my UNO R4 Wifi @ 1Mhz 12Mhz and 24Mhz SPI clock using BMP388-Basic without interrupt. Works fine.
I've also made a spin on my Portenta C33 and find out the clock is capped @ 2Mhz independently from the configuration.
I will doubleckeck with @aentinger to understand how and if he can get 24Mhz out of his C33.

@pennam
It appears that the Portenta C33 SPI on the board pins are Serial Communications Interface (SCI). SCI provides "simple SPI".
See the pinout for SPI on D7, D8, D9, D10.
At first glance I don't see the max rate for SCI Simple SPI.
This mod is only for this interface.

The Serial Peripheral Interface (SPI) has 2 channels. The SPI provides high-speed full-duplex synchronous serial
communications with multiple processors and peripheral devices.

Looks like the Portenta Breakout has an SPI port on one of the full featured ports.  I have a C33 but no breakout so I can't test it.

I've also made a spin on my Portenta C33 and find out the clock is capped @ 2MHz independently from the configuration.

SCI SPI (7-10) is capping at around 12.5 MHz for me. There is definitely some limitation, but its not 2 MHz.

Testing was performed at 1MHz and 10MHz SPI clock rate using BMP388-Basic from 107-Arduino-BMP388. The example exercises SPI both from interrupt and normal execution context.
Uno R4 WiFi: SPI :heavy_check_mark: .
Portenta C33: SPI :heavy_check_mark:, SPI1 :heavy_check_mark: .
My key take-away is that the FSP code that calculates the bitrate settings warrants a closer look.


Merged DodgyCoin with Kebab

Here's the fork I use in DodgyCoin. I added typings for the Wrapper function and UserList (which need to be checked).

Okay I am actually stupid, I did not check recent kebab commits, just mindlessly merged everything. I will close this for now while I fix the problems.

you could mark it as draft

Will add


I broke Brackets and need some assistance.

While trying to open a new project folder, I accidentally opened an entire shared server at work. I'm sure I've maxed out the program memory, and nothing in the Brackets window will respond. I've tried opening a new Brackets window, but it just comes up blank. I've tried opening the folder again, but it doesn't seem to work. Is there anything I can do other than deleting and re-installing Brackets?
Thanks.

After waiting half an hour after opening Brackets to the public drive, an error message appeared and Brackets closed the directory itself. Thanks anyway!

In case you need it in the future, Resetting cache & preferences:
https://github.com/adobe/brackets/wiki/cache-folder#resetting-cache--preferences


Tests using the scan partitioner are broken on at least gcc 4.7 and intel compiler

See buildbot for more detail:
http://hermione.cct.lsu.edu/builders/hpx_gcc_4_7_boost_1_51_debian_x86_64_release/builds/301/steps/run_unit_tests/logs/stdio
and
http://hermione.cct.lsu.edu/builders/hpx_intel_13_boost_1_54_debian_x86_64_release/builds/300/steps/run_unit_tests/logs/stdio

This was fixed by dc37df32d2faac3665f087d6cc06f8d11561cac8


Add PrivExchange

A function which sends a push notification to the Exchange

This is great work @panagioto! Thanks for the valuable contribution.
Apologies for the delay in merging this. I've made a couple small changes to rename and standardize some comments, function names, and parameter names.
Could you verify that these functions still work as you intended? In particular, I don't have an environment available to me at the moment to test the "PrivExchange" attack.
If these function still work as intended and you are okay with the changes, then I'll go ahead and merge!

@cobbr No worries at all. I just tested the changes and seem to work as expected.

Thanks @panagioto this looks great!


WIP: Fixed android regressions, updated mym-utils modules (release after 2.0.2 mymutils published)

Merged into #91


Bitcode rewrite

I rewrote the entire library. docs
It's been tested/fuzzed but still needs some work before release.
# If you want to try it early
bitcode = "=0.6.0-alpha.2"

New features:

much faster
very compressible
doesn't require any hints (determines them at runtime)
deserialize &str

Alpha release:

[x] big endian platforms (compile error right now)
[x] signed integer size reduction (currently treated as unsigned)
[x] usize
[x] Result

Full release:

[ ] handle derive macro errors instead of panicking
[ ] remove lifetime bound on DecodeBuffer
[ ] recursive types
[ ] #[bitcode(with_serde)] (can only do bitcode::serialize right now)
[ ] #![forbid(unsafe_code)] feature flag (serde only and slightly slower)
[ ] CString
[ ] IpAddr
[ ] deserialize &[u8]

Benchmarks for those who are interested. Previous version of bitcode isn't shown here but it has speed similar to bincode, size simliar to new bitcode, and compressed size 20% worse than bincode.

Format
Compression
Size (bytes)
Serialize (ns)
Deserialize (ns)

bincode

49.1
35
115

bincode
lz4
16.1
86
115

bincode
deflate-fast
13.1
166
176

bincode
deflate-best
8.9
3708
141

bincode
zstd-0
12.4
172
146

bincode
zstd-22
8.5
32312
133

bincode-varint

22.3
36
116

bincode-varint
lz4
10.8
71
119

bincode-varint
deflate-fast
10.1
146
165

bincode-varint
deflate-best
8.0
2664
153

bincode-varint
zstd-0
8.2
123
131

bincode-varint
zstd-22
7.8
23939
136

bitcode

16.9
29
104

bitcode
lz4
9.8
42
108

bitcode
deflate-fast
8.3
86
137

bitcode
deflate-best
6.8
1661
125

bitcode
zstd-0
7.1
58
115

bitcode
zstd-22
6.2
24713
115

bitcode-derive

16.9
10
12

bitcode-derive
lz4
9.7
25
17

bitcode-derive
deflate-fast
8.3
68
45

bitcode-derive
deflate-best
6.8
1597
33

bitcode-derive
zstd-0
7.1
40
24

bitcode-derive
zstd-22
6.2
24905
25

doesn't require any hints (determines them at runtime)

This is remarkable, is there any particular part of the branch you could point me to so I could see how it's done?
Side question, do you see any benefits in hinting on top of runtime determined?
Hope it's okay to ask the questions in your PR :) thanks for making this library

doesn't require any hints (determines them at runtime)

This is remarkable, is there any particular part of the branch you could point me to so I could see how it's done?

https://github.com/SoftbearStudios/bitcode/blob/5bdc22ba943d0ba8de092a763327b8167656611f/src/pack.rs
https://github.com/SoftbearStudios/bitcode/blob/5bdc22ba943d0ba8de092a763327b8167656611f/src/pack_ints.rs
https://github.com/SoftbearStudios/bitcode/blob/5bdc22ba943d0ba8de092a763327b8167656611f/src/f32.rs (albiet this one requires the aid of a compression algorithm like deflate/lz4/zstd to do anything)

Side question, do you see any benefits in hinting on top of runtime determined?

After adding hints to bitcode, I didn't use them as much as I thought I would because it was tedious. The types of "packing" I'm using in this new version are designed to quickly determine if they're applicable and pack the data. I'm probably not going to add manual hints back because most people don't benefit from them (me included). Also this new version prioritizes working with general purpose compression which some of the old hints got in the way of (e.g. #[bitcode_hint(ascii)] made characters 7 bits which confused byte-wise compression algorithms).

Hope it's okay to ask the questions in your PR :) thanks for making this library

Yeah! I made this PR public before being finished to see what people think about it.

Very impressive!!!

I just released bitcode = "=0.6.0-alpha.1" which has the core features done.

Good work! I read through the code and have a few thoughts:

Looks like enum with more than 256 variants is not supported, will it be supported in full release?
This definitely is a huge improvement over 0.5 for large messages with a lot of repeated patterns. But I'm guessing it won't do as well as 0.5 for small messages. It would be nice to measure how it compares against 0.5 in terms of speed and size for messages that have little long vec's but might have deep structures.
If it's worse against 0.5 for this use case, would it make sense to keep 0.5 around and release this in a new crate?
Would it make sense to optimize the case where len of a vec type is always 0? If an vec type always have len of 0, don't bother store the elements. Same thing goes for enum variant (didn't understand the code enough to know whether it already does). This is beneficial for deprecated data.
I'm personally interested in this crate but want it to support schema evolution. Opened a ticket: [#21].
I have a hard time imaging how recursive data can be supported. If it's hard (in terms of effort or ser/de speed), I think you might not want to support it at all.

Looks like enum with more than 256 variants is not supported, will it be supported in full release?

Not unless multiple people complain about it and even then it would probably have sub optimal performance. I've been avoiding it because calculating a histogram of an unbounded number of elements would require a HashMap.

This definitely is a huge improvement over 0.5 for large messages with a lot of repeated patterns. But I'm guessing it won't do as well as 0.5 for small messages. It would be nice to measure how it compares against 0.5 in terms of speed and size for messages that have little long vec's but might have deep structures.

You are right that speed is worse on small messages, however I don't see this as a problem since small messages are probably not a bottleneck anyways. The main overhead is allocating a Vec for each field, but you can keep an EncodeBuffer around to reuse allocations. On the size front, small messages might be bigger uncompressed, but smaller if you have cross message compression since 0.5 doesn't compress well.

If it's worse against 0.5 for this use case, would it make sense to keep 0.5 around and release this in a new crate?

I thought about this, but I don't want to maintain two crates. Someone can always fork 0.5 and maintain it if they want.

Would it make sense to optimize the case where len of a vec type is always 0? If an vec type always have len of 0, don't bother store the elements. Same thing goes for enum variant (didn't understand the code enough to know whether it already does). This is beneficial for deprecated data.

I considered this (including integers that are always 0), but it allows for a zip bomb. Currently deserialization memory usage is proportional to input size.

I'm personally interested in this crate but want it to support schema evolution. Opened a ticket: [FR: allow schema evolution (in or after 0.6) #21].

See my comment.

I have a hard time imaging how recursive data can be supported. If it's hard (in terms of effort or ser/de speed), I think you might not want to support it at all.

One way would be to have a max recursion depth N and have N nested encoders.

I think enum with more 256 variants can be easily supported with a compile time IF branch
For 0.5 vs 0.6  on small messages, is there a benchmark to measure this?
For vec with only zero length, I think zip bomb can still be avoided. I'll give an example:

struct A {
  ... // a lot of nested structures
}
struct B(Vec<Inner>);
fn serialize(b: &B) {...}

If I understand correctly, if b.0.len() is always 0, we would still have a lot of data storing A's definition, which is unnessesary. We still store b.0.len() as 0, which takes some space, so zip bomb is not possible.
4. IMO, zip bomb should not be a consideration on encoding spec, but on implementation. Interesting read on how capnproto handles zip bomb: https://capnproto.org/encoding.html#amplification-attack

I think enum with more 256 variants can be easily supported with a compile time IF branch

Yes, certainly for derive (although this would require some code duplication). The tricky part is serde, where you don't know the total number of variants ahead of time (I suppose it could always encode them as u32 or something but it's not elegant).

For 0.5 vs 0.6  on small messages, is there a benchmark to measure this?

No, because they're both called bitcode and they don't like to be imported at the same time. My comments about compressed size of small messages are based on us testing both version on our projects and eyeballing the results. Feel free to do your own benchmarks and share the results here!

For vec with only zero length, I think zip bomb can still be avoided. I'll give an example:

struct A {
  ... // a lot of nested structures
}
struct B(Vec<Inner>);
fn serialize(b: &B) {...}

If I understand correctly, if b.0.len() is always 0, we would still have a lot of data storing A's definition, which is unnessesary. We still store b.0.len() as 0, which takes some space, so zip bomb is not possible.

Consider Vec<Vec<u8>> where all the inner Vecs are 0 length.

IMO, zip bomb should not be a consideration on encoding spec, but on implementation. Interesting read on how capnproto handles zip bomb: https://capnproto.org/encoding.html#amplification-attack

Yeah this isn't a bad idea, but I didn't really want to worry about this for the initial version. We should revisit this once 0.6 is released.

Consider Vec<Vec> where all the inner Vecs are 0 length.

Let's use Vec<Vec<A>> as example.
This still isn't a problem, say it's [[],[],[]]. We store 3, 0, 0, 0, but we shouldn't store anything about A. Maybe I'm misunderstanding it and it's the current behavior?

Consider Vec where all the inner Vecs are 0 length.

Let's use Vec<Vec<A>> as example. This still isn't a problem, say it's [[],[],[]]. We store 3, 0, 0, 0, but we shouldn't store anything about A. Maybe I'm misunderstanding it and it's the current behavior?

Oh, nothing is currently stored for A even though the encoders exist. I thought you meant the 0s wouldn't get stored because all the Vecs are empty.

I'm curious how you're testing auto vectorisation. Are you manually enabling cpu features with rustflags, or using something like target native?

I'm curious how you're testing auto vectorisation. Are you manually enabling cpu features with rustflags, or using something like target native?

Just using target-cpu=native .cargo/config.toml. x86_64 always has SSE2, so even without target-cpu=native there is still some autovectorization.
The autovectorization mainly improves encode_vectored by making it require about half the mov instructions. LLVM can replace scalar load+store with scalar load+vectored store since encode_vectored chunks 64 of each field a time. Loads are scalar because As aren't adjacent in memory when encoding Vec<(A, B)>, but output looks like [A, A, A, ...] so vectored store can be used.

the alpha version on benchmarks looks nuts :)

Thanks :)

I also plan to utilize autovectorization in decode with something like decode_vectored_in_place but it's currently limited by the inability to decode enums in place with MaybeUninit. Hopefully Rust will add a way to initialize the variant of an uninitialized enum.

I'm interested in hearing opinions about making bitcode::encode/bitcode::decode use a thread local bitcode::Buffer.
This is how we internally use bitcode so I'm considering upstreaming it.
thread_local! {
    pub static BUFFER: std::cell::RefCell<Buffer> = Default::default();
}
pub fn encode<T: Encode + ?Sized>(t: &T) -> Vec<u8> {
    BUFFER.with(|b| b.borrow_mut().encode(t).to_vec())
}
pub fn decode<'a, T: Decode<'a> + ?Sized>(bytes: &'a [u8]) -> Result<T, Error> {
    BUFFER.with(|b| b.borrow_mut().decode(bytes))
}

Pros:

Small messages encode/decode 2x faster
Large messages encode 20% faster and decode 10% faster

Cons:

Keeps large amounts of memory when encoding/decoding a single large message
Unexpected behavior

You could always opt out by creating a new buffer for each encode/decode call.

@caibear https://docs.rs/bitcode/0.5.0/bitcode/struct.Buffer.html#method.deserialize
So buffer.deserialize() is now removed? I'm guessing if I want to deserialize with reusable buffer, I have to use encode/decode?

@caibear https://docs.rs/bitcode/0.5.0/bitcode/struct.Buffer.html#method.deserialize So buffer.deserialize() is now removed? I'm guessing if I want to deserialize with reusable buffer, I have to use encode/decode?

Yes this was removed. Currently you have to use Encode/Decode if you want to reuse allocations.
Buffer::{serialize, deserialize} might be reimplemented in a future version, but doing so is non-trivial.
Note: Saving allocations is an optimization that's usually 10% faster on large messages and 50% faster on small messages.
If you care about speed, you probably want to use Encode/Decode anyway because they're usually 2-5x faster.

@caibear Ahh I see. I'm using hecs(ecs) and other libs and would need to derive Encode/Decode on them as well so I'll stick with 0.5.0 in the meantime.

@caibear Ahh I see. I'm using hecs(ecs) and other libs and would need to derive Encode/Decode on them as well so I'll stick with 0.5.0 in the meantime.

Have you benchmarked 0.6 with allocations against 0.5 without allocations for your usecase? 0.6 with allocations might be faster if your messages are large enough.

Wow, 0.6 is really that better huh? I have not benchmarked just yet. What do you mean by "large enough" in terms of size? My game sends position updates 30 times a sec and the average size is ~3-5kilobytes

@caibear Wow, 0.6 is really that better huh? I have not benchmarked just yet. What do you mean by "large enough" in terms of size? My game sends position updates 30 times a sec and the average size is ~3-5kilobytes

0.6 is generally faster/smaller than 0.5 across all benchmarks. The question here is if the additional allocations outweigh the gain in speed. I just benchmarked deserializing 5kb of messages  and 0.6 is 30% faster. I don't know your exact structs, but this should be a good baseline.


Update fuuid, fuuid-doobie to 0.6.1

Updates

io.chrisdavenport:fuuid
io.chrisdavenport:fuuid-doobie

from 0.2.0 to 0.6.1.
GitHub Release Notes - Changelog - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (0.2.0).
You might want to review and update them manually.
build.sbt

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.chrisdavenport" } ]

labels: library-update, semver-minor, old-version-remains

Superseded by #161.


Pass PIOASM_EXTRA_SOURCE_FILES to Pioasm sub-cmake and add OUTPUT_FORMAT option (fixes #827)

This PR changes FindPioasm.cmake to pass through the PIOASM_EXTRA_SOURCE_FILES variable, so that it is actually able to be changed by users of the SDK without modifying it.
This PR also adds an OUTPUT_FORMAT option to pico_generate_pio_header that sets the output format used so users can also easily use their custom output formats.
Example usage:
file(GLOB pioasm-sources pioasm-extensions/*.cpp)
set(PIOASM_EXTRA_SOURCE_FILES ${pioasm-sources})

pico_generate_pio_header(my-target thing.pio OUTPUT_FORMAT sdk-custom)

sure!


Apple Community Support

Add the ability to give alerts on this forum! https://discussions.apple.com/welcome

Sure.


STM32 single-wire UART not working when poll-out more than 1 char

Describe the bug
When the STM32 single-wire UART sample is modified such that to poll out more than just one byte, no characters are received by the polling in side.
I'm currently testing on a ST Nucleo F103RB using USART1 (sending) and USART3 (receiving).
For reference, I tested the default single-wire sample (1 char sent and received) on the Nucleo F103RB and it works just fine.
To Reproduce
Build the sample using the code for main.c and nucleo_f103rb.overlay in the Additional Context below
Expected behavior
Data sent from USART1 shall be received at USART3
Impact
Showstopper
Environment (please complete the following information):

OS: Linux
Zephyr SDK 0.15.0
Zephyr 3.2.0

Additional context
main.c
#include <zephyr/kernel.h>
#include <zephyr/device.h>
#include <zephyr/devicetree.h>
#include <zephyr/drivers/uart.h>

#define UART_NODE1 DT_ALIAS(uart1)
#define UART_NODE2 DT_ALIAS(uart2)

const struct device *const sl_uart1 = DEVICE_DT_GET(UART_NODE1);
const struct device *const sl_uart2 = DEVICE_DT_GET(UART_NODE2);

void main(void)
{
	unsigned char recv;

	if (!device_is_ready(sl_uart1) || !device_is_ready(sl_uart2)) {
		printk("uart devices not ready\n");
		return;
	}

	while (true) {

		uint8_t i;

		// TX
		//uart_poll_out(sl_uart1, 'c');
		unsigned char tx_buf[4] = { 0x0A, 0x0B, 0x0C, 0x0D };
		for(i=0; i<sizeof(tx_buf); i++) {
			uart_poll_out(sl_uart1, tx_buf[i]);
		}

		/* give the uarts some time to get the data through */
		k_msleep(50);

		int ret = uart_poll_in(sl_uart2, &recv);
		if (ret < 0) {
			printk("Receiving failed. Error: %d\n", ret);
		} else {
			printk("Received %c\n", recv);
		}

		// RX
		/*i=0;
		while(i<4) {
			ret = uart_poll_in(sl_uart2, &recv[i]);
			if( ret < 0 ) {
				k_usleep(10);
				//printk("err\n");
			} else {
				printk(" %02X ", &recv[i]);
				i++;
			}
		}
		printk("\n");*/

		k_sleep(K_MSEC(2000));
	}
}

nucleo_f103rb.overlay
/ {
	aliases {
	   uart1 = &usart1;
	   uart2 = &usart3;
   };
};

&usart1 {
	single-wire;
	pinctrl-0 = <&usart1_tx_pa9>;
};

&usart2 {
	// !!! default ST-LINK / debug port (zephyr console),
	// do not use here
};

&usart3 {
	single-wire;
	pinctrl-0 = <&usart3_tx_pb10>;
	status = "okay";
};

Forgot to mention that I had to add an external pull-up 10k resistor to my  "loopback fixture" (to use the same jargon used in the existing sample docs) for my test to work, according to what stated in the F103 reference manual sec. 27.3.10

I tried your code and I can confirm that no data is received in your case.
I pushed things a bit further to try to understand what is happening.
First thing I saw is that when sending two bytes in a row, an overrun error occurs on both uart. It can be checked by calling the uart_err_check function after poll_out.
In your case, after the second poll_out, the overrun error bits of both uart are raised, and never cleared, so it is stuck in error.
In any case, since we are using polling functions without dma or interrupts, writing 4 bytes in a row necessarily results in lost data, even with your commented code, since the uart does not have access to anything to store the received data.

@gautierg-st I took some time to go through the docs and run more tests.
I was familiar with the mcu RM and knew about the pins being connected together. I assumed the chip logic, when put in single-wire, would prevent local echo of characters sent at register level, which I understand from you it's not the case.
with that out of the way, and since half-duplex is a mandatory requirement, I perfected the polling example I've got to prevent overrun readings all characters sent out. doing so I can poll multiple chars on a command-response pattern no problem.
here's the snippet to do that:
`
// command
int i;
for(i=0; i<N_CMD; i++) {
uart_poll_out(cfg->uart_dev, tx_buf[i]);
while( uart_poll_in(cfg->uart_dev, &buf)) {
}
}
// response
for(i=0; i<N_RSP; i++) {
	while( uart_poll_in(cfg->uart_dev, &data->rd_data[i])) {
	}
}

`


Removes IDisplayedContentItemHandler in favour of ICacheTagProvider (#7792)

See #7792 for more info.

Talked to Chris at length about this and reviewed these changes with him. GTG as far as I'm concerned.


Image is broken

Now it's confirmed, the image is using resctl-demo available in crates.io
So publishing new version in crates.io will solve the broken image problem.
@htejun : I don't seems to have permission to login with organization permission. Could you confirm if you want to do it ?

PR:

wait unit boot partition is recognised and also debugging

https://github.com/iocost-benchmark/resctl-demo-image-recipe/pull/39

Make pivot as default option

https://github.com/iocost-benchmark/resctl-demo-image-recipe/pull/40

duplicate of resctl-demo-image-recipe#


Train VRC on single categories produces bad results

Hi! I have retrained your model on another dataset (derived from ShapeNet) on single categories, (i.e. for each class I train a separate model). I found that VRC has the same performances of ECG for some categories and perform even worser on other categories. There's no an overall improvement as is demonstrated by training on all categories. What do you think could be the reason behind this behaviour. I trained with point cloud resolution set to 2048 and the same settings specified in your config file.

HI! Thanks for your feedback, I am using the dataset proposed in https://arxiv.org/abs/2108.00205 that contains a larger number of samples with respect to MVP C3D or PCN. Actually I just wanted to instert VRC in the experiment table in order to have a strong reference for comparison but since I am not getting very good results, I'm trying to understand whether the problem lies in the data difference or in the model. Do you think that VRC could be affected by beign trained on a single category? Or maybe the model implicitly expects point clouds within a certain scale? I'm normalizing inputs to the unit sphere. Now I'm training on the whole set of categories together but since the dataset is huge it will take time..

Hello! When I carried out experiments on ShapeNet34/55 datasets proposed in https://github.com/yuxumin/PoinTr, I found that when I trained the ECG in 200 or more epochs, the results is better than many networks such as CRN, ASFMNet, and even PoinTr. I generate 8192 points and the settings can be found as follow:


Peer to peer morning session code review

Hi @DJ-MrJay,
congratulation, you have made a great job and your project is working properly but you still have to validate your input, so that the user will not be able to submit an empty field.

Thanks. Noted. I'll work on it asap!


Modify AccountList display component to not include online sources

Since they don't have their own account number and go to CC instead

complete


Add feature sentiment analysis

I add to feature for sentiment analysis from text because if you going to another country that speaks English and you not yet, it is a good idea to check if your phrase makes sense and which sentiment you show at this moment.
For this, I used the API SentimentAnalysis of Azure Cognitive Services.
Positive Sentiment:

Negative Sentiment:

Thanks for completing the challenge! Fill out this form so we can send you the swag! https://aka.ms/cog-svcs-swag


text is not translated :(

Hello,
I have install all the prerequisites and follow all the installation steps but I don't see any text translated, only the main menu (new game, load game, etc..) and some textures but nothing else.
What am I doing wrong?
Thanks in advance,
FP

It could be literally anything, try asking in help channel on the discord server or try using the RG HF Patch to make sure everything is actually installed properly.


Text alignment

Implement a decent way to align text ... probably requires refac of Style class beforehand.
Maybe something along the lines of:
(new Style())->doStuff()->align()->centerCenter()

class Alignment(Style)
public function topLeft();
public function topCenter();
public function topRight();
public function centerLeft();
public function centerCenter();
public function centerRight();
public function bottomLeft();
public function bottomCenter();
public function bottomRight();

no need


cannot unpack non-iterable NoneType object on edge tts generation

literally just tried to use edge tts with a freshly trained model (it worked last night)

Your trained model has spaces in the name. That's probably why it's not working

Your trained model has spaces in the name. That's probably why it's not working

i replaced all the spaces with an underscore and got the same result, any other possibilities?

It's quite peculiar. Does it function with a different model?

It's quite peculiar. Does it function with a different model? And can you send all the log?

perfectly fine, no idea why

I checked the code of the tts but it's al working properly and because you tried with another model and worked what options did you used to train the model that didn't work??


Implement operator traits

Operators are currently exported as plain functions. We need to implement operator traits (Add, Sub, ...) instead.

Implemented.


Fix bug for select-language intro + Remove need for HTML

Related AB#1694

@jannisvisser @RubenGeo
I think we should do a similar exercise for the other interfaces (all in 1 go, maybe) to remove the HTML from the translations-file(s).

@elwinschmitz good idea. just to check: for the more-info-popups we will still have HTML in the jsons right?

Yes, we still need to find a pretty solution for that.
But that is already on the backlog somewhere....


Salt hash on BSSID with public-prefix to avoid corelation with other sources

See issue #271 for additonal details.
sha1 is used on BSSID without applying salting, thus correlation is possible with sources:

collections of bssids (just hash them, check if it's matched to what stored on the device)
collections of bssids which use the same algorithm (such as another app who use sha1(bssids) and leaks it to server-side)

Thus its better to salt them with public prefix such as:
sha256('hamagenapp' + bssid)
which will avoid the second possible correlation (as only hamagen app will use hamagenapp prefix), and increase the cost required for building rainbowtable (it will apply only for hamagen app)
Depending on the intended usage of this feature it might be possible to add in addition a unique prefix as well.

The file with the code mentioned here was removed at 4ff7b4815801abac19a8f5c71cf91a317d1e12b9


Configurable logging

Related to #11
Logging is now configurable in the config.js file.  There are 3 settings (info, error and silent).  I've given it a few goes on my machine and this seems to work fine.  However. I have not done extensive testing.
 /*Log Levels - default is "info"
     *
     *  This setting controls all logging into
     *  the server.log file - other than the
     *  restart message which will ALWAYS show.
     *
     * "info" -> log info and error events
     * "error" -> log error events only
     * "silent" -> do not log at all
     *
     * debug, warn, fatal, error, http could be
     * implemented as well, but not done yet*/
    logLevel: "info",

Note that I tidied up the spacing in the Logging.js file as well as it was using double tabs rather than 4 x spaces like in other files.  Just wanted to mention this in case the number of lines changed is a little alarming for a minor change
Feedback welcome!

Oh - I also added some changes in server.js which should make the logging more consistent.

Excellent work! :]

Am hoping I've not broken anything yet!


More detailed error message for component failure

Where can you get more detailed information when a component fails to init?
INIT_COMPONENT_FAILURE
I have this in my logs only but I can't see why it failed
WARN[0005] error processing component, daprd process will exit gracefully  app_id=example-actor instance=WIN-QI7NL08UUIM.reddog.microsoft.com scope=dapr.runtime type=log ver=1.9.5

INFO[0005] dapr shutting down.                           app_id=example-actor instance=WIN-QI7NL08UUIM.reddog.microsoft.com scope=dapr.runtime type=log ver=1.9.5
INFO[0010] Shutting down all remaining components        app_id=example-actor instance=WIN-QI7NL08UUIM.reddog.microsoft.com scope=dapr.runtime type=log ver=1.9.5

FATA[0010] process component state-mongo error: [INIT_COMPONENT_FAILURE]: initialization error occurred for state-mongo (state.mongodb/v1): init timeout for component state-mongo exceeded after 5s  app_id=example-actor instance=WIN-QI7NL08UUIM.reddog.microsoft.com scope=dapr.runtime type=log ver=1.9.5

Sadly it’s a bit component-dependent. Some components do a better job at explaining what went wrong than others. Do you see something else in the lines above? Sometimes the error from the component is a bit higher.

The Dapr components are part of Dapr too?
In order to figure out the actual error, I had to look at the Mongo library that Dapr uses, create a Go application that uses it, build a docker image ad deploy it to my Kubernetes cluster, to simulate the conditions. Could the mongo state store not log the underlying error?

I'm moving this issue to the components-contrib repos as it's specific to the MongoDB component.
What was the error?


The code snippet provided on this page throws an error

when executed from notebook only these commands fail. Other pyspark and sparkSQL cmd work ---
java.util.concurrent.ExecutionException: HTTP Error 401; url='https://login.microsoftonline.com/{tenant_id}/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/{tenant_id}/oauth2/token failed for getting token from AzureAD.; requestId='c36258bc-9bd2-485c-acf6-531d1101aa00'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret is provided.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ba47f948-5b90-9098-1987-1f6605c607c8
Version Independent ID: efb7d68e-10fd-6bc8-84d8-c5e2de2c9180
Content: How to create table DDLs to import into an external metastore - Azure Databricks - Workspace
Content Source: databricks/kb/metastore/create-table-ddl-for-metastore.md
Service: azure-databricks
GitHub Login: @mssaperla
Microsoft Alias: saperla

@mamccrea

@prasadvaze
Thank you for reaching out. At this time we are reviewing the ask and will provide an update as appropriate

@himanshusinha-msft and @KranthiPakala-MSFT   please disregard this issue.  The code works .  I had something wrong in my azure databricks workspace which was causing this error.


Objective/quest list being auto collapsed

The objective/quest list is being randomly collapsed sometimes after completing parts of quests. It isn't consistent and I couldn't find an option in HMP that was directly causing this.

Fixed in 10af1f1.


chore: Better names for (⌊a⌋₊ : α) = ⌊a⌋ again

Note to other reviewers: I'm expecting Eric and no one else to review this.

bors d+

bors merge