Integrate onnx-go as backend to remove onnxruntime dependency

Hey! The onnx-go project is being maintained again 🥳  See https://github.com/oramasearch/onnx-go.
Would the hugot team consider adding support for it as a backend? That way the external dependency on the onnx runtime can be (optionally) removed.

Hi Greg, thanks for the heads up. I'm a little reluctant to do the work to integrate this one, as it relies on gorgonia for its backend, which appears to be dormant (no new features for 2 years), i suspect it doesnt work with current models.  Go-native backends often seem to have a problem keeping up with the state of the art, at least until the Go ML community grows.  We are looking however at alternative backends and are closely working with GoMLX to natively integrate with their tooling.  It is however still C based behind the scenes, but some frameworks are being developed there to make dependency management much easier in future!

That's cool to hear.  I should mention that https://github.com/oramasearch/onnx-go was one of the projects that inspired us to do onnx-gomlx, by showing the conversion was possible and a reasonable thing to do.
Greg, I'm not sure what are your interests/incentives on the https://github.com/oramasearch/onnx-go project, but I wonder if there are other ways we could collaborate on this.
One thing that comes to mind: GoMLX started with XLA as a backend (for its performance/efficiency), but it was designed with multiple-backends in mind. In particular, it's on my TODOs to use Gorgonia as a backend: while slower (a pure Go engine will never be a top-performance), being pure Go has its advantages during development (no C libraries to install), or for smaller ML models.
Anyway, just an idea. If you would like to brainstorm sometime and discuss potential other collaborating ideas, pls reach out.
Btw, I should mention I'm meeting this week folks from https://github.com/AdvancedClimateSystems/gonnx (another ONNX engine for Go), also to discuss potential collaborating ideas -- looping in @Swopper050 on this.

Greg, I'm not sure what are your interests/incentives on the https://github.com/oramasearch/onnx-go project

Thanks for the response! I'm pretty agnostic as to what framework is used, since I'm wanting integrate some more ML capabilties to Bento, a Golang stream-processing tool I help maintain.
While speed/performance is not actually a top priority, some users run Bento serverlessly (i.e in AWS Lambda) so resource usage is a consideration. A top priority though is minimising external dependencies. One of the key-selling points of Bento is that it is packaged, and runs, as a single-binary and should just work out-the-box.
I actually already have a working POC I built using Hugot that works brilliantly (see PR) but the reliance on the external C and Rust dependencies means we have to package and release the distro with Hugot seperately -- which has turned out to be a dealbreaker.

... being pure Go has its advantages during development (no C libraries to install), or for smaller ML models.

Hugot has abstracted away a lot of the scary linear algebra which makes it a lot easier to develop against. If we could help create an easy-to-use Hugot distro that runs in pure Go, that would be 🥇 !

Anyway, just an idea. If you would like to brainstorm sometime and discuss potential other collaborating ideas, pls reach out.

I would love to help out however I can! I think the Go ecosystem could really benefit from a stable library for ML inference.

Bento looks very cool!
And I understand the desire for minimal dependencies, and in particular not wanting to manage the installation of separate Rust/C packages -- I hate that too, mainly because C++ various build systems are so complicated and just don't work together, or more specifically, they don't work together with the go tooling.
For Rust I wish the go tooling could depend on Cargo, and we could just add a dependency and everything would automatically, and statically be linked in. Maybe one day ...
Btw, from your perspective, would it help if every C dependency were linked statically ? So nothing to install in the workers ? I mean, it would still require library installation in the box building the binaries, but at least the final binary wouldn't have any dependencies ?

Bento looks very cool!

Thanks! I envision that it would plug-in really nicely to the tool and be a great value-add. Especially since it'll make ML inference a lot more accessible to our user-base.

And I understand the desire for minimal dependencies, and in particular not wanting to manage the installation of separate Rust/C packages -- I hate that too, mainly because C++ various build systems are so complicated and just don't work together, or more specifically, they don't work together with the go tooling.

Yeah exactly. I ended up not taking this approach on due to the overhead of needing to compile a libtokenizers + libonnxruntime for each GOARCH + GOOS combination we were wanting to support. Felt like I was introducing a lot of complexity to our CI and creating a complicated installation process for users, pretty much defeating the point of using Go in the first place.
Also, not to mention, that a target sys may have a different version of C installed (leading to this irritating ld: warning: object file (...) was built for newer macOS version (<ver>) than being linked (<other ver>) issue that I have cursed my local Go development environment with).

Btw, from your perspective, would it help if every C dependency were linked statically ?

I was actually looking into this during my Hugot integration! It would definitely help but, at the time, I didn't feel comfortable releasing + supporting a distro with statically linked libraries, especially given my unfamilarity with the CGO ecosystem.
With all this said, I would be very happy with a POC of a distro with statically linked libs for a single GOOS/GOARCH combination, say Linux ARM64. This way, I could see whether said libraries play nicely with Bento and what trade-offs will be made -- with larger memory usage and binary size being the obvious ones.

Hey there! @wipsel and I are the maintainers of gonnx. I saw this issue and I just wanted to let you guys know that should you want to integrate gonnx as a backend we would be happy to help things get started. Also if operators would be missing that you guys would require we could implement them.
Feel free to contact us should you want to integrate gonnx or would want to know some more about the project!

Hi @Swopper050,  i have just pushed a branch with go-onnx as an alternative backend. @riccardopinosio will do a pass to see if we can abstract the backends away further, but i think this is enough for you to take a look.
If you run the tests, first thing you will see is that all of our test models fail due to unsupported opset version. Obviously it will take time to increase the opset version, so perhaps step one is to downgrade the models opset. We tried first with distilbert and were still missing some ops, so perhaps step one is to iterate on the test models and implement the remaining ops from 13 so that we can run the test models (exported with opset 13) with gonnx.

Hi @RJKeevil, awesome.
I'll have a look which models you are using and which opsets and operators are missing and see if we can easily add this!

As an experiment, ive also just pushed to this branch a commit that when running with "GO" as runtime, it will use https://github.com/sugarme/tokenizer instead of bindings to the rust tokenizers. If all works ok we now have a means for running pure-go Hugot.

OK, so now we have a branch: alt-backends, if run with no build args it replicates the previous behaviour of Hugot (apart from also testing everything also with Cuda).

If run with -tags XLA you can call NewXLASession to create a session with a goMLXbackend, and you can run hugot_xla_tests.go (which currently all fail due to gcc incompatibility) @janpfeifer.
Once these unit tests pass we can hide ORT behind a similar build flag, so it ceases to be required at compile time. Then we could e.g. default to GO to ease of use, and allow users to switch to ORT or XLA for performance/compatibility/accelerators/training etc etc
Let me know if this new structure works for you, it was a pretty big change!

Wow thanks for this! I'm currently getting issues with clang and the tokenizers library (which I assume shouldn't be a dependency anymore when building with the -tags GO)
ld: library 'tokenizers' not found

Full log dump:
<EMAIL_ADDRESS>running clang failed: exit status 1
/usr/bin/clang -arch arm64 -Wl,-S -Wl,-x -o $WORK/b001/huggingface.test -Qunused-arguments /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/go.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000000.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000001.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000002.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000003.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000004.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000005.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000006.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000007.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000008.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000009.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000010.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000011.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000012.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000013.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000014.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000015.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000016.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000017.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000018.o /var/folders/jm/jdck4c8n6wz1b79b7pb6lmpw0000gn/T/go-link-1271651113/000019.o -lresolv -O2 -g -O2 -g -ldl -O2 -g -ltokenizers -ldl -lm -lstdc++ -O2 -g -framework CoreFoundation -framework CoreFoundation -framework Security
ld: warning: ignoring duplicate libraries: '-ldl'
ld: library 'tokenizers' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)

At the moment I default to ORT and rust tokenizers being required, as they are still default for now.  But i will add a temporary NOORT tag to be able to build without ORT, give me 10 mins

Ok @gregfurman with the latest commit this should now work with -tags NOORT,GO .  I have it that way so go is opt in for now, ORT is opt out, so no tags = ORT.  Will revisit this once all backends fully work.

Some delay from my side on the build of GoMLX for amazonlinux (with the older glibc). New ETA is Monday. (But if there is an important deadline or something let me know, I'll find some time).

@RJKeevil OK awesome. Am able to compile but (as you mentioned) am getting an unsupported opset version error. Will that reduceMean PR fix this?
Thanks for being so quick on this! Feel we're super close 😄

@gregfurman it wont fix it outright, but it is the first operator we notice is missing if we artificially reduce the opset required by our test models to v13 (i.e. the max currently supported by gonnx).
So it might be a bit of a journey: downgrade models to 13, get them working with gonnx by adding any/all missing operators, then eventually add more operators to gonnx until it supports our test models out of the box (with no downgrades).  Hopefully it triggers some community contributions to gonnx, as discussed at the top of the thread, go native ports are fantastic, if they have momentum to keep up with the original project.

Hmm your question triggered me to check what opset we require, so i printed out the versions per model with the python code:
model = onnx.load(modelPath)
opset_version = model.opset_import[0].version if len(model.opset_import) > 0 else None

sentence-transformers_all-MiniLM-L6-v2\onnx\model.onnx: 14
SamLowe_roberta-base-go_emotions-onnx\onnx\model.onnx: 11
protectai_deberta-v3-base-zeroshot-v1-onnx\model.onnx: 12
KnightsAnalytics_distilbert-NER\model.onnx: 11
KnightsAnalytics_distilbert-base-uncased-finetuned-sst-2-english\model.onnx: 11
So something else is up here, ill see what i can figure out

I'm assuming it's the sentence transformer that triggers the unsupported opset?

Its anything that isnt exactly opset 13, so all of them.  I'm playing with a test fork that uses 13 for anything below 13, lets see what happens.

Can probably just export all test models to onnx with opset 13 exactly

I've pushed a change that uses a forked gonnx that uses the lowest opset larger than the model's opset.  This gives us the true errors, feature extraction needs 14, TextClassification and TokenClassification probably work but ive messed up some logic reading their go outputs, ZeroShot has a missing operator in gonnx despite opset 12

Hello,
Currently I have limited time, but from next week I'll have more time. I created a label called hugot missing op that can be used to label existing issues concerning missing operators. Feel free to add more missing operators. If they are all labeled we could create a milestone for it so we have an overview of what needs to be implemented.
Implementing support for other opset versions was already on the list but it might require a bit of a refactor since the way we currently resolve opset versions is not optimal for supporting multiple versions.

I personally just use my IDE integration to run all unit tests, providing the first dockerfile step as the target and adding -tags NOORT,GO to the build arguments (to build hugot in go-only mode).
To use Make @riccardopinosio is better placed to comment, it might need changes due to the build args chnages i've done on this branch.  The current state of the docs can be found here

Thanks to @janpfeifer the XLA/goMLX tests now fully pass for all test models, so we have our second full fledged hugot backend! Try it out with -tags XLA (or -tags NOORT,XLA)

@Swopper050 for gonnx, if you look at this branch, you shouldn't have to use the docker container (i use that for tests in cicd). It should be sufficient to download the models with
go run ./testData/downloadModels.go
And then run the tests in the go file in your IDE. Sadly i couldn't convert the mini-LM and roberta to opset 13 (requires at least 14) but i could convert the others so if you run those tests we'll see what other operators are missing, we tried with the reduceMean branch and the next one seems to be Pow.

@RJKeevil @riccardopinosio I've managed to get the tests running.
As far as I could see all different operator sets of the models are:
19, 14, 13 and 12
Furthermore I noticed the following missing operators:

ReduceMean (PR already open)
Pow
Where
Sqrt
Erf
Cumsum

I've created a milestone for this with all issues that need to be fixed in order to run the Hugot models (hopefully).
The operators shouldn't be hard. Challenging part is refactoring the opset version handling, but I will work on that next week.

Quick heads up @Swopper050 @wipsel i've just released the pluggable backends/goMLX integration, so ive made a branch dedicated to adding gonnx (unsuprisingly called "gonnx"). The alt-backends branch has been removed.

Quick heads up @Swopper050 @wipsel i've just released the pluggable backends/goMLX integration, so ive made a branch dedicated to adding gonnx (unsuprisingly called "gonnx"). The alt-backends branch has been removed.

Alright! @wipsel and I are working on a big refactor to support the multiple operator sets, once that's done we'll update here as well

Fantastic, looks like some serious work is happening there!

@RJKeevil The big refactor is complete! Only the operators are remaning now. Will work on that the upcoming time.

Excellent stuff! When I get a second i'll try out a merge of master, i know its probably too early but maybe i can test at least the opset selection logic, perhaps some of the older models work already.

Updated the gonnx branch, integration looks fine but yep need the rest of your milestone, tests fail either with a higher opset than 13, or pass the opset check but have missing operators. Another step closer, thanks.

We'll get there!

@RJKeevil & @Swopper050 Thanks for all the work on this!
@janpfeifer I've been thinking about the statically linked approach:

Btw, from your perspective, would it help if every C dependency were linked statically ? So nothing to install in the workers ? I mean, it would still require library installation in the box building the binaries, but at least the final binary wouldn't have any dependencies ?

I'm not super familiar with the C/CGO ecosystem but am curious how much work you think this would entail?

@gregfurman gomlx now statically links the C stuff, e.g. if you look at the hugot dockerfile here
https://github.com/knights-analytics/hugot/blob/4652323efe426462fa451da860242ea6bd2a5345/Dockerfile#L32
we're grabbing the linux/amd libs for awslinux and building with those.
See also https://github.com/gomlx/gopjrt/tree/main/cmd for the other platforms.
Tokenizers is also statically linked so one can build a single binary for a platform if inference is via the XLA route.

@riccardopinosio Ahh that is awesome. Just some questions:

Have you noticed any performance hits as a result of using the statically linked deps? Unsure if Hugot has benchmarking on this.
How has this affected the image/binary size? Would be curious for those wanting to run in host-mode as opposed to a container.
Do we see significantly increased memory usage when running a Hugot instance with these static libs baked in?

Tokenizers is also statically linked so one can build a single binary for a platform if inference is via the XLA route.

Was this always statically linked or is this a new addition? Is the library embedded in the binary or does it just reside in the containers volumes.

I'd have to explicitly test this to be absolutely sure (travelling right now) but typically statically linked libraries are if anything very mildly faster due to less overhead calling them. The binary size will of course be larger, roughly by the same size as the size of the linked library. Memory usage should be unaffected by static Vs dynamic linking.  Lastly, the rust tokenizer has been statically linked in Hugot compiled binary from day one.

@RJKeevil & @Swopper050 Thanks for all the work on this!
@janpfeifer I've been thinking about the statically linked approach:

Btw, from your perspective, would it help if every C dependency were linked statically ? So nothing to install in the workers ? I mean, it would still require library installation in the box building the binaries, but at least the final binary wouldn't have any dependencies ?

I'm not super familiar with the C/CGO ecosystem but am curious how much work you think this would entail?

Just a couple of comments:
Statically linking the CPU XLA/PJRT is optional for GoMLX (actually, underlying gopjrt project). One simply has to include the github.com/gomlx/gopjrt/pjrt/c/cpu/static package -- it's 10 lines of code + 20 lines of comments if you are curious.
The implications of statically linking:

The obvious one, no need to install the PJRT plugin where one deploys the binary.
Non obvious one: it's large (the static .a PJRT is 800Mb with all the symbols, while the  the dynamic library is only 200Mb), and statically linking is much slower than dynamic linking. Compiling the binaries went from "instantaneously" (as usual with Go) to taking 4 seconds. Not an issue if you are only running the binary, but very annoying during development -- hence, it's optional.

@RJKeevil @gregfurman Small update here.
All missing operators are now implemented.
Only the changelogs of opsets 14-19 remaining.

Amazing! I just gave it a quick test on the branch, but i somehow get the error "Test failed with error unsupported operator: Expand" for some models.  I see that you have a commit for that operator, but maybe some metadata is missing?  Or maybe its missing for the opset version of that model?

Amazing! I just gave it a quick test on the branch, but i somehow get the error "Test failed with error unsupported operator: Expand" for some models. I see that you have a commit for that operator, but maybe some metadata is missing? Or maybe its missing for the opset version of that model?

Hmmmm I'll have a look!

Fab yes solved, next one: Test failed with error input 0 for op cast v9 does not allow dtype bool.  Think were on the last few issues here!


[TO_REVIEW] Add DomainAndLabelStratifiedSubsampleTransformer + Fix DomainStratifiedSubsampleTransformer

Fix TypeError: DomainStratifiedSubsampleTransformer.transform() got an unexpected keyword argument 'allow_source'with DomainStratifiedSubsampleTransformer
Add DomainAndLabelStratifiedSubsampleTransformer to have a transformer preserving the percentage of samples
for each class and for each sample domain (To prevent issues like: ValueError: This solver needs samples of at least 2 classes in the data, but the data contains only one class: 1)

Then, we have to be consistent with the splitters. IMO, we have to rename the subsamplers as:

DomainStratifiedSubsampleTransformer --> DomainSubsampleTransformer
DomainAndLabelStratifiedSubsampleTransformer --> StratifiedDomainSubsampleTransformer

BTW, we can even get rid of 'Transformer':

DomainStratifiedSubsampleTransformer --> DomainSubsampler
DomainAndLabelStratifiedSubsampleTransformer --> StratifiedDomainSubsampler

I agree


[ 0.9.6-nightly-43a4201] Minor bug in time clause

Hi,
I am trying to find the timestamp of the first value in a series, so I have been using
Select value from metric order by time asc limit 1
I have had serious performance problems, and a couple of times the database crashed. So I have started to put a time limit on the query.
Select value from metric where time > now() - 30d order by time asc limit 1
This seems to work much better, but I did notice that
time > now() -30d
i.e. no space before the time operator returns no rows or an error message.

This may be a parser issue, the code to examine is here: https://github.com/influxdb/influxdb/tree/master/influxql

I found what is causing the behavior but I'm not sure of an elegant solution. I think there needs to be some additional logic in the parser to scan ahead for multiple non-tokens?

I was assuming that you would throw query parser error.  Something that tells the user its expecting a space.

I have just tried it against the 0.9.6 and it if I run the query without the space, I get no results and also it blocks writes (this is using the bz1 engine)

I would like to help on this. I see the file scanner.go - scanNumber() being pointed out for the place where number parsing is done.
Any other place where I should look at.

@nitishagar -- all code that builds the abstract syntax tree is performed inside the influxql package. That is definitely the place to look, and I would be surprised if the fix wasn't in there.
If you want to understand more about how the package works, I suggest you read this article:
https://blog.gopheracademy.com/advent-2014/parsers-lexers/

Hi!
I've taken a look into this, and I'd be glad to try at a fix, assuming @nitishagar isn't still on it. (If so, let me know.)
AFAICS, the issue boils down to the parseFill function (https://github.com/influxdata/influxdb/blob/master/influxql/parser.go#L2013), which apparently swallows expressions that do not match the expected fill(...) format. parseFill only does a single unscan when it encounters a non-fill(...)-expression, although such an expression may consist of an arbitrary number of tokens. In particular, it will unscan the subsequent whitespace token in cases such as @ivanscattergood's, and thus happily continue parsing.
One simple fix is to rewrite parseFill so that it first checks whether the next token is an IDENT with the literal value "fill", and then either backs out at once if not, or continues with parsing a fill(...) expression, returning a parse error if it fails. If a fix like that sounds OK, I would be happy to give it a try. (Another would be to add an option to save the state of the scanner before the parseExpr call.)
P.S.: I haven't looked for (possible) similar cases other places in the parser code. And, thank you for an awesome piece of software.

@edwkar You can go ahead with this. Somehow I didn't receive notification for the last 2 mentions and also was occupied with stuff at work. I look at picking up smoother item :)

This seems to now correctly report an error during parsing. I'm going to close this issue since it seems to be fixed in 0.13.


Add dig for etcd problem determination.

IMO: I don't think this is really needed, or helpful.  dig'ing from the start container is not the connection to worry about.   You need to dig from the etcd/etcdEvent ec2 instances themselves....(and those already have the DNS tools).

Right, it only adds ~ 0.5 MB to the size of the image. Since I use k2-tools for all of my development, I expect basic PD tools to be available. That said, it's not like I don't immediately fall back to nslookup when necessary...
Honestly, I think sometimes we over-optimize the size of our images. Developer productivity is usually more important than saving a small amount of space.


ESP32-WROOM-32UE doesn't work with CYW89359 bluetooth chip (IDFGH-6167)

Hello Guys,
Could you help me? ESP32_DevKitc_v4 doesn't work with Bluetooth chip (My VW ID.3 car) CYW89359 - https://www.cypress.com/file/350976/download
I have ESP32_DevKitc_v4 (ESP32-WROOM-32UE).
Build example from the master https://github.com/espressif/esp-idf/tree/master/examples/bluetooth/hci/controller_hci_uart_esp32 and flashed the device.
Connected it to BlueZ stack
sudo btattach -B /dev/ttyUSB1 -S 921600
My ESP32 device:
sudo hciconfig -a hci1
hci1:	Type: Primary  Bus: UART
BD Address: E0:E2:E6:B2:E3:06  ACL MTU: 1021:9  SCO MTU: 255:4
UP RUNNING
RX bytes:1847 acl:0 sco:0 events:106 errors:0
TX bytes:4038 acl:0 sco:0 commands:105 errors:0
Features: 0xbf 0xee 0xcd 0xfe 0xdb 0xff 0x7b 0x87
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV3
Link policy: RSWITCH SNIFF
Link mode: SLAVE ACCEPT
Name: 'ubuntu #2'
Class: 0x0c0000
Service Classes: Rendering, Capturing
Device Class: Miscellaneous,
HCI Version: 4.2 (0x8)  Revision: 0x30e
LMP Version: 4.2 (0x8)  Subversion: 0x30e
Manufacturer: RivieraWaves S.A.S (96)
It works well with my Raspberry Pi Bluetooth, Android phone, iPhone...
But it doesn't work with Bluetooth chip (My VW ID.3 car) CYW89359.
This is the information I requested from Raspberry Pi Bluetooth, [b]not via esp32!!![/b]
root@raspberrypi-bt:~# hcitool info "18:48:CA:10:7E:AC" Requesting information ...         BD Address:  18:48:CA:10:7E:AC         Device Name: My VW 3816         LMP Version: 4.2 (0x8) LMP Subversion: 0x420d         Manufacturer: Broadcom Corporation (15)         Features page 0: 0xbf 0xfe 0xcf 0xfe 0xdb 0xff 0x7b 0x87                 <3-slot packets> <5-slot packets> <encryption> <slot offset>                 <timing accuracy> <role switch> <sniff mode> <RSSI>                 <channel quality> <SCO link> <HV2 packets> <HV3 packets>                 <u-law log> <A-law log> <CVSD> <paging scheme> <power control>                 <transparent SCO> <broadcast encrypt> <EDR ACL 2 Mbps>                 <EDR ACL 3 Mbps> <enhanced iscan> <interlaced iscan>                 <interlaced pscan> <inquiry with RSSI> <extended SCO>                 <EV4 packets> <EV5 packets> <AFH cap. slave>                 <AFH class. slave> <LE support> <3-slot EDR ACL>                 <5-slot EDR ACL> <sniff subrating> <pause encryption>                 <AFH cap. master> <AFH class. master> <EDR eSCO 2 Mbps>                 <EDR eSCO 3 Mbps> <3-slot EDR eSCO> <extended inquiry>                 <LE and BR/EDR> <simple pairing> <encapsulated PDU>                 <err. data report> <non-flush flag> <LSTO> <inquiry TX power>                 <EPC> <extended features>         Features page 1: 0x09 0x00 0x00 0x00 0x00 0x00 0x00 0x00         Features page 2: 0x7f 0x0b 0x00 0x00 0x00 0x00 0x00 0x00
Could you help me, maybe I should tunning some parameters in ESP32 firmware?
`Executing action: monitor
Running idf_monitor in directory /home/user/esp/esp-idf/examples/bluetooth/hci/controller_hci_uart_esp32
Executing "/home/user/.espressif/python_env/idf4.4_py3.8_env/bin/python /home/user/esp/esp-idf/tools/idf_monitor.py -p /dev/ttyUSB0 -b 115200 --toolchain-prefix xtensa-esp32-elf- --target esp32 --revision 0 /home/user/esp/esp-idf/examples/bluetooth/hci/controller_hci_uart_esp32/build/controller_hci_uart.elf -m '/home/user/.espressif/python_env/idf4.4_py3.8_env/bin/python' '/home/user/esp/esp-idf/tools/idf.py'"...
--- idf_monitor on /dev/ttyUSB0 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jul 29 2019 12:21:46
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6744
load:0x40078000,len:14812
ho 0 tail 12 room 4
load:0x40080400,len:3792
0x40080400: _init at ??:?
entry 0x40080694
I (29) boot: ESP-IDF v4.4-dev-3401-gb86fe0c66c-dirty 2nd stage bootloader
I (29) boot: compile time 00:52:30
I (30) boot: chip revision: 3
I (34) boot_comm: chip revision: 3, min. bootloader chip revision: 0
I (41) boot.esp32: SPI Speed      : 40MHz
I (46) boot.esp32: SPI Mode       : DIO
I (50) boot.esp32: SPI Flash Size : 2MB
I (55) boot: Enabling RNG early entropy source...
I (60) boot: Partition Table:
I (64) boot: ## Label            Usage          Type ST Offset   Length
I (71) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (79) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (86) boot:  2 factory          factory app      00 00 00010000 00100000
I (94) boot: End of partition table
I (98) boot_comm: chip revision: 3, min. application chip revision: 0
I (105) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=0b954h ( 47444) map
I (131) esp_image: segment 1: paddr=0001b97c vaddr=3ffbdb60 size=04604h ( 17924) load
I (138) esp_image: segment 2: paddr=0001ff88 vaddr=40080000 size=00090h (   144) load
I (138) esp_image: segment 3: paddr=00020020 vaddr=400d0020 size=35e14h (220692) map
I (225) esp_image: segment 4: paddr=00055e3c vaddr=40080090 size=16888h ( 92296) load
I (263) esp_image: segment 5: paddr=0006c6cc vaddr=50000000 size=00010h (    16) load
I (274) boot: Loaded app from partition at offset 0x10000
I (274) boot: Disabling RNG early entropy source...
I (286) cpu_start: Pro cpu up.
I (286) cpu_start: Starting app cpu, entry point is 0x4008115c
0x4008115c: call_start_cpu1 at /home/user/esp/esp-idf/components/esp_system/port/cpu_start.c:147
I (0) cpu_start: App cpu up.
I (300) cpu_start: Pro cpu start user code
I (300) cpu_start: cpu freq: 160000000
I (300) cpu_start: Application information:
I (305) cpu_start: Project name:     controller_hci_uart
I (311) cpu_start: App version:      v4.4-dev-3401-gb86fe0c66c-dirty
I (318) cpu_start: Compile time:     Oct 15 2021 02:07:23
I (324) cpu_start: ELF file SHA256:  7870c22910fcaf00...
I (330) cpu_start: ESP-IDF:          v4.4-dev-3401-gb86fe0c66c-dirty
I (337) heap_init: Initializing. RAM available for dynamic allocation:
I (344) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (350) heap_init: At 3FFB7CD8 len 00000328 (0 KiB): DRAM
I (356) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
I (363) heap_init: At 3FFC37E8 len 0001C818 (114 KiB): DRAM
I (369) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (375) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (382) heap_init: At 40096918 len 000096E8 (37 KiB): IRAM
I (389) spi_flash: detected chip: generic
I (392) spi_flash: flash io: dio
W (396) spi_flash: Detected size(4096k) larger than the size in the binary image header(2048k). Using the size in the binary image header.
I (411) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (444) CONTROLLER_UART_HCI: HCI UART1 Pin select: TX 5, RX, 18, CTS 23, RTS 19
I (454) BTDM_INIT: BT controller compile version [078d492]
I (454) system_api: Base MAC address is not set
I (454) system_api: read default base MAC address from EFUSE
I (464) phy_init: phy_version 4670,719f9f6,Feb 18 2021,17:07:07
`
Best Regards.

Hello for fast answer,
Doesn’t work mean, I can’t connect, request name, create pair.
For example i can’t request name
hcitool name  18:48:CA:10:7E:AC
I don’t see any communication with chip with hcidump, I think problem in Inquiry/Page procedure.
How can I collect additional logs Inquiry/Page procedure?
Best Regards

@7erduk  We will look into this issue. Please wait for the reply.

Hi @7erduk,
It will do help if you provide some LOGs with btmon. I need the logs in btmon CLI when you run hcitool name 18:48:CA:10:7E:AC in another CLI.

Hello,
btmon logs
hcitool name 18:48:CA:10:7E:AC
`Bluetooth monitor ver 5.53
= Note: Linux version 5.8.0-50-generic (x86_64)                                                                                0.567270
= Note: Bluetooth subsystem version 2.22                                                                                0.567272
= New Index: E0:E2:E6:B2:E3:06 (Primary,UART,hci1)                                                                                [hci1] 0.567273
= Open Index: E0:E2:E6:B2:E3:06                                                                                [hci1] 0.567273
= Index Info: E0:E2:E6:B2:E3:06 (RivieraWaves S.A.S)                                                                                [hci1] 0.567273
= New Index: C8:58:C0:25:B9:BB (Primary,USB,hci0)                                                                                [hci0] 0.567274
@ RAW Open: dumpcap (privileged) version 2.22                                                                                {0x0002} [hci0] 0.567274
@ MGMT Open: bluetoothd (privileged) version 1.17                                                                                {0x0001} 0.567275
@ MGMT Open: btmon (privileged) version 1.17                                                                                {0x0003} 0.567285
@ RAW Open: hcitool version 2.22                                                                                {0x0004} 3.608963
@ RAW Close: hcitool                                                                                {0x0004} 3.609330
@ RAW Open: hcitool version 2.22                                                                                {0x0004} [hci1] 3.609386
< HCI Command: Remote Name Request (0x01|0x0019) plen 10                                                                                #1 [hci1] 3.609737
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Page scan repetition mode: R2 (0x02)
Page scan mode: Mandatory (0x00)
Clock offset: 0x0000

HCI Event: Command Status (0x0f) plen 4                                                                                #2 [hci1] 3.623396
Remote Name Request (0x01|0x0019) ncmd 5
Status: Success (0x00)
HCI Event: Remote Name Req Complete (0x07) plen 255                                                                                #3 [hci1] 8.750761
Status: Page Timeout (0x04)
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Name:
@ RAW Close: hcitool                                                                                {0x0004} [hci1] 8.750893
@ RAW Open: hcitool version 2.22                                                                                {0x0004} 10.148010
@ RAW Close: hcitool                                                                                {0x0004} 10.148103
@ RAW Open: hcitool version 2.22                                                                                {0x0004} [hci1] 10.148131
< HCI Command: Remote Name Request (0x01|0x0019) plen 10                                                                                #4 [hci1] 10.148173
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Page scan repetition mode: R2 (0x02)
Page scan mode: Mandatory (0x00)
Clock offset: 0x0000
HCI Event: Command Status (0x0f) plen 4                                                                                #5 [hci1] 10.157088
Remote Name Request (0x01|0x0019) ncmd 5
Status: Success (0x00)
HCI Event: Remote Name Req Complete (0x07) plen 255                                                                                #6 [hci1] 15.288187
Status: Page Timeout (0x04)
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Name:
@ RAW Close: hcitool                                                                                {0x0004} [hci1] 15.288326
@ RAW Open: hcitool version 2.22                                                                                {0x0004} 19.993537
@ RAW Close: hcitool                                                                                {0x0004} 19.993675
@ RAW Open: hcitool version 2.22                                                                                {0x0004} [hci1] 19.993708
< HCI Command: Remote Name Request (0x01|0x0019) plen 10                                                                                #7 [hci1] 19.993790
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Page scan repetition mode: R2 (0x02)
Page scan mode: Mandatory (0x00)
Clock offset: 0x0000
HCI Event: Command Status (0x0f) plen 4                                                                                #8 [hci1] 20.002732
Remote Name Request (0x01|0x0019) ncmd 5
Status: Success (0x00)
HCI Event: Remote Name Req Complete (0x07) plen 255                                                                                #9 [hci1] 25.136224
Status: Page Timeout (0x04)
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Name:
@ RAW Close: hcitool                                                                                {0x0004} [hci1] 25.136586
@ RAW Open: hcitool version 2.22                                                                                {0x0004} 58.837702
@ RAW Close: hcitool                                                                                {0x0004} 58.837745
@ RAW Open: hcitool version 2.22                                                                                {0x0004} [hci1] 58.837754
< HCI Command: Remote Name Request (0x01|0x0019) plen 10                                                                                #10 [hci1] 58.837812
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Page scan repetition mode: R2 (0x02)
Page scan mode: Mandatory (0x00)
Clock offset: 0x0000
HCI Event: Command Status (0x0f) plen 4                                                                                #11 [hci1] 58.842467
Remote Name Request (0x01|0x0019) ncmd 5
Status: Success (0x00)
HCI Event: Remote Name Req Complete (0x07) plen 255                                                                                #12 [hci1] 63.979359
Status: Page Timeout (0x04)
Address: 18:48:CA:10:7E:AC (Murata Manufacturing Co., Ltd.)
Name:
@ RAW Close: hcitool
`

@7erduk We will look into this issue. Please wait for the reply.
For logs, maybe you can capture some HCI logs using WireShark.

snoop file with hcidump tool
https://mega.nz/file/9jpWFarS#JHGPQDVgyzHKDeE6lG0hX_duSjRa2SzMG_4JmHi3VAM

Also, you can see HCI log from raspberry pi, it is not ESP32
The car chip CYW89359 answer on hcitool name 18:48:CA:10:7E:AC
https://mega.nz/file/U3hwwYAC#gPwZNs0_CAoxLCzITBA0kZl8e7bJpEY4XCR0VYLpRJg

Maybe I can somehow to make a log with Inquiry/Page procedure. Do you have any idea?

@7erduk,
From the btmon LOGs, the command was just correctly passed to controller (ESP32) from host stack. And the Command Status Event returned successfully. All the things looks like the systems worked well.
As for "HCI Event: Remote Name Req Complete" with error code "Page Timeout", it was just "page timeout".
From my perspective, there is no problem on Interoperability between the host and controller. Maybe you should check it again.
Thanks.

@7erduk,
From the btmon LOGs, the command was just correctly passed to controller (ESP32) from host stack. And the Command Status Event returned successfully. All the things looks like the systems worked well.
As for "HCI Event: Remote Name Req Complete" with error code "Page Timeout", it was just "page timeout".
From my perspective, there is no problem on Interoperability between the host and controller. Maybe you should check it again.
Thanks.

Hello BetterJincheng,
I have tested it many many  many times, I have only got "Page Timeout".  Other my devices don't have such a problem with CYW89359.

Hello Guys,
I have connected an antenna to my esp32, Now everything is working well. Thank you!

Thanks for reporting, and glad to hear everything is working well now, feel free to reopen.


useElementScroll is deprecated

Current version: v2.0.0
Error message:

useElementScroll is deprecated. Convert to useScroll({ container: ref }).
Warning: useLayoutEffect does nothing on the server, because its effect cannot be encoded into the server renderer's output format. This will lead to a mismatch between the initial, non-hydrated UI and the intended UI. To avoid this, useLayoutEffect should only be used in components that render exclusively on the client. See https://reactjs.org/link/uselayouteffect-ssr for common fixes.

hey guys, will get a fix in for this shortly 👍

Fixed in 2.0.1!


Bound the regularization smoothing factor $r$

Perhaps we can ramp $r$, similar to how one might ramp boundary conditions to help the Newton method converge. So for a given time step, we can first obtain a solution for large $r$, and then reduce $r$ sequentially. I'm not yet sure how difficult this will be to do with AMR.

That idea from April works quite nicely. Still there seems to always be a lower limit for $r$. This is probably dependent on some parameters, most importantly the grid spacing near the phase interface. We have yet to quantify this.

Using the continuation algorithm in my submitted paper, it is not troublesome to reduce $r$ until the interface is limited by the mesh cell size.


refactor: Remove metakey concept

Which issue does this PR close?

Part of https://github.com/apache/opendal/issues/5314
Rationale for this change

To make Lister behavior more predictable.
What changes are included in this PR?

Remove the metakey concept.
Are there any user-facing changes?

Merging #5319 will degrade performances by 15.89%

cc @dqhl76, I'm considering disabling tests for those cases or simply refactoring them. They've never been reported correctly...
Any ideas?

Any ideas?

Feel the same with you. It can be a bit distracting. For those who aren't familiar with what it means, it might cause unnecessary concern.


Not sure how to use it

Downloaded the packages from releases, running both executable at the same time, not sure what's the correct config, but have created a text file with url links in it, was able to login in through the code, but then when you start scrapping it says: Main Exception local variable 'driver' referenced before assignment
Anything you want me to try apart from what you've mentioned in the readme? A little help would be appreciated.

Don't use the releases since I have not updated the releases to the latest version yet.
Clone the repo,
Download the respective Chrome Binary for your operating system from releases section and extract it inside Chrome-bin folder,
Run python chromedriver.py in one terminal
Run python educative_scraper.py in the second terminal
The rest should work fine.
If possible create a video demonstration.

Did that, same issue.
Need help with creating config file, when it's asking for URL, should i be pasting the url for the course or the file path for url containing text file? Tried with both and still have the same issue.

File path -> C:/urls.txt

@chandangm is your issue fixed?

@anilabhadatta hi bro I have a quey. Where does the log.txt file gets saved?

@arnabs542 save path


An error occurred doing POST card_token

Describe the bug:
mp.createCardToken return the following error: { code: "E603" description: "an error occurred doing POST card_token" }
To reproduce running the example:
const cardToken = await mp.createCardToken({ cardNumber: '375365153556885', cardholderName: 'APRO', cardExpirationMonth: '11', cardExpirationYear: '25', securityCode: '1234', identificationType: 'CPF', identificationNumber: '12345678910', });

Hi @juliocesar757 ,
I tried the exactly same code and it went well, without any errors.

Could you confirm me if the error still persists?
Thanks

Hello @NicholasPedroso
The error is intermittent.
Sometimes you get the token on the first API call. Sometimes I need to redo the call to get the token without the E603 error.

@NicholasPedroso
Second try, it worked without errors.

Hi @juliocesar757
Thanks for the info, I'm working on it and I'll let you know when I get some status on it.

I'm closing this issue as it is related to issue #43


Add support for Xcode 14.3.1 / Swift 5.8.1

Describe the feature
The latest version of Xcode, 14.3.1, supports up to Swift 5.8.1. Projects dependent on these versions are unable to use smithy-swift.
Use Case
Any Swift or iOS features which require Xcode 14.3 or 14.3.1, or Swift 5.8 or 5.8.1
Proposed Solution
No response
Other Information
No response
Acknowledgements

[ ] I may be able to implement this feature request
[X] This feature might incur a breaking change

Closing this issue.  Feel free to reopen & clarify your request if you are still having problems with this project on latest Xcode.


Logout page

Per the team's last discussion, we've agreed that we should provide a dedicated page for the user to log out. If I remember correctly, we have yet to determine how this page will be reached and I don't believe we have an example of how it should look at this time.

Closed by 1482f8c


swap to words-n-numbers custom emoji regex

Check if it will do what I need.

If it does, the text regex needs to be custom and work with gi-flags


Cplex runtime not found

I installed anaconda, cplex 12.9 on my mac. I also installed Docplex using  the command pip install docplex and set the variable environment : PYTHONPATH as it is explained in IBM website. However each time I try to run a code in python I got this error : Cplex runtime not found. Please install Cplex. I can’t figure out where is the problem. Any help would be very appreciated. Thank you

Hi,
If you installed cplex studio, can you please make sure that $CPLEX_STUDIO_DIR129/cplex/python/3.6/x64_win64 (replace 3.6 by your actual python version) directory is in your PYTHONPATH ?
If you did not install cplex studio but just the cplex lib from anaconda or pypi, I will need you to dump your PYTHONPATH and PATH to check your environment.
Thanks

Hi,
Thanks for reply
I installed culex studio on my pc and I did set the PYTHONPATH variable as shown in the attached screenshots. I still got the same error

Let's check that cplex is working.
Can you please run this in a shell ?
$python -c "import cplex; print(cplex.__version__)"
(it should print <IP_ADDRESS>)
If this works, then we will need to check that the python env you are using with your docplex model knows about the PYTHONPATH you set there. In your python script where you have your "cplex not found error", right before you create a model, can you print the contents of sys.path ?

I run it but I got this error :

As for you 2nd question, I got the ' DOcplexException: CPLEX runtime not found: please install CPLEX or solve this model on DOcplexcloud' error after I created the model

Now it works

sys.path does not contain your /Applications/CPLEX_Studio129/cplex/python/3.7/x86_64-osx directory, as you set in your .bash_profile.
You set the PYTHONPATH in your .bash_profile, so you need to run your python in a subprocess of a bash.
I'm not an OSX user, but as far as I remember, you must use launchctl to set system environment variables (something like launchctl setenv PYTHONPATH /Application/CPLEX_Studio129 etc...)
Can you please set PYTHONPATH as a system environment variable and try again ?
Thanks

I dont know how to set system environment. I will  do s research nd let you know if it works

I used launchctl setenv as you recommended but I still get the same error.

If you don't know how to set environment variables, I would advise you to just install the CPLEX python libraries in your python environment. This is described in the 'Setting up CPLEX' (aka installation guide) of CPLEX: https://www.ibm.com/support/knowledgecenter/SSSA5P_12.9.0/ilog.odms.cplex.help/CPLEX/GettingStarted/topics/set_up/Python_setup.html
Just make sure you are running the python interpreter that goes with your /Users/infotech/anaconda3
Thanks

Setting up Python API using either the script setup.py or the environment variable Pythonparh is exactly what I've done ( and as far as I know, to be able to do so cplex studio must be installed. Am I wrong ?)
The result I get whenever I run code in python was : cplex runtime not found. It has to noted that I used this method in a windows PC and it worked perfectly. In my mac I couldn't figure why it doesn't work and how to fix the problem.
As for your last remark. I didn't get it. How to make sure that the interpret goes with the directory. And should I need to uninstall cplex studio ?
Thanks

Hello,
No, you didn't run the setup script. If you did, then cplex would have been installed in your sys.path. This is different from setting up a PYTHONPATH.
On mac, 100% of the errors of this type I have seen are due to users running python in a terminal, and using another python in their IDE. This is because Mac comes with its own installation fof python, which your IDE could have picked up (I can't check, you deleted your post with the screenshots of your sys.path). On windows this is easier: There is no system python so when you install python, there's only one. On Mac, I often see at least 2 installations.
You can check the python executable you use in your script where you have your error by checking the value of sys.executable.
Please also do not delete comments in this issue, it makes it difficult to follow what happened and makes it difficult for me to check what we already tryied (in this instance, I cannot check the paths you previously copy here).
Thanks

Hello,
No, you didn't run the setup script. If you did, then cplex would have been installed in your sys.path. This is different from setting up a PYTHONPATH.
On mac, 100% of the errors of this type I have seen are due to users running python in a terminal, and using another python in their IDE. This is because Mac comes with its own installation fof python, which your IDE could have picked up (I can't check, you deleted your post with the screenshots of your sys.path). On windows this is easier: There is no system python so when you install python, there's only one. On Mac, I often see at least 2 installations.
You can check the python executable you use in your script where you have your error by checking the value of sys.executable (This is probably a good test: print(sys.executable) from a python you run in a Terminal window, then same thing in your script and we'll be sure you are using the right python).
Please also do not delete comments in this issue, it makes it difficult to follow what happened and makes it difficult for me to check what we already tryied (in this instance, I cannot check the paths you previously copy here).
Edit: you don't need to install your cplex stdio. Just run CPLEX_Studio129\cplex\python\3.7\x86_64-osx\setup.py with the same python interpreter than the one used by your IDE.
Thanks

Problem solved !! Thank you so much


Productize and harden our functional testing infrastructure

This is too hard for our users to get right and we currently have many copies of this. We should make this into a shipped package. This is not just the TestHost, it needs to include all of the MVC-specific things that we do.
This needs to include validation that some of the common problems are addressed:

Shadow copy off
Look for missing deps files
Set working directory correctly
Prevent assembly crosstalk (through ApplicationPartManager)

And new stuff:

Get a RequestDelegate or HttpClient

See https://github.com/aspnet/Mvc/issues/5793 for other pointers, discussions, and scenarios.

@javiercn - has started peeling the onion on this btw as part of his testing effort for identityservice

Any ETA on an MVP or updates? With regard to #276, I'm not sure how to approach implementing TDD within an application that performs HasClaims checks on just about every HttpPost controller action.

@javiercn any updates here?

I'm working on other things at the moment, but I'll be able to come back to this sometime this week.

Leaving this open because we won't officially have this done until 2.1.0.

Is this going to be fixed for 1.1.x as well? We are currently having the same hacks in place in order to have web tests with an in-process kestrel server.

@drauch - this is a new feature we're building for the 2.1 release, so there isn't really anything to fix for 1.x because this feature wasn't there.

@Eilon : Maybe I'm mistaken here, but hosting Kestrel in-process to do integration testing doesn't seem to be a "new" feature. We do web testing this way using ASP.NET Core 1.1 and ran into various problems all described in the initial post here. We would very much like it to not have to use shadow copying, create the deps file with a special target in the build file, etc. Would be great if this thingy (i.e., whatever mitigates these problems) would be available for 1.1 too.

@drauch - ah, well, the way we're fixing this is with new features we're adding in the 2.1 release, so we're not planning on back-porting this to the 1.x releases. Perhaps you can take what we've done so far in the 'dev' branch (or even the 'rel/2.0.0' branch) and see if that addresses the issues you have in 1.x?

We need an extensibility point to configure the WebHostBuilder like real applications do. See: https://github.com/aspnet/MetaPackages/blob/396f413b389f5983e413be465f917cbfdbda26c9/src/Microsoft.AspNetCore/WebHost.cs#L152

Hi, I got here from https://github.com/aspnet/Razor/issues/1212 and see this issue is closed but I am still experienced the original problem. Has this been tested before closing the issue?

@dustinmoris I’ve perused through that issue and seems like it’s an issue with razor missing references to assemblies.
I put together a blog post a while ago on how to do this.
https://blogs.msdn.microsoft.com/webdev/2017/12/07/testing-asp-net-core-mvc-web-apps-in-memory/ can you try the instructions in the blog post (and related sample) and let me know if you still have issues?

@javiercn Thanks that works, but that feels extremely complex for a simple unit test project for a razor MVC app. Hope this can be simplified one day...

@dustinmoris This will actually be simplified in the next release. See https://github.com/aspnet/Announcements/issues/275 for details.

Shadow copying is still required to be turned off? That's a bummer : (

Shadow copying will still be required to be turned off? That's a bummer : (

MVC needs to be able to find the view files on disk to be able to do compilation. This is also relevant if you have any other file content that you are serving. If you don't need to do any of these things you should be OK to use shadow copying.
If you do need these things, let's talk, I'd love to understand what you think we could do to improve this.

View files and content is found by setting the content directory accordingly, that's not an inherent problem with shadow copying turned on. However, it is important that the compilation-time assembly references for Razor compilation are properly shadow copied. I hope this second issue is resolved by the infrastructure, at the moment, we have to do the copying ourselves.
We need it primarily to run all our web tests in the background.

@drauch are you doing this already on your tests? If so, how are you setting up the content root?


Divert mode select doesn't work

Hi @firstof9,
Just updated to the latest version (2.1.3) and the divert mode select still doesn't set the right mode when any option is selected. Curiously, listening for updates on changes from the web UI itself works great.
I'm on wifi version 4.1.4.dev, Firmware: <IP_ADDRESS>P and Home Assistant 2022.08.6.
Let me know how I can help debug this.

Just noticed my divert_active binary sensor is also showing as off when divert mode is enabled.

Yes, I have to look at it again. I don't think the API is working quite right for divert, I may need to submit a issue to OpenEVSE.

Possible workaround in the meantime :
Use MQTT to  control the Divertmode /divertmode/set [1 | 2] : enable (1)/ disable (2) divert mode
Toggling it in the OpenEVSE webpage changes the selection in HA.

I'm still using this workaround on the latest version of OpenEVSE WiFi.

I've been having trouble getting MQTT to change the divert mode too, maybe just me though... That's what led me to this awesome HA integration, I was trying to do everything through Node-Red before.

Divert mode has been updated to fast / eco and a toggle added to enable/disable the Solar PV Divert feature.


Right-click while dragging causes drag adorner to remain shown

If the mouse is right-clicked while dragging, the drag adorner remains visible (on top of all other desktop windows) until a new drag is started.
I would expect the click to cancel the drag at least, and potentially open a context menu (i.e. pass through the event to the underlying control).
To reproduce:

Drag any item with a drag adorner
Right-click mouse while still dragging

Seen on Win10 1909 in Showcase.WPF.DragDrop.exe
Can also reproduce by clicking mouse wheel.
Note: if 'CanDragWithMouseRightButton' selected, then clicking the left-button also causes this bug.

Thank you for this info! I've have users of our app complain about adorners that don't go away, but I haven't been able to reproduce it. I'll look into making a fix sometime this week.


Routing over Reticulum?

Describe the bug
Transport level security vulnerabilities: https://www.youtube.com/watch?v=LBd-ewnjn8M
To reproduce
Steps to reproduce the behaviour:
See: https://www.youtube.com/watch?v=LBd-ewnjn8M
Expected behaviour
Messages are secure
Additional context
Where secure messaging is needed could you route the packets over reticulum: https://unsigned.io/projects/reticulum/

So I have seen Reticulum before and have been meaning to take a deeper dive. Looked really interesting. Been reading Mark Qvist's blog every now and then. @samuk have you used Reticulum before?

TBH I've been waiting for the Android & ESP32 stuff which has only come out in the last week or so. I plan to give it a go soon,

Definitely share your thoughts when you try it out. I think I would like to bring this up as a topic during our next town hall


MarkerCluster options

Here we should be able to pass the markerClusterGroup options.
Something like :
...

interface Props {
  leafletObject: Map
  markers: MarkerProps[]
  options: MarkerClusterGroupOptions
}

export const useLMarkerCluster = async (props: Props) => {
  // Get Leaflet from the window object
  const L = window.L

  // Lazy-load leaflet.markercluster
  // Importing it at the top level will cause errors because it could be loaded before the Leaflet library
  const { MarkerClusterGroup } = await import('leaflet.markercluster')

  // Initialize marker cluster
  const markerCluster = new MarkerClusterGroup(props.options)

Oh that's true ! I'll work on that as soon as possible 👍

Released in v1.2.5 👍

Hi! Thanks for your works on it!
The definition of the interface MarkerClusterGroupOptions is missing many options. All options seem visible here : https://github.com/Leaflet/Leaflet.markercluster/blob/c0f055bd5dcd1d6a733090d0cf024d7362d77bc8/src/MarkerClusterGroup.js#L7
Sadly, leaflet.markercluster isn't typed. But you can find all types for the option here : https://github.com/DefinitelyTyped/DefinitelyTyped/blob/61576c3c5bcb6518f86224fcb0a2314ac68e4ebb/types/leaflet.markercluster/index.d.ts#L36
Instead of re-creating markercluster group options definition, maybe it will make more sense to use DefinitelyTyped. It will be possible to import types definition for leaflet and leaflet.markercluster. Ex:
pnpm add -D @types/leaflet
pnpm add -D @types/leaflet.markercluster

So MarkerClusterGroupOptions will be available in leaflet type :
import type { MarkerOptions, Map, Marker, MarkerClusterGroupOptions } from 'leaflet'

This is merged and released in v1.2.6, thanks @ManUtopiK !


Merge pull request #170 from sourceryinstitute/add-install-script-boi…

…lerplate
Boilerplate installation scripts & mv install_prerequisites

Current coverage is 38.73%

Merging #171 into add-install-script-boilerplate will not affect coverage as of a82eabd

@@            add-install-script-boilerplate    #171   diff @@
==============================================================
  Files                                    2       2
  Stmts                                  901     901
  Branches                                 0       0
  Methods                                  0       0
==============================================================
  Hit                                    349     349
  Partial                                  0       0
  Missed                                 552     552

Review entire Coverage Diff as of a82eabd

Uncovered Suggestions

+1.90% via src/mpi/mpi_caf.c#1906...1922
+1.56% via src/mpi/mpi_caf.c#2004...2017
+1.56% via src/mpi/mpi_caf.c#1559...1572
See 7 more...

Powered by Codecov. Updated on successful CI builds.


Can't make triple column

Skelet uses a 16-column grid system instead of the regular 12-column one. This gives fewer grid options, since 16 is only divisible by 1, 2, 4 and 8. 12 is divisible by 1, 2, 3, 4 and 6 giving a better range. How can I solve this issue?

Hi @rockthrower69,
sorry for the late reply, busy period.
16 is the Max supported number of columns you can have in Skelet.
If you need a 12-column grid you can simply set it like this:
<grid columns="12">
<grid columns="12">
<c span="1-6>Hey</c>
<c span="7..">There</c>
</grid>

here's a JSfiddle


Adding Landscape Lock

Fixes #204
Describe the changes you have made in this PR -
Adding Portrait Lock
Screenshots of the changes (If any) -
Tablet View :-

Mobile View:-

Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

@Prerna-0202 @Arnabdaz PTAL
Thank You : )

Closing this PR this because of branch related issues.


ansible-navigator run can't directly access a playbook provided by a collection

ISSUE TYPE

Bug Report

SUMMARY
ansible-playbook is able to directly access a playbook provided by a collection. Trying to do the same thing with ansible-navigator run results in the tool trying to resolve it as a filename.
ANSIBLE-NAVIGATOR VERSION

ansible-navigator 2.2.0

STEPS TO REPRODUCE

Create a collection with a playbook located in the playbooks/ directory
Try to run ansible-navigator run namespace.collection.playbook_name --eei container-with-collection-installed

EXPECTED RESULTS

Playbook runs
ACTUAL RESULTS

Fully-qualified playbook name gets parsed as though it were a file and an error like the following is returned:
ERROR! the playbook: /working/dir/namespace.collection.playbook_name could not be found

ADDITIONAL INFORMATION

Running ansible-playbook namespace.collection.playbook_name does succeed with the collection installed locally, so this is something Ansible is supposed to be able to do.
Can be worked around by creating and running a one-line playbook like the following
- import_playbook: namespace.collection.playbook_name

Thanks for this! We'll look into it and post back with findings


Can't build the solution

Describe the bug
I am following the instructions given in https://docs.demo.saas.apptoku.com/getting-started/run-project/run-new-project
I then get an error like bellow and my containers are not starting
core: Warning: run-commands command "nx run-many --target=compose-build-image --projects=backend,workers" exited with non-zero status code
make: *** [Makefile.base.mk:37: up] Error 1
Steps to reproduce
make up
System Info
python --version
Python 3.11.5

docker --version
Docker version 20.10.24, build 297e128

node --version
v18.17.1

pnpm --version
8.7.0

OS version
cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

Logs
The error log is attache below.
my-wellformatted-outputl.pdf
sri-saas.txt
Validations

[X] Follow our Code of Conduct.
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.

So far, anything related to this problem that I found says you should reinstall your docker on your machine.
When you look into logs it says something about permissions.
Do you have a fresh docker installation?
How did you install exactly?
Is is possible you used snap to install docker? I've seen this being given as a reason.

yes, I used snap to install docker. What's the other way I need to install docker on ubuntu to preven this happening?

Using your tip, I managed to get the docker build working. The snap installation of docker does not work. (as described here https://stackoverflow.com/questions/61876116/how-to-remove-a-snap-application-docker-completely) .
A few missing steps.

docker-compose (v1) has to be in place. Not docker compose (v2)
docker volume create --name=saas-web-backend-db-data  needs to be run before Make up command

I can see my containers are running but I can't access them through http://locahost:5001

got this working. I pruned all docker containers, images and volume then ran the steps with default values.


input code over study break to make a start

Note to anyone reading this: the code will probably be really bad. But I'm not an expert. Pls forgive. :-)

get on with it


能否折叠表格?

能否折叠表格?
比如我的入参是一个对象A, A中有个属性是对象B.
通常情况下, 会写两个表格, 并在A中通过锚点跳转到B.
能否做到点击表格中A的属性B来弹出B的表格?

Translation:
Can I collapse the table?
For example, my input parameter is an object A, and an attribute in A is object B.
Normally, two tables will be written, and in A will jump to B through the anchor point.
Is it possible to click the attribute B of A in the form to pop up the form of B?


[Mouse Highlighter] Show highlight on mouse drag

Summary of the Pull Request
What is this about:
On mouse drag, have the highlight follow the pointer.
To land on top of https://github.com/microsoft/PowerToys/pull/14496
@niels9001 , could you please give it a test to see if this is what you intended?

@jaimecbernardo Can you add a .GIF?

Thanks a lot @niels9001 .
I'll just mash this into the currently opened PR ;)


Configure Azure Pipelines

This pipeline configuration builds the application and produces a build for the Release configuration.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: BaylorBrad sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Updated the usage of the grep utility

Updated the usage of the grep utility so it works both on Linux and BSD (MacOS).
Tested on Mac using:
grep (BSD grep, GNU compatible) 2.6.0-FreeBSD
ggrep (GNU grep) 3.8
Not tested on Linux

Hey @kamilgryniewicz thank you for the contribution!
I will check it out on both Mac and Linux.

@kamil tested, looks good, thank you! 🎉

@yankeexe my pleasure, thanks!


Attaching duckdb types while fetching data

When I fetch data from duckdb wasm, I guess type of data base on Apache Arrow Field. Sometimes, type couldn't be guessed exactly.
Eg, For this query "SELECT -9223372036854775809;".

DuckDB Type HUGEINT
Apache Arrow store it as DECIMAL

Another example: JSON type is store as Utf8 type in Apache Arrow.
Please let me know if having any way to retrieve exactly type from duckdb.

If you insert into a table, you could query the table metadata!
https://duckdb.org/docs/sql/information_schema

Hi @Alex-Monahan ,
For Select from table, I could retrieve data types from Duckdb.
I mainly focus on SELECT from values. Eg: SELECT interval 1 day;
Thanks


if Attachment.HTMLRelated is set, mark attachment as inline

Properly setting "Content-Disposition: inline" for attachments marked as
related. Otherwise some email clients (e.g. Thunderbird) are not showing
inline images correctly.

This seems reasonable to me. I appreciate you going the extra mile to keep things backwards compatible. 💚


VSCode Markdown Parser Bug for Extensions README.md

I am about to launch a new theme next week, called Shades of Purple.
The issue is that the README.md file is being parsed completely fine at GitHub and at VSCode Marketplace. But inside VSCode the markup is not being parsed :(
Take a look at this screenshot.

VSCode Version: 1.21.1

Environment:
  OS: macOS Sierra 10.12.6
  CPU: x64 Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
  Node: 9.7.1
  Yarn: 1.5.1
  npm: 5.7.1
  Watchman: Not Found

Browsers:
  Chrome: 65.0.3325.181
  Chrome Canary: Not Found
  Firefox: 57.0.1
  Firefox Developer Edition: 57.0
  Firefox Nightly: Not Found
  Safari: 11.0
  Safari Technology Preview: Not Found

Bug in marked library. Opened https://github.com/markedjs/marked/issues/1195 to track this upstream

Fixed by picking up a new copy of marked.
For verification, create an extension that has this README.md: https://raw.githubusercontent.com/ahmadawais/shades-of-purple-vscode/2828bb0c51ba950790cf3e084475d125205a1921/README.md
Some headers and other content used to show up raw, i.e. as ### header in the output. This should not happen in fixed vscode versions


Add publication_name config setting

We need to support this field in go-fivetran SDK first.

I didn't know how to request that, so I was hoping this could kick off an internal request :)

I wasn't sure where the sdk code was located. I found it and added a PR: https://github.com/fivetran/go-fivetran/pull/34. Hopefully that helps 🤷

I wasn't sure where the sdk code was located. I found it and added a PR: fivetran/go-fivetran#34. Hopefully that helps 🤷

Thank you! I will merge all the things.
Once go-fivetran new version will be released I'll update dependency here and merge it too.

@jgeurts I've released a new version - so you can update go-fivetran version here

@jgeurts all is set! Thank you!
testing fails - but it is OK, we haven't set the testing creds for external PR's for e2e tests.


Set non zero query parameter for end in series query API

Current behavior

When we query for series metric, we pass zero as value for end query key.
Ex : http://localhost:9090/api/v1/series?match[]=my_metric&start=1651129852&end=0'
The error is similar to this.
While this works on self hosted prometheus, I faced an issue where seriesQuery started throwing error on managed prometheus provided by GCP.
It worked after I started using custom build passing valid end time.

/remove-lifecycle stale

Thank you for the contribution @srinidhis94 :)
/lgtm
/approve
/ok-to-test

/remove-lifecycle stale

@srinidhis94 will you have some time to finish this work? It seems to be waiting to be rebased and the tests need to be fixed

Hey I'll check this weekend .

Hi @srinidhis94
Do you think you'll be able to rebase this PR?


[Model] Support SmolVLM

soon to be released model by Hugging Face. We would like to mention that it is integrated in VLMEvalKit when we launch it. Thank you!

Cool, will help with evaluation after the model is released.


Add latest sensor datapoint to graphs

Easiest way to test this is to set the history length to 1 week on the dashboard, set graph type to Acceleration Z, then rotate a sensor and the graph should reflect that when the data updates.

QA passed


feat: GridServer is now unsupproted

Did we deprecate it before?

Did we deprecate it before?

Well it was said that we would remove it, but i'm waiting on @ddubuc 's confirmation

Bye Bye GridServer RIP


[BEAM-8996] Auto-generate pipeline options documentation for FlinkRunner

The documentation of the pipeline options on the runner page easily becomes
outdated. In order for it to stay up to date, we auto-generate the documentation
from the FlinkPipelineOptions class. This is done for both Java and Python.

Thanks for the review @lgajowy! I agree that we can generify this further. This is the first step to keeping the Flink pipeline options up to date. We can work together with the other runner authors to adapt this to their needs.

Unrelated failure in Cassandra test:
12:34:36 FAILURE: Build failed with an exception.
12:34:36
12:34:36 * What went wrong:
12:34:36 Execution failed for task ':sdks:java:io:cassandra:test'.
12:34:36 > There were failing tests. See the report at: file:///home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit/src/sdks/java/io/cassandra/build/reports/tests/test/index.html

@mxm this PR was not accepted but it was merged anyway - we shouldn't act this way. :(

I thought I had addressed all your comments. That's why I merged. The only open discussion point seems to resolve about code style, which can be subjective. I definitely always check if any blockers are still open before merging. I merged fast here, though there was sufficient time for the reviewers.
Please consider, this is not a mission-critical code path. I'm not changing anything related to the Beam runtime. This just generates docs which always became outdated before. I'm essentially doing this on the side and if the review discussion is only about code style, I don't see a reason to block this in case of an isolated class, which is solely used to generate a table in the documentation.
While I'm also guilty of this occasionally, it is worth to take a step back and take a look at the value of a change, instead of insisting on details, which might make it better subjectively, but do not provide more value, or even increase the maintenance costs. I believe I've just recently reviewed some of your PRs, where I tried to live up to that standards.

Please don't get me the wrong way - I really think this is a good change
and I believe that in my comments I suggested improvements not only to
style but also to functionality (template vs generating code using
system.out.println). I wouldn't say those are subjective. What is more, in
my opinion, after a review round or two we could get even more value from
this pr.
The reason why I think merging this should wait for the reviewer's approval
(not necessarily mine) is that it would prevent missunderstandings like the
one we have right now. I agree this is not "mission critical" code but I
still think we shouldn't judge that on our own.
pt., 20 gru 2019, 14:44 użytkownik Maximilian Michels <
<EMAIL_ADDRESS>napisał:

I thought I had addressed all your comments. That's why I merged. The only
open discussion point seems to resolve about code style, which can be
subjective. I definitely always check if any blockers are still open before
merging. I merged fast here, though there was sufficient time for the
reviewers.
Please consider, this is not a mission-critical code path. I'm not
changing anything related to the Beam runtime. This just generates docs
which always became outdated before. I'm essentially doing this on the side
and if the review discussion is only about code style, I don't see a reason
to block this in case of an isolated class, which is solely used to
generate a table in the documentation.
While I'm also guilty of this occasionally, it is worth to take a step
back and take a look at the value of a change, instead of insisting on
details, which might make it better subjectively, but do not provide more
value, or even increase the maintenance costs. I believe I've just recently
reviewed some of your PRs, where I tried to live up to that standards.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/10424?email_source=notifications&email_token=AAOXWDMNZQVKZD2DBUCOW43QZTD4VA5CNFSM4J5HDT7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHM6RRQ#issuecomment-567929030,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOXWDOZ6JLH5ZS5V2KZ7FLQZTD4VANCNFSM4J5HDT7A
.

@mxm thanks for working in this, it is a good improvement.
However, regarding the merge if this PR, I would prefer more time is given to allow for comments and consideration of comments. We should wait for an approval, not just assume everything was reviewed and addressed.
There is no reason to rush this type of change.

 An approval is not strictly necessary when a change provides an improvement or feature which is undoubtedly useful or necessary for the success of the project.
Last time I checked we had close to 200 open pull requests. One reason is that there are simply not enough people reviewing pull requests. Another reason is that the conversation in the PRs progressively slows down because of questions, requests, wishes, or nitpicks. I know some people who have given up contributing to Beam because of too much wishful thinking in the conversations. That is not to say that those comments are not valid, but there is a trade off between in the dimensions of time, complexity, and maintenance that should be carefully thought about when reviewing PRs.
I always consider all the comments I receive, but I'm only willing to compromise my time to a degree. If a suggestion does not convince me or it simply does not provide enough benefit for the time and mental energy it would to take to implement it, I won't incorporate it, but reject it politely.
The feature here has already at least twice led to an updated documentation. It works flawlessly and I'm more than happy to see it being improved, generalized, extended to use templating, or xyz. I'm just not going to do this myself at the moment because I would like to focus my mental energy on other work.
Contributing to Beam should be as hard as necessary but not any harder.
On January 2, 2020 7:39:58 PM GMT+01:00, Thomas Weise<EMAIL_ADDRESS>wrote:

@mxm thanks for working in this, it is a good improvement.
However, regarding the merge if this PR, I would prefer more time is
given to allow for comments and consideration of comments. We should
wait for an approval, not just assume everything was reviewed and
addressed.
There is no reason to rush this type of change.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/apache/beam/pull/10424#issuecomment-570300807


fix/update-api-docs-readme

updating api-docs readme with additional installation of npm packages
Closes #
Describe your proposed changes here.
add installation guide for redoc, redoc-cli and redocly-openapi-cli

[ ] Signed the InfluxData CLA
(if necessary)
[ ] Rebased/mergeable

@jstirnaman
how many people should I be adding to these types of PR requests?
is there a specific format for the branch title i should be following?
do we usually merge directly into master?

@jstirnaman how many people should I be adding to these types of PR requests? is there a specific format for the branch title i should be following? do we usually merge directly into master?

For docs content PRs, we ideally want one Docs team member, the Product Manager, and maybe the engineer involved. For docs technical/pipeline changes, request Scott or me (esp. for API stuff).

The team shouldn't need to install these packages; the scripts use npx to fetch them and execute the commands. I would be OK with making your addition a developer's note.
Also, I forgot that I've made some updates to this README and scripts in the v2.2 branch (that we expected to release a while ago).
Sorry for the confusion.

No worries, thanks for the clarification. I didn't see your updates and I'll close out / remove this PR since my changes unnecessary after seeing the v2.2 branch


Fix B+Tree Index or Remove it

I think that we should keep the BTreeIndex around in case we need something more simple than the BwTreeIndex. But it currently does not work with any of our test cases. I think that @jarulraj also has a SkipListIndex somewhere too.
We should either fix these indexes and re-enable them in IndexPerformanceTests or remove them.

Here are the errors if you re-enable INDEX_TYPE_BTREE in the IndexPerformanceTests:
/home/pavlo/Documents/Peloton/Github/peloton/test/performance/index_performance_test.cpp:288: Failure
Value of: location_ptrs.size()
  Actual: 0
Expected: num_thread * num_key
Which is: 1048576
[INFO ] [test/performance/index_performance_test.cpp:293:TestIndexPerformance()] 2017-01-03 09:46:47 - InsertTest1 :: Type=BTREE; Duration=0.21
[INFO ] [test/performance/index_performance_test.cpp:314:TestIndexPerformance()] 2017-01-03 09:46:47 - DeleteTest1 :: Type=BTREE; Duration=0.79
/home/pavlo/Documents/Peloton/Github/peloton/test/performance/index_performance_test.cpp:330: Failure
Value of: location_ptrs.size()
  Actual: 0
Expected: num_thread * num_key
Which is: 1048576
[INFO ] [test/performance/index_performance_test.cpp:335:TestIndexPerformance()] 2017-01-03 09:46:47 - InsertTest2 :: Type=BTREE; Duration=1.02
[INFO ] [test/performance/index_performance_test.cpp:356:TestIndexPerformance()] 2017-01-03 09:46:48 - DeleteTest :: Type=BTREE; Duration=1.74
[  FAILED  ] IndexPerformanceTests.BTreeMultiThreadedTest (1746 ms)

The B+Tree index could be used as a backup plan, in a sense that if the fanout for a key in a table is too large (e.g. > 100) then BwTree has miserable performance on Insert() and Delete() because it has to keep operations atomic on a large set of records. In this case maybe using a B+Tree might be beneficial. This could be made as a knob for tuning and let the user make decision

Right, but we would have to modify it to make it concurrent.

Large fan out is important in disk based b tree indexing. That's because
processing the index page is cheap compared to a disk read. (100th the
cost). So the equation was log(n,b) disk reads.
But for memory based indexes you now have to count memory cycles. And
processing an index node takes cycles also. So it is now something like
log(n,b)*b memory reads.
On Tue, Jan 3, 2017 at 5:18 PM Andy Pavlo<EMAIL_ADDRESS>wrote:

Right, but we would have to modify it to make it concurrent.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cmu-db/peloton/issues/453#issuecomment-270253558, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABS8HNVJfMZnEDaeUB-WkWNCcishzAVgks5rOtc_gaJpZM4LZrhP
.


Feature Add admin set partition version statement

Proposed changes
Last week, we encountered a situation where we had to recover master's metadata from image without bdbje log.
After recovering the master, we found that the partition visible version of the master's metadata was inconsistent with the actual replica in Be.
This commit add a statement to modify partition visible version.

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

some docs still need to be updated

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.74 seconds
stream load tsv:          536 seconds loaded<PHONE_NUMBER>9 Bytes, about 133 MB/s
stream load json:         21 seconds loaded<PHONE_NUMBER> Bytes, about 107 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select:          29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size:<PHONE_NUMBER>6 Bytes

run buildall

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.13 seconds
stream load tsv:          542 seconds loaded<PHONE_NUMBER>9 Bytes, about 131 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          66 seconds loaded<PHONE_NUMBER> Bytes, about 15 MB/s
stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select:          29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size:<PHONE_NUMBER>6 Bytes

@morningman

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.07 seconds
stream load tsv:          542 seconds loaded<PHONE_NUMBER>9 Bytes, about 131 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          66 seconds loaded<PHONE_NUMBER> Bytes, about 15 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.5 seconds inserted 10000000 Rows, about 338K ops/s
storage size:<PHONE_NUMBER>6 Bytes

run buildall

run clickbench

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.97 seconds
stream load tsv:          536 seconds loaded<PHONE_NUMBER>9 Bytes, about 133 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size:<PHONE_NUMBER>2 Bytes

revert this PR in branch-2.0 since it modify fe metadata OperationType and cause rollback from 2.0.2 to 2.0.1/2.0.0


Make cassandra schema update tool idempotent

Expected Behavior
When schema update partially failed, a retry should succeed.
Actual Behavior
If the table is created but the version was not updated, the retry would keep failing on table already exists error.

Thank you for reporting this issue. This is a known issue and we are actively working on fixing it.
For now, you can work around this issue by:

Manually updating the schema version. You can find more information on how to do this here.
Using the Upsert option when creating the table. This option will overwrite the existing table if it already exists. You can learn more about this option here.

We appreciate your patience and understanding as we work on a permanent solution.

Thank you for reporting this issue.
This is a known issue with Temporal.  We are working on a solution to retry schema updates that partially failed.
In the meantime, you can work around this issue by manually updating the schema version in your Temporal database.
For more information on updating schema versions, please refer to the documentation:  https://docs.temporal.io/docs/temporal-server/schema/schema-management

Thank you for reporting this issue.  I understand that you are experiencing an issue where schema updates fail to retry when the table is created but the version is not updated.
This is a known issue and is tracked in Temporal issue tracker. You can follow the issue for updates and potential solutions.
In the meantime, you can workaround this issue by ensuring that the table is created and the version is updated in a single transaction. This can be achieved by using a transaction management system like MySQL or PostgreSQL.

Thank you for reporting this issue.
We understand that this behavior can be frustrating.
Currently, Temporal does not retry schema updates if the table already exists. This is because Temporal uses a single transaction to both create the table and update the schema version. If the table already exists, the transaction fails.
A possible workaround would be to use a separate transaction to check if the table already exists and only create it if it doesn't. If the table already exists, you could then update the schema version in a separate transaction.
For more details on schema management in Temporal, please refer to the Temporal documentation.


enhancement Move storage policy info from show create table to show partitions (#39923)

Proposed changes
pick: #39923

run buildall

run buildall

run feut


Automatic GraphEdit node connection hot zone height 2

The graph editor in 4.0 has a too-small hot zone for connecting graph nodes, which can make snapping unintuitive. This PR changes how hot zones are calculated and now automatically sets their height based on the accompanying UI elements instead of using a constant.
red is what we currently have, green is what I am proposing:

However, this PR exacerbates another problem with the GraphEditor: When the input is filtered it doesn't take child nodes into consideration, which means that hot zones can block the user from editing fields below them. To fix this we would need to pass the mouse down event to the children first before the GraphEditor handles it, and I have no clue how to properly do that.
Would appreciate it if someone could help me with this. Perhaps I am missing something obvious.
This PR superseeds #49705, for more information please see godotengine/godot-proposals#2790.
TODO

[ ] figure out how to handle overlapping hot zones and input field ui elements

if impossible just use differents hot zones for drag and drop.

[ ] add highlights to connections on hover

This is still design approved, but of course the details need to be solved.

When the input is filtered it doesn't take child nodes into consideration, which means that hot zones can block the user from editing fields below them. To fix this we would need to pass the mouse down event to the children first before the GraphEditor handles it, and I have no clue how to properly do that.

In theory you can have two different zones for picking from a port and for dropping a connection. The former would be tighter and smaller and won't overlap with controls, the latter would be bigger and more forgiving.

That's what I will do should I be unable to come up with a better solution. I am currently working my way up through Godot's input code, but so far I haven't found anything that looks promising.

Superseded by #61515.


Styled blocks should use the (Context, AttributeSet, style: Int) constructor where possible

I would love to see this fixed! I would be willing to take a look at this but I am not really sure where to get started. From what I gather the Factories for views are generated here: https://github.com/Kotlin/anko/blob/0f99e9fd5e4c8b44f5df645206005d3e2d7d958d/anko/library/generator/src/org/jetbrains/android/anko/render/viewRenderers.kt#L107-L112
So I guess you could generate code using the 3-argument constructor instead but I am really unsure of the details (like where to take the arguments for the constructor from). I will look into this some more but if anyone could provide a few details it would certainly help a lot.

No I have not made any progress, sorry. With some guidance I'd still be up for but I am mostly working on unrelated projects currently.

I investigated in applying xml styles programmatically and saw that it's only possible with the 4 arguments View constructor available only on select views on API 21+, excluding Android support libraries/AndroidX views, so there's no viable option. The third parameter expects an attribute resource id, not a style resource id, so it only works for styles defined in the theme like buttonStyle.
Ok, Airbnb engineers found an over-engineered way to apply xml styles programmatically: https://github.com/airbnb/paris

@hatzel any progress on this bug?

Nope, don't see this happening anytime soon for me. Not building apps at the moment and I'd probably be using flutter for new projects.

oh sad :( , btw I was able to style my button using themedButton(), but any background style is not applied to it..

Just FYI, it's possible to hack the theme a little to be able to use xml styles. That's what I'm doing in Splitties Views DSL (and the Material and AppCompat extension modules).


Hot reload not publishing changes

Laravel Mix Version: 0.11.4
Node Version:  v6.10.3
NPM Version: 3.10.10
OS: Ubuntu / Docker container

Description:
Im running laravel with a laravel-mix configuration inside a docker container, but cant seem to get hot reload to work, I can access the static js file, but while running hot and changing files, and even reloading manually the page Im not getting the changes even when the console output from npm run hot seems to be compiling the files.
Steps To Reproduce:
webpack.mix.js

mix
	.options({
		processCssUrls: false
	})
	//.sourceMaps()
	.js('resources/assets/js/admin.js', 'public/js')
       .sass('resources/assets/sass/admin.scss', 'public/css')

Package.json hot reload configuration
"hot": "cross-env NODE_ENV=development node_modules/webpack-dev-server/bin/webpack-dev-server.js --host=<IP_ADDRESS> --public=<IP_ADDRESS> --inline --hot --config=node_modules/laravel-mix/setup/webpack.config.js",

U can see I made some changes so the host machine can access the webpack-dev-server
In my index I import the js this way (maybe this could be the problem)
<script src="http://<IP_ADDRESS>:8080/public/js/admin.js"></script>

I have checked that I have no WebSocket connections open with the server, even when the websockets code seems to be in my admin.js file
/******/ (function(modules) { // webpackBootstrap
/******/ 	function hotDisposeChunk(chunkId) {
/******/ 		delete installedChunks[chunkId];
/******/ 	}
/******/ 	var parentHotUpdateCallback = this["webpackHotUpdate"];
/******/ 	this["webpackHotUpdate"] =
/******/ 	function webpackHotUpdateCallback(chunkId, moreModules) { // eslint-disable-line no-unused-vars
/******/ 		hotAddUpdateChunk(chunkId, moreModules);
/******/ 		if(parentHotUpdateCallback) parentHotUpdateCallback(chunkId, moreModules);
/******/ 	} ;

just a small part of the start of the bundled file
the file also is being loaded on the page

theres some extra configuration needed for this to work? or could be a problem brought by the webpack server configuration?

You should include <script src="http://<IP_ADDRESS>:8080/js/admin.js"></script> for hot reloading

Thanks, I'll test it out tomorrow and Ill close the issue if it works correctly.

seems to workbut still having some config issues
/#/:1 XMLHttpRequest cannot load http://<IP_ADDRESS>/sockjs-node/info?t=1497480609221. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://admin.laravel.dev' is therefore not allowed access.  No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://admin.laravel.dev' is therefore not allowed access. The response had HTTP status code 404.
2admin.js:13923 GET http://<IP_ADDRESS>/sockjs-node/info?t=1497480612786
AbstractXHRObject._start @ admin.js:13923
(anonymous) @ admin.js:13812
/#/:1 XMLHttpRequest cannot load http://<IP_ADDRESS>/sockjs-node/info?t=1497480612786. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://admin.laravel.dev' is therefore not allowed access. The response had HTTP status code 404.

I see some weird behaviour http://<IP_ADDRESS>/sockjs-node/info?t=1497480609221 when it should be requesting it trough port 8080? seems like Ill have to modify the webpack server configuration but not a clue where to start

Removed previous comment
The suggested change and changing the hot reload script to
"hot": "cross-env NODE_ENV=development node_modules/webpack-dev-server/bin/webpack-dev-server.js --host=<IP_ADDRESS> --public=<IP_ADDRESS>:8080 --inline --hot --config=node_modules/laravel-mix/setup/webpack.config.js",

helped a bit, but getting still this output when running
Project is running at http://<IP_ADDRESS>:8080/
webpack output is served from http://localhost:8080/
404s will fallback to /index.html

this means when it requests the  hot-update.json it request it to localhost (which fails), ill keep trying to see if I found a solution

Hi @alejodiazg , I was following your approach as well and hit the same walls :(
I finally get my HMR working!
The last piece missing for this config to work was the 'localhost:8080' still used to load the .hot-update.json. And it was apparently due to the generated manifest.js still having 'localhost:8080' from the output config in the laravel-mix WebpackConfig.js file
I just added:
output: {     publicPath: Mix.isUsing('hmr') ? 'http://my-vm-ip:8080/' : '', },
to my local webpack.mix.js > mix.webpackConfig, and actually worked!
Hope it could be helpful,
Cheers,

@jonatangerman thanks ill try it in a few days to see if it works, right now im in my vacations.