generate a navmesh from an heightfield

possible solution from recast: steps 3 to 5 of https://9631ec60-a-62cb3a1a-s-sites.googlegroups.com/site/recastnavigation/MikkoMononen_RecastSlides.pdf

watershed partitioning to find regions
contour the regions to get polygons
triangulate the polygons to get a trimesh

The recast slides linked aren't working for me, but here's an alternative algorithm.
https://www.cs.cmu.edu/~garland/scape/scape.pdf
Implementations:
https://github.com/fogleman/hmm (MIT)
https://github.com/mapbox/delatin (ISC, based on hmm)

Thanks!
I had a local copy of the slides, here they are: MikkoMononen_RecastSlides.pdf


[ Error ] on run-midi command

Hi. Thank you for sharing your code.
I run your code as you said and use the below command, replacing the $LOGDIR with the model directory.
However, I got this error
Replace $LOGDIR with the model directory, containing the model.yaml configuration file (e.g. one of the directories under experiments).
To run the trained model on a single pair of MIDI files, use the run-midi command, e.g.:
python -m groove2groove.models.roll2seq_style_transfer --logdir $LOGDIR run-midi --sample home/project/Desktop/Groove2groove/groove2groove/experiments/v01_drums_vel/v01_drums_vel/latest_checkpoint --softmax-temperature 0.6 content.mid style.mid output.mid

When i run this command, the following error occured.
OSError: no MTrk header at start of track

Your command looks incorrect – you are passing the path to your checkpoint as an argument and it's probably being interpreted as a MIDI file, hence the error. Just remove the path to the checkpoint and you should be fine.

@cifkao, I have followed all the steps in the main run file. I would like to list out the steps that i have performed during the running process.

Created a conda environment (Pip env not done as it is mentioned inside the environment.yml file)
Installed the package with (pip install './code[gpu]')
We went to synth folder and ran the 4 commands (download.sh, train, val and test)
After the above commands were finished, we ran the following command
 python -m groove2groove.models.roll2seq_style_transfer --logdir $LOGDIR run-midi --sample --softmax-temperature 0.6 <inputfile1>.mid <inputfile2>.mid output.mid

Not sure what to run in sequence, After running the above command i got the following error:

Have you set $LOGDIR to your model directory? E.g.:
LOGDIR=/path/to/v01_drums_vel

Also, have you extracted the checkpoint to that directory? The directory specified here should contain the files latest_checkpoint, latest.ckpt-24361.data-00000-of-00001 etc.

By the way, you do not need to prepare the training dataset if you are not training a model.

@cifkao, I was providing wrong checkpoint filename. Now it is working.
Thanks :)

@cifkao, Can you help me on how can i implement this idea in my web app interface as you did on the supplementary website demo?


Marlin fp8

Fixes https://github.com/huggingface/optimum-quanto/issues/238

Compared to calling the kernel in isolation, end to end benchmark is slower using quanto, due to many overheads.

A separate PR may be needed to remove some of them first (or after this one - if we don't care about perf).

Compared to calling the kernel in isolation, in my config end to end benchmark is slower using quanto, due to many overheads.
A separate PR may be needed to remove some of them first (or after this one - if we don't care about perf).
I'll try with a larger model / different GPU see if python latency is hidden or not.
@dacorvo Am I doing something wrong here?

Can you explain how you deduce there is an overhead here, as compared to the standard call from inside torch.nn.functional.linear to torch.ops.aten.mm ? BTW, what tool did you se ?

For sure. Last week I was using quanto benchmark script with
python evaluate_model.py --device cuda --metric decode-latency --quantizer quanto --weights float8_e4m3fn --activations none --dtype fp16 --batch_size 1
python evaluate_model.py --device cuda --metric decode-latency --quantizer quanto --weights none --activations none --dtype fp16 --batch_size 1

and did not have speedups, but I am unsure why. From the profile, I assume it is some overhead from quanto's dispatch, but could be something else.
The following script gives the profile below: https://gist.github.com/fxmarty/1aff830cdd57aa650412f34bd4076b3b, comparing a linear call through a quanto module, through direct torch.ops.quanto_ext.fp8_gemm call, or through fp16 torch.nn.functional.linear.

Benchmarking the same (https://gist.github.com/fxmarty/e449c55e4a1dbf9b1657f395aa542eb4) for a single linear, there indeed seems to be a 10-25% overhead (in my problem setup) from torch_function, multiple torch.library dispatch, etc (first and second col). Still faster than native fp16 though, so have to see

@dacorvo Running python evaluate_model.py --device cuda --metric decode-latency --quantizer quanto --weights float8_e4m3fn --activations none --dtype fp16 --batch_size 1 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 between https://github.com/huggingface/optimum-quanto/pull/241/commits/f45711c58b78a5971143e915ef0864a61148138a and https://github.com/huggingface/optimum-quanto/pull/241/commits/d52e44a9c99ffc922e4bdda266100132a6348780 (avoid dispatch when not necessary),
I get:

For f45711c58b78a5971143e915ef0864a61148138a: 11.855 ms / token
For d52e44a9c99ffc922e4bdda266100132a6348780: 9.964 ms / token

To me this is a substantial enough difference to care.

@dacorvo Running python evaluate_model.py --device cuda --metric decode-latency --quantizer quanto --weights float8_e4m3fn --activations none --dtype fp16 --batch_size 1 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 on my laptop between b8dbdf0 and d52e44a (avoid dispatch when not necessary),
I get:

For b8dbdf0: 11.855 ms / token
For d52e44a: 9.964 ms / token

To me this is a substantial enough difference to care.
Edit: on A100, for llama 3 8B, with end-to-end (prefill + decode), I do get with mixed fp16+fp8:

For b8dbdf0: 40.350 ms / token
For d52e44a: 33.952 ms / token

^ this is not faster than fp16-fp16 in any case (getting 32.681 ms), weird

That is indeed a substantial difference. It would be great if you could:

run the unit tests to identify the extent of the consequences of your changes on other features,
maybe clean up the branch, group your commits into meaningful groups of changes and use conventional commits, so that we can run the CI.

the restriction to per-tensor scale makes the kernel unusable

I think this can be easily changed in a later PR. Neither vllm nor tgi support per-column scales, and yet they achieve nice memory reductions & speedup.

On par with my tests in TGI, we have decent speedup with this kernel only when using cudagraph. I don't really explain myself why.
Using transformers + A100 + 8B model & measuring decode latency at batch size = 1:
fp8 + compile static
Peak memory during benchmark: 9.7056 GB
Average decode latency per token: 8.833560943603516 ms

fp16 + compile static
Peak memory during benchmark: 15.3295 GB
Average decode latency per token: 12.276221656799317 ms

fp8 + eager
Peak memory during benchmark: 8.5034 GB
Average decode latency per token: 33.871917724609375 ms

fp16 + eager
Peak memory during benchmark: 14.9994 GB
Average decode latency per token: 32.10729331970215 ms

Perplexity bench is very decent as well


Update instructions for setting environment variable

Before submitting this PR

Description: This updates the instructions for testing since the current instructions do not include how to define environment variables in Windows.
Jira task / GitHub issue: DDS-1528
How to test: Use the specified commands in Windows cmd and PowerShell. (Already tested)
Type of change: Check the relevant boxes in the section below
Add docstrings and comments to code, even if you personally think it's obvious.

What type of change(s) does the PR contain?

[ ] New feature

[ ] Breaking: Please describe the reason for the break and how we can fix it.
[ ] Non-breaking

[ ] Bug fix

[ ] Breaking: Please describe the reason for the break and how we can fix it.
[ ] Non-breaking

[ ] Security Alert fix
[x] Documentation
[ ] Tests (only)
[ ] Workflow

Checklist

Sprintlog

[x] Added
[ ] Not needed (E.g. PR contains only tests)

Rebase / Update / Merge from base branch (the branch from which the current is forked)

[ ] Done
[x] Not needed

Blocking PRs

[ ] Merged
[x] No blocking PRs

PR to master branch

[ ] Yes: Read the release instructions

[ ] I have followed steps 1-7.

[x] No

Actions / Scans

Black: Python code formatter. Does not execute. Only tests.
Run black . locally to execute formatting.

[x] Passed

Prettier: General code formatter. Our use case: MD and yaml mainly.
Run npx prettier --write . locally to execute formatting.

[x] Passed

Yamllint: Linting of yaml files.

[x] Passed

Tests: Pytest to verify that functionality works as expected.

[ ] New tests added
[x] No new tests
[x] Passed

TestPyPi: Build CLI and publish to TestPyPi in order to verify before release.

[x] Passed

CodeQL: Scan for security vulnerabilities, bugs, errors

[ ] New alerts: Go through them and either fix, dismiss och ignore. Add reasoning in items below.
[ ] Alerts fixed: What?
[ ] Alerts ignored / dismissed: Why?
[x] Passed

Trivy: Security scanner

[ ] New alerts: Go through them and either fix, dismiss och ignore. Add reasoning in items below.
[ ] Alerts fixed: What?
[ ] Alerts ignored / dismissed: Why?
[x] Passed

Snyk: Security scanner

[ ] New alerts: Go through them and either fix, dismiss och ignore. Add reasoning in items below.
[ ] Alerts fixed: What?
[ ] Alerts ignored / dismissed: Why?
[x] Passed

since I saw the warning ...testing.rst: WARNING: document isn't included in any toctree I wonder if it shouldn't be included in index.rst? (like the password one)

Yeah I haven't quite decided on that yet. Excluded it for now to reduce the risk of confusion for those using in production


add retry strategy in cache sync

since there may be an situation that when sync request call a node that is down temporary and is restarting, sync cache may fail.  So I add a retry strategy to avoid that situation

My JIRA link

@shaofengshi  hi, please check this, it passed UT

Thanks Chao, I will re-try.

@lichaojacobs The patch has merged at https://github.com/apache/kylin/commit/ecc01458c4ad361aaf863505884d51474a8fec9d
Could you close this issue?

ok, thanks


As a cluster owner, I want to have more control over Node draining during rolling upgrades

How to categorize this issue?

/kind enhancement
What would you like to be added:
Kubernetes version updates (or updates to the node os image) are performed in a rolling upgrade fashion.
As nicely described here https://cloud.google.com/kubernetes-engine/docs/concepts/node-pool-upgrade-strategies , Gardener is offering a functionality similar to "surge upgrades" in GKE.
In certain scenarios, this procedure is not adequate to certain workloads: Imagine, many data transformation or computation jobs are running. Those jobs are usually idempotent but do not do state snapshotting, meaning: if the job is terminated and restarted, computation starts from the beginning. In this case, the pod eviction caused by the node draining is too disruptive.
Instead it would be nice, that nodes with old version get condoned (i.e. do not get new load), but draining starts after some configurable grace period. If the grace period is large enough (in my use case: several hours), the majority of jobs can complete without disruption.
Why is this needed:
The current principle of surge updates is very well suited for services, especially for this which are able to scale via replicas.
For workloads such a computation or data transfer, often a "snapshot" of the current state is not available, thus a restart of this job leads to a full recomputation.
The GKE implementation of the BGU upgrade for node pools shows, that there is a clear business need and that the surge upgrades are not appropriate for all kinds of K8S load.

Issue is related to https://github.com/gardener/gardener/issues/10121

Non-interruptable workload might be described and protected by a corresponding PodDisruptionBudget, i.e. not allowing any disruption.
Gardener will then only forcefully evict these pods when the machineDrainTimeout is exceeded.
Could you please double check this approach?

@martinhartig Did you have a chance to check @timuthy's suggestion to see whether it resolves the issue you've described?


Override not editable fields during updates

Fixes https://github.com/opendatahub-io/model-registry/issues/199
Description

As per related issue description, this PR aims to fix the update issue for InferenceService and ServeModel.
Created OverrideNotEditableFor<Entity> converters that take <existing, update> and override the NOT editable fields in the update object by taking those values from the existing.
With this approach I was able to remove the Name override in the core functions.
NOTE: everytime we add a new field in the openapi spec that can be editable, we should ignore it in the OverrideNotEditableFor<Entity> converters.

Since I have updated the goverter version, I recommend to run make clean/deps and then make build  to download te proper version of goverter.

How Has This Been Tested?

make test

Merge criteria:

[x] The commits and have meaningful messages; the author will squash them after approval or will ask to merge with squash.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

As discussed offline, if there is a quick way via introspection/parsing to determine the // annotation so to make sure is updated with the list of fields using pointer semantic in the struct, it would be awesome. If that takes too much time, could be postponed for later

+1 I will try to investigate if there is an easy way to ensure that all fields that are editable are NOT overridden, keep you posted here.

+1 I will try to investigate if there is an easy way to ensure that all fields that are editable are NOT overridden, keep you posted here.

I think I have found a possible approach for testing this using reflection and inspection of the goverter methods directives. Hope to provide a proposal on Monday :)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (06ea855) 74.33% compared to head (2e01650) 86.85%.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #200       +/-   ##
===========================================
+ Coverage   74.33%   86.85%   +12.51%
===========================================
  Files          15       11        -4
  Lines        2159      540     -1619
  Branches       68       68
===========================================
- Hits         1605      469     -1136
+ Misses        405       53      -352
+ Partials      149       18      -131

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Going to merge this to unblock the development of https://github.com/opendatahub-io/model-registry/issues/173


[WFCORE-4890] Upgrade WildFly Elytron to 1.11.3.Final

https://issues.redhat.com/browse/WFCORE-4890

@darranl thanks


render xml properly in the grid

#4648
Root cause: we escaped the strings in profiler as a result the xml cannot be displayed properly.
Fix: remove the escape call in profiler, this is safe since the underlying slickgrid already have the escaping logic.

I'm not even sure if the original bug is actually a bug. In their exec string they aren't actually using < and > , they are using the entity value for those characters, which is then being escaped since we escape &. If they instead use < and > it would show < and >. This is the same in the query editor.
We are displaying their string correctly as they gave it to us, but they are expecting us to not escape characters, which is not a good approach.

@anthonydresser Thanks for the comments, you are right, it cannot handle html properly, I will look more into it.

The example I used that broke the grid: Exec dbo.ExampleWithXML @XML = 'abdcdfd'
I looked further into the problem and found the problem is actually caused by the following code in slick.grid.js

it is using the value directly without encoding/escaping it. I can use the custom dataItemColumValueExtractor to provide a object that contains a ariaLabel property but that is not ideal. I think we should fix the issue in slick.grid.js so that we don't have to change all the references of it.
@anthonydresser thoughts?

@anthonydresser I just pushed another commit showing the fixes I had to do within ADS.


Design script

This is a possible way to split the basic setup script from design specific applications.

I took your stuff an incorporated it into https://github.com/pivotal/workstation-setup/commit/560756e7f8d1f368a738c7ab0b393d3249c180cf


Update lint checks to include ruff format check as well in the pipeline

Replace the black, flake8, pydocstyle linters with ruff.
And enable import sorting as well.

We are using ruff in the pyproject.toml and the github pipeline, are you having any issues with it? Here's the relevant portion of the code:
[tool.ruff]
exclude = ["data-files", "docs", ".git", "pycache", "venv"]
src = ["src/", "tests/unit/"]
show-fixes = true
line-length = 88
[tool.ruff.lint]
Adds Pyflakes, Mccabe, Pydocstyle, etc. run ruff linter to see all available linters
extend-select = ["F", "C90", "I", "N", "ICN"]
ignore = [ "N801", "N803", "N806", "N815", "N818", "E721"]
Allows a single underscore to be an unused variable
dummy-variable-rgx = "^_$"
[tool.ruff.lint.mccabe]
max-complexity = 18
[tool.ruff.format]
[tool.ruff.lint.isort]
order-by-type = false
section-order = ["future", "standard-library", "third-party", "first-party", "local-folder"]
[tool.ruff.lint.pydocstyle]
convention = "numpy"
ignore-decorators = ["property", "pytest.fixture", "locator"]

Sorry I had looked at an older pull request and saw that the Checks were still using Black etc. Updated the title to be specific to ruff format

created a PR for this: https://github.com/alteryx/locust-grasshopper/pull/42

Fixed in 1.3.23!


Setup the Kanban Board for the Recipe_app.

Hello there.... ✋🏾

In this Recipe_app Ruby on Rails project we are teams of two:

[X] @codeWithHyder
[X] @Besufekad-HAZ

Here is the link to our project page

STATUS: APPROVED 👍🎯
Hello @Besufekad-HAZ  and @codeWithHyder 👋
Your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉
Highlights ⭐
✔cards created and divided equally
Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.

@tugsem Thanks for the professional review. 👍🏾


'.. only:' is not working as expected in documnetaiton

Describe the bug
We do have this role based content in our documentation that stopped working:
Zephyr Project Documentation
############################

.. only:: release

   Welcome to the Zephyr Project's documentation for version |version|.

   Documentation for the latest (main) development branch of Zephyr
   can be found at https://docs.zephyrproject.org/

.. only:: (development or daily)

   **Welcome to the Zephyr Project's documentation
   for the main tree under development** (version |version|).

The published documentation does not have any of the variants.
To Reproduce
check published docs and compare with source
Expected behavior
Need one variant at lease published
Impact
There is no way for us to create content based on the desired output, for example with
.. only:: html

   Use the version selection menu on the left to view
   documentation for a specific version of Zephyr.

removing eager_only extension makes this work again, not sure what did break. Maybe only implementation in sphinx has changed and the extension is not compatible anymore?
@pfalcon  FYI

Why do we need the eager_only extension? Does documentation build fine without?

Why do we need the eager_only extension? Does documentation build fine without?

builds fine, I think as a first step we should just remove it.

removing eager_only extension makes this work again, not sure what did break. Maybe only implementation in sphinx has changed and the extension is not compatible anymore?
@pfalcon FYI

I wrote that extension quite some time ago for a particular project, which no longer uses it, and I didn't use it for a bunch of time either. So, I'm not sure about status of it. Of course, if needed, I can look into it. Otherwise, if there's an idea that it might not be needed any longer, that may be a good way too.


Updating api_installer to register paths in a deterministic order to get a deterministic swagger spec

Fixing our code for https://github.com/GoogleCloudPlatform/kubernetes/issues/5425

Also, why do you get the cia:no tag?

Re. CLA: because @nikhiljindal didn't join the https://github.com/google org.

I did join that org. I see the Google badge on my profile and I see my name there in "people" section of the group.
My membership was private. Have changed that to public now.


setup.sh: verify that find is from findutils

Description of changes
PR #168413 causes setup.sh to use find.  Packages which put busybox into their nativeBuildInputs will have $busybox/bin/find ahead of $findutils/bin/find in $PATH when setup.sh is run, causing setup.sh to fail.
Let's check that find is really from findutils so we can provide a more useful error message.
The setup.sh script already does a similar check to make sure that the shell it is running under is bash rather than some other shell.
A longer-term solution to the problem of busybox's outpaths conflicting with stdenvs is described here:
https://github.com/NixOS/nixpkgs/pull/177682#issuecomment-1155730371
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Could there be reasons for people to want to use find from busybox? This seems quite strict, though I understand where you're coming from.

Hmm, I thought it was abuild that was expecting find with printf. I don't see -printf in the fixLibtool PR.

What features of GNU findutils do we need that Busybox's find doesn't have?

Just -execdir, which I used out of habit.  It isn't needed here because TOCTOU attacks on nix's builders' temporary directories (/tmp/nix-build-*) shouldn't be possible.

Could there be reasons for people to want to use find from busybox?

I'm not sure.  But it can't hurt.

Would it be possible to make setup.sh compatible with both?

Sure: https://github.com/NixOS/nixpkgs/pull/177789

It would be really nice if build inputs looked at meta.priority, IMO.  There are other situations where it'd be nice — e.g. I have a shell.nix with both virtiofsd and qemu, and I have to make sure to specify them in the right order so I don't get QEMU's deprecated built-in virtiofsd ahead in PATH.

It would be really nice if build inputs looked at meta.priority

Hrm, is that possible?
I thought that changing a package's meta would not affect its drv-hash (or the drv-hashes of packages which depend on it).
If builders could see their inputs' meta.priority then changes to meta.priority would affect the build process, and therefore would have to affect the drv-hash.

Also while we're wishing for things, I wish meta.priority were a partial ordering rather than a total $\omega_0$-ordering.  So for each package you list the other packages it is strictly-less-than or strictly-greater than.
I can never seem to remember integer priority systems like the one in nftables.


convert table to list/array

Are there built-in functions to convert a table to a list of lists, (preferably with the column names as the first list), and/or an array?

Reasonable request. You can do the following now (just added).
(vellum:pipeline (table) (cl-ds.alg:to-list :key (vellum.selection::row-to-list)))
By default row-to-list will extract all rows, but it accepts the same arguments as the VS function. CL-DS.ALG also contains to-vector and to-hash-table. Those are aggregation function so it works just fine with cl-ds.alg:group-by. There is also peculiar macro vellum:brr that can be used to convert some of the row to list (it has a double mode, it will yield a single value is there is one argument, and a list if there are multiple arguments). I will add ro

Oh, and you could actually attempt using aggregate-columns but this would be a two-stage process which does not make it convinient.

I also added row-to-vector. I will close this now.


Store !help that return nothing

Store somewhere the asked word when someone send a !help word that return nothing.
So that admin can see what people want to know.

Transfer to https://github.com/LaBaleineFr/ToolsBaleine


fix the regex for publishing binaries

Description
A few sentences describing the overall goals of the pull request's commits.
Fill ALL sections
Documentation

[ ] I updated the relevant .md documentation.
[ ] I updated the relevant website documentation.
[ ] This PR does not require documentation updates.

Dependencies

[ ] This PR contains new dependencies
[ ] This PR does not have any change in dependencies.

Testing

[ ] I provided sufficient test coverage for the modified code.
[ ] The existing tests capture the requirements for the modified code.
[ ] The bulk of my code covered by unit tests.
[ ] I executed manual tests to validate my changes and to avoid regressions.

Testing Strategy
A few words describing your testing strategy, e.g., manual tests, automated tests, extra validations, etc.
Dependencies, related issues, and other notes

List change dependencies, if any.
List related issues, if any.

(The failing chart release is because I tagged this branch as the release, but the chart release for that tag had already run)


General: Improve the Selection of Exercise Details Shortcut Buttons

Checklist
General

[x] I tested all changes and their related features with all corresponding user types on a test server.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Client

[x] I followed the coding and design guidelines.
[x] I added multiple screenshots/screencasts of my UI changes.
[x] I translated all newly inserted strings into English and German.

Motivation and Context
The current selection of the exercise details shortcut buttons has a few drawbacks:

Tutors have no selection of exercise details button shown. If they want to open the detail view, the scores or the participation page, they must navigate into the Course Management -> Exercise and select the exercise from the list.
Editors are displayed with the button Submissions, but they are not allowed to open the submission page for Text- and Modelling-Exercises
Participations are only shown for Programming Exercises, however, the Participation Page is used to set the presentation score
Quiz-Exercises lack the feature to open the preview or the example solution for the exercise

Description
The corresponding course-exercise-details.component.html was modified to present a selection of shortcut buttons to Tutors, Editors, and Instructors.

The Shortcut-Buttons are either labeled Tutor actions or Instructor actions
Tutors have shortcuts to view the details page, open the scores or the participations
Editors can only open the Submissions for Programming Exercises. For Text-, Modelling- and Quiz-Exercises, they can open the Participations instead
For Quiz-Exercises, shortcuts to the preview and the example solutions are added

A detailed overview can be found below.
Steps for Testing

1 Instructor
1 Editor
1 Tutor
1 Exercise of each type (Programming-, Modelling-, Text-, Quiz-, File-Upload)

Log in to Artemis
Open a course
Open the 5 Exercises with all 3 roles and make sure, that the buttons are displayed as shown in the screenshot below. Also make sure, that all buttons are functional.
(Note: For Quiz-Exercises, editors should only be able to edit the Quiz before the start of the quiz. During the quiz, both editors and instructors should not be able to edit the quiz. After the quiz, only instructors should be able to re-evaluate the quiz. But this is independent of this PR)

Code Review

[ ] Review 1
[ ] Review 2

Manual Tests

[ ] Test 1
[ ] Test 2

Screenshots

Seems to work well mostly.
For a file upload exercise, I got this as editor instead of what you provided in your screenshot (stats is missing):

The same happens for text exercises for both tutors and instructors (stats missing).
Stats are there for quiz exercises, though.
I don't know if I did something wrong :D

Thanks for your review @bassner 👍
You did not do anything wrong, I just got mixed up when putting the buttons together.
All exercises have Scores, but only modelling and quiz exercises have Statistics additionally.
(And for the sake of completeness, you cannot View a quiz and quizzes have no Participation.)
I have updated the screenshot above.


Bigints out of range in the query results' new UI

Expected behavior
Running
should return

test

1675679957658

Actual behavior
Running
select<PHONE_NUMBER>658 as test

actually returns

test

-2147483648

Steps To Reproduce
Open a .sql file, and run
select<PHONE_NUMBER>658 as test

Log output/Screenshots
This number does not seem "random", and I believe something is up with the new UI.
select<PHONE_NUMBER>664

will also return the same value.
I also tried
select<PHONE_NUMBER>658::bigint as test

A teammate did ask all-mighty ChatGPT:

The value -2147483648 is at the boundary of the integer range in many systems (including PostgreSQL, which Redshift is based on). The integer data type typically ranges from -2147483648 to<PHONE_NUMBER>. If colA is a bigint, this shouldn't be an issue, but if there's any chance that colA is actually an integer, the query might not behave as expected. Ensure that colA is indeed a bigint.

Here is what we get in the new UI:

while it works fine in the legacy UI:

We originally had the problem on a bigint column in our dwh.
Operating System
MacOS Ventura 13.5.2 (22G91)
dbt version
Core: 1.6.6
dbt Adapter
Redshift: 1.6.2
dbt Power User version
v0.24.3
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

This issue seemed to be fixed when I tested it.

@david-whimsical, could you please test the latest version of the extension and let us know if this issue has been fixed?

Hey @anandgupta42!
I'm still experiencing the problem with version v0.27.0 – it seems like it's the latest available in my end, is that right?

I have the same problem. I have tested with version 0.27.2
I use the dbt adapter for databricks
>dbt debug
10:02:32  Running with dbt=1.6.6
10:02:32  dbt version: 1.6.6
10:02:32  python version: 3.11.6
10:02:32  os info: Windows-10-10.0.22621-SP0
10:02:32  adapter type: databricks
10:02:32  adapter version: 1.6.6

@stegus64 and @david-whimsical, we have fixed the issue across all dialects and adapters. Can you please upgrade to v0.31.4 and verify if the issue persists?

Hey @anandgupta42 – well done, this is fixed! Thanks a ton 😊


Latest deployment breaks previous URLs

This linked was shared a couple of weeks ago: https://map.openaerialmap.org/#/-72.24601549919855,18.64219774452739,12/0322112030100/59bebf9646e19904aab0ae4a?_k=nv0j3h but after the latest deployment, it doesn't seem to work anymore.
The corresponding new URL might be https://map.openaerialmap.org/#/-72.29072570800781,18.60525215054155,10/square/032211203001/59bebf9646e19904aab0ae4a?_k=nv0j3h but doesn't seem to work.
What is the new /square/ parameter added to the URL?

@cgiovando Square is the grid square that is selected. OAM urls have never been perfectly sharable since we've relied on a grid which is different per zoom. This will change again in the future as we adjust how the grid is shown. Were you able to find the image by searching and selecting a grid square (as opposed to clicking on an old url)?


Player Dimension

Is your feature request related to a problem? Please describe.
No.
Describe the solution you'd like
We should build a player dimension containing player metadata. It will be a great starting point for building this play by play model.
Describe alternatives you've considered
No good alternatives. We could just jam all the player attributes into on hyperdenormalized table.
Additional context
@JJohnson-DA already did the work to pull the player data into a table.

We'll need to find column definitions for all that data and add it to its own dbt schema .yml file.
We'll need to parse the columns we care about and build a dbt model for the player dimension

In the future we may want to make the player_dim a SCD


Refactor (ci): Remove .github/workflows/ci-release and use only ci-master-pr and CalVer-based workflow

lgtm, will fix broken packages in another MR


[BUG] Keydown event because it had an old focus tree

Hey
I've noticed strange behaviour.
Demo:
https://codesandbox.io/s/react-hotkeys-6vexl?fontsize=14
If we focus "container 1" and press esc we'll see "Container  1 ESC" that is ok.
If we press the "change" button we'll show "container 2" and after focus it and press ESC in the console we'll see "Container  2 ESC" - like expected.
And now we can change focus as much as we want. We can show and hide container 2 by button and everything work as expected.
Bug
But... Container 2 got action for "enter" key. This action doing the same thing as the "change" button but by HotKeys handler. After we press "enter" key when container 2 is focused -  ESC handlers don't work anymore on any container.
Strange
If i add
simulateMissingKeyPressEvents: false

option. The handler will work on 3rd ESC press
Expected behavior
ESC handlers work after we use "enter' when container 2 is focused.
Platform

react-hotkeys 2.0.0
chrome
OS: Ubuntu

Thanks for posting your issue.
Unfortunately I do not have the time to actively work on this package, but I am seeking other active maintainers. If you are willing to create a pull request or help out, that would be an excellent way of moving this forward.


add region field to project configuration

From #883
Add region field to project.json.
The priority of region specification will be following:
(Most Prior) --region > project.json 🆕 > AWS_REGION env > ~/.aws/config (Least prior)

thanks!


0.9.5 doesn't work without defining password for redis anymore

Hello!
After upgrading to 0.9.5 if no password defined for redis url will throw an error:
redis.exceptions.DataError: Invalid input of type: 'NoneType'. Convert to a bytes, string, int or float first.
redis.exceptions.AuthenticationWrongNumberOfArgsError: wrong number of arguments for 'auth' command
On 0.9.4 works fine.
BR,
Dacian

Most likely relates to #218
@erdos4d any ideas?

I'd like to add more details about the issue that @jab3z described above.
If there are no username and password in a redis url, django-defender sets username to 'default' and password to None by default.
Since the username is not None or an empty string, redis-client tries to authorize and raises AuthenticationWrongNumberOfArgsError and DataError.
IMO, a way should be added to specify username as None or an empty string during the StrictRedis initialization in django-defender if a user's redis url doesn't contain username.

I'm on Ubuntu 22.04 with a clean install of redis 6.0.16 and defender at 0.9.5. I can set DEFENDER_REDIS_URL to redis://<IP_ADDRESS>:6379 and it works fine, I can login to the admin and see the attempt. By my reading of the code, this should set username as default and password as None in the StrictRedis constructor. Am I missing something to reproduce this? Can someone share a url/config that will reproduce? Thanks and sorry if this messed anyone up out there.

@erdos4d forgot to mention, redis://<IP_ADDRESS>:6379 works fine for me too. I get the error while using a real production redis url, something like redis://abc.def.amazonaws.com:6379/1. Redis v5.0.6 and redis-client v4.3.3 are used in production.

Hello,
I haven't debugged why and how the error occurs, simply found the faulty package, checked the release log to not have major changes and reverted to the old version.
Tested only locally, but the following Redis URLs were throwing the error when login into admin:
redis://<IP_ADDRESS>:6379
redis://<IP_ADDRESS>:6379/1
redis://localhost:6379
redis://localhost:6379/1

Let me know if you'd like me to raise a PR with the fix.
BR,
Dacian

That error happend to me too and also only on the production server. What I did was:

Setting password "mypassword" for redis on the server (by editting /etc/redis/redis.conf or via redis-cli):

root@server:~# redis-cli
<IP_ADDRESS>:6379> config set requirepass mypassword
<IP_ADDRESS>:6379> quit
root@server:~# service redis-server restart

Adding DEFENDER_REDIS_URL to Django's settings.py:

DEFENDER_REDIS_URL = 'redis://:mypassword@localhost:6379/0'

To see blocked users/ips:
root@server:~# redis-cli
<IP_ADDRESS>:6379> auth mypassword
<IP_ADDRESS>:6379> scan 0

That it. To quick check, for example, does the user "admin" is blocked:
<IP_ADDRESS>:6379> get defender:blocked:username:admin

To quick quick unblock user "admin":
<IP_ADDRESS>:6379> del defender:blocked:username:admin

I am getting this error with django-defender 0.9.6 and redis 4.3.5. My Django is 3.2.16.
I cannot set a password on redis since it is used by several other applications. Is there a workaround?
--p

What do you have for defender settings? Specifically "DEFENDER_REDIS_URL" and "DEFENDER_REDIS_NAME" ?

Also, what version of redis server are you using?

Redis server v=5.0.3
DEFENDER_REDIS_URL = 'redis://<IP_ADDRESS>:6379'
DEFENDER_REDIS_NAME is not set (docs say that if _NAME is set, it overrides _URL).
Not necessarily relevant, but the other settings in base.py are:
DEFENDER_LOGIN_FAILURE_LIMIT = 5
DEFENDER_BEHIND_REVERSE_PROXY = True
DEFENDER_LOCK_OUT_BY_IP_AND_USERNAME = True
DEFENDER_ACCESS_ATTEMPT_EXPIRATION = 1
DEFENDER_STORE_ACCESS_ATTEMPTS = True
DEFENDER_COOLOFF_TIME = 900
DEFENDER_LOCKOUT_TEMPLATE = 'registration/login_failed.html'

Additional details: And the system is on Alma 8. Python version is 3.6.8.

Here is the full traceback. Dump happens right when the initial login form loads. Well before user enters anything into the credentials field.
Environment:
Request Method: GET
Request URL: https://<IP_ADDRESS>/login/
Django Version: 3.2.16
Python Version: 3.6.8
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'django_filters',
'rest_framework.authtoken',
'rest_framework',
'allauth',
'allauth.account',
'allauth.socialaccount',
'allauth.socialaccount.providers.amazon',
'allauth.socialaccount.providers.google',
'allauth.socialaccount.providers.linkedin',
'defender',
'resources',
'rulesstatus',
'rules',
'profiles',
'users',
'reports',
'aws',
'ibm_broker']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'defender.middleware.FailedLoginMiddleware']
Traceback (most recent call last):
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 713, in on_connect
auth_response = self.read_response()
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 839, in read_response
raise response
During handling of the above exception (wrong number of arguments for 'auth' command), another exception occurred:
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/defender/decorators.py", line 21, in wrapper
if utils.is_already_locked(request):
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/defender/utils.py", line 393, in is_already_locked
ip_blocked = is_source_ip_already_locked(get_ip(request))
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/defender/utils.py", line 386, in is_source_ip_already_locked
return REDIS_SERVER.get(get_ip_blocked_cache_key(ip_address))
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/commands/core.py", line 1705, in get
return self.execute_command("GET", name)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/client.py", line 1235, in execute_command
conn = self.connection or pool.get_connection(command_name, **options)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 1387, in get_connection
connection.connect()
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 623, in connect
self.on_connect()
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 719, in on_connect
self.send_command("AUTH", self.password, check_health=False)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 800, in send_command
self.pack_command(*args), check_health=kwargs.get("check_health", True)
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 858, in pack_command
for arg in map(self.encoder.encode, args):
File "/home/raxak/.virtualenvs/raxak/lib/python3.6/site-packages/redis/connection.py", line 109, in encode
f"Invalid input of type: '{typename}'. "
Exception Type: DataError at /login/
Exception Value: Invalid input of type: 'NoneType'. Convert to a bytes, string, int or float first.

@pmulgaonkar any chance you can try this version out? I think it will fix your issue, but not 100% https://github.com/jazzband/django-defender/pull/227

Sorry. I don't understand. You want me to pull the file [defender/connection.py] and replace the one in my current build?
Or do you want me to try with redis >= 6
--p

Try with the 'connection.py' file that is in that PR/branch on your local build, using your current redis server.
Ken

Tried it. Error went away and login was permitted. However defender does not trigger even after multiple incorrect password attempts.

I'm glad the error went away, but I'm confused as to why defender isn't working. Does it work fine with the older version 0.9.4?

Will revert version and try again, but we have been using defender successfully for a long time with defender 0.6.0. Now going through the process of upgrading along with redis and django, and started with 0.9.6.
But downgrading to 0.9.4 also does not not work. No redis error, but defender functionality (blocking) does not happen.
Probably something in my code since we are using a custom login function that we had to modify while upgrading django. (Old system was still on django 1.11) Probably something we are doing wrong.
Will test and report.

Well... reinstalled everything and downgraded to django-defender 0.9.4. Now I can see the process playing out correctly. Backend database DEFENDER_ACCESSATTEMPTS correctly updates everytime I try to login. Debug statements show that authentication is successful. However the final redirect from the login code seems to put things right back at the login screen. The code itself is like this:
`
nxt = request.GET.get('next','/')
print_err("nxt = {0}".format(nxt))
return redirect(nxt)
`
If I disable defender from the middleware list, and remove the @watch_login on the login code, this construct works fine and the redirect takes me to the correct page. Am I missing something obvious?

My previous comment vanished. I systematically redid the code and now everything is working, but with 0.9.4.

Great, my fix has been merged into the master branch. Feel free to try that version, hopefully that works for you.
I'll cut a new release this weekend when I get a chance.

@pmulgaonkar @jab3z  I released 0.9.7 with the fix for using Redis 5 with no password; see if that works for you.

am having the same problem here, redis server activates the localhost url even when i hardcode the redis url from a managed redis server

What versions are you using?

Am using
redis == 5.0.0
django-redis == 5.3.0
django-defender == 0.9.7
The managed redis works just fine but the moment I start implementing
Django defender functionalities then it stops working. Like it loads the
localhost redis url...
This is happening in production environment and even in development
environment the managed redis url does not load....
On Mon, Nov 20, 2023, 20:41 Ken Cochrane @.***> wrote:

What versions are you using?
—
Reply to this email directly, view it on GitHub
https://github.com/jazzband/django-defender/issues/219#issuecomment-1819611475,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A4I6IMGEHAY7WIO7NGYGMBDYFOP4TAVCNFSM5YNQQL42U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRHE3DCMJUG42Q
.
You are receiving this because you commented.Message ID:
@.***>

Noted that I have not provided the redis name and password since the
managed redis server only gives you the redis url....
On Mon, Nov 20, 2023, 21:14 Hope Sain @.***> wrote:

Am using
redis == 5.0.0
django-redis == 5.3.0
django-defender == 0.9.7
The managed redis works just fine but the moment I start implementing
Django defender functionalities then it stops working. Like it loads the
localhost redis url...
This is happening in production environment and even in development
environment the managed redis url does not load....
On Mon, Nov 20, 2023, 20:41 Ken Cochrane @.***> wrote:

What versions are you using?
—
Reply to this email directly, view it on GitHub
https://github.com/jazzband/django-defender/issues/219#issuecomment-1819611475,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A4I6IMGEHAY7WIO7NGYGMBDYFOP4TAVCNFSM5YNQQL42U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRHE3DCMJUG42Q
.
You are receiving this because you commented.Message ID:
@.***>

And you don’t see any errors or messages in the logs at all?
do you see the same error when running the version that is on the master branch?

The error message I see is that ConnectionError
Error  10061 connecting to localhost:6379. No connection could be made
because the target machine actively refused it.
This happens even when I use a managed redis url.
Like instead of using the managed redis url then it uses the localhost..
On Mon, Nov 20, 2023, 21:26 Ken Cochrane @.***> wrote:

And you don’t see any errors or messages in the logs at all?
do you see the same error when running the version that is on the master
branch?
—
Reply to this email directly, view it on GitHub
https://github.com/jazzband/django-defender/issues/219#issuecomment-1819668944,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A4I6IMAFZA7MT5UX6HOXST3YFOVGHAVCNFSM5YNQQL42U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRHE3DMOBZGQ2A
.
You are receiving this because you commented.Message ID:
@.***>

Ok, do you see that same error using the latest version on the master branch?

Yes, am seeing the same error.  even with the latest versions.  and it is a stressful experience...

Ok, does your managed URL have any special characters in it? Could you post the config that you are using (mask out any sensitive info) so we can try and duplicate what you are seeing.

One more question, how are you installing the latest version on master?

i do not get this but am using pip install ... the statement am not fully getting 'One more question, how are you installing the latest version on master?'
as for the caches..
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "*********************:6379",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
}
}
and the answer is yes the url has special characters, just to be precise am using render.com managed redis. you can try it out it is completely free....

So you did
pip install git+https://github.com/jazzband/django-defender.git

Have you tried to set the DEFENDER_REDIS_URL  or DEFENDER_REDIS_NAME ? or any of the other DEFENDER_ variables?

setting the DEFENDER_REDIS_URL, was all i wanted and everything now works fine, if anything then i will let you know..
in your documentation explicitly outline that DEFENDER_REDIS_URL has to be for the managed url or they have to set it clearly in production otherwise it loads default redis localhost. this will raise errors....


Add requirements.txt

To every sample.

Example: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/dataproc/requirements.txt

See pending PR.


[Issue] HttpProxy query in script is different from blog

Describe the work
Script discrepancy
Additional Context
Found some discrepancy between this post and the script currently in github.
https://www.microsoft.com/security/blog/2021/03/02/hafnium-targeting-exchange-servers/
On the post above, for CVE-2021-26855, it contains -
Where-Object {  $.AuthenticatedUser -eq '' -and $.AnchorMailbox -like 'ServerInfo~/' }
But current version on github it doesn't contain ($_.AuthenticatedUser -eq '') condition.
Would you please explain why the current script did not include the authenticateduser as one of conditions or should it be included? Thanks.

The thinking with -AuthenticatedUser -eq '' is that we don't need to look at requests that are authenticated. From what I see, the script has never had that parameter, even back to its initial version. As far as I can tell, we're not actually seeing any authenticated requests that have a ServerInfo hit.
So the short answer is that it makes no practical difference based on the data we're seeing right now. However, we should probably add that parameter just for consistency with the blog post.

Thanks for a lot for the detailed explanations. We are currently re-run modified script with condition added and will keep you posted if we see different results.

Yes rerunning returned same results. Again thanks for the quick response.


top_count_keys does not work with multiple query_key due to a unexpected leading space in filter

I have set up a frequency rule as follows.
name: test

index: filebeat-*

type: frequency

num_events: 10

timeframe:
  hours: 1

filter:
- bool:
    must:
      - exists:
          field: "data.srcip"
      - exists:
          field: "data.dstip"

top_count_keys: ["data.dstuser", "data.srcip", "data.srcport", "data.dstip", "data.dstport"]

top_count_number: 5

raw_count_keys: false

query_key: ["data.srcip", "data.dstip"]

It will fail with top_count_keys:
Traceback (most recent call last):,
File "/usr/local/lib/python3.6/site-packages/elastalert-0.2.4-py3.6.egg/elastalert/elastalert.py", line 1450, in alert,
  return self.send_alert(matches, rule, alert_time=alert_time, retried=retried),
File "/usr/local/lib/python3.6/site-packages/elastalert-0.2.4-py3.6.egg/elastalert/elastalert.py", line 1485, in send_alert,
  counts = self.get_top_counts(rule, start, end, keys, qk=qk),
File "/usr/local/lib/python3.6/site-packages/elastalert-0.2.4-py3.6.egg/elastalert/elastalert.py", line 1997, in get_top_counts,
  buckets = list(hits_terms.values())[0], IndexError: list index out of range

I found out it is because there is a leading space for the second query_key in the filter query which leading query parsing error in elasticsearch for my case. (" YYY.YY.YY.YY" instead of "YYY.YY.YY.YY")
{
  "term": {
    "data.srcip": "XXX.XX.XX.XX"
  }
},
{
  "term": {
    "data.dstip": " YYY.YY.YY.YY"
  }
}

The reason is because a space is added for compound key during the join()
            if rule.get('compound_query_key'):
                values = [lookup_es_key(hit['_source'], key) for key in rule['compound_query_key']]
                hit['_source'][rule['query_key']] = ', '.join([str(value) for value in values])

            if rule.get('compound_aggregation_key'):
                values = [lookup_es_key(hit['_source'], key) for key in rule['compound_aggregation_key']]
                hit['_source'][rule['aggregation_key']] = ', '.join([str(value) for value in values])

but the qk_list is splitted by a single comma only which causing the issue
def get_hits_terms(self, rule, starttime, endtime, index, key, qk=None, size=None):
        rule_filter = copy.copy(rule['filter'])
        if qk:
            qk_list = qk.split(",")

For a fix, use ','.join instead of ', '.join for compound key.

We also recommend asking questions on the Gitter channel
https://gitter.im/Yelp/elastalert

Yelp/elastalert is no longer maintained. Please use jertel/elastalert. Questions to the discussion below
https://github.com/jertel/elastalert/discussions


update text field wording for clarity

I fumbled around putting in the wrong model information until I got it right. This change will make it more clear to the user since "Model Name" is the specific column to use when referencing ollama list output. I also updated the placeholder to make it even more clear.

Hey @saoudrizwan, let me know if this needs any other edits in your opinion. Small wording change on the Ollama provider's page.


Build ztouch Probing Function

Hi everyone,
In this PR I am integrating a way of probing the height of non-conductive materials, e.g. 96-well plates, and standard tubes, to enable their automated definition generation.
The Problem
In PR #69 I integrated the function probe_z_height_using_channel which uses Hamilton STAR(let) channels' capacitive Liquid Level Detection (cLLD) capability to detect the z-height of any conductive material.
Though very useful and highly precise, this function does not allow z-height probing of the resources/labware we have to most commonly generate definitions for, plates and tubes, because these resources are made of highly insulating/non-conductive plastic.
Task definition: We therefore urgently need a function that uses only the machine available (STAR(let)) to identify the z-height of these non-conductive materials.
PR Content
Here I ...

introduce a new function called STAR.ztouch_probe_z_height_using_channel(channel_idx: int, tip_len: float): this function uses the same functionality that aspirate and dispense with lld_mode=4, i.e. z-touchoff, has. In other words, the channel is conducting a controlled "crash" of the tip, which triggers some form of force-sensor in the Z-drive.
This enables the probing of non-conductive materials, including plates and tubes.
Rename STAR.probe_z_height_using_channel() to STAR.clld_probe_z_height_using_channel(), to clearly distinguish these two probing functions,
convert both functions to accept attributes in millimetre (mm) (and homogenise their attributes as much as possible to make it easier to switch between the two functions),
convert both functions to use 0-based channel indexing (0 is the backmost channel from a user standing in front of the machine).

The current version of the new function:
class STAR(HamiltonLiquidHandler):
  ...
  async def ztouch_probe_z_height_using_channel(
    self,
    channel_idx: int, # 0-based indexing of channels!
    tip_len: float, # mm
    lowest_immers_pos: float = 100.0, # mm
    start_pos_search: float = 330.0, # mm
    channel_speed: float = 10.0, # mm/sec
    channel_acceleration: float = 800.0, # mm/sec**2
    channel_speed_upwards: float = 125.0, # mm
    detection_limiter_in_PWM: int = 1,
    push_down_force_in_PWM: int = 0,
    post_detection_dist: float = 2.0, # mm
    move_channels_to_save_pos_after: bool = False
    ) -> float:
    """ Probes the Z-height below the specified channel on a Hamilton STAR liquid handling machine
    using the channels 'z-touchoff' capabilities, i.e. a controlled triggering of the z-drive,
    aka a controlled 'crash'.
    ...
    """

Next Steps
I have added one TODO to STAR.ztouch_probe_z_height_using_channel():
At the moment, users have to directly declare the total_tip_length to the tip_len argument.
This can easily be automated at runtime using:
measured_z_touch = await lh.backend.ztouch_probe_z_height_using_channel(
    channel_idx=0,
    tip_len=lh.head[0].get_tip().total_tip_length,
    channel_speed=6.0, start_pos_search=250.0,
    detection_limiter_in_PWM=0, push_down_force_in_PWM=0,
    move_channels_to_save_pos_after=False
)

i.e. by accessing the total_tip_length via the liquid_handler.
I would like to remove this attribute altogether and have the function retrieve the total_tip_length information by itself at the STAR level, rather than the liquid_handler level.
Notes
How accurate is STAR.ztouch_probe_z_height_using_channel()?
I performed an array of tests in which I ztouch_probed conductive materials and compared their measured z-height against the same positions clld_probed height:
I still need to write the complete analysis up, but in short: ztouch_probing deviates between 0 (minimum) - 320 (maximum) micrometers, in my hands.
This is in absolute terms.
Does this matter?
Whether this deviation matters depends on what you are trying to measure:
If you try to identify absolute locations, it likely matters, and I recommend using clld_probing in these cases - if possible, because the material is conductive (e.g. PlateCarrierSite mapping).
However, if you try to identify relative positions you can use ztouch_probing for measuring both/multiple locations of interest with low to no concern about this deviation:
e.g. all Containers require defining their material_z_thickness
see PR #183; example:

This attribute of Container is very difficult to actually measure (e.g. because calipers cannot enter the container, ...).
But using ztouch_probing one can identify the well_cavity_bottom, then remove the plate, and use ztouch_probing again to identify the height of the site below.
If the PlateCarrierSite has a pedestal, the difference between these two measurements is the material_z_height ... measured, and therefore automatable.
Since both measurements can be assumed to experience any absolute deviation in the same way, their difference holds true, regardless of absolute deviation between ztouch_probing and clld_probing :)
Does it damage the channel?
There might be some concern about damaging the channel when using controlled "crashes" to probe resources.
Nothing can guarantee no damage.
But as long as the force and detection attributes are not pushed to their extremes (i.e. detection_limiter_in_PWM & push_down_force_in_PWM), channel damage is deemed very unlikely.
I defaulted these values to 1 and 0 respectively, making the force-sensing extremely responsive and essentially generating no noticeable force, to decrease damaging risks in the odd case that anyone accidentally finds these functions and doesn't read this PR/the documentation pages (work in progress).
What tips can you use?
All standard tips, i.e. 10 ul, 50 ul, 300 ul, 1000 ul, work using ztouch_probing, and you are not limited to using their conductive version but might also use their transparent or needle/metal version.
That being said, there might be a slight change in accuracy depending on tip choice:
Lower-volume tips are thinner and can buckle under the ztouch_probing force generated which might give a lower z-height than real.
I therefore switched to using a teaching needle for probing (see image above of probing a PCR 96-well plate).
However, if one does not have needles/metal tips available and requires a quick measurement it is possible to successfully use standard tips, taking a mental note of the caveats discussed here.

Please let me know whether you have any suggestions on how to further improve these two functions.
I expect these updates to save use all time, and to lead to highly robust PLR Resource Library contributions.
Happy automation 🦾

I would like to remove this attribute altogether and have the function retrieve the total_tip_length information by itself at the STAR level, rather than the liquid_handler level.

This information depends on the tip that is currently mounted, which is only known by LiquidHandler. I tried to make STAR as stateless as possible. The tips used are passed to say STAR.aspirate by LiquidHandler.aspirate through the ops array when doing an aspiration.
In this case, a call is made directly to the backend where this information is not available. One way to do this is to go through LH, but that seems suboptimal because this functionality is not generally applicable. The alternative seems to be passing the parameter directly the backend (if we want to keep STAR with minimal state). We can add a convenience feature to LiquidHandler for getting the length of the currently mounted tips if helpful.

The alternative seems to be passing the parameter directly the backend (if we want to keep STAR with minimal state). We can add a convenience feature to LiquidHandler for getting the length of the currently mounted tips if helpful.

I am not sure. Do you think a convenience feature would be clearer than my example from above?:

This can easily be automated at runtime using:

measured_z_touch = await lh.backend.ztouch_probe_z_height_using_channel(
    channel_idx=0,
    tip_len=lh.head[0].get_tip().total_tip_length,
    channel_speed=6.0, start_pos_search=250.0,
    detection_limiter_in_PWM=0, push_down_force_in_PWM=0,
    move_channels_to_save_pos_after=False
)

i.e. by accessing the total_tip_length via the liquid_handler.

I guess lh.head[0].get_tip().total_tip_length is already taking this feature's role.
I understand the reason for statelessness in the STAR backend. Keeping the resource management system and the machine control system, i.e. machine backend, separate has definitely many advantages.
But it does seem like it limits the backend in this case, and makes it dependent on the liquid_handler.
I don't see a path around this either and we'll just have to hand the backend the tip_length.
...what I don't understand is how STAR.clld_probe_z_height_using_channel() appears to somehow circumvent this issue.
We don't tell the cLLD function what the tip_len is, and yet it works with all conductive tips....

i had to add
lowest_immers_pos=100,

because AssertionError: Lowest immersion position must be between 99.98 and 334.7 mm..
unfortunately it seems i have to update the channel firmware
STARFirmwareError: {'Pipetting channel 2': UnknownHamiltonError('Unknown command')}, P2ZHid0019er30

. Do you think a convenience feature would be clearer than my example from above?:

not necessarily, up to you

But it does seem like it limits the backend in this case, and makes it dependent on the liquid_handler.

Very much by design. LiquidHandler is the state manager.

...what I don't understand is how STAR.clld_probe_z_height_using_channel() appears to somehow circumvent this issue.
We don't tell the cLLD function what the tip_len is, and yet it works with all conductive tips....

I guess it knows which tip is mounted, and the length is defined with TT earlier?
(also, only tangentially related, when discarding tips to trash it will just move until it feels resistance, and then discard. but you don't get a reading from that)

ztouch_probe_z_height_using_channel should then also know this, but maybe we can't assume all the firmware commands are at the same level/written at the same time

unfortunately it seems i have to update the channel firmware

What is your return from?
await lh.backend.send_command(
    module="PX",
    command="RF"
    )

ztouch_probe_z_height_using_channel should then also know this, but maybe we can't assume all the firmware commands are at the same level/written at the same time

I completely agree; I'm not sure how the firmware keeps memory of the tip but I think you are right that we cannot assume different firmware commands are written the same way.
In fact the clld_probing and ztouch_probing return the measured z_height in completely different formats:
clld_probing returns the z_height in dmm
and ztouch_probing does in increment ... I abstracted all this complexity in the functions I created because the user should not have to be bothered by this inconsistency.
Btw, what do people think about the name "ztouch_probing"? My alternative suggestion is "force_probing" but since I don't know the mechanism of action behind the force sensing I was hesitant to use it, and "ztouch_probing" is a reference to the lld_mode=4 / "z-touchoff" that inspired me to create this function.
But happy to hear what the community finds more intuitive.

'P1RFid0012rf4.0S f 2020-07-31'
I like ztouch_probing!

I guess it knows which tip is mounted, and the length is defined with TT earlier?

ztouch_probe_z_height_using_channel should then also know this, but maybe we can't assume all the firmware commands are at the same level/written at the same time

This seems to be discrepancy between the two commands which might or might not be intended. Basically the tip length would also be available through TT.

This seems to be discrepancy between the two commands which might or might not be intended. Basically the tip length would also be available through TT.

I'm not sure I understand what you mean, @jrast: C0TT, PLR-exposed as STAR.define_tip_needle(), is a definition command.
To my understanding, it does not enable "request tip_length on channel n", which is what we'd need to remove the attribute tip_len from STAR.ztouch_probe_z_height_using_channel(channel_idx: int, tip_len: float).

Slightly off-topic: Having the LiquidHandler as state manager is probably the right way to go. At the same time the .deck on the Backend also handles state (of course it's also available at other levels) and the backend has easy access to this state. Maybe something similar would also be nice for the state of the liquid handler?

This is an amazing idea!
It would enable a whole array of automated requests, and solve the TODO I placed into  STAR.ztouch_probe_z_height_using_channel(channel_idx: int, tip_len: float).

To finish this PR:
With @rickwierenga currently not having the necessary 2022+ firmware version of module="PX", is there anybody else who could verify/peer review the changes/additions to PLR from this PR?

This seems to be discrepancy between the two commands which might or might not be intended. Basically the tip length would also be available through TT.

I'm not sure I understand what you mean, @jrast: C0TT, PLR-exposed as STAR.define_tip_needle(), is a definition command.
To my understanding, it does not enable "request tip_length on channel n", which is what we'd need to remove the attribute tip_len from STAR.ztouch_probe_z_height_using_channel(channel_idx: int, tip_len: float).

Sorry, this was not clear. I was talking about the Firmware which would have access to the tip length by looking at what's in the tip definition table written with the TT command. And indeed, the STAR firmware offers no command to query the tip definition table, at least I'm not aware of a command. Newer devices offer a command to read the table.

@jrast, I had a bit of an epiphany and invented a way for the STAR to measure the tip length on a specified channel, even though it is impossible to request that information from the firmware, and made it into a standalone function for us:
async def request_tip_len_on_channel(
    self,
    channel_idx: int, # 0-based indexing of channels!
    ) -> float:
    """
    Measures the length of the tip attached to the specified pipetting channel.

    Checks if a tip is present on the given channel. If present, moves all channels
    to THE safe Z position, 334.3 mm, measures the tip bottom Z-coordinate, and calculates
    the total tip length. Supports tips of lengths 50.4 mm, 59.9 mm, and 95.1 mm.
    Raises an error if the tip length is unsupported or if no tip is present.

    Parameters:
        channel_idx (int): Index of the pipetting channel (0-based).

    Returns:
        float: The measured tip length in millimeters.

    Raises:
        ValueError: If no tip is present on the channel or if the tip length is
          unsupported.
    """

    # Check there is a tip on the channel
    all_channel_occupancy = await self.request_tip_presence()

    if all_channel_occupancy[channel_idx]:
      # Level all channels
      await self.move_all_channels_in_z_safety()
      known_top_position_channel_head = 334.3 # mm
      fitting_depth_of_all_standard_channel_tips = 8 # mm
      unknown_offset_for_all_tips = 0.4 # mm

      # Request z-coordinate of channel+tip bottom
      tip_bottom_z_coordinate = await self.request_z_pos_channel_n(
        pipetting_channel_index=channel_idx
        )

      total_tip_len = round(known_top_position_channel_head - (
        tip_bottom_z_coordinate - fitting_depth_of_all_standard_channel_tips - \
        unknown_offset_for_all_tips
      ),1)

      if total_tip_len in [50.4, 59.9, 95.1]: # 50ul, 300ul, 1000ul
        return total_tip_len
      else:
        raise ValueError(f"Tip of length {total_tip_len} not yet supported")

    else:
      raise ValueError(f"No tip present on channel {channel_idx}")

Does anyone have an idea where the unknown_offset_for_all_tips = 0.4 # mm come from?
They are also part of the tip definitions.
My best guess is that they represent the curvature between tip shaft and tip_fitting_depth compartment?

Maybe something similar would also be nice for the state of the liquid handler?

Where should this go though? It is fundamentally a property of LH. deck is externally managed (in the pylabrobot.resources package) and so both the frontend and backend can point to it.

request_tip_len_on_channel

I love this!
if total_tip_len in [50.4, 59.9, 95.1]: # 50ul, 300ul, 1000ul
        return total_tip_len
      else:
        raise ValueError(f"Tip of length {total_tip_len} not yet supported")

why not just return total_tip_len?

I love this!

There's almost always a code solution :)
Imagine the benefit to Hamilton:
Instead of having to create a new firmware command to do the same task...

which not only requires development time and money but would also require accessing every STAR(let) ever sold to be manually firmware updated by a Hamilton engineer (because in contrast to Opentrons machines, Hamilton machines are not online, i.e. their firmware cannot be remotely upgrade [to my knowledge]) -
...every STAR(let) user in the world can use this functionality as soon as this PR is merged, even if they have an older model/firmware :)

Caveat: to get this information the channel_head will move to the safe z-height=334.3 mm (pretty much the opposite of a danger to the machine, but creates a little bit of a bouncing movement)

why not just return total_tip_len?

Because we don't know any potential offset for other tips.
I could only measure the offset for 50ul, 300ul, and 1_000ul Hamilton CO-REII tips.
Slim tips, wide-bore tips and 10ul have not been tested. 10ul tips have a good chance of having the same offset.
But since we don't know the real source of this 0.4 mm offset yet, we cannot assume that for it will be the same for the remaining tips.
This way people who do want to use these tips have the ability to simply add the corresponding tip_len - after empirical validation - to this list.
(Though I would not recommend trying slim fit tips from a design perspective because of the above mentioned buckling issue, and wide-bore tips are unlikely to reach the very bottom narrow cavity, e.g. PCR_wellplate well bottoms)

So to catch up on some of the questions:

Does anyone have an idea where the unknown_offset_for_all_tips = 0.4 # mm comes from?

I can't say for sure on the STAR platform. I know of some internal position offsets / shifts on the Vantage / STAR V platform which are related to the CORE-I / CORE-II (tip coupling mechanisms) change.
On Vantage there should also be a easier way to get the tip length of the mounted tip, when I find the code snipped I can provide some guidance here.

... (because in contrast to Opentrons machines, Hamilton machines are not online, i.e. their firmware cannot be remotely upgraded [to my knowledge]) -

I can confirm this: Hamilton has currently no means to update (or access) the device remotly.

Maybe something similar would also be nice for the state of the liquid handler?

Where should this go though? It is fundamentally a property of LH. deck is externally managed (in the pylabrobot.resources package) and so both the frontend and backend can point to it.

Good question... And if the state of the LH is exposed, the next question would be "what is considered part of the state"?

i don't think it works with 50uL tips LOL (as you pointed out in 69, these tips are good for safety precisely because they are soft)

with 10uL tips it works though! I see a tiny bit of bending which makes me question the accuracy

how would we quantify accuracy? can we use the PLR model as a ground truth? (precision is easy to measure)

i don't think it works with 50uL tips LOL (as you pointed out in 69, these tips are good for safety precisely because they are soft)

with 10uL tips it works though! I see a tiny bit of bending which makes me question the accuracy

ohh wow, what detection_limiter_in_PWM and push_down_force_in_PWM settings are you using?
I would say though, yes, 50ul tips are the softest and buckle quite easily (even though I haven't seen them being crushed likes this before.

how would we quantify accuracy? can we use the PLR model as a ground truth? (precision is easy to measure)

The question here is Where does the PLR model come from?
Some resource dimensions are not realistically measurable by hand, e.g. the material_z_thickness of a well.
This new ztouch_probing finally gives us an automatable way of measuring this value.
This means we might be able to compare the ztouch_probing measurements to a PLR model because there is no PLR model yet.
-> My alternative: use the metal teaching needle that all machines already have:
Definition-wise it is just a 300ul / "ST" tip.
But it's metal (I believe stainless steel) and won't buckle.
Then take 5+ measurements of each position you want to z-measure and let the data guide you towards the true value.

One update I was thinking of after our last conversation that I would like to implement before we merge:
adding an "RF" (request firmware) command at the beginning of the ztouch_probing function.
Raise an error if firmware <2022, explaining that their machines PX module has to be upgraded to enable this functionality.

Really cool change that will make calibrating labware a lot easier!


Make the icons show in dev mode

The icons are not showing in the dev mode, because they are served from a different domain without CORS header.

@mxinden is there any reason to not use Elm reactor in dev mode to serve /lib files?

@w0rm I am not aware of any reason why we are not using Elm reactor to serve the libs. I haven't tried whether that works or not. Can you try it out and send a PR?

In case anyone is bugged by this you can use the following extension while developing to see the icons:
https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi


Add ADC3 internal inputs and example for internal temperature

Implemented Enabled/Disabled type states!
bors try

I will merge shortly if no further feedback

bors r+


allow users to provide an existing k8s secret

Hello, we are installing our helm charts using gitops, and we need a way to provide infra/logs token without specifying them in values directly (as this would require us to commit them to git).
I've adapted the chart so we can use it and i'm creating a PR with the same changes to see if this is something that you guys would like to have in the main chart.
Let me know what you think :)

Thanks for the PR @hpdobrica, I pushed small fixes for keeping compatibility.
For me is ready to be merged. @SohamChakraborty can review?


feat: update wording for upcoming campaign

https://byebuyglobaloperations.atlassian.net/browse/CHIN-1252

:tada: This PR is included in version 2.7.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Setup test suite

I created a testapp by running wagtail start testapp.
I also added boilerplate files.

Thanks @Tijani-Dia +1
You are on the right track. I do have a few comments.
1. Test folder structure
I'm not keen on having a tests folder inside a testapp folder. I think it should be the other way around, like wagtail-factories for example: https://github.com/wagtail/wagtail-factories/tree/master/tests
2. Running tests
As for running tests, I think we should consider using pytest in combination with tox to run tests. Instead of Django's manage.py test.
Pytest is a testing library that is a bit more advanced than Django's default testing mechanism. It supports a ton of useful plugins as well.
Tox is a tool that can be used to test a lot of different Python / package combinations. Since we support a few different Wagtail / Django combinations[1] I think we should properly test these combinations.
Take a look at wagtail-factories for inspiration on how pytest and tox are used: https://github.com/wagtail/wagtail-factories/blob/master/tox.ini
Next steps: add linting
In my opinion the next step (after merging this PR) would be to add linting with flake8 and isort. This ensures code in this package follows a code style and keeps things generally 'tidy'. We care about high-quality code.
Wagtail-factories uses flake8 and isort too: https://github.com/wagtail/wagtail-factories/blob/master/tox.ini#L38
This may all be a bit overwhelming sweat_smile
Please feel free to reach out to me or other mentors on Slack if you have any questions or need guidance implementing these suggestions.

Thanks @Stormheg  for the helpful feedback! I agree with you on all you said above and I will take it into consideration.
We talked about linting in the first meeting and If I understood we are going to use black instead of flake8 but I'm not sure.

We talked about linting in the first meeting and If I understood we are going to use black instead of flake8 but I'm not sure.

Black is great! I'm using black instead of flake8 in my own projects so I'd be keen to use black instead of flake8.

As discussed during today's progress meeting I've contributed improvements to the test setup.
@allcaps can you review this and flag anything that doesn't seem right?
I've added black and isort linting. I'd prefer to have some sort of command to automatically fix linting issues. Would a Makefile be okay? (e.g. make format, make lint..)

I've added black and isort linting.

I added flake8 because I found their logs useful except for the line too long one.
Do you mind keeping it?

Do you mind keeping it?

I don't mind at all, added it! 👍  Included a makefile as an easy shortcut to run all tools.
Waiting for your approval @allcaps

Black is a formatting tool and Flake coding style rules. They both have something to say about code style, but Flake has a much wider scope and Black fits within Flake rules.
So both is fine.
Concrete example. 'No unused variable' is a Flake rule (and a good one to be alerted about) bit black just doesn't care.
Flake and Black sometimes collide. Like with line length, the trick is to give Flake the same or longer length.
Did you consider isort? It is related and complements the Flake and Black.

Yes, I've been using all three.

Flake and Black sometimes collide. Like with line length, the trick is to give Flake the same or longer length.

Alright, black and isort collide too sometimes.

@Tijani-Dia isort conf can go into setup.cfg and look like:
[isort]
; Be black compatible
multi_line_output=3
include_trailing_comma=True
force_grid_wrap=0
use_parentheses=True
line_length=88

; Project
default_section = THIRDPARTY
known_first_party = testapp
known_django = django
known_wagtail = wagtail
sections = FUTURE,STDLIB,THIRDPARTY,DJANGO,WAGTAIL,FIRSTPARTY,LOCALFOLDER
skip = migrations/*

Oops, missed your comment @allcaps. Let's create a follow up PR for the isort config.


cmake compiler settings & logger improvement

Hello Dustin,
The goal of this PR was first to make the SCION::LOGGER consistent accross platforms and compilers which is not the case due to the variadic macro '##VA_ARGS' not working when Args... are empty on some compilers... I also wanted to reduce the number of macros and #ifdef WIN32, #ifdef __linux etc when possible.
To achieve this, I did the following changes:

Use of 'VA_OPT' instead which is conforming with stdc++11 and newer. It is working well but MSVC compiler requires to have the flag /Zc:__cplusplus in the command line...

To make this PR worth it and achieve this commit, I felt it might be the time to give the ability to the user of those libraries to be able to tweak the compiler, enable/disable warnings, etc...

Added 'CompilerSettings.cmake' in the root/cmake/' directory, populated with standard compiler and linker options (including /Zc:__cplusplus) - called from root CMakeLists.txt.

To enable the compilers settings accross subprojects, the following must be included in every project CMakeLists.txt (already updated accross the SCION libraries):
target_compile_options(<LIB_NAME> PRIVATE $<$<COMPILE_LANGUAGE:CXX>:${CXX_COMPILE_FLAGS}>)

Added 'CMakeSettings.json' in the root directory to allow the user to choose the build type from Visual Studio when working with cmake projects (Debug build are usually very slow when playing with OpenGL). 'CMakeSettings.json'  is automatically parsed by Visual Studio like cmake script files:

Other:

Fixed an annoying warning in 'Logger.inl' where GCC was complaining about static std:string being a non-POD type.
Slightly updated README.md

@333fps Hey Ronan, this is great. I did not know about Soil2.
Approved.

@333fps Hey Ronan, the only issue I just found after compiling the changes, I didn't notice in the review, was the change to the button (Play/Stop) colors in the scene display. Now both buttons can be green at the same time.

No problem though, I will push I fix on my next commit. I have a couple coming as well.
Thanks

Hello Dustin,,
Sorry for that... just pushed a new PR to fix this. It's just a proposal, but I guess it safer to not rely on 'imgui_internal.h" which is a private api and not stable.

@333fps Hey Ronan,
No problem at all, I appreciate all of the work and help that you are putting in. I do agree, the UI will become more sophisticated later on and it is better to not have to change ImGui if not needed.
Thanks.


Update theme styling to use lowercase, regular weight font

Requested by @timpaul

Note - I have used a hard 19px unit on mobile as I do not want it to change to 16px on mobile.

You can preview this change here:
Built with commit 4641b3642eb75145851c67f3b08d7edbead5180c
https://deploy-preview-243--govuk-design-system-preview.netlify.com

vs

@nickcolley I fixed the alignment yesterday, which was then merged quickly...I'll rebase

@nickcolley Should be good now...


Bug: TSDBDatabaseMetaData get metadata return "invalid system table name"

Environment:

server version: tdengine-<IP_ADDRESS>
jdbc version: 3.2.8
jdk version: JDK 17

Reproduce:
  private boolean isTableExist(Connection connection, String tableName) throws SQLException {
    var dbname = connection.getCatalog();
    var metadata = connection.getMetaData().unwrap(TSDBDatabaseMetaData.class);
    try (var ignore = metadata.getColumns(dbname, "", tableName, "")) {
      return true;
    } catch (SQLException e) {
      return false;
    }
  }

When running with the above code, exception is raised: TDengine ERROR (0x80002662): Fail to get table info, error: Invalid system table name.
This issue was introduced with the latest jdbc connector, and the same piece of codes run ok with 3.2.4 version.
Would you please help to investigate?

3.2.8 only support  TDengine <IP_ADDRESS>+


Simplify expanded menus

Expanded menus are using quantity queries to define the item widths. This generates a lot of unnecessary CSS output. There is no need to define a width if table-layout: fixed is used.
Here’s a little demo: http://codepen.io/anon/pen/zvQXQq
This could also be useful for button groups.

Yeah, this is something we're interested in. If you have time for a PR we'd LOVE that!


unimo-text采用fp16预测报错

请提出你的问题
unimo-text采用fp16预测报错，不采用fp16能够跑通
Infer begin...
Traceback (most recent call last):
File "inference.py", line 285, in
infer(args, predictor, data_loader, tokenizer)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 354, in _decorate_function
return func(*args, **kwargs)
File "inference.py", line 237, in infer
predictor.run()
ValueError: In user code:
File "export_model.py", line 101, in
do_predict(args)
File "export_model.py", line 93, in do_predict
os.path.join(args.inference_model_dir, "unimo_text"))
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py", line 631, in wrapper
func(layer, path, input_spec, **configs)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 51, in impl
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py", line 861, in save
inner_input_spec, with_hook=with_hook)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 528, in concrete_program_specify_input_spec
*desired_input_spec, with_hook=with_hook)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 436, in get_concrete_program
concrete_program, partial_program_layer = self._program_cache[cache_key]
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 801, in getitem
self._caches[item_id] = self._build_once(item)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 790, in _build_once
**cache_key.kwargs)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 51, in impl
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 733, in from_func_spec
outputs = static_func(*inputs)
File "/usr/local/lib/python3.7/dist-packages/paddlenlp/ops/faster_transformer/transformer/faster_transformer.py", line 1238, in forward
model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddlenlp/ops/faster_transformer/transformer/faster_transformer.py", line 1149, in prepare_inputs_for_generation
attention_mask = paddle.cast(attention_mask == 0, dtype="float16" if self._use_fp16_decoding else "float32")
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/layers/math_op_patch.py", line 347, in impl
attrs={'axis': axis})
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 3621, in append_op
attrs=kwargs.get("attrs", None))
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 2635, in init
for frame in traceback.extract_stack():
InvalidArgumentError: The type of data we are trying to retrieve does not match the type of data currently contained in the container.
[Hint: Expected dtype() == paddle::experimental::CppTypeToDataType::Type(), but received dtype():10 != paddle::experimental::CppTypeToDataType::Type():13.] (at /paddle/paddle/phi/core/dense_tensor.cc:137)
[operator < equal > error]

这个已经有PR在修复了 https://github.com/PaddlePaddle/PaddleNLP/pull/3936 ，可以先试下设置下环境变量 export FLAGS_enable_eager_mode=0

看报错应该不是 FLAGS_enable_eager_mode 的问题，能否再提供些使用的脚本以及如何执行的信息？


fix windows terminal output escape characters

This PR fixes Windows screen and piped output on cmd, PowerShell, and mintty terminals.

What does this PR change? What problem does it solve?
In Windows, piping the output of restic resulted in errant control characters being sent to the terminal / destination file.
Invoking restic backup ... > test.log before this fix:
using parent snapshot eee10060

creating VSS snapshot for [c:\]

successfully created snapshot for [c:\]
...

After this fix:
using parent snapshot eee10060
creating VSS snapshot for [c:\]
successfully created snapshot for [c:\]
...

Was the change discussed in an issue or in the forum before?
Closes #3111

Checklist

[x] I have read the Contribution Guidelines
[x] I have enabled maintainer edits for this PR
[ ] I have added tests for all changes in this PR
[ ] I have added documentation for the changes (in the manual)
[ ] There's a new file in changelog/unreleased/ that describes the changes for our users (template here)
[x] I have run gofmt on the code in all commits
[x] All commit messages are formatted in the same style as the other commits in the repo
[x] I'm done, this Pull Request is ready for review

This is my first PR into the project. I was looking into the history of #3111 and noticed that these checks were removed between 0.11 and 0.12, when the issue first appeared for me. I reverted them and tested them and it looks to be the root cause, so I thought I'd issue the PR vs. just telling you all about it.

Newbie question - how can I get an .exe built with this patch to test?
Is building it myself from source the only option, until it gets merged/released?
Thanks.

@jdiercks There are no prebuilt binaries for PRs. So you'll have to checkout this branch and the build it using go build ./cmd/restic. Or if you have a linux (or similar) machine then run GOOS=windows GOARCH=amd64 go build ./cmd/restic.
This PR should only be thought of as a temporary workaround. It won't be merged in its current form.

OK thanks @MichaelEischer. I'd be willing to test and report results, but don't have the time to set up a build environment at the moment. Not a critical condition for me, just a nuisance that the output is messy. I'l wait for a proper full fix release.

@MichaelEischer - I've tested this on cmd, PowerShell, and mintty.
In all terminals, terminal.IsTerminal(int(fd)) returned true if I was running directly on the terminal. And false if the output was piped to a file or command. This simplified the code dramatically.
I can confirm that status updates only are shown for non-piped output, and that there are no errant characters.
I'm unfamiliar with the go.sum linting errors that are occurring - I'm new to go, so any help resolving those would be appreciated.

I've tested this on cmd, PowerShell, and mintty.

@kmwoley Which version of mintty did you use exactly? For me mintty included in MSYS2 still uses pipes on Windows 10.

@kmwoley Which version of mintty did you use exactly? For me mintty included in MSYS2 still uses pipes on Windows 10.

I tested with Git Bash terminal that installed with Git for Windows - my understanding is that it is a mintty terminal. I may be mistaken there.

@MichaelEischer - what do you think? Should I close this PR in favor of the one you put together?

I think we should use PR #3325, so let's close this one.


added goartistapi to list of music apis

https://goartistapi.xyz
Thank you for taking the time to work on a Pull Request for this project!
To ensure your PR is dealt with swiftly please check the following:

[x] Your submissions are formatted according to the guidelines in the contributing guide
[ ] Your additions are ordered alphabetically
[ ] Your submission has a useful description
[ ] The description does not end with punctuation
[ ] Each table column should be padded with one space on either side
[ ] You have searched the repository for any relevant issues or pull requests
[ ] Any category you are creating has the minimum requirement of 3 items
[ ] All changes have been squashed into a single commit

Please address build failures- thanks.


[Feature]: cubefs surpport autofs mount

Is your feature request related to a problem? Please describe.
build  ladp+automount+cubefs-client automition mount workflow，Support for easier mount management
Describe the solution you'd like
cubefs launches tools and detailed process instructions to help users quickly use autofs+cubefs

@leonrayang good job

done: https://github.com/cubefs/cubefs/pull/1973