Deprecated joblib

If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS 10.15.4
TensorFlow/Keras version: 1.14.0/2.3.1
Python version: 3.7.0

Describe the problem
In the project setup.py requirements there is a dependency on 'scikit-learn>=0.19.1', but in the module anago.preprocessing there is a dependency on joblib from sklearn.externals which is deprecated after scikit-learn version 0.23.
Source code / logs
https://github.com/Hironsan/anago/blob/80efcfe9dc6781afaa1a349c2beb6f8df95a5f47/anago/preprocessing.py#L10
 File "/usr/lib/python3.7/site-packages/anago/__init__.py", line 3, in <module>
    from anago.wrapper import Sequence
  File "/usr/lib/python3.7/site-packages/anago/wrapper.py", line 7, in <module>
    from anago.preprocessing import IndexTransformer
  File "/usr/lib/python3.7/site-packages/anago/preprocessing.py", line 9, in <module>
    from sklearn.externals import joblib
ImportError: cannot import name 'joblib' from 'sklearn.externals' (/usr/lib/python3.7/site-packages/sklearn/externals/__init__.py)

Temp fix:
pip uninstall scikit-learn
pip install scikit-learn==0.21.3


Types auto imported by Nuxt can't be used for defining components' props

Environment

Operating System: Darwin
Node Version:     v18.17.1
Nuxt Version:     3.8.2
CLI Version:      3.10.0
Nitro Version:    2.8.0
Package Manager<EMAIL_ADDRESS>Builder:          -
User Config:      devtools, imports
Runtime Modules:  -
Build Modules:    -

Reproduction

go to https://github.com/DawidKopys/nuxt-props-with-global-types/ and clone this repo
npm i
npm run type-check

Expected result: we get typescript error in App.vue in place of using CompUsingGlobalType1 as we are passing number instead of string to its prop foo.bar.
Actual result: there is no typescript error in above place.
Describe the bug
First of all, let me thank all of the people involved in developing Nuxt.
Now back to the issue:
It seems that types auto imported by Nuxt can't be used for defining components' props (without importing them explicitly, for example from #imports). In fact it's even worse - they can be used, we don't get any ts errors, it just doesn't seem to work. It looks that it fails silently.
What is more, it also presents that globally available types can be used for defining props, just not if they are made available globally the way it is currently done in Nuxt.
To keep it brief:

global types added to the global scope in the way Nuxt does that don't work when used for defining vue components' props. It seems that Nuxt is generating something alike below snippet inside .nuxt/types/imports.d.ts

declare global {
  // @ts-ignore
  export type { GlobalExternalType1 } from './GlobalExternalType1'
}

it seems that it could work, if Nuxt was generating something like below instead:

declare global {
  interface GlobalExternalType2 {
    bar: string
  }
}

It would possibly require to copy over entire types' declarations in order to do that. But maybe it wouldn't be a problem since TypeScript is a  structural type system and when comparing types, TypeScript only takes into account the members on the type.
Additional context
For additional info see https://github.com/DawidKopys/vue-props-with-global-types#readme as well. The same issue but with plain vue project, without Nuxt.
Logs
No response

I think this would work automatically if we did explicitly add a type import in the unimport transform. We can possibly fix this upstream. What do you think @antfu?

@danielroe it's not the same as transform, because this is type checking problem that happens in the editor instead of the pipeline.

After a bit of digging, this is quite a fun behavior. The type does not work because they are type-only modules. If you add any runtime export like export const a = 1, it will magically work. It turns out that export type  will not make TypeScript to look into that file, so it only knows there is an export but the type structure is never initiated (so it fallbacks to any). The fix is actually simple, we need to add an import() statement in the dts to reference that file, forcing TS to initiate that file.
Made a fix in unimport v3.6.0: https://github.com/unjs/unimport/commit/17f28945eb5572986e27a387a1d1aa3bb55fd3a1
@DawidKopys You can fix the issue by rebuilding your lock, or force your package manager to use the latest unimport. In npm, it would be:
  "overrides": {
    "unimport": "^3.6.0"
  }

I can comfirm that bumping to<EMAIL_ADDRESS>did fix the issue. Thanks a lot for super quick fix!

Para mim não funcionou no nuxt 3.11


Fix for issue #1733

Description
Solves issue #1733, but needs checking from people, who possibly could explain why textures in RenderTexture::display() previously have set m_texture.m_pixelsFlipped = true;

This does not fix the problem.
See my comment in the original issue.

This does not fix the problem.
See my comment in the original issue.


Stale || Fetching doesn't actually represent when operations are in progress while offline

Describe the bug
When using the offlineExchange there is no way to tell if values are still being loaded or not. Based on the documentation a union between fetching || stale should represent if there is a potentially new value still being loaded, however this is not the case.
In the below repro turn off the network and open console. You will see the following

fetching and stale are both false and the data is null
then a values comes back while fetching and stale are still both false.
This means there is effectively no way to tell if a query is 'settled' or not when the user is offline.
It appears like this occurs with all relevant cache policies as well.
Reproduction
https://stackblitz.com/edit/github-cnyjhe?file=src%2FApp.jsx
Urql version
"dependencies": {
"@urql/core": "^4.2.0",
"@urql/exchange-graphcache": "^6.4.0",
"graphql": "^16.6.0",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"urql": "^4.0.6"
},
Validations

[X] I can confirm that this is a bug report, and not a feature request, RFC, question, or discussion, for which GitHub Discussions should be used
[X] Read the docs.
[X] Follow our Code of Conduct

When the user is offline we settle the query and put it on a queue 😅 you are right in saying this is an unobservable event atm

After thinking about this, when the user goes offline we don't really want to show these as in progress as they're operations that got aborted by the network going offline and we put it in an internal queue for when the user comes back online. Depending how folks do loading, i.e. Suspense for a query could become problematic or if the whole form is blocked out the user can doubt whether it's worth navigating away from the page it could confuse the user... I personally think marking it as resolved and leaving the UX concerns up to the application to highlight "user you are offline" is correct in this context

The main issue here for me is that its impossible to determine if urql is still doing something in the background. Assuming we want to redirect to a new page if some information cannot be found in the cache while the user is offline, there is no information surfaced by urql to tell us that it is safe to redirect.
There is no loading state
There is no stale state
There is no error state
There is no data
It appears like the query has been settled, and then some new information comes in.
Currently we just use a timeout after everything is settled and just wait to see if new data comes. This is not really a solution just a bandaid. If we rely on navigator.onLine this is still just a bandaid solution because we will just need to set a timeout and see if any extra information comes.

Hiya 👋
Just to highlight the problem here.
It's impossible for us to indicate that someone is offline, hence the default that has been chosen. Even if there was a reliable property or signal for us to detect this, network requests may still fail.
In other words, there isn't a 100% fail safe method of detecting that a device is offline due to network instability issues.
To then explain what's going on, from a bindings perspective it's important to note that fetching: false only indicates that no result is actively being waited for.
Meanwhile, stale: true means that an operation has been initiated and a current result is expected to be updated.
However, when offline, even if we set stale: true there is nothing to be gained here. All that it's telling you is that something is still actively being fetched, which may or may not fail.
There are other ways to indicate this to the user, but doing so at the site of the bindings rather than globally doesn't gain you much imho.

Assuming we want to redirect to a new page if some information cannot be found in the cache while the user is offline, there is no information surfaced by urql to tell us that it is safe to redirect.

This sounds more like a schema design issue to me though.
You have neither a value (like a special object type on a union) nor null here to check for.
So, i'm not sure what you're expecting to check for, but this state seems unambiguous to me. Even if you have an offline cache, when you fail to retrieve an up-to-date value, you have no choice but to trust the stored/cached value and to use it.
Basically, it's irrelevant that the request failed if you have no retry on it and are offline. Either you got some data you can act on or you don't 😅
This is where some opinions shine through, but tl;dr a complete solution would likely have a special signal/value associated with onOnline on storage that corresponds to the device being online again.
However, to put it short, without us injecting an opinion on how that online/offline state is kept, while also handling that any fetch request can fail regardless of this value/network state due to the device being offline and this value being out-of-date, there simply isn't something useful that the UI bindings side can surface.
To make it obvious, even if we set stale back to true, you would then be unaware and uncertain when it may do so. Just like a result being able to come in any time, this is also something that could then happen any time.

Closing this because of inactivity and the question itself being answered. Feel free to re-open or continue this discussion


File not available after validation in SPA mode

I am trying to use a file input in SPA mode.
While the formData seems to be updated correctly, once submitted (and successfully validated), no file information is present.
Not sure if I am doing something wrong, or if this is a bug. Any help is much appreciated :)
Reproduction
I modified the file upload example to be purly client side. Issue can be reproduced by selecting a file, and then hitting submit. The message that should display the submitted form data shows no file information.
https://www.sveltelab.dev/q3nc7s4qqzustmz?files=.%2Fsrc%2Froutes%2F%2Bpage.svelte

You can't JSON.stringify a File, you need to access its properties directly. Try setMessage(form, form.data.image.size) for example and you'll see the real value.


Model is not running

What is the issue?
PS C:\Users\Admin> ollama run mistral-nemo
Error: llama runner process has terminated: error loading model: unable to allocate backend buffer
whenever I try to run this model it shows me this error I don't know how to solve it.. i have 16gb ram with rtx3050
OS
Windows
GPU
Nvidia
CPU
AMD
Ollama version
0.3.9

Server logs will aid in debugging.

@rick-github Thanks for the help after checking server logs I found that my memory on the computer was insufficient.


Installation issues on Debian 8.9

Hello,
During the project installation on a Debian version 8.9 I have the following issues:
"CMake Error at /usr/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:136 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/usr/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:343 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake-3.0/Modules/FindThreads.cmake:178 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
src/CMakeLists.txt:235 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/audrey/Bureau/USBWALL/usbwall-master/_build/CMakeFiles/CMakeOutput.log".
See also "/home/audrey/Bureau/USBWALL/usbwall-master/_build/CMakeFiles/CMakeError.log"."
The project's compilation stops when it checks if the compiler accepts -pthread. See the error bellow:
"-- Check if compiler accepts -pthread - no
CMake Error at /usr/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:136 (message): ... "
The version of the compiler is GNU 4.9.2 : -- The C compiler identification is GNU 4.9.2

Hi,
this is a bug from cmake on Debian 8.9 when using gcc.
A working solution is here : https://stackoverflow.com/questions/24813827/cmake-failing-to-detect-pthreads-due-to-warnings/25130590#25130590
You could update your distribution to stable (Debian 9) or use clang instead of gcc.


Cannot read Spur images

Squeak 5 is in Spur format, we need to support it.

Is this fixed now? :)

Yep.


Update boilerplate.txt

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Deploy preview for docsdocker ready!
Built with commit 398d26622ec0d1e2120f9236c1b1a752f81d76aa
https://deploy-preview-7594--docsdocker.netlify.com

Looks like this was just for testing GitHub; let me close


Known Tested Modems: replace SIM7100E with SIM7600E

The docs mention we support the SIM7100E modem: https://www.balena.io/docs/reference/OS/network/2.x/#known-tested-modems-
meta-balena doesn't mention the SIM7100E but the SIM7600E instead:
https://github.com/balena-os/meta-balena/blob/master/README.md#modems

@floion and @shaunmulligan have confirmed that "meta-balena is correct, the SIM7100E is deprecated in favour of the 7600."
 Front conversations

closed by https://github.com/balena-io/docs/pull/1072


Add support for controls API

The example at https://developers.google.com/maps/documentation/javascript/geolocation allows you to add custom buttons to the map easily, via map.controls[google.maps.ControlPosition.TOP_CENTER].push(locationButton);. It would be excellent to expose this API in Java, so that I could add my own "center" Vaadin button easily. Relates to #114

This code seems to work:
map.getElement().executeJs("this.map.controls[google.maps.ControlPosition.TOP_CENTER].push($0);", button);

but it's extremely limited: the map must be visible and initialized (otherwise the JS code will throw an exception that this.map is null); also the button needs to be added in the layout somewhere in the current UI. Investigating further.

Okay, this works well-ish:
			map.getElement().addEventListener("google-map-ready", e -> {
				final Button button = new RoundIconButton(VaadinIcon.CROSSHAIRS);
				map.getElement().appendChild(button.getElement());
				map.getElement().executeJs("this.map.controls[google.maps.ControlPosition.RIGHT_BOTTOM].push($0);", button);
			});

Couldn't find the docs for ControlPosition, here are couple of links I found:

https://developers.google.com/maps/documentation/javascript/examples/control-positioning
https://stackoverflow.com/questions/27769309/google-maps-javascript-positioning-control-maps

Looks like there are the following available constants: TOP_CENTER, BOTTOM_RIGHT, MIDDLE_LEFT and their combinations (TOP_LEFT) etc. Looks like swapped names work as well, e.g. RIGHT_BOTTOM.


"Killed" (processing from 3MP input sample mp4)

When processing a small rectangle out of a large video, trainbot seems to detect a train but then ends up "Killed." (exit code 137) inside my VM which has 3.8Gi of RAM plus 11Gi of swap added which lets it run slightly longer after starting to process a train. This is with a local sample file which it processes through ffmpeg. The source file is 2304 × 1296 pixels but the crop region is only e.g. 192 x 133.
If I pre-crop the video sample video e.g. ffmpeg -i sample-rot.mp4 -filter:v "crop=198:133:1403:284" sample-crop.mp4 then I am able to completely process the feed with trainbot. So I wonder if it is somehow trying to keep in memory not just the crop region but all the original (whole) frames too?

We crop the image and then pass around only the cropped image. The original frame is thrown away.
However, I have as slight suspicion for where this could be going wrong. Please check commit 03bcf15b035c385913321b2b87c3f40eaae5a28c to see if that fixes the problem.

@jo-m Had a chance to circle back to this this evening. Using the trainbot-arm64 from the binaries archive on https://github.com/jo-m/trainbot/actions/runs/7159296241 (so the most recent https://github.com/jo-m/trainbot/commit/8afc520a5921451cf4947941b668e64b3960d6ae rather than the original, but afaict the fix should still be in that) —
unfortunately I still get Killed. on a large file when used directly.

Hm, any chance you could send me an example video? Can be a link to GDrive via email or similar.

So, the options would be

reduce frame rate, especially if you have slow but long trains
scale down video

The issue is simple - it eats up too much memory, it ate all my 16GiB or RAM and then 16GiB of swap and then crashed by OOM-killer.

Ah, okay so iiuc it's now no longer a memory "leak" (i.e. unintentional) but just hitting a large amount of memory expected to be used now? That would make sense since my own testing is on VMs without much allocated probably shouldn't even both on the Raspberry Pi itself then which is already past its limit just providing the full-size feed itself 😂
Thanks for looking into this one too, and glad you could get this one closed out even if just something I need to avoid 👍


Switch CP4D to express installation pattern

To get CP4D & DB2 modules delivered, switch to the express installation pattern, as defined here (everything in one namespace):
https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=installing
However, this is just a temporary solution.  Long term we need to be able to support the specialized installation method that is described in the same link (This is where we originally started).   It is a very real scenario that OpenShift admins will want to have deployed resources in different namespaces.  We will need to be able to support the specialized configuration, and I've created a separate tech debt issue to bring it back here:  https://github.com/cloud-native-toolkit/software-everywhere/issues/191

Perhaps we can make this a installation option... the user can choose whether they want express or specialized installation.

currently we are using custom install.. we will revisit if there is interest from GSI


added solution for Bonfire : Exact Change

works for all given test cases.

Confirmed.
Excellent work, thank you!
:+1:

glad to help :)


<pull 108>JSON structure for deregistration response in WINNF.FT.C.RLQ.5 is incorrect

JSON structure for deregistration response in WINNF.FT.C.RLQ.5 is incorrect

"deregistrationResponse" :
			[
				{
				  "response": {
					 "responseCode": 103,
					 "responseData": "cbsdId"
				  }
				}
			]
		}

}
As response data is list of strings. So data type need to be corrected.

I think you mean WINNF.FT.C.DRG.5 instead be WINNF.FT.C.RLQ.5?
I ask this as I see deregistrationResponse.

yes .. updated the same


Unit Testing

We should write tests.

In place and running at 75% coverage on RestCollection. Need to test GraphService and util

Yay! ed40e9fccbcf493a2fdacca9a9c20dbb90e231a5

I'll close this out once tests are set up for the rest of the modules.


Add ports folder for building platform

/check-enforcer reset

@RickWinter, was this PR superseded by https://github.com/Azure/azure-sdk-for-c/pull/904? Can we close this one?

Tracking via new PR


repo.changset with custom UpdateChangeset

Support using a custom UpdateChangsets in rom-repository.
I basically wanna do user_repo.changeset(user.id, params) but with my own UpdateUserChangeset class

Hey @kwando, it looks like you'd need to do:
(repo.changeset(UpdateUserChangeset).data(id: user.id, **params))[http://rom-rb.org/learn/repositories/custom-changesets/]
Does that achieve what you wanted?

That's not gonna work, the only way to restrict changet's relation by PK is to use changeset(pk_value, new_data) so this does require extending existing API. I don't have any idea how it should look like though. Maybe:
user_repo.changeset(user.id, params, UpdateUserChangeset)

?

Why not take the pk from the data?

@jasper-lyons because it's too implicit. you may have PKs like UUID and then we can't determine whether it's meant to be a new tuple or we want to update existing one

That makes sense. Does 'by_pk()' return an object you could call changesets on?

Yes. Structs are coercible to hashes so they work with changesets and commands

My immediate thoughts here were to do something like:
user_repo.by_pk(user.id).changeset(params, UpdateUserChangeset)

But that is quite the detraction from the existing API.

@jasper-lyons I'm glad you mentioned this, if you take a look at the rom 4.0 roadmap I posted earlier today, you'll see that it talks about moving changeset to core, or to rom-changeset. This would be a good opportunity to think about having Relation#changeset method, which is almost what you proposed. The difference is that repository methods are meant to return data, not relations, so this means we could have this:
users.by_pk(user.id).changeset(params, UpdateUserChangeset)

...however, this is stuff for rom 4.0, now we have changeset in repos only :)


[Question]: Parsing batches of files stuck at 50%

Describe your problem
When a batch of documents are parsed all of them are stalled at 50% forever. Manual restart to reparse one by one again (and sometime 3 times) is necessary.
In addition pls kindly add support for AMD GPU acceleration.  Ragflow appears to only support Nvidia by using docker-compose-gpu.yml.   I cannot find corresponding yml file for AMD. Am I missing something? Much appreciated for your assistance.  I installed ragflow docker locally with Ollama deploying "nomic-embed-text "in Ubuntu 24.04.

driver: nvidia
Maybe change the driver in docker compose yaml?
You need to start multiple task executor to enlarge the throughput of parsing, please enter the container and check this out.

The solution to this problem is to start multiple instances of the task_executor.py program, correct?

driver: nvidia
Maybe change the driver in docker compose yaml?
You need to start multiple task executor to enlarge the throughput of parsing, please enter the container and check this out.

Sorry. I changed the driver to amdgpu. Docker compose did not fire up. Out of ideas I even attempted to add these and that did not work either:
devices:
- /dev/kfd:/dev/kfd
- /dev/dri:/dev/dri
environment:
- 'HSA_OVERRIDE_GFX_VERSION=${HSA_OVERRIDE_GFX_VERSION-11.0.0}'
Sorry I have no idea how to run multiple task_executor.py (thanks to Darry's help) in a container.

driver: nvidia
Maybe change the driver in docker compose yaml?
You need to start multiple task executor to enlarge the throughput of parsing, please enter the container and check this out.

Sorry. I changed the driver to amdgpu. Docker compose did not fire up. Out of ideas I even attempted to add these and that did not work either: devices: - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri environment: - 'HSA_OVERRIDE_GFX_VERSION=${HSA_OVERRIDE_GFX_VERSION-11.0.0}' Sorry I have no idea how to run multiple task_executor.py (thanks to Darry's help) in a container.

We're shrinking the RAGFlow docker image size by removing some python modules and some of the built-in modeling. This will be finished before next release. This might make RAGFlow run in your environment.


time cache causes the timeout to expire and the worker process enters an endless loop

What
the worker process may go into a dead cycle after the reload command is executed. as you can see, there are multiple master processes：

sudo strace -p 15399:
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147
nanosleep({0, 10000000}, 0x7ffca30db420) = 0
write(10, "2021/06/25 17:12:37 [warn] 15399"..., 147) = 147

sudo pstack 15399:
#0  0x000000318e00eff0 in __nanosleep_nocancel () from /lib64/libpthread.so.0
#1  0x00007fad7ee8a39e in timeout_lua_sleep () from /home/webedit/yanxuan-ianus/luarocks/lib/lua/5.1/socket/core.so
#2  0x00007fadc3cdf6ca in lj_BC_FUNCC () from /home/webedit/yanxuan-ianus/luajit/lib/libluajit-5.1.so.2
#3  0x00007fadc3cef8ad in lua_pcall () from /home/webedit/yanxuan-ianus/luajit/lib/libluajit-5.1.so.2
#4  0x0000000000515271 in ngx_http_lua_do_call ()
#5  0x000000000052751b in ngx_http_lua_init_worker_by_inline ()
#6  0x00000000005271c9 in ngx_http_lua_init_worker ()
#7  0x0000000000479da6 in ngx_worker_process_init ()
#8  0x000000000047a30b in ngx_worker_process_cycle ()
#9  0x0000000000478e14 in ngx_spawn_process ()
#10 0x000000000047a7b4 in ngx_start_worker_processes ()
#11 0x000000000047b1ab in ngx_master_process_cycle ()
#12 0x0000000000454d58 in main ()

Why
https://github.com/Kong/lua-resty-worker-events/blob/master/lib/resty/worker/events.lua#L384
ngx.now()  is cache time. when get_event_data returns nil for both data and err, the time may not be updated. so, the program loops through sleep.
Solution
https://github.com/Kong/lua-resty-worker-events/pull/42

Good catch! thank you very much.
I think though that the problem has some more nuances to it. During a sleep the time will be updated, the problem however occurs if we're in a phase that cannot yield, like init. Because then the pcall fails, the sleep isn't executed, and then the deadlock occurs. In a phase where yielding is supported, the sleep does get executed and the cached time will be updated at some point.
This also means that the provided solution is not correct, because the pcall fails, it won't sleep, it will loop _wait_max / _wait_interval times and exit immediately (within a fraction of a second, no matter how _wait_max or _wait_interval have been set).

I think we should update;
pcall(sleep, _wait_interval)

with
if not pcall(sleep, _wait_interval) then
  -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
  -- must force update the cached time reported by `ngx.now`.
  ngx.update_time()
end

and then that should fix the issue. Wdyt @jinhailang ?

I think we should update;
pcall(sleep, _wait_interval)

with
if not pcall(sleep, _wait_interval) then
  -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
  -- must force update the cached time reported by `ngx.now`.
  ngx.update_time()
end

and then that should fix the issue. Wdyt @jinhailang ?

That's right, but I'm referring to the https://github.com/openresty/lua-resty-lock/blob/master/lib/resty/lock.lua#L155 solution, maybe it's more efficient?
resty.lock:
while timeout > 0 do
        sleep(step)
        elapsed = elapsed + step
        timeout = timeout - step

        local ok, err = dict:add(key, true, exptime)
        if ok then
            cdata.key_id = ref_obj(key)
            self.key = key
            return elapsed
        end

        if err ~= "exists" then
            return nil, err
        end

        if timeout <= 0 then
            break
        end

        step = min(max(0.001, step * ratio), timeout, max_step)
    end

That code will not run in the init phase. Whereas worker-events will do a busy-wait in the init phase, which isn't a problem since it is single threaded anyway, and runs very short during startup.

@jinhailang will you update the PR? (and sign the CLA) otherwise I'll commit the fix myself. It's severe enough that it warrants a swift fix and release imo.

@jinhailang will you update the PR? (and sign the CLA) otherwise I'll commit the fix myself. It's severe enough that it warrants a swift fix and release imo.

I have resubmitted it  https://github.com/Kong/lua-resty-worker-events/pull/43

merged! Thanks again @jinhailang !

2.0.1 released and pushed to LuaRocks

Good catch! thank you very much.
I think though that the problem has some more nuances to it. During a sleep the time will be updated, the problem however occurs if we're in a phase that cannot yield, like init. Because then the pcall fails, the sleep isn't executed, and then the deadlock occurs. In a phase where yielding is supported, the sleep does get executed and the cached time will be updated at some point.
This also means that the provided solution is not correct, because the pcall fails, it won't sleep, it will loop _wait_max / _wait_interval times and exit immediately (within a fraction of a second, no matter how _wait_max or _wait_interval have been set).

Hello, i tested it today and found that pacall did not return an error in "init_worker", can you try it on?

@jinhailang where did you test? in case you tested with Kong, that carries a patch to have sleep work in the init_worker phase (becasue we needed lua-resty-lock to work). See https://github.com/Kong/kong/blob/master/kong/globalpatches.lua#L42-L61

when testing with plain OpenResty, calling ngx.sleep(1) gives me;
2021/07/04 07:41:56 [notice] 94177#0: start worker process 94178
2021/07/04 07:41:56 [error] 94178#0: init_worker_by_lua error: ./resty-test.lua:8: API disabled in the context of init_worker_by_lua*
stack traceback:
        [C]: in function 'sleep'
        ./resty-test.lua:8: in function <./resty-test.lua:7>
        init_worker_by_lua:2: in main chunk

@jinhailang where did you test? in case you tested with Kong, that carries a patch to have sleep work in the init_worker phase (becasue we needed lua-resty-lock to work). See https://github.com/Kong/kong/blob/master/kong/globalpatches.lua#L42-L61

I know that. I mean pcall(sleep, _wait_interval) doesn't return an error, so our previous fix didn't solve the problem? You can try.
if not pcall(sleep, _wait_interval) then
  -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
  -- must force update the cached time reported by `ngx.now`.
  ngx.update_time()
end

hmmm....
when I run this:
local app = {}

local sleep = ngx.sleep
local _wait_interval = 0.1

function app.init()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init failed to sleep!")
  end
end

function app.init_worker()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init-worker failed to sleep!")
  end

  ngx.log(ngx.ERR, "one")
  local ok, err = pcall(ngx.sleep,1)
  ngx.log(ngx.ERR, "two", ok, err)
  ngx.sleep(1)
  ngx.log(ngx.ERR, "three")
end

I get this:
2021/07/05 12:57:42 [notice] 87500#0: [lua] resty-test.lua:11: init failed to sleep!
2021/07/05 12:57:42 [notice] 87500#0: using the "kqueue" event method
2021/07/05 12:57:42 [warn] 87500#0: 1024 worker_connections exceed open file resource limit: 256
2021/07/05 12:57:42 [notice] 87500#0: openresty/<IP_ADDRESS>
2021/07/05 12:57:42 [notice] 87500#0: OS: Darwin 20.4.0
2021/07/05 12:57:42 [notice] 87500#0: hw.ncpu: 12
2021/07/05 12:57:42 [notice] 87500#0: net.inet.tcp.sendspace: 131072
2021/07/05 12:57:42 [notice] 87500#0: kern.ipc.somaxconn: 128
2021/07/05 12:57:42 [notice] 87500#0: getrlimit(RLIMIT_NOFILE): 256:9223372036854775807
2021/07/05 12:57:42 [notice] 87501#0: start worker processes
2021/07/05 12:57:42 [notice] 87501#0: start worker process 87502
2021/07/05 12:57:42 [notice] 87502#0: *1 [lua] resty-test.lua:20: init-worker failed to sleep!, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:23: one, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:25: twofalseAPI disabled in the context of init_worker_by_lua*, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: init_worker_by_lua error: ./resty-test.lua:26: API disabled in the context of init_worker_by_lua*
stack traceback:
	[C]: in function 'sleep'
	./resty-test.lua:26: in function <./resty-test.lua:15>
	init_worker_by_lua:2: in main chunk

So I see everything I expect to see, only the last log line isn't logged because the non-protected sleep just before breaks the code flow.

hmmm....
when I run this:
local app = {}

local sleep = ngx.sleep
local _wait_interval = 0.1

function app.init()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init failed to sleep!")
  end
end

function app.init_worker()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init-worker failed to sleep!")
  end

  ngx.log(ngx.ERR, "one")
  local ok, err = pcall(ngx.sleep,1)
  ngx.log(ngx.ERR, "two", ok, err)
  ngx.sleep(1)
  ngx.log(ngx.ERR, "three")
end

I get this:
2021/07/05 12:57:42 [notice] 87500#0: [lua] resty-test.lua:11: init failed to sleep!
2021/07/05 12:57:42 [notice] 87500#0: using the "kqueue" event method
2021/07/05 12:57:42 [warn] 87500#0: 1024 worker_connections exceed open file resource limit: 256
2021/07/05 12:57:42 [notice] 87500#0: openresty/<IP_ADDRESS>
2021/07/05 12:57:42 [notice] 87500#0: OS: Darwin 20.4.0
2021/07/05 12:57:42 [notice] 87500#0: hw.ncpu: 12
2021/07/05 12:57:42 [notice] 87500#0: net.inet.tcp.sendspace: 131072
2021/07/05 12:57:42 [notice] 87500#0: kern.ipc.somaxconn: 128
2021/07/05 12:57:42 [notice] 87500#0: getrlimit(RLIMIT_NOFILE): 256:9223372036854775807
2021/07/05 12:57:42 [notice] 87501#0: start worker processes
2021/07/05 12:57:42 [notice] 87501#0: start worker process 87502
2021/07/05 12:57:42 [notice] 87502#0: *1 [lua] resty-test.lua:20: init-worker failed to sleep!, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:23: one, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:25: twofalseAPI disabled in the context of init_worker_by_lua*, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: init_worker_by_lua error: ./resty-test.lua:26: API disabled in the context of init_worker_by_lua*
stack traceback:
	[C]: in function 'sleep'
	./resty-test.lua:26: in function <./resty-test.lua:15>
	init_worker_by_lua:2: in main chunk

So I see everything I expect to see, only the last log line isn't logged because the non-protected sleep just before breaks the code flow.

hmmm....
when I run this:
local app = {}

local sleep = ngx.sleep
local _wait_interval = 0.1

function app.init()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init failed to sleep!")
  end
end

function app.init_worker()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init-worker failed to sleep!")
  end

  ngx.log(ngx.ERR, "one")
  local ok, err = pcall(ngx.sleep,1)
  ngx.log(ngx.ERR, "two", ok, err)
  ngx.sleep(1)
  ngx.log(ngx.ERR, "three")
end

I get this:
2021/07/05 12:57:42 [notice] 87500#0: [lua] resty-test.lua:11: init failed to sleep!
2021/07/05 12:57:42 [notice] 87500#0: using the "kqueue" event method
2021/07/05 12:57:42 [warn] 87500#0: 1024 worker_connections exceed open file resource limit: 256
2021/07/05 12:57:42 [notice] 87500#0: openresty/<IP_ADDRESS>
2021/07/05 12:57:42 [notice] 87500#0: OS: Darwin 20.4.0
2021/07/05 12:57:42 [notice] 87500#0: hw.ncpu: 12
2021/07/05 12:57:42 [notice] 87500#0: net.inet.tcp.sendspace: 131072
2021/07/05 12:57:42 [notice] 87500#0: kern.ipc.somaxconn: 128
2021/07/05 12:57:42 [notice] 87500#0: getrlimit(RLIMIT_NOFILE): 256:9223372036854775807
2021/07/05 12:57:42 [notice] 87501#0: start worker processes
2021/07/05 12:57:42 [notice] 87501#0: start worker process 87502
2021/07/05 12:57:42 [notice] 87502#0: *1 [lua] resty-test.lua:20: init-worker failed to sleep!, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:23: one, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:25: twofalseAPI disabled in the context of init_worker_by_lua*, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: init_worker_by_lua error: ./resty-test.lua:26: API disabled in the context of init_worker_by_lua*
stack traceback:
	[C]: in function 'sleep'
	./resty-test.lua:26: in function <./resty-test.lua:15>
	init_worker_by_lua:2: in main chunk

So I see everything I expect to see, only the last log line isn't logged because the non-protected sleep just before breaks the code flow.

Did you  tested with Kong?

hmmm....
when I run this:
local app = {}

local sleep = ngx.sleep
local _wait_interval = 0.1

function app.init()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init failed to sleep!")
  end
end

function app.init_worker()
  if not pcall(sleep, _wait_interval) then
    -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
    -- must force update the cached time reported by `ngx.now`.
    ngx.update_time()
    print("init-worker failed to sleep!")
  end

  ngx.log(ngx.ERR, "one")
  local ok, err = pcall(ngx.sleep,1)
  ngx.log(ngx.ERR, "two", ok, err)
  ngx.sleep(1)
  ngx.log(ngx.ERR, "three")
end

I get this:
2021/07/05 12:57:42 [notice] 87500#0: [lua] resty-test.lua:11: init failed to sleep!
2021/07/05 12:57:42 [notice] 87500#0: using the "kqueue" event method
2021/07/05 12:57:42 [warn] 87500#0: 1024 worker_connections exceed open file resource limit: 256
2021/07/05 12:57:42 [notice] 87500#0: openresty/<IP_ADDRESS>
2021/07/05 12:57:42 [notice] 87500#0: OS: Darwin 20.4.0
2021/07/05 12:57:42 [notice] 87500#0: hw.ncpu: 12
2021/07/05 12:57:42 [notice] 87500#0: net.inet.tcp.sendspace: 131072
2021/07/05 12:57:42 [notice] 87500#0: kern.ipc.somaxconn: 128
2021/07/05 12:57:42 [notice] 87500#0: getrlimit(RLIMIT_NOFILE): 256:9223372036854775807
2021/07/05 12:57:42 [notice] 87501#0: start worker processes
2021/07/05 12:57:42 [notice] 87501#0: start worker process 87502
2021/07/05 12:57:42 [notice] 87502#0: *1 [lua] resty-test.lua:20: init-worker failed to sleep!, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:23: one, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: *1 [lua] resty-test.lua:25: twofalseAPI disabled in the context of init_worker_by_lua*, context: init_worker_by_lua*
2021/07/05 12:57:42 [error] 87502#0: init_worker_by_lua error: ./resty-test.lua:26: API disabled in the context of init_worker_by_lua*
stack traceback:
	[C]: in function 'sleep'
	./resty-test.lua:26: in function <./resty-test.lua:15>
	init_worker_by_lua:2: in main chunk

So I see everything I expect to see, only the last log line isn't logged because the non-protected sleep just before breaks the code flow.

Did you tested with Kong? This error appears to be due to the use of native ngx.sleep instead of alternative_sleep, if you use Kong, this should be alternative_sleep. See https://github.com/Kong/kong/blob/master/kong/globalpatches.lua#L42-L61

No tested plain openresty. I get the idea now. That is a bug with Kong then.

You can repeat it using the following code test.
local expire = now() + _wait_max
  for idx = 1, count do
    local data = cache_data[idx]
    local err = cache_err[idx]

    data, err = nil, nil  -- the temporary test

    while not data do
      if err then
        log(ERR, "worker-events: error fetching event data: ", err)
        break
      else
        -- just nil, so must wait for data to appear
        if now() >= expire then
          break
        end
        -- wait and retry
        -- if the `sleep` function is unavailable in the current openresty
        -- 'context' (eg. 'init_worker'), then the pcall fails. We're not
        -- checking the result, but will effectively be doing a busy-wait
        -- by looping until it hits the time-out, or the data is retrieved
        _busy_polling = true  -- need to flag because `sleep` will yield control
                              -- and another coroutine might re-enter
        if not pcall(sleep, _wait_interval) then
          -- `pcall` failed, so this is a non-yieldable phase to prevent a deadlock we
          -- must force update the cached time reported by `ngx.now`.
          ngx.update_time()
        end
        _busy_polling = nil
        data, err = get_event_data(_last_event - count + idx)

       data, err =nil, nil --  the temporary test
      end
    end

@jinhailang can you still reproduce after applying this:
at the top replace:
local sleep = ngx.sleep

with
local sleep = function(t)
  ngx.sleep(t)
  ngx.update_time()
end

Good idea

@jinhailang did you test?, did that solve the issue with Kong?

@jinhailang did you test?, did that solve the issue with Kong?

I tested it,ngx.update_time() can change the time, and fix the problem.

great thanks!


fix: Set IdleTimeout for http.Server

Description
What this PR does / why we need it:
Setup IdleTimeout for ratify http server
Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes https://github.com/deislabs/ratify/issues/1416
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Yes. I tested this change on an EKS cluster with OPA gatekeeper. Without an idle timeout, gatekeeper audit controller keeps creating new connections till it reaches the OS limit and then any new connection fails. After the change, gatekeeper audit controller still creates new requests, but the connections are short-lived and I no longer encounter the error.
Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

[x] Manually verified number of connections to ratify using ss command on gatekeeper-audit container using ephemeral debug container

Checklist:

[ ] Does the affected code have corresponding tests?
[ ] Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
[ ] Does this introduce breaking changes that would require an announcement or bumping the major version?
[ ] Do all new files have appropriate license header?

Post Merge Requirements

[ ] MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

@mannbiher thanks for investigating and fixing it! @susanshi @akashsinghal do you think it would be beneficial to configure it to handle varying levels of load in the future?

@mannbiher thanks for investigating and fixing it! @susanshi @akashsinghal do you think it would be beneficial to configure it to handle varying levels of load in the future?

Potentially. I think the idle timeout is one factor among others for higher request counts. We'd only be able to determine after another round of perf testing and that too perf testing that can run for longer period of time. My opinion would be to proceed forward with a set default right now like PR is doing and we can revisit if this becomes a concern in the future.


Bump Dependencies #8804474403

Bumps dependencies for product-is. Link : https://github.com/wso2/product-is/actions/runs/8804474403

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[youtube_player_iframe] hide interface

Whenever you click the screen for a long time, the same interface as the picture appears. Is there a way not to show this interface? [youtube_player_flutter] I don't see the interface, so I think there's a way to hide it, but I can't find it

Facing the same issue with no solution.

You can turn off desktop mode to avoid the pop up.

I think the desktop mode is false (disabled) by default and I also set it explicitly. But when clicked on the player, it shows the popup.

This is the workaround I found for youtube_player_iframe.
Inside YoutubePlayerControllerProvider.body > LayoutBuilder.builder (check example code) I placed this code:
if (!kIsWeb) {
  _controller.invokeJavascript('''
    console.log('Youtube page: disable menu');
    var iframe = document.querySelector('iframe').contentWindow.document;
    iframe.querySelector('div video').addEventListener('contextmenu', e => e.preventDefault());
    iframe.querySelector('body').insertAdjacentHTML('afterbegin', '<style>.ytp-contextmenu{visibility:hidden;}</style>');
  ''');
}

Unfortunately due the CORS limitations (can't change content of iframes) it doesn't work in web browsers.

Unfortunately, the above javascript works only on Android devices, not iOS.
Does anyone have an alternative solution?


Enforce usage of Greaseweazle firmware v0.24 after adapting CMD_READ_…

…FLUX and CMD_WRITE_FLUX

Sorry about the delay --- been working on other things...
Thanks very much for this! However I would rather like to keep compatibility with v23 as well as v24, as given that I'm a third-party client I don't really want to demand that people upgrade their Greaseweazle firmware. Any chance of selecting the appropriate API based on the reported version? It should be a simple matter of stashing the version number in a global variable in a member variable and then using one or the other based on the value.

Thanks. Yeah, I will look into a flexible approach to allow v0.22+. May take some time!

I've actually overhauled all the USB stuff (and, sadly, had to take out the GreaseWeazle autodetection because I can't make it work on Windows), and merged your code along with it. Thanks very much!


Services Service - Allow Service Release without technicalUserProfile

Summary
The current implementation the service release process (dataspace services) at least 1 technical user role is expected to be selected to connect the service to central applications.
Till now a technical user was mandatorily needed - BUT changed recently and we need to allow with 24.05. that an user can release an dataspace service even without technical user connection.
To allow the same on backend side, following endpoints need to get adjusted:

PUT /api/services/ServiceReleaseProcess/{serviceId}/submit

Update the endpoint business logic - in the current implementation the technicalUserProfile is set as MUST to allow the user to run PUT /api/services/ServiceReleaseProcess/{serviceId}/submit for a specific service.
This logic needs to get removed. It is allowed to submit an service request without technicalUserProfile configured
Note the services service is getting updated as well - described in ticket: https://github.com/eclipse-tractusx/portal-backend/issues/497
Linked FE ticket: https://github.com/eclipse-tractusx/portal-frontend/issues/501

@jjeroch Technical user profile check is not present for services release process. Repository changes is done with #497 ticket PR #566 which is common for both apps and services.

@jjeroch #566 PR is merged. As this ticket is also linked with #497 ticket implementation changing in review state


ignoreMissingPatterns not working

Which version and edition of Flyway are you using?
10.11.0
Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)
Maven plugin
in Flyway 8 we were relying on the setting (in Maven)
<ignoreMissingMigrations>true</ignoreMissingMigrations>
According to the release notes here https://rd.gt/416ObMi the new alternative for this option is now "ignoreMigrationPatterns", which due to the documentation here https://documentation.red-gate.com/fd/ignore-migration-patterns-224919720.html is used as follows:
<ignoreMigrationPatterns><ignoreMigrationPattern>*:missing</ignoreMigrationPattern></ignoreMigrationPatterns>
The problem ist that this option does not seem to apply at all. Even worse, when I replace the pattern like
<ignoreMigrationPatterns><ignoreMigrationPattern>versioned:missing</ignoreMigrationPattern></ignoreMigrationPatterns>
I get the error message

Flyway Redgate Edition Required: ignoreMigrationPattern with type 'versioned' is not supported by OSS Edition

which seems to suggest that this option is no longer part of the community edition. But that contradicts the documentation linked above.
Could somebody please clarify the situation?

Sorry, it was actually not MISSING, but FUTURE migrations that caused the problem. I let myself be mislead by the error message

Detected applied migration not resolved locally: [...]
If you removed this migration intentionally, run repair to mark the migration as deleted.

Thank you for your issue. I'm not sure why the documentation is out of sync. However, from the latest release 10.12.0, these features have been made available at the open source edition.
https://documentation.red-gate.com/flyway/release-notes-and-older-versions/release-notes-for-flyway-engine

Just to follow up — does this ticket still need attention?


Update virtual-machines-mariadb-cluster.md

Article still relevant. Updating ms.date.

Hi @sabbour, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
We will now validate the agreement and then real humans will evaluate your PR.
TTYL, AZPRBOT;


Please consider current Moq privacy issues and replace with something else

Article: https://learn.microsoft.com/en-us/ef/ef6/fundamentals/testing/mocking
Privacy issues with SponsorLink, starting from version 4.20:
https://github.com/moq/moq/issues/1372
Video "Remove Moq From Your .NET Projects Right NOW!" with more details:
https://www.youtube.com/watch?v=A06nNjBKV7I
I won't expect Microsoft will continue to advice using of Moq considering this info.

This issue has been closed because EF6 is no longer being actively developed. We are instead focusing on stability of the codebase, which means we will only make changes to address security issues. See the repo README for more information.


New icon: No-IP

(Same PR message as #547)

No-IP icon merged.
I removed T-Mobile icon, because it's not part of this PR. Recover it from here, if you want to make next PR: https://github.com/antun3s/aegis-icons/commit/5039c9cc6ffb6db689f84f0267961f35b63cb754.


refactor(text-value-as-string): no longer account for richText gui attribute (DEV-1471)

resolves DEV-1471

I have a hard time giving reasonable feedback on this. But from what I understand, it should be good.
I know there is more enough to refactor in the App. But imho the whole "text-value-as-string/html/xml" logic is far from ideal because it makes the app depend on what exactly the API returns, rather than trying to capture the actual logic of it. If you ever want to tackle that, I may have some ideas.

yes, I also don't like how it's implemented but that's a task for future me 🙃


Clay statues

Statues made of clay that can be found in in excavation sites or hidden rooms (#35)

Possible inspiration: Statues from Terraria


Standard does not warn of adding properties to non-extensible objects.

What version of standard?
12.0.1
What operating system, Node.js, and npm version?
OS: Ubuntu 18.04
Node.js: v8.12.0
npm: 6.4.1
Example code
'use strict'

var obj = {}
Object.preventExtensions(obj)

obj.test = 'this throws a TypeError'

What did you expect to happen?
Standardjs to warn me of the TypeError that running this code throws.
What actually happened?
Standardjs gives no warning and the code throws a TypeError.

Is there an ESLint rule that would warn against that?

I'm not sure if a new one would need to be written or if one already exists.

After discussion in the ESLint gitter, it seems unlikely that such a rule will be added, so closing this issue.


Compile for android description is broken

executing "git clone git://github.com/kivy/kivy-remote-shell" as stated leads to:
The unauthenticated git protocol on port 9418 is no longer supported.
it should be:  git clone https://github.com/kivy/kivy-remote-shell
Besides the fact that the precompiled link is broken as well, this makes it hard to get started

Also, buildozer android_new debug deploy run logcat fails now, as android_new has been renamed to android and androidrenamed toandroid_old`.
So, this should be corrected, and the link to the precompiled binary updated, please.


Relogin without login

Is there a possibility that i do not have to log in after every hour? At this time the refresh token is 1 hour valid and then i have to relogin.
I would like to have the same user experience i have myself when i use for example the Azure Cloud. I can revisit the Azure Portal after several days and i am still logged in and can manage my things without relogin.
Is there a setting i have not seen at this time?

Hi,
I'm also facing this issue. What can we do to avoid the user having to login every hour?

As mentioned in issue #61 this library uses the preferred implicit flow. Which is according to Azure AD documentation the preferred way for SPA's.

@rohitnarula7176 @jmprieur It would be nice if one of you could answer the question. What do we have to do, that the user do not have to log in on every visit of the page? Is this not supported?

@VSDekar , I went into a confused path as well. In short, from what I understand, TokenCache in MSAL is the replacement/abstraction for refresh_token. You need to save TokenCache for later access. (acquireTokenSilent will require an IAccount as parameter which you would only get from a saved TokenCache)
Provided a bit more of my understanding here: https://stackoverflow.com/a/58482305/1578962

@AAATechGuy : I don't think that there is a refresh token in MSAL.js (which uses the implicit flow, which is a bit different from the other OAuth 2.0 flows)
What you saw applies to the other MSAL libraries (MSAL.NET, MSAL for iOS and Mac OS, MSAL Android, Python, Java), but I don't think it applies to .JS
cc: @negoe to confirm.

Yes @jmprieur, refresh token does not apply for implicit flow grant which is the current implementation for msal js as that was the recommended flow for SPAs when we started. We are extending the library to support auth-code flow in browsers as it is the new recommendation for SPAs in the upcoming months.


Transparency not overrides by alpha value.

In case of filled 32bit depth buffer as 0xFFFFFF00, transparency is 0x00 and color is 0xFFFFFF (white).
If FLFTRender set to color any, and transparency set to 0xFF, FLFTRender not draws correct font rendering on the buffer.

Strange alpha calculation.

Solved, not a bug.


Limbo npm package?

Shall we do a npm package, similar to<EMAIL_ADDRESS>

We have a limbo-wasm package now.


Auto Start: "need to restart terminal" warning is too easy to miss

After turning on "Auto Attach",  it never works for me until I notice the tiny warning icon on the right side of the terminal.

IMO this is not a good flow. I'm still tricked by this every other day...
If users explicitly set an Auto Attach option, they expect that the feature works after executing the action. They do not want to hunt down a warning and then have another action...
I suggest to ask users immediately from the Quickpick for restart confirmation.
In addition I do not understand why the terminal needs to be restarted?
Setting an env var in the terminal should be possible without restarting, correct?

This is what we ended up landing on after a bit of back and forth around the UX in the syncs and tests https://github.com/microsoft/vscode/issues/96300.
Initially the warning was going to be much more in your face but that was rejected as only sometimes is this something that we would want to interrupt the user on. There's a request to restart terminals automatically to help with this if they have not been interacted with https://github.com/microsoft/vscode/issues/100193. I'm open to a different flow but going for something more obvious will also tick users off.

In addition I do not understand why the terminal needs to be restarted?
Setting an env var in the terminal should be possible without restarting, correct?

I considered adding another item in the hover's actions to send commands to update the environment but decided against it to simplify the already complex hover and I don't think I got good feedback in the UX sync. It's also difficult to get this right reliably (differing shells, shell levels, apps running in the shell taking the input), the only reliable way is to restart which is typically an inexpensive action to take.

This is what we ended up landing on after a bit of back and forth around the UX in the syncs and tests https://github.com/microsoft/vscode/issues/96300.
Initially the warning was going to be much more in your face but that was rejected as only sometimes is this something that we would want to interrupt the user on. There's a request to restart terminals automatically to help with this if they have not been interacted with https://github.com/microsoft/vscode/issues/100193. I'm open to a different flow but going for something more obvious will also tick users off.

In addition I do not understand why the terminal needs to be restarted?
Setting an env var in the terminal should be possible without restarting, correct?

I considered adding another item in the hover's actions to send commands to update the environment but decided against it to simplify the already complex hover and I don't think I got good feedback in the UX sync. It's also difficult to get this right reliably (differing shells, shell levels, apps running in the shell taking the input), the only reliable way is to restart which is typically an inexpensive action to take.

The existing warning icon might be OK for cases where users don't want to be interrupted (but I still wonder what those case would be..)
But turning on "Auto Attach" is definitely not such a case!
If users explicitly enable Auto Attach, they expect that the feature works after executing the action.
Showing a modal dialog and asking them whether to restart the terminal is the only way to make this flow.

The existing warning icon might be OK for cases where users don't want to be interrupted (but I still wonder what those case would be..)
But turning on "Auto Attach" is definitely not such a case!
If users explicitly enable Auto Attach, they expect that the feature works after executing the action.
Showing a modal dialog and asking them whether to restart the terminal is the only way to make this flow.

This command?

You could have that command also show a notification saying that terminals need to be restarted if any terminals exist? workbench.action.terminal.relaunch is also available to relaunch the active terminal if needed, I could also create a relaunchAll command if desired.
This API is not just for debugging and we don't want it to be interrupting users in the general case.

This command?

You could have that command also show a notification saying that terminals need to be restarted if any terminals exist? workbench.action.terminal.relaunch is also available to relaunch the active terminal if needed, I could also create a relaunchAll command if desired.
This API is not just for debugging and we don't want it to be interrupting users in the general case.

Yes, this issue is only about the "Auto Attach" use case, not about setting env vars in the general case.
@connor4312 I like Daniel's suggestion to show a notification (or alert) saying that terminals need to be restarted.

Yes, this issue is only about the "Auto Attach" use case, not about setting env vars in the general case.
@connor4312 I like Daniel's suggestion to show a notification (or alert) saying that terminals need to be restarted.

I think a relaunchAll action would be useful here for a notification button

@connor4312 with "notification button" you mean an action in the alert or notification?

Yea

@connor4312 I'm not sure relaunch all action is needed anymore since I pushed the auto restarting yesterday (https://github.com/microsoft/vscode/issues/100193)?

cool! One thing I ran into (adding an option to restart a single terminal) is that I don't know whether the terminal is about to restart on its own, so could show an extraneous notification.
I wonder if the auto restart you pushed solves the case in general... the ⚠ is still easy to miss, but now it will always be caused by a user interaction and not just by opening a workspace.

@connor4312 don't you just need to wait a little bit before showing a notification?

That works. I have it about ready to go but currently this triggers a bad bug https://github.com/microsoft/vscode/issues/115927

@connor4312 we're aiming to fix that for Feb, it's a lot standing issue https://github.com/microsoft/node-pty/pull/415


fatfs/drivers/fatfs_sd.c cannot be compiled with GCC 7.3.1

Using gcc version 7.3.1 20180622 (release) [ARM/embedded-7-branch revision 261907] (GNU Tools for Arm Embedded Processors 7-2018-q3-update)
I'm getting the error
`fatfs/drivers/fatfs_sd.c:158:12: error: conflicting types for 'select'
/usr/arm-none-eabi/include/sys/select.h:62:5: note: previous declaration of 'select' was here
int select __P ((int __n, fd_set *__readfds, fd_set *__writefds,`
Shouldn't the use of system function names be considered as bad practice and exterminated?

@alpo yes, could be. Could you do please a pull request with valid change?


How to password protect the root directory only?

Hey maple3142, it's a great script. Can you please tell me if it's possible to password-protect the root directory only. I want to keep the root directory password protected not other ones. Is it possible to keep child directories unlocked and only the root directory protected?

Alright, I've found the solution on another issue. Sorry for opening a new issue.


discussion: global precommit hook for eslint via husky

I'm curious if we can add a precommit hook for yarn run lint in lodestar via this package husky.  Is this something that we would want to have implemented in the repo?  Or do people think this is annoying?
Here's an example of the usage via husky here (which hooks in directly to the package.json scripts section):
https://medium.com/netscape/git-hooks-with-husky-8b98f2556363

Closing in favor of individual dev git hooks


A few questions

Hello, thank you for your work!
I have a few questions:

Can create_traj and other codes that interact with the environment use player instead of agent? The agent is a robot, and the viewing angle is relatively low.
"Legent launch --scene 1" cannot successfully render the environment.
Could you open source the create_traj of the grab task?
Could the grab action specify the object ID? That is, use the object ID as an input parameter of grab, and the same applies to the put down action.
Unable to use the top-down view.

Hello, sorry for the late reply. We are currently working on these issues, and they will all be resolved by October.

You can initialize a view similar to the player's by using env = Environment(env_path="auto", use_animation=False, camera_resolution_width=896, camera_resolution_height=448, camera_field_of_view=60). This will set the vertical field of view to 60 degrees, with an image aspect ratio close to 2:1. Additionally, we are developing a module for customizing the agent, including the 3D model and camera position of the agent.

We did not include this scene in the latest build. We are organizing more prefab scenes in a better format, which will be released together with the scene information.

Yes, we will open source the data generation codes for all task templates, including the diversified data generation codes, in October.

This functionality already exists in the environment but hasn't been provided as an API yet. We will include it in future updates.

The top-down view has been replaced with a free view and scene editing tool currently under development, which will also include the top-down view functionality. The free view is very useful and we will update the documentation on how to use this free view.

Thank you very much for developing this awesome platform. I am interested in using a realistic environment. Do you have an idea when the legent launch --scene 1 feature will be available?

Now the command legent launch --scene 1 is available. Before using it, update the client with legent download or legent download --thu.

Thank you very much, will give it a try!


SeqXMLFile error

Hi i work on matlab 2021b, MRILab-1.3 and I got SeqXMLFile error

I don't know why this error is happening.  Can you give me some advice?

You need to choose and load a sequence first to run the simulation.


HIVE-24788: Backport HIVE-23338 to branch-3.1

Trying to backport 31935896a78f95ae0792ae7f29960d1b604fbe9d for HIVE-24788

Tests seem to pass, removing WIP-label

This is my first time contributing to hive. I was surprised that another CI run appeared 24h later. It fails, but says
There are 0 new tests failing, 26 existing failing and 53 skipped.

That sounds like the failures are pre-existing? Anything I need to do here?

Hello @jcamachor @pvargacl, I'm unfamiliar with the process here, would you be able to have a look at this PR (or tell me what's missing), please? 🙃

@h-vetinari  The precommit tests are unfortunately flaky on the branch 3.1, so do not expect a green run there.
@klcopp  can you take a look

All 26 test failures were seen previously on http://ci.hive.apache.org/job/hive-precommit/job/PR-1952/1/#showFailuresLink, so don't worry about those failures. +1 and I can commit in 24h.


TLC modal: link to support doc needs updating

Current behavior
The tip length calibration modal contains a link to a support doc that is no longer working:
https://support.opentrons.com/en/articles/4523313
Expected behavior
We should update this link to the new URL for the support doc:
https://support.opentrons.com/s/article/Recalibrating-tip-length-before-running-a-protocol
Acceptance criteria

[ ] The updated link on the TLC modal loads correctly in the browser

Realized we probably shouldn't be hunting for these manually. Here's a list of all the old-format URLs left in edge:
grep -rn "support.opentrons.com/en" app/src
app/src/pages/Robots/RobotSettings/CalibrationCard.tsx:52:  'https://support.opentrons.com/en/articles/3499692-how-calibration-works-on-the-ot-2'
app/src/pages/Devices/DevicesLanding/NewRobotSetupHelp.tsx:20:  'https://support.opentrons.com/en/collections/1559720-ot-2-get-started'
app/src/organisms/ProtocolSetup/RunSetupCard/ModuleSetup/MultipleModulesModal.tsx:31:  'https://support.opentrons.com/en/articles/5167312-using-modules-of-the-same-type-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/ModuleSetup/__tests__/MultipleModuleModal.test.tsx:43:      'https://support.opentrons.com/en/articles/5167312-using-modules-of-the-same-type-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/LabwareSetup/LabwareOffsetModal.tsx:29:  'https://support.opentrons.com/en/articles/3499692-how-positional-calibration-works-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/LabwareSetup/LabwareOffsetModal.tsx:31:  'http://support.opentrons.com/en/articles/5742955-how-labware-offsets-work-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/LabwareSetup/__tests__/LabwareOffsetModal.test.tsx:60:      'https://support.opentrons.com/en/articles/3499692-how-positional-calibration-works-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/LabwareSetup/__tests__/LabwareOffsetModal.test.tsx:70:      'http://support.opentrons.com/en/articles/5742955-how-labware-offsets-work-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/RobotCalibration/PipetteCalibration.tsx:32:  'https://support.opentrons.com/en/articles/3450143-gen2-pipette-compatibility'
app/src/organisms/ProtocolSetup/RunSetupCard/RobotCalibration/__tests__/DeckCalibrationModal.test.tsx:57:      'https://support.opentrons.com/en/articles/3499692-how-positional-calibration-works-on-the-ot-2'
app/src/organisms/ProtocolSetup/RunSetupCard/RobotCalibration/DeckCalibrationModal.tsx:30:  'https://support.opentrons.com/en/articles/3499692-how-positional-calibration-works-on-the-ot-2'
app/src/organisms/CalibrateTipLength/ConfirmRecalibrationModal.tsx:33:const CALIBRATION_URL = 'https://support.opentrons.com/en/articles/4523313'
app/src/organisms/ProtocolUpload/hooks/__tests__/RerunningProtocolModal.test.tsx:44:      'http://support.opentrons.com/en/articles/5742955-how-labware-offsets-work-on-the-ot-2'
app/src/organisms/ProtocolUpload/RerunningProtocolModal.tsx:34:  'http://support.opentrons.com/en/articles/5742955-how-labware-offsets-work-on-the-ot-2'
app/src/organisms/CalibrationPanels/NeedHelpLink.tsx:6:  'https://support.opentrons.com/en/collections/2426956-ot-2-calibration'
app/src/organisms/AppSettings/__tests__/ConnectRobotSlideout.test.tsx:98:      'https://support.opentrons.com/en/articles/2934336-manually-adding-a-robot-s-ip-address'
app/src/organisms/AppSettings/PreviousVersionModal.tsx:17:  'https://support.opentrons.com/en/articles/2393514-uninstall-the-opentrons-app'
app/src/organisms/AppSettings/GeneralSettings.tsx:45:  'https://support.opentrons.com/en/articles/1795303-get-started-update-your-ot-2#:~:text=It%E2%80%99s%20important%20to%20understand,that%20runs%20your%20protocols).'
app/src/organisms/AppSettings/ConnectRobotSlideout.tsx:28:  'https://support.opentrons.com/en/articles/2934336-manually-adding-a-robot-s-ip-address'
app/src/organisms/Devices/DevicesEmptyState.tsx:28:  'https://support.opentrons.com/en/collections/1559720-ot-2-get-started'
app/src/organisms/Devices/DevicesEmptyState.tsx:30:  'https://support.opentrons.com/en/articles/2687601-troubleshooting-connection-problems'
app/src/organisms/Devices/RobotSettings/RobotSettingsNetworking.tsx:37:  'https://support.opentrons.com/en/articles/2687586-get-started-connect-to-your-ot-2-over-usb'
app/src/organisms/Devices/RobotSettings/__tests__/RobotSettingsNetworking.test.tsx:195:      'https://support.opentrons.com/en/articles/2687586-get-started-connect-to-your-ot-2-over-usb'
app/src/organisms/Devices/ConnectionTroubleshootingModal.tsx:20:  'https://support.opentrons.com/en/articles/2687601-troubleshooting-connection-problems'
app/src/organisms/Devices/ProtocolRun/SetupPipetteCalibrationItem.tsx:36:  'https://support.opentrons.com/en/articles/3450143-gen2-pipette-compatibility'

They appear to all go to a login screen.

@aedocw @mikecardarelli Do we need to update all of the links in that comment?

Updating this ticket to cover all the broken links in OT App


CDC -> ELSST integration

CDC SO, ELSST SO and CESSDA discussed the posibilities of creating integrations between the services. From the CDC end, there is a desire to be able to access information on ELSST keywords, and utilize the ELSST UI to discover other keywords, which in turn can be searched for in CDC.
In order to create this possibility there needs to be a path from the CDC to ELSST which first need to consider the following:

Needs to be a string search (since identifiers are not present in metadata from SPs)
Needs to be able to handle multilingual keywords

The string search can be solved by using the ELSST API to find the correct term in ELSST. Information on language of the keyword clicked can be extracted from metadata at CDC end. ELSST URLs have "clang=language code" at the end, which enables the user to get the same language as the one they clicked in CDC.
Secondly, not all keywords used by SPs are from ELSST. The best experience for the user is if this functionality is only present for valid ELSST keywords. To do this, the CDC needs to check if the keywords present in a metadata record are valid ELSST keywords. Matthew mentioned that a lookup can be done for all the keywords present on a study landing page. There might be some performance concerns here with the requests required to do this lookup, so this should be considered in the solution. A mentioned option is to use caching.
Thirdly, the path from CDC to ELSST, should not replace the existing functionality of filtering directly in the CDC. I.e. clicking a keyword on a landing page and getting a new result page with all studies containing that keyword. So there needs to be a good solution in the UI for how to handle this (also needs to be technically possible)
Suggestion 1: add 2 links under the keyword which is displayed at the landing page. For keywords only present in CDC, only the CDC link is shown:

Suggestion 2: the box with the links is shown after the user clicks the keyword, adds a 2nd step to the process but it will not be that invasive on the UI. Will require a reshuffling of the list after the click. If the keyword clicked does not exist in ELSST, the CDC executes a filter search on the keyword:

Suggestion 3: the box with the links is shown as a popup window after the user clicks the keyword avoiding the reshuffling of the list. If the keyword clicked does not exist in ELSST, the CDC executes a filter search on the keyword:

Step 1 is to create a mechanism for checking if a given keyword exists in ELSST

Step 1 has been implemented, but without caching at the moment.
A demo UI has been implemented at https://github.com/cessda/cessda.cdc.searchkit/tree/feature/617-cdc-elsst-integration which changes any detected ELSST terms from internal CDC links to ELSST links.

Caching has been implemented server-side

Not sure if it is possible to have the metadata language carry over from CDC to ELSST, meaning a user clicking a keyword in a specific language in CDC should be taken to that specific language in ELSST. Currently the user is always brought to the english term, independant of metadata language in CDC. See examples
Example study 1 Clicking the term Nuorisorikollisuus leads to Juvenile Deliquency in ELSST, where it instead should be Nuorisorikollisuus  in ELSST. With the "?clang=fi" in the URL
Example study 2 Clicking the term Gesundheit leads to Health in ELSST, where it instead should be Gesundheit. With the "?clang=de" in the URL
Example study 3 Clicking the term ΣΤΑΣΕΙΣ (ΑΝΤΙΛΗΨΕΙΣ) leads to Attitudes in ELSST, where it instead should be ΣΤΑΣΕΙΣ (ΑΝΤΙΛΗΨΕΙΣ). With the "?clang=el" in the URL

It is possible by adding clang parameter like you mentioned but the problem is that it can't simply be added to the uri given by ELSST's API. Using the first example, it is searched with https://thesauri.cessda.eu/rest/v1/elsst-4/lookup?label=NUORISORIKOLLISUUS&lang=fi which results in:
{
  "@context": {
    "skos": "http://www.w3.org/2004/02/skos/core#",
    "uri": "@id",
    "type": "@type",
    "onki": "http://schema.onki.fi/onki#",
    "results": {
      "@id": "onki:results"
    },
    "prefLabel": "skos:prefLabel",
    "altLabel": "skos:altLabel",
    "hiddenLabel": "skos:hiddenLabel",
    "@language": "fi"
  },
  "result": [
    {
      "uri": "https://elsst.cessda.eu/id/4/b5f727f0-2a03-4790-a79f-ec0b442b9752",
      "type": ["skos:Concept"],
      "localname": "b5f727f0-2a03-4790-a79f-ec0b442b9752",
      "prefLabel": "NUORISORIKOLLISUUS",
      "lang": "fi",
      "vocab": "elsst-4"
    }
  ]
}

The uri from result here is https://elsst.cessda.eu/id/4/b5f727f0-2a03-4790-a79f-ec0b442b9752 and adding ?clang=fi to it doesn't work (404 not found). The link without clang parameter gets redirected to https://thesauri.cessda.eu/elsst-4/en/page/b5f727f0-2a03-4790-a79f-ec0b442b9752 when opened but the full redirect chain when checked with curl is:
$ curl -v -L https://elsst.cessda.eu/id/4/b5f727f0-2a03-4790-a79f-ec0b442b9752 2>&1 | grep -i "^< location:"
< location: https://thesauri.cessda.eu/entity?vocab=elsst-4&uri=https://elsst.cessda.eu/id/4/b5f727f0-2a03-4790-a79f-ec0b442b9752
< location: https://thesauri.cessda.eu/elsst-4/page/b5f727f0-2a03-4790-a79f-ec0b442b9752
< location: https://thesauri.cessda.eu/elsst-4/en/page/b5f727f0-2a03-4790-a79f-ec0b442b9752

Parameter clang works when added to https://thesauri.cessda.eu/elsst-4/page/b5f727f0-2a03-4790-a79f-ec0b442b9752?clang=fi (still redirects to the next location) or https://thesauri.cessda.eu/elsst-4/en/page/b5f727f0-2a03-4790-a79f-ec0b442b9752?clang=fi (the final location).
Easiest solution would be to have https://thesauri.cessda.eu/elsst-4/en/page/ as a variable in CDC and add localname value from JSON to it along with ?clang=fi so basically in code it would be something like:
`${ELSST_URL}/${results.result[0].localname}?clang=${lang}`

and end up as:
https://thesauri.cessda.eu/elsst-4/en/page/b5f727f0-2a03-4790-a79f-ec0b442b9752?clang=fi

Currently there is:
const SKOSMOS_URL = process.env.SEARCHKIT_SKOSMOS_URL || "https://thesauri.cessda.eu";
const ELSST_VOCABULARY = process.env.SEARCHKIT_ELSST_VOCABULARY || "elsst-4";

so basically ELSST_URL could be:
const ELSST_URL = process.env.SEARCHKIT_ELSST_URL || `${SKOSMOS_URL}/${ELSST_VOCABULARY}/en/page`;

I don't really know much about ELSST so I don't know if there is some better solution to this. I can see configuring ELSST_URL like this causing issues if the uri returned by API gets redirected to even slightly different address in the future since it wouldn't be used here anymore so it would require updating the value in ELSST_URL to reflect the change.

Unfortunatly Skosmos doesn't support specifying a language when using the /entity API. It's unfortunate because using redirects like this is the officially prescribed way to do this, see https://github.com/NatLibFi/Skosmos/wiki/ServingLinkedData.
We could specify a known prefix (ideally set with an environment variable) that is trimmed from the start of the URIs so we are left with only the UUIDs, and then use that to construct the final URLs directly

Thank you for the detailed clarification Markus, we understand now why it does not work. As for both of your proposals, I think you are best left to chose this. But it seems like a good idea to avoid anything that might be vulnerable to a change down the line, but seeing as both these services are CESSDA that perhaps can easily avoided.

It's an ELSST issue that I'm currently investigating

See https://github.com/cessda/cessda.elsst.skosmos/issues/56


Error handling #262

Description

Add global error handler for the get requests
add global retry handler
remove console.log for the react query  functions

Testing instructions
Add a set up instructions describing how the reviewer should test the code

[ ] Review code
[ ] Check Actions build
[ ] Review changes to test coverage
[ ] {more steps here}

Agile board tracking
closes #262

waiting on auth in the backend

Add error handling for the moveto and copyto

do the todos in the api functions

I copy loads of catalogue categories and then I was trying to delete them it done this when it triggered a refresh @joelvdavies


Adding code changes for Hot Shards RCA.

Description of changes: The RCA currrntly identifies a hot shard within an index based on following dimensions:
1. CPU Utilization
2. IO Total SysCallRate
3. IO Total Throughput
Tests: Unit Test, Dev stack Testing
Code coverage percentage for this patch:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Now that we have added the muting RCA PR, we should add the nodes of this for muting right ?


Sync non-spring code to spring boot 3 branch before release

Description
Sync non-spring code to spring boot 3 branch before release
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

/azp run java - spring - tests

close to use https://github.com/Azure/azure-sdk-for-java/pull/41011


Feature request: request history list

please add "the request history list" likes Postman. thx

Good point, @zjtv I'll look into it. what you think about having it like a tab in treeview? so user can switch between the tree view and list of recently called apis.

good idea

good idea


Missing 7.17.x versions

Is the Elastic team still supporting Elastic stack 7.17.x releases for these Helm charts? It looks like most of the charts were last updated about six months ago to use the 7.17.3 images by default whereas 7.17.6 is the latest patch release.
If there are no plans to update the 7.17 branch here, could that be pinned as a GitHub Issue?

Hey @brsolomon-deloitte,
Indeed, we didn't release versions after 7.17.3, and decided to not release new version of charts for every patch version.
Not sure if we'll release newer 7.17.x versions but you can use the 7.17.3 chart version with new stack images by setting imageTag value.

Helm best practice is to not override the image version in a helm chart (to ensure compatibility between the deployment and the container image).
Could elastic please reconsider their position on this?

Helm best practice is to not override the image version in a helm chart

Can you add more details and sources?
90% of the Helm charts do have a value to override the image tag. This is also part of the official Helm chart template when using the helm create command and has been mentioned as a best practice by Helm maintainers in the PR that added it: https://github.com/helm/helm/pull/7878#issuecomment-609628459
While we ourself recommend using the same version for the Helm chart and the Docker image in the doc, we don't release any more new charts version for all patch releases, so you can override the Docker image tag for a more recent patch version (for example 7.17.3 chart with 7.17.5 images).

Opened https://github.com/elastic/helm-charts/pull/1723 to make it clear in the doc.

Can you add more details and sources? 90% of the Helm charts do have a value to override the image tag.

Yes, but 90% of Helm charts also follow the best practices of:

Keeping the default image tag at an up-to-date (latest patch release) version
Properly bumping the chart version whenever the default image tag is bumped

Bitnami charts are one good example.


Added URL to calendar event.

I added an url to calendar events. The url can be serialized yet I did not yet get to add the parsing.

Looks good, thanks for the PR!


⚠ Jenkins Plugins Artifactory has degraded performance

In 8872471, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1368 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 64f6db0 after 28 minutes.


Gambit Crashes during SMS Broadcasts and Niche Import

Description

We've recently seen a spike in request timeouts, which most often happens after we send SMS broadcasts or when we import niche users.  The app crashes and the user receives the mdata error response message.

Heroku outlines potential causes of request timeouts here: https://devcenter.heroku.com/articles/request-timeout#timeout-behavior

Potential solutions include background jobs, which Heroku outlines here: https://devcenter.heroku.com/articles/background-jobs-queueing

Refs #704

 GAMBIT: Investigate solutions for timeout requests

 GAMBIT: Investigate solutions for timeout requests

Notes from the Gambit Request Timeout Meeting
AS: "Hunch is that the culprit = external calls to Phoenix / Northstar"
MH: "New Relic would show us how calls are affecting gambit performance"
When understanding the issue, biggest things to look for are:

What slows down NS, Phoenix and the app

Examples of users with calls taking >30 seconds:

95% of transaction cost is talking to contentful... if we're hitting stuff hard on contentful we should look into why a call to contentful would take that long.... AS: "can be two calls to contentful bc the keywords and campaigns live in separate documents... if KW comes in during incoming request then when we render the message we query contentful to see if it has a field set in campaign flow, if not then query again for the default campaign." MH: "possibly that these are one offs."

Post to the signups endpoint: MH "really long request to NS for user ID... signing them up for a campaign or pulling their info (i.e. phone number)... why is it taking over 30 seconds?

RP: "My theory is that the endpoint itself doesn't become unstable but rather the application. Would be good to know if we're stressing the system in a different way... e.g. If I generate as many requests as my computer can handle, but it wasn't because MoCo could send them to me but rather than my computer couldn't handle it.. so it's possible that we're stressing our only dyno."

Takeaways:

Slow requests to contentful (why would a request to your super fast CDN take so long?)
NS gets bogged down

Next Steps:

Set up autoscaling
Analyze timeouts in New Relix
If timeouts persist, fire up a third dyno

I think it's safe to rename this as "timeouts" instead of crashes. The app crashes seem to have been caused by #811.

While load testing Gambit thor, it became apparent to me that using an aggressive concurrency setting ended in a high error rate of requests made to Northstar thor.

Today's broadcast doesn't seem to support the theory that it's always an external request that causes this. We started receiving the Connection failed. Errno::ETIMEDOUT CC: 2. E: '' emails from mData failures. I checked Heroku logs to find plenty of H12's:
 at=error code=H12 desc="Request timeout" method=POST path="/v1/chatbot" host=ds-mdata-responder.herokuapp.com request_id=29da5ad4-257a-4ee7-81d8-2d099b6b81be fwd="<IP_ADDRESS>" dyno=web.2 connect=1ms service=30000ms status=503 bytes=0 protocol=https

In New Relic from what I can tell -- it seems to be a problem within Node today vs Phoenix timeouts:

I restarted all dynos, and the H12's stop -- but it seems that broadcast triggered them in the first place. Today's broadcast went out to 30k members.

From Heroku docs defining a H12:

An HTTP request took longer than 30 seconds to complete ... This 30-second limit is measured by the router, and includes all time spent in the dyno, including the kernel’s incoming connection queue and the app itself.

From https://devcenter.heroku.com/articles/request-timeout:

When a timeout is detected the router will return a customizable error page to the client and an H12 error is emitted to your application logs. While the router has returned a response to the client, your application will not know that the request it is processing has reached a time-out, and your application will continue to work on the request. To avoid this situation Heroku recommends setting a timeout within your application and keeping the value well under 30 seconds, such as 10 or 15 seconds. Unlike the routing timeout, these timers will begin when the request begins being processed by your application. You can read more about this below in Timeout behavior.

From https://devcenter.heroku.com/articles/request-timeout#debugging-request-timeouts, numerous database queries (which we do, by writing/readingsignup documents for each request) as well as numerous API queries (which we've already documented we do) could be a culprit.
Here's ideas for next steps:

Per https://github.com/DoSomething/gambit/issues/810#issuecomment-286783237, a next step for the immediate future might be to try a 3rd dyno.

Related: Experiment with the number of members we send a single broadcast to, to better determine what # of requests cause the H12's.

Set a timeout in Gambit to 30 seconds to anticipate (and prevent) H12 errors.

This brings to mind the idea returning a response of "200 OK" as soon as we receive a request as a quick way to avoid H12's immediately (https://github.com/DoSomething/gambit/issues/704#issuecomment-275727492). Not sure if this affects our New Relic metrics (but we can use the Gambit Local and Thor apps to see)

It might be time to start thinking about working on background tasks with Blink.

We could experiment with setting a timeout setting on our HTTP lib of say 20 seconds (to prevent any race condition between Timeout settings at Heroku and MoCo). We should probably pair that with a message to the user to try again later. There are a few things that are not so straight forward with this since external calls do have side effects (like posting a sign up) and we do not have a way to rollback in case a later step times out.

I created https://github.com/DoSomething/gambit/issues/776 from https://github.com/DoSomething/gambit/issues/810#issuecomment-286783237 but closed it as a "this doesn't happen a lot" but a look at https://secure.mcommons.com/mdatas/13720/mdata_queries (instead of just relying on the email alerts sent from Mobile Commons) shows this still happens a lot.
I'm thinking it might be our best bet for stopping these H12's any time soon (or trying out that 3rd dyno).
A problem still though is that we can't (or at least I'm not sure how) replicate a broadcast locally -- @rafa would it make sense to start a wiki page (or add to one of the existing Testing/Tests pages) to document how you've gone about load testing?

I'm not sure how the third Dyno would help. There is a big chance the problem gets worst by just sending more concurrent connections to Northstar/Phoenix-Ashes, thus increasing timeouts.
I'll definitely document load testing.

Tonight's broadcast to 30k members hasn't triggered any H12 errors (knocking on wood). We're at 13% memory usage and slowly increasing... when Gambit was timing out earlier today we were at 17% (and restarting dynos made H12's disappear -- see Heroku metrics below)

Per work done in #849, the chatbot endpoint will return a 504 if a request is taking too long (#704).
Per this afternoon, our production mData chatbot now posts incoming messages to Blink instead of Gambit. Blink posts the incoming mData request to Gambit and will retry it again if it receives a 504.


chore(changelog): put commit id on the same line as the message

Is that not what it does already?

What's the difference?

What's the difference?

GitHub releases use GFM for displaying release notes. GfM interprets the previous format as a hard break instead of a soft break.


Wrap exceptions thrown by beforeStart with MappableException

Problem:
We have a custom HibernateBundle that provides multi-tenancy and must support the case where tenant database instances are offline. We'd like to handle the errors caused by an offline tenant using ExceptionMapper, but that's impossible because exceptions thrown during the beforeStart phase are not wrapped in MappableException. (See #5501.)
This is similar to issue #949. Back in those good old times, it was impossible to create ExceptionMappers to handle persistence exceptions thrown during the afterEnd phase of the unit of work. The fix in PR #958 wraps the exception with Jersey's MappableExceptionbefore rethrowing.
This gist is a minimal test case.
Solution:

HibernateExceptions thrown by UnitOfWorkAspect.beforeStart are wrapped in MappableException. This is stricter than what is done during afterEnd, where any Exception is wrapped.
I'm unsure about the test: I had to config the H2 TCP server, so that I could turn it off and trigger an error during beforeStart. That might not be appropriate in the project's build infrastructure.
It might be sensible to wrap any exception thrown by the UnitOfWorkEventListener. I didn't do that though.

Result:
If an exception is thrown during the beforeStart stage, the user can create an ExceptionMapper to produce a web response in a way similar to what is already possible during the afterEnd stage.

Hi @andref. I've now checked, if we can simplify the test and stop using the server.
The following code closes the database's Session and produces a JdbcSQLNonTransientConnectionException too:
try (Session session = hibernate.getSessionFactory().openSession()) {
    session.doWork(connection -> connection.unwrap(JdbcConnection.class).getSession().close());
}

Maybe we can utilize this to avoid introducing the H2 server.

Hi @zUniQueX! Unfortunately, by the time we have a current Session from which to extract the JdbcConnection, the beforeStart phase has already run and the error must happen inside the UnitOfWorkAspect.beforeStart.
I'm fixing the other items in your review, thanks!

@andref To obtain the JdbcConnection, the beforeStart method doesn't have to be invoked. You can obtain the SessionFactory from your application to the executing test and instantiate the Session manually without the need to use the UnitOfWorkAspect. Or am I missing something here?

Or am I missing something here?

@zUniQueX Only because I didn't explain myself properly. 😅
The integration test case I wrote is verifying specifically whether an exception thrown during the beforeStart phase can be handled by an ExceptionMapper — that is the behavior this PR implements. As such, how to obtain a Session and close its connection is beside the point if the exception that would cause can't happen inside UnitOfWorkAspect.beforeStart.
I'm wondering that maybe setting up the DataSourceFactory with a bogus validation query and checkConnectonOnBorrow = true might do the trick.

@zUniQueX I replaced the integration test case with a simple unit test. The change I'm proposing is just too small.

@andref As we've now discussed how to write an integration test, I added a simple one :wink:

@andref As we've now discussed how to write an integration test, I added a simple one 😉

Oh... OH! Now I understand.


Update api.md

Fix reference to function name in mapDispatchToProps documentation.

Original:

If you don't want to subscribe to store updates, pass null or undefined in place of mapStateToProps.

My change:

If you don't want to subscribe to store updates, pass null or undefined in place of mapDispatchToProps.

Alternatively, it may have been intended to be expressed as follows:

If you don't want to subscribe to store updates, pass null or undefined as the second argument of connect, just after mapStateToProps, e.g., connect(mapStateToProps, null).

Unless I'm wrong, that's correct as-is. If you don't want to subscribe to state, you shouldn't map state.

The original statement is indeed true as-is. However, it kind of jarred me out of context, when reading about it within the documentation for mapDispatchToProps. It felt to me like it was referring to passing null or undefined in as the second argument being discussed, and mapStateToProps seemed like an IDE-autocomplete typo.
Simply close this if you disagree, and Ill try and correct perception on my end :).

To be honest, the current wording feels like the result of the mapState paragraph being copied and edited, rather than being written from scratch.  I think that sentence should just be removed.


[openrr-gui] Add velocity_sender

cc #159
How to run
urdf-viz ./openrr-planner/sample.urdf &

cargo run -p openrr-apps --bin openrr_apps_velocity_sender -- --config-path ./openrr-apps/config/sample_robot_client_config_for_urdf_viz.toml

https://user-images.githubusercontent.com/43724913/119468180-2b4f9a00-bd81-11eb-999e-c1874c431643.mov

I think it's better to show details default ON.

Ok. Changed to show details by default.


Some way to hide React 16 error boundary warnings 🙈

Do you want to request a feature or report a bug?
Somewhere in between?
What is the current behavior?
When I'm running tests on my library, there are some behaviors that I want to test do throw an error. These currently result in:
Consider adding an error boundary to your tree to customize error handling behavior.
You can learn more about error boundaries at https://fb.me/react-error-boundaries.

...being output to the console. This error is great in an application, but not so great for a library test.
What is the expected behavior?
It would be great if I could do something like:
expect(() => {
  TestUtils.renderIntoDocument(<DoSomething naughty/>)
})
.toThrow(/Bad developer!/)
.andCatch() // <---- prevents React 16 error boundary warning

Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
See also

https://github.com/facebook/react/issues/11083
https://github.com/facebook/react/issues/10474#issuecomment-334164834

cc @gaearon

LOL. @gaearon knows. 😄

To clarify I meant this should probably be filed in React repo 😛
I don't think we saw these warnings. But we probably mocked something in Jest at some point and that's why.

Sorry I my comment was confusing. I meant "seeing it in Jest" as a phrase, not "file ... in Jest" 😄

yep, closing here then.

I filed as https://github.com/facebook/react/issues/11098.


Added July 2016 CU hotfix (KB3163912) to Windows 10 x86 and x64 LTSB …

…default lab media registrations.

Thanks - LGTM!