Define the NetworkChannel interface

Lasse, have you made any progress in this regard? I am somewhat blocked on this because I want to implement start-tag distribution and fix akwardness of startup.
My current propsal is this:
struct NetworkChannel {
  size_t dest_channel_id; // So that we can "address" one of several NetworkChannel's at the other end.
  lf_ret_t (*open_connection(NetworkChannel *self);
  lf_ret_t (*is_connected(NetworkChannel *self);
  lf_ret_t (*send_blocking)(NetworkChannel *self, TaggedMessage *message);
  void *(register_receive_callback(...)
};

Key design decisions here that we can discuss are:

It does not make the difference between server/client. This might be a bad decision and we might reconsider and add a flag to the constructor.
It introduces the notion of a dest_channel_id. This is an address into the federate on the other side. It lets you say that this message is destined for a particular NetworkChannel at the other side. It is redundant with TCP/UDP since we can just use ports. We might argue that it is not a good idea to have this in the NetworkChannel interface and instead let the different networking stacks handle it.
Only send_blocking is blocking. I.e. open_connection does not guarantee that a connection is established. Only when is_connected returns LF_OK can we conclude that a connection is established. This has the advantage that a single-threaded runtime can "open" all his connections and then in the end poll all of them in a while loop until all are connected. Any suggestions for an alternative approach?

What about close_connection?
Is this something that just does not happen in Lingua Franca?

So your idea is to put the server-client information into the tcp_ip_channel concrete type? At least somehow the channel needs to know it it should call accept and bind or connect etc.

Yes. When constructing e.g. a TcpIpChannel with TcpIpChannel_ctor() you need to tell it if its the client or server, and the IP adddress + port number of the server. If other networking stacks also have this asymmetry, i.e. there is a server and a client in the communication, we can consider making this part of the  network abstraction yes.

As far as I can see for now I think this is indeed duplicated. At least for BLE we have mac addresses and for LoRaWAN we have their specific IDs. Or is there the need for multiple channels to the same device? If that is the case I think we must have this channel ID anyways.

We must support a single device having multiple NetworkChannels. A federate running on a device will have one NetworkChannel per other federate he talks to. However, we will only have a single NetworkChannel between two federates. So all "LF Connections" between two federates are combined into a "Bundle" together with a single NetworkChannel.

The idea sounds good, but I am not sure about the implementation details yet though. For TCP at the server side we need to first call bind and then accept .
Currently my examples dont work if the client calls connect before the server calls bind or accept (I dont know which of the two it is). We need to implement this in a way that is guaranteed deadlock-free and not dependent on timing at all.

3.1. Would the idea be to call the bind method inside of open_connection and then accept function in a non-blocking mode inside of the is_connected function? (It is somewhat strange to let is_connected "do work", but then that just depends on the perspective from which you are looking at it ;))

The idea would be to do the work in is_connected yes. The problem it solves is that we might have many NetworkChannels coming in and out and there might be cycles. We cannot have a blocking API where we end up blocking in a cycle = a deadlock. So we must be able to wait on all NetworkChannels at the same time.

3.2. Or maybe call bind and accept in open_connection and then continuously call open_connection in a non-blocking way in addition to is_connected?
I am no expert on sockets. But currently the main problem is actually that connection fails if client calls connect to early. So would maybe put bind in open_connection (if server, and nothing if client). And then is_connected (can rename to do_connection or whatever), does non-blocking connect/accept. It is important though that we do not depend on the order in which these these things are done. We can only expect that open_connection comes before is_connected at a particular federate, not accross them.

Thanks for feedback Lasse. This proposal is quick-and-dirty and I am very open to changing it, just want the discussion rolling.

Thanks for feedback Lasse. This proposal is quick-and-dirty and I am very open to changing it, just want the discussion rolling.

I think it is actually already a very good proposal as long as it is clear what function is doing what and why, but we can do that in the header file description.

I am no expert on sockets. But currently the main problem is actually that connection fails if client calls connect to early. So would maybe put bind in open_connection (if server, and nothing if client). And then is_connected (can rename to do_connection or whatever), does non-blocking connect/accept. It is important though that we do not depend on the order in which these these things are done. We can only expect that open_connection comes before is_connected at a particular federate, not accross them.

I just wanted to come up with the same idea after reading your feedback :) I think this should work at least for TCP.
Calling connect in open_conneciton for the client indeed makes no sense because it can fail and then is not called again, so yes it makes sense to put it into do_connection or maybe try_connect and call it non-blocking as it can fail as long as the server has not called bind. The network stack takes care of the tcp handshake to complete for the client even if the server has not yet called accept. This should not be a problem.
Another point:
I am thinking in theory we could also drop open_connection and just do whatever we do when we call try_connect or do_connection the first time? So the first while loop iteration would implicitly be a open_connection and the second while loop and so on would be do_connection or try_connect? Or do you think that could have disadvantages that I am missing?

I just wanted to come up with the same idea after reading your feedback :) I think this should work at least for TCP. Calling connect in open_conneciton for the client indeed makes no sense because it can fail and then is not called again, so yes it makes sense to put it into do_connection or maybe try_connect and call it non-blocking as it can fail as long as the server has not called bind. The network stack takes care of the tcp handshake to complete for the client even if the server has not yet called accept. This should not be a problem.

Naming is very important, and I think try_connect is a much better name yes. I have to look closer into the issue that I observed at Zephyr. I sometimes observe that the client is blocked at connect while the server is blocked at accept. Could it be because client called connect even before server called bind? I will investigate this today, I solved it by adding some sleep's for now.

Another point: I am thinking in theory we could also drop open_connection and just do whatever we do when we call try_connect or do_connection the first time? So the first while loop iteration would implicitly be a open_connection and the second while loop and so on would be do_connection or try_connect? Or do you think that could have disadvantages that I am missing?

This makes sense yes, probably no need to have two API calls for this.
A good excercise would be to see how this would work for BLE and LoraWAN also

Could it be because client called connect even before server called bind?

Yes connect by default is blocking. But it can be configured to be called in a non blocking way using the SOCK_NONBLOCK flag.

A good excercise would be to see how this would work for BLE and LoraWAN also

Yes this will get interesting. My fingers can't wait to start playing around with it :)

I have started to implement the UdpSocket and am wondering if I could maybe use this new interface already and migrate the tcp to it?
Just want to make sure we don't duplicate work

Yes, that would be great. The earlier we converge on this new interface the better. If you can take the lead on it, that would be really great :D


Fix documentation link for Docker Cloud vs Hub

Looks like Docker changed some links around and the old link is now only available when logged in. Changed the link to something which should work even for users without DockerHub accounts.

@theacodes noticed this while pushing a docker build for 2019.4.11, which should be live on DockerHub soon!

Perfect, thank you, @TheKevJames!


How to get TLS working when the container provides its own self-signed certificate

Describe the bug
The jgraph/drawio image internally exposes two ports 8080 (HTTP) and 8443 (HTTPS), and HTTPS port is backed by a self-signed certificate generated during startup: https://github.com/jgraph/docker-drawio/blob/dev/main/docker-entrypoint.sh#L146
This results in https://container.orb.local to use the self-signed certificate instead of the OrbStack one. I wonder if there's a way to override this and use the OrbStack certificate.
To Reproduce
No response
Expected behavior
No response
Diagnostic report (REQUIRED)
OrbStack info:
Version: 1.7.3
Commit: dc0aa82035784e357b70ee5beeca03547747fbc1 (v1.7.3)
System info:
macOS: 15.0 (24A335)
CPU: arm64, 12 cores
CPU model: Apple M3 Pro
Model: Mac15,6
Memory: 36 GiB
Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2024-09-25T08-53-29.269038Z.zip

Screenshots and additional context (optional)
No response

Confirmed fixed in the 1.8.1 update, closing.


Enable Sourcegraph

I want to use Sourcegraph for local-government-online-apps code search, browsing, and usage examples. Can an admin enable Sourcegraph for this repository? Just go to https://sourcegraph.com/github.com/Esri/local-government-online-apps. (It should only take 30 seconds.)
Thank you!

According to the help, you can use sourcegraph by installing one of the browser add-ins.
https://sourcegraph.com/help/doc


updated queue

Description

Urgency

[ ] Blocker
[x] High
[ ] Medium

Requirements

Test all commands and procedures.
Build content locally and test for formatting/links.
Add redirects to dcos-website/redirect-files.
Change all affected versions (e.g. 1.7, 1.8, and 1.9).
See the contribution guidelines.

@pleia2 or @joerg84 please merge if you have a sec :)

Actually not yet! found a mistake

Ok fixed


Demo testing pose groupstate

What this change adds to the repository?
A test for the pose_groupstate example provided by the crane_x7_ros repository. This change tests that the arm has reached the vertical pose and the ready pose state.
Methods for Testing
Created a new node that publishes joint_states to the terminal, then the test has an assert that checks the published joint_states against the vertical and ready values.
Preview of Expected Change
https://github.com/art-e-fact/crane_x7_ros/assets/72196105/cbab0683-f585-4179-8b51-ec08a3f19ada
How to test the change
launch_test pose_groupstate.launch.py

2 Trouble spots with the test that need addressing before merging.

There is a timer component in charge of the node subscription that I am uncertain with what to replace with.
The subscription run synchronously. As a result I cannot check the data as it is coming in, do you perhaps have an idea about how to make the submitted code asynchronous?


add testing for agent status syncers.

ref: https://github.com/stolostron/backlog/issues/26232
Signed-off-by: morvencao<EMAIL_ADDRESS>

Depends on https://github.com/stolostron/multicluster-global-hub/pull/270

/test sonarcloud

/retest

/LGTM


Better error handling

Some improvements to error handling, including:

an opentelemetry fix
tracebacks in logs for errors
an IrrecoverablePredictorError to signal serious problems with a predictor

Details in the commits.

It looks it's not the timing error I was trying to fix in 8f9ff5f – it seems the traceback isn't getting sent through as logs at all. @andreasjansson do the tests pass locally for you?

This is mostly obsolete after #795. If we want to keep something like IrrecoverablePredictorFailure then we'll need to rework this.


Make tracer creation more robust

This is an attempt to make CCL more robust to bad n(z). See e.g. #927, #919.

Added spline integration for the n(z) normalisation integral and the lensing kernel integral
Added flags in gsl_params to switch between spline and GSL quadrature integration for the two cases. Since spline integration is more robust and seems to be faster in the case where the n(z) is not super smooth or not very finely sampled, this is chosen as the default.
When using GSL quadrature for the lensing integral, the integration error is compared to the peak of the lensing kernel, instead of just the relative accuracy of the integral. This solves issues with numerical convergence of the integrals in the tails of the kernel where the absolute values are small but the integration method struggles.
Added option to choose lower number of samples for the lensing kernel than samples in the n(z). This is done using the n_samples argument, like in CMBLensingTracer. Right now there is an inconsitency in the API, in that the tSZ, CIB, and ISW tracers use n_chi for the same thing. This should be unified but as this involves a change to existing API, I've not included it here. @damonge Let me know what you think.
Added the option to call Tracer.get_kernels without specifying chi. In this case, the internal spline representation is returned. This is mostly useful for debugging purposes.
Enabled warnings by default (can be turned off with pyccl.debug_mode(False). Many of the warnings are there for good reason and warning the user that internal computations might produce garbage results is probably preferable to silent failures.
Added a warning when the status message gets overwritten. Right now there's no error stack or traceback. If multiple errors get written to cosmo.status_message, earlier messages get lost, which makes debugging difficult. Partially addresses #672. But this should really be handled cleaner (let's just switch to C++ and get exceptions and smart pointers!)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 97.052%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
4411

Relevant Lines:
4545

💛  - Coveralls

On cleaning up warnings: I agree... we should try to sort that out before V3
On nchi vs. nsamples: I also agree, but let's not break the API yet. Perhaps this could be added to #889

Unrelated to the title of this PR but definitely related to C-level warnings: currently, GSL raises an error when a value outside of the interpolation range is requested. For some reason, this is caught and converted to a warning in CCL, and the code ends up failing with an unexplained error CCLError: Error 1: . Would we consider promoting GSL out-of-bounds warnings to errors (perhaps in this PR)?
Try, for example, ccl.CosmologyVanillaLCDM().sigmaM(M=1e5, a=1).
As the vast majority (citation needed?) of GSL failures in CCL are caused by the queried values being out of bounds, maybe it's worth either catching this case specifically and promoting to an error (GSL_EDOM=-1 here or just promoting every GSL failure to an error.

OK, a more complete solution to error handling is for another PR. @tilmantroester see my suggestion above for the poorly-sampled case. Not sure if it's clear. I can provide a diagram if not.

@damonge The poorly-sampled case should be addressed now.
@nikfilippas I added a warning if the n(z) is poorly sampled that points used to changing the integration method, which is set by a flag in gsl_params. Once your PR is in that warning needs to be adjusted accordingly.

@tilmantroester I noticed while running some tests that this PR introduces a bunch of unnecessary warnings in the python unit tests and in the benchmarks. It would be nice to get rid of them in a separate PR to keep the output as clean as possible.


env: node\r: No such file or directory

Hey,
I'd like to use nunjucks with npm. So I installed nunjucks with:
npm i -g nunjucks-cli
If I try to run nunjucks from commandline with:
nunjucks index.html or just nunjucks
I get this error:
env: node\r: No such file or directory
Has someone the same issue?
I am running on OSX 10.11.4

I found a solution here:
here
brew install dos2unix sudo dos2unix /usr/local/lib/node_modules/task-app/src/task-app.js

It seems to be due to the CLRF endlines, as I created the module on Windows. I'll take car of it.

There you go. Thanks for the heads up.

I still have this issue on Arch Linux

Ok I think I fixed it, tested on Ubuntu and correction worked.
Could you give me a hint ?

Confirmed fixed as of the latest version on NPM.

Thanks!

Hi, I just pulled version 0.5.0 on Linux and still seeing line ending problems:
sudo npm install --global nunjucks-cli
/usr/bin/nunjucks -> /usr/lib/node_modules/nunjucks-cli/index.js
/usr/lib
└──<EMAIL_ADDRESS>
file -L /usr/bin/nunjucks
/usr/bin/nunjucks: a /usr/bin/env node\015 script, ASCII text executable, with CRLF line terminators

Yep sorry, I'm gonna set gitattributes to eol=LF to fix this once and for all.

Could you check again with 0.5.1 ?

@jeremyben Yep, that's working fine. Thanks!


Exclude sass files from ng-build

I use sass-loader in webpack.config to compile scss files to one bundled minified CSS file and I will manually reference this file in my "Index.html" but i need to exclude scss files from "ng build", I excluded it in tsconfig.app   "exclude": ["../**/*.scss"] but it didn't work
Versions.

@angular/cli: 1.2.0
node: 7.10.0
os: win32 x64
@angular/common: 4.2.5
@angular/compiler: 4.2.5
@angular/core: 4.2.5
@angular/forms: 4.2.5
@angular/http: 4.2.5
@angular/platform-browser: 4.2.5
@angular/platform-browser-dynamic: 4.2.5
@angular/router: 4.2.5
@angular/cli: 1.2.0
@angular/compiler-cli: 4.2.5
@angular/language-service: 4.2.5
The log given by the failure.

Still included in dist folder
Desired functionality.

Excluded from dist output folder

I was adding sass files in assets and try yo exclude them on same time. I think that was the problem


Selection tool

One critical thing is adding a selection tool so it's easy to copy/paste parts of an image around. This is decently complex, as we will need to track changes for history as well as making sure we don't break other tools.
Currently, all that is available is copying the selected sprite to another tile (ctrl/cmd + arrow keys) or shifting the currently selected sprite (shift + arrow keys). Shifting isn't perfect as it leaves existing pixels as they are when shifting.
I think I like these quick keys for copying sprites, which comes in handy when creating and editing animations, but a proper selection tool would be really nice.

I have recently added a first pass at a mask-based selection tool. I would love some feedback. My thinking here is that a lasso-based selection tool similar to photoshop would be less precise. The current tool supports the same shapes and sizes as the pencil/eraser tool. You can create new selections by just using the tool without any additional keys, while cmd/ctrl adds to the current selection and shift removes from the current selection.
After a selection is made, cmd/ctrl + t will cut and transform the current selection, cmd/ctrl + x will cut the selection or cmd/ctrl + c will copy the selection for pasting with cmd/ctrl + v.
If you have any suggestions for changes here, I'd love to hear them!


withBody + getBody results in an empty body

I have the following library code, which is implemented in another Symfony application with httplug and auto discovery.
// Both `$requestFactory` and `$streamFactory` are the same instance of `Nyholm\Psr7\Factory\HttplugFactory` (through auto discovery).

$request = $requestFactory->createRequest($method, $url);

$body = json_encode(['test' => 'test']);
$request->withBody($streamFactory->createStream($body));

echo (string)$request->getBody(); // empty string "" (!)

I've traced this problem to the MessageTrait, the withBody method (line 126), does this:
        $new = clone $this;
        $new->stream = $body;  // Note: `stream` is a PRIVATE property and that's probably why this does nothing.

If I change the code to this, it works as expected:
        $this->stream = $body;
        $new = clone $this;

What I don't understand is that nobody else runs into this problem ;)
My version of nyholm/psr7 is 1.2.1

I'm baffled, but when I change my code to do this, it works
$request = $this->requestFactory->createRequest($method, $this->host.$url)
    ->withBody($this->streamFactory->createStream($body));

echo (string)$request->getBody(); // now returns the $body as expected!

I really did not expect php to see direct chaining differently than using a local variable in between.
In any case, I stlil think this is very werid behavior and my first code should work too or otherwise give some kind of comprehensible exception why I shouldn't do that..
I still don't understand why the two statements are different

Every time you call a method on an object, it actually returns a completely new object rather than updating the existing one. So when you call $request->withBody(…) this call returns a new instance of a Request object while the original object $request is unchanged.
By making sure you redefine the variable $request you can make sure you always have the latest created object ready to use. You can do this by either chaining the method calls (because the final output returned gets assigned to your variable) or by redeclaring the variable like so:
$request = $requestFactory->createRequest($method, $url);

$body = json_encode(['test' => 'test']);
$request = $request->withBody($streamFactory->createStream($body));

echo (string)$request->getBody(); // empty string "" (!)

Note how I have updated line 4 of your example code to start with $request.
This is because PSR-7 objects are defined to be immutable. Meaning (in short) they cannot be changed after being created. To quote from the PSR-7 definition of a Request object:

 * Requests are considered immutable; all methods that might change state MUST
 * be implemented such that they retain the internal state of the current
 * message and return an instance that contains the changed state.

You can read more about this on the PHP-FIG’s website for PSR-7, under “Why value objects?”.
I am closing this issue as there is nothing here for this library to address. But if you have any follow-up questions, feel free to post them as I am happy to answer them :)


Stream & Sink Error types could benefit from core::fmt::Debug bound

I find one quanta of friction making it a bit harder to work with generic streams & sinks is they have this associated error type which is usually awesome because you control the error type, but in the case of streams and sinks, the low level details are really hard, so it's much more likely for a wide variety of developers to wind up using a smaller number of streams and sinks at a high level. Anyway, since more people are using streams and sinks than developing new streams and sinks, that means all the users are stuck with the trait bound on type Error, which doesn't exist,
Problem is, type Error doesn't have any trait bounds, it could be anything; this means it's hard to functionally map the Err branches because we need a custom error handler for each kind of Stream and Sink, you can't debug print your errors from Streams / Sinks, you're screwed, can't turn it into anyhow::Error because it's not a std::error::Error.
One wonders how much of a breaking change it would be for Stream and Sink Error to both be bounded by "std::error::Error" ?  big issue is this weirdly breaks with anyhow because they didnt implement std::error::Error for anyhow::Error, a decision which makes no sense except as a hack to avoid From conflicts. Therefore, I meekly suggest we need an easier bound for Stream / Sink errors than std::error::Error, but more than no bound..
A type Error: core::fmt::Debug bound would at least let us print out the errors regardless of the stream/sink we're dealing with. That'd be the minimalist approach and would work for anyhow::Error as well as custom or default error types. I just think it's important to have a fully functional interface because otherwise we have to re-implement differently for each kind of stream (source) and sink
LMK what you think, it's probably going to break stuff, but at least this way we could always generically debug print our error messages in streaming programs. Thank You!

example of the problem:
        match sink.send(output).await {
            Err(e) => Err(anyhow!("unprintable error from the outpipe sink")),

you'd need
error[E0277]: `<S2 as futures::Sink<O>>::Error` doesn't implement `std::fmt::Debug`
   --> src/flow.rs:148:76
    |
148 |             Err(e) => Err(anyhow!("unprintable error from the outpipe sink {e:#?}")),
    |                                                                            ^^^^^^ `<S2 as futures::Sink<O>>::Error` cannot be formatted using `{:?}` because it doesn't implement `std::fmt::Debug`
    |
    = help: the trait `std::fmt::Debug` is not implemented for `<S2 as futures::Sink<O>>::Error`
    = note: this error originates in the macro `$crate::__private::format_args` which comes from the expansion of the macro `anyhow` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider further restricting the associated type
    |
144 |     async fn load(&mut self, output: O) -> anyhow::Result<()> where <S2 as futures::Sink<O>>::Error: std::fmt::Debug {
    |                                                               ++++++++++++++++++++++++++++++++++++++++++++++++++++++


Create the Tasks, easy to start the server and UI

Added tasks.json under .vscode, this helps in starting the server/UI in a single click with the extension "Tasks" (https://marketplace.visualstudio.com/items?itemName=actboy168.tasks)
The tasks can also be started by Ctrl+Shift+P => Tasks: Run Task, and choose the task.

The task cannot be universally compatible with all operating systems; instead, we can develop two or more sets tailored for each OS individually.

copy of https://github.com/stitionai/devika/pull/345
we recommand to use >= 3.10 and < 3.12. I tested it with that not an issue. currently, so many projects doesn't support 3.12

not suitable for production and all users. you can share this in discussion threads or discord server threads that maybe helpful for someone.


OnEnter formatting Test is failing on CI

2018-12-12T23:36:08.0636695Z
2018-12-12T23:36:08.0636912Z     ✖ "before each" hook for "Formatting line with @" -- error: Failed to show text document file:///d%3A/a/1/s/src/test/pythonFiles/formatting/fileToFormatOnEnter.py, should show in editor #vs.editor.ICodeEditor:1,$model283
2018-12-12T23:36:08.0637113Z
2018-12-12T23:36:08.0717483Z ##[error]out\test\format\extension.onEnterFormat.test.js(,): error : FAILED Formatting - OnEnter provider :: "before each" hook for "Formatting line with @"
2018-12-12T23:36:08.0726663Z ##[debug]Processed: ##vso[task.logissue type=error;sourcepath=D:\a\1\s\out\test\format\extension.onEnterFormat.test.js;]FAILED Formatting - OnEnter provider :: "before each" hook for "Formatting line with @"
2018-12-12T23:36:08.0726772Z

No longer occurring.


Internet Observatory

Currently the Internet Observatory covers US metro areas - can we get data exports for all areas covered?

@pkoutroumpis  Our sincere apologies for not seeing this issue and getting back to you. We have discussed expanding Observatory to include other areas present in the M-Lab dataset, and may decide to pursue new visualizations for other regions in the coming year.
Our data are available to anyone however, and there are several ways to gain access and export it, as well as use it in your own applications. If you're comfortable with using Python scripts, you could check out Telescope. Its a Python framework for extracting raw measurements from M-Lab as CSV files for your own research/use. It does require a Google developer account to be whitelisted, which you can do by subscribing the account you'd like to use with it to M-Lab's Discuss list.
If there's anything more we can do to help, please let us know.


update(multiple): add Lighthouse 2.0 changes

Updated add-to-homescreen icons for the blog app that is used in
multiple labs. Changed the sizes of the icons to align with
the suggested sizes in Google documentation
and in Lighthouse 2.0 metrics. Updated the manifest files and meta
tags accordingly. Also modified the apps link colors to meet Lighthouse
accessibility requirements. Finally, added lang attribute to all
HTML files, again to meet Lighthouse 2.0 accessibility requirements.

Looks good. Merged!


DBT - Do not force to have both manifest and catalog

We should be a bit more flexible and allow either of them.
The issue happens in two places:

For example for Local DBT config, if we don't pass both files, the pydantic model is not properly parsed and thinks we are talking about S3 config
We will need to revisit the loading functions and how we use the info later to make sure that if either is missing, there are no exceptions

@OnkarVO7 will it be handled in DBT refactoring

Yes @harshach


Add way to run jax program on CPU

In my case, I have a small numpy array and a relative simple computation.
Is there a way to place the computation on the CPU to benchmark what's the overhead of transfering the computation to the GPU is?
Related to #957.

Indeed, @levskaya added this functionality in #1211! In fact, #957 is fixed too, we just haven't documented it...
The APIs are a bit in flux, though, as we're trying to simplify a couple things.
For placing on CPU:
def f(x): return x**2

f_cpu = jit(f, backend='cpu')
f_gpu = jit(f, backend='gpu')

cc @skye who helped with this example code
I'm going to leave this issue open as a documentation request.

@skye pointed out that there is a mention in the jit docstring

device – This is an experimental feature and the API is likely to change. Optional, the Device the jitted function will run on. (Available devices can be retrieved via jax.devices().) The default is inherited from XLA’s DeviceAssignment logic and is usually to use jax.devices()[0].

So let's actually close this issue, but leave up #957 as a request to finalize the API and document device placement of computations in a "Note" at jax.readthedocs.io. (I think one of the changes might be to unify "device" and "backend" arguments.)
By the way, if you want to measure just the transfer time, you can also use jax.device_put which itself also takes device and backend arguments.

Please reopen if the above answers aren't satisfactory!

Thanks for the quick replies!

If I understand correctly, there is no way to specify that all computations should be on CPU, right? It seems we need to adjust all the code to put all jits on CPU backend. It would be useful to have something similar to "config.update("jax_disable_jit", True)" to test your code when it won't run properly on GPU. Is such a config update available?

You can set the default platform globally with:
import jax
jax.config.update('jax_platform_name', platform)

How do I tell jnp.array to put the data on the CPU from the start? I can’t afford the perf hit of it going to the TPU and then getting it back. (The whole point of using the CPU is to avoid such a trip for small networks.)

You can use the device argument of jax.device_put to specify which device the data should be put on. For example, on a TPU runtime:
import jax
import numpy as np

x = jax.device_put(np.arange(5), device=jax.devices("tpu_driver")[0])
print(x.device_buffer.device())
# TPU_0(host=0,(0,0,0,0))

y = jax.device_put(np.arange(5), device=jax.devices("cpu")[0])
print(y.device_buffer.device())
# TFRT_CPU_0


Sign in with google

Is it possible to integrate the sign in with google method?

Of course. next-firebase-auth-edge is about validating token on the backend and storing credentials in user cookies. It doesn’t care where the token is coming from :-) In order to authenticate user with Google you can follow official library docs https://firebase.google.com/docs/auth/web/google-signin


[ENHANCEMENT] Configuration to use an already existing Elasticsearch

Describe the use case:
I would like to connect Camunda Platform (Zeebe, Operate, Tasklist and Optmize) to an external managed Elasticsearch.
Describe the enhancement/feature:
Here is the values.yaml file to achieve this : (tested with an Elasticsearch provided on https://cloud.elastic.co/)
global:
  elasticsearch:
    url: "https://camunda-platform-elasticsearch.com"
    host: camunda-platform-elasticsearch.com
    port: 443

zeebe:
  # ClusterSize defines the amount of brokers (=replicas), which are deployed via helm
  clusterSize: "1"
  # PartitionCount defines how many zeebe partitions are set up in the cluster
  partitionCount: "1"
  # ReplicationFactor defines how each partition is replicated, the value defines the number of nodes
  replicationFactor: "1"

  resources:
    requests:
      cpu: 800m
      memory: 1200Mi
    limits:
      cpu: 800m
      memory: 1200Mi

  # PvcSize defines the persistent volume claim size, which is used by each broker pod https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims
  pvcSize: "16Gi"

  env:
    - name: ZEEBE_BROKER_DATA_SNAPSHOTPERIOD
      value: "5m"
    - name: ZEEBE_BROKER_DATA_DISKUSAGECOMMANDWATERMARK
      value: "0.85"
    - name: ZEEBE_BROKER_DATA_DISKUSAGEREPLICATIONWATERMARK
      value: "0.87"
    - name: ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_AUTHENTICATION_USERNAME
      value: "elastic"
    - name: ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_AUTHENTICATION_PASSWORD
      value: "mysecretpassword"

zeebe-gateway:
  # Replicas defines how many standalone gateways are deployed
  replicas: 1

operate:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

  env:
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"

tasklist:
  resources:
    requests:
      cpu: 400m
      memory: 1Gi
    limits:
      cpu: 400m
      memory: 1Gi

  env:
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"
    - name: CAMUNDA_TASKLIST_ZEEBE_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_TASKLIST_ZEEBE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_TASKLIST_ZEEBE_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"

optimize:
  # PartitionCount defines how many Zeebe partitions are set up in the cluster and which should be imported by Optimize
  partitionCount: "1"

  image:
    # Image.repository defines which image repository to use
    repository: camunda/optimize
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    # Don't change the comment after the value, it's needed due to a bug yq. Check Makefile for more details.
    tag: 3.9.2 # optimize.image.tag

  env:
    - name: CAMUNDA_OPTIMIZE_ELASTICSEARCH_SSL_ENABLED
      value: "true"
    - name: CAMUNDA_OPTIMIZE_ELASTICSEARCH_SECURITY_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPTIMIZE_ELASTICSEARCH_SECURITY_PASSWORD
      value: "mysecretpassword"

  resources:
    requests:
      cpu: 600m
      memory: 1Gi
    limits:
      cpu: 600m
      memory: 1Gi

identity:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

elasticsearch:
  enabled: false

It would be very nice if we could also configure CA certificate or truststore for use cases where elasticsearch is running with security local in same cluster.  Or set some config to not validate certificates in this use case.
Using above example, zeebe tries to connect but fails certificate validation because the elasticsearch in the cluster is running with self signed certificates.
Zeebe log:
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Same goes for the other services operate/optimize/tasklist. Tested with operate, it connects to elastic, but fails on certificate validation:
2022-11-07 14:08:59.252  WARN 7 --- [           main] i.c.o.u.RetryOperation                   : java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Hello @mhuijgen  !
I have prepared a values.yaml example for this, but it still a work in progress.
Do you want me to share it ?

You should first create a keystore from your certificate :
keytool -import -alias elasticsearch  -keystore elasticsearch.jks -file elasticsearch.cer -storepass changeit -noprompt

Then create a secret from this certificate :
kubectl create secret generic elasticsearch-jks --from-file=elasticsearch.jks

Mount the secret in each pod (See example below) :

global:
  # Image configuration to be used in each sub chart
  image:
    # Image.tag defines the tag / version which should be used in the chart
    tag: 8.1.2

  elasticsearch:
    url: "https://camunda-platform-elasticsearch.com"
    host: camunda-platform-elasticsearch.com
    port: 443

zeebe:
  # ClusterSize defines the amount of brokers (=replicas), which are deployed via helm
  clusterSize: "1"
  # PartitionCount defines how many zeebe partitions are set up in the cluster
  partitionCount: "1"
  # ReplicationFactor defines how each partition is replicated, the value defines the number of nodes
  replicationFactor: "1"

  resources:
    requests:
      cpu: 800m
      memory: 1200Mi
    limits:
      cpu: 800m
      memory: 1200Mi

  # PvcSize defines the persistent volume claim size, which is used by each broker pod https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims
  pvcSize: "16Gi"

  env:
    - name: ZEEBE_BROKER_DATA_SNAPSHOTPERIOD
      value: "5m"
    - name: ZEEBE_BROKER_DATA_DISKUSAGECOMMANDWATERMARK
      value: "0.85"
    - name: ZEEBE_BROKER_DATA_DISKUSAGEREPLICATIONWATERMARK
      value: "0.87"
    - name: ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_AUTHENTICATION_USERNAME
      value: "elastic"
    - name: ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_AUTHENTICATION_PASSWORD
      value: "mysecretpassword"

  javaOpts: >-
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:HeapDumpPath=/usr/local/zeebe/data
    -XX:ErrorFile=/usr/local/zeebe/data/zeebe_error%p.log
    -XX:+ExitOnOutOfMemoryError
    -Djavax.net.ssl.trustStore=/usr/local/zeebe/certificates/elasticsearch.jks

  extraVolumeMounts:
    - name: keystore
      mountPath: /usr/local/zeebe/certificates/elasticsearch.jks
      subPath: elasticsearch.jks
  extraVolumes:
    - name: keystore
      secret:
        secretName: elasticsearch-jks
        optional: false

zeebe-gateway:
  # Replicas defines how many standalone gateways are deployed
  replicas: 1

operate:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

  env:
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"
    - name: JAVA_TOOL_OPTIONS
      value: "-Djavax.net.ssl.trustStore=/usr/local/operate/certificates/elasticsearch.jks"

  extraVolumeMounts:
    - name: keystore
      mountPath: /usr/local/operate/certificates/elasticsearch.jks
      subPath: elasticsearch.jks
  extraVolumes:
    - name: keystore
      secret:
        secretName: elasticsearch-jks
        optional: false

tasklist:
  resources:
    requests:
      cpu: 400m
      memory: 1Gi
    limits:
      cpu: 400m
      memory: 1Gi

  env:
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"
    - name: CAMUNDA_TASKLIST_ZEEBE_ELASTICSEARCH_URL
      value: "https://camunda-platform-elasticsearch.com"
    - name: CAMUNDA_TASKLIST_ZEEBE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_TASKLIST_ZEEBE_ELASTICSEARCH_PASSWORD
      value: "mysecretpassword"
    - name: JAVA_TOOL_OPTIONS
      value: "-Djavax.net.ssl.trustStore=/app/certificates/elasticsearch.jks"

  extraVolumeMounts:
    - name: keystore
      mountPath: /app/certificates/elasticsearch.jks
      subPath: elasticsearch.jks
  extraVolumes:
    - name: keystore
      secret:
        secretName: elasticsearch-jks
        optional: false

  ingress:
    enabled: true
    className: nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      cert-manager.io/cluster-issuer: letsencrypt
    host: "tasklist.cwa.wap.ibminteractive.fr"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: tasklist.cwa.wap.ibminteractive.fr

optimize:
  # PartitionCount defines how many Zeebe partitions are set up in the cluster and which should be imported by Optimize
  partitionCount: "1"

  image:
    # Image.repository defines which image repository to use
    repository: camunda/optimize
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    # Don't change the comment after the value, it's needed due to a bug yq. Check Makefile for more details.
    tag: 3.9.2 # optimize.image.tag

  env:
    - name: CAMUNDA_OPTIMIZE_ELASTICSEARCH_SSL_ENABLED
      value: "true"
    - name: CAMUNDA_OPTIMIZE_ELASTICSEARCH_SECURITY_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPTIMIZE_ELASTICSEARCH_SECURITY_PASSWORD
      value: "mysecretpassword"
    - name: JAVA_TOOL_OPTIONS
      value: "-Djavax.net.ssl.trustStore=/optimize/certificates/elasticsearch.jks"

  resources:
    requests:
      cpu: 600m
      memory: 1Gi
    limits:
      cpu: 600m
      memory: 1Gi

  extraVolumeMounts:
    - name: keystore
      mountPath: /optimize/certificates/elasticsearch.jks
      subPath: elasticsearch.jks
  extraVolumes:
    - name: keystore
      secret:
        secretName: elasticsearch-jks
        optional: false

identity:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

elasticsearch:
  enabled: false

⚠  It is still a Work In Progress ⚠

To debug, you can add -Djavax.net.debug=all in your JAVA_TOOL_OPTIONS
    - name: JAVA_TOOL_OPTIONS
      value: "-Djavax.net.debug=all -Djavax.net.ssl.trustStore=/usr/local/operate/certificates/elasticsearch.jks"

Tested this work in progress values file after creating the secret based on the ca.crt exported from the secret created by the bitnami/elasticsearch chart and it works right away. That is, it can connect now just fine and operate starts to create some indexes. However it seems that camunda/operate:8.1.2does not yet support ES 8.x ?
https://forum.camunda.io/t/operate-8-0-1-not-supporting-elasticsearch-8-1-3/37298
2022-11-07 18:45:05.229 DEBUG 7 --- [           main] i.c.o.s.ElasticsearchSchemaManager       : Template [operate-incident-1.3.0_template] was successfully created
2022-11-07 18:45:05.234  WARN 7 --- [           main] o.e.c.RestClient                         : request [HEAD https://elastic-elasticsearch:9200/operate-incident-1.3.0_?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=false] returned 1 warnings: [299 Elasticsearch-8.5.0-c94b4700cda13820dad5aa74fae6db185ca5c304 "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2022-11-07 18:45:05.531 DEBUG 7 --- [           main] i.c.o.s.ElasticsearchSchemaManager       : Index [operate-incident-1.3.0_] was successfully created
2022-11-07 18:45:05.539  WARN 7 --- [           main] o.e.c.RestClient                         : request [HEAD https://elastic-elasticsearch:9200/operate-web-session-1.1.0_?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=false] returned 1 warnings: [299 Elasticsearch-8.5.0-c94b4700cda13820dad5aa74fae6db185ca5c304 "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2022-11-07 18:45:05.635  WARN 7 --- [           main] i.c.o.u.RetryOperation                   : Elasticsearch exception [type=illegal_argument_exception, reason=Unknown token [VALUE_NULL] in alias [operate-web-session-1.1.0_alias]]

@mhuijgen Zeebe support only version 7.x.x

Looks like the latest bitnam/elasticsearch chart 19.0.5 still configures elasticsearch 7.17.7 correctly if I override the image tag. Redeployed camunda using based on @vctrmn 's values.yaml and operate now created all indexes succesfully using a secured elasticsearch with self signed certificates.

Notes from discussions, we can split this down into multiple topics:

ease the configuration to configure in a central place an external elasticsearch
support self-signed certificate for elasticsearch or other external systems

@aabouzaid we could break this down into the two issues, and already prioritize the first topic, for the second one we have to discuss when and how we want to implement https://github.com/camunda/product-hub/issues/583

Tested this work in progress values file after creating the secret based on the ca.crt exported from the secret created by the bitnami/elasticsearch chart and it works right away. That is, it can connect now just fine and operate starts to create some indexes. However it seems that camunda/operate:8.1.2does not yet support ES 8.x ?
https://forum.camunda.io/t/operate-8-0-1-not-supporting-elasticsearch-8-1-3/37298
2022-11-07 18:45:05.229 DEBUG 7 --- [ main] i.c.o.s.ElasticsearchSchemaManager : Template [operate-incident-1.3.0_template] was successfully created 2022-11-07 18:45:05.234 WARN 7 --- [ main] o.e.c.RestClient : request [HEAD https://elastic-elasticsearch:9200/operate-incident-1.3.0_?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=false] returned 1 warnings: [299 Elasticsearch-8.5.0-c94b4700cda13820dad5aa74fae6db185ca5c304 "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."] 2022-11-07 18:45:05.531 DEBUG 7 --- [ main] i.c.o.s.ElasticsearchSchemaManager : Index [operate-incident-1.3.0_] was successfully created 2022-11-07 18:45:05.539 WARN 7 --- [ main] o.e.c.RestClient : request [HEAD https://elastic-elasticsearch:9200/operate-web-session-1.1.0_?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=false] returned 1 warnings: [299 Elasticsearch-8.5.0-c94b4700cda13820dad5aa74fae6db185ca5c304 "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."] 2022-11-07 18:45:05.635 WARN 7 --- [ main] i.c.o.u.RetryOperation : Elasticsearch exception [type=illegal_argument_exception, reason=Unknown token [VALUE_NULL] in alias [operate-web-session-1.1.0_alias]]

Hi @mhuijgen
Can you please quickly document what you did with bitnami elasticsearch? What is your value file?

@hoangduy1912  you are correct that Camunda does not support ES 8.x.
What I did is use the bitnami/elasticsearch chart version 19.5.0, which by default uses 8.x, but I overrode the image tag to use latest 7.17.x at the time. The values I used for this chart:
global:
  kibanaEnabled: true
image:
  tag: 7.17.7-debian-11-r4
master:
  replicaCount: 1
  masterOnly: false
  heapSize: 1024m
  persistence:
    size: 40Gi
data:
  replicaCount: 0
coordinating:
  replicaCount: 0
ingest:
  replicaCount: 0
security:
  enabled: true
  tls:
    autoGenerated: true
kibana:
  image:
    tag: 7.17.7-debian-11-r4
  persistence:
    enabled: false
  elasticsearch:
    hosts:
      - elastic-elasticsearch
    port: 9200
    security:
      auth:
        enabled: true
        createSystemUser: true
        kibanaPassword: secretPass
        elasticsearchPasswordSecret: elastic-elasticsearch
      tls:
        enabled: true
        existingSecret: elastic-elasticsearch-master-crt
        usePemCerts: true

elasticsearch-jks

Thanks @mhuijgen
How about the secret? I followed these steps but the following error occured
2023-03-09 09:27:18.168  INFO 1 --- [           main] i.c.t.e.ElasticsearchConnector           : Retrying #7 connect to Elasticsearch at https://bitnami-es-elasticsearch:9200 due to ElasticsearchException[java.util.concurrent.ExecutionException: javax.net.ssl.SSLException: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty]; nested: ExecutionException[javax.net.ssl.SSLException: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty]; nested: SSLException[java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty]; nested: InvalidAlgorithmParameterException[the trustAnchors parameter must be non-empty];

You should first create a keystore from your certificate :
keytool -import -alias elasticsearch  -keystore elasticsearch.jks -file elasticsearch.cer -storepass changeit -noprompt

Then create a secret from this certificate :
kubectl create secret generic elasticsearch-jks --from-file=elasticsearch.jks

To create the secret with java keystore I used:
kubectl  get secret elastic-elasticsearch-master-crt -o jsonpath='{.data.ca\.crt}'|base64 -d >elastic-ca.crt
keytool -import -alias elasticsearch -keystore elasticsearch.jks -storetype jks -file elastic-ca.crt -storepass changeit -noprompt
kubectl  create secret generic elasticsearch-jks --from-file=elasticsearch.jks

Deployed using camunda helm chart version 8.1.0 using the following values (showing just the bits for zeebe and operate). Depending on the deployment name for your elasticsearch you might need to tweak some names a bit.
global:
  elasticsearch:
    url: "https://elastic-elasticsearch:9200"
    host: "elastic-elasticsearch"
    port: 9200

zeebe:
  clusterSize: 1
  partitionCount: 1
  replicationFactor: 1
  env:
    - name: ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_AUTHENTICATION_USERNAME
      value: "elastic"
    - name: ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_AUTHENTICATION_PASSWORD
      valueFrom:
        secretKeyRef:
          name: elastic-elasticsearch
          key: elasticsearch-password
  javaOpts: >-
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:HeapDumpPath=/usr/local/zeebe/data
    -XX:ErrorFile=/usr/local/zeebe/data/zeebe_error%p.log
    -XX:+ExitOnOutOfMemoryError
    -Djavax.net.ssl.trustStore=/usr/local/zeebe/certificates/elasticsearch.jks
  extraVolumeMounts:
    - name: keystore
      mountPath: /usr/local/zeebe/certificates/elasticsearch.jks
      subPath: elasticsearch.jks
  extraVolumes:
    - name: keystore
      secret:
        secretName: elasticsearch-jks
        optional: false

operate:
  enabled: true
  env:
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_URL
      value: "https://elastic-elasticsearch:9200"
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_PASSWORD
      valueFrom:
        secretKeyRef:
          name: elastic-elasticsearch
          key: elasticsearch-password
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_URL
      value: "https://elastic-elasticsearch:9200"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_USERNAME
      value: "elastic"
    - name: CAMUNDA_OPERATE_ZEEBE_ELASTICSEARCH_PASSWORD
      valueFrom:
        secretKeyRef:
          name: elastic-elasticsearch
          key: elasticsearch-password
    - name: JAVA_TOOL_OPTIONS
      value: "-Djavax.net.ssl.trustStore=/usr/local/operate/certificates/elasticsearch.jks"
  extraVolumeMounts:
    - name: keystore
      mountPath: /usr/local/operate/certificates/elasticsearch.jks
      subPath: elasticsearch.jks
  extraVolumes:
    - name: keystore
      secret:
        secretName: elasticsearch-jks
        optional: false

Also note that every time you helm upgrade bitnami/elasticsearch with values
  tls:
    autoGenerated: true

It will generate a new elastic-ca certificate after which you will need to update the JKS secret and rollout restart the camunda services.

Thanks a lot @mhuijgen
It worked.

We plan to start on this once https://github.com/camunda/camunda-platform-helm/issues/655 is done.

Connecting to es-cloud is currently working on my PR with the values.yaml configuration that @aabouzaid and I agreed with.
Now,I am currently working on connecting to elasticsearch on a separate namespace.

Connecting to external self managed elasticsearch is working for all components. The user can also provide a tls secret that will be used to connect over https.
Connecting to external AWS managed opensearch is working for all components currently, except optimize. The functionality for providing a tls secret is not implemented in the opensearch section since it is not needed for specifically AWS managed  opensearch.

All changes for external elasticsearch support are merged.


Make TestAppleSimulatorOSType.py more flexible

Different versions of Xcode have different outputs for the simctl command
git-svn-id: https://llvm.org/svn/llvm-project/lldb/trunk@347117 91177308-0d34-0410-b5e6-96231b3b80d8
(cherry picked from commit 47916947abc087c447044448d8e6f3159af623d5)

@swift-ci test and merge


Greatest common Divisor in java

This program calculates the nth fibonacci number where n is inputted from the user
Fixes issue:
fixes #364
Changes:
added a java program that calculates GCD of 2 numbers inputted from user

Thank you for contributing @Aaryan2808
Keep it up


[AutoPR compute/resource-manager] Adding "proximityPlacementGroup" property in "VirtualMachineScaleSetUpdateProperties". Adding "includeColocationStatus" optional flag as part of Get ProximityPlacementGroup URL

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/7745

This PR has been merged into https://github.com/Azure/azure-sdk-for-go/pull/6354


Set swift version in config.xml

I am using XCode 9 beta which only supports Swift 3.2 and 4. This pull requests adds the possibility to set the Swift version in config.xml.
What do you think?

Hello ! It sounds like a good idea ! Thx


Only subdirectories in /var/lib/kubelet should be unmounted at reset time

If /var/lib/kubelet has initially its own partition, it should not be unmounted.

@mirwan  why?

@riverzhang correct me if I'm wrong but, independently of kubelet_deployment_type, kubespray does not specifically mount /var/lib/kubelet on the host, it only creates the directory.
Thus, at reset time, it should only remove the directory.
At least, it shouldn't umount it when kubelet_deployment_type is set to host (even if I don't think that it would change anything as the "service stop" would free the docker/rkt volume).
Anyway, on bare metal, having a dedicated device for /var/lib/kubelet can happen, if it is mounted before kubespray, it should not be unmounted at reset time


6.4.x

I need jbpm-designer

Can one of the admins verify this PR? Comment with 'ok to test' to start the build.

Can one of the admins verify this PR? Comment with 'ok to test' to start the build.

Was this PR created by accident ?

Looks like it sure was created by accident. Closing.
@ajaymuj if you have jBPM Designer related questions please ask at https://groups.google.com/forum/#!forum/jbpm-usage


I can't use lettuce-core to link to a Docker-built "bitnami-Redis-cluster" cluster.

I can't use lettuce-core to link to a Docker-built "bitnami-Redis-cluster" cluster.
It always takes the internal IP of the Docker and links it to the port number.

The cluster is not on the local machine and cannot be resolved using "network_mode: host". This is the main solution on the stack, but this problem can occur if you want to debug with a remote cluster. @mp911de

This is a forum for Lettuce-specific topics, we cannot help with general Docker connectivity issues.


Cannot see available app builds when clicking email link

As soon as a build is completed successfully, an email is sent to all the collaboration team members. Recently and when clicking to the install button over the email we get redirected to appcenter's following screen instead of the list with the available builds.

Please let me know what more info you require, thanks

@marudy Thanks for reporting it. Our team will look into this issue.

HI @Marudy, can you please open a support ticket on appcenter.ms in intercom so we can take a deeper look into your app?

Hey @botatoes thanks for the reply. Can you show me to the direction of intercom? I mean some link to follow? thanks


Model Checkpoints

Hey Amal,
thanks for providing this code! Would it be possible to get access to the model checkpoint of the DenseCL version trained on COCO2VOC?
Best
Pascal

Hi @psclklnk! Sure. I will try in a few days.

Try to reach it with this link (Google Drive)


Pattern Matching

ActorForth's first (primary, only, preferred, best?) form of code flow control will be pattern matching.
Support syntax such as the following: (don't know why the damn indentations in the code go away)
fib : nat -> nat;
fib : 0 -> 0.
fib : 1 -> 1.
fib : _ -> nat;
dup
1 -
fib
swap
2 -
fib
+.
Whether "fib" needs to be re-declared before each ":" is open to debate. How ";" and "." are treated becomes relevant here.
"" is a replacement for the "Any" type. We may ultimately replace "Any" with "" especially for type checking things like
swap : Any1 Any2 -> Any2 Any2;

with:
swap: _1 _2 -> _2 _1;
or something like that.

Implemented as of checkin: 492e6cdd9823bedda606d8974b35aee99e9f10f9


bs-close-button does not exist

hi
on bsmodalheader there is an import to @lit-element-bootstrap/button/bs-close-button.js
and this js file does not exist can you upload it or change it?

The latest and stable version of @lit-element-bootstrap/button is not backwards compatible with any component in alpha version. @lit-element-bootstrap/modal is in alpha version should it should be using @lit-element-bootstrap/button with version 1.0.0-alpha.7 and the 1.0.0


() is missing in parse_log.py

When I use 'Vitis-AI/models/AI-Model-Zoo/caffe-xilinx/tools/extra/parse_log.py', an errors shows,
File "/workspace/tutorials/caffe-xilinx/tools/extra/parse_log.py", line 166
print 'Wrote %s' % output_filename
I think () is missing in the sentence.
print ('Wrote %s' % output_filename) is correct.

Wich version of python do you use?

Thank you for your reply.
I'm using caffe-xilinx in docker (vitis-ai-gpu:latest), so python version is 3.6.12.
I copied the caffe-xilinx directory under the tutorials directory and use it.

Ok, I see. Actually, The caffe-xilinx you copied can be only used with python 2 and there is a caffe-xilinx of python 3 in docker.

I thought AI-Model-Zoo/caffe-xilinx would work in docker. Thank you.


I want to help in translating this product to Hindi and Marathi languages.

Prerequisites

[x] I'm using latest version:  v0.17.0

Description
[There are nearly 54% users in India who speak Hindi as opposed to 12% English speakers. It would be a great opportunity to increase reach of this software. Also, I genuinely want this to be in Hindi so that elderly,office workers who I have noticed keep seating from 10am to 6 am makes me worried about their health.  I'm ready to contribute, waiting for @everyone's input. Thanks.]
Operating system:
Expected behavior: [It should have an option to show instructions in Hindi and Marathis]
Actual behavior: [Currently I could not find an option to do so]
Additional Information
I'm a native speaker of both the languages and have helped other open source softwares in translation.

for reference  https://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers_in_India

Hi ;]
There is no way right now to translate break ideas, only the interface. There is a translation for Hindi (https://github.com/hovancik/stretchly/blob/master/app/locales/hi.json) which could use an update. There is no version for Marathi, yet, so feel free to add it.
Latest version is v0.19.1 btw ;]

Hi ;]
There is no way right now to translate break ideas, only the interface. There is a translation for Hindi (https://github.com/hovancik/stretchly/blob/master/app/locales/hi.json) which could use an update. There is no version for Marathi, yet, so feel free to add it.
Latest version is v0.19.1 btw ;]

Yes,I updated the app. And to be honest the hindi translation seems like done by someone who just simply used google translate, did not care about actual words that are used on daily basis. I will send a pr soon.

checkout #402


ReversingLabs A1000 - Correct dynamicHostUrl to dynamichosturl

When submitting a connector, please make sure that you follow the requirements below, otherwise your PR might be rejected. We want to make you have a well-built connector, a smooth certification experience, and your users are happy :)
If this is your first time submitting to GitHub and you need some help, please sign up for this session.

[x] I attest that the connector doesn't exist on the Power Platform today. I've verified by checking the pull requests in GitHub and by searching for the connector on the platform or in the documentation.
[x] I attest that the connector works and I verified by deploying and testing all the operations.
[x] I attest that I have added detailed descriptions for all operations and parameters in the swagger file.
[x] I attest that I have added response schemas to my actions, unless the response schema is dynamic.
[x] I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
[x] If this is a certified connector, I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff. If this is an independent publisher connector, I confirm that I am not submitting a connector icon.

If you are an Independent Publisher, you must also attest to the following to ensure a smooth publishing process:

[ ] I have named this PR after the pattern of "Connector Name (Independent Publisher)" ex: HubSpot Marketing (Independent Publisher)
[ ] Within this PR markdown file, I have pasted screenshots that show: 3 unique operations (actions/triggers) working within a Flow. This can be in one flow or part of multiple flows. For each one of those flows, I have pasted in screenshots of the Flow succeeding.
[ ] Within this PR markdown file, I have pasted in a screenshot from the Test operations section within the Custom Connector UI.
[ ] If the connector uses OAuth, I have provided detailed steps on how to create an app in the readme.md.

@Amjed-Ayoub I only changed the templateId from "dynamicHostUrl" to "dynamichosturl" since I can see that the vast majority of other connectors have it that way and our connection did not work.
https://github.com/microsoft/PowerPlatformConnectors/pull/2742/files#diff-ed82c2474ad561ceadd51d8fea1fd2ac5993120f85d45f3bf7d52d3fbde329bcR38

Hello @MislavReversingLabs,
can you please fix the conflict.

Hello @MislavReversingLabs, can you please fix the conflict.

Why are those conflicts? Those are precisely the lines I wanted to change. We want the new values that we commited in this PR.

@Amjed-Ayoub Done.

@Amjed-Ayoub @sriyen-msft Is this PR now merged or not? I received a message from Srikanth in ISV Studio that the A1000 connector is now published to the preview environment and I can find it there.
Also, I am getting this error when I try to create a connection now that I corrected the "dynamichosturl" key:
Failed to create connection for connection id '/providers/Microsoft.PowerApps/apis/shared_reversinglabsa1000/connections/shared-reversinglabs-273bec14-88d3-4fa8-a4f3-af6fe626afb8'. The request failed with client error: 'Input parameters are invalid. See details for more information. Details:errorCode: ParameterNotDefined. Message: Parameter 'apiKey' is not allowed on the connection since it was not defined as a connection parameter when the API was registered..'. The correlation Id is 'dc0bcb72-70fc-4e7d-84c1-739d6c6a71bd'.


Document Source and entryUUID

Section Number
https://profiles.ihe.net/ITI/MHD/a_issues.html
CP-ITI-1119: Made clear that Document Source must not populate any entryUUID.
Issue
CP-ITI-1119: Made clear that Document Source must not populate any entryUUID.
According to the original CP I cannot find out the summary of the above sentence.
In addition Table 4.3.1-3: Sending Actor Metadata Attribute Optionality sets entryUUID as Required for the sending Actor.
See also discussion in google group
Proposed Change
Remove the summary of CP-ITI-1119 and enforce entryUUID also in MHD, see also https://github.com/IHE/ITI.MHD/issues/13.
Priority:

High: Important issue where there is major issue to be resolved. Requires discussion and debate.

Move CP-ITI-1119 to closed issues, and enforce entryUUID in MHD.


Add additional hint text to facility form during initial sign-up

Related Issue

Closes #4023

Changes Proposed

For facility registration post-sign-up, display some additional help text aimed at users targeting the CSV Uploader workflow

Screenshots / Demos

@vz3 it looks like some of the existing help text on the Manage Devices section was previously set to be a full-width paragraph. Should I narrow the paragraph width for that as well?

this is a design thought and not code feedback, but seems like the text in the manage devices should all be the same style. at least, it's not clear to me why one part is grey and the other part is darker

this is a design thought and not code feedback, but seems like the text in the manage devices should all be the same style. at least, it's not clear to me why one part is grey and the other part is darker

💯, hoping @vz3 can adjudicate on how we want the styling to look here. I would guess we'd want to unify the presentation of the bits of help text but I'm not sure which style we'd like to standardize on.

Can we move the "If you don't see a device" warning to just underneath the "Device Types" input field? Also I think that should probably get sentence casing i.e. "Device types" cc @lauraykerr for visibility

Can we move the "If you don't see a device" warning to just underneath the "Device Types" input field? Also I think that should probably get sentence casing i.e. "Device types" cc @lauraykerr for visibility

@vz3 good catch on the sentence casing 👍 here is what the form looks like with the hint text moved below the input field; let me know if you think the spacing/layout will need any tweaking


Report invalid GTFS to UI

I am uploading a ZIP of 18 GTFS feeds and OSM data. The upload never seems to complete (based on the UI spinner). Looking at the server logs, I see that the GTFS feed is missing all required tables.
Oddly, it also reports that the feed does not contain a feed ID and it's using the file name 1b376653-c553-4d28-a752-3753d607952c. This ZIP file does not contain any file by that name, all GTFS files have been given the names of their respective agencies.
06:50:48.916 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table feed_info was missing but it is not required.
06:50:48.916 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - Feed ID is undefined, pester maintainers to include a feed ID. Using file name 1b376653-c553-4d28-a752-3753d607952c.
06:50:48.916 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table calendar_dates was missing but it is not required.
06:50:48.916 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table fare_attributes was missing but it is not required.
06:50:48.916 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table fare_rules was missing but it is not required.
06:50:48.916 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table shapes was missing but it is not required.
06:50:48.917 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table transfers was missing but it is not required.
06:50:48.917 [Thread-16] INFO  c.conveyal.gtfs.model.Entity$Loader - Table frequencies was missing but it is not required.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - 6 errors
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: agency line 0: This table is required by the GTFS specification but is missing.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: calendar line 0: This table is required by the GTFS specification but is missing.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: routes line 0: This table is required by the GTFS specification but is missing.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 0: This table is required by the GTFS specification but is missing.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stops line 0: This table is required by the GTFS specification but is missing.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: trips line 0: This table is required by the GTFS specification but is missing.
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - Building stop to stop times index
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - Building trips per service index
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - Building services per date index
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - Generating unique names for patterns
06:50:48.917 [Thread-16] INFO  com.conveyal.gtfs.GTFSFeed - Total patterns: 0
06:50:49.029 [Thread-16] INFO  com.conveyal.gtfs.GTFSCache - Writing feed to s3 cache
06:50:51.322 [Thread-16] INFO  com.conveyal.gtfs.GTFSCache - Zip file written.
06:50:51.500 [Thread-16] INFO  com.conveyal.gtfs.GTFSCache - db files written.
Exception in thread "Thread-16" java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 0
        at gnu.trove.list.array.TDoubleArrayList.get(TDoubleArrayList.java:281)
        at com.conveyal.taui.controllers.BundleController.lambda$create$10(BundleController.java:157)
        at java.lang.Thread.run(Unknown Source)

Finally finding the geographic center fails because of course the feed has no stops. It seems clear that the server is attempting to interpret some ZIP file as a GTFS feed, when the file is not really a GTFS feed. But I unzipped the file I uploaded and unzipped all the GTFS files it contains, and they all look like valid GTFS with routes, stops etc.
The simplest explanation is that the server is interpreting my ZIP of 18xGTFS+OSM as a single GTFS feed, which would lead to exactly these errors. I see that my 117MB archive appears in the analyst disk cache prefixed with the same UUID that the logs report as the corrupt GTFS's file name.

Oh, looking at the BudleController code now I undestand. The new UI cannot handle uploading a ZIP of GTFS and OSM. It expects you to multiple-select the files and upload them all, which is not unreasonable. We should fail cleanly when a non-GTFS zip is uploaded and tell the user.

I'm also getting errors like
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73193: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73194: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73195: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73225: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73226: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73227: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73257: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73258: Number 10.0 outside of acceptable range [0.0,3.0].
07:56:57.730 [Thread-18] INFO  com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 73259: Number 10.0 outside of acceptable range [0.0,3.0].

that I'm only even aware of if I sift through the server logs.

Related report from @abyrd:
AnalysisServerException L42 (GraphQL exception) includes only a message and not a stacktrace

Related report from @trevorgerhardt:
While processing GTFS feeds we do not show GTFS errors found to the user. GTFS errors have four priorities:
    // Something that is likely to break routing results, e.g. stop times out of sequence or high-speed travel
    HIGH,
    // Something that is likely to break display, but still give accurate routing results, e.g. broken shapes or route long name containing route short name.
    MEDIUM,
    // Something that will not affect user experience but should be corrected as time permits, e.g. unused stops.
    LOW,
    // An error for which we do not have a priority
    UNKNOWN

There can be millions of errors for a given GTFS feed, therefore it's not feasible to save and show them all. Ideally we could collect a summary of the errors and show the contents of 10 High or Medium priority messages alongside.
Something like:
This GTFS feed contains errors: 7 high priority, 13 medium, and 347.
High Priority Errors:
1.
...
7.
Medium Priority Errors:
1.
2.
3.
Please correct all high and medium priority errors before using this feed for analysis.

We should also provide stats on feed validity (e.g. service hours per day per mode), which would help users avoid choosing dates without intended service.
Another visualization example: http://tidytransit.r-transit.org/articles/servicepatterns.html

Yes, we need to make sure we return and display the full chain of exception causes (or the root cause at the end of the chain), not just the outermost one.


no connection headscale-ui and headscale

I run headscale, headscale-ui with the nginx proxy manager via docker. I can access the headscal ui but I can't reach headscale, a red X appears so I can't save the API key. SyntaxError: The string did not match the expected appears briefly. What did I miss configuring?

Nice to see that I'm not alone. I struggle with the same error pattern no matter what I try.

same setup and error here

I solved the problem in my case! To do this, I entered the following CORS headers in the corresponding location block for the API (/api). This is necessary to allow cross-origin requests from the UI to the API.
location /api {
add_header Access-Control-Allow-Origin *;
add_header Access-Control-Allow-Methods "GET, POST, OPTIONS";
add_header Access-Control-Allow-Headers "Authorization, Content-Type";
## Weiterleitung an die Headscale-API
proxy_pass http://headscale:8080/api;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

## Real-IP Einstellungen
set_real_ip_from <IP_ADDRESS>/16;
real_ip_header X-Forwarded-For;

## Timeout Einstellungen
send_timeout 5m;
proxy_read_timeout 240;
proxy_send_timeout 240;
proxy_connect_timeout 240;

}


loads optional deps only if required

This loads postcss and autoprefixer and CssUrlRewriter using System.import on runtime to speed up loading of sass/scss if autoprefixer/rewriteUrl is disabled.
Thanks for the great plugin. Let me know if I missed something.

Many thanks for the pull request 👍
Merged 😎