[KOGITO-94] - Introducing KogitoInfra CRD to install Infinispan

TODO:

[x] - Add integration with DataIndex
[x] - Add dependency with Infinispan Operator and test on an actual cluster having the OLM installing the operator for us
[x] - Updating CLI to not require infinispan URL anymore

--------- 8< --------------
See:
https://issues.jboss.org/browse/KOGITO-94
I'm opening this one as a draft so we can discuss the introduction of a new controller to the operator that will handle infrastructure installations for Kogito Services and the Data Index itself.
Motivation
The goal of this CR is to install Infinispan/Kafka/DataIndex/Keycloak by creating CRs for those operators, this way we wouldn't handle the installations itself, just delegate them to others. The idea is to not have a heavy burden to the operator to carry, but at the same time ease the way for developers to have a complex infrastructure to support all Kogito use cases in the cloud.
Scope
(Beginning simple!)
For this first use case, we're going to leverage from OLM  to install the Infinispan Operator for us during the Kogito Operator install via the dependencies mechanism.
After having the Infinispan Operator installed in the same namespace, we will offer the KogitoInfra CR, which will create Infinispan CRs for us.
Only one KogitoInfra per namespace is allowed. Can be created manually via yaml/spec or created on demand by the Data Index or any other Kogito Service that requires persistence.
Won't be owned by any other CR, but by the operator. So the first Kogito Service in the namespace that requires persistence or the Data Index will create one if not exist, but won't delete it if it gets undeployed/uninstalled.
The KogitoInfra CR will only hold the basic spec for each infra actor. In this first use case we'll have something like this:
{
  "apiVersion": "app.kiegroup.org/v1alpha1",
  "kind": "KogitoInfra",
       "metadata": {
           "name": "kogito-infra"
        },
        "spec": {
           "installInfinispan": true
        },
        "status": {
          "infinispan": {
              "service": "kogito-infinispan:112222",
              "credentialSecret": "kogito-infinispan-secret",
              "name": "kogito-infinispan",
              "condition": {
                  "type": "Success",
                  "status": true,
                  "message": "Infinispan successfully deployed"
              }
          }
       }
}

We'll keep this CR with minimal information since replicating Infinispan CR here would end up being a anti-pattern. Users should fine tune the installation themselves to adequate each sceario, like increasing the replica numbers. The operator only need to know that a installation exists to tie it to other services in the namespace.
The information in the KogitoInfra CR is the minimum needed for the operator to inject into a service. In this example we have the Secret with credentials and the Service that will be injected as environment variables. See: https://issues.jboss.org/browse/KOGITO-309
Please review the pkg/apis/app/v1alpha1/kogitoinfra_types.go file to understand the structure and suggest changes.

@xiezhang7 could you do a quick review in the controller? I'm still working on it, but just to make sure that I'm using the utils correctly.
I should finish this one tomorrow to get prepared for 0.6.0 release.

@xiezhang7 could you do a quick review in the controller? I'm still working on it, but just to make sure that I'm using the utils correctly.
I should finish this one tomorrow to get prepared for 0.6.0 release.

It looks good. Thanks.

TODO:
[ ] - Add integration with DataIndex
[ ] - Add dependency with Infinispan Operator and test on an actual cluster having the OLM installing the operator for us
[ ] - Updating CLI to not require infinispan URL anymore

@radtriste @spolti @xiezhang7 please take a careful look into this, mainly in the README part.
Now we can leverage from the Infinispan Operator to deploy Infinispan server for Data Index for us.
I intend to run more tests tomorrow to get prepared to the release on Monday, I know that's too much, but please, help me here by reviewing it during the morning.
Many, many thanks.

Also you changed some versions in there. Is that on purpose ?
I didn't get it. If you meant the scripts to generate the CSV, yes I removed because the SDK was messing with the new CRD

Also you changed some versions in there. Is that on purpose ?

I didn't get it. If you meant the scripts to generate the CSV, yes I removed because the SDK was messing with the new CRD

For example, in "deploy/examples/data-index.yaml", you changed the version to 0.6.0-rc1

Also you changed some versions in there. Is that on purpose ?

I didn't get it. If you meant the scripts to generate the CSV, yes I removed because the SDK was messing with the new CRD

For example, in "deploy/examples/data-index.yaml", you changed the version to 0.6.0-rc1

Good point. This one was a leftover. I’ll fix it on Monday and any other that I have left.

@xiezhang7 @spolti many thanks guys, helped a lot


Set default-value for gds-token

Hi @fniephaus,
Thanks for your work on GraalVM and on this GitHub action.
I've found the following bug: Currently there is no default value set for 'gds-token'. When I try to use GraalVM CE in this GitHub-action I get the error message 'Bad credentials'. It's not possible to manually set the 'gds-token' to an empty string, because GitHub ignores this.
Solution: Set the default value to an empty string (as described in the readme).
Best wishes!

Hi @johannes-link,
Thanks for the PR. Could you please share a link to a build that is failing due to this? We are testing different setups and I haven't seen the error you've described.
It'd be great if you could sign the OCA, so that I can merge your contribution.

Is this still needed?

Hi @fniephaus, sorry for my late reply. I set up a GitHub action in the public GitHub and there it worked without any issues. In my real project I use GitHub Enterprise and self-hosted runners. There I get the 'bad credentials'-message. So I think there is a issue with our configurations and not with the GitHub action. We can close this PR. Thank you!

There I get the 'bad credentials'-message.

So do you still get this error in your GitHub Enterprise instance? If setting the default helps, I'm happy to fix this for you.

Hi @fniephaus, we have switched to the GraalVM Maven plugin. This works fine, so we don't need the change anymore. If nobody else has the same problem, I think it was an issue with our instance and not a general problem. But thanks for your support!

Ok, great. Glad to hear this is working for you! Also, with the new GraalVM for JDK 17/20 releases made available under the GFTC license, the gds-token option is fortunately no longer needed :)


pull request for e2e test at 11/11/2016 15:15:09

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

Details

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[feature] semantic research for small paragraph

https://www.kaggle.com/code/aisuko/in-document-search-cross-encoder

We already add sentence similarity score for the backend API https://github.com/SkywardAI/chat-backend/pull/151


Black rectangle at the bottom of Spatial plot by using transformed seurat Rds object?

Seurat Rds was converted by following the manual. Could anyone tell me how to remove the black region？

Try not to overlay your expression matrix on the image may work, at least it works for me. This is possibly due to the wrong coordinate or size of the raw image and expression matrix.

Hello, could you please send me the running code for review?


Draft of FAQ answers

First draft of FAQ answers.
@dongbohu would you mind taking a look at the TODO under How can I find out what versions of software/packages were used to process the data? and leaving your thoughts? Thanks!

Yep, thanks @dongbohu I've filed #27

I'm going to merge this as is. If we want to make tweaks to the API processor list bit, we can address that at the same time as #27.


TYP: add type hints to library functions

Add type hints to library functions. These are all fairly concrete and not (yet) ready for full static type checker support, which will probably require some thought about dealing with numpy and scalar types.
The PR is still work in progress:

[X] fields.py
[X] galaxies.py
[X] lensing.py
[X] matter.py
[X] observations.py
[ ] points.py
[ ] shapes.py
[ ] user.py

Merging this even though some modules are still missing, so that other things can happen in the meantime.


Bug in masking of output in K.rnn(..., unroll=False) (for tensorflow and cntk)

Summary
Outputs are not masked correctly in tensorflow_backend.rnn(..., unroll=False). The issue is that states[0] is assumed to be equal to the output of the step_function in this line (not so in other backends or with unroll=True). This holds for the built-in RNNCells, which is the reason the bug has gone undetected. Especially since the introduction of output_size in the RNNCell it is clear that this should not generally be assumed.
Implications

RNN returns the wrong output when mask is used and the output is not equal to states[0] but has same size - i.e. a quiet error:

class Cell(keras.layers.Layer):

    def __init__(self):
        self.state_size = None
        self.output_size = None
        super(Cell, self).__init__()

    def build(self, input_shape):
        self.state_size = input_shape[-1]
        self.output_size = input_shape[-1]

    def call(self, inputs, states):
        return inputs, [s + 1 for s in states]

x = Input((3, 1), name="x")
x_masked = Masking()(x)
s_0 = Input((1,), name="s_0")
y, s = recurrent.RNN(Cell(),
                     return_state=True,
                     unroll=False)(x_masked, initial_state=s_0)
model = Model([x, s_0], [y, s])
model.compile(optimizer='sgd', loss='mse')

# last time step masked
x_arr = np.array([[[1.],[2.],[0.]]])
s_0_arr = np.array([[10.]])
y_arr, s_arr = model.predict([x_arr, s_0_arr])

# 1 is added to initial state two times
assert_allclose(s_arr, s_0_arr + 2)
# expect last output to be the same as last output before masking
assert_allclose(y_arr, x_arr[:, 1, :])  # Fails!

Gives:
       AssertionError:
       Not equal to tolerance rtol=1e-07, atol=0

       (mismatch 100.0%)
        x: array([12.], dtype=float32)
        y: array([2.])

Exception is raised when trying to apply an RNN with a cell which output_size != state_size[0]

class Cell(keras.layers.Layer):

    def __init__(self):
        self.state_size = None
        self.output_size = None
        super(Cell, self).__init__()

    def build(self, input_shape):
        self.state_size = input_shape[-1]
        self.output_size = input_shape[-1] * 2

    def call(self, inputs, states):
        return keras.layers.concatenate([inputs]*2), [s + 1 for s in states]

x = Input((3, 1), name="x")
x_masked = Masking()(x)
s_0 = Input((1,), name="s_0")
y, s = recurrent.RNN(Cell(),
                     return_state=True,
                     unroll=False)(x_masked, initial_state=s_0)  # Fails!

Gives:
ValueError: Dimension 1 in both shapes must be equal, but are 2 and 1. Shapes are [?,2] and [?,1]. for 'rnn_1/while/Select' (op: 'Select') with input shapes: [?,?], [?,2], [?,1].

Hi @qlzh727, since you came with the (great 👍) addition of explicit output_size, you might want to be aware of this issue.

Is this only considered a "minor" bug @gabrieldemarmiesse? It quietly breaks the promise of the RNN api (even if it is not that well documented) and gives inconsistent behaviour depending on backend and unroll True/False. This can cause a lot of confusion.

As of @souptc's comment here:

My understanding is that cntk does not support multiple input/output variables, so we can't handle the case that input and state has different shape. The current "RNN" api implementation in cntk_backend is kind of compromise to Keras api, base on the assumption that the last states is the output.

I understand that we can't (there is not ambition to) solve this for cntk? But it shouldn't return the wrong output quietly?

Thanks for the notice. If the cause of the issue is that customized cell does not hold the contract of state_size[0] == output_size, shouldn't the fix be applied to those cells? Maybe a check of state_size[0] == output_size could be added before the return of call() so that the error message could be more explicit.
Also adding @fchollet here who did the original review of my PR.

First of all, I don't see any reason to enforce that

I mean, why should cell/step function return outputs AND states at all if output should always be equal to states[0]?

Maybe a check of state_size[0] == output_size could be added before the return of call() so that the error message could be more explicit

Note that is not only a problem with shapes, as stated in implication 1) it is also about getting the wrong output when shapes are equal - at least different output with unroll=True/False (so one must be wrong).

Sorry for the confusion. There are three items in the scope right now.
a. cell.state_size[0]
b. cell.output_size
c. Actual size for the returned tensor from the cell.call() method.
The original contract was a == c, and I introduced b in my PR to make it more explicit. We want to have a == b == c. We would prefer strict b == c, and if b is not defined, then at least a == c should hold. For any of the cell that does not even hold a == c, then those cell should be updated. We could add an more explicit error message to make this contract more clear in the call() method.
Hope this clarify my previous message.

I think the original reason to have state_size[0] == output_size is because of LSTM which has two states. For historical reason, I think most of the RNN implementation has its state same as the output, but its not necessary always True for all the cells. This is also why I added cell.output_size to make the output shape more explicit.
In the case that unroll = True/False, they should return same shape of output, regardless of the state_size/output_size.

The output content should be same given the same input/initial state when unroll = True and False. unroll flag only control how the graph is generated (eg, tf.while loop) and should not have behavior difference. If the output returns different content when toggle the unroll flag, then there must be some bug on either code path.

Thanks for the clarification @qlzh727, however, I'm still not sure about your position, these statements are conflicting to me:

We want to have a == b == c. We would prefer strict b == c, and if b is not defined, then at least a == c should hold. For any of the cell that does not even hold a == c, then those cell should be updated

I think most of the RNN implementation has its state same as the output, but its not necessary always True for all the cells. This is also why I added cell.output_size to make the output shape more explicit.

Summary:

Right now it is required that output == states[0] for output, states = cell.call(inputs, states_tm1) when using layers.RNN for masking to work accurately/consistently.

This "contract" is not stated in docs of layers.RNN. In docs of cell.state_size it says that "(which should be the same as the size of the cell output)" but nothing else.

Right now it is required that output == states[0] for output, states = step_function(inputs, states_tm1) when using K.rnn for masking to work accurately/consistently.

Here, the "contract"  is stated in the docs of K.rnn: "The first state in the list must be the output tensor at the previous timestep" - So strictly speaking there is not a bug in K.rnn but rather in layers.RNN or just missing docs.
My question is: why do we need/want this contract? And why then even let the cell/step_function return output separately so that it is possible to break the contract? @fchollet.
If it is absolutely certain that we want this contract, then this issue can be closed with an update of documentation of layers.RNN (and prefereably some better verification and error messages) and some clarification of what cell.output_size is used for and why it can differ from cell.state_size[0].

Ah, I see your point.
My intention for introduce b) is to replace the contract of a == c, so that state size can be different from output size. For any existing cell without cell.output_size defined, a == c should still hold. cell.state_size[0] is just a fallback for any existing custom cell without cell.output_size.

This "contract" is not stated in docs of layers.RNN. In docs of cell.state_size it says that "(which should be the same as the size of the cell output)" but nothing else.

I think it might just be a doc issue but the contract should still holds. Otherwise the method for compute_output_shape probably won't be able to compute the correct shape for output tensor. In your proposal (no contract), what will be logic for compute_output_shape?

Please note that I'm not only/primarily talking about the contract of shapes, I'm talking about the actual values of the tensors, please read "Implication 1" in my orginal description of this issue. If the shape of output  is allowed to differ from shape of states[0] where output, states = cell.call(inputs, states_tm1) then it follows that there cannot be a contract saying that output == states[0] (again, including actual values) - in which case there is a bug when using masks in layers.RNN.

In your proposal (no contract), what will be logic for compute_output_shape?

I suggest no changes in how output shape is computed. I.e. cell.output_size is by default cell.state_size[0] if hasattr(self.cell.state_size, '__len__') else cell.state_size if the cell.output_size is not implemented. This is not what I refer to as the "contract" this is just the reasonable "special case" to deal with backward compatibility.
The reason I really appriciate your addition of output_size is that there is no longer anything blocking us from removing the artifificial constraint (what I refer to as the "contract") that output == states[0]

Ah, Thank for the detail explanation.
I think the line in question for implication 1 is at https://github.com/keras-team/keras/blob/master/keras/backend/tensorflow_backend.py#L3055.
It uses the value from state[0] to mask the output, whereas in the unroll == True, it uses the output from previous timestep or zeros for timestep0.
I think the proper fix is to read the output_ta_t for previous timestep or generate zeros instead of use state[0] if timestep is 0.

Yes, thanks, that was the fix I made in the PR I opened together with this issue ;)


Unexpected Automatic Date Conversion

Issue Overview
One of our systems which uses sheetjs for parsing csv/xlsx data produced an unexpected result this week. The data which caused the issue was "Mayslanding, NJ 08234", which the library coerced to a date as { t: 'n', v: 2313568, w: '5/1/34' }. My hunch is that the code which attempts to guess if a cell might be a date is too lax given some inputs.
This happens when raw: false, which is the default. We have worked around it by simply setting raw: true and doing our own type conversions. Still, this unexpected result seems likely to trip up others using the default setting.
I am willing to volunteer to submit a PR to fix this issue but would like some feedback from a maintainer before writing any code. My guess is that any changes to this date conversion code could very easily cause other unexpected side effects, so advice would be appreciated.
Steps to Reproduce:
Given the (somewhat fabricated) csv data below, the following code will produce the result:
const wkbk = xlsx.readFile('./datesville.csv')

CSV input:
Line 1
"Januaryville, NJ 08234"
"Februaryville, NJ 08234"
"Marchville, NJ 08234"
"Aprilville, NJ 08234"
"Mayslanding, NJ 08234"
"Juneville, NJ 08234"
"Julyville, NJ 08234"
"Augustville, NJ 08234"
"Septemberville, NJ 08234"
"Octoberville, NJ 08234"
"Novemberlanding, NJ 08330"
"Decembertown, NJ 08330"

Unexpected result:
{
  A1: { t: 's', v: 'Line 1' },
  A2: { t: 'n', v: 2313448, w: '1/1/34' },
  A3: { t: 'n', v: 2313479, w: '2/1/34' },
  A4: { t: 'n', v: 2313507, w: '3/1/34' },
  A5: { t: 'n', v: 2313538, w: '4/1/34' },
  A6: { t: 'n', v: 2313568, w: '5/1/34' },
  A7: { t: 'n', v: 2313599, w: '6/1/34' },
  A8: { t: 'n', v: 2313629, w: '7/1/34' },
  A9: { t: 'n', v: 2313660, w: '8/1/34' },
  A10: { t: 'n', v: 2313691, w: '9/1/34' },
  A11: { t: 'n', v: 2313721, w: '10/1/34' },
  A12: { t: 'n', v: 2348815, w: '11/1/30' },
  A13: { t: 'n', v: 2348845, w: '12/1/30' },
  '!ref': 'A1:A13'
}

To be clear, this ultimately boils down to how V8 (Chrome/Node) handles dates:
new Date("Mayslanding, NJ 08234"); // Thu May 01 8234 00:00:00 in your timezone

We try to correct for it in fuzzydate by looking for one of the month labels.  I suspect the month name regex can be cleaned up a bit to explicitly match the short or long form for names, like
if(s.toLowerCase().match(/\b(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\b/)) return o;
if(s.toLowerCase().match(/\b(january|february|march|april|may|june|july|august|september|october|november|december)\b/)) return o;

I'll take a closer look at those and put together a PR when I get a chance.

I am also seeing this when creating an excel file in my project. In our case, the data is a string "1/$1.99" It gets turned into a date with some of the numbers but some randomness as well. 1/2/1999

@jbull328, just to confirm, you're seeing that behavior on creation/output of a document with that data? I am not seeing it on input in xlsx (with several column formats) or csv.

Correct @matthew-macgregor we did see that on output. I ended up abandoning using the module.

@jbull328 If this is showing up when you write a CSV, that's Excel's automatic conversion.  For example, consider the CSV
1/3,2/3,3/3

If you write that as plaintext to a file and open in Excel, it will interpret those as dates:

In any case that's an issue unrelated to this one.

I want to confirm that the date parsing strategy of sheetjs is a bit too aggressive for my taste.
Effectively I can only use raw reading mode for csv files, as random decimal numbers are interpreted as date, which is not foreseeable so not reliable.
There are dozens of issues on this topic.
If I switch on raw, this problem disappears, however everything is a string then. I have to parse numbers then.
Is there something in between? So a config that will parse numbers if possible, and leaves the rest as string?

@flaushi it's aggressive because V8 (chrome/node) is aggressive.  fuzzydate attempts some light validation.  Unfortunately this story probably ends with a hard-coded list of acceptable date formats, as discussed in https://github.com/SheetJS/sheetjs/issues/1300#issuecomment-530266048

Moving this to #1300

0.18.1 fixes this issue.  To verify in NodeJS:
$ for v in 0.17.4 0.17.5 0.18.0 0.18.1; do npm i "xlsx@$v"; node -pe 'var XLSX = require("xlsx"); [XLSX.version, XLSX.readFile("t.csv").Sheets.Sheet1.A1]'; done

[ '0.17.4', { t: 'n', v: 2313448, w: '1/1/34' } ]
[ '0.17.5', { t: 'n', v: 2313448, w: '1/1/34' } ]
[ '0.18.0', { t: 'n', v: 2313448, w: '1/1/34' } ]
[ '0.18.1', { t: 's', v: 'Januaryville, NJ 08234' } ]


can it be used on discord?

did you mean send it to discord?

Yes

did you want to making pull request?


Feature: Added ability to specify indentation for ordered and unordered lists.

Updated:

Gradle build tools (required to get it running in Android Studio 3.1).
Bintray Release plugin (to 0.8.1).
Gradle version from 3.1 ---> 4.1
Compile / Target SDK versions to 27.
compile ---> implementation.
google() repo (required for android gradle build tools).

Let me know if its absolutely necessary to support back to API 7. I figured enough people were at least above 14 to warrant the upgrade.

Awesome, released as 3.6


feat: add name underneath users logo

Motivation
Added name of user underneath logo on users page #1040
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
Ran yarn prettier, yarn lint and yarn test

@JoelMarcey Updated

Apologies @mptap we decided to go with another implementation. Thanks for your efforts 😄


terraform_1: 1.7.3 -> 1.7.4

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/hashicorp/terraform/releases.
meta.description for terraform_1 is: Tool for building, changing, and versioning infrastructure
meta.homepage for terraform_1 is: https://www.terraform.io/
meta.changelog for terraform_1 is: https://github.com/hashicorp/terraform/blob/v1.7.4/CHANGELOG.md
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.7.4 with grep in /nix/store/kys6596zkf0l4l8gs2nmkq1nqy7nh70p-terraform-1.7.4
found 1.7.4 in filename of file in /nix/store/kys6596zkf0l4l8gs2nmkq1nqy7nh70p-terraform-1.7.4

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
coder
terraform
terraform_1
terraform_plugins_test

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/kys6596zkf0l4l8gs2nmkq1nqy7nh70p-terraform-1.7.4 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A terraform_1 https://github.com/r-ryantm/nixpkgs/archive/e79968952a8b7047d2b52e61c2c34304efac8a5a.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/e79968952a8b7047d2b52e61c2c34304efac8a5a#terraform_1

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/kys6596zkf0l4l8gs2nmkq1nqy7nh70p-terraform-1.7.4
ls -la /nix/store/kys6596zkf0l4l8gs2nmkq1nqy7nh70p-terraform-1.7.4/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package failed to build:

coder

2 packages built:

terraform
terraform_plugins_test

Maintainer pings
cc @Chili-Man @babariviere @kalbasit @amaxine @timstott @zimbatm @zowoq @techknowlogick @qjoly for testing.

Duplicate of https://github.com/NixOS/nixpkgs/pull/290516


Specify a placeholder TTD

Allowing the successful exchange of the transition config values before the TTD is known allows users to setup their EL+CL today. This helps ensure the network is Ready For The Merge™ and helps with Testing The Merge™.
The consensus specs specify a placeholder TTD of<PHONE_NUMBER>37316195423570985008687907853269984665640564039457584007913129638912, which is equal to 2**256-2**10. This value is absurdly high, but not the full size of a uint256 such that we can still perform tests with greater-than-TTD values.
AFAIK, there is no equivalent placeholder value in the EL specs. Geth presently returns a TTD of null and Nethermind a value of 0.
For reasons specified earlier, I think CL and EL should agree on these placeholder values. Since a specification already exists for the CL (although it is somewhat implicit), I propose that we explicitly apply the same specification to both EL and CL.

I'm okay with this
@paulhauner do you see this as necessary given the mainnet timeline?

@paulhauner do you see this as necessary given the mainnet timeline?

I'm not sure about must-have, but I think it's worthwhile. Not having this in the specification has already proved itself to be a mistake; it prevented users from proactively preparing for the merge. Adding this to the spec doesn't seem immediately necessary for mainnet, but I see it as tidying a loose-end and and paying it forward to future testnets and other chains who want to follow the roads we've paved (e.g., gnosis).
 I certainly won't be heartbroken if it doesn't get merged, though. Definitely not dying on (or feeling even mildly uncomfortable on) this hill.


Add grpc_pollset_set to grpc_resolve_address

As grpc_resolve_address is an async interface, the caller should provide a grpc_pollset_set.
Part of #7771

Updated with #8795

@y-zeng could you update the documentation for the addition of the new pollset argument in src/core/ext/client_channel/resolver_registry.h?


Added Etcher

In case other maintainers stumble on this. I’m not being able to access the homepage, so I wasn’t able to verify the URL.

Also not accessible for me

http://www.etcher.io/ is working for me

https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.2/Etcher-darwin-x64.dmg is verified.
Please update the url comment as stated above and the cask is good to go.

Will fix this tomorrow!

Thank you for the contribution. It needed some fixes, so they were made in 9bf47be9f7e1fce80318aa7a2ba0b3af96591def. Your contribution is still included (and still credited to you), with the appropriate modifications. Please feel free to ask about any of the changes.

Seems like homepage 'https://etcher.io/' is still wrong?


Flyway parsing bug: unable to decrease block depth below 0 in version 6.4.2

Which version and edition of Flyway are you using?
6.4.2
If this is not the latest version, can you reproduce the issue with the latest one as well?
(Many bugs are fixed in newer releases and upgrading will often resolve the issue)
I have checked also version 6.5.1 and there still was this error
Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)
Java API
Which database are you using (type & version)?
Oracle 19
Which operating system are you using?
Linux and Windows
What did you do?
(Please include the content causing the issue, any relevant configuration settings, the SQL statement that failed (if relevant) and the command you ran.)
In v6.2.4 i was having an error "unable to decrease block depth below 0" when i try to execute script which contains create package, procedure and dbms_scheluder.create_job.
Example code:
CREATE OR REPLACE PACKAGE some_jobs AS
    PROCEDURE some_procedure;
END some_jobs;

CREATE OR REPLACE PACKAGE BODY some_jobs AS

    PROCEDURE some_procedure IS
      CURSOR C_TO_CLOSE
      IS
        --some select

    BEGIN
      FOR row IN C_TO_CLOSE
      LOOP
       --update
      END LOOP;
    END;
END;

BEGIN
 DBMS_SCHEDULER.create_job (
    job_name        => 'some_procedure_job',
    job_type        => 'STORED_PROCEDURE',
    job_action      => 'some_jobs.some_procedure',
    start_date      => SYSTIMESTAMP,
    repeat_interval => 'FREQ=DAILY; BYHOUR=1',
    end_date        => NULL,
    enabled         => TRUE,
    comments        => 'some job');
END;

Workaround from issue https://github.com/flyway/flyway/issues/2715 (downgrade to v6.2.0) works.
But same issue exists in version 6.5.2.

If you add a / after the end of the package and the package body, Flyway will parse them correctly again.
In Flyway 6.2.0 this parsed by accident, as packages were not being correctly separated out which would in some cases mean that the sql was invalid when executed.
Fixing that bug has meant that packages need to explicitly be deliminated by a / to be detected properly. If I'm not mistaken this is the same behaviour that SQL*Plus exhibits isnt it?

Thank you for very fast answer. Adding / helped. Now it works corretly.


ZMQ-operator

I tried to compile your custom-operator on my machine and get
Compiling user ops ...
make: Entering directory '/home/patwie/git/tensorpack/tensorpack/user_ops'
[dep] zmq_recv_op.cc ...
In file included from zmq_conn.h:8:0,
                 from zmq_recv_op.cc:10:
zmq.hpp:84:36: error: missing binary operator before token "("
 #if ZMQ_VERSION >= ZMQ_MAKE_VERSION(3, 3, 0)

Can you shortly comment, which zmq version do you use. I had to change
//#include <zmq.hpp> into
#include "zmq.hpp"

and use
https://github.com/zeromq/cppzmq
But still getting the error.

I'm using 4.2.1-1.  cppzmq only contains the cpp header but maybe the c header is incompatible.

It looks like my package manager automatically includes zmq.hpp for me. (https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/zeromq)
And none of the other packaging system seems to do this. Yay archlinux..

After the following commands
git clone https://github.com/zeromq/libzmq
cd libzmq
./configure --prefix=$HOME/zeromq
make
make install

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/zeromq/include/zmq.hpp
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$HOME/zeromq/lib/pkgconfig/
wget -O $HOME/zeromq/include/zmq.hpp https://raw.githubusercontent.com/zeromq/cppzmq/master/zmq.hpp

and explicitly adding
CXXFLAGS += -I/home/patwie/zeromq/include/ -L/home/patwie/zeromq/lib/ -lzmq

I get at least linking errors from tensorflow.

As long as you did export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$HOME/zeromq/lib/pkgconfig/, I suppose you don't need CXXFLAGS += -I/home/patwie/zeromq/include/ -L/home/patwie/zeromq/lib/ -lzmq.

I have tested it and I have to add it. I currently try to use bazel. Here I managed to let bazel download all dependencies automatically.

Could you confirm pkg-config --cflags --libs libzmq produce your CXXFLAGS after setting PKG_CONFIG_PATH?
Also, the Makefile at first doesn't include libzmq in LIBS. If that's the case you may need to pull.

You definitely need Gitter or Slack. I will try that.

https://gitter.im/tensorpack/dev try this

Just a reminder:
As already discussed this only works with the correct protobuf version. When installing TF from source it is easy to find the correct protobuf version in
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/workspace.bzl#L309
Then
wget https://github.com/google/protobuf/archive/2b7430d96aeff2bb624c8d52182ff5e4b9f7f18a.tar.gz
cd protobuf-2b7430d96aeff2bb624c8d52182ff5e4b9f7f18a

replace_by_sed() {   local regex="${1}";   shift;   if echo "${OSTYPE}" | grep -q darwin; then     sed -i '' -e "${regex}" "$@";   else     sed -i -e "${regex}" "$@";   fi; }
replace_by_sed 's#https://googlemock.googlecode.com/files/gmock-1.7.0.zip#http://download.tensorflow.org/deps/gmock-1.7.0.zip#'   "autogen.sh"

./autogen.sh
./configure --prefix=$HOME/protobuf

make -j 4
make install

export LD_LIBRARY_PATH=$HOME/protobuf/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$HOME/protobuf/lib/pkgconfig

I strongly argue against sending Tensor-Protos over ZeroMQ. I tried to create a C++ example, but the documentation of TensorFlow clearly says, that they want you to put custom C++ code in the TF directory. I vote for using msgpack directly with a custom dataformat to further reduce the overall requirements when sending from C++. This would remove the TensorFlow-dependency.

Which document is it? I thought the dynamic-loader design was just aimed to allow people to put ops anywhere.
There is still a small protobuf dependency (only used for data type and shape), and I don't like it either.

It has protobuf dependency. I'll see if TF is willing to export the symbols otherwise it'll be very hard to build.
Only dozens of bytes of protobuf is used, but it consumes about 1/4 of the runtime.

Since it's only data type and shape, I will definitely try a new format (probably a custom one to avoid more dependencies).

https://github.com/tensorflow/tensorflow/pull/8728
might be related

Seems a good reference!
It requires known shape of tensor? That's a big limitation

But I like the tf.decode part, such that you just need to send raw data. So it can directly send msg packed data.

But there is no existing serialization protocol for "list of tensors"

Maybe a better design would be to implement two ops:

zmq_receive
msg_decode

It looks better from the interface. But it'll introduce two copies. One from zmq to a string tensor, and one from the whole string tensor to decoded list of tensors.
If another queue is added after the decode, the latency of copy can be hidden.

We do most of our decoding in python (for now) but intend to migrate to using [tf.train.Example](https://www.tensorflow.org/api_docs/python/tf/train/Example) which support variable size tensors and are natively supported by tensorflow.
Let me know if you want to collaborate on this issue. Would be great to hear your requirements and use cases.

Thanks for the note. I've checked the Example proto, it supports variable size, but it doesn't contain the meta data inside (the size and dtype). So it's still not an ideal way to transfer "list of tensors".
Also we tend to move away from protobuf because they are slow, and causing build issues.

As TF does not export these symbols, I wrote a small tensor_msg class which can be serialized by msgpack:
https://github.com/PatWie/tf_zmq

Nice work! Looks like it's using msgpack to replace protobuf. I guess the next question is how to do the equivalent encode method in Python (so that we can send a DataFlow to TF through zmq).

I have though about that. But if you are working in python, then it is easier to directly generate tf.Tensors and just send them. Anyway, my latest attempt to create a write.py is here:
https://github.com/PatWie/tf_zmq/blob/bin/python/example.py#L9

Generate tf.Tensors and with what protocol to send them?

Ah, I see your point.
I was experimenting with pybind (seems easier than swig) and msgpack.
Then,one just needs the sbuffer from msgpack.

Alright, it seems to be quite easy to send and receive the tensor_msg format with native python.

As you already implemented the ZMQ operator --> close.

But it doesn't build smoothly..
I won't have time for this one in the near future.

Should I create an example with that zmq and polish the implementation to add it to TensorPack?

It would be nice to have this op! But recently I don't have to time to look at your implementation in details. I guess there are at least these points to clarify:

Make it support more tensor types? (including string)
How does Python side and C++ side agree on the serialization format they use? Maybe that's some msgpack internals (packb) but I'm not very familiar.
The performance? If it's still slower than queues, it would be necessary to at least understand what amount of time copy and deserialization contribute.

So the usual business: make it compile, make it link, make it work, make it correct, make it fast, make it flexible.
I will ping this thread, when I prepared something. But it will take sometime as it has low priority.

The only problem with the current zmq op seems to be protobuf, and it's easy to solve: just need to serialize shape + dtype somehow. Maybe it is easier to do this way.

I just removed protobuf dependency. The op should be at least runnable now but there is still work to do.


v14 - Fluent API or DTO Types?

@MrRoundRobin initially proposed a Fluent API for the client here.
Having a fluent API is nice and when implemented well, could instruct clients how to make the request properly without getting exception.
On the other hand, defining bot API request and respons types (DTOs), would help to implement the request in response to webhook feature. @Olfi01 has already made a PR for it here.
In both of the cases, we could offer some validations before sending the request as well (maybe by using DataAnnotations api).
Which way should we go to? Either/or or Both? We also need to ensure we won't end up with duplicating ourselves.

hi, don't know if it's already clear but "request in response to webhook" is also possible with the fluent api

That's fine. My concern is more about duplicating the field names when making requests. I'm not in favor of using the dictionary. A set of DTO classes with inheritance between them seems a better approach to me.
Maybe both features can go together but I'm not sure about how implementation might look like.

Matrix's API is very similar to Telegram. Here you can see how I tried my best to reuse code between DTOs.

That project is in its early days so it's not perfect.

https://github.com/pouladpld/PeykOn/tree/master/src/Matrix.NET.Client


GetUpdates fails multiple times with an UnsupportedContentTypeException

I run a bot on a server and I noticed that there are some reoccuring exceptions with the Polling. The Exception reads as follows:
ERROR - GetUpdates failed
akka.http.scaladsl.unmarshalling.Unmarshaller$UnsupportedContentTypeException: Unsupported Content-Type, supported: application/json
        at akka.http.scaladsl.unmarshalling.Unmarshaller$UnsupportedContentTypeException$.apply(Unmarshaller.scala:158)
        at akka.http.scaladsl.unmarshalling.Unmarshaller$EnhancedFromEntityUnmarshaller$.$anonfun$forContentTypes$3(Unmarshaller.scala:114)
        at akka.http.scaladsl.unmarshalling.Unmarshaller$$anon$1.apply(Unmarshaller.scala:58)
        at akka.http.scaladsl.unmarshalling.Unmarshaller.$anonfun$transform$3(Unmarshaller.scala:23)
        at akka.http.scaladsl.unmarshalling.Unmarshaller$$anon$1.apply(Unmarshaller.scala:58)
        at akka.http.scaladsl.unmarshalling.Unmarshal.to(Unmarshal.scala:25)
        at info.mukel.telegrambot4s.clients.AkkaClient.toApiResponse(AkkaClient.scala:38)
        at info.mukel.telegrambot4s.clients.AkkaClient.$anonfun$apply$2(AkkaClient.scala:50)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:304)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
        at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:140)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

The really bad part about this is not the uncaught exception, but that the error occurs multiple times afterwards. I checked the logs and apparently the error appeared 382 times in about 12 seconds, in my example between 2:32:25 and 2:32:37. To show the frequency, I'll add an excerpt of the logs:
2017-12-09 02:32:25,980 ERROR h.i.t.IWINewsBot [Polling.scala:39] GetUpdates failed
2017-12-09 02:32:26,006 ERROR h.i.t.IWINewsBot [Polling.scala:39] GetUpdates failed
2017-12-09 02:32:26,035 ERROR h.i.t.IWINewsBot [Polling.scala:39] GetUpdates failed
2017-12-09 02:32:26,060 ERROR h.i.t.IWINewsBot [Polling.scala:39] GetUpdates failed
2017-12-09 02:32:26,090 ERROR h.i.t.IWINewsBot [Polling.scala:39] GetUpdates failed
2017-12-09 02:32:26,113 ERROR h.i.t.IWINewsBot [Polling.scala:39] GetUpdates failed

Closing this.
Please re-open if needed


add forChild()

TranslateModule can be imported multiple times: once per lazily-loaded bundle.
That is why there are two ways to create the module: TranslateModule.forRoot and TranslateModule.forChild.
forRoot creates a module that contains all the directives, the given loader, and the TranslateService service itself.
forChild creates a module that does not include the TranslateService service but a new given loader for the module (useful for splitting translation files).
When registered at the root, the module should be used as follows
@NgModule({
  imports: [TranslateModule.forRoot(Loader)]
})
class MyNgModule {}

For submodules and lazy loaded submodules the module should be used as follows:
@NgModule({
  imports: [TranslateModule.forChild(Loader)]
})
class MyNgModule {}

Hi, could you explain what is the idea behind this PR ? Why is it needed, what does it offer ?

@ocombe oups, sorry I updated my comment

oh ok, so this allows you to have a loader for each of your modules, it might be a different type of loader if you want, and each loader has its own translations that it doesn't share with the other modules ?
I have to think about this, because it's a major change.
Allowing each module to load its own translations is one thing, but making all of them independent is another, I can already imagine people who will want those translations to be shared among module.

@ocombe that's it, this is the same idea than angular router, this allow lazy-loading...
how do you think moduled should share translations? only one big translation object merging new other from child modules (what about performance on large app) ?

I'm not saying it's a bad idea, it's just that I need to think about it before I merge this, because once it's in there is no going back ;-)

this could be improved a bit by adding a parent to each module, if there is no translation on it, try its upper parents...

Ok so I finally took the time to review this PR (sorry it took me so long).
At first I didn't get why you generated uids and why you moved a lot of things into the module loader. The reason is that we want "one" instance of the service, so that we keep one list of languages and translations, but we want different loaders. The loader being injected into the service, it couldn't work for sub-modules.
So you created a new service that gets registered into the main service, and when the pipe / directive need a translation, it gives the id to the main service so that it uses the local instance that uses the local loader.
But I don't like that because it complicates a lot the model, and it changes the signature of all the methods to add a new "uid" parameter.
I searched how we could achieve this modularity without changing the methods, here is what I think would be a better way to do it:

forRoot defines a service / loader / directive / pipe and store
this store is new, it will be used to share common stuff between each instance of the service
forChild defines a service / loader / directive / pipe, but no store, the store is unique to the root module, it will be injected in each instance of the service
since the service is local, it will use the loader loader, which means that the methods don't change in the pipe / directive and components that use the service
the service will call this shared store when it wants global stuff (the list of languages, translations, ...)
if you want totally independent modules that don't share the translations/languages/..., you can use forRoot in each independent module, it will create a local instance of the store

What do you think? It would be much simpler, don't you think?

I totally agree with @ocombe. I think that's the way to go.
One remark though is when you have the same keys and different module translation files. For example, if we have this translation file for the root
{
    "foo": "bar"
}

And for a feature module we have this translation file
{
    "foo": "hello world"
}

The feature module will overwrite the value that we already have in the store and the foo translation key will map to hello world in every other module.
So what if we did it like Angular's injector. Create a store per feature module. If someone asks for a translation, it will look for it at the module store where it lives in. If the translation key could not be found it will navigate one up and asks the parent store for the translation and so on. It's just an idea though and not sure if it can be easily implemented. Just wanted to share the idea :).

@ocombe why not using multiple instance of service if it's simpler... does service would generate their own id to buid a tree in store or would you like all keys to be merged in a flat object? @SamVerschueren this is exactly what i was thinking...

@ocombe maybe store could have an option mergeChild when active all translations are merged in the same object else every service translations are stored in a tree (if a key does not exist on child, search recursively in parent)

Ok so I tried to implement my method, and wrote a test for it but... it fails.
I had 2 different modules, each one importing TranslateModule with forChild, and tried to define "useStore" to false, but both modules shared the same TranslateService instance, even if forChild provided it twice.
I asked Rob Wormald why that was the case, and apparently even if you use different modules, the providers get flatten at bootstrap and the last "provide" wins.
Obviously that's not the case for lazy loaded modules because they are not defined at bootstrap. They get their own injectors, which means that if you provide a service there, it will be a new instance. But since everything runs inside the same angular application, the new injector inherits from the root injector, and you can still get parent instances (if you don't provide a new one in the lazy loaded module).
So the only way to have different loaders / services in different modules will be to lazy load them (or use the providers property in your components, which creates a new injector too).
It means that you can do all of that, it will just not be possible to have 2 different instances in 2 different modules that are not lazy loaded.

@ocombe
why would you like to use forChild without lazy loaded module?
do you mean it should call every loaders at the same time when asking for new language?
could you publish your code?
maybe we could use store instance in .forChild method to register the new loader.
my problem with lazy loaded module is to know who is the parent service instance.

You don't really have to use forChild if you don't use lazy loaded modules. You're right about the observables (on lang change), it's gonna be messy if the service is a new instance every time...

@ocombe so what's going on?

The store & forChild are available in the ngx-translate branch (along with a bunch of other things).
There is an "isolate" property that you can set to "true" if you want to isolate your service (it only works with lazy loaded modules, or if you use the "providers" property in a component).
I will publish a beta of the new ngx-translate branch soon.

New 6.0.0-beta.1 with forChild: https://github.com/ngx-translate/core/releases/tag/6.0.0-beta.1
Be careful it is based on the new ngx-translate branch, and a lot of things have changed.

@ocombe after some test with the 6.0.0-beta.1 i found some problems with this implementation:

as there is one instance of translate service by module, we need to reconfigure it in all modules (with use, langs,...), we could move this values in store to fix this.
with isole: false once a lang has been loaded by an other module, the current module can't load his own translations
with isole: true, as there is one instance of store by module, we can't fix first problem and this is not possible to use parent module translations when no translation is defined for the key.(see below)

i give an sample of what we expect to do:
i have two module first have 2 translation key in 'en' lang:
{
  'title': 'welcome on my site',
  'section': 'home'
}

and the second lazy loaded by first one has only one key:
{
  'section': 'contact us'
}

each of them show on top page title and section so first on show:
welcome on my site
home

and second show:
welcome on my site
contact us

second module has no translation for title,so higher module's one is used
i think we should isolate module translations in same store instance

Creating a sample demo with Angular CLI now and I noticed the same thing @atiertant already described. In a feature module where you configure TranslateModule.forChild with a custom loader, you again have to call .use() and configure the service. I think all the different service should actually share the same TranslateConfig or something. A quick example of that could be something like this
@Injectable()
class TranslateConfig {

    lang$ = new ReplaySubject<string>();

    use(lang: string) {
        this.lang$.next(lang);
    }
}

That TranslateConfig is only created in the forRoot call so that it is shared between all the TranslateService instances. They can then subscribe to changes on the config object like the lang$ observable. If a service changes the language, all the other service will also respond and load the correct language.
Just a quick idea :)

Hi,
I need this functionality for my devs, do you know when it will be available in npm?
Can I directly point my package.json to this repo?
Thanks for your help

It's available in 6.0.0

Thanks for the quick answer :). I found some documentation in the README, but do you have a working example with a CustomLoader doing an http get?

Not really but you can take a look at the http loader source code: https://github.com/ngx-translate/http-loader/blob/master/src/http-loader.ts

I still have issue... I got a error caused by: No provider for TranslateStore!
My current set up is:

in my module:
TranslateModule.forChild({     loader: {       provide: TranslateLoader,       useClass: CustomLoader,       deps: [Http]     }, isolate: true   }),
my loader:
export class CustomLoader implements TranslateLoader { constructor(private http: Http) { } getTranslation(lang: string): Observable<any> {   return this.http.get('...?language='+lang)     .map((res: Response) => res.json()); } }
in my main component in my module:
 constructor(translate: TranslateService) {       translate.setDefaultLang('en');       translate.use('en');   }
I am missing something ? Should I do a TranslateModule.forRoot() at the root level ?
Many thanks for your help

@ocombe, I tried different set up, but I never saw the TranslateService using the CustomLoader set in the Translate.forChild. Is the TranslateService managed the different loaders? for me it looks like it always use the one set in the forRoot.
Many Thanks
Coco

@ocombe is forChild still supposed to work in 8.0.0? I tried to use it with a different loader than the one in forRoot, and using isolate: true and isolate: false, but it doesn't load the translations at all. I think the idea of having children load their own translations but listening to lang change on the parent is crucial when using modules


fix: return undefined instead of null for non existing keys

🔗 Linked issue
Resolves #205
❓ Type of change

[ ] 📖 Documentation (updates to the documentation, readme, or JSdoc annotations)
[x] 🐞 Bug fix (a non-breaking change that fixes an issue)
[x] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
This is a rework of #206
By changes to this PR, storage returns undefined instead of null when a key does not exist.
On the downside, this change makes unstorage less consistent with local storage and Cloudflare KV when getItem returns null when the key does not exist.
On the benefits, more thinking this is less a breaking change for users but actually a fix since we were already listing null as a valid Storage value (which does exist!). And we reduce use of evil null. (https://github.com/sindresorhus/meta/discussions/7)
📝 Checklist

[ ] I have linked an issue or discussion.
[ ] I have updated the documentation accordingly.

PR stalled. Let's track via #205


Add un-publish buttons

It would be useful to add un-publish buttons to documents and also in the Browse view. The Helix Sidekick has this and it's visible on the published page view.

Added this to: https://github.com/adobe/da-live/issues/227


GPL-517 Use 'LHR Stock' not 'Stock Plate'

User story
As a Sample Management user, I would like the plates created by the lighthouse service to come into Sequencescape as purpose 'LHR Stock', not 'Stock Plate', to keep them consistent with extracts coming in from CGaP (see GPL-502 - https://github.com/sanger/wrangler/issues/56)
Who are the primary contacts for this story
Danni
Acceptance criteria

Lighthouse service creates plates in Sequencescape with purpose 'LHR Stock'

Closing this as it's in the Done column


Slack logs - hide slugs for thumbnails

A little noisy in the slack thread when thumbnails are shown.  Just show the main slug.
CA-Drought-Update-10-25-21-pdf-116x150,
CA-Drought-Update-10-25-21-pdf-232x300,
CA-Drought-Update-10-25-21-pdf-386x500,
CA-Drought-Update-10-25-21-pdf,
CA-Drought-Update-10-25-21

...should just be...
CA-Drought-Update-10-25-21
continue to list all the files in the replies.

Done...https://github.com/cagov/wordpress-to-github/pull/97


QOI settings is bad translated

Microsoft PowerToys version
0.77
Utility with translation issue
Settings
🌐 Language affected
Dutch
❌ Actual phrase(s)
"Afbeelding is redelijk ok"

✔ Expected phrase(s)
"Quite OK Image"
ℹ Why is the current translation wrong
It's a quite literal translation of a file type name. Which is strange by itself, but...
  <data name="FileExplorerPreview_ToggleSwitch_Thumbnail_QOI.Header" xml:space="preserve">
    <value>Quite OK Image</value>
    <comment>File type, do not translate</comment>
  </data>

[!NOTE]
This applies to both FileExplorerPreview_ToggleSwitch_Preview_QOI and FileExplorerPreview_ToggleSwitch_Thumbnail_QOI

@crutkas @niels9001 if strings like these are explicitly forbidden to be altered, maybe we could hard-code these?

We need to use the lock command.

↳ ref #31036

Hi, this has been worked on for the 0.78 development cycle. 😄
Please update PowerToys to the latest: https://github.com/microsoft/PowerToys/releases


Support for version in Amazon API and filenames

Currently filenames are coded only with a timestamp...  ideally there would be provisions for creating the filename based on a version string.  The same input could then in turn be used to set the version as per the AWS Lambda API:
https://docs.aws.amazon.com/lambda/latest/dg/versioning-intro.html
This could be implemented by adding a "version: x.x.x" line to the config.yaml file, if present then the version provided will be used, otherwise the code will exhibit the current behavior for backward compatibility. Another option would be to add a '--version=x.x.x' command line parameter to 'lambda build' and the pending S3 change 'lambda upload'.
It would break 100% compatibility with current outputs, but it would also likely be more intuitive to have the filenames generated be in the format -<version|timestamp>.zip rather than the current -.zip.

Tagging my other non-work account @kilna

Also, bears mentioning... looking at the codebase, I can do this. Just curious if others have the same need.

Me too

Yes, I think this would be a very useful feature. I was thinking about submitting a patch for this along the same lines, so if you're doing it, I'll definitely support and use it.


Bugfix: Do not inject prototype scoped objects

Injecting prototype scoped objects prevents deserialization and thus
caching of a form definition (which is helpful with larger node based
forms).
Closes #94

It would be greate if this change could be merged.


use an artifact for the documentation

To avoid rebuilding it 5 times

no need it's only build once


Telegram summary improvement

The summary alert sent to telegram needs improvement

summary/detail option
group by algo
total hash

Added summary option by status
Summary by algo can be added later if anyone requests it
Telegram is low priority now that web site is functional. Resources are better spent improving the summary display on the web site.


[backport] Import numpy before Cython

Backport of #5482 by @emcastillo

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit 68d9bc7306f686951087411e702dff8bc6811ffb, target branch v9) succeeded!


Question: Best way to find connections between clusters

We would like to find the relations between the clusters produced with the default parameters. Basically some kind of clustering of the clusters (or hierarchical clustering). Is there a way to do that using mmseqs? (Other than running clustering again with higher sensitivity or taking the representative of each cluster and clustering aligning them.)
Thanks

We don't have a good way to produce a tree from a cascaded clustering currently.
The default cascaded clustering workflow produces exactly that however, if you look into the tmp folder you will find the clustering results of previous steps, there you can find whatever representatives from the previous iterations and reconstruct a tree based on these.

Can you elaborate a little bit more? do I look at the files: aln_step1...? what should I look for?

MMseqs uses internally a cascaded workflow. This means it will first cluster with a low sensitivity and get step by step higher ( see https://github.com/soedinglab/MMseqs2/wiki#cascaded-clustering). But there is also a way to recluster clustered databases again to create a hierarchicy. Please check out the guide https://github.com/soedinglab/MMseqs2/wiki#how-to-manually-cascade-cluster
You could also use profiles to cluster more senstive. See issue https://github.com/soedinglab/MMseqs2/issues/66.

I have clustering results and I would like to see if any of the clusters are related, so I guess adding another step using the manual cascade clustering is what I need.
So 2 questions about this process:

Do I need to add a result2repseq step before createsubdb? (I think it is missing in the wiki link that you attached)
If the first step was done using mmseqs cluster with default parameters, do I need to change the sensitivity for the manual step? (should I increase or reduce it?)

Alternatively, is it possible to create a profile for each cluster and then cluster the profiles (like in issue #66)?

1.) createsubdb is enough. No need to use result2repseq
2.) I would search all sequence all-against-all to find the relatedness.
# first clustering run
mmseqs cluster sequenceDB clu1 tmp1
# create a subset of the sequenceDB only with representative sequences
mmseqs createsubdb clu1 sequenceDB cluSequenceDB
# search representative all against all
mmseqs search cluSequenceDB cluSequenceDB cluSequenceDB_aln tmp

3.)  You can turn a clustering result into a profile and search with them profile-again-consensus
# first clustering run
mmseqs cluster sequenceDB clu1 tmp1
# create a subset of the sequenceDB only with representative sequences
mmseqs result2profile sequenceDB sequenceDB profileDB
mmseqs createsubdb clu1 profileDB cluProfileDB
mmseqs createsubdb clu1 profileDB_consensus cluProfileDB_consensus
# search representative all against all
mmseqs search cluProfileDB cluProfileDB_consensus cluProfileDB_aln tmp --add-self-matches

Thanks Martin.
Few issues regarding the profile option:

The result2profile command requires 4 DBs, I'm assuming clu1 is the missing one?
mmseqs result2profile sequenceDB sequenceDB clu1 profileDB
If I'm correct about the first issue - then the createsubdb steps are unnecessary?
A possible bug: the results2profile command resulted in 2 files: profileDB_tmp_0.dbtype and profileDB_tmp_0_consensus.dbtype which I had to rename manually in order to move forward (remove the _tmp_0)

MMseqs2 Version: 9f344f3d6765a1131f22794d9a6fd9fcd8ca2cdf

The _tmp_0 indicates that something went wrong (the merge was unsuccessful). Could you post the log?

Hi Milot,
This is the log:
MPI Init...
Rank: 0 Size: 1
Program call:
all_genomes.genes.db all_genomes.genes.db all_genomes.genes.clu4.db all_genomes.genes.clu4.profile.db

MMseqs Version:                    	9f344f3d6765a1131f22794d9a6fd9fcd8ca2cdf
Sub Matrix                         	blosum62.out
Profile e-value threshold          	0.001
Compositional bias                 	1
Use global sequence weighting      	false
Filter MSA                         	1
Maximum sequence identity threshold	0.9
Minimum seq. id.                   	0
Minimum score per column           	-20
Minimum coverage                   	0
Select n most diverse seqs         	1000
Pseudo count a                     	0
Pseudo count b                     	1.5
Omit Consensus                     	false
No preload                         	false
Early exit                         	false
Threads                            	56
Verbosity                          	3

Compute split from 0 to 6159847
Start computing profiles.
Query database type: Aminoacid
Target database type: Aminoacid
..............................................................................................................................................................	2 Mio. sequences processed
...	1 Mio. sequences processed
............................	4 Mio. sequences processed
............................................................	3 Mio. sequences processed
.....................................................................	5 Mio. sequences processed
.............................	6 Mio. sequences processed
......................................................................................................................................................................................................................................................................Time for merging files: 0 h 0 m 21 s Time for merging files: 0 h 13 m 43 s

Done.
Time for merging files: 0 h 0 m 1 s
Time for merging files: 0 h 0 m 1 s
Time for processing: 2 h 53 m 18s

@adimil do you run mmseqs using MPI? If not, please compile MMseqs without MPI. This might resolve the issue.

Is this issue resolved?

Hi Martin, I didn't try to run it again, but the results were fine.

Hey Adimil, thanks for the answer. I will close it. Please let me know if the issues occurs again.

I had this same issue when trying to use mmseqs2. Are you sure it's safe to simply rename these files?


Test bot commands

[ ] HELP
[ ] RELAYS LIST
[ ] RELAYS ADD
[ ] RELAYS DELETE
[ ] RELAYS CLEAR
[ ] CONDITIONS LIST
[ ] CONDITIONS ADD [--amount ][--requiredLength ] [--requiredPhrase ]
[ ] CONDITIONS UP
[ ] CONDITIONS DELETE
[ ] CONDITIONS CLEAR
[ ] PROFILE [--name ] [--picture ] [--banner ] [--description ]
[ ] ZAPMESSAGE
[ ] EVENT
[ ] CREDITS ADD
[ ] STATUS
[ ] SUPPORT

All functional and added more for BALANCE, ENABLE, DISABLE.
The messages coming back to the user can be confusing on some clients that ignore line returns, (web clients that dont render as a <br> or other tag to force appearing as a new line)


feat: Add functionality to enforce strong password for form signups

Pull Request Template
Use this template to quickly create a well written pull request. Delete all quotes before creating the pull request.

Description

Add a TL;DR when description is extra long (helps content team)
Please include a summary of the changes and which issue has been fixed. Please also include relevant motivation
and context. List any dependencies that are required for this change
Links to Notion, Figma or any other documents that might be relevant to the PR

PR fixes following issue(s)
Fixes # (issue number)

if no issue exists, please create an issue and ask the maintainers about this first

Media

A video or a GIF is preferred. when using Loom, don’t embed because it looks like it’s a GIF. instead, just link to the video

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Chore (housekeeping or task changes that don't impact user perception)
This change requires a documentation update

Testing

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Also list any relevant details for your test configuration.
Delete anything that is not relevant

[ ] Manual
[ ] JUnit
[ ] Jest
[ ] Cypress

Test Plan

Add Testsmith test cases links that relate to this PR

Issues raised during DP testing

Link issues raised during DP testing for better visiblity and tracking (copy link from comments dropped on this PR)

Checklist:
Dev activity

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Speedbreak features have been covered
[ ] Test plan covers all impacted features and areas of interest
[ ] Test plan has been peer reviewed by project stakeholders and other QA members
[ ] Manually tested functionality on DP
[ ] We had an implementation alignment call with stakeholders post QA Round 2
[ ] Cypress test cases have been added and approved by SDET/manual QA
[ ] Added Test Plan Approved label after Cypress tests were reviewed
[ ] Added Test Plan Approved label after JUnit tests were reviewed

Summary by CodeRabbit

New Features

Introduced a strong password policy feature that can be enabled or disabled per tenant.

Bug Fixes

Added a new error type and code INSUFFICIENT_PASSWORD_STRENGTH for handling cases of insufficient password strength, improving user feedback on password requirements.

Refactor

Updated password validation logic to enforce stronger password criteria when the strong password policy is enabled.
Refactored user creation and password reset flows to consider tenant-specific password policies, enhancing security measures.

/ok-to-test<EMAIL_ADDRESS>

/ok-to-test<EMAIL_ADDRESS>

/ok-to-test<EMAIL_ADDRESS>


Reconstruct both the left and right hands

Have you taken into account the scenario involving both hands? If I aim to reconstruct both the left and right hands simultaneously, would I require two Token Generator models? Alternatively, should I directly mirror the feature map of the Token Generator, or is it necessary to mirror the input before feeding it into the Token Generator model?
Thank you.

This method can indeed be extended to scenarios involving both hands, but it requires training a new model since there is very little data on dual-hand interactions in FreiHAND. Considering that a single image may contain two hands, each image in the point sampling stage needs to select 42 points. These 42 points can come from the same feature map, or separate feature maps can be set up for the left and right hands.
It is important to note that during the mesh regressor stage, the output of joints and vertices needs to be changed to 21x2 and 778x2, respectively, to simultaneously output both hands. If you use the parameters of both hands for supervision, there is no need to mirror the feature map. However, if you still want to use only the right hand's parameters, you will need to model the left hand using the mirrored feature map.
BTW, have you managed to run the training code successfully?

Thank you for your response. I haven't debugged the code yet.

你好，请问可以详细解释一下吗，如果需要同时重建左右手的话，需要用新的数据集训练吗，因为您提到了freihand中关于双手交互的场景很少；还有在只有一只手参数的情况下，如何使用镜像的特征图对左手进行建模,谢谢
Hello!!Can you explain in detail, if I need to reconstruct the left and right hands at the same time, do I need to train with a new dataset, because you mentioned that there are very few scenes of two-hand interaction in Freihand;And how to model the left hand using a mirrored feature map when there are only one hand parameters，thanks


No resoutce status is displayed if delete cloud resources in the console

Version: main
UI: dev-6168160
Commit ID: 2123c63e14cb8461eb00fc5a594999b5bc82483d
Env: MacOS 13.1, Apple M2
Test Steps:

Deploy an aliyun rds instance github.com/Finenyaco/modules_test//alicloud/aliyun-rds.
Delete the rds in aliyun console.
Check the status of alicloud_db_instance resource.

Test Result:

The status of alicloud_db_instance is empty.

Expected Result:

Display something like ResourceNotFound.

Environment

Walrus version: v0.5.1

Test Result:
The component's status does not update after deleting the resource in the console.


Operation aborted error on mobile Chrome / iOS (Flutter Web)

Scanning works fine in apps and web ran on desktop, but on mobile iOS Chrome it does not ask for a permission and shows black screen. Going to the dev console on the iPhone it shows this error:
LOG MobileScannerWeb: AbortError: The operation was aborted.
The app was built with Canvaskit. Any way to solve?

Can you please try again with version 3.0.0?


[infra] Lit logo image that works in dark mode / on other backgrounds

Description
The Lit Logo is not visible properly in GitHub dark mode.
Steps to Reproduce

Go to GitHub.com/lit/lit
Change the GitHub theme to dark
See that the lit logo is not visible.

Expected Results
The logo should be both dark and light mode compatible

Actual Results
The logo is not visible

Browsers Affected

[x] Chrome
[ ] Firefox
[ ] Edge
[ ] Safari 11
[ ] Safari 10
[ ] IE 11

might be helpful to include some screenshots in the description to make it a bit easier to understand / see the issue.

Sure @thescientist13 .

Here it is :

Any updates on this @thescientist13 ?

I'm not really a graphic designer nor am I an owner of the Lit repo, so my ability to help is very limited other than to help triage and assist from the outside. 😅
That said, your screenshot is helpful as it it looks like it is the text aspect to the image that doesn't render properly in dark mode, naturally, given the text is dark.
I saw Adam Wathan (creator of Tailwind) had a similar question / challenge.  I'm not sure what he means by sticker, but effectively he added a white border to the text so that in light mode, it looks the same, but in dark mode, the text has a stroke around it to keep it from getting lost in dark mode.

Maybe that would be helpful for the Lit team to consdier?

Hey @justinfagnani I know a good friend of mine who could help us. Are you ok if he helps?

Hey @TarunavBA we have an agency that produces our assets for us. @graynorton is going to forward this to them.

GitHub supports light/dark image toggling now. Adding a dark theme variant in https://github.com/lit/lit/pull/2314.

See Ocupop guidance in #2314.

Hey @TarunavBA we have an agency that produces our assets for us. @graynorton is going to forward this to them.

Ok @justinfagnani

Fixed in https://github.com/lit/lit/pull/3457


[BUG] Suggestions inside Copilot Chat

I believe it to be a bug after updating the last version: NestJs code 'snippets' started to appear within Copilot Chat. After deactivating the extension, Chat behavior returns to normal.

Hi @felipfr ,
I understand that the report refers to a different extension, one of my snippet extensions for NestJS, maybe vscode-nestjs-snippets.
I have carefully reviewed your report and it is actually normal VSCode behavior. Apparently Copilot Chat loads all extension nippets that are defined for plain text (In my case the defined scope is plain text and typescript).
This is the first report I have received on this, I could certainly remove plain text support from my extension. However, that doesn't mean it can't happen again with another extension with the same scope.

Thanks for the answer.
I disabled this extension here: https://marketplace.visualstudio.com/items?itemName=imgildev.vscode-nestjs-generator
After doing this, it worked normally again, without any snippet appearing. I don't know what it could be, but if you want, I can record the screen to show you. Or if you know how to disable this behavior in Copilot, I would appreciate it.

thank you very much for your information
It is a totally unexpected effect, but apparently the snippets extension is totally dependent on this extension, disabling this one disables both, but you can disable or uninstall the snippets extension without affecting this extension.

Hello @felipfr ,
I am interested in knowing what decision you made to close the issue.
Did you disable or remove the snippets extension? Do you want to continue using that extension but need to remove the plain text scope?

Actually, I uninstalled the Prisma snippets extension and uninstalled the other one along with it.
What setting do I change in Copilot so I can continue using the extension? And if you want, you can close the issue.

Unfortunately copilot chat does not have a configuration option for that purpose. It is not something that has to do directly with the development of the extensions I have made but a behavior of Copilot Chat, but it does affect me as the creator.
In such a case, I am forced to remove the plain text scope from all my snippet extensions so I am asking for some time as there are 6 extensions just for NestJS with more than 1000 snippets in total.
I recommend you to transfer your questions to the GitHub Copilot issue tracker while I make changes to my extensions.
Translated with DeepL.com (free version)

Hi @felipfr ,
All snippet extensions have been updated to remove the plain text scope. You can use them without them interfering with Copilot Chat.
I would be very grateful if you can give a good rating to the extensions in the marketplace, and if you have any ideas to improve any of the extensions, I am open to comments.


code generator takes increasingly long time when using 'build_runner watch'

I use the 'watch' feature of build_runner to regenerate my dart code when input files change.
I have noticed that when changing drift input files, each regeneration takes an increasing amount of time.
For example, when first launching build_running, the generation takes about 5 seconds. Simply by touching the input file (adding a space), regen then takes 30 seconds, then 3 minutes...
My suspicion is that something in the internal cache is accumulating - causing unnecessarily repeated work to be performed.
== UPDATE ==
I've tracked it down to DriftBuildBackend.loadElementDeclaration.
 @override
  Future<AstNode?> loadElementDeclaration(Element element) {
    return _buildStep.resolver.astNodeFor(element, resolve: true);
  }

putting a stopwatch around this reveals on first execution, it takes about 100ms - then on the second run it takes 500ms - thats a 5x increase, I don't know why....
While debugging, I found a bug that made it harder to track down timing issues.
A missing await in resolver.dart at DriftResolver._restoreOrResolve
  Future<DriftElement> _restoreOrResolve(DriftElementId element) async {
    try {
      if (await driver.readStoredAnalysisResult(element.libraryUri) != null) {
        return await _deserializer.readDriftElement(element);
      }
    } on CouldNotDeserializeException catch (e, s) {
      driver.backend.log.fine('Could not deserialize $element', e, s);
    }

    // We can't resolve the element from cache, so we need to resolve it.
    final owningFile = driver.cache.stateForUri(element.libraryUri);
    await driver.discoverIfNecessary(owningFile);
    final discovered = owningFile.discovery!.locallyDefinedElements
        .firstWhere((e) => e.ownId == element);

    return _resolveDiscovered(discovered); // this should be awaited
  }

Here's the fix
  Future<DriftElement> _restoreOrResolve(DriftElementId element) async {
    try {
      if (await driver.readStoredAnalysisResult(element.libraryUri) != null) {
        return await _deserializer.readDriftElement(element);
      }
    } on CouldNotDeserializeException catch (e, s) {
      driver.backend.log.fine('Could not deserialize $element', e, s);
    }

    // We can't resolve the element from cache, so we need to resolve it.
    final owningFile = driver.cache.stateForUri(element.libraryUri);
    await driver.discoverIfNecessary(owningFile);
    final discovered = owningFile.discovery!.locallyDefinedElements
        .firstWhere((e) => e.ownId == element);

    return await _resolveDiscovered(discovered);
  }

Thanks for the detailed description and for the investigation. This does look like a build resolver issue (drift doesn't reuse its driver across build steps).


packspec

Hey, just wanted to let you know that we want to standardize semver in plugins and dependencies here: https://github.com/nvim-lua/nvim-package-specification

Yeah I saw that a few days after I made Pact.
If I do go down the route of dependency-chain-sniffing I would definitely reference that spec over writing my own.
(Maybe you can support a packspec.fnl format too :grin:)


ng-class is not working properly ie8 + document mode: IE7 standards after 1.2.0-rc.3

Assume we have a snippet:

Hello

and we have a meta on the page:
<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7" />

model preperties updates correctly, but nothing changes.
Demo: http://jsbin.com/IDexoga/5/edit?html,js
You could try to open it in IE8/9/10.
In RC2 it works fine.

+1

1.2.28 is unsupported, and IE8, too.


feat: get compiler settings

What I did
added get_compiler_settings to ape solidity
How I did it
with help from @unparalleled-js
How to verify it
Checklist

[ ] Passes all linting checks (pre-commit and CI jobs)
[ ] New test cases have been added and are passing
[ ] Documentation has been updated
[ ] PR title follows Conventional Commit standard (will be automatically included in the changelog)

needs tests

functional checks failing because https://github.com/ApeWorX/ape/pull/979 not merged yet.


Should we output introspection warnings to stderr when using prisma introspect --print?

Right now we only show the warnings in CLI output when doing prisma introspect (without --print)

This means for example having this at the end of the print
*** WARNING ***

These models do not have a unique identifier or id and are therefore commented out.
- "no_unique_identifier"

These fields were commented out because of invalid names. Please provide valid ones that match [a-zA-Z][a-zA-Z0-9_]*.
- Model: "column_name_that_becomes_empty_string"
  Field(s): "12345"
- Model: "column_name_that_becomes_empty_string2"
  Field(s): "1", "10", "11", "2", "3", "4", "5", "6", "7", "8", "9", "999999999999"

These fields were commented out because we currently do not support their types.
- Model: "no_unique_identifier" Field: "field1" Raw Datatype: "integer key"
- Model: "unsupported_type" Field: "unsupported" Raw Datatype: "binary(50)"


Exposing matrix implementation, automatic drawer recognition

I will wait a bit with my version of the renderer, there's still plenty of irrelevant code...

@Aiosa Most of this looks great. I'd like a couple small things to be tweaked and then I'll merge it.

Done, The last merge was just a conflict resolution.


Run opam-admin check to detect cycles

opam admin check --cycles [--ignore-test-doc] is super useful to detect issue with dependency cycles. That'd be useful to run it automatically on every PR to detect this kind of issue early.

I'm currently working on the lint phase so I can add this in there.


pay the iron price does not allow greyjoy player to place attachment on opponent's character

nt

Thanks for reporting. Which attachment have you tried it with?

The Martell icon removal ones. Like I couldn't put Attainted on his char. Maybe it's because those don't say opponent's character only.

To anyone tackling this issue: the likely problem is that the attachment.canAttach check happens while the card is still in the opponent's discard pile. This means that attachments that can only go on opponent's characters (like the Martell icon strippers) can only be attached to your own character, and probably, vice versa (like Venomous Blade only going on opponent's characters).


Relicense under dual MIT/Apache-2.0

OK, those tagged here have seen this before.  I need your consent to relicense as we've done in other Rust projects (see https://github.com/energymon/energy-monitor-rs/issues/2 for an example with more info).
Please state in a comment:
I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

@Manishearth
@vvuk
Thanks.

I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

Thank you, we're now re-licensed as of 67a34c4.


Support for active storage preview

Fixes #1

@tachyons thanks for the PR, it is nice that the previewer can resize the image. Let me test this.
But please note this doesn't fix #1 ("show image preview right after image is selected from local.")

Okay, I missed that. But since preview is to be generated using backend tool, previewing right after uploading  won't be easy

Also, in the case of an image this code is not reached. I am wondering if this code can just replace the conditional?

I tried it, But got following error
ActiveStorage::UnpreviewableError: ActiveStorage::UnpreviewableError

I see. It appears that previewable? only works with PDF and video files. For images we need #variable?. Let me do some more testing tonight before I merge your PR.


Compile error

I was compiling Openblas but it didn't work.
What should I do next?
C:\MinGW\bin\ar.exe: sgemm_kernel.obj: No such file or directory
make[1]: *** [libs] Error 1
make[1]: Leaving directory `/c/nopenblas/1/kernel'
make: *** [libs] Error 1

What's your CPU?

AMD A8 8600p

make clean
make TARGET=EXCAVATOR

Did you manage to get it working in the meantime with xianyi's suggestions, or do you need further assistance ?


Negative r, g or b values.

Why does this code:
ColorMix.setGradient([{ 	reference: 0, 	color: { red: 255, green: 0, blue: 0 } }, { 		reference: 26, 		color: { red: 255, green: 255, blue: 0 } 	}, 	{ 		reference: 50, 		color: { red: 150, green: 150, blue: 150 } 	}, 	{ 		reference: 75, 		color: { red: 41, green: 55, blue: 255 } 	}, 	{ 		reference: 99, 		color: { red: 0, green: 255, blue: 26 } 	}]) console.log(ColorMix.blend(4))
... result in:

{red: 259, green: 82, blue: -5}?

I seem to be able to generate a lot of these negative values. Is this expected? If so, how do I convert them to hex values?

Hi @iVoteSize, no, this is not expected!
Thanks for the report, I will try to commit a solution next week (at least a Math.min)


How to run a local llamaserver without using openaikey

How to run a local llamaserver without using openaikey

Right now it's not possible. But I agree that it shouldn't be this way. Let me work on it over the weekend.


Other way to get log file

Nice package!
Another way to get the log file could be:
Log::getMonolog()->getHandlers()[0]->getUrl();

Thanks for the suggestion, maybe I'll change that up when I have some more time.


add ekco v0.15.0 to automatically renew contour and envoy certificates

What type of PR is this?
type::feature
What this PR does / why we need it:
adds ekco v0.15.0 which includes the functionality for automatically renewing the contour and envoy certificates
Does this PR introduce a user-facing change?
Adds [EKCO](/docs/add-ons/ekco) version 0.15.0 which will handle automatically renewing the contour and envoy certificates.

Does this PR require documentation?
NONE

You also need to add the new version to the version.js file. I forgot this last time so I thought I could pay it forward on this one.


training part and data loading part

Thanks for the great respo.
I'm really enjoying your work, but I'm a starter in pytorch.
Would it be possible for you to release the training part and data loading part? So this project can be used to training models based on local data.
Looking forward to your reply.
Edward

Thank for your interesting. I am working on another repo and the new repo will be released soon.

Hi @pykao , I'm looking forward to the new repo.
If you need some code testing, I will be happy to do it.

Sure, I will let you know once we make the repo public.


Specs data fixes

Some fixes for the Specs/Data that have been the result of running the validator against that data.
These changes often affect many files, but the same modification is applied to all files. So it may be best to review that on a "per-commit" basis instead of looking at the files.
Notes:
Some the Specs data has apparently been auto-generated. Probably. Somehow. For example, there is the Specs/Data/Geometry subfolder without any information about where exactly these files come from. Iff there is a known tool that was used for creating them, then these fixes should preferably be done in the tool , and not manually in its output. Some of these fixes are already the result of fixes from https://github.com/CesiumGS/3d-tiles-samples-generator/pull/2 , and are therefore automated. But the overall approach for maintaining the Specs data is not perfectly transparent.
This does not yet fix all errors in the data (and some of them are not even supposed to be fixed). Also, I did not run the CesiumJS unit tests against the results (yet). But as a ballpark estimate: Running the validator against the Cesium3DTiles directory of the main state validated 204 files, where 146 files had errors. Now, only 34 files have errors. (For now...)

Thank you so much for the pull request @javagl! I noticed this is your first pull request and I wanted to say welcome to the Cesium community!
The Pull Request Guidelines is a handy reference for making sure your PR gets accepted quickly, so make sure to skim that.

:x: Missing CONTRIBUTORS.md entry.

Please add yourself to the contributors file!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

Reviewers, don't forget to make sure that:

[ ] Cesium Viewer works.
[ ] Works in 2D/CV.

So the build is failing. Running the tests locally on the current main causes 4 failures for me. Running them on the PR branch causes 4 failures for me. I have a hard time understanding the travis output ...
  ✗ clipping planes cull tiles completely inside clipping region for i3dm
	Unhandled promise rejection: RuntimeError: Failed to load model: http://localhost:9876/Data/Cesium3DTiles/Tilesets/TilesetWithExternalResources/external.i3dm?v=1.2.3
	Failed to load glTF
	Failed to load texture
	Failed to load image: bricks.jpg
	undefined
	Original stack:
	Original stack:
	Original stack:
	Original stack:
	undefined

These seem to be different failures than in the local tests (and I'm reasonably sure that I did not touch any file called bricks.jpg in this PR...). Maybe I'll have a closer look at this at a later point in time.

There have been some further fixes, based on the last validator state ( https://github.com/CesiumGS/3d-tiles-validator/tree/a33be6b016b10b195f187f9298f168902e6f9594 , supposed to become release 0.2.0 soon)
With the support for configuration files, one can run the validator with
npx ts-node ./src/main.ts --configFile validationConfigSpecs.json
where the validationConfigSpecs.json is
{
  "tilesetsDirectory": "../cesium/Specs/Data/Cesium3DTiles/",
  "tilesetGlobPattern": "../**/*tileset*.json",
  "writeReports": false,
  "options": {
    "excludeContentTypes": [
        "CONTENT_TYPE_GEOJSON",
        "CONTENT_TYPE_VCTR",
        "CONTENT_TYPE_GEOM"
    ]
  }
}

(This assumes that cesium is a sibling directory of the validator - otherwise, the path has to be adjusted accordingly)
There are still some errors. Some of them are anticipated (e.g. the spec files with legacy B3DM headers, or files that do not exist because they are not needed for the specs). Others are a bit non-trivial to fix (inconsistent availability- or buffer lengths). Bur reducing the number of errors from 146 to 4 is not a bad start.

Many of the current build errors have been caused by an image that was not found.
Other failures have been due to the test expecting Cartesian3 objects, where the result was actually a Cartesian4.

Both should be fixed now.

Details:
For 1.: The missing image was hidden behind several layers of "this-is-not-what-this-test-is-about", causing some misleading error messages. The 3d-tiles-samples-generator generates many of the specs tilesets. One of them is a tileset that uses an I3DM that does not contain a GLB, but only refer to a GLB with a URI. The GLB that this I3DM refers to is https://github.com/CesiumGS/3d-tiles-samples-generator/tree/dbad666839838626674399637fd61995278a2be5/data/textured_box_separate . Apparently, at some point, this GLB had been modified (manually?) in the specs data, to replace an image URI that referred to a bricks.jpg with one that referred to a file called Cesium_Logo_Flat.png. So the output of the samples generator still referred to bricks.jpg, but this file was not found here in the specs data. (I considered modifying the file in the samples generator, to point to the Cesium_Logo_Flat.png - but unless there are specifi reasons to do that (copyright or Corporate Identity), replacing the Cesium_Logo_Flat.png with the bricks.jpg should be fine.
For 2.: The colors should be Cartesian4, because the metadata schema defines the colors to be 4D vectors. The metadata JSON only defined them as 3D vectors. This had already been updated in https://github.com/CesiumGS/cesium/pull/10852/commits/29c53e64f0bfb4d06678c342475bad8e1e2231e5 and https://github.com/CesiumGS/cesium/pull/10852/commits/50b52ab9bcdabc20543d436640d0e24d4c72a7e4 , but the spec had not been updated yet.

All checks have passed 🎉

Thanks @javagl for tracking down all the issues!


Implement parallel get

Signed-off-by: Igoshev, Iaroslav<EMAIL_ADDRESS>What do these changes do?

[ ] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

[ ] passes flake8 .
[ ] passes black .
[ ] signed commit with git commit -s
[ ] Resolves #?
[ ] tests added and passing
[ ] module layout described at docs/developer/architecture.rst is up-to-date

Closed in favor of #359.


initial jinja2 setup

@artcz why the move to jinja as the template engine?

@umgelurgel investigating macros. :)

(rebased on latest dev/ep2019)


From trait breaks dependencies

Looks like commit<PHONE_NUMBER>87a028ed3a952fa505b1781ac391cc introduced a general from implementation that overlaps with any T that implements certain traits. This will break code that tries to provide its own implementation of From<T> (example attached).
https://github.com/cucapra/futil/blob/80aed57976ddac86a99c54dc546d42be16f073cc/calyx/src/passes/simplify_guards.rs#L10-L20
AFAIK, rust's trait system doesn't allow for multiple conflicting implementations so the upstream crate has to hide the implementation/not provide it.
On a side note, thanks for implementing this crate and open sourcing it!

Sorry about that! I hadn't fully thought through the implications of providing the From impl there. I agree that allowing flexibility is the right policy, so it's better to allow custom From impls.
I went ahead and removed this and published version 0.4.3, then yanked 0.4.2 since it was an inadvertent API-break.
@arlyon sorry about this -- hopefully adding the simple From impl on the user side won't be too much inconvenience in your use-case!

Sorry about that! I hadn't fully thought through the implications of providing the From impl there. I agree that allowing flexibility is the right policy, so it's better to allow custom From impls.
I went ahead and removed this and published version 0.4.3, then yanked 0.4.2 since it was an inadvertent API-break.
@arlyon sorry about this -- hopefully adding the simple From impl on the user side won't be too much inconvenience in your use-case!

No worries, in retrospect it is clearly a restriction not a convenience so I agree.

No worries, in retrospect it is clearly a restriction not a convenience so I agree.


Regr test: Fix trigger

Proposed changes:

Remove always() for combine_reports job, Re-write if logic

Status (please check what you already did):

[ ] added some tests for the functionality
[ ] updated the documentation
[ ] updated the changelog (please check changelog for instructions)
[ ] reformat files using black (please check Readme for instructions)

/modeltest
include:
 - dataset: ["financial-demo"]
   config: ["Sparse + BERT + DIET(seq) + ResponseSelector(t2t)"]


Hello logo

Fixes https://github.com/CartoDB/cartodb/issues/8473.
CR: @alonsogarciapablo
cc @saleiva @urbanphes @piensaenpixel

@nobuti, using the overlays in order to render the logo, but the name has changed, let's get rid of the cartodb prefix. So just logo, will it fit with your map options behaviour?

Yes! 🚀

I'll take those comments into account in other PR, don't worry @nobuti :).


Option tags and skiptags has no effect when applying ansible playbook

The options tags and skiptags are not passed to ansible-playbook command when applying playbook.
--tags "${TAGS}" --skip-tags "${SKIPTAGS}"

I have prepared a fix to look at and merge if it is right for you.