[Bug] [Resources] 3.2.0 The resource page displays "resource not exist"

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
Upgrade to version 3.2.0. The resource page displays "resource not exist". Both "LOCAL" and "HDFS" have the same problem. The file path exists, the file exists, and upgrade-schema.sh and migration-resource.sh have been executed, successfully modifying the mysql library tables and successfully migrating the data.
The problem is shown below:

When I create a folder, both "Success" and "resource not exist" are displayed. The file is successfully created in the configured path, but is not displayed on the resource page, as shown in the following figure:

What you expected to happen
I expect to be able to display resources properly and use the resource center properly
How to reproduce
After the configuration is complete, the system starts and goes to the "Resources" page.
Anything else
No response
Version
3.2.x
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @l838224553 , can you provide some error logs?

Hi @l838224553 , can you provide some error logs?
There are no errors. I printed the debug log:

Do you configure correct resource.storage.upload.base.path?

Do you configure correct resource.storage.upload.base.path?

Yes.It's configured correctly. And create folders will be prompted to "success", and the resource. The storage. The upload. Base. The path will generate the corresponding folder path, but will not display.

Do you configure correct resource.storage.upload.base.path?

Yes.It's configured correctly. In addition, the message "Success" is displayed when the folder is created, and the corresponding folder is generated in the "resource.storage.upload.base.path", but it is not displayed.

Please check server logs again, I can not reproduce this bug, seems sth wrong when querying resource list.

Please check server logs again, I can not reproduce this bug, seems sth wrong when querying resource list.

ok.Thanks.

So, how to solve it ？


Help in finetuning generator

Hi,
I'm trying an experiment, to fine-tune the generator with a small set of molecules with specific properties, (so it will generate new molecules with similar properties) but I'm running into some errors that I have been unable to solve. I'd really appreciate if anyone could shed some light into what I'm doing wrong.
What I'm doing:

From a set of 10 molecules, split into 80:10:10 for train:valid:test. Put into folder finetune_moler/input.
Run MoLeR in pre-process mode:
$ molecule_generation preprocess finetune_moler/input finetune_moler/output finetune_moler/trace
Then try to finetune the pre-trained model provided with the small set of molecules above: ```

molecule_generation train MoLeR finetune_moler/trace \
				--load-saved-model ./PRETRAINED_MODEL/GNN_Edge_MLP_MoLeR__2022-02-24_07-16-23_best.pkl \
				--load-weights-only \
				--save-dir finetune_moler/tuned_model

The pre-process step seems to run just fine. But in the fine-tuning step, I'm getting the following error:
(dumps a lot of informational messages)
Traceback (most recent call last):
  File "/opt/miniconda3/envs/moler-env/bin/molecule_generation", line 33, in <module>
    sys.exit(load_entry_point('molecule-generation', 'console_scripts', 'molecule_generation')())
  File "/home/seabra/work/source/repos/microsoft/molecule-generation/molecule_generation/cli/cli.py", line 35, in main
    run_and_debug(lambda: commands[args.command].run_from_args(args), getattr(args, "debug", False))
  File "/opt/miniconda3/envs/moler-env/lib/python3.10/site-packages/dpu_utils/utils/debughelper.py", line 21, in run_and_debug
    func()
  File "/home/seabra/work/source/repos/microsoft/molecule-generation/molecule_generation/cli/cli.py", line 35, in <lambda>
    run_and_debug(lambda: commands[args.command].run_from_args(args), getattr(args, "debug", False))
  File "/home/seabra/work/source/repos/microsoft/molecule-generation/molecule_generation/cli/train.py", line 140, in run_from_args
    loaded_model_dataset = training_utils.get_model_and_dataset(
  File "/opt/miniconda3/envs/moler-env/lib/python3.10/site-packages/tf2_gnn/cli_utils/model_utils.py", line 319, in get_model_and_dataset
    load_weights_verbosely(trained_model_file, model)
  File "/opt/miniconda3/envs/moler-env/lib/python3.10/site-packages/tf2_gnn/cli_utils/model_utils.py", line 148, in load_weights_verbosely
    K.batch_set_value(tfvar_weight_tuples)
  File "/opt/miniconda3/envs/moler-env/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/opt/miniconda3/envs/moler-env/lib/python3.10/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 911, in assign
    raise ValueError(
ValueError: Cannot assign value to variable ' decoder/node_categorical_features_embedding/categorical_features_embedding:0': Shape mismatch.The variable shape (98, 64), and the assigned value shape (166, 64) are incompatible.

Could someone point me to what I'm doing wrong? Would it be possible to get an example of successfully fine-tuning the model?
Thanks a lot!
Gustavo.

The steps you tried are quite reasonable! The missing ingredient is that preprocessing computes various vocabularies of motif/atom types based on the data, which affects the shapes of some layers in the model, but those shapes are already fixed if a pretrained model is loaded, hence the shape mismatch (intuitively, this is saying that the number of motif/atom types found in your finetuning dataset was smaller than the number of types found in the original dataset, which is expected).
So, one would have to tell preprocessing to use the vocabularies from the pretrained checkpoint instead of computing them afresh. This isn't supported in the current code (we did briefly experiment with fine-tuning, but not enough for this to end up in the release), but shouldn't be hard to add. I'll hack something together this week and then share with you as a branch; once you test it out I can then make a PR and merge it into main.

Oh, I see, I get now.
Thank you so much!

@gmseabra: can you pull kmaziarz/finetuning, re-install the package from there, and try fine-tuning again?
The only change to the workflow you described would be passing --pretrained-model-path when doing preprocessing. However, note that by default molecule_generation train will do validation every 5000 steps, and wait until there is no improvement on the validation dataset. If you're fine-tuning on a small set of molecules, it may make sense to set this to something lower (so that training has a chance to stop before it overfits) and/or limit the total number of such rounds of validation. For example, passing
--model-params-override '{"num_train_steps_between_valid": 10}' --max-epochs 8

means that you will do some multiple of 50 steps, at most 8 * 50, but possibly less if validation stops improving.
Let me know how this goes!

@gmseabra Did you have any luck with fine-tuning?

Hi! I've been trying to replicate this example with the steps you provided above, where I try to finetune it on a small set of 3K molecules, I still encounter the following error (running everything on Colab now). I just took the existing checkpoint, and I wanted to finetune it to a smaller set:
!molecule_generation preprocess input output trace
--pretrained-model-path  --load-saved-model /content/drive/MyDrive/subset_gpu_finetuning/moler/molecule-generation/best_model/GNN_Edge_MLP_MoLeR__2022-02-24_07-16-23_best.pkl
!molecule_generation train MoLeR trace
--model-params-override '{"num_train_steps_between_valid": 50}' --max-epochs 8
--load-saved-model /content/drive/MyDrive/subset_gpu_finetuning/moler/molecule-generation/best_model/GNN_Edge_MLP_MoLeR__2022-02-24_07-16-23_best.pkl
--load-weights-only
But even when aligning the metadata that the model was originally trained with (I took the same Guacamol files), it still didn't want to run. This is the error I encountered:
Traceback (most recent call last):
  File "/usr/local/bin/molecule_generation", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/molecule_generation/cli/cli.py", line 35, in main
    run_and_debug(lambda: commands[args.command].run_from_args(args), getattr(args, "debug", False))
  File "/usr/local/lib/python3.10/site-packages/dpu_utils/utils/debughelper.py", line 21, in run_and_debug
    func()
  File "/usr/local/lib/python3.10/site-packages/molecule_generation/cli/cli.py", line 35, in <lambda>
    run_and_debug(lambda: commands[args.command].run_from_args(args), getattr(args, "debug", False))
  File "/usr/local/lib/python3.10/site-packages/molecule_generation/cli/train.py", line 140, in run_from_args
    loaded_model_dataset = training_utils.get_model_and_dataset(
  File "/usr/local/lib/python3.10/site-packages/tf2_gnn/cli_utils/model_utils.py", line 319, in get_model_and_dataset
    load_weights_verbosely(trained_model_file, model)
  File "/usr/local/lib/python3.10/site-packages/tf2_gnn/cli_utils/model_utils.py", line 148, in load_weights_verbosely
    K.batch_set_value(tfvar_weight_tuples)
  File "/usr/local/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.10/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1022, in assign
    raise ValueError(
ValueError: Cannot assign value to variable ' decoder/node_type_selector/MLP_final_layer/kernel:0': Shape mismatch.The variable shape (256, 142), and the assigned value shape (256, 167) are incompatible.

I'm not sure whether I did something wrong, and where I could fix it!

Hi @MeilinaR, sorry for the silence, I forgot to respond to this one. Are you still having issues? I noticed the command you pasted passes --load-saved-model to preprocess; is this a typo?

Closing due to lack of activity.


refresh auto-generated samples for mgmt sdk

Contributing to the Azure SDK
Fix https://github.com/Azure/azure-sdk-for-net/issues/33936

refresh auto-generated sample codes

Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

Dont enable this now as the CodeGen can't generate samples properly for some RPs

OK. I will close this PR.


Download Links Broken?

[x] I am on the latest Hyper.app version
[x] I have searched the issues of this repo and believe that this is not a duplicate

Issue
I have been trying to download the latest version of Hyper, but it seems that all the links are completely broken.  Is there something I am missing?

The links on hyper.is/#installation works fine for me – feel free to reopen with more info (completely broken is not very descriptive 😛)

Work fine to me too

My apologies, for not leaving more information, thought it would be pretty straightforward 😬
I am trying to download the macOS version of the app from hyper.is/#installation as you have linked above.  the actual link is this:
https://github-production-release-asset-2e65be.s3.amazonaws.com/62367558/3a4a781c-90d6-11e7-8138-07b99403f502?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20170904%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170904T054101Z&X-Amz-Expires=300&X-Amz-Signature=c1a76d69a234d906e83c89427b256d8efdfa1847a9d26e63c2da2eb97a20b643&X-Amz-SignedHeaders=host&actor_id=1544580&response-content-disposition=attachment%3B%20filename%3Dhyper-1.4.3-mac.zip&response-content-type=application%2Foctet-stream
That link completely fails for me (any browser I try).
I also tried downloading directly from the GitHub releases section, with the same results.
Just for fun, I tried downloading all the other versions, just to see, and they all failed.
If you have any suggestions, even if it is to help me figure out if this could be something local to my machine, I would be very appreciative

seems to be throwing connection-refused error:

The URL you posted doesn't work here too:

The URL I get from hyper.is, however, is one: https://github-production-release-asset-2e65be.s3.amazonaws.com/62367558/3a4a781c-90d6-11e7-8138-07b99403f502?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20170904%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170904T060255Z&X-Amz-Expires=300&X-Amz-Signature=6f225c38b7aeabfaf8030cf978387e506e1fc022835f89845e216ff4ea826e2e&X-Amz-SignedHeaders=host&actor_id=4721750&response-content-disposition=attachment%3B filename%3Dhyper-1.4.3-mac.zip&response-content-type=application%2Foctet-stream
And it works like a charm 🤔 Maybe S3 is somehow broken for you? Where are you from?

@n8stowell82 maybe your firewall

Very odd, clicking that link fails for me as well.  Maybe it is trying to redirect? I am in Dallas, TX

@chabou firewall is off 😕

not sure if you changed something on your end @chabou but it magically started working 🤷♂.  Thanks for your time and help

Now the link I posted doesn't work anymore for me too – looks like they expire after 300 seconds: X-Amz-Expires=300
@leo any clues?

Oh nice @n8stowell82, probably some caching issue – we didn't deploy any changes 👌

Yea, I'm assuming it's related to GitHub's CDN and some caching issue, like @matheuss said. Hazel is pulling every 10/15 minutes (not sure what the value i set was), which means that broken download links will be automatically fixed by getting the most recent one by GitHub.

@leo I think it is related to GitHub's CDN and @n8stowell82 Internet infrastructure.
You can see "ERR_CONNECTION_REFUSED" on his screenshot. This is a transport layer (TCP) issue, not application layer. If cached is expired on CDN, you'll see a formated error message (404 ou 403).
Pretty sure this is a firewalling/proxy temporary issue somewhere between @n8stowell82's computer and GitHub.


Decoder topological circuit readout string

@JazzyCH  implement TopologicalCircuitDecoder that can parse a readout string from a tcirc composed of tqubits of the same type by Jun 16.

Waiting on #44


Build an android tensorflow binary with support DT_BOOL

Hey guys,
I have the following error by running a network on android:
Unhandled Exception:TensorFlow.TFException: No OpKernel was registered to support Op 'Switch' with these attrs.  Registered devices: [CPU], Registered kernels:device='GPU'; T in [DT_STRING]device='GPU'; T in [DT_BOOL]device='GPU'; T in [DT_INT32]device='GPU'; T in [DT_FLOAT]device='CPU'; T in [DT_FLOAT]device='CPU'; T in [DT_INT32][[Node: model/comb1/comb1/cond/Switch = Switch[T=DT_BOOL](ph/training_ph, ph/training_ph)]] occurred
It looks like the type bool is not support on the cpu version of tensorflow.
I found this fix: https://github.com/tensorflow/models/issues/1740
but wasnt able to manage it....

Did you build own native library?

No not yet... I tried to use the gpu version provided by miguel. But there is something else going wrong..
Can anyone provide the built native library for bool support? I suppose I am not the only one having trouble with that. Would spare me a lot of time getting started with building the c-lib myself.
Thanks for any help :)

I builted bool supported native lib. Google Drive
It worked fine with my own model (included BatchNorm Layers that have Switch Ops).
And you can find gpu supported binaries in GitIgnoredDatas.zip too.  It is the zip file that I uploaded.
You just include into TensorFlowSharp.Windows instead of CPU version of libtensorflow.dll.
Follow files should be included to use gpu on window.

/GitIgnoredDatas/windows/gpu/libtensorflow.dll
/GitIgnoredDatas/windows/libs/*.dll

If you want to get dll yourself, copy _pywrap_tensorflow_internal.pyd from python package. And rename .pyd as libtensorflow.dll. Also you need dependency dlls too.

Thanks man! Your new built native library is working for me :)

Great!

Hi @gmlwns2000 , I have tried your built libraries and it's working for me. And I'm pretty interested how to build libraries like that as well. May you share how to make it ? Thanks

Hi @hgffly ,
This is how to build tensorflow C lib for android. I hope this will help you.

Make tensorflow for android support DT_BOOL.
Edit following line #define TF_CALL_bool(m) to #define TF_CALL_bool(m) m(bool)
Source: stackoverflow
Change build option to appear TF_* functions in libtensorflow_inference.so.
This file defines which functions should be export. Add a line TF_* in global definition

VERS_1.0 {
  global:
    Java_*;
    JNI_OnLoad;
    JNI_OnUnload;
    TF_*;
  local:
    *;
};

Build a libtensorflow_inference.so. You should build several time to support platforms (arm64-v8a, armeabi-v7a, x86, x86_64)
Enter the command on the root of tensorflow dir.

bazel build -c opt //tensorflow/contrib/android:libtensorflow_inference.so \
   --crosstool_top=//external:android/crosstool \
   --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
   --cpu=armeabi-v7a

To change target platform, you can just change --cpu=armeabi-v7a.
If you ran out of memory while building, add --verbose_failures --local_resources 4096,4.0,1.0 -j 1

So now, you can find tensorflow_inference.so in bazel-bin/tensorflow/contrib/android. Copy libtensorflow_inference.so into other place, go back to step 3, change cpu target, and build it again for several platforms.
After building whole binary for each platform, rename libtensorflow_inference.so to libtensorflow.so. And then copy those files into correct place
(ex: TensorflowSharp/GitIgnoredData/android/armeabi-v7a/)

My solution is not official and not clear too, if you guys have some better solution for building native library for android, please share how to make it :)
Thanks

Hi gmlwns2000,
Appreciate you for sharing how to build the library for android
But I encountered some errors in step 3:
ERROR:
/home/wilson/.cache/bazel/_bazel_wilson/416ee1e5c1dd220e496a2567f04ee5b5/external/protobuf/BUILD:113:1:
C++ compilation of rule '@protobuf//:protobuf' failed: false failed: error
executing command /bin/false -MD -MF
bazel-out/stub_armeabi-v7a-py3-opt/bin/external/protobuf/_objs/protobuf/external/protobuf/src/google/protobuf/wrappers.pb.pic.d
... (remaining 26 argument(s) skipped):
com.google.devtools.build.lib.shell.BadExitStatusException: Process exited
with status 1.
Target //tensorflow/contrib/android:libtensorflow_inference.so failed to
build
I may need to investigate how to resolve it
After resolving it, I will try it soon, thanks
Which version of Tensorflow do you use? My Tensorflow is r1.2
Thanks for help!
2017-08-02 12:25 GMT+08:00 AinL<EMAIL_ADDRESS>
Hi @hgffly https://github.com/hgffly ,
This is how to build tensorflow C lib for android. I hope this will help
you.

Make tensorflow for android support DT_BOOL.
Edit following line
https://github.com/tensorflow/tensorflow/blob/v1.1.0-rc2/tensorflow/core/framework/register_types.h#L125 #define
TF_CALL_bool(m) to #define TF_CALL_bool(m) m(bool)
Source: stackoverflow
https://stackoverflow.com/questions/40855271/no-opkernel-was-registered-to-support-op-switch-with-these-attrs-on-ios/43627334#43627334
Change build option to appear TF_* functions in
libtensorflow_inference.so.
This file
https://github.com/tensorflow/tensorflow/blob/v1.1.0-rc2/tensorflow/contrib/android/jni/version_script.lds
defines which functions should be export. Add a line TF_* in global
definition

VERS_1.0 {
global:
Java_;
JNI_OnLoad;
JNI_OnUnload;
TF_;
local:
*;
};

Build a libtensorflow_inference.so. You should build several time
to support platforms (arm64-v8a, armeabi-v7a, x86, x86_64)
Enter the command on the root of tensorflow dir.

bazel build -c opt //tensorflow/contrib/android:libtensorflow_inference.so
--crosstool_top=//external:android/crosstool
--host_crosstool_top=@bazel_tools//tools/cpp:toolchain
--cpu=armeabi-v7a

To change target platform, you can just change --cpu=armeabi-v7a.
If you ran out of memory while building, add --verbose_failures
--local_resources 4096,4.0,1.0 -j 1

So now, you can find tensorflow_inference.so in
bazel-bin/tensorflow/contrib/android. Copy libtensorflow_inference.so
into other place, go back to step 3, change cpu target, and build it again
for several platforms.
After building whole binary for each platform, rename
libtensorflow_inference.so to libtensorflow.so. And then copy those
files into correct place
(ex: TensorflowSharp/GitIgnoredData/android/armeabi-v7a/)

My solution is not official and not clear too, if you guys have some
better solution for building native library for android, please share how
to make it :)
Thanks
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gmlwns2000/TensorFlowSharp/issues/3#issuecomment-319563717,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATOuLfJABWK8pd35Avra5WxctloOHN39ks5sT_o4gaJpZM4OP5YF
.

Hi, @hgffly ,
I used tensorflow 1.2.0-r0 for build a library.
I'm sorry but I am not good at bazel building, so I can't help you :(

Hi gmlwns2000,
It's fine, you have already helped a lot, thanks so much!
http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
不含病毒。www.avg.com
http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
2017-08-03 19:18 GMT+08:00 AinL<EMAIL_ADDRESS>
Hi, @hgffly https://github.com/hgffly ,
I used tensorflow 1.2.0-r0 for build a library.
I'm sorry but I am not good at bazel building, so I can't help you :(
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gmlwns2000/TensorFlowSharp/issues/3#issuecomment-319941138,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATOuLXlhP49eyPaBuHmPOdfOi0oXbLlHks5sUayYgaJpZM4OP5YF
.


Proposal: Add :prepare option for Postgres Adapter

Because of the way Postgres generates plans for prepared statements, sometimes it is beneficial to disable them (i.e. use an unnamed prepared statement). See this Postgrex issue for more details.
You can currently use unnamed prepared statements, but they are set at the Repo config level and would apply to all queries. This means the user can't pick and choose which queries benefit from this feature.
This proposal is to allow the Postgres adapter to recognize a :prepare option and to overwrite the Ecto-generated name with empty string. This is similar to what happens in Postgrex when prepare: :unnamed is configured on the connection. See this excerpt from Postgrex.Protocol:
def handle_prepare(%Query{} = query, opts, %{queries: nil} = s) do
  # always use unnamed if no cache
  handle_prepare(%Query{query | name: ""}, opts, s)
end

This way you still get the benefits of the Ecto cache but without the potentially harmful effects of the Postgres cache. One potential pitfall is that once you issue the query you can't change the :prepare option because the prepare step will be skipped. I think that's ok but let me know what you guys think.
The change is restricted to the Postgres adapter because I don't think the other adapters handle empty string names the same way. I could be mistaken though, since I'm not as familiar with them. Postgres will treat this as a "one shot" name and allow it to be re-defined without explicitly telling it that you're done with the old definition.
Note that this option is meant as a convenience. You can currently issue an unnamed query using Ecto.Adapters.SQL.query but then you don't get all the nice Ecto conveniences.
If you guys like this proposal then I think the following things also need to be done:

Update the Ecto docs to say this option exists
Add integration tests in Ecto to make sure the queries work the same when that option is specified
Maybe add an integration test in this repo. If I could create one that inspects the results from Ecto.Adapters.Postgres.Connection.prepare_execute then I could inspect the name of the query.

My only concern with this option is the :prepare applying permanently for future operations. It feels perhaps we should make it part of the query cache somehow?

My only concern with this option is the :prepare applying permanently for future operations. It feels perhaps we should make it part of the query cache somehow?

I'll take a closer look at the cache handling and come back with a different proposal. Thanks!

It looks like something like this can be done:
In the ecto repository, specifically Ecto.Query.Planner.finalize_cache can add {:prepare, :named | :unnamed} to the cache. This can be done either by:

adding a new prepare argument toEcto.Query.Planner.query
adding a new prepare field to the query struct and making sure it's in the query before it goes to the planner

Then in ecto_sql can make a change similar to what is in this PR. The combo of these 2 will allow for having both unnamed and named versions of the same query.
I think it's possible to ignore this option for everything that's not Postgres and default it to named. But I have to investigate a bit more to figure out the details.

Oh, I think we may be fine with the current approach, as long as we store if the query was prepared or not in the cache? If the query was not prepared and then it is later requested to be prepared, then do so and we update the cache? We can use the callback functions listed here: https://github.com/elixir-ecto/ecto/blob/master/lib/ecto/query/planner.ex#L129-L135

That makes sense. I'm trying to see if it's necessary to use the prepare_execute callback every time  the user changes the prepare value.
It looks like if you use the execute callback, Postgrex will re-prepare if it has to. This would mean someone going back and forth several between named and unnamed could skip the prepare step for the named query.
I'm just not sure if somehow this code segment can get hit when we wouldn't want it to. If it hits this then potentially something that needs to be prepared wouldn't be
defp handle_execute_result(%{ref: ref} = query, params, opts, %{postgres: {_, ref}} = s) do
  # ref in lock so query is prepared
  status = new_status(opts)

  case query do
    %{name: ""} -> bind_execute_close(s, status, query, params)
    _ -> bind_execute(s, status, query, params)
  end
end

Do you think it's safer to always call prepare_execute because going back and forth like that should be a very rare occurrence? I can dig into the Postgrex code more if you think it's worth considering.

Actually nevermind I think the query ref changes every time the query is parsed. So you will lose the ref to the named query if you go from named -> unnamed -> named and it will need to re-prepare.
It seems like prepare_execute always needs to be called.

I will submit a PR to Ecto with the following changes, if you are alright with them:

Will add prepare value to the cache.
If query was already cached, issue prepare_execute instead of execute if the opts[:prepare] is not the same as the cache value. Also update the cache with the new query + prepare value.
I'll try to see if I can find a new way to ignore opts[:prepare] when the adapter is not Postgres and force it to be :named.

Can’t we handle it in ecto_sql exclusively?

Ah yes sorry I see what you mean now. I'll submit something soon.

@josevalim Thanks for your guidance. I think this is a lot better but wanted to get your opinion on the approach before I start on writing tests.

Oh yeah if you prefer to leave it as prepare: :named | :unnamed then I can switch it back. I originally had it that way but my thinking was that prepare: :named | :unnamed is very Postgres specific, both the option name and the potential values. I was trying to convert it into a neutral term in case other adapters in the future need a similar capability to bypass the cache but use different option names/values.

I could also leave the option name as :prepare but convert its value to true/false because prepare is a pretty neutral name. But I think the idea of unnamed prepared statements if very Postgres specific, if I'm not mistaken.

I didn't dive a lot into this issue, so I may be saying some non-sense here, but doesn't this do the same as passing cache_statement: "" as an option? (see: https://github.com/elixir-ecto/ecto_sql/pull/352/files)

I think using :prepare is fine. All adapters based on DBConnection have a prepare nomenclature.
cache_statement is different caching mechanism that compares the name with the statement and reuses it if the statement matches. The prepare query does not keep the original statement at all (and it is therefore superior).

@josevalim Sorry just to be clear, did you mean to use :prepare as a boolean for the downstream?

I think it's a good way to go. It works for everything existing now. And if by some fluke a future built-in adapter has a :prepare option with different values, can think about normalizing it then.

@josevalim Think it's good to go. Did the following things:

got rid of :use_cache
documented the new option
added integration test

I had one thought actually. I think it's possible custom adapters can specify their own :prepare option to use in the Connection callback module however they want?
Then would it make more sense to add a prepare argument to Ecto.Adapters.SQL.execute(adapter_meta, query_meta, query, params, opts) instead of making it part of opts?

Beautiful! The solution you landed on is quite elegant! :green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart:


ForControl-Radio: TypeError: Cannot read properties of undefined (reading '0')

Description
says TypeError: Cannot read properties of undefined (reading '0') when checking one of the raido input in the radiogroup
CodeSandbox/Snack link
https://gluestack.io/ui/docs/components/form-control
Steps to reproduce

Go to 'https://gluestack.io/ui/docs/components/form-control'
Click on any of the raido inputs in The Radio Component
See error says (TypeError: Cannot read properties of undefined (reading '0') )

gluestack-ui Version
3.3.1
Platform

[ ] Expo
[ ] React Native CLI
[X] Next
[ ] Web
[ ] Android
[ ] iOS

Other Platform
No response
Additional Information
No response

The issue is fixed so, I am closing it.
Thank You


Create replace_relations.tsv

Adding a file for @balhoff to review for replacing relations in GO-CAM models

@balhoff
Here's a draft tsv file for updating three of the RO relations used in GO-CAMs.  Will this work?
If not, please suggest what would be better.  Thx.

Thanks, @balhoff
@kltm - where should the two tsv files for non-automatic ontology updates live?

@vanaukenk I would suggest that this is noctua-models metadata, not minerva metadata.

Okay.  I can upload the file to nocuta-models.  Is this the preferred directory:
https://github.com/geneontology/noctua-models/tree/master/migrations
If not, please let me know where the file should go.
Thx.

@vanaukenk I think that is a good place for that for the moment.

For https://github.com/geneontology/minerva/issues/465

I think this can be closed now.


Bot crashes if hunter configuration is not specified

Expected Behavior
When the bot starts, it should begin moving to fort, catching Pokemon etc, depending on the options specified within the config.json file.
Actual Behavior
When the bot starts. it crashes with the an error due to the PokemonHunter section not being present in config.json file.
Output when issue occurred
Traceback (most recent call last):
  File "pokecli.py", line 865, in <module>
    main()
  File "pokecli.py", line 206, in main
    bot.tick()
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/pokemongo_bot/__init__.py", line 778, in tick
    if worker.work() == WorkerResult.RUNNING:
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/pokemongo_bot/cell_workers/catch_pokemon.py", line 55, in work
    if self.bot.hunter_locked_target != None:
AttributeError: 'PokemonGoBot' object has no attribute 'hunter_locked_target'
[2017-02-08 23:25:28] [sentry.errors] [ERROR] Sentry responded with an error: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128) (url: https://app.getsentry.com/api/90254/store/)
Traceback (most recent call last):
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync
    super(ThreadedHTTPTransport, self).send(data, headers)
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send
    ca_certs=self.ca_certs,
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open
    response = self._open(req, data)
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open
    '_open', req)
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain
    result = func(*args)
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open
    return self.do_open(ValidHTTPSConnection, req)
  File "/Users/ashley/POKEMONGO/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1053, in request
    self._send_request(method, url, body, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1093, in _send_request
    self.endheaders(body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1049, in endheaders
    self._send_output(message_body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 891, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)
[2017-02-08 23:25:28] [sentry.errors.uncaught] [ERROR] [u"AttributeError: 'PokemonGoBot' object has no attribute 'hunter_locked_target'", u'  File "pokecli.py", line 865, in <module>', u'  File "pokecli.py", line 206, in main', u'  File "pokemongo_bot/__init__.py", line 778, in tick', u'  File "pokemongo_bot/cell_workers/catch_pokemon.py", line 55, in work']

Steps to Reproduce

Clone repo
Check out dev branch
Run ./setup -i
Don't have a PokemonHunter section in config.json
Start bot

Other Information
OS: Mac OSX
Branch: dev
Git Commit: 8a677a12d22e9b02cd6f8f6d4143f15854ea0bf4
Python Version: Python 2.7.10

Workaround is to add this to the config.json file
        {
            "type": "PokemonHunter",
            "config": {
                "enabled": false,
                "max_distance": 1500,
                "hunt_all": false,
                "hunt_vip": true,
                "hunt_pokedex": true,
                "lock_on_target": false
            }
        }

But ideally the bot should have better handling of configuration items that aren't specified instead of crashing.

It's already being submitted as buxfix #5904
Waiting to be merged.

Merged, thanks for the patch. let me close this issue.


Upgrade IOV-Core to 0.13.3 and integrate ERC20 tokens

But tests are failing, some problem related to faucet. Do you know how to fix them?

Arg, good point! I only ran non-blockchain tests locally. The ERC20 configuration only works for rinkeby. So we need different configurations for tests and testnet. I will try to move the Ethereum options into a config.

Travis automatic deployment:
https://pr324-272_wallet-demo_iov.surge.sh
Storybook book automatic deployment:
https://storybook_pr324-272_wallet-demo_iov.surge.sh

ERC20 config moved into config files. All tests green.


Next Generation of Spark-Bench

spark-bench Rewrite
SparkTC/spark-bench is a project that fulfills the objective of providing a standard set of mostly ML-focused benchmarks for Apache Spark. This PR is a rewrite that seeks to expand upon the initial goals of the spark-bench paper.
The Vision: Going Beyond Benchmarking
SparkTC/spark-bench fulfills one particular user story, and does so quite well:
As a researcher, I want to run a set of standard benchmarks with generated data against one instance of Spark, one time.
This PR takes that vision and expands on it greatly. In this new version of spark-bench, users can simulate, test, compare, exercise, and still benchmark, all within one flexible tool. From the original user story, we crafted many more:

As a Spark developer, I want to run regression tests with my changes in order to provide some authoritative performance stats with my PR.

As a Spark developer, I want to see regression tests run against Spark nightly builds

As a Parquet developer, I want to compare performance between Spark built with several different versions of Parquet

As a Spark developer, I want to simulate multiple notebook users hitting the same cluster all at once so that I can test my changes to the dynamic allocation algorithm.

As an enterprise user of Spark, I want to make my daily batch ingest job run faster by tuning parameters.

As a DevOps professional, I want to see how performance changes for our daily jobs as a I add more nodes to our prod cluster.

As a data scientist, I want to run several machine learning algorithms over the same set of data in order to compare results.

What We Did In The Rewrite
Key New Features

Create and launch multiple spark-submits, serially or in parallel
Run workloads within those spark-submits serially or in parallel
Repeat benchmark runs easily and configurably
Centralized configuration
Configurable input and output files (supports local files, HDFS, S3, etc.)
Configurable Spark parameters and workload parameters
Configurable results output in multiple formats (CSV, parquet, etc.)
BYOW: Bring Your Own Workload.jar and run it using spark-bench

Currently Available Workloads

Kmeans
Logistic Regression
SparkPi
Sleep
SQL (run any given SQL query)
Kmeans Data Generation
Linear Regression Data Generation

Transition Plan

Preserve Existing SparkTC/spark-bench in a branch
Port issues and current in-progress work from ecurtin/spark-bench to SparkTC/spark-bench
Port All SparkTC/spark-bench Workloads To New Version
Create Github Pages site for easier documentation views

Plan for Supporting Multiple Spark Versions
This rewrite currently uses Spark version 2.3.
Supporting a greater number of Spark versions both older and newer is an important goal that must be carefully managed.
Branch Structure
We propose mirroring the branching strategy of Apache Spark itself.
Spark has a master branch where new feature development occurs. This master branch is "cut" periodically to become a new versioned release of Spark, and this version exists in its own branch. Thus, new features are developed in master and fixes are backported as needed and as appropriate to version branches.
In spark-bench, as it nears feature-completeness with the legacy branch, we should create a branch structure as follows:

legacy: preserves existing SparkTC/spark-bench as developed by the original authors.
gh-pages: no spark-bench source, only the source for the Github Pages site.
master: built against the next upcoming version of Spark. For example, 2.2.0 is the current stable version, so master should be built against 2.3.
spark-2.2: preserves all functionality of master but with any necessary changes to adapt to the Spark 2.2 API
spark-2.1: as above, but against Spark 2.1
spark-2.0: as above
spark-1.6: as above

As the Spark 1.6 branch is the oldest "active" branch on the Spark project itself, we propose that the cost-benefit of maintaining versions of spark-bench against anything older than Spark 1.6 would require significant effort without producing significant benefit.
Build/Release
The rewrite in this PR is built using TravisCI. Currently new PRs to master will trigger a build of the release distribution that is uploaded automatically to Github Releases. This workflow would require a bit of modification to support multiple Spark versions:

TravisCI is currently configured to pull down a certain version of Spark into the build environment to run tests against.
Change this pull down the configured version of Spark appropriate for that branch.

Continue to post releases to Github Releases, but instead of pointing users to the releases page, organize links to the appropriate releases supporting various Spark versions on a page in the Github Pages website. If these links could be curated automatically, all the better, but for now manual editing will suffice.

What's Next

Port workloads and data generators from SparkTC/spark-bench
Add hardware stats to results
Beef up data generators

More custom capabilities
Null, corrupted value generation
Dataset downloader

Generate data based on a given schema
SparkR, PySpark
Streaming workloads and data generators
Add standard SQL focused workloads such as TPC-DS and others
So much more!!!

Closing this PR. Will submit a new one shortly with the commits squashed.

@ecurtin  is tpcds related workload added ? I didn't manage to find them

@zinking It was not. Significant effort was extended in that direction but we ultimately found that (at the time) SparkSQL was limited in its ability to express the complex queries in the TPC-DS suite.


TypeError: expected str, bytes or os.PathLike object, not CatBoostRegressor

Hi there!
I don't understand why dice_ml doesn't want to accept a pre-trained model (CatBoostRegressor in my case) and requires to provide the path to the model. The code is
import dice_ml

d = dice_ml.Data(
    features={
        'season': ['spring', 'summer', 'fall', 'winter'],
        'month': list(range(1, 13)),
        'hour': list(range(0, 24)),
        'holiday': ['False', 'True'],
        'weekday': list(range(0, 7)),
        'workingday': ['False', 'True'],
        'weather': ['clear', 'misty', 'rain'],
        'temp': [-10.0, 50.0],
        'feel_temp': [-10.0, 50.0],
        'humidity': [0.0, 1.0],
        'windspeed': [0.0, 70.0]
    },
    outcome_name='bike_rentals'
)
m = dice_ml.Model(
    model_path=cb_model, backend='sklearn', model_type='regressor'
)
exp = dice_ml.Dice(d, m, method='random')

It follows one of the tutorials regarding calling the functions, so I don't understand why I get
TypeError                                 Traceback (most recent call last)
[<ipython-input-65-0c0c9dbbcb6d>](https://localhost:8080/#) in <cell line: 28>()
     26     model_path=cb_model, backend='sklearn', model_type='regressor'
     27 )
---> 28 exp = dice_ml.Dice(d, m, method='random')

3 frames
[/usr/local/lib/python3.10/dist-packages/dice_ml/dice.py](https://localhost:8080/#) in __init__(self, data_interface, model_interface, method, **kwargs)
     20         :param method: Name of the method to use for generating counterfactuals
     21         """
---> 22         self.decide_implementation_type(data_interface, model_interface, method, **kwargs)
     23
     24     def decide_implementation_type(self, data_interface, model_interface, method, **kwargs):

[/usr/local/lib/python3.10/dist-packages/dice_ml/dice.py](https://localhost:8080/#) in decide_implementation_type(self, data_interface, model_interface, method, **kwargs)
     30                     ' since kdtree explainer needs access to entire training data')
     31         self.__class__ = decide(model_interface, method)
---> 32         self.__init__(data_interface, model_interface, **kwargs)
     33
     34     def _generate_counterfactuals(self, query_instance, total_CFs,

[/usr/local/lib/python3.10/dist-packages/dice_ml/explainer_interfaces/dice_random.py](https://localhost:8080/#) in __init__(self, data_interface, model_interface)
     26
     27         self.model = model_interface
---> 28         self.model.load_model()  # loading pickled trained model if applicable
     29         self.model.transformer.feed_data_params(data_interface)
     30         self.model.transformer.initialize_transform_func()

[/usr/local/lib/python3.10/dist-packages/dice_ml/model_interfaces/base_model.py](https://localhost:8080/#) in load_model(self)
     40     def load_model(self):
     41         if self.model_path != '':
---> 42             with open(self.model_path, 'rb') as filehandle:
     43                 self.model = pickle.load(filehandle)
     44

TypeError: expected str, bytes or os.PathLike object, not CatBoostRegressor

Moreover, the documentation lacks module __init__() parameters for some reason while they are present in the source code:

Is this done on purpose?

@TopCoder2K, you could use model parameter instead of model_path.
https://github.com/interpretml/DiCE/blob/48832802c2a0822a9b203f3057e6def9e8ba0d0a/dice_ml/model_interfaces/base_model.py#L16.

Thank you for you help! Now it's working.


Gives the capability to customize Acorn.parse

@marijnh I'm writing a tern plugin to use ESLint. This tern plugin is not performant because :

acorn parse text file to use AST with tern
ESLint tern plugin use file text to call ESLint#verify which parses the given text with Espree

In other words, file text is parsed twice.
To improve that I would like to use Acorn AST with ESLint, but it requires that Acorn must be configurated:
var ast = Acorn.parse(text, {
 ranges: true,
 locations: true,
 onToken: function(token) {
  tokens.push(_convertToEsprima(token));
 },
 onComment: comments,
});

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=432940#c17 for more information
It should be cool if tern could provide a mean (perhaps a passes like postParse?) to customize Acorn.parse

This would be kind of scary, because it would be very easy to pass an option that breaks Tern's own expectations about the AST. When you say your current situation is 'not performant', do you mean it really is too slow, or you just consider the double parse theoretically wasteful? Doing extra work to keep things cleanly separated, when you can afford that work, is often a good idea.

When you say your current situation is 'not performant', do you mean it really is too slow, or you just consider the double parse theoretically wasteful?

Yes I consider the double parse theoretically wasteful (I'm afraid with memory too with big projects liek dojotoolkit). I will do tests.
But the second thing is that there is a Orion guy which works on the topic to use Acorn with ESLint to integrates Tern inside Orion (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=432940#c17)
If Orion integrates Tern, Eclipse community will accept more tern, it means more users of tern:)

Does ESLint keep the AST around? I thought it just creates it and throws it away again after analyzing. In that case, memory should not be a big concern.

@mrennie could you please gives us information please about Acorn+ESLint integration. Thanks!

The mentioned link ( https://bugs.eclipse.org/bugs/show_bug.cgi?id=432940#c17) pretty much sums up where the work is at right now - I have been focusing on our Orion 8.0 release and have not had time to work on it more.

could you please gives us informations about Acorn+ESLint integration

ESLint in Orion is not the stock version, I have customized it to accept an AST in addition to just text. This allows us to get around the  parse > verify > throw away cycle Marijn mentioned and use our cached AST - more simply, it does not matter how we parse, just so long as the ASTs are compatible.
The rules used in Orion are also not the ones from the ESLint repo (this is due to an IP provenance issue), so we have written our own, making it easy to customize them as needed to adapt to Acorn.

My advice would be to use Acorn's regular AST, and rely on the start and end properties for looking up line/column positions. There is a rather efficient, caching implementation of converting between character offsets and line/column objects in the Tern codebase (asLineChar in lib/tern.js), which you could steal. You could even add computed properties to the Node prototype that lazily initialize the properties you need from the start/end numbers.


Fix multiple issues in xExchInstall

Pull Request (PR) description
Fixes for issues https://github.com/PowerShell/xExchange/issues/200 and https://github.com/PowerShell/xExchange/issues/241
This Pull Request (PR) fixes the following issues
Fixes #200
Fixes #241

This change is

Reviewed 2 of 2 files at r1.
Review status: all files reviewed, 1 unresolved discussion (waiting on @mhendric)

CHANGELOG.md, line 6 at r1 (raw file):

- Fixed issue in xExchInstall where winrm config command fails to execute
- Fixed issue in xExchInstall where a failed Exchange setup run is not reported, and subsequent DSC resources are allowed to run

This row is over 80 chars (need to break it to sveral rows). Tests failing,
https://ci.appveyor.com/project/PowerShell/xexchange/build/1.13.547.0?fullLog=true#L1086

Comments from Reviewable

Codecov Report

Merging #242 into dev will decrease coverage by <.01%.
The diff coverage is 0%.

@@           Coverage Diff            @@
##             dev    #242      +/-   ##
========================================
- Coverage   0.97%   0.96%   -0.01%
========================================
  Files         36      36
  Lines       3917    3925       +8
========================================
  Hits          38      38
- Misses      3879    3887       +8

Impacted Files
Coverage Δ

...Resources/MSFT_xExchInstall/MSFT_xExchInstall.psm1
1.12% <0%> (-0.12%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update faaeb58...59cabef. Read the comment docs.


Feature request: Julia-style "next command" down key in the REPL

Julia's REPL has a really nice feature: when you press the down key, if you just picked and executed a command in the history, it will select the following command in the history.
For example:
julia> 1+1
2
julia> 2+2
4
julia> 3+3
6
julia> 4+4
8
julia> 2+2        #<-- this command was selected in the history by type <up><up><up>
4
julia> 3+3        #<-- this command was selected simply by typing <down>

Rationale: it's very common to want to execute several commands in a row from the history (perhaps skipping or editing a few of them). If these commands are far up in the history, then it's cumbersome to have to go back up for each one. Thanks to this "next command" feature, you only need to go back up once.
It's probably not very hard to implement, and it's a real pleasure to use. Why not in Roc's REPL?

I'd be fine with that addition, we rely on rustyline for these things. Would you be willing to add an issue to rustyline for this? That way all projects relying on rustyline will be able to use this feature :)

Yes, I started looking at that yesterday and I noticed that it's rustyline under the hood. I'll file an issue over there. 👍

Here's the rustyline issue I just created: https://github.com/kkawakam/rustyline/issues/781


⚠ Processors has degraded performance

In ab6a870, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 43 ms

Resolved: Processors performance has improved in 9ba39cc.


Allow for non unique state IDs

Right now, Verbs snapshots have a unique id primary key, which means that you can only have one of any given ID snapshotted at any given time. This is fine if you use snowflakes for everything, but if you don't that can be an issue.
@skylerkatz and I paired on this for a while, and I know there were issues remaining, but I can't remember what they are. I'm going to post this up as a draft while I review it and maybe add some more tests.

@skylerkatz I forgot to push up a couple changes. Can you take another look?
Also—I agree that we should think about allowing Verbs to run without snowflakes at all. Let's discuss at some point.


Restrict deploy from master, to master only

Coverage remained the same at 85.612% when pulling 20da37eedce5fee54bc43bc6971b9592ec4105e4 on fbiville:master into 145892eeead19cc98fc9d9a0d4215c7d2de82b00 on liquigraph:master.

Coverage remained the same at 85.612% when pulling 20da37eedce5fee54bc43bc6971b9592ec4105e4 on fbiville:master into 145892eeead19cc98fc9d9a0d4215c7d2de82b00 on liquigraph:master.

Coverage remained the same at 85.612% when pulling 20da37eedce5fee54bc43bc6971b9592ec4105e4 on fbiville:master into 145892eeead19cc98fc9d9a0d4215c7d2de82b00 on liquigraph:master.


Make clean URL generation optional (default true)

Current behaviour is to turn each page into a folder containing an index.html file, so pages/about.md turns into about/index.html for a nice clean extension-less URL of /about/. (As requested in #5)
Unfortunately, I'm looking to migrate an existing site that doesn't have those nice extension-less URLs, and I need to preserve the existing URLs by mapping directly from pages/about.md to /about.html.
This PR adds a new option to the site.config.js file in the build section: cleanUrls which defaults to true, preserving the existing behaviour of generating clean extension-less URLs.

Thanks for the contribution! I'll take a closer look later today and approve it.


Multi-valued properties and _

Regarding Multi-valued properties, could you please cover the use of the _ (underscore) operator and explain it in more depth. e.g. "(device.devicePhysicalIDs -any _ -contains "[ZTDId]")"
See the following blog post for an examle of it:
https://blogs.technet.microsoft.com/mniehaus/2018/06/13/autopilot-profile-assignment-using-intune/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5f7a4070-1be8-5f1f-0092-68fb6b9e2305
Version Independent ID: cd34312f-b507-affe-555e-1e0c63e7d494
Content: Attribute-based dynamic group membership in Azure Active Directory
Content Source: articles/active-directory/active-directory-groups-dynamic-membership-azure-portal.md
Service: active-directory
GitHub Login: @curtand
Microsoft Alias: curtand

@gutjahr Thanks for the feedback! I have assigned the issue to the author to investigate and update as appropriate.

Hi @gutjahr I'm investigating this now with the PM.
#in-progress

Hi @gutjahr I'm assured I'll have an answer in a few days. Early next week (July 2 or 3).

Hi @gutjahr I was too optimistic about what gets done around July 4th. I'm almost ready with an answer published in documentation. I'm adding this to link this to my work item DB.
https://identitydivision.visualstudio.com/Technical Content/_workitems/edit/462056
but I do have a pull request too
https://github.com/MicrosoftDocs/azure-docs-pr/pull/46127

It's live now
@MohitGargMSFT #please-close

@gutjahr We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Monthly Chat Agenda January 2023 (2023-01-02 and 2022-01-16)

This is the Agenda for the two Monthly CRS Chats.
The general chat is going to happen on https://owasp.slack.com in the channel #coreruleset on Monday, 2023-01-02, at 20:30 CET. That's the 1st Monday of the month. A separate issue chat is happening at the same location, same time on Monday, 2023-01-16. That's the 3rd Monday of the month. Please note that we have a CRS calendar (maintained by @fzipi).
Archived previous meetings and their decision are here.
What happend in the meantime since the chat last month
Outside development

Setup tutorial for caddy&coraza
Separate ModSecurity rule set for WordPress usage WPRS (by our very own @theMiddleBlue, somewhat outdated, but never reported here before)

Inside development
Rules

The biggest task / roadblocker is the overhaul of 932150 in PR #3061. @theseion will work on this during the week and we will try to review / merge afterwards.
We are not getting any traction with the keyword list updates. It's not quite clear why, but the BB still being open is certainly a problem. There is also the idea to pay somebody to get this done, but we would rather avoid this. But if anybody is interested in that, then please speak up.

CRS Sandbox

@theMiddleBlue is currently creating a new challenge for the SQL in JSON bypass technique, in order to ask to try to bypass the new related rule

CRS Bug Bounty and Security

The PR to solve the 2nd list of bypass findings by  Shivam Bathla's is not merged yet (#2926). But close.
Bug Bounty Findings stats: Coverage at PL1: 78.13% (up from 76.86%), PL2: 94.27% (92.57%), PL3: 94.69% (92.99%), PL4 98.94% (98.51%). Currently, 25 tests fail at PL3. We assume most of these will be fixed by the updated 932150. See above.

Plugins

New task: Now that tests can be added to plugins, add tests to all plugins (#3051).

Moved this lengthy item to project discussions (see below).

WordPress plugin needs to be update with all new FPs found in the latest WP version.

Remove FPs for the top installed WP plugins (SEO, Woocommerce, etc...)
Convert/refactor WPRS project into a CRS plugin (https://github.com/Rev3rseSecurity/wordpress-modsecurity-ruleset)

Documentation and Public Relations

There is an effort on trying to get a release notes in #3072
The INSTALL document needs attention. We currently have four different install guides (INSTALL, 'Quick Start Guide', 'Extended Install', and coreruleset.org/installation/). We should not maintain four guides: we should squash them into a single source of truth for install documentation. @RedXanadu has agreed to look at this some time in the new year; it's a lot of work…
The next developer portrait is going to be released coming weeks

Project Administration and Sponsor relationships

All payments for CRS have been issues, not sure everything arrived before the end of the year.
CRS sponsoring newsletter 2022.Q4 is pending
CRS finances for 2022 not completed; once done 2023 budget has to be drafted

Tools

no news

Testing incl. Seaweed and many future plans

@vandanrohatgi is around again 🎉 . He offered to help with additional Seaweed plans. @fzipi will be working with him in getting additional features and see what he wants to help with.

Containers

There is an outstanding problem with the certificate generation in issue https://github.com/coreruleset/modsecurity-crs-docker/issues/106.

CRS Status Page

A few more 'missing PL 1 test cases' investigated and resolved.

Project discussions and decisions

Should we remove response body audit logging from all rules? (I.e. remove ctl:auditLogParts=+E everywhere.)

We need to be consistent with our rules. We need to implement and document a CRS project policy on this:

CRS rules never audit log responses?
CRS rules sometimes audit log responses? If so, when?

Response bodies can contain sensitive data, so maybe CRS shouldn't ever trigger them to be logged and stored.
Would we need to provide a config item to (re-)activate response body audit logging for users who must keep it enabled?
What about users/integrators who might have a setting of their own?
Associated PR: #3034 (@fzipi note: all four people involved in this PR were in sync that this was needed, so I went there and merged it).

We talked about this in our retreat, but looks like we need a broader discussion, so let's have it: #2948

Now that tests can be added to plugins, add tests to all plugins (#3051).

Realistically, who is going to do this work?
This task needs a person responsible for it or it won't get done. It looks like an enormous amount of work. Should we pay someone to do this project? Last year we had budgeted money for 'projects' that we did not use. Open for discussion.
What is needed?
Step 1: Copy the template-plugin github actions workflow directory to the destination plugin.
Step 2: Create the tests subdirectory 👉 tests/regression/<the plugin name>
Step 3: Create a test file for at least one rule. What are we testing? That the plugin works. Meaning? If this is for an exclusion rule, then use any REQUEST_URI that we are removing rules from, and then see that after GET/POST ing to that url with whatever the rule is targeting, no_log_contains the rule we are excluding.
Is there any example of this crazy magic? Yes, please see the one in the phpmyadmin-rule-exclusions-plugin
But wait a minute, do we need to add a test for all rules also? 👉 @fzipi suggests we delay doing this. We need to test that the plugin itself is working, we don't need 100% coverage now. We are testing that our decoupling from the core still makes the plugin work.
So it is just copying that .workflows directory then, and writing ONE TEST FILE with ONE TEST INSIDE then? Pretty much, yes.
And what about complex plugins that require Lua extensions? We cannot cover those now. That's future work. As we are using the "regular" CRS docker, for that we will need to extend it to have the new lua files, etc. This is future work.

Rules development, key project numbers
PRs that have been merged since the last meeting

#3034
#3026
#3048
#3043
#3067
#3063
#3066
#3052
#3055
#3058
#3057
#3020
#3053
#2946
#3041
#3050
#2573

We merged 17 PRs since the last monthly project chat.
Open PRs
Open PRs marked DRAFT or work in progress or needs action

#2933
#3059
#3040
#3056
#3045
#3019
#3029
#2954
#2878
#2948
#2637
#2666

Open issues and PRs

As of Monday, we have FIXME open issues.
As of Monday, we have FIXME open pull requests.

Separate 2nd Meeting (Monday, 2023-01-16)

FIXME: Agenda to be defined

How to get to our slack and join the meeting?
If you are not yet on the OWASP Slack, here is your invite: https://owasp.org/slack/invite .
Everybody is welcome to join our community chat.

Decisions Jan 2

Decisions Jan 16 (issue chat meeting)

BB Status of refactoring of rule 932150 and friends is the final showstopper. @theseion is working on this, with @fzipi and @dune73 waiting for a PR they volunteered to review.
#3079 ->
Unless there is someone who is keen to fix a corner case bug with the IP reputation rules in CRS 3.x, I think this issue will realistically become a 'Won't fix' issue.
-> Decision: Wont Fix' and we flag it in the future if someone writes a plugin for IP reputation.
#3087
@anuraaga from Coraza proposed a feature request which replaces ARGS/ARGS_NAMES with ARGS_GET/ARGS_GET_NAMES and ARGS_POST/ARGS_POST_NAMES. We need to think deeply about the consequences.
Initialization of collections (see https://github.com/coreruleset/coreruleset/issues/3068)

CRS no longer needs collections, but plugins may want to work with collections
** -> Decisions**: CRS initializes them for the plugins, off by default

CRS Community Summit in Dublin Feb 14: Participation, staying (AirBnB?), program
We stated we want to get serious about Coraza on NGINX in 2023. Is it time to start?

we provide a collection initialization plugin, we need to find a way to make sure it runs before the plugins that need it.
20:42 Uhr
And - besides - we defined that plugins should try to live independently, this not interfering with one another.
20:42 Uhr
With that being said, I personally lean somewhat in the direction of CRS doing the initialization, if an optional variable is set (off by default).


”Bunești-Averești” is listed in the drop-down menu

Description:
"Bunești-Averești" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Bunești-Averești" in the search bar.

Expected results
The location is listed in the drop-down menu as "Comuna Bunești-Averești, VASLUI".


Allow the user final say with the CLANG_FORMAT_DIFF variable

I'd like to run my clang-format-diff with a special Python, and the
previous code would strip out my special Python and run it with whatever
'python' binary was in the path.
This allows the user final say over how to run clang-format-diff, and
brings the behaviour in line with what the documentation implies.

Thank you for this tool, by the way!

Thank you! The patch looks good!

I made a small change to make bash's variable declaration happy. Just accessing $CLANG_FORMAT_DIFF fails if it doesn't exist. ${CLANG_FORMAT_DIFF:-} fixes it (That :-} looks like a smiley...).


typo in kubelet --help

kubelet --help prints

Other than from an PodSpec from the apiserver…

an PodSpec → a PodSpect

Hello,
This is a mirror repo and is not monitored: https://github.com/kubernetes/kubelet#where-does-it-come-from
Please file issues against https://github.com/kubernetes/kubernetes/issues
/close


chore(readme): fixing npm command to start dev server

I've updated the Readme and fixed the command to start the dev server 🫡

@gruselhaus these have been fixed since. Thanks for the PR though.


Setup code coverage

Use an automated code analysis/quality tool to get static analysis, cyclomatic complexity, duplication and code unit test coverage changes.

Using Codacy as default code coverage application.

Since there is actually no unit test on this project, the Codacy code coverage can't be enabled, see #100 for more informations.

Project quality fixed on Codacy. Closing the issue.


Additional test cases and adding configuration options

Summary

Checklist
Delete items not relevant to your PR:

[ ] Unit and integration tests covering the common scenarios were added
[ ] A human-readable description of the changes was provided to include in CHANGELOG
[ ] For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials

@Paultagoras we did not updated CHANGELOG.md


() nil exception

I am looking for some guidance. I have been using the exceptional(!)-php interface for a few months now and enjoying it, but I am getting exceptions reported at get exceptional.com frequently that look like (screenshot sorta from get exceptional.com):
BACK TO OPEN EXCEPTIONS
() nil
11 hours ago
2 months old
Number of exceptions: 2942 Number of comments:0
() nil
Occurrences
NEWER 10 of 30 OLDER
Time
07 Dec at 11:17AM
Params
{ }

Given the total lack of details I am trying to figure out how to identify whether these are...
1- My code doing something odd in throwing an exception (my suspicion but where??)
2 - The exceptional-php interface to getexceptional.com handling an exception in a strange way ...
3 - something on getexceptional.com misrepresenting my exception
As you can see I have 2942 of these at present. I get enough exceptions using this tool that merely looking for what happened at that time is leading me nowhere, and throwing an exception in exceptional-php is probably not the proper approach. How would you suggest I locate the root cause???
Thanks for a fine interface product and any help.

Hey TJ, I'm leaning towards 2 or 3. The library should handle all types of exceptions gracefully as long as the setup files parses correctly. Can you upload a screenshot and post the link?

I have posted a screenshot for what it is worth - no more data than I entered above. Hence why I am at a loss to locate... Let me know what else would help.
http://www.schellingpoint.com/images/screenshot_742.png

Thanks, I'll talk with the Exceptional team and see what's going on.

Hey TJ, I tried recreating the bug with no success. What version of PHP are you using?
One thing you could try is writing all exceptions to a log file. The best place to do that is line 85.
https://github.com/ankane/exceptional-php/blob/master/exceptional/data.php#L85
If you log the $data variable, you can see what the PHP library is sending to the server. $data includes the time of the exception, so you can try matching the times with those on getexceptional.com.

Well - that was going to be my next option. SO I will try to get that into place ASAP. I'll let you know what I find.
On Mon, Dec 12, 11, at 3:01 AM, Andrew Kane wrote:

Hey TJ, I tried recreating the bug with no success. What version of PHP are you using?
One thing you could try is writing all exceptions to a log file. The best place to do that is line 85.
https://github.com/ankane/exceptional-php/blob/master/exceptional/data.php#L85
If you log the $data variable, you can see what the PHP library is sending to the server. $data includes the time of the exception, so you can try matching the times with those on getexceptional.com.

Reply to this email directly or view it on GitHub:
https://github.com/ankane/exceptional-php/issues/9#issuecomment-3103088

Any update on this?

Not ignoring your request - just have been swamped.
Am getting back to this now. I have 2 types of errors: one which is  "Undefined Offset" that always references exceptional itself line 71 and that is it. And The second is the nil problem. The nil ONLY occurs on Production, so I have to be careful about that. The other happens all the time.
So I am going to make a change on test and see if I can find the undefined offset issue (appears it could be a shutdown related issue, but the offset referenced varies).

On another note I was not getting my controller sent to getexceptional.com and I stumbled over the fact that the controller AND action must both be set or you don't include them (data.php line 70). I am going to change my code to send both, but I think either or for PHP exceptions should still be sent?
Thanks again for your guidance. Responding soon.
Tim
On Sun, Jan 15, 12, at 2:47 AM, Andrew Kane wrote:

Any update on this?

Reply to this email directly or view it on GitHub:
https://github.com/ankane/exceptional-php/issues/9#issuecomment-3497589

I added a separate issue for tracking the similar but distinct undefined offset problem I have. There is somewhat more data for that. IN the next 2 days I anticipate a production release which will add this logging into production, so I can further uncover the "nil ()" problems source. Will keep you updated. Thanks for the guidance.

Great, I'll look into the issues this weekend.

I was able to make the logging available to our production environment (to catch the () nil) and so far none have occurred. WIll let you know when I get a hit.

Sounds good.

Great NEWS!
I got my first nil exception since we have been trying to collect by capturing the array!!
I am forwarding you my exception screen (pretty empty), and the equivalent NIL exception as caught by your suggestion of capturing the array inside your API interface into a log file.
Perhaps you can when time permits take a look at it and see if you can suggest to me why it translated to a nil in getexceptional.com.
http://media.schellingpoint.com/exc/screenshot_745.png
http://media.schellingpoint.com/exc/nil.log
Thanks,
Tim

Wanted to add that if this is the primary source exception, it would explain why it has not been occurring so frequently, because the number of times this code path is used has been significantly reduced over the last 3 months, so the opportunity for this source line to have an exception is far lass frequent.
My inference, without prejudicing your investigation, is this has something to do with the sheer size of the results parameter being posted through this interface and being passed on as part of the exception package to exceptional.com.

Cleaning up issues. RIP Exceptional :(


Fix tests, add Makefile

Currently on main, four tests fail, because the format that is returned from the plugin functions has changed without these tests being updated. This PR updates the tests to pass, utilizing a new helper function. It also adds a Makefile which serves as a rudimentary command runner (and form of documentation) for the test and format commands.
I added the format command into the Makefile, but did not run it. It's going to make a reasonable size diff, which should be done independently. I personally feel we should rip that bandaid off, but I'll leave that decision to @cthulahoops .
I'd also like to add a GitHub Action which will run the tests, if that contribution is welcome. This would help prevent further test regressions.

I'd also like to add a GitHub Action which will run the tests, if that contribution is welcome. This would help prevent further test regressions.

I'd appreciate that. I don't think I've ever set it up, so it would be good to see how it's done.

Awesome, I'll add the GH Action in a separate PR @cthulahoops , they're a little finicky being YAML and all, so it'll take me a few commits in my fork

I've merged this by hand because I wanted to have access to the working tests... not sure how to convince Github that this is merged other than by closing.

Hi, @ntietz. We had a coding with LLMs pairing session and ended up asked GPT to write this: https://github.com/cthulahoops/chatcli/blob/main/.github/workflows/test.yml

Oh cool! I'm glad that it exists now @cthulahoops :smile: I wanted to get back to that but life got in the way, sorry for leaving you hanging on it.

No problem, @ntietz, and thanks for suggesting. It's remarkably easy to get working. :grin:


No Output is given on Output 2

When using the scheduler only output 1, "Outputs new schedule", seems to do anything
Output 2 never gives any output
Really liking the dashboard system, will be great for my project
The flow I'm using is below
[{"id":"92432b2a.a282a8","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"68a5924d.c5851c","type":"comment","z":"92432b2a.a282a8","name":"### This scenario is permanent and requires a file to read an write to ###","info":"","x":360,"y":440,"wires":[]},{"id":"8b78b561.41dae8","type":"inject","z":"92432b2a.a282a8","name":"FireAfterReboot","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":0.1,"x":200,"y":500,"wires":[["301cea20.0ae426"]]},{"id":"988a4680.f98f38","type":"ui_time_scheduler","z":"92432b2a.a282a8","group":"c66eb395.fc787","name":"Pump","order":2,"width":"3","height":"6","x":710,"y":500,"wires":[["c8fdc395.71a1b","57bf3c62.a67a64"],["3fbdc154.4e6bde","f428d818.6068a8"]]},{"id":"3fbdc154.4e6bde","type":"ui_switch","z":"92432b2a.a282a8","name":"","label":"Pump","tooltip":"","group":"c66eb395.fc787","order":1,"width":0,"height":0,"passthru":true,"decouple":"false","topic":"","style":"","onvalue":"true","onvalueType":"bool","onicon":"","oncolor":"","offvalue":"false","offvalueType":"bool","officon":"","offcolor":"","x":1030,"y":520,"wires":[[]]},{"id":"301cea20.0ae426","type":"function","z":"92432b2a.a282a8","name":"Get Schedule","func":"msg.payload = global.get(\"schedule\");\nreturn msg;","outputs":1,"noerr":0,"x":440,"y":500,"wires":[["988a4680.f98f38"]]},{"id":"c8fdc395.71a1b","type":"function","z":"92432b2a.a282a8","name":"Set onOff","func":"global.set(\"schedule\",msg.payload);\nreturn msg;","outputs":1,"noerr":0,"x":1040,"y":480,"wires":[[]]},{"id":"57bf3c62.a67a64","type":"debug","z":"92432b2a.a282a8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1050,"y":440,"wires":[]},{"id":"f428d818.6068a8","type":"debug","z":"92432b2a.a282a8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1050,"y":580,"wires":[]},{"id":"c66eb395.fc787","type":"ui_group","z":"","name":"PermDemo","tab":"885c13b9.5ae63","order":5,"disp":true,"width":"6","collapse":false},{"id":"885c13b9.5ae63","type":"ui_tab","z":"","name":"Scheduler Demo","icon":"dashboard","order":1,"disabled":false,"hidden":false}]

Okay so the debug node at output 1, shows the following payload after I added one timer (default settings) via the dashboard (with your flow):
"{"timers":[{"starttime":42720000,"endtime":43020000,"days":[0,0,0,0,0,1,0],"sendContinuous":true,"onlySendStart":{"isActivated":false,"valueToRun":true}}]}"
and output 2 sends
true
at the defined start time which is correct.

This is the same methodology as I used to test but I do not receive anything on the second debug node
"{"timers":[{"starttime":49140000,"endtime":49440000,"days":[0,0,0,0,0,1,0],"sendContinuous":true,"onlySendStart":{"isActivated":false,"valueToRun":true}}]}"

Definitely strange, since I have tested this on multiple devices and virtual machines.
Could you try to create a timer that has all days selected, and see if it works then?
What operating system/device are you using?

Timer using all days gives same result as does setting every day 00:00 to 23:59
I'm using NodeRED docker hosted on Unraid OS

So I installed an unraid trial via virtualbox and installed the node-red docker container.
The timer didn't work as expected, but like I suspected it seemed to be a date/time problem.
Can you add this to your flow and tell me if it outputs the correct local time?
[{"id":"19db767e.566f9a","type":"ui_text","z":"a9305d8b.a333a","group":"50aa414d.fc336","order":2,"width":0,"height":0,"name":"","label":"TimeNow","format":"{{msg.payload}}","layout":"row-spread","x":680,"y":260,"wires":[]},{"id":"e6f5f011.5054a","type":"inject","z":"a9305d8b.a333a","name":"FireAfterReboot","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"1","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":170,"y":260,"wires":[["73735a40.187cf4"]]},{"id":"73735a40.187cf4","type":"function","z":"a9305d8b.a333a","name":"GetTime","func":"const days = ['SU','MO','TU','WE','TH','FR','SA'];\n\nconst date = new Date();\nconst today = date.getDay();\nconst currentHour = padZero(date.getHours());\nconst currentMinute = padZero(date.getMinutes());\n\nmsg.payload = days[today] + \" - \" + currentHour + \":\" + currentMinute;\nreturn msg;\n\nfunction padZero(i) {\n    return i < 10 ? \"0\"+i : i;\n}","outputs":1,"noerr":0,"initialize":"","finalize":"","x":380,"y":260,"wires":[["19db767e.566f9a"]]},{"id":"50aa414d.fc336","type":"ui_group","z":"","name":"PermDemo","tab":"681cf323.a5097c","order":5,"disp":true,"width":"6","collapse":false},{"id":"681cf323.a5097c","type":"ui_tab","z":"","name":"Scheduler Demo","icon":"dashboard","order":1,"disabled":false,"hidden":false}]
In order to get in run, I had to change the timezone in the unraid dashboard (settings-date and time) and then !reinstall the node-red docker container. A restart had no effect. Please let me know if this helps.

Reinstalling NodeRed docker did fix the issue
The Flow above always displayed correct local time so I'm unsure why the main flow wasn't working before


Anerdw: Watch eftkar456(?:@gmail.com)?

Anerdw requests the watch of the watch_keyword eftkar456(?:@gmail\.com)?. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

eftkar456(?:@gmail\.com)? has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Cow in Charcoal HQ


Example Build Fails - Can't Find vscode-languageserver/lib/protocol?

Mac OS High Sierra 10.13.3
npm 5.6.0
node v8.11.1
Code appears to exist, but when running the theia-electron example with npm run start, I get the following error:
Uncaught Exception:
Error: Cannot find module 'vscode-languageserver/lib/protocol'
at Module._resolveFilename (module.js:470:15)
at Function.Module._resolveFilename
Within the theia-electron directory, I do see the *.js file for protocol:
ls node_modules/vscode-languageserver/lib/pro*
node_modules/vscode-languageserver/lib/protocol.d.ts	node_modules/vscode-languageserver/lib/protocol.js
Is there a version dependency on npm/node that would cause this to build improperly? I didn't see any errors during npm install nor npm run build.
thea-app example fails in apparently the same way.

theia-dsl-app@ start:backend /Users/blewis/git/theia-sprotty-example/theia-app
[watch-deps] > node ./lib/backend/main.js --root-dir=../workspace
[watch-deps]
[watch-deps] module.js:549
[watch-deps]     throw err;
[watch-deps]     ^
[watch-deps]
[watch-deps] Error: Cannot find module 'vscode-languageserver/lib/protocol'
[watch-deps]     at Function.Module._resolveFilename (module.js:547:15)

Sorry, I'm sure these issues are meant for code base only, but I didn't know who else to advise. Google searches were not any help.

The theia-sprotty-example is build on old versions of the underlying libs and frameworks and is no longer maintained. Sorry.
Please have a look at the @theia/yangster repository for an uptodate integration of theia and sprotty.


[Spike] Server side copy of blob - extended evaluation

Server side copy of blob

Clarified the best way to copy blobs on server side with an Azure Storage developer.
Documented the outcome:

Explained the most optimal way to copy blobs on server side with java SDK.
Added code snippets and tested it.

ADR
Created simple ADR explaining the approach to copy blobs on server side.

Can we some indication of which block size to use? Is something preset in DML/azcopy?

Yes, azcopy uses blocks of ~8 MB, but we didn't find any recommendation in the docs.

Can we some indication of which block size to use? Is something preset in DML/azcopy?

@algattik  Yes, azcopy used blocks of ~8 MB, but we didn;t find any recommendation in the documentation.


Not possible to ingest MLmodels via OpenApi

Describe the bug
There is a bug related to MLModel schema. The structure is ok, I also managed to ingest other entities as glossaryterms following this "structure or instructions" that appear in the swagger of datahub.
The error is always related to the name of the aspect "MLModelProperties". I send the correct name but  it is detected as if it was written with the first m in lowercase.
ERROR : "Request processing failed; nested exception is java.lang.RuntimeException: Unknown aspect mLModelProperties for entity MLModel"
REQUEST BODY:
[   {     "entityType": "MLModel",     "entityUrn": "urn:li:mlModel:(urn:li:dataPlatform:science,sciencePatri,PROD)",     "entityKeyAspect": {       "__type": "MLModelKey",       "platform":"urn:li:dataPlatform:science",      "name":"sciencePatri",      "origin":"PROD"     },     "aspect": {       "__type": "MLModelProperties",       "customProperties": {         "additionalProp1": "value1",         "additionalProp2": "value2",         "additionalProp3": "value3"       },       "externalUrl": "url",       "description":"description",       "tags":["urn:li:tag:hola"]     }   } ]
To Reproduce
Steps to reproduce the behavior:

Go to 'https://demo.datahubproject.io/openapi/swagger-ui/index.html#
Create the tags you are going to associate to your MLModel (tags are a required field)
Create the platform entity where you are going to have the MLModel, or choose the urn of a platform that already exists
Copy the structure attached above in to the POST /entities/v1 query
Check the error in the console

Expected behavior
The response should be 200 sucess and the mlModel should be created.
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

This issue is still a bug. Tested in last version 10.0.0

Also managed to reproduce in 0.10.0, but weirdly enough I managed to create an empty MlModel entity without any aspects or ownership

This is the body I used
[
  {
    "entityType": "MlModel",
    "entityUrn": "urn:li:mlModel:(urn:li:dataPlatform:testplatform,testtest,DEV)",
    "aspect": {
        "__type": "Ownership"
    }
  }
]

But I couldn't add any aspects to it or features, just an empty shell

Thanks!! But yes, still aspects are necessary.


Can't add LP webpart to SharePoint-site even after adding LP app to that site

The LP app is only working for some websites on my tenant.
After adding the LP app, it works on some sites to add the webpart to a site, but on others it simply does not.
How can I fix this issue?

I'm going to need a bit more to go on then it simply does not work.
We have issue templates to try and guide you to give details that will help us help you, but in lieu of that maybe you could explain more about your install, what you've done to get it working in the sites where it doesn't work and what exactly isn't working.

Hi @juliemturner,
thanks a lot for getting back to me so quickly.
We deployed the Learning Pathways as explained in the documentation and also created an app catalog for this. Then, we went to an already existing SharePoint page and added the app to the site. The app got added to the site without any problems. Then, we opened an existing site and clicked on "edit" and then on the "+" to add a webpart to the site.
Unfortunately, the LP web part icon did not show up in the pop-up menu, and we were unable to add the LP webpart.
We then went to a different, less frequented SharePoint site with the same user (SharePoint admin) and added the LP app to this site as well. Afterwards, we again tried to add the LP webpart, and on this site it showed up in the pop up menu and we were able to add it.
As a result, on the more frequented site which is a main communications site at the customer, we were unable to add the LP webpart but were able to add the LP app.
In the less frequented site, we were able to add the LP webpart and the LP app as regular.
Thanks a lot for your help and looking into this!

I think you're missing the step that you need to add the Microsoft 365 learning pathways app to the site, once you do that then the web part will be in the gallery.  The solution isn't deployed to the whole tenant by default so you need to add the app on a site collection by site collection basis.

Hi @juliemturner,
as written, we have added the LP app to each site we wanted to add the web part to.
For one site, where the LP webpart needs to be deployed, the web part nevertheless did not show up in the pop-up menu for adding a new webpart.
At another site, we followed the same steps (adding the app to the site, then adding a web part), and here the LP webpart did show up in the list of all available web parts.
How can we ensure that for the site where the LP web part did not show up in the list of all web parts, that it will be available?

um... the only reason I can think of is that you didn't wait long enough for the app to get deployed.  Validate that the Microsoft 365 learning pathways app is in the site collection... if that's true then if you navigate to a page and edit it and then add the web part it should be there.

I'm going to close this issue as answered. If you have further issues, please feel free to create a new issue and reference this one.


Synchronize cursorline, cursorcolumn when cursorbind is set

When cursorbind, cursorline, cursorcolumn are set in two windows, cursorline, cursorcolumn does not synchronize unless some events (e.g. scrolling or switching window) occur.
The gif of example is shown at vim-jp/issues#1009

for the sharing issue
before

after

Masanori Misono wrote:

When cursorbind, cursorline, cursorcolumn are set in two
windows, cursorline, cursorcolumn does not synchronize unless some
events (e.g. scrolling or switching window) occur.
The gif of example is shown at vim-jp/issues#1009

To me it looks like only the addition of validate_cursor() is needed.
Redrawing with VALID does appear to work.
--
hundred-and-one symptoms of being an internet addict:
260. Co-workers have to E-mail you about the fire alarm to get
you out of the building.
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Hi Bram,

To me it looks like only the addition of validate_cursor() is needed.

You're right.
I confirmed that just adding validate_cursor() works as intended.
diff --git a/src/move.c b/src/move.c
index 912e51ccd..7519da4c9 100644
--- a/src/move.c
+++ b/src/move.c
@@ -2841,6 +2841,7 @@ do_check_cursorbind(void)
            restart_edit_save = restart_edit;
            restart_edit = TRUE;
            check_cursor();
+           validate_cursor();
            restart_edit = restart_edit_save;
 # ifdef FEAT_MBYTE
            /* Correct cursor for multi-byte character. */


some style nits

basically just cleared up the intellij warnings in two files I was working in for #1277:

unused imports
many no-op else records branches of RDD-instrumenting-if-statements
some .size->.length changes
some unnecessary .toSeqs

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1621/
Test PASSed.

Would it be ok to close this without merging?

@heueh +1


Initial Mosek interface (SOCP only). With tests.

I tested it and it works.
Install Mosek https://www.mosek.com/resources/downloads and get a free academic license.
So far I only saw improved performance (over both CVXOPT and ECOS).

looks great!

Thanks!
next steps:

integer programming (should be fairly straightforward)
SDP (I still have to understand a couple of things, but I don't think it will be more than another 50 lines)
solver options (I'm using mosek for work right now, will figure something out).

Enzo

On Aug 4, 2015, at 15:48, Steven Diamond<EMAIL_ADDRESS>wrote:
looks great!
—
Reply to this email directly or view it on GitHub https://github.com/cvxgrp/cvxpy/pull/211#issuecomment-127733177.


Copy cell formatting from one cell to another

Im having an issue copying the cell formatting from a formatted  cell to an adjacent blank cell, when I attempt to copy the cell style using
$Excel = Open-ExcelPackage -Path 'C:\Temp\Excel-Doc.xlsx
$Excel_CellStype = ($excel.sheet1.Cells[11,3]).Style
($Excel.sheet1.Cells[12,3]).Style = $excel_Cellstyle
but I am getting error:
InvalidOperation: 'Style' is a ReadOnly property.
if I try doing the same with sub portions its successful-ish, as I can copy the Style.fill color, but with only that once closed the cell doesnt match so im assuming there is more areas I need to copy but am hitting a bit of a wall.

Have not tried that. Style is a top-level object and many properties/nested properties underneath. Using ($Excel.sheet1.Cells[12,3]) | Get-Member, Style does not have a set on the property, makes it read-only.
($Excel.sheet1.Cells[12,3]) | Get-Member * -force, there are some set_* methods, few for *Style* type properties.
You'll probably need to set the ones you want individually.

Hi @dfinke,
I was able to lookthru Set-Cell's body and found that I can use $Excel.sheet1.Cells[12,3]).Style.Fill.Backgroudcolor.rgb to get the color then with a table of rgb to color get the color name and then use $Excel.sheet1.Cells[12,3]).Style.Fill.Backgroudcolor.setcolor("Yellow") for example to set the color.
Thank you for responding im hoping this will help others as well.


Plot brush sometimes "sticky" and doesnt clear as expected

I'm not entirely sure how to reproduce this, but sometimes the previous selection will not be cleared when a second selection is made. Very rarely, the previous selection will be used as the active selection instead of the newer selection.

This issue is still around and I'm still unable to reproduce it, but I've been told by others that it occasionally happens to them as well. I've added a button to clear the selection, so hopefully that works as a bandaid for the time being. Apparently the plot brush doesn't distinguish between a click event and a click+hold (for brushing), so I can't map session$resetBrush to a click event.

See also discussion here, I think it's referencing the same problem: https://github.com/rstudio/shiny/pull/3683

Anecdotally this seems to happen very frequently for MacOS users, and may be related to moving between windows or desktops. Sadly the implemented clear selection button doesn't target the sticky boxes since they're no longer related to input$plot_brush (which I half expected but nice to have confirmation).

Based on the idea in the above comment, I've applied a band-aid fix such that pressing the clear select button will not only reset the current brush but it will also run a line of javascript to delete the div with the id pulsePlot_brush. When there's a "sticky" box there are 2 divs with the same id, but running session$resetBrush should remove the first/active brush div, then the javascript should target the remaining one. However, you'd need to keep pressing it if there are more on the screen.
This is still difficult to reproduce, but you can hack it into existence by drawing a brush, opening up the HTML in the dev tools, copying the pulsePlot_brush element, then pasting it after making a new box. The pasted box will persist. If you drag a new box, then navigate to another file, then that new box will also start to persist. In other words, once you have one sticky box, you'll start getting a lot more sticky boxes:

I'll push a version that uses a loop to delete all the boxes at once in a moment. In JS, document.getElementById("pulsePlot_brush") will return null if no element with the given id exists.

This is now fixed in the sense that there exists a solution to get rid of the boxes, but I do think the underlying issue is related to the plot brush debouncing behavior described in the cited PR from before. So, we have a band-aid of a fix now and until Shiny itself is updated to fix the brush I don't think it's going to be solved.


index.html should be a template

How to make the index.html page template, for example, I hope to build a package, create a meta field, for example, like this below.
<meta name="releaseDate" content="2017-12-06 19:21">
For example, I played this excellent demo-angular-starter, it supports because he himself is by this plugin-html-webpack-plugin , so that we can use html expressions in the page, according to the definition of variables to make judgments, Output different html. In fact, the above is a very normal demand.
demo-angular-starter

https://github.com/angular/angular-cli/issues/1509 related.

my solution https://github.com/alanhg/angular-demo/issues/32

Any progress on version 6?

Is there any good workaround for this?

@filipesilva some news here? What does Angular Team think about it?

Closing as this is not currently in the CLI roadmap to provide this option out of the box.
This can currently be achieved by used a 3rd party builder that extends the browser builder to transformer the index.html.


Configure timeout

Fixes #80. Defaults are as before - 5 seconds for Android, Blackberry and Windows (since they use the Buzz client), and 60.
ping @andrewsmm @emiliomg - could you both test and see if this helps you?

Thank you. I will review this PR in the evening :-)

The PR does not work yet, you did not set the necessary "rms_push_notifications.INSERT_PLATFORM_HERE.timeout" parameter in RMSPushNotificationsExtension.php, so the DI cannot resolve the requested parameter key.

Ah yeah, that'll teach me to try and do it quickly :-) I've added it now, could you check?

Thank you. Yes, I will check it out as soon as I can.

Works fine with android. Didn't test the rest ;-)

i raised the limit to 120 (for iOS) but it still timeouts. maybe my problem is somewhere else, but where? sometimes notifications pass, sometimes they don't :/

So, can this branch be merged? :)

Sure thing.


The operation couldn’t be completed. (ABAddressBookErrorDomain error 1.)

on IOS i am getting this error. did not understand

what is your code?


fix: Use bytes instead of string comparison in delete validator queue (backport cosmos/cosmos-sdk#12303)

Description

Cherry-pick of:

cosmos/cosmos-sdk#12303

Motivation and context

How has this been tested?

Screenshots (if appropriate):
Checklist:

[x] I followed the contributing guidelines and code of conduct.
[x] I have added a relevant changelog to CHANGELOG.md
[ ] I have added tests to cover my changes.
[ ] I have updated the documentation accordingly.
[ ] I have updated API documentation client/docs/swagger-ui/swagger.yaml

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please ignore CLA.

Please ignore coverage on panic scenarios.


Swagger ingest api doc

A re-issue of @prabh-t's branch that would be otherwise lost due to a rewind of the master branch.
I suggest we either review when we can or once we update to the latest springfox version (3.0.0, currently 2.9.2) to address the issue mentioned in the doc https://docs.google.com/document/d/1x6qOQ_u6GoMzTYyK7u5hffmOiHkY9fOLYuMq7BiHOQw.

Give me a chance to test this on Monday. It is possible that this change was responsible for some odd functionality I was seeing in endpoints of ingest-core. Just want to prove it was not this change before merging.