Mapping UK

New interface with RtD3 works well apart from in the UK. Shapefiles are mostly a closed book to me so any ideas appreciated. May be partially fixed by upstream work on RtD3 to making linking easier. Any ideas @kathsherratt  or @hamishgibbs?

If data is coming in NHS regions, not Local Authorities, you will need a custom shapefile of those regions in order for the map and plots to play together correctly. I am not familiar with your pipeline so not sure how challenging it is to add custom geodata.
Something else that I have found working with NHS regions  is that they change often so important to make sure that that we use the correct shapefile (RtD3 should warn about name issues).

Yes all good points. @kathsherratt built one out that worked well but for unknown reasons I couldn't link it with RtD3 when we rolled that out. I'll have a look around and try and work out what the problem was (🤓 ).

Mapping temporarily removed from the site due to instability issues.


Failed to get registration from op code CUSTOM

Hi developers, hi all!
I'm making some experiments with LSTM networks using Keras, TensorFlow, and I'm working on a Colab environment, using TF 2.11.0-dev20220915 installed via pip.
On the model authoring side, I'm able to successfully build the model, apply compression techniques and get the equivalent Lite model, and the .cc file. Specifically, starting from the base model I apply the pruning technique first, and then fully integer quantization. During the conversion process to Lite model, I do not get any errors.
However, when it's time to flash the model onto the device (in this case,  an ESP32), I always get the following error Failed to get registration from op code CUSTOM AllocateTensors() failed.
Is this error generated because there is something in the model not yet supported? is there a way to discover which is, or are, the OPs that are not supported?
I really would like to understand the reason for this error and how could I solve it.
Here attached you can find a zip folder with two .tflite files (one it's related to the model not quantized yet and the other to the quantized one), the third one is the .cc file of the quantized model.
material_pruning-fullyintquant.zip

Hi @ChicchiPhD !
you can use netron app to see any operator which are not either supported in select ops or built in list.
I found a SO thread with similar stack trace which suggest on porting lite to micro ops (perhaps you are using some flex ops)
Could you also  attach the C++ inference code used in ESP 32 inference.
Thank you!

Hi @mohantym,
thank you so much for your reply, and for pointing all the links out. I've actually realized that one of the sources of the problem was related to the fact that I had not a fixed input size in the model. I was able to repeat all tests, without the need to use tf.lite.OpsSet.SELECT_TF_OPS on model authoring side, during conversion.
Everything works fine, so, I'm able to apply the pruning technique first, fully integer quantization then, and obtain .tflite .cc files. But when I flash the model onto the device, I get a new error:
**VAR_HANDLE requires resource variables. Please create ResourceVariables and pass it to the interpreter.
Node VAR_HANDLE (number 1f) failed to prepare with status 1
**
Here attached you can find a zip folder containing the two tflite models (one of the pruned model but not quantized yet, the other one of the pruned and quantized model), and the two .cc file (one is related to the pruned model but not quantized yet, the other one of the pruned and quantized model). In both cases, I get the same error.
I've found this ticket https://github.com/tensorflow/tensorflow/issues/56932 related to the same problem. However, for me, the proposed solution did not work. Not only the proposed solution did not work, but the size of both .cc file increased a lot (around 7 MB), so that I was not even able to run them on the device.
1_pruning_fullyquant.zip

Ok @ChicchiPhD !
Thanks for the update. Could you convert the model to TFlite again with below flag and let us know the inference result.
converter.experimental_enable_resource_variables = True # Ref
Thank you!

Hi @mohantym,
if I try to convert the model to TFLite with that flag, I still get the same error
**VAR_HANDLE requires resource variables. Please create ResourceVariables and pass it to the interpreter.
Node VAR_HANDLE (number 1f) failed to prepare with status 1
**
However, I've done several attempts and I've noticed the following behaviors:

whether I use the flag or not, if I apply pruning and then TFLite conversion, or pruning, followed by fully integer quantization and then I apply TFLite conversion, onto the device I always get the VAR_HANDLE error, in both cases;
whether I use the flag or not, if I do not apply pruning technique but only quantization, or none of the two techniques, and then I apply TFLite conversion, onto the device I do not get any errors.

So, it seems something related to the fact that I apply the pruning technique.
Thank for your support!

Ok @ChicchiPhD !
Thanks for the update.
Can we consider this as resolved then as it seem you were able to resolve this issue by skipping prunning technique/other methods.
Thank you!

Hy @mohantym,
I would say that skipping pruning is particularly a workaround, more than a solution, but at the moment is fine for me. Investigating why the error happens when pruning is applied would be interesting, but we can consider this solved.

@ChicchiPhD !
Please share your pruning code snippets to check for any issue.
Thank you.

I have the same problem, I have a Keras model converted to tflite and then to cc array, but the convertion to tlite model is simple without any addition flags, and I'm getting the same error but without solution
Failed to get registration from op code CUSTOM

@fathiabdelmalek is that an LSTM model? Can you share the output from netron app?

I have the same problem. I have added the code I am working on and the SVG of the Netron app. There re two models that I need to run on my ESP32C3.
Arduino_TensorFlowLite-1.15.0-ALPHA.zip

error:
ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0x1 (POWERON),boot:0xc (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5820,len:0x458
load:0x403cc710,len:0x814
load:0x403ce710,len:0x2880
entry 0x403cc710
Failed to get registration from op code CUSTOM
AllocateTensors() failed
models:
def create_state_model(input_dim):
model = Sequential()
model.add(Reshape((input_dim, 1), input_shape=(input_dim,)))
model.add(LSTM(64, return_sequences=False))
model.add(Dropout(0.3))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.3))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
def create_temp_model(input_dim, output_dim):
model = Sequential()
model.add(Reshape((input_dim, 1), input_shape=(input_dim,)))
model.add(LSTM(64, return_sequences=False))
model.add(Dropout(0.3))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.3))
model.add(Dense(output_dim, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
This is the code that I have used for conversion:
converter = tf.lite.TFLiteConverter.from_keras_model(state_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,  # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS  # enable TensorFlow ops.
]
state_model_tflite = converter.convert()
Save the model.
with open('state_model.tflite', 'wb') as f:
f.write(state_model_tflite)
converter = tf.lite.TFLiteConverter.from_keras_model(temp_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,  # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS  # enable TensorFlow ops.
]
temp_model_tflite = converter.convert()
Save the model.
with open('temp_model.tflite', 'wb') as f:
f.write(temp_model_tflite)

I am facing the same issue and its an lstm model.


Add ProposalCreator

Depends on #146 and #150.

[x] docs
[x] tests

Since it is confusing to separate PRs into multiple ones, I will close this.


Add support for linux gnome

All wallpaper setting logic has been moved to wallpaper.py
Wallpaper can be changed using the set_wallpaper(file) function exposed. The function changes depending on the operating system, so other scripts dont need to be aware which platform the system is running on.
Usage is as simple as:
from pathlib import Path
from wallpaperChanger import wallpaper

wallpaper.set_wallpaper(Path('path/to/file'))

The function also checks if the specified file exists, if not found, raises FileNotFoundError

Alright, looks good! I'll merge this now.
Thanks for contributing!


provisioning windows nodes with custom scripts during cluster creation

Hello!
Is there any approach to apply few lines of code during nodes creation process. Do you have some field like a user_data inside agent_pool_profile or windows_profile blocks?
I can solve the problem with helps ssh pipe to Windows node, but it's a bit complex, when i will try to autoscale nodepool.
Thank you!

Hi @ivanguravel ,Thansk for your suggestion. Currently, the field "user_data" is not available in azurerm_kubernetes_cluster.


Release 2.1.1

Issues: https://github.com/cerner/cerner_tomcat/issues?q=milestone%3A2.1.1
Test Evidence: https://gist.github.com/bbaugher/5b352b06672925e40ba1e71dde845613

+1

+1

Released


Added contrbuting.md

Added CONTRBUTING.md file and created a image folder inside public folder for images related to CONTRBUTING.md.

Hey @OverthinkersArea thank you for the wonderful contribution I'm going ahead and merging this pull request

@all-contributors please add @OverthinkersArea  for Documentation

pls add  "hacktoberfest-accepted" label to this pull request @monizb

Hey @OverthinkersArea thank you for the wonderful contribution I'm going ahead and merging this pull request

thanks man

pls add "hacktoberfest-accepted" label to this pull request @monizb

Hey @OverthinkersArea you dont need that label as long as it has been merged


AttributeError: 'LSTM' object has no attribute 'weight_hh_l0'

I'm using pytorch 1.4.0 and cuda 10.1 now. When I run
python main.py --batch_size 20 --data data/penn --dropouti 0.4 --dropouth 0.25 --seed 141 --epoch 500 --save PTB.pt

I get following error message
Producing dataset...
Applying weight drop of 0.5 to weight_hh_l0
Applying weight drop of 0.5 to weight_hh_l0
Applying weight drop of 0.5 to weight_hh_l0
[WeightDrop(
  (module): LSTM(400, 1150)
), WeightDrop(
  (module): LSTM(1150, 1150)
), WeightDrop(
  (module): LSTM(1150, 400)
)]
Using []
Traceback (most recent call last):
  File "main.py", line 144, in <module>
    model = model.cuda()
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 304, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 201, in _apply
    module._apply(fn)
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 201, in _apply
    module._apply(fn)
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 201, in _apply
    module._apply(fn)
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 137, in _apply
    self._flat_weights = [getattr(self, weight) for weight in self._flat_weights_names]
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 137, in <listcomp>
    self._flat_weights = [getattr(self, weight) for weight in self._flat_weights_names]
  File "/data/scratch/owaski/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 576, in __getattr__
    type(self).__name__, name))
AttributeError: 'LSTM' object has no attribute 'weight_hh_l0'

I guess that's a problem with newer version of pytorch. Can someone tell me how to fix this?

Updating to pytorch 1.5.0 solves this issue.


When to use the MuJoCo viewer or renderer, in particular in combination with MJX

Hi,
I'm a student and I'm trying to use MuJoCo for a space robotics application.
I have a few questions about the MuJoCo viewer and renderer:

Is it possible to add geometries to the renderer in a similar fashion to viewer.user_scn.geoms, or do you have to change the scene by adapting renderer.scene.geoms? If so, is the scene reset again after calling renderer.update_scene(data, cam)?
Is there a way to save an image of what happens in the viewer, similar to renderer.render().copy()?
Is it possible to use the MuJoCo viewer in combination with MJX? Because if I understand correctly, after using the step function in MJX, you would have to re-allocate mujoco_data using mjx.get_data(mujoco_model, mjx_data) (at least that is the way with the renderer), but the changes in the data do not seem to be transmitted to the passive viewer.
What is the best way to change mjx_data.ctrl field since it is frozen? Would that be something along the lines of mjx_data = mjx_data.replace(ctrl=jax.numpy.asarray(your_ctrl))?

Thank you very much in advance.

Hello - I can answer your last two questions:

Is it possible to use the MuJoCo viewer in combination with MJX?

Yes, see an example of MJX hooked up to drive the simulate viewer here:
https://github.com/google-deepmind/mujoco/blob/main/mjx/mujoco/mjx/viewer.py
You can start this via the command mjx-viewer which is installed in your path when you pip install mujoco-mjx.

What is the best way to change mjx_data.ctrl field since it is frozen?

Yes, that's right.  JAX is designed to be functionally pure and that extends to its pytrees - replace ensures you're creating a copy rather than modifying the original.

Regarding the first question: Yes. mjv_updateScene completely redraws the scene from scratch each time. It should really be called mjv_drawScene since there is no state from the previous call, "update" is confusing.
Regarding the second question, I'm not sure, maybe @kevinzakka knows?

Thank you very much for your quick responses. Using mjx.get_data_into(d, m, dx) fixed the visualisation issue in MJX.

Hi,
I saw the example of viewer abow from Erikfrey, thank you.
I wanted to write a simpler code like this one:
import jax
import mujoco
from mujoco import mjx
import mujoco_viewer

mj_model = mujoco.MjModel.from_xml_path("data/cubes.xml")
mj_data = mujoco.MjData(mj_model)
viewer = mujoco_viewer.MujocoViewer(mj_model, mj_data)

mjx_model = mjx.device_put(mj_model)
mjx_data = mjx.device_put(mj_data)
jit_step = jax.jit(mjx.step)
mjx_data = mjx.put_data(mj_model, mj_data)

with_mjx = False
while viewer.is_alive:
    if not with_mjx:
        # ok without MJX
        mujoco.mj_step(mj_model, mj_data)
    else:
        # does not work with MJX
        mjx_data = jit_step(mjx_model, mjx_data)
        mj_data = mjx.get_data(mj_model, mjx_data)
    viewer.render()
viewer.close()

It works without MJX but with MJX the scene is not displayed.
I transfer the Jax-MJX data to reguler mujoco data with get_data but that doesn't seem to be enough ?
Thanks,

Hi! Not sure why the code using mujoco_viewer does not work, but this should:
import jax
import mujoco
from mujoco import mjx

mj_model = mujoco.MjModel.from_xml_path("data/cubes.xml")
mj_data = mujoco.MjData(mj_model)

mjx_model = mjx.put_model(mj_model)  # mjx.device_put was deprecated
mjx_data = mjx.put_data(mj_model, mj_data)
jit_step = jax.jit(mjx.step)

viewer = mujoco.viewer.launch_passive(mj_model, mj_data)
with_mjx = False
with viewer:
    if not with_mjx:
        mujoco.mj_step(mj_model, mj_data)
    else:
        mjx_data = jit_step(mjx_model, mjx_data)
        mj_data = mjx.get_data(mj_model, mjx_data)
viewer.close()


Dependency: Internal error

Library was working perfectly fine, suddenly i am getting those errors after scraping a number of tweets, what does that mean?
2023-10-12 17:12:17.427 | WARNING  | twscrape.queue_client:req:199 - Unknown error <class 'twscrape.queue_client.ApiError'>: ApiError on 32/50 - xxxx (code 200): Dependency: Internal error. (131)
2023-10-12 17:12:17.437 | INFO     | twscrape.accounts_pool:get_for_queue_or_wait:260 - No account available for queue "UserTweets". Next available at 17:18:24
2023-10-12 17:18:27.607 | INFO     | twscrape.accounts_pool:get_for_queue_or_wait:266 - Account available for queue UserTweets
2023-10-12 17:18:33.407 | WARNING  | twscrape.queue_client:req:199 - Unknown error <class 'httpx.ConnectTimeout'>:
2023-10-12 17:18:33.418 | INFO     | twscrape.accounts_pool:get_for_queue_or_wait:260 - No account available for queue "UserTweets". Next available at 17:27:17

same issue for me, do you have any idea?

same issue for me, do you have any idea?

This is unknown error from Twitter. Should be fixed by PR https://github.com/vladkens/twscrape/pull/80 from v0.9


toolset ignored, need CMAKE_GENERATOR_TOOLSET

Brief Issue Summary
The CMakePresets.json has an option "toolset" but the CMake Tools extension ignores it.
If I run from command line cmake.exe detects it, but using the extension it doesn't detect clang-cl.
I need to add CMAKE_GENERATOR_TOOLSET for the extension to detect clang-cl, but as said, the command line doesn't need it.
        {
            "name": "clang-x64-debug",
            "displayName": "Win Clang x64 Debug",
            "inherits": "x64-debug",
            "toolset": "ClangCL", // command line only needs this
            "cacheVariables": {
                "CMAKE_GENERATOR_TOOLSET": "ClangCL" // extension needs this
            }
        },

CMake Tools Diagnostics
No response
Debug Log
No response
Additional Information
No response

@brccabral We're wondering if this has possibly been fixed with the most recent release, could you test again? Thank you.

Hi @gcampbell-msft , yes, v1.16.31 works with just "toolset": "ClangCL"

@gcampbell-msft I find cmake-tools(v1.19.52) doesn't support the format:
{
    "toolset":
   {
       "value": "ClangCL"
   }
}

but cmake command does


npm start fails

Hi there,
I'm trying to get the latest version (1.5.1) to work on an Rpi, but I fail at npm start. I'm building groovebasin and libgroove from source. Can I get some help here?

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'start' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart<EMAIL_ADDRESS>6 info start<EMAIL_ADDRESS>7 verbose unsafe-perm in lifecycle true
8 info<EMAIL_ADDRESS>Failed to exec start script
9 verbose stack Error<EMAIL_ADDRESS>start: node lib/server.js
9 verbose stack Exit status 1
9 verbose stack     at EventEmitter. (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:214:16)
9 verbose stack     at emitTwo (events.js:87:13)
9 verbose stack     at EventEmitter.emit (events.js:172:7)
9 verbose stack     at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:24:14)
9 verbose stack     at emitTwo (events.js:87:13)
9 verbose stack     at ChildProcess.emit (events.js:172:7)
9 verbose stack     at maybeClose (internal/child_process.js:817:16)
9 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
10 verbose pkgid<EMAIL_ADDRESS>11 verbose cwd /home/jia/groovebasin
12 error Linux 4.1.19+
13 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
14 error node v4.0.0
15 error npm  v2.14.2
16 error code ELIFECYCLE
17 error<EMAIL_ADDRESS>start: node lib/server.js
17 error Exit status 1
18 error Failed at the<EMAIL_ADDRESS>start script 'node lib/server.js'.
18 error This is most likely a problem with the groovebasin package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error     node lib/server.js
18 error You can get their info via:
18 error     npm owner ls groovebasin
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

You ever find a solution? I am having similar error at npm start on my ubuntu vps, although I am running node v0.12.10 vs your v4.0.0.


[App]: Filters do not work well

Current Behavior
On Mar. 14th, The filters cannot search for a string, e.g., when I search in the ID filter as follows, it doesn't appear any results.

Expected Behavior
The filters work well as yesterday.
Steps To Reproduce

Add a filter
select 'ID' or Name in the key
click the  'Select a value'
Nothing can be searched

Screenshots
No response
Environment
OS: Windows, Android
Browsers: Chrome, Edge
Version:
Additional Context
I have hundreds of runs, maybe it takes too much time to fetch all the values? But it works well before today.

Sorry that you encountered this! We are pushing out a fix, should be out shortly (in the next half hour or so)

@WanchengWildMan this is now fixed by the team. If you encounter any problems, please feel free to reopen the conversation.


Increased memory usage when calling to_hdf5()

Issue Description
I've noticed that since version 0.17.2 (up to 0.17.5), the output hdf5 file (generated by the code below) is written to disk only when the processing graph is about to complete (i.e., close to 100% according to the progress bar). As a result, memory usage increases significantly and leads to a crash. This is not a problem for version 0.17.1.
import numpy as np
import dask.array as da
import dask.diagnostics as dd

chunks = (2000, 2000)

# create list of dask arrays based on memmap
number_of_images = 60
dask_arrays_list = [
    da.from_array(np.memmap('data/data_{}.img'.format(idx),
                            shape=(65802, 29372), dtype='>f4',
                            offset=0, mode='r'),
                  chunks=chunks)[19296:42509, 0:29372].rechunk(chunks)
    for idx in range(number_of_images)
]

# stack dask arrays
dask_array = da.stack(dask_arrays_list, axis=0).rechunk({0: number_of_images})

# apply map_blocks to stacked dask arrays
result = da.map_blocks(lambda x: np.mean(x, axis=0),
                       dask_array, dtype='f8', drop_axis=0)

# compute/save to hdf5
with dd.ProgressBar():
    da.to_hdf5('data_test.hdf5', '/data', result)

Note that this piece of code is just to reproduce the problem, i.e., the first rechunk (...[19296:42509, 0:29372].rechunk(chunks)) is not necessary for this example (in fact, removing it resolves the issue for this particular case). Also, the use of mean() in the lambda function passed to map_blocks() is a placeholder for a more involved computation.
I did some digging and the problem is resolved if these lines:
deps = [d for d in dependents[item]
        if d not in result and not (d in seen and waiting[d])]

are switched back to:
deps = [d for d in dependents[item] if d not in result]

This change/improvement was introduced in this commit.
I'm not sure how this impacts other use cases or whether this is expected behavior. Any suggestion?
Thanks!
Version Information

Python version: 3.5.2

Dask 0.17.1; pip freeze output:
click==6.7
cloudpickle==0.5.3
dask==0.17.1
distributed==1.21.8
h5py==2.7.1
HeapDict==1.0.0
locket==0.2.0
msgpack==0.5.6
numpy==1.14.3
pandas==0.23.0
partd==0.3.8
psutil==5.4.5
python-dateutil==2.7.3
pytz==2018.4
six==1.11.0
sortedcontainers==2.0.2
tblib==1.3.2
toolz==0.9.0
tornado==5.0.2
zict==0.1.3

Dask 0.17.2; pip freeze output:
click==6.7
cloudpickle==0.5.3
dask==0.17.2
distributed==1.21.8
h5py==2.7.1
HeapDict==1.0.0
locket==0.2.0
msgpack==0.5.6
numpy==1.14.3
pandas==0.23.0
partd==0.3.8
psutil==5.4.5
python-dateutil==2.7.3
pytz==2018.4
six==1.11.0
sortedcontainers==2.0.2
tblib==1.3.2
toolz==0.9.0
tornado==5.0.2
zict==0.1.3

Dask 0.17.5; pip freeze output:
click==6.7
cloudpickle==0.5.3
dask==0.17.5
distributed==1.21.8
h5py==2.7.1
HeapDict==1.0.0
locket==0.2.0
msgpack==0.5.6
numpy==1.14.3
pandas==0.23.0
partd==0.3.8
psutil==5.4.5
python-dateutil==2.7.3
pytz==2018.4
six==1.11.0
sortedcontainers==2.0.2
tblib==1.3.2
toolz==0.9.0
tornado==5.0.2
zict==0.1.3

(Edit)
The script above needs some fake data to run. The following steps generate the required dataset:

Create a directory called data:

$ mkdir data

Run this to create a single fake image:

import numpy as np
shape = (65802, 29372)
fake_data = np.memmap('data/data.img',
                      shape=shape, dtype='>f4',
                      offset=0, mode='w+')
fake_data[:,:] = 7777
del fake_data

Create 60 fake images:

$ cd data
$ for i in {0..59}; do ln data.img data_$i.img; done

Thank you for the excellent bug report, and for identifying the commit where things change.  This is very helpful. I wish that I could return the favor with a rapid fix but alas I'm somewhat slammed this week and most of next.  I have this on my personal TODO list though.  If you don't hear back in two weeks I encourage you to ping angrily on this issue :)
I also have a long plane ride coming up.  I'll download your analysis here and hopefully have the will to work on it while in the air.

Ping!
Let me know if there's anything I could do to further pinpoint the issue. Thanks!

Am not aware of this having been fixed in 0.18.0. That said, it would still be good to try and confirm it is still an issue.

Thanks for replying. I just tested with v0.18.0 and the problem is still there. Is there anything else I could test and report back on?

I have time to look into this and try to fix it myself. I'll get started already but any thoughts/leads are welcome.

I tried copy-pasting this example but it looks like it needs some files.  If there is a way to make this example easier to run it will make it easier for other people to reproduce the problem.

I've just added instructions on how to generate the required data files.

Closing..., as it is superseded by this one.


TypeError: Input 'strides' of 'StridedSlice' Op has type int32 that does not match type int64 of argument 'begin'.

I just want to get a slice of a tensor. It works for int32 but not for int64:
In [35]: import tensorflow as tf
    ...: a=tf.Variable([0,1,2], dtype=tf.int64)
    ...: i = tf.constant(1, dtype=tf.int64)
    ...: a[i]
    ...:
<long stack trace>
TypeError: Input 'strides' of 'StridedSlice' Op has type int32 that does not match type int64 of argument 'begin'.
In [36]: import tensorflow as tf
    ...: a=tf.Variable([0,1,2], dtype=tf.int32)
    ...: i = tf.constant(1, dtype=tf.int32)
    ...: a[i]
    ...:
Out[36]: <tf.Tensor 'strided_slice_22:0' shape=() dtype=int32>

This question is better asked on  StackOverflow since it is not a  bug or feature request. There is also a larger community that reads questions there. Thanks!

It looks like a bug for me... I mean, why can't int64 be used as a simple index? What happen if I have a vector of more than 2^32 elements?

Invoking @vrv who has suffered a long time working on the differences in behavior between int32 and int64.
Can you not do a cast from int64 to int32 to get the effect you want? I.e., do you really have a >4B-element vector that you want to access in practice?

StridedSlice definitely supports int64 indices, I think the problem is that the datatype of 'begin' is int64 and the datatype of 'strides' is int32, and that's a type mismatch.
@aselle: my guess is that there's some kind of missing cast to match the dtype of 'begin' somewhere in the SliceHelper.  Do you know the code well enough to know what the fix is quickly?

@minhlab knowing the version of TensorFlow and all the information requested in the template would help to make sure it's not already fixed.

In fact, I think this was already fixed two months ago in https://github.com/tensorflow/tensorflow/pull/9000/files, and you are probably using an older version of TF that doesn't have the bug fix.

Hi @vrv, you're right. After running pip install --upgrade, the problem was gone.


Oauth: The resource owner or authorization server denied the request

I'm very new to Oauth so please bear with me if I have something terribly wrong, but I cannot get the oauth/token route to work, after following all the steps. I always get the response:
{error: "invalid_client", error_description: "Client authentication failed",…}
error: "invalid_client"
error_description: "Client authentication failed"
message: "Client authentication failed"

With logs in the Drupal backend:
League\OAuth2\Server\Exception\OAuthServerException: The resource owner or authorization server denied the request. in League\OAuth2\Server\Exception\OAuthServerException::accessDenied() (line 243 of /app/drupal/vendor/league/oauth2-server/src/Exception/OAuthServerException.php).
Symfony\Component\HttpKernel\Exception\HttpException: The JWT string must have two dots in Drupal\simple_oauth\Authentication\Provider\SimpleOauthAuthenticationProvider->authenticate() (line 81 of /app/drupal/web/modules/contrib/simple_oauth/src/Authentication/Provider/SimpleOauthAuthenticationProvider.php).
Doing a bit of debugging I found that the simple_oauth module appears to require the client_id explicitly set in the body of the post, whereas the next-js package doesn't include it:
Oauth2Token.php
if (!empty($body['client_id'])) {
      $consumer_storage = $this->entityTypeManager()->getStorage('consumer');
      $client_drupal_entities = $consumer_storage
        ->loadByProperties([
          'uuid' => $body['client_id'],
        ]);
      if (empty($client_drupal_entities)) {
        return OAuthServerException::invalidClient($request)
          ->generateHttpResponse(new Response());
      }
      $client_drupal_entity = reset($client_drupal_entities);
    }

get-access-token.ts
const response = await fetch(
    ${process.env.NEXT_PUBLIC_DRUPAL_BASE_URL}/oauth/token,
    {
      method: "POST",
      headers: {
        Authorization: `Basic ${basic}`,
        "Content-Type": "application/x-www-form-urlencoded",
      },
      body: querystring.stringify({
        grant_type: "client_credentials",
      }),
    }
)

Logging the body of the POST
Array
(
    [grant_type] => client_credentials
    [client_id] => undefined
    [client_secret] => undefined
)

As I say, I might be horribly wrong here, but I'm just desperately trying to get this working as it looks awesome. Any help appreciated!

No worries. This is very early alpha and I know there's a lot to configure to get this working properly (we're working on making this simpler).
Let's do a quick preflight check.
Are your environment variables configured and properly loaded?
You need the following:
NEXT_PUBLIC_DRUPAL_BASE_URL=http://localhost:8080
NEXT_IMAGE_DOMAIN=localhost
DRUPAL_SITE_ID=
DRUPAL_CLIENT_ID=
DRUPAL_CLIENT_SECRET=
DRUPAL_PREVIEW_SECRET=

You can test this by console.log(process.env) in _app.jsx.

It's weird, when I run the app I see the following:
info  - Loaded env from /../nextjs/next/.env.local
And:
 NODE_ENV: 'development',
  __NEXT_PROCESSED_ENV: 'true',
  NEXT_PUBLIC_DRUPAL_BASE_URL: 'https://next.lndo.site',
  NEXT_IMAGE_DOMAIN: 'next.lndo.site',
  DRUPAL_SITE_ID: '0a44cc38-ed29-4e76-802b-5dec85bc41cf',
  DRUPAL_PREVIEW_SECRET: 'rushhour',
  DRUPAL_FRONT_PAGE: '/node',
  DRUPAL_CLIENT_ID: 'cf068b84-4813-456a-969e-148dbbe8e92a',
  DRUPAL_CLIENT_SECRET: 'secret',
  NEXT_PUBLIC_TEST: 'testing',
  TRACE_ID: '68ca412def1ef638'

But when I try to console log process.env it's empty. Not really anything to do with this package I know, but frustrating...
In any case, it appears to have everything correct even though I can't console.log(), here are the headers being sent on the oauth/token call:
:authority: next.lndo.site
:method: POST
:path: /oauth/token
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-GB,en-US;q=0.9,en;q=0.8
authorization: Basic dW5kZWZpbmVkOnVuZGVmaW5lZA==
cache-control: no-cache
content-length: 29
content-type: application/x-www-form-urlencoded
origin: http://localhost:3000
pragma: no-cache
referer: http://localhost:3000/
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"
sec-ch-ua-mobile: ?0
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

Thanks for your help!

I also just tried running my drupal site on localhost:1234 instead of the lando url incase it's a domain issue to no avail (both running on https)

EDIT: It's working! I was using getStaticProps() completely wrong, I need to read up on how to use Nextjs :)
Still not sure why I can't console.log the process.env though???

Marking this issue as closed. Thank you.


Failure to install on Jython since 2.12.0

(jython) r@r-V ~ $ python --version
Jython 2.7.1b3
(jython) r@r-V ~ $ pip install requests==2.11.1
Collecting requests==2.11.1
  Using cached requests-2.11.1-py2.py3-none-any.whl
Installing collected packages: requests
  Found existing installation: requests 2.11.0
    Uninstalling requests-2.11.0:
      Successfully uninstalled requests-2.11.0
Successfully installed requests-2.11.1
(jython) r@r-V ~ $ pip install requests==2.12.0
Collecting requests==2.12.0
  Using cached requests-2.12.0-py2.py3-none-any.whl
Installing collected packages: requests
  Found existing installation: requests 2.11.1
    Uninstalling requests-2.11.1:
      Successfully uninstalled requests-2.11.1
  Rolling back uninstall of requests
Exception:
Traceback (most recent call last):
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/commands/install.py", line 338, in run
    requirement_set.install(
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/commands/install.py", line 338, in run
    requirement_set.install(
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/req/req_set.py", line 780, in install
    requirement.install(
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/req/req_install.py", line 1057, in move_wheel_files
    move_wheel_files(
  File "/home/r/.virtualenvs/jython/Lib/site-packages/pip/wheel.py", line 272, in move_wheel_files
    compileall.compile_dir(source, force=True, quiet=True)
  File "/home/r/jython/Lib/compileall.py", line 56, in compile_dir
    if not compile_dir(fullname, maxlevels - 1, dfile, force, rx,
  File "/home/r/jython/Lib/compileall.py", line 56, in compile_dir
    if not compile_dir(fullname, maxlevels - 1, dfile, force, rx,
  File "/home/r/jython/Lib/compileall.py", line 56, in compile_dir
    if not compile_dir(fullname, maxlevels - 1, dfile, force, rx,
  File "/home/r/jython/Lib/compileall.py", line 50, in compile_dir
    if not compile_file(fullname, ddir, force, rx, quiet):
  File "/home/r/jython/Lib/compileall.py", line 99, in compile_file
    ok = py_compile.compile(fullname, None, dfile, True)
  File "/home/r/jython/Lib/compileall.py", line 99, in compile_file
    ok = py_compile.compile(fullname, None, dfile, True)
  File "/home/r/jython/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/home/r/jython/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
RuntimeException: java.lang.RuntimeException: Method code too large!
(jython) r@r-V ~ $

This might not be a bug in requests, but perhaps it is workaroundable? Has any change been done in 2.12.0 that caused a very big method to be created?

In the future, please search closed and open issues before creating new ones that are duplicates.
This is a duplicate of the open #3711

I found that issue, but it did not contain "RuntimeException: java.lang.RuntimeException: Method code too large!" and it mentioned usage, not installation, with something called Indna, which I didn't think I use.
Sorry for the trouble.

Hey @ppolewicz, it looks like this is specifically a limitation of how Jython uses the JVM as noted in this ticket. I'd ensure you have an updated version of Jython2.7 (which allegedly fixes the problem), and attempt to use Requests 2.12.4.
If that doesn't solve it, I'm afraid there's not much we can do at this time since the bug isn't really with Requests.

(for people searching for a solution later and finding this)
That ticket says:

Issue will get solved if the jython version is 2.7 without applying work around(Splitting of method content).

...but my Jython version is 2.7.1b3 and it still fails.
requests==2.12.4 fail to install for the same reason.


Greenkeeper/sequelize 4.20.1

Coverage remained the same at 98.105% when pulling ce1ff6c64874a50652d8c2cac2847fd09ce48fcd on greenkeeper/sequelize-4.20.1 into 32b8055d267111cba38c6eb763a6a81625cbadd4 on master.


Support Maven jdk toolchain config in dev mode

Parallel to https://github.com/OpenLiberty/ci.gradle/issues/792, add support for Maven toolchains https://maven.apache.org/plugins/maven-toolchains-plugin/ to independently set the JDK used to run Maven itself vs. the JDK used to run other goals,  most importantly (but not exclusively) the Liberty server.
See discussion especially here.
Some dimensions to the problem:

Honor the configuration of the toolchain plugin itself for plugin goal invocation
Honor configuration of individual plugins with jdk toolchain config, for executing goals of those plugins
Honor the config when dev mode executes a Java/JDK invocation outside of executing maven plugins,e.g. when we do the javac recompile in Maven dev mode.

The last is perhaps debatable?  So maybe a bit of design thought is needed before proceeding.

I would be happy to create a PR once there is a user request (please 👍🏻  if you would like to see it).

I would be happy to create a PR once there is a user request (please 👍🏻 if you would like to see it).

Sorry I didn't respond to this before.   If/whenever you still have the time and interest, that would be great.  I would be happy to collaborate on this.
We actually did get a request just recently:
https://github.com/OpenLiberty/ci.maven/issues/1781


Robust Font Loading without a Polyfill

The caveats and benefits of cutting the mustard for your web fonts on CSS Font Loading API support.
Compare to previous methods that required a polyfill:

limitations with font-synthesis and segmented font-family names
Independently/optionally eliminate the need for fonts-loaded guarding classes.
browser support differences, most notably Internet Explorer/Edge

You could also mention this brilliant and robust method: https://twitter.com/zachleat/status/803347265742106628

@zengabor Unconventional but innovative, truly the work of brilliant and humble mind 😂

See also https://twitter.com/zachleat/status/900406984297590785


closure_js_library typecheck fails when depending on j2cl jre

This might be a bug with rules_closure, or with how j2cl's //jre/java:jre uses it, I'm not quite clear.
Describe the bug
Using closure_js_library emits several build rules to facilitate js libraries building and type checking in parallel. One of those rules emits a .i.js file, and another collects the .i.js files produced by dependencies and runs rules_closure's JsCompiler on them with --checks_only.
To Reproduce

Check out latest j2cl, currently 3ecf6f737b6eddab233644072b18025dbcfe8d16
cd samples/helloworld
bazel build 'src/main/java/com/google/j2cl/samples/helloworld:app_typecheck'

Bazel version
Please include version of Bazel that you are running J2CL with:
Bazelisk version: v1.15.0
Build label: 5.4.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 15 16:14:25 2022<PHONE_NUMBER>)
Build timestamp:<PHONE_NUMBER>
Build timestamp as int:<PHONE_NUMBER>

Expected behavior
Type checks for trivial programs pass.
Actual behavior
INFO: Analyzed target //src/main/java/com/google/j2cl/samples/helloworld:app_typecheck (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/colin/workspace/j2cl/samples/helloworld/src/main/java/com/google/j2cl/samples/helloworld/BUILD:9:19: Doing library-level typechecking of //src/main/java/com/google/j2cl/samples/helloworld:app failed: (Exit 1): ClosureWorker failed: error executing command bazel-out/k8-opt-exec-2B5CBBC6/bin/external/io_bazel_rules_closure/java/io/bazel/rules/closure/ClosureWorker JsCompiler --checks_only --warning_level VERBOSE --jscomp_off reportUnknownTypes ... (remaining 21 arguments skipped)
bazel-out/k8-fastbuild/bin/external/com_google_j2cl/jre/java/jre.i.js:10776: ERROR - A file cannot be both a goog.module and a script file that contains at least one goog.provide.
goog.provide("jre");
^
  Codes: JSC_MIXED_MODULE_TYPE

bazel-out/k8-fastbuild/bin/external/com_google_j2cl/jre/java/jre.i.js:10781: ERROR - A file cannot be both a goog.module and a script file that contains at least one goog.provide.
goog.provide("jre.checks");
^
  Codes: JSC_MIXED_MODULE_TYPE

bazel-out/k8-fastbuild/bin/external/com_google_j2cl/jre/java/jre.i.js:10794: ERROR - A file cannot be both a goog.module and a script file that contains at least one goog.provide.
goog.provide("jre.logging");
^
  Codes: JSC_MIXED_MODULE_TYPE

3 error(s), 0 warning(s)

Target //src/main/java/com/google/j2cl/samples/helloworld:app_typecheck failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.942s, Critical Path: 1.77s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

The issue appears to be that the ijs task emits code that fails checks for j2cl's //jre/java:jre target, as the jre output contains both goog.module and goog.provide declarations. It could be appropriate to split these into separate j2cl_library declarations (that might also make it possible for downstream projects to selectively replace certain java.* packages with their own implementations).
It also might be possible to add js_suppress args, but at first try JSC_MIXED_MODULE_TYPE isnt a valid suppression that can be provided.
Consider adding this task to the samples CI to validate output?

This is a problem with rules_closure. I actually had a patch to fix it but ended up not sending it. Will resurrect the CL though I'm not sure if it fixes all the problems.

https://github.com/bazelbuild/rules_closure/pull/615


Executing a callback sequence with an empty sequence succeeds

For issue: https://github.com/tumblr/colossus/issues/211
Fills in with callback(Success(Nil))
@DanSimon

:+1:


Add helm revoke functionality: build on #50

This implements small refactored improvements on https://github.com/gruntwork-io/kubergrunt/pull/50 so that it can be merged:

Moved placement of error types so that they are in helm/errors.go
Removed RevocationErrors since it turned out to be redundant with MultiHelmError.
Refactored call to revokeAccessFromRBACEntity: Instead of calling it for each entity type (RBAC Group, User, and ServiceAccount), we build the entity list first and call it once. This works better with the error handling using MultiHelmError.
Reword some error messages so that it is clearer what is going on.
Removed all the overrides in the dependency list, as they are now resolved from the later versions of kube. Upgraded terratest to latest and fixed build failures.

Going to merge this in now that build is passing, since it is building on heavily reviewed code, with a few minor improvements.

Roger


Controlnet vsd

Recreated and modified #244. Found a good guidance scale via tuning.
Some results:
condition_scale: 2.0 with guidance_scale 7.5:
https://github.com/threestudio-project/threestudio/assets/36912641/73a7c4b8-1ff3-4c60-90a3-475f07f53a22
condition_scale 1.5 with guidance_scale 7.5:
https://github.com/threestudio-project/threestudio/assets/36912641/c6f19ac1-0c10-410c-bd2b-dc1680496c72

Just saw unified guidance has been implemented. I'll let you guys decide what to do with this branch.


Add Replay Overrides for Functions that Retrieve Array Sizes

Implement overrides for functions that retrieve an array size and are used with the following pattern:
vkEnumeratePhysicalDevices(instance, &size, nullptr);
data = new VkPhsicalDevice[size];
vkEnumeratePhysicalDevices(instance, size, data);

Current replay behavior is to always use the size value read from the file.  If the size value read from the capture file is smaller than the size value returned by the first call in the above example, the replay of the second call will produce a VK_INCOMPLETE return value.
The replay tool will print an error message for the VK_INCOMPLETE return value, but the replay tool will not use the data retrieved by the call, so it should be safe to ignore the message.
Short term, this could be addressed by silencing the VK_INCOMPLETE error message, but long term we will need to implement a portability override, which does the following to make the call with the correct array size:

When the array parameter is null, skip replay of the call.
When the array parameter is not null, first call the function with a null array value to obtain the appropriate size, then allocate memory for the array and make the call with retrieved size and allocated memory.

Although this is primarily a portability issue, where the size value would only be different for replay on a different GPU/driver, the vkGetPipelineCacheData has been observed to return different sizes when replayed on the same system used for capture, resulting in the VK_INCOMPLETE error message on replay, which should be addressed for the 'non-portability' replay case.

Closed with #349


Change QuickAccent to Quick Format

Description of the new feature / enhancement
Make QuickAccent more versatile by giving more text editing shortcuts, such as convert to single line (to get rid of linebreaks when copy pasting from a pdf), capitalize (correctly capitalize a sentence / convert to upper/lowercase), etc.
Scenario when this would be used?
This feature could be used during essay writing. When copying information from a pdf over to for example a google doc, it would be helpful to be able to select the text and paste it as a single line.
Supporting information
No response

This seems more like a feature request for the Advanced Paste component, doesn't it?

@CrazeXD
Could you please rename the issue to something lik "[Advanced Paste] Add advanced text formatting"?

My thoughts on this: Great idea but the new fornats should be available through the ui too. (Maybe configurable which one.) There are so many shortcuts in PowerToys already 🤯 and I can't imagine everyone if I need it.

What I was thinking about was something similiar to what word does. On paste, a small tooltip pops up with different options where you can press a keybind to change the way something pastes in. This could be toggled on or off, with the additional options of what you mentioned.

What I was thinking about was something similiar to what word does. On paste, a small tooltip pops up with different options where you can press a keybind to change the way something pastes in. This could be toggled on or off, with the additional options of what you mentioned.

Sounds interesting. Can you share a demo or mock up?

I don't really have any experience in creating mockups, but here is a little google drawing I made as an example.

I like this idea! I was actually noodling on and started prototyping a feature that would allow users to add their own custom shortcut that they could reuse. Ideally, we could make it so that the user can select which SLM they want to use for the custom shortcut and/or decide to use the Open AI API flow we already have today.
It sounds like your enhancement is slightly different because some of the functionality doesn't really need an AI model, but rather just additional logic baked into advanced paste (which sounds nice).
I'd love to coordinate our ideas and efforts here, so we don't do overlapping work with the additional shortcuts flow. I'm working to get my prototype design to a point where I can share the designs out in this thread and we can bounce ideas off one another!

Sounds great, let me know!

Thinking if instead of having this be a whole new shortcut, perhaps being options for Text as Plain text, changing that text in some ways we select? Just an idea, though.

That's what we were discussing above, integration with advanced paste.

Apologies for the delay - I didn't have a chance to clean up the UI more than what I had already mocked up for just the settings, so here is roughly the idea that I was playing around with:

The overall goal of this feature is to allow the user to have autonomy over various copy/paste flows they want to include. It would be great to be able to choose from a set of local models and/or choose from whatever cloud based models we have available.
Please be aware this is purely a mockup of the settings functionality with essentially random strings in places that I've just used to communicate the idea. @CrazeXD @jaimecbernardo @craigloewen-msft - would be interesting in getting your thoughts.
@CrazeXD, I know your suggestion is a bit different, but I think we can integrate both of our ideas in a cohesive way.

I really like that mockup, I think it definetly expands upon the ideas I originally was thinking about. Both of these features definetly have a place in Advanced Paste and I think I have a way of integrating them as well.
Instead of having a global option "Enable Paste with AI" as you have at the top, what we could do is just use the presets and inside the preset customizer have the option to enable AI, at which point we would be able to see all of the different options and modals that you designed in your demo. We could also have optionality for regex, capitalization, single line, etc. to give as much control over the paste function as we can think of (baked in with regular code rather than an AI model as in my original issue).
Let me know what you think about this combination @joadoumie

Seems like a good approach to me. Not sure if this is what you had in mind exactly, but having a regex type input that would literally just apply some regex formula to text and paste it would be epic.

That's what I was thinking, sounds like a useful feature.

Please be aware this is purely a mockup of the settings functionality with essentially random strings in places that I've just used to communicate the idea. @CrazeXD @jaimecbernardo @craigloewen-msft - would be interested to get your thoughts.

It sounds like an interesting feature. A sort of quick prompts. I wonder if it makes sense having them on the same section as the others or having a distinct section like "custom prompts", to distinguish local run logic from AI based prompts. 🤔

My thought process was that inside of a separate modal for preset editing, we have a toggle similar to the original mockup that enables Paste with AI and then opens up many more options relevant to an AI based feature. This could reduce some of the confusion/clutter within the page (at least in my opinion).

@CrazeXD - maybe some artwork to help illustrate your last suggestion could be helpful. Just pen on paper works, nothing too fancy!
I'm really excited about merging these ideas and continuing to build them out!

@joadoumie Here is my pencil and paper crude drawing. Apologize for the delay, I've been busy with finals.

The idea is that when a user clicks on the "Add custom shortcut " in the above prototype, they see this as a pop-up modal. A cool feature idea I had while drawing this was "Custom string code", in which a user gets to make a custom  C# script or other language (not getting too gritty on those details) which will be run on paste.
This would be in addition to the Google drawing I have at the top, which would show up on a different paste key one, or perhaps it could be one of the different icons.

Love these ideas thank you - sorry for the slow turnaround! I'm juggling a few ideas coming in from various areas - great idea for the scripting logic (C#, python etc.). There are a lot of moving parts coming from different directions that I'd like to bring into the mix and need to figure out the best way to prioritize with the team.
I'm working on some proposals this week and will share out here as soon as I can.

Hey @joadoumie, I was wondering if you hard any updates in regards to this feature.

Still working on publishing a short-term roadmap so we can get the ball rolling. If you're interested in working on a branch to integrate some of your ideas in code I'm open to it we'll just need to sync and gutcheck before we can pull anything in

#33261 for reference of a bug that could potentially be solved with advanced text formatting.

@CrazeXD - we're working on adding custom paste actions for the upcoming release.
https://github.com/microsoft/PowerToys/tree/dev/ani/advanced-paste-custom-actions
You can take a look at the code in the branch there. For now, it's focused on adding custom prompts to the existing paste actions. This means it simply takes a name for your action and a prompt:

It uses a modal similar to what we do for custom color formats in color picker. If you are still interested in playing around with some of the ideas you've shared above, I think we can leverage the same modal and just make it more robust with different panes potentially at the top that you can select.
@drawbyperpetual @craigloewen-msft for visibility

@joadoumie Sounds good, I'll start  working on it (as soon as I can figure out how to use the repository lol).

Let me know if you need any help -- happy to help you get up and running.

Actually that would be great, I was having some trouble with the Visual Studio solution (I've never really worked with it before, I'm still a student so it's very likely something basic).
Thanks!

Sure thing - it's a really large repo. I recommend looking at the file diffs in the branch that I just shared above because that will help you understand where things have to be changed in order to add similar functionality.
In terms of building/compiling the project, check out https://github.com/microsoft/PowerToys/tree/main/doc/devdocs#compiling-powertoys to get started.
Let me know if you have any specific questions I'm more than happy to help

@joadoumie Hey, I was working on trying to implement some of the features (I started with single line since it's super similiar to paste as plain text) and was having some issues (probably because I still haven't figured out entirely how the repo works), and was wondering if you could take a look and help me out? Whenever you get a chance.

Do you have a fork I can take a look at? Happy to look into it.

@joadoumie I don't know if I should make a commit with the changes I have (I can if it would work better) but here is a cloud patch that you can take a look at (let me know if you have any issues accessing it).
https://gitkraken.dev/drafts/9996b635-9108-43f6-b763-3ed4f4ee69c4?%3Fsource%3Dgitlens

no worries at all! I'm still learning myself. I'll try my best to take when I get the chance -- likely within about 1 week.

@CrazeXD
Any updates? Is it planned to support custom text formatting without using ai?

@htcfreek Sorry I've been kind of swamped and this took a backseat, but implementation has started.

@joadoumie , @CrazeXD
Any updates on this?

@htcfreek uhhhh no I kinda got stuck and don't know what I'm doing lol


WIP: Add Devices Page

[x] Add a device page
[x] Be able to navigate to the scan page from the device page
[x] Populate device page with registered devices
[x] Fix a bug where devices got added to the PairedStorage on each pair attempt
[ ] ???
Meant to fix #24

@crc-32 @matejdro Any thoughts on what else the device page should have?

I recommend you check out @piggehperson's designs: https://www.figma.com/file/mXl0svBSnlGV2MKfTj8H7e/Mobile-app?node-id=1818%3A5671


Support Windows's DpiScale in Avalonia Designer

Describe the bug
Avalonia Designer is very convinient but only when your Windows DpiScale is 100%.
On Laptop, for instance, when DpiScale is 125% or 150%, there is an offset on the hit tests.
You have to move the mouse somewhere in the down-right of the controls to see the pointerover effect, or to click on controls. For example, the minimized scrollbar is almost impossible to grab.
To Reproduce
Try to use the Designer when Windows DpiScale is not 100%.

You speak of the Visual Studio previewer? I think you have to raise the issue here: https://github.com/AvaloniaUI/AvaloniaVS

Ah exact, there was a separate git, thank you!


TestXdsProxyHealthCheck test flake

https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/1355198515185192960

ran the test multiple times locally without luck of reproducing it. How do you recommend reproducing it?

I added https://github.com/istio/istio/wiki/Test-Flakes#reproducing-test-flakes with some instructions. let me know if its not clear, I just wrote this

Thanks for adding it. I modified a few things but besides that its very understandable. I am finally able to reproduce the flake.

https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/1395876045185355776


Let feeds use the json plugin

But, modify the json plugin to allow parsing to be optional (since the feeds need to be parsed differently, not the same way Flickr API json is parsed)
Update tests
cc @jeremyruppel

Feeds knows how to parse its own responses as JSON now https://github.com/flickr/flickr-sdk/blob/next/services/feeds.js#L32


Plugin Graphs: make history file configurable

Resolves #517

@MoonMaker I've been using this:
cat $HOME/.gitconfig
[alias]
  pr = "!f() { git fetch -fu ${2:-upstream} refs/pull/$1/head:pr/$1 && git checkout pr/$1; }; f"

Allows for git pr 518 and checks out the PR as a local branch


[1.20.1]incompatibility between Sodium 0.5 and TC

Minecraft Version
1.20.1
Fabric/Quilt Version
0.14.22
Fabric/Quilt API Version
0.87.0-1.20.1
Hephaestus Version
1.20.1-<IP_ADDRESS>
Describe your issue
crash with sodium 0.5.2 when entering world
here is the crash report:
crash-2023-08-22_23.39.35-client.txt
here is the latest log:
latest.log
Crash Report
https://github.com/Alpha-s-Stuff/TinkersConstruct/files/12410418/crash-2023-08-22_23.39.35-client.txt
Other mods
sodium-fabric-mc1.20.1-0.5.2
Tried reproducing with just Tinkers?
Yes
Performance Enchancers
Sodium
Searched for known issues?
Checked pinned issues

Fixed in latest release https://modrinth.com/mod/hephaestus/version/1.20.1-<IP_ADDRESS>


Trouble with inference

I am trying to run inference on some images using the KITTI (ConvNeXt-L) linked on the README.
the model runs without error, but outputs look like this:

I believe something is going wrong since this doesn't match the supposed results.
I read some of this repo's previous issues that had similar images, but didn't find any full configuration setups that fixed the issue.
I am running inference with this command from the README
python test_simple_SQL_config.py ./args_files/args_test_simple_kitti_320x1024.txt
Where can I find a combination of:

model from the Pretrained Weights Section
config file from /SfMNeXt-Impl-main/args_files
contents of config file that correctly access the model above and set it up without error

Thanks!

For the ConvNeXt model, the args_file is ./conf/cvnXt.txt, and the command should be
python test_simple_SQL_config.py ./conf/cvnXt.txt

Thanks for the fast response!
That worked. Thank you!

Hi, I have the same issue when I run inference with KITTI(ResNet-50) , the results look like this:

I am running inference with this command from the README
python test_simple_SQL_config.py ./args_files/args_test_simple_kitti_320x1024.txt
with the following contents:
--load_weights_folder ./pretrained/KITTI_320x1024_models/
--image_path ./images/
--height 320
--width 1024
--model_dim 32
--patch_size 20
--dim_out 128
--query_nums 128
--min_depth 0.001
--max_depth 80.0
--ext jpg
the weights is download from KITTI (ResNet-50)
Thanks!

You should set --load_pretrained_weights, and set --load_weights_folder to /path/to/your/model.

Thank you very much fro your quick replay!
I add the --load_pretrained_model to the config, and set --load_pt_folder to my/model/path, the full config like this:
--height 320
--width 1024
--model_dim 32
--patch_size 20
--dim_out 100
--query_nums 128
--min_depth 0.01
--max_depth 80.0
--load_pretrained_model
--load_pt_folder ./pretrained/KITTI_320x1024_models/
--image_path ./images/
--ext jpg
But got the following error:
log.txt
Any idea?
Thanks again!

When I add --model_name resnet_320x1024 to the config, it starts to download something from 'rwightman/gen-efficientnet-pytorch', and timed out. I cannot figure out what it download, so I cannot download by hand.
Here is the log:
log 2.txt
Best wishes!

When I add --model_name resnet_320x1024 to the config, it starts to download something from 'rwightman/gen-efficientnet-pytorch', and timed out. I cannot figure out what it download, so I cannot download by hand.
Check out this, https://github.com/hisfog/SfMNeXt-Impl/issues/15#issuecomment-1837342744,  set --backbone to resnet_lite

--model_type cvnxt_L
--model_dim 32
--patch_size 32
--dim_out 64
--query_nums 64
--dec_channels 1024 512 256 128
--min_depth 0.001
--max_depth 80.0
--load_pretrained_model /data1/hanmina/SfMNeXt-Impl-main/ConvNeXt_Large_SQLdepth/encoder.pth
--load_pt_folder /data1/hanmina/GAN/GAN_10.7/result/LLVIP/images/val/
--height 320
--width 1024
--image_path /data1/hanmina/GAN/GAN_10.7/result/LLVIP/images/val/
--ext jpg
这样设置是有问题的吗？报错信息是：test_simple_SQL_config.py: error: unrecognized arguments: /data1/hanmina/SfMNeXt-Impl-main/ConvNeXt_Large_SQLdepth/encoder.pth

When I add --model_name resnet_320x1024 to the config, it starts to download something from 'rwightman/gen-efficientnet-pytorch', and timed out. I cannot figure out what it download, so I cannot download by hand.
Here is the log: log 2.txt
Best wishes!

after add following parameters, this script would be works(commit id: 6a1e997).
--dim_out 128
--num_features 256


[wip] gettext: add setup hook populating GETTEXTDATADIRS

Motivation for this change
Fixes #32296: packages expect gettext to be able to find the files they install into $prefex/share/gettext, such as appstream-glib with its /nix/store/…-appstream-glib-0.7.2/share/gettext/its/appdata.loc.
https://www.gnu.org/software/gettext/manual/html_node/Preparing-ITS-Rules.html
https://github.com/autotools-mirror/gettext/blob/981c523ddbd1462970335a1069573ca3bdd3df5e/NEWS#L19
Things done
WIP: needs testing.

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Most hooks run once during the build, env hooks run once for each native build input (which includes buildInputs unless cross compiling) with the store path to the input as the first argument: https://github.com/NixOS/nixpkgs/blob/d747c273876f07bd2649f7dd25dba5bbbc9bffd9/pkgs/stdenv/generic/setup.sh#L407-L409

I have confirmed that this fixes #32296.


Fix GoDoc comment formatting in t3c-apply source

PR #6960 was erroneously merged with go fmt test failures, resulting in our GHAs failing on master and PR branches thereon based.
This fixes the issue, and also fixes a bug where t3-apply would consider attempting to tail a non-existent file to be a success with no output.

Which Traffic Control components are affected by this PR?

Traffic Control Cache Config (t3c, formerly ORT)

What is the best way to verify this PR?
Make sure existing tests pass, ensure the go fmt GHA now passes
If this is a bugfix, which Traffic Control versions contained the bug?

master

PR submission checklist

[x] This PR utilizes existing tests
[x] This PR has GoDoc comment documentation
[x] This PR doesn't ned a CHANGELOG.md entry
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY

The tests are reportedly failing on master. Since I think my code changes fixed a bug, I'm happy to give it a day or two while someone looks into the failures on master and then rebase. If they continue to fail by, say, the end of Monday, I'll just file an issue and take the changes out.


Fix bad unrolling on the exciting APW basis set

Sebastian Tillack from the SOL group pointed out to me that assignments of the APW wavefunctions was flawed.
I traced the problem back to bugs in the extraction of the <custom> and <lo> tags.
I also used this opportunity to clean up the code.

@ladinesa If Sebastian replies in time, I could also add a test here.

@ladinesa Got a question: the updates to the exciting parser show up in the JSON output of the archive. However, when I parse with appworker, it doesn't show up in the GUI, neither locally, nor when deployed. No modifications were made to the metainfo.
Do you have an idea of where the problem could be?

Do you have an idea of where the problem could be?

Maybe it is not using the correct parser commit or that you did not install using -e

Maybe it is not using the correct parser commit or that you did not install using -e

Triple-checked the commit. I always install with -e and that wouldn't explain why the deployment fails too:
https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1342?commit_id=8b2b4e28c2d0e7bd9dc60bad27241b267672066e

Maybe it is not using the correct parser commit or that you did not install using -e

Triple-checked the commit. I always install with -e and that wouldn't explain why the deployment fails too: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1342?commit_id=8b2b4e28c2d0e7bd9dc60bad27241b267672066e

maybe put a raise in the parser to see if the worker picks it up.

maybe put a raise in the parser to see if the worker picks it up.

Okay, traced it back to ExcitingParser.file_exists(): it doesn't find any species files when run from appworker, but it does as a local parser

maybe put a raise in the parser to see if the worker picks it up.

Okay, traced it back to ExcitingParser.file_exists(): it doesn't find any species files when run from appworker, but it does as a local parser

I was just looking at this earlier.There is a utils function get_files. Maybe we can repurpose this.

@ladinesa Solved the problem locally, but not on the deployment... Could you please take a look?

@ladinesa Solved the problem locally, but not on the deployment... Could you please take a look?

Just needed time, all good now


Call to protected method PhpCsFixer\Tokenizer\Tokens::generatePartialCode() from context 'PhpCsFixer\Tokenizer\Analyzer\NamespacesAnalyzer'

All of a sudden I'm getting this error in some of my repositories, but not others. All of them are using custom fixers that I wrote.
  PhpCsFixer\Tokenizer\Analyzer\NamespacesAnalyzer->getDeclarations()
    in phar:///usr/local/bin/php-cs-fixer/src/Fixer/Import/FullyQualifiedStrictTypesFixer.php at line 79
  PhpCsFixer\Fixer\Import\FullyQualifiedStrictTypesFixer->isCandidate()
    in phar:///usr/local/bin/php-cs-fixer/src/AbstractFixer.php at line 74
  PhpCsFixer\AbstractFixer->fix()
    in /my_project/vendor/friendsofphp/php-cs-fixer/src/Runner/Runner.php at line 190
  PhpCsFixer\Runner\Runner->fixFile()
    in /my_project/vendor/friendsofphp/php-cs-fixer/src/Runner/Runner.php at line 132
  PhpCsFixer\Runner\Runner->fix()
    in phar:///usr/local/bin/php-cs-fixer/src/Console/Command/FixCommand.php at line 220
  PhpCsFixer\Console\Command\FixCommand->execute()
    in phar:///usr/local/bin/php-cs-fixer/vendor/symfony/console/Command/Command.php at line 251

The PHP version you are using ($ php -v):
PHP 7.0.17 (cli) (built: Mar 18 2017 20:13:50) ( NTS )
PHP CS Fixer version you are using ($ php-cs-fixer -V):
PHP CS Fixer 2.12.2 Long Journey by Fabien Potencier and Dariusz Ruminski (dcc87d5)

@jbnv , what was the reason of issue ?
(I see you had closed it, so probably you had figured it out on your own already ;) )


Add version flag

Type of change

[ ] Refactor
[x] New feature
[ ] Bug fix
[ ] Optimization
[ ] Documentation Update

Description
Add the version flag using the go-commons code.

Related Tickets & Documents

Related Issue #
Closes #

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.

Testing

Please describe the System Under Test.
Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

/lgtm


Add a token refresh load testing scenario

Description
There is currently no scenario to test refreshing access tokens.
The new suggested Gatling scenario steps or the new scenario are:

Login 100_000 users as fast as possible to have a lot of active sessions
Refresh access tokens when 60 seconds are left for the access token

Assumptions:

Under peak load, Keycloak will slow down for the responses to the access tokens. As the clients will calculate the next refresh time based on the response, the refreshes will level over time automatically.

Expected outcomes/learnings:

What Keycloak configuration (how many pods/vCPUs) can support this load?
How does Keycloak behave with many active sessions if we kill pods during the first and second phase?
Are there bottlenecks?

Motivation
This is one of the most common activities (even more frequent than user logins), so there should be one for this.

@ahus1 , I agree this scenario needs to exist.  Browsers left open all refreshing their tokens over time is a realistic use-case.  And just how many users can be doing this before problem start to happen.
I heard one story that since refresh tokens are stored only in the infinispan cache, which is about 10k entries, then users start to be logged out.  True(?) or not, it makes sense if infinispan isn't overflow to disk or db.
So another thing to check for is if the logged in users start to fail to refresh because of this.
Have you started any work on this yourself yet?

@ahus1, I started a draft PR attempting to address this.  Never done Scala, but I think I'm getting it.
So far... (links to PR)

KeycloakScenarioBuilder.exchangeCode() now saves the refresh token
(new method) KeycloakScenarioBuilder.refreshToken() attempts to refresh the id, access and refresh tokens by supplying the refresh token saved when exchangeCode was used.  It POSTs to the same TOKEN_ENDPOINT as exchangeCode()
defined RefreshToken scenario class which logs in, and then will repeatedly invoke refreshToken()  (not fully implemented)

But I'm having an issue with the refreshToken() routine.  It makes the request, but Keycloak is giving the following HTTP 401 error in the response:
{"error":"unauthorized_client","error_description":"Invalid client or Invalid client credentials"}
Keycloak itself logs this:
2023-07-19 06:41:01,479 WARN  [org.keycloak.events] (executor-thread-119) type=REFRESH_TOKEN_ERROR, realmId=f37169e7-43c1-492d-be41-f26a8f0842d9, clientId=client-0, userId=null, ipAddress=<IP_ADDRESS>, error=invalid_client_credentials, grant_type=refresh_token
The userId is null, which I thought looked odd.
I was thinking this might be due to the cookies set during auth that are marked Secure not being submitted by Gatling since it's not using TLS.  So, for the moment, I made it re-create those cookies without the secure flag, but it still gives the same error.
As far as I can tell, the HTTP POST Gatling is submitting all the same values that my normal application (of course, using a different instance of keycloak and client) does when it successfully refreshes the tokens.
I checked through the test-realm's settings to see if something is disallowing refreshing tokens.  e.g. Token lifetimes > 0, user-0's roles, "Use refresh tokens" set to On, "Use refresh tokens for client credentials grant" set to On, etc.
Any ideas what could be going on?  TIA

I found the problem to be that client-0 -> Settings -> Capability config -> Client authentication was set to ON
Turning it OFF allows the refresh token operation to work.

@ddurham2 thanks for picking up this task to create a new refresh token scenario. I am not fully aware why the publicClient is set to false TBH., but that is what we had for the service enabled account, does this have something to do with the cookie's we are setting for a particular session in your code ? Do we really need to set those cookies for this scenario to work ?

But I'm having an issue with the refreshToken() routine. It makes the request, but Keycloak is giving the following HTTP 401 error in the response:

how are you exactly testing this routine ? the Service enabled client expects the correct client-secret and client name for the scenario to work properly. if you are using test-realm the client secret is by default set to setup-for-benchmark

All, now I'm to the point of wanting to actually figure out how this test should work.  Some questions.

Presumably the tester should be able to specify the number of users (which is already determined by --concurrent-users)

The tester needs a way to specify the pause between refreshes.  An argument?  Or do we want to just let that be determined by the expiration of the tokens which can be configured in KC admin.  That is, if you want a rapid refresh, then you make the token time, like, 10 seconds, and presumably the refresh operation follows that schedule - w/ 10% remaining or so.  If you want a realistic test, leave it 5m and it'll refresh every 4m30s (i.e. 10% remaining).  The downside is that setting the token time very short may break other (future?) scenarios.

The test has to end at some point.  Do we have a fixed token refresh count, specified by an arg.  Or should it read the --mesurement duration given to the test? And it simply refreshes on the above interval until that time has elapsed?

The original description talks about 100k users.  I would agree that we need to be testing in that range (or more).  But currently the setup only creates user-0.  I'm going to assume that if we tried to do all the testing with a single user account (logged in 100k times), that it isn't a real-world scenario since db/cache indexes and such aren't really in play like they would be.  So that will require initialize-keycloak-entities.sh to learn how to create N users, and for the test to be told how many users there are.  Correct?

Finally, I've already discovered with the minimal test that it is, that it is really not hard at all to crash keycloak.  This is probably something that should be discussed in the keycloak project, but I'd be willing to discuss that with somewhere here first if you like.  I'm using a fairly standard setup in minikube.

@ddurham2 I am glad you were able to figure out the issue around invalid client credentials issue. And all of your questions related to the scenario are valid and here is my attempt to answer them to the best of my knowledge.
Not sure if you have seen this documentation already, I will be referring it in few places of my answers., https://www.keycloak.org/keycloak-benchmark/benchmark-guide/latest/

Presumably the tester should be able to specify the number of users (which is already determined by --concurrent-users)

As mentioned in the configuration options that you can specify for all the benchmark scenarios., there are two ways to specify users one is the --concurrent-users(Closed Model) which you already pointed out and the other one is --users-per-sec (Open Model). To read more about closed and open model and how it works in gatling please refer to this page from gatling official docs.

The tester needs a way to specify the pause between refreshes. An argument?

I think we did something like this in one of the previous attempts of creating a refresh token scenario for a particular situation modeled by @tkyjovsk ., we could follow a similar pattern to use --user-think-time and implement that config inside the specific scenario which enables refreshToken
    .openLoginPage(true)

    .userThinkPause()
    .loginUsernamePassword()
    .exchangeCode()

    .userThinkPause()
    .refreshToken()

    .userThinkPause()
    .refreshToken()

    .userThinkPause()
    .refreshToken()

    .userThinkPause()
    .logout(true))

This is one of the ways, but more than happy to hear your ideas around this implementation.

The test has to end at some point. Do we have a fixed token refresh count, specified by an arg. Or should it read the --mesurement duration given to the test? And it simply refreshes on the above interval until that time has elapsed?

Your assumption is right about using --measurement. That would be the best way to end the test and it takes a default value of 30 seconds which is pointed out in the docs here.

The original description talks about 100k users.

Sorry our docs didn't have a proper way to describe this, but I think we can highlight this in the https://www.keycloak.org/keycloak-benchmark/benchmark-guide/latest/preparing-keycloak section somehow. We do have a dataset module related documentation, and in there we specify how to generate large amounts of datasets. For example to generate a realm with 100k users, you use something like the below command once the dataset module is installed on the target Keycloak instance's providers directory.
https://keycloak-example-host.com/realms/master/dataset/create-realms?realm-name=realm-0&count=1&threads-count=10&users-per-realm=100000

And the benchmark scenarios are built in such a way when you do not specify a specific user or a client, it anticipates a specific pattern of clients, users and realms to be existing inside the system and runs the requests off of those patterns - see here and here. This allows you to scale realistically for large datasets and makes the test conditions field like

Finally, I've already discovered with the minimal test that it is, that it is really not hard at all to crash keycloak.
We are always learning new ways to trouble Keycloak and would be more than happy to hear it via a new GH Issue discussion and see if thats something we have already encountered or not. Again thanks for the continued interest in improving the health of Keycloak product in general. Cheers !

Concerning the crash.. I think that it may just be minikube shutting it down due to the failed liveness probes under load.  So, false alarm I think.

I've locally hit 100k users refreshing.   But granted, different systems will handle it differently.
I spun it up in minikube, with 6 pods limited to 2 cores and 4GB RAM, and running it as follows works for me:

created 100k users in realm-0 using the dataset provider
kcb.sh --scenario=keycloak.scenario.authentication.AuthCodeWithRefreshToken --server-url=$KC_URL --realm-name=realm-0 --ramp-up=500 --users-per-realm 100000 --user-think-time=100 measurement=600 --concurrent-users=100000

================================================================================
2023-07-29 01:19:56                                        1718s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=1400000 KO=0     )
> Browser to Log In Endpoint                               (OK=100000 KO=0     )
> Browser posts correct credentials                        (OK=100000 KO=0     )
> Exchange Code                                            (OK=100000 KO=0     )
> RefreshToken                                             (OK=1000000 KO=0     )
> Browser logout                                           (OK=100000 KO=0     )

---- Authentication - Authorization Code Username/Password with Refresh Token --
          active: 0      / done: 100000
================================================================================

Simulation keycloak.scenario.authentication.AuthCodeWithRefreshToken completed in 1718 seconds
Parsing log file(s)...
Parsing log file(s) done
Generating reports...

================================================================================
---- Global Information --------------------------------------------------------
> request count                                    1400000 (OK=1400000 KO=0     )
> min response time                                      2 (OK=2      KO=-     )
> max response time                                  34309 (OK=34309  KO=-     )
> mean response time                                  4509 (OK=4509   KO=-     )
> std deviation                                       7873 (OK=7873   KO=-     )
> response time 50th percentile                        328 (OK=328    KO=-     )
> response time 75th percentile                       3392 (OK=3386   KO=-     )
> response time 95th percentile                      23990 (OK=23989  KO=-     )
> response time 99th percentile                      27971 (OK=27971  KO=-     )
> mean requests/sec                                814.427 (OK=814.427 KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                        822712 ( 59%)
> 800 ms < t < 1200 ms                               61965 (  4%)
> t > 1200 ms                                       515323 ( 37%)
> failed                                                 0 (  0%)
================================================================================

... not always the best response times.
Do we want to change anything about the actual testing of the refresh operation itself?  If not, then, how to move this across the finish line...

How do we want to specify the number of refreshes to make?

We are using userThinkTime between refreshes.  I don't love it, but it works.

During the test I'm not sure it was really using all of the 100k users I created.  In the admin console I could only see that user-0 had session created (how many it doesn't display).  But user-1 had none.  It has been my understanding that specifying --users-per-realm would cause it to authenticate, cycling through these users.  Is that correct?
Is that final report it spits out adequate?  It doesn't seem to focus just on the refresh times which is what we're more interested in.  Is it always just a general report?
Do we want to augment the documentation to mention this scenario?  I assume doc/dataset/modules/ROOT/pages/ is the correct place to make these changes?  I'll take a stab at explaining it.

I'll put appropriate links and instructions for using the dataset provider, since it's pretty helpful in this scenario.

.. and for kicks I did some more runs.  A pod given 12 CPUs and only 4GB of RAM appears to perform better than the 6 pod configuration above (I ran the 6 pod configuration again and got the same poorer results).  The single pod configuration's CPU actually hovered around 5-6 cores utilized and about topped out around 1.5GB RAM.  I'm sure there's many factors, but I thought it worth mentioning.
Results of 1 Pod w/ 12 CPU and 4GB limit
================================================================================
2023-07-29 02:11:13                                        1610s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=1400000 KO=0     )
> Browser to Log In Endpoint                               (OK=100000 KO=0     )
> Browser posts correct credentials                        (OK=100000 KO=0     )
> Exchange Code                                            (OK=100000 KO=0     )
> RefreshToken                                             (OK=1000000 KO=0     )
> Browser logout                                           (OK=100000 KO=0     )

---- Authentication - Authorization Code Username/Password with Refresh Token --
          active: 0      / done: 100000
================================================================================

Simulation keycloak.scenario.authentication.AuthCodeWithRefreshToken completed in 1610 seconds
Parsing log file(s)...
Parsing log file(s) done
Generating reports...

================================================================================
---- Global Information --------------------------------------------------------
> request count                                    1400000 (OK=1400000 KO=0     )
> min response time                                      0 (OK=0      KO=-     )
> max response time                                   2570 (OK=2570   KO=-     )
> mean response time                                    19 (OK=19     KO=-     )
> std deviation                                         50 (OK=50     KO=-     )
> response time 50th percentile                          4 (OK=4      KO=-     )
> response time 75th percentile                          9 (OK=9      KO=-     )
> response time 95th percentile                         99 (OK=99     KO=-     )
> response time 99th percentile                        201 (OK=201    KO=-     )
> mean requests/sec                                869.025 (OK=869.025 KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                       1399335 (100%)
> 800 ms < t < 1200 ms                                 583 (  0%)
> t > 1200 ms                                           82 (  0%)
> failed                                                 0 (  0%)
================================================================================

kcb.sh --scenario=keycloak.scenario.authentication.AuthCodeWithRefreshToken --server-url=$KC_URL --realm-name=realm-0 --ramp-up=500 --users-per-realm 100000 --user-think-time=100 measurement=600 --concurrent-users=100000

I am still going through all the changes you did but one thing which popped out at me was the missing = for 'users-per-realm 100000' I believe if its passed correctly like this 'users-per-realm=100000' the test should rotate the logins between 100k users.

Ah! I didn't realize that mattered.  But looking at the sh script, I see it does.  I will correct it (and in the comment).
After correcting the argument, results are still similar.  So that's good.

@kami619, for the refresh token frequency, I found that there's an existing --refresh-token-period=<seconds> config value already handled.  It's used by the AdminConsoleScenarioBuilder.scala.
Do you think that's more appropriate than --user-think-time?  If so, I can change it.

For now, I've introduced --refresh-token-count= to parameterized the number of refreshes.  Let me know if that's cool or not..

@kami619, it occurs to me now, that if one were to specify --refresh-token-count=0 to this new scenario, then it's rather equivalent to the existing AuthorizationCode scenario.
Any reason not to simply:

introduce --refresh-token-count=0 (default)
use --refresh-token-period for the time between refreshes (or something new if you disagree with reusing that existing arg).   The point of this is so that user-think-time can continue to specify how long after opening the login page it waits.
make the AuthorizationCode scenario call refresh() with these parameters
document the new available option in the AuthorizationCode docs

Sound more reasonable than introducing a new scenaior?

@kami619, for the refresh token frequency, I found that there's an existing --refresh-token-period=<seconds> config value already handled. It's used by the AdminConsoleScenarioBuilder.scala.
Do you think that's more appropriate than --user-think-time? If so, I can change it.

You are correct, using refresh-token-period cli input is more appropriate for this than using the user-think-time, please make that change.

that if one were to specify --refresh-token-count=0 to this new scenario, then it's rather equivalent to the existing AuthorizationCode scenario.

I think this is worth the change and merge both scenario's into one, to make code more consice and clean. Thanks for suggesting the change. I am good with that approach.

all changes complete as far as I'm concerned

@ddurham2 thanks much, I will give it another go and merge if everything lines up.


Add an issue to multiple projects

Feature Description
Would be great if we could add an issue to multiple projects. Especially with the new feature that allows org wide projects.
Screenshots
No response

This would be very helpful indeed!

I created a PR https://github.com/go-gitea/gitea/pull/25164


docs(release): also update the version number in SECURITY.md

Proposed Changes
We need to make sure to update supported version number in SECURITY.md when creating a new release.

Hello, I am a PR summary agent on flows.network.
It could take a few minutes for me to analyze this PR. Relax, grab a cup of coffee and check back later. Thanks!


Dead link http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=710000003474

Found in docs/load-test/reference-qa.md

@alexhomer1 Can you fix the broken links in this article? It looks like some stuff has moved and I don't know enough about that area to replace it with the new good links.

I cannot find this link in the source of the topic. Maybe it was already fixed? If not, let me know where it is in the topic. Thank you.


Update brew formula for 1.0

$ wget https://binaries.cockroachdb.com/cockroach-v1.0.src.tgz
2017-05-10 09:30:26 (18.3 MB/s) - ‘cockroach-v1.0.src.tgz’ saved
[68753479/68753479]
$ sha2 cockroach-v1.0.src.tgz
SHA-256 (cockroach-v1.0.src.tgz) =
ca87b10eec688195e0df4f85431b019f2980ae4b511ee321f91f945315efeb76
SHA-384 (cockroach-v1.0.src.tgz) =
852945017c596e75fd48f6c528f39a347b47f86e20d7eb2d6d6e366047f9dd7492f794d5ee90153a21b7dec1bd59b29d
SHA-512 (cockroach-v1.0.src.tgz) =
d74ad9546908535f9a7ad4172a04a5f86f5ed22698bbb005aec7478a52632e0497a966b859fd68b40894eb7ee68bfa80dc4188d198f2f23725b9cadd50a55191

:tada:
We should switch over to the new procedure
def install
  system "make", "install", "prefix=#{prefix}"
end

but doesn't have to be now.

Done. Might as well.

Checking now!

How confident are we in the new procedure? Switching now seems a bit risky if we aren't very sure about it.

Pretty confident, this was the goal of one of the PRs going into 1.0. It's also easy to test by having homebrew use a local checkout of the branch (except for my homebrew for some reason).

$ brew uninstall --force cockroach && brew install cockroach
==> Installing cockroach from cockroachdb/cockroach
==> Downloading https://binaries.cockroachdb.com/cockroach-v1.0.src.tgz
Already downloaded: /Users/benesch/Library/Caches/Homebrew/cockroach-1.0.tgz
==> make install prefix=/usr/local/Cellar/cockroach/1.0
==> Caveats
CockroachDB is a distributed database intended for multi-server deployments.
For local development only, this formula ships a launchd configuration to
start a single-node cluster that stores its data under:
  /usr/local/var/cockroach/
Instead of the default port of 8080, the node serves its admin UI at:
  http://localhost:26256

Do NOT use this cluster to store data you care about; it runs in insecure
mode and may expose data publicly in e.g. a DNS rebinding attack. To run
CockroachDB securely, please see:
  https://www.cockroachlabs.com/docs/secure-a-cluster.html

To have launchd start cockroachdb/cockroach/cockroach now and restart at login:
  brew services start cockroachdb/cockroach/cockroach
Or, if you don't want/need a background service you can just run:
  cockroach start --insecure
==> Summary
🍺  /usr/local/Cellar/cockroach/1.0: 5 files, 60.7MB, built in 2 minutes 7 seconds

$ cockroach version
Build Tag:    v1.0
Build Time:   2017/05/10 13:46:03
Distribution: CCL
Platform:     darwin amd64
Go Version:   go1.8.1-parallelbuilds
C Compiler:   4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)
Build SHA-1:  43aa1ed04cd817348921ac27e11e25725f634aa6
Build Type:   development

:beers:


double click on a reference will not open target window editor

hi, if you select a method, right click and select search reference.
Then if you double click in a reference, if the ref are in other window editor, the editor not open scaling.
Can we fix this bug plz :)

@djmisterjon can you check your setting workbench.useExperimentalGridLayout

it activated but what this should do ?

@djmisterjon It seems that setting does not make a difference for me anyway. I see the collapsed editor group is not expanded when focused this way. Additionally, I see that there is some extra whitespace at the bottom of the image, is that happening in your window?

yes it focus on the correct line with highlight, but i want auto expanded the window.
And yes you true, it work in debug Mode with breakpoint, when travel in breakpoint it auto open and expanded window needed.
But in this ways, it not work, maybe a dev forget add the feature ?

I think the issue is as follows:

there is an editor group minimized that is not active
clicking on the reference will prefer to reveal the editor from the group instead of opening it in the current group
we preserve focus in the reference view because the interaction is by mouse and as such we do not activate the minimized group
as such the group does not maximize

I pushed a change so that a minimized editor group will maximize even if not getting activated.

@bpasero yes, sorry, some difficult to explain my brain in english but yes you say all.

@djmisterjon no worries. I am actually thinking that maybe a better fix would be to not just restore the minimized group but also set it active. It feels weird to me that you click on an item and you are not getting any feedback if the item is already opened in some group. At the very minimum I think it should not only restore (if minimized) but also activate the group. Would you agree?

@djmisterjon yes +1 for active group 👍

OK, pushed that via https://github.com/microsoft/vscode/issues/79798

@mjbvz good catch, this was obviously broken when I made all extensions go back to the previous solution and the "Find References" view is an extension.
I added another value to the EditorActivation option: RESTORE and now all extensions (including your webview stuff) will use that option. This option will NOT activate the group (so we preserve todays behavior), but will restore a group IF it is minimized.

test in version 1.38.0 and it just work fine.
Thanks you so much dude , am so happy now.


Enhancement: Update exclude_fields_from_list, exclude_fields_from_detail, exclude_fields_from_create, exclude_fields_from_edit conditionally on runtime.

I wan't to disable/enable - show/hide fields conditionally.
Let's say the current user has permission to visit Company view but if the current user doesn't have SUPERUSER role, they shouldn't see the company contact info in the list view.
Another scenario could be like this:
I have two tables, articles and authors. Articles can be created by any author and any article can have multiple authors, but onl the author who created that article should be able to edit the title and other authors can only see the title in the list and details view.
How can I implement this one? I did try something the other day but I was not able to.

You can achieve this by overriding the ModelView.get_list_query function. Here is an example


Local CA (between hosts)

For if we once require a local CA to establish trust between hosts, this might be a starting point: https://github.com/numtide/devshell/pull/74

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Probably best solution https://www.tweag.io/blog/2020-12-16-trustix-announcement

Another discovery in this problem space: https://github.com/NixOS/nixpkgs/pull/112322


ZeroDivisionError

I just tried the end to end pipeline from project_common with the latest changes on dev. It is working well for me. The only problem I have now is when there is a speech input without any keywords (eg. i said "nasal") which returns an empty list and we get division by zero in tts: ZeroDivisionError: float division by zero

I tried to reproduce the error both on the tts repository and on the end to end pipeline from project_common.
I wasn't able to reproduce the error after many trials.
I spoke to Amit and we agreed to close this issue for now, if the problem comes back, the issue will be re-opened with more details on how to reproduce it


Removed options from AmazonFulfillmentOrderCreator::updateOrder that aren't required

I removed several of the options that were required to update an order. The only required option should be the SellerFulfillmentOrderId, as per the UpdateFulfillmentOrder request specification.
What are your requirements for tests on this? I could modify the existing tests to make sure this works if you would like. I forked so that I could update my Amazon orders without inputting redundant data each time.

Looks pretty straight-forward. I admit that I did not notice this difference between create and update when I added that action, so thanks for bringing it up. As it turns out, I have a pull request (#87) that makes a bunch of fixes for those files and others. If it's okay with you, I can implement your changes on my branch, since otherwise this one will conflict with it.

Of course! It was just a quick delete so I could use it right away. I'll go ahead and close this and you can add it to #87


Hunting the burned once

What happened?
Viki was like: let me try Darshana's PR on this fish

Fish is burned

Task is:

Go through app
check where we can find this edgecases - (burned should not be visible on any list page)
fix the queries

Please reproduce in steps
🔥
Expected Behavior
I've expected see only existing NFTs on the app
What browsers are you seeing the problem on?
Chrome
Are you logged in?
No
At which address did you encounter bug?
kodadot.xyz
Screenshots
Paste from your clipboard or upload screenshots/video
Relevant log output
No response
Payment link for reward
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@vikiival I'll check this