Incorrect number of weights in custom layer containing keras layer

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.12.0
Python version: 3.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: 9.0, 7.1
GPU model and memory: Quadro

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
class CustomLayer(tf.keras.layers.Layer):
    def __init__(self, dims, **kwargs):
        super(CustomLayer, self).__init__(**kwargs)
        self.dims = dims
        self.dense = tf.keras.layers.Dense(units=self.dims)

    def call(self, x, mask=None):
        return self.dense(x)

    def get_config(self):
        return {"dims": self.dims}

inp = tf.keras.layers.Input(shape=(10,))
x = CustomLayer(32)(inp)
model = tf.keras.models.Model(inp, x)
print(model.summary())

The summary contains no weights:
Layer (type)                 Output Shape              Param #
=================================================================
input_5 (InputLayer)         (None, 10)                0
_________________________________________________________________
custom_layer_2 (CustomLayer) (None, 32)                0
=================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0

If I try to train, it doesn't seem to do anything
model.compile("sgd","mse")
X,Y = np.random.rand(1000,10), np.zeros((1000, 32))
model.fit(X,Y, epochs=20)

Epoch 1/20
1000/1000 [==============================] - 0s 28us/step - loss: 0.1625
Epoch 2/20
1000/1000 [==============================] - 0s 25us/step - loss: 0.1625
Epoch 3/20
1000/1000 [==============================] - 0s 29us/step - loss: 0.1625
Epoch 4/20
1000/1000 [==============================] - 0s 28us/step - loss: 0.1625
Epoch 5/20
1000/1000 [==============================] - 0s 24us/step - loss: 0.1625
Epoch 6/20
1000/1000 [==============================] - 0s 24us/step - loss: 0.1625
Epoch 7/20
1000/1000 [==============================] - 0s 26us/step - loss: 0.1625
Epoch 8/20
1000/1000 [==============================] - 0s 22us/step - loss: 0.1625
Epoch 9/20
1000/1000 [==============================] - 0s 22us/step - loss: 0.1625
Epoch 10/20
1000/1000 [==============================] - 0s 25us/step - loss: 0.1625
Epoch 11/20
1000/1000 [==============================] - 0s 22us/step - loss: 0.1625
Epoch 12/20
1000/1000 [==============================] - 0s 25us/step - loss: 0.1625
Epoch 13/20
1000/1000 [==============================] - 0s 23us/step - loss: 0.1625
Epoch 14/20
1000/1000 [==============================] - 0s 26us/step - loss: 0.1625
Epoch 15/20
1000/1000 [==============================] - 0s 23us/step - loss: 0.1625
Epoch 16/20
1000/1000 [==============================] - 0s 26us/step - loss: 0.1625
Epoch 17/20
1000/1000 [==============================] - 0s 23us/step - loss: 0.1625
Epoch 18/20
1000/1000 [==============================] - 0s 23us/step - loss: 0.1625
Epoch 19/20
1000/1000 [==============================] - 0s 23us/step - loss: 0.1625
Epoch 20/20
1000/1000 [==============================] - 0s 23us/step - loss: 0.1625

Describe the expected behavior
I would like to use Keras layers within the custom layer, and have them trainable.

Is it possible for you to try latest TF versions (!pip install tensorflow-gpu==1.15.0-rc1)and let us know whether the issue persists? There were lots of performance improvements in the latest versions. Thanks!

Seems to work appropriately in 1.15. Thanks!


Enabling WSL then causes BSOD when AIDA64 starts.

Enabling WSL then causes BSOD when AIDA64 starts.And installing the SteelSeries mouse driver also got the BSOD.
BSOD stop code: SYSTEM_SERVICE_EXCEPTION
Failed operation：kerneld.x64
kerneld.x64 is a file from AIDA64.
My Laptop Specs:
Laptop Name: HP Omen 15
Processor: AMD Ryzen 5 4800h
Build Version: 2004, 19041.450

Enabling WSL then causes BSOD when AIDA64 starts.And installing the SteelSeries mouse driver also got the BSOD.
BSOD stop code: SYSTEM_SERVICE_EXCEPTION
Failed operation：kerneld.x64
kerneld.x64 is a file from AIDA64.
My Laptop Specs:
Laptop Name: HP Omen 15
Processor: AMD Ryzen 5 4800h
Build Version: 2004, 19041.450

If you recently upgrade to KB4566782, it's related to this thread #5760

it's related to this thread #5760

There is no data to support the supposition that the BSOD is in any way related to a WslRegisterDistribution failed with error: 0x80041002 other than the fact you both did a windows update.
/dupe #5770


get_image_size bug

When the picture has an EXIF information, the Image size may get the opposite value of the width and height, and Bboxes will generate an error coordinate. In order to solve this problem, I temporarily modified part of the code:
def get_image_size(filepath):
    # return Image.open(filepath).size
    img = Image.open(filepath)
    img = ImageOps.exif_transpose(img)
    return img.size

Then, the returned results can be displayed normally in label-studio.

Could you please create a PR ?

Hi, I have created a PR to fix this bug（https://github.com/heartexlabs/label-studio-ml-backend/pull/158）.

@ChenPengf https://github.com/heartexlabs/label-studio-ml-backend/pull/158#pullrequestreview-1090303202 please check this comment

Merged into master


Update aggregation.md

Remove strange extra characters.

@ktschau : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@mcleanbyron : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Thanks for submitting these fixes!


Move to cpu takes extra memory usage after .gather()

System Info
- `Accelerate` version: 0.30.1
- Platform: Linux-5.4.119-19.0009.28-x86_64-with-glibc2.31
- `accelerate` bash location: /home/jovyan/conda-env/envs/gloq/bin/accelerate
- Python version: 3.10.14
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.3.0+cu121 (True)
- PyTorch XPU available: False
- PyTorch NPU available: False
- PyTorch MLU available: False
- System RAM: 1878.24 GB
- GPU type: NVIDIA H800
- `Accelerate` default config:
        Not found

Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
[X] My own task or dataset (give details below)

Reproduction
Hi! I'm doing multi-gpu inference of a specific layer of a model, and trying to save the IOs. To save gpu RAM, I only move the tensors to gpus when computing.
Here is my code:
layer = layer.to(accelerator.device)
    layer_outputs = []
      for i in tqdm(range(0, len(layer_inputs), batch_size*accelerator.num_processes)):
          with accelerator.split_between_processes(layer_inputs[i: i+batch_size*accelerator.num_processes]) as sharded_inputs:
              batch_inputs = torch.stack(sharded_inputs).to(accelerator.device)
              batch_outputs = layer(batch_inputs)
          batch_outputs = accelerator.gather(batch_outputs)
          batch_outputs = batch_outputs.to('cpu')
          for j in batch_outputs:
              layer_outputs.append(j)
layer = layer.to('cpu')

The issue is when batch_outputs = batch_outputs.to('cpu') happens, it takes accelerator.num_processes times memory than expected.
Expected behavior
I want to save the gathered inference outputs into layer_outputs list only once. The memory usage should be same as layer_inputs.

I'm not sure if I fully understand your issue, but would it help if you run this only on one process? I.e.:
if accelerator.is_main_process():
    # your code

@BenjaminBossan Thank you for your reply.
I tried:
if accelerator.is_main_process:
    batch_outputs = batch_outputs.to('cpu')
    for j in batch_outputs:
        layer_outputs.append(j)

But the whole process hangs.

Hmm, not sure why it would hang in that case. Hopefully someone else has a good idea here.

I managed to solve my problem with native pytorch.distributed operations. Thanks for the help!

It hangs because we need to declare the catchup point.
Or:
if accelerator.is_main_process:
    batch_outputs = batch_outputs.to('cpu')
    for j in batch_outputs:
        layer_outputs.append(j)
accelerator.wait_for_everyone()

Would love to know your solution though @xinghaow99 and I can translate it over to what should work in accelerate :)

I used torch.distributed.scatter_object_list() instead of accelerator.split_between_processes(), and I refactored my script to avoid using gather(). I'm new to Accelerate and not sure if the problem is with Accelerate.
I suspect that the problem arises when using gather(), as it syncs the object across all devices, the memory consumption on each device is increased by 8x (lets say there are 8 processes). After calling .cpu(), these objects on different devices use separate space in CPU memory, thus taking up 8x the space they should be. And this maybe can be avoided by using torch.distributed.gather_object()? (I'm not sure)
I hope this is informative to you.


[FEATURE] Add better support for backwards compatibility with previously saved DataPanels

Since meerkat is changing quite quickly, and folks are often working off the dev branch, it's hard to ensure that every DataPanel that gets saved is backwards compatible with future versions of Meerkat. Eventually, once Meerkat is stable and folks are working off of major and minor versions, we should support backwards compatibility. But for the time being, when everyone's working off of dev, how should we support them? I see two options:
(1) Save the meerkat commit with every DataPanel and column, and create a conversion script that allows for converting saved DataPanel's between commits.
(2) Try to support backwards compatibility in the read and write code directly.
I think (2) will be quite challenging because we rely on Pickle, which runs into issues when classes and names change. One approach is to offer an "approximate" read that skips all data in pickles and raises appropriate warnings. This seems not ideal though and adds a lot of mess to the code.
I think my preference is for (1), but curious to hear other thoughts.

@krandiash @ad12


updated cal view and list view button to have separate buttons

Added a button for both list view and calendar view separately and highlight the activated page.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

can you link issue #480 so it will close when the pr merges?

getting some weird whitespace below the greenbar when i'm looking at the comparison view. also it should show both buttons in the comparison view. i attached a screenshot for reference

getting some weird whitespace below the greenbar when i'm looking at the comparison view. also it should show both buttons in the comparison view. i attached a screenshot for reference

I didnt know we wanted it in both places.  I can get that updated too.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.

Your review app has been deployed and is available at 'http://<IP_ADDRESS>:80'.Learn More about how to test and provide feedback for the app.


Deploy to npm with travis

This adds the npm deploy step to the Travis file, with the on: branch: master condition. It is also worth noting that NPM publish will only work if the version in package.json has been updated since a version can only be used once on NPM.
I added the $NPM_EMAIL and $NPM_TOKEN values in travis using a token from my account, but it may be more appropriate for someone elses info to be there.
I also added a .npmignore file to try to ignore everything except build-lib/, readme, changelog, and license.
^update: I removed the .npmignore

I did read more about Git Flow. It makes sense but I think having release branches may be overkill here and just clutter our branches. My preference would be to just use git tags to tag release commits instead, and npm version does this automatically.
I changed the README to remove the mention of the git tag command and instead reference the npm version command.
About the npm user - in this model there is no need to worry about the npm user in our local CLIs since that is handled by travis.

This was not urgent so I did not want to distract from more important things. I have added information about where the NPM information is stored in the Travis CI settings.
But I would like to clarify something before merging this:

after the last release and making a PR from dev into master, I noticed that the dev branch says "This branch is 17 commits ahead of master.", even after I run git pull origin master on the dev branch. Is this because the PR from dev into master was a "Squash"?

Do you, @manzt, or @ilan-gold know if there is a more clean way of doing this, so that our dev and master comparisons do not list the commits that have already been merged into master?

If there is not a way to clean this up, are we alright with that, or should we go back to just using master and feature branches?

I was never opposed to master/release branches so going back is ok with me.  I am not sure how to fix that - I have found that sometimes these things are github specific and not actually related to the underlying git versioning.  This might be a dumb question, but did you push the subsequent changes back to dev?  Sometimes git/github can be confused when there are no "changes" even though there is a commit.

My only alternative is to not squash the dev branch? Squash on feature PRs/bug fixes into dev and then merge master. This would keep a nice history of each PR (like before) since right now all those changes are reduced to v1.1 etc

That makes sense. I think for now we can keep the dev branch then and just now squash when merging dev into master.


Stremio Server Added

Hello,
Added Stremio Server to resolve the "Streaming Server is not available" error.
Added a /stremio volume to keep installed Addons after Docker restart.

The Dockerfile is extending stremio/server
https://github.com/psyb0t/safe-stremio/blob/master/Dockerfile#L1
So you need to use the already existing directory if you want.

Sorry for the confusion. How do you explain the Streaming Server is not available error?

@Korben00 thx!


added comment

@anti43 I'm leaving this open so that you can later add what you're working on after your vacation. Enjoy :)


Frameit: Speed up processing of skipped screenshotsf

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context

Fixes #12976
Description

** What's changed?**

Created a helper method should_skip? to determine whether or not Frameit should skip over a Screenshot due to not having a filter defined for the screenshot in the Framefile.json.
Call the should_skip? method in the Runner class to avoid processing any screenshots if not necessary.  This was done in the Runner opposed to the Editor as this is the place the end user is notified about the start of  framing via the Helper.show_loading_indicator("Framing Screenshot...") call.  There is now a conditional there that instead displays to the user that the screenshot is being skipped if should_skip? returns true.
Removed the UI.user_error message in editor.rb when the title wasn't found as this message is now handled in Runner as mentioned above. A notable difference is that this is now a UI.message not a UI.user_error as this scenario can be typical and shouldn't be flagged as an error.
Moved the instantiation of the Editor/MacEditor  from Screenshot to Runner to give Runner the ability to call to should_skip? on the Editor

** Testing **

Added unit tests
Exploratory tests using a local project.  Confirmed no functional changes to framed images, only a significant increase in performance when dealing with screenshots that don't need to be processed by Frameit. Went from ~1.5/sec per screenshot -> near zero.  Makes a big difference when dealing with 100s of screenshots!

Hey @janpio !  I finally got around to #12976 as we talked on earlier in the year!  Could you take a look?  Thanks!


Broken decoding of Issue without a body (when its a PR)

Issues endpoints also return PRs. Body in a PR is Option, but isn't in Issue. This breaks at runtime during deserialisation.
github4s.GithubResponses$JsonParsingException: String: DownField(body)

Good catch @YarekTyshchenko, we have two different operations for Issues and PRs. I'm wondering what will be the right behavior here. I guess the easy one is just to make optional the body in the Issue model.

Could filter out PRs from the issue endpoint, but it would cause unnecessary traffic. I'd be happy with an Optional body for now, since even in the docs they tell you to filter them yourself based on the presence of the pr urls field

It seems like #236  has introduced a random test failure, which I've ignored here: https://github.com/47deg/github4s/pull/243/commits/24dd5566da0fd93fa95716889f5714ca32ca1174, as part of https://github.com/47deg/github4s/pull/243.
We might want to re-visit this issue later on.

I've done some debugging.
That test is failing because issue https://github.com/47deg/github4s/pull/1 has a body, and the test expects an empty body.
But if we use other issue with an empty body (that is a PR), like #154 , instead of receiving and empty body decoded to None, circe is decoding it to Some() :dizzy_face:
More debugging is needed to know why that is happening :thinking:


Clipboard.getString() not retrieving clipboard data.

Description
I was using a bluetooth barcode scanner which pastes the scanned data onto clipboard, the data scanned is pasted to notes(proof that data once scanned is pasted to clipboard).When i tried to get the clipboard data with Clipboard.getString(),it doesn't show any data.Am i doing something wrong?
Reproduction Steps and Sample Code
export default class ReadClipboard extends Component {
    getValue(){
        Clipboard.getString((res) => {
            alert(res)
        })
render() {
        return (
            <View>
                <TouchableOpacity
  style={{height:70,width:100,backgroundColor:'cyan',alignSelf:'center'}}
  onPress={this.getValue.bind(this)}
               >
                    <Text>clipboard</Text>
                </TouchableOpacity>
            </View>
 );
    }
}

Additional Information

React Native version: 0.44.0
Platform: Both
Development Operating System: macOS

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with Canny! Check out its new home: https://react-native.canny.io/feature-requests/p/clipboardgetstring-not-retrieving-clipboard-data

Still isn't working as of 0.50


Double-dash separator can be used between group name and commands, has odd behavior

This is not a bug, but an enhancement request -- I'd like to be able to turn off this behavior.
Right now, Click allows a group name and a command name to be separated using the -- separator.
I think this is confusing, and I would like it to raise an error in my CLI.
To explain and provide an example, imagine a command group foo with a command in it named bar.
You can write
$ foo bar --help

and things are happy.
You can also write
$ foo -- bar

and things work fine.
This behavior makes sense in my head, because bar is a positional argument to foo in a sense, but things are about to get messy and weird.
$ foo -- bar --help

This works, producing help output, and it really looks to me like it shouldn't. I guess the parsing for bar comes into play when it's read by foo and -- is just forgotten? I'm not sure.
I would like to entirely disable the usage of -- within the command tree. That is, I think it should be usable after a command name, but not between a group and a command.
So, foo -- ... should be entirely disallowed because it's a group. The only reason to use -- is to pass things like --help as arguments, and the parsing behavior here is confusing.
I don't see a viable use-case for writing foo -- bar ... over foo bar -- ....
I could see this being added as an optional behavior, since making it the default will potentially break existing scripts.
In the meantime, is there some way that I can disable this for my own application in Click 5 or 6?
I'm also happy to write this and submit a PR, if I can get a little bit of direction on (1) where I should be looking to customize the parsing and (2) whether or not this is a feasible small change (vs. uprooting a lot of well-established code, which I'm somewhat averse to).

Feel free to submit a PR with your proposed change. However, I think it would be hard to make an API change this significant, because as you said it could break a lot of peoples existing code.

Hi, I worked on this issue for quite a while and would like to submit a PR. In order to make the  "Add an entry in CHANGES.rst summarizing the change and linking to the issue" point of checklist done there are  questions:

Is it enough to write 'Using double-dash separator can be turned off. :issue:619' in CHANGES.rst?
Should some notes appear in docs/command.rst file?
P.S. the relevant notes in docstrings and comments in the code are done.

Yes to both. Thanks for working on this!

I've got fiew more questions, perhaps they are a little bit silly, but nevertheless:)
my local tox check showed some... problems. Tests succeded but there are those, which got skipped, because of my OS, and those about Python 3.6, 3.7, 3,8, pypy3, so my questions are:

Should I just install those interpreters to my machine and try again?
Skipped tests will work out later according to this: "CI will run the full suite when you submit your pull
request"?
There are typing errors... but those which I am linked to are just 3 of them, these others in _winconsole.py, _compat.py, _termui_impl.py. Those files I haven't touched, excplicitly.
Actually this is all.
P.S. If I am asking in the wrong place, I would be grateful for the redirection :)

I don't think this is possible as asked for in the op, although the PR does demonstrate a way to disable it everywhere. The op wants to disable it only until the last subcommand. One problem is that a given command doesn't know it's the last command until after it has finished its parsing/processing. Another is that a group can also be a valid command without any subcommands, and can take arguments which may look like options, just like any other command.
Parsing is handled by each command separately. The first command (group) parses the entire command line. Then it looks at any unknown tokens, picks the first one as the next command name, and passes the remaining tokens to be parsed by that command. If the "--" was passed on to each subsequent command after the first command encountered it, no command would be able to parse options. If the "--" wasn't allowed, no command would be able to have argument that look like options.
I don't think this is possible but will keep thinking about it while rewriting the parser.

I'm hesitant to allow disabling this. Perhaps if there is very clear documentation around "If you disable this on a command, it will be impossible for your users to provide an argument that looks like an option. Make sure this situation will never come up before doing this." The developer would also need to disable this individually for every command they do not want it on. Each command in the nested processing handles parsing individually, so each one might take an argument that could look like an option.

Hello, David. Glad to hear you, although we have war in Ukraine today, I will try to focus...
I see your point, and yes you are wright about the individual case on each command. I thought about different scenarios too, but then decided to take a literally straight approach just to hear your opinion about this and to hear if that is enough or I should consider different use cases too. So, basically now I am all listening the technical task if there will be such one from you. Honestly speaking I don't understand the case when we need <group> <--> <subcommand>. I think that an option is an option explicitly when we have <command> <--option>.

What you implemented in your PR is a good solution for this issue, assuming that the issue should be addressed. I'm currently rewriting the parser, which is why I came back to this.
Without discussing whether this particular CLI design is good, consider the following setup.
@click.group()
@click.pass_context
@click.argument("path", type=click.Path())
def process(ctx, path):
    click.ensure_object(dict)["path"] = path

@process.command()
@click.pass_context
@click.option("--save")
def reformat(ctx, save):
    ...

Let's say you want to process a file whose name starts with a -, such as -example.py. You would need to be able to do the following: process -- -example.py reformat --save However, if -- isn't allowed, then it's impossible to specify the file, it will raise an "unknown option" error. And if -- is only allowed once, not per command, then --save will raise an "extra argument" error instead of being treated as the save option.

As the OP, I want to mention something. When I filed this, we had just written a CLI application with click at work. I was concerned about ways in which users could confuse themselves, and tried all sorts of stuff like playing with -- to see what would happen.
In the 5+ years since then, I haven't seen a single user get mixed up by this behavior.
I think -- between a group and command is very odd. It would be preferable, IMO, to reject it. But I don't think this really needs to change, and have the aforementioned anecdotal data to back that up.

OK, based on the further discussion here, I'm going to close this.


feat: use request and response examples, see #764

See #764.
With this PR provided example request and responses are used for the code snippets and the example response card.
If you provide multiple examples, you’ll get a select. Here’s the full version:

Has some CSS issues, which are resolved in #810.

Moved the example picker to the bottom left
https://github.com/scalar/scalar/assets/6374090/6a36a3ef-0088-4d45-bb56-7f5545f35707


Select 3d object jetpack compose.

Hello everyone! I created an AR application with jetpack compose. I added 5 models to the screen using a tap, I did everything as in the example.
When I use
	val modelInstances = remember { mutableListOf<ModelInstance>() }
	val childNodes = rememberNodes()

onGestureListener = rememberOnGestureListener(
	onSingleTapConfirmed = { motionEvent, node ->

	}
)

I can't find 'node ->' in 'childNodes'
PS - Objects are added correctly. The childNodes list is the correct size.
How can I select multiple objects to change their color and delete them?

I was able to change the color of the selected model. Tell me, is this the optimal solution?
			onGestureListener = rememberOnGestureListener(
				onSingleTapConfirmed = { motionEvent, node ->
					if (node != null) {

						val nodes = node.childNodes
						val renderableNodes = nodes.mapNotNull {
							it as? ModelNode.RenderableNode
						}
						renderableNodes.forEach {
							it.materialInstance.setColor(colorModel.copy(0.1f))
						}
					}
				}
			)

The question remains - how can I select certain models in order to delete them later? If I do childNodes.indexOf(node) - I get an exception


Quick Actions: Foreground image actions

Context
Setting up more of the quick actions
Summary
Adds the quick actions when selecting a 'foreground image'.
Relevant Technical Choices
n/a
To-do

[ ] hide snackbar when the user clicks undo. maybe show a snackbar stating that the animations were re-applied
[x] finish karma tests for 'clear animations' action
[ ] update karma tests once #7522 is merged. Allows us to focus the tabs in the library when clicking a quick action.

User-facing changes

Gifs are here but very large

Action
Gif

Replace Media

Add Animation

Add Link

Clear Animations

Testing Instructions

Make sure quick actions experiment is enabled
Go to editor
Add image to the canvas (not as a background image)
Mess with actions - clicking and keyboard. Actions should do the following:

Replace media: should show the user the media3p panel in the library. Should focus the media3p tab.
Add animation: should focus the animation panel in the design panel. Should focus the dropdown.
Add link: should focus the link panel in the design panel. Should focus the input.
Clear animations: should remove the animations from the element. If there are no animations nothing should happen. Clicking undo in the snackbar that's shown should add the previously removed animation back to the element

QA

[ ] This is a non-user-facing change and requires no QA

UAT

[x] UAT should use the same steps as above.

Reviews
Does this PR have a security-related impact?
no
Does this PR change what data or activity we track or use?
no
Does this PR have a legal-related impact?
no
Checklist

[x] This PR addresses an existing issue and I have linked this PR to it in ZenHub
[x] I have tested this code to the best of my abilities
[x] I have verified accessibility to the best of my abilities (docs)
[x] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
[x] This PR contains automated tests (unit, integration, and/or e2e) to verify the code works as intended (docs)
[x] I have added documentation where necessary
[x] I have added a matching Type: XYZ label to the PR

Fixes #6143

One note that I will address tomorrow:
Clicking the undo button on the snackbar does not clear the snackbar. This lets the user click the undo button multiple times. I think clicking the undo button should clear the snackbar

Since this is behind a feature flag, I don't see any reason not to split up the additional work you've listed in 'todo' as separate PRs to avoid making this larger.

Dope - I'll do the other work in smaller prs 👍

Designs don't have this in them, but to be proactive I think we should disable the ability to clear animations/filters if there are none on the element. I think this should be separate work. These menus are a little sneaky and that's kinda why I tried to point them all rather large. I made a new ticket here: #7602

Awesome, thanks!

One thing I did notice is that the Link panel isn't expanding when I click on that menu item, wasn't sure if this is known or not. Totally cool to leave this for later. I think is ample opportunity in other menus to make sure that this behavior is present.

Good catch - I think we should come back to that in a different pr. I don't know why that's going on but it should be a simple add later on 🤞


UI problem in the second factor authentication (v.3.10.0)

Hi,
I wonder if you could help me.
I have an issue with the 3.10.0 version of authelia in the /secondfactor page.
I have no u2f extension installed in Firefox. But there is an iframe in the page which source is not accessible (because of the missing extension).

And this is ugly:

Could you please make it hidden in case of the absence of the extension? Thank you!

Hello @szimszon , thank you for reporting. I will try to find a way to detect if the browser has U2F enabled and load the lib only if necessary. That should solve this issue.
I'll let you know when it's done.
Thanks.

Hello @szimszon , I created a new UI for Authelia and I handled the case of the browser not handling U2F. In that case, the option is not even shown to the end user. I let you test it.
In the meantime, I'm closing the issue. Re-open it if you have any issue.


bash history

Guys, is it possible to add saving bash history between runs of develop jobs from the box? For example, create a file with history on storage in a project folder during setup and mount this file to file system in develop jobs.

Good idea! Though, we need to be very careful there. From security perspective, it might be a problem  if the user's bash history file is sent to the Platform without asking for permission as it might contain sensitive information. I would think of it more carefully and if needed, implement this feature with a trigger-flag disabled by default.

I see one possible implementation, yet its super-ugly:

after starting the job, copy bash history file from the storage
update bash history file on storage:
during the job run - sync the file with the storage.

This leads to:

develop job singleton. we cannot dump the updates from several jobs.
obligatory mount of project folder into the job with R/W mode
Not everybody will be sattisfied with that.

Do you need that @atselousov?
If yes, maybe it's a matter of docs. We might construct a config file for you, with help of which you will be able to sync bash history between job runs.

@atselousov, @mariyadavydova, another idea came into my mind.
What if we add a mixin, which will:

use a volume (or even define, in future) for storing bash/zsh history + *.rc files
point to the local bash/zsh files
upload(volume) to the storage on job startup and mount it to the job's /root or /home/user folder


Fix: ambiguity with newly introduced divisibility symbol

In newer lean versions, the divisibility symbol $\mid$ (\mid) is already defined, hence
infix:50 " ∣ " => divides

example (h₁ : divides x y) (h₂ : y = z) : divides x (2*z) :=
  calc
    x ∣ y   := h₁
    _ = z   := h₂
    _ ∣ 2*z := divides_mul ..

results in an ambiguity. So I replaced this symbol with double pipe $|$ (\|) and updated the explanation as to why we use this symbol instead of the canonical divisibility symbol.

Thanks! I also ran into this. Your code change fixed it for me.


Ability to disable/hide options that have already been selected

Is it possible to disable options in the dropdown in some way? In a multi select I would like to be able to hide the already selected options from the dropdown. I tried to use display: none; but when I navigate through the options with the arrow keys the hidden options are still highlighted. I'm also trying to hide the new tag option when it is a duplicate of an already available option, but since it is the first element in the dropdown if the user presses enter this will be the selected one even if not visible...
If not possible to disable is there an event being triggered on highlight? This way I could mark the options as disabled and trigger the highlight event in the next one if the current one is disabled / hidden.

:+1:

I have a use case for this too.  I was attempting to stylize the "disabled" options so it was visually obvious which elements were flagged as disabled without having to mouseover any scrutinize the cursors hover state or add text to the end of the displayed value.  (The Select2 demo for disabled options requires manually adding text "(disabled)" to the end of the label to indicate the option is disabled.)  There's no special disabled CSS class to the element, only aria-disabled="true".
After a little research, I've found that I could apply styles to disabled options by using the following CSS aria selector. (NOTE: I'm not sure if this will work in all browsers. I tested it using Firefox Dev 46.):
.select2-results__option[aria-disabled="true"] {color:#f00 !important;}

Hello
I came across this same solution earlier today, however there is an issue with navigation. When using arrow keys to navigate, the hidden options still "take up" an entry. So perhaps there is an improvement that could be made in this regard such that arrow navigation could skip over hidden options?
This could be done by changing the results:next trigger selector line from
var $options = self.$results.find('[aria-selected]');
var $options = self.$results.find('[aria-selected=false]');
but, i am not very familiar with this library to know of any drawbacks to this approach

There is a good reason that you wouldn't want to remove a selected item from the list completely.  See Kevin's explanation in #1538:

Right now the only way to unselect arbitrary (other than the last) options using the keyboard is to type CTRL + SPACE when the result is highlighted. This is to command for toggling a result, so it will select or unselect the highlighted result depending on what state it was previously in. Screen readers should read out the state of the option (whether it is selected or not) as they move through the results list, which is usually important.

Without an alternate way to unselect arbitrary options using the keyboard, the UX problems caused by that far outweigh the UX annoyances that you currently have by showing the selected results.

This should probably be documented since it is such a common question.

If this is an accessibility choice, would it not make more sense for screen readers to simply use the original <select> item instead of select2? What is the benefit of screen readers using select2 at all? (Please note, I'm not trying to talk negative about select2, but I genuinely don't know the answer. You could also simplify the code to not use any aria attributes).
Anyway, here's a way that allows you to hide the elements from browsers and keep them "visible" for screen readers:
.select2-results__option[aria-selected="true"] {
    position: absolute !important;
    clip: rect(1px 1px 1px 1px); /* In case you want to support IE6, IE7 */
    clip: rect(1px, 1px, 1px, 1px);
}

And then to fix the arrow keys, as @719media more or less said above, in the results:previous and results:next functions, simply do:
var $options = self.$results.find('[aria-selected=false]');

This should be the best of both worlds. Then you could include some sort of configuration option to accomplish this programatically. I'm assuming you would want to use the javascript to conditionally add some sort of data attribute to the selected options (in addition to aria), then the CSS would also match data instead of aria.

I don't think it's just about screen readers, though.  There are lots of contexts where you might see the Select2 control, but only be able to interact with it via the keyboard.
You're right though, this shouldn't be blocking on a configurable option to hide selected items.  Perhaps the ideal solution would be to make the actual selection pills navigable, so that they could be unselected directly via keyboard.  Then options could be safely hidden without the problem of items being unremovable in keyboard-only contexts.

Perhaps the ideal solution would be to make the actual selection pills navigable, so that they could be unselected directly via keyboard

That's how StackOverflow handles it for the question tagging. Pushing the left arrow when on the leftmost character changes the current tag back into a pill and makes the tag to the left become editable. The opposite for pushing the right arrow when there is a pill after the current tag being edited.

Ah, I was talking about tabbing through the x buttons to be able to actually clear entire selections (with or without tagging enabled).

Already commented at #4991 , but since it's closed I'll add it here as well.
I would suggest to be able to modify the original select and then properly display (after re-initialising).
I already saw several issues related to the fact that Select2 doesn't care about CSS styling (display:none), and I even noticed that it doesn't even care about the hidden attribute.
CodePen: https://codepen.io/anon/pen/JrJKBq
As you can see the default handling (at least in Chrome) will hide the options which have either display:none in the CSS or have the hidden attribute. At least it follows the disabled attribute.

For anyone else experiencing difficulties regarding this issue, as I did:
Finally, I reached the following solution(until the select2 lib supports this by itself).
I didn't need multiple selects however and I have not tested it in all browsers, so you may want to investigate these things if they are of concern to you.
First, to hide the selected value, I used the already mentioned CSS solution:
.select2-dropdown .select2-results__option[aria-selected=true] {
    display: none;
}

Next, to fix the arrow keys issue, I set up some event handling with JQ:
$selectElement.data('select2').on('results:next', function (event) {
    // find the currently highlighted element:
    var $currentSelection = $selectElement.data('select2').$dropdown.find('.select2-results__option--highlighted');
    // findout if its the currently selected element:
    if ($currentSelection.is('[aria-selected="true"]')) {
        // if it is, we trigger another key event, to skip this element
        // the following if is required to avoid recursion and to achieve default select2 behaviour
        if ($currentSelection.is(':last-child')) {
            // if its the last child, while pressing the down arrow key, we need to go back to the previous one, instead of going to the next one
            $selectElement.data('select2').trigger('results:previous');
        } else {
            // if its not the last child, while pressing the down arrow key, just go forward to the next element and skip this one
            $selectElement.data('select2').trigger('results:next');
        }
    }
});

// same as the first event, but in reverse:
$selectElement.data('select2').on('results:previous', function (event) {
    var $currentSelection = $selectElement.data('select2').$dropdown.find('.select2-results__option--highlighted');
    if ($currentSelection.is('[aria-selected="true"]')) {
        if ($currentSelection.is(':first-child')) {
            $selectElement.data('select2').trigger('results:next');
        } else {
            $selectElement.data('select2').trigger('results:previous');
        }
    }
});

And finally, my select dropdown has round border edges, so I needed to add border-radius only to the last element in the select list. But if the last element is the selected one, it will be invisible. After a few hours of trying to fix this using CSS, I figured that is impossible with CSS(maybe I am wrong). So I ended up using JQ to add a special CSS selector to the element that is the last visible select option element:
// this is binded to the highlighted elements only
$selectElement.data('select2').$dropdown.on('mouseenter', '.select2-results__option--highlighted', function () {
    if ($(this).is(':last-child')) {
        // if the last child is the current element, then give the css class to the prev element, otherwise give it to this element
        if ($(this).is('[aria-selected="true"]')) {
            if (!$(this).prev().hasClass('select2-real-last-child-fix')) {
                // remove any current elements with this css, can happen under special circumstances
                self.$statusSelect.data('select2').$dropdown.find('.select2-results__option').removeClass('select2-real-last-child-fix');
                $(this).prev().addClass('select2-real-last-child-fix');
            }
        } else {
            // same as the other if, but applied to this element, instead of the prev element
            if (!$(this).hasClass('select2-real-last-child-fix')) {
                self.$statusSelect.data('select2').$dropdown.find('.select2-results__option').removeClass('select2-real-last-child-fix');
                $(this).addClass('select2-real-last-child-fix');
            }
        }
    }
});

The templateResult callback option seems to work great for me:
templateResult: function formatState (state) {
    return state.selected ? null : state.text;
},

It is possible to use sorter function to remove such elements. It's quite sad though, that the "No data" message is checked to be displayed before sorting, not after it. That's why in the following snippet I just close the dropdown if it's about to be empty.
$(this.dropdown).select2({sorter: (data) => {
   let results = data.filter(item => !item.selected);
   if (data.length && !results.length) {
       $(this.dropdown).select2('close');
   }
   return results;
}

We don't have immediate plans to provide this. We are focused to fix some major UI bugs (that are majority of issues and PR's). But if you open a PR with unit tests, I will be glad to review and approve if everything is ok :+1:

+1

$("body").on('click', 'button.edit', function () {
        $('select.extra_future').val(null).trigger('change');
        var features = $(this).data('data');
        $.each(features, (ind, val) => {

            $('select.extra_future option[value="' + val + '"]').prop('selected', true).trigger('change');

            $('select.extra_future option[value="' + val + '"]').closest('optgroup').find('option').attr('disabled', true).trigger('change');
            $('select.extra_future option[value="' + val + '"]').attr('disabled', false).trigger('change');

            $('.extra_future').select2({
                placeholder: "SELECT",
                allowClear: true
            });

        })

    });

+1

Here's a workaround for cascading dropdowns using Select2, without Ajax.
https://codepen.io/lflier/pen/xxWXwpg

$(document).on('mouseenter', '.select2-results__option', function () {
$(this).css('background-color', '#97c121');
$(this).css('color', 'white');
});
// When the mouse leaves an option, remove the custom background color
$(document).on('mouseleave', '.select2-results__option', function () {
$(this).removeClass('select2-results__option--highlighted');
$(this).css('background-color', '');
$(this).css('color', '');
});
this works for me


resolved chat horizontal  scrolling

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Vikram Singh seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

tested and it works perfectly thanks @vkrm-segta !


update deployments function; better third party resource handling

update deployments function; better third party resource handling

LGTM

LGTM


[PROTOCOL-112] ATMGovernance contract

It includes:

ATMGovernance contract
IATMGovernance interface
New unit tests
Deploy script changed to include ATMGovernance deploy.

Great job! I added some questions/comments.


Implement POST endpoints for validators and validator_balances

https://github.com/ethereum/beacon-APIs/pull/367

Hey May I work on this issue ? @nflaig

That would be great, I assigned the issue to you

Great, but I see the PR you mentioned is already merged. Is it relevant now ?

Hi @lazyfuhrer , if you're still working on this and have any questions on how to potentially implement this, please feel free to join the ChainSafe Discord and find us on the lodestar group of channels!

This is really straight forward to implement and we should consider including that in the next release as for example Teku VC calls this API by default and has to fallback to GET API if it gets a 404.
Related issue https://github.com/Consensys/teku/issues/8118#issuecomment-2043814649


Numpy dtype

Hello,
I'm running Python 3.5 on Arch. I use Conda instead of pip, and noticed a package I use stopped working (seaborn). As soon as I ran import seaborn, there was a stack trace. I've traced the issue back to scipy, and managed to reproduce it. To reproduce it, you need Arch's package python-scipy or Conda's latest scipy package, and run import scipy.special._ufuncs. Here's the full stacktrace:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-256b318df750> in <module>()
      1 get_ipython().magic('matplotlib inline')
----> 2 import seaborn as snb

/home/matthias/.local/lib/python3.5/site-packages/seaborn/__init__.py in <module>()
----> 1 from .rcmod import *
      2 from .utils import *
      3 from .palettes import *
      4 from .linearmodels import *
      5 from .categorical import *

/home/matthias/.local/lib/python3.5/site-packages/seaborn/rcmod.py in <module>()
      3 import matplotlib as mpl
      4
----> 5 from . import palettes
      6
      7

/home/matthias/.local/lib/python3.5/site-packages/seaborn/palettes.py in <module>()
     12 from .external.six.moves import range
     13
---> 14 from .utils import desaturate, set_hls_values
     15 from .xkcd_rgb import xkcd_rgb
     16 from .crayons import crayons

/home/matthias/.local/lib/python3.5/site-packages/seaborn/utils.py in <module>()
      6
      7 import numpy as np
----> 8 from scipy import stats
      9 import pandas as pd
     10 import matplotlib.colors as mplcol

/home/matthias/miniconda3/envs/tipe/lib/python3.5/site-packages/scipy/stats/__init__.py in <module>()
    319 from __future__ import division, print_function, absolute_import
    320
--> 321 from .stats import *
    322 from .distributions import *
    323 from .morestats import *

/home/matthias/miniconda3/envs/tipe/lib/python3.5/site-packages/scipy/stats/stats.py in <module>()
    178 from scipy._lib.six import callable, string_types
    179 from numpy import array, asarray, ma, zeros
--> 180 import scipy.special as special
    181 import scipy.linalg as linalg
    182 import numpy as np

/home/matthias/miniconda3/envs/tipe/lib/python3.5/site-packages/scipy/special/__init__.py in <module>()
    599 from __future__ import division, print_function, absolute_import
    600
--> 601 from ._ufuncs import *
    602
    603 from .basic import *

__init__.pxd in init scipy.special._ufuncs (scipy/special/_ufuncs.c:27221)()

ValueError: numpy.dtype has the wrong size, try recompiling

As I said, this issue exists with both Arch's Scipy package (0.16.1), and Conda's (0.16.0). Numpy is in version 1.10.1 for both the Arch package and the Conda one. Here's the full list of packages in my Conda environment.
Thanks in advance for your attention

This error usually means that you're using a scipy build which is compiled against a newer version of numpy than you have installed. Can you verify that if you import numpy, you get version 1.10.1?

You're right, numpy.__version__ returns 1.9.3 whereas conda says 1.10.1. That's quite weird.

Look at numpy.__file__ to make sure that that numpy is really from within your conda install. If so, you can report this as a Conda bug.


[R] Set factor codes to 0-based when writing to Feather file

Closes #108
I had to update some Feather files in R/inst/feather because this changed the on-disk files. I still need to add a version number to the file metadata.

moving this comment out of the PR description
cc @hadley @krlmlr I am getting the following test failures that are outside my domain of knowledge:
> testthat::test_package("feather")
class : .......1.2.............
read : .34
roundtrip-vector : ....................

1. Failure (at test-class.R#17): basic access ----------------------------------
iris.f[, 1:5] is not identical to iris.tbl[, 1:5]. Differences:
Objects equal but not identical

2. Failure (at test-class.R#19): basic access ----------------------------------
iris.f[1:5] is not identical to iris.tbl[1:5]. Differences:
Objects equal but not identical

3. Failure (at test-read.R#6): multiplication works ----------------------------
read_feather(iris_example, columns = 1:3) is not identical to as_data_frame(iris)[1:3]. Differences:
Objects equal but not identical

4. Failure (at test-read.R#8): multiplication works ----------------------------
read_feather(iris_example, columns = "Species") is not identical to as_data_frame(iris)["Species"]. Differences:
Objects equal but not identical
Error: Test failures

I added a version number. Let me know if this looks good, as we should get this shipped soon to avoid accidental bad data getting read by Python users =|

I can't replicate the test failures in dad02f13d7.

Bumping the commit hash to get a new build, then merging on green. We should advise Feather users to update. I will tag a v0.2.0 release if that sounds good to all

@hadley @krlmlr it occurs to me that NA values in factors are probably handled incorrectly here. I opened #117 to track -- could one of you write a test case for this to verify one way or the other (since the codes were being written unmodified before, it is likely incorrect now)? Might have to test this with an integration test (Python generating null data in an int8 or int16-encoded factor).

I'll try to write some tests to check the cases I'm talking about (these are bugs that I may have introduced in https://github.com/wesm/feather/commit/06e69b0af92505e94ac785daf6c7f7d09236580c).


Rm rrule

Looks reasonable, anything in particular you want me to look at?
I guess it relies on https://github.com/ITensor/ITensors.jl/pull/843 to get merged before this is merged?


Tile rendering leaves a hole on globe

ION tileset id is 986442
You can see holes within the terrain. CesiumJS rendering is perfectly fine.

Also with my custom tiles, where Cesium JS  renders perfectly again, Unreal Engine plugin leaves a hole where it should up sample parent tile (because child tile is not available, layer.json says so) in addition to same bug that also present at ION tileset that i just described above. I can supply that as well if you want to check that out.

@MuhammedKalkan what version of Cesium for Unreal are you using? Check the Edit -> Plugins -> Geospatial page. I believe we fixed this problem in 1.13.

@kring Downloaded latest release (1.31.1) and extracted into Unreal 5.
Have you also seen before whole tile missing  thing before or just strip like hole ? Cant decide if you also consider that as a bug and require tile files

Prior to 1.13, layered terrain didn't work right at all in Cesium for Unreal. It would ignore the underlying layer, and try to fill in missing tiles with aggressive (and perhaps unsuccessful) upsampling. Are you still seeing a problem in 1.13.1?

Yup problem is in 1.13.1. My custom tileset uses layer.json rather than metadata extension so as you have pointed out, with this release (1.13.1) i have just tested it.
About ION generated terrain, never tried with older plugin versions.
Both tilesets are generated using same DEM file btw

Ok, can you please supply two Sandcastles with the two problematic datasets? Then we'll investigate why they aren't working the same in Cesium for Unreal. You can email them to me directly if they contain keys that you don't want to share publicly (kevin@cesium.com). Thanks!

@kring Sure!
I have sent you an email for both tilesets. Please feel free to ask for any assistance about custom tileset if you have any problems

I received your email, thanks @MuhammedKalkan. The problem with the ion tileset appears to be a known issue that @joseph-kaile happens to be working on right now. It's possibly related to https://github.com/CesiumGS/cesium-native/issues/385, but Joseph is still investigating.

@kring Thats great.
I have emailed you a sandcastle example with hosted custom tileset.
Please let me know if you  have any troubles

Fixed by https://github.com/CesiumGS/cesium-native/pull/500.


Event Time support in java client

Feature Request
This issue tracks adding event-time support in java client

introduce new Event Time field in message protobuf definition
introduce Getter and Setter in java client

This is part of #732 . See details at PIP-5.

The implementation is merged at #734


Getting import error

ImportError                               Traceback (most recent call last)
 in <cell line: 1>()
----> 1 from pdfminer.high_level import extract_text
2 import pdfminer.high_level
/usr/local/lib/python3.10/dist-packages/pdfminer/high_level.py in
6 from typing import Any, BinaryIO, Container, Iterator, Optional, cast
7
----> 8 from .converter import (
9     XMLConverter,
10     HTMLConverter,
ImportError: cannot import name 'HOCRConverter' from 'pdfminer.converter' (/usr/local/lib/python3.10/dist-packages/pdfminer/converter.py)

Having the same issue
from pdfminer.high_level import extract_text
extract_text(file_path)

Got an error:
ImportError: cannot import name 'HOCRConverter' from 'pdfminer.converter' (/usr/local/lib/python3.10/site-packages/pdfminer/converter.py)

same here

Same here

Same here

same here

same

Solved through
pip3 uninstall pdfminer
pip3 uninstall pdfminer-six
pip3 install pdfminer-six

Solved through
pip3 uninstall pdfminer
pip3 uninstall pdfminer-six
pip3 install pdfminer-six

import io

from pdfminer.converter import TextConverter, HTMLConverter
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage

pdf_path = "example.pdf"

input_file = pdf_path

i_f = open(input_file,'rb')
resMgr = PDFResourceManager()
retData = io.StringIO()
codec = 'utf-8'

TxtConverter = HTMLConverter(resMgr,retData, laparams= LAParams(), codec = codec)

interpreter = PDFPageInterpreter(resMgr,TxtConverter)
for page in PDFPage.get_pages(i_f):
    interpreter.process_page(page)

html_tags = retData.getvalue()
print(html_tags)

`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[23], line 17
14 retData = io.StringIO()
15 codec = 'utf-8'
---> 17 TxtConverter = HTMLConverter(resMgr,retData, laparams= LAParams(), codec = codec)
19 interpreter = PDFPageInterpreter(resMgr,TxtConverter)
20 for page in PDFPage.get_pages(i_f):
File ~/anaconda3/lib/python3.10/site-packages/pdfminer/converter.py:393, in HTMLConverter.init(self, rsrcmgr, outfp, codec, pageno, laparams, scale, fontscale, layoutmode, showpageno, pagemargin, imagewriter, debug, rect_colors, text_colors)
391 # write() assumes a codec for binary I/O, or no codec for text I/O.
392 if self.outfp_binary == (not self.codec):
--> 393     raise ValueError("Codec is required for a binary I/O output")
395 if text_colors is None:
396     text_colors = {"char": "black"}
ValueError: Codec is required for a binary I/O output`

Ran into the same issue but couldn't fix it. Finally had to go back to the previous version [20220524].

Solved through pip3 uninstall pdfminer pip3 uninstall pdfminer-six pip3 install pdfminer-six
If it doesn't work, you need to restart your computer / mac, it works for me.
Even though I don't know why, rebooting solves everything :)

pip3 uninstall pdfminer
pip3 uninstall pdfminer-six
pip3 install pdfminer-six

If it doesn't work, you need to restart your computer / mac, it works for me.
Even though I don't know why, rebooting solves everything :)


Cloud code 141 unauthorized even after setting useMasterKey:true

Issue Description
I Have the following cloud code which I am trying to execute without session token on android i am getting unauthorized even after setting useMasterKey as mentioned in docs. Am i missing something?
Parse.Cloud.define("sendotp", function (request, response) {
    var mobile_no = request.params.mobile_no;
    var Usr = new Parse.Query(Parse.User)
    Usr.equalTo("phone", mobile_no);
    Usr.first({ useMasterKey: true,
      success: function (usr) {
        // The object was retrieved successfully.
        if (usr.id != null) {
          Parse.Cloud.httpRequest({
              method: 'POST',
              headers: {
                'Content-Type': 'application/json;charset=utf-8',
                'application-Key': 'KEYREMOVED'
              },
              url: 'https://sendotp.msg91.com/api/generateOTP',
              body: {
                countryCode: "91",
                mobileNumber: mobile_no,
                getGeneratedOTP: true
              },
              sucess: function (httpResponse) {
                // success
                var otpres = JSON.parse(httpResponse.text);
                usr.set("newOTP", otpres.response.oneTimePassword);
                usr.save(null, { useMasterKey: true,
                    success: function (user) {
                        response.success('OTP Sent Successfully!')
                    },
                    error: function () {
                        response.error('Sending OTP Failed! Please try again!');
                    }
                });
              },
              error: function (httpResponse) {
                // error
                response.error('Sending OTP Failed! Please try again!');
              }
            });
        }
      },
      error: function (error) {
        response.error("No such user is found, kindly check your mobile no!");
      }
    });
});

Expected Results
{
  "message": "OTP Sent Successfully!"
}

Actual Outcome
{
  "code": 141,
  "error": "unauthorized"
}

Environment Setup

Server

parse-server version : 2.3.6
parse node module version: 1.9.2
Operating System:     Linux on Heroku
Hardware:            Heroku
Localhost or remote server? (AWS, Heroku, Azure, Digital Ocean, etc): Heroku

Database

Hosted on: mLab

Logs/Trace
Heroku Logs
2017-03-11T21:38:30.762860+00:00 heroku[router]: at=info method=POST path="/api/functions/sendotp" host=******.herokuapp.com request_id=a22f5699-093f-4b15-a5fe-b121e0cf2ebe fwd="<IP_ADDRESS>" dyno=web.1 connect=0ms service=3ms status=403 bytes=466 protocol=https
2017-03-11T21:38:30.725571+00:00 app[web.1]: warn: Making outgoing webhook request without webhookKey being set!
2017-03-11T21:38:30.764286+00:00 app[web.1]: error: Failed running cloud function sendotp for user undefined with:
2017-03-11T21:38:30.764290+00:00 app[web.1]:   Input: {"mobile_no":"9908640059"}
2017-03-11T21:38:30.764292+00:00 app[web.1]:   Error: {"code":141,"message":"unauthorized"} functionName=sendotp, code=141, message=unauthorized, mobile_no=9908640059, user=undefined
2017-03-11T21:38:30.765000+00:00 app[web.1]: error: Error generating response. ParseError { code: 141, message: 'unauthorized' } code=141, message=unauthorized
2017-03-11T21:38:30.765999+00:00 app[web.1]: error: unauthorized code=141, message=unauthorized

In your code, you seem to handle all errors inside your cloud function
Can you make sure:

The function is actually run
What part of the code is actually yielding an error

Can you provide the parse-server logs when running with VERBOSE=1 ? That may help finding the issue.

@flovilmart
Here are the Logs after setting verbose=1
2017-03-14T07:40:02.886299+00:00 heroku[web.1]: Restarting
2017-03-14T07:40:02.887373+00:00 heroku[web.1]: State changed from up to starting
2017-03-14T07:40:03.578746+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2017-03-14T07:40:03.706102+00:00 heroku[web.1]: Process exited with status 143
2017-03-14T07:40:06.973809+00:00 heroku[web.1]: Starting process with command `node ./index.js`
2017-03-14T07:40:09.868950+00:00 app[web.1]: API is running on port 32008.
2017-03-14T07:40:10.332227+00:00 heroku[web.1]: State changed from starting to up
2017-03-14T07:40:39.467890+00:00 heroku[router]: at=info method=POST path="/api/functions/sendotp" host=******.herokuapp.com request_id=5c256d32-36bf-45d9-b150-6aa209669eaf fwd="<IP_ADDRESS>" dyno=web.1 connect=0ms service=5ms status=403 bytes=466 protocol=https
2017-03-14T07:40:39.491347+00:00 heroku[router]: at=info method=POST path="/api/functions/sendotp" host=******.herokuapp.com request_id=29015a1e-d74b-4c4a-9ebb-92cd8a570e14 fwd="<IP_ADDRESS>" dyno=web.1 connect=1ms service=82ms status=400 bytes=563 protocol=https
2017-03-14T07:40:39.433493+00:00 app[web.1]: verbose: REQUEST for [POST] /api/functions/sendotp: {
2017-03-14T07:40:39.433503+00:00 app[web.1]:   "mobile_no": "9908640059"
2017-03-14T07:40:39.433511+00:00 app[web.1]: } method=POST, url=/api/functions/sendotp, host=*******.herokuapp.com, connection=close, origin=chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop, postman-token=09bf1b69-0522-2d0c-5607-b0b01034514f, cache-control=no-cache, x-parse-master-key=<REMOVED>, x-parse-application-id=<REMOVED>, user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36, content-type=text/plain;charset=UTF-8, accept=*/*, dnt=1, accept-encoding=gzip, deflate, br, accept-language=en-US,en;q=0.8,te;q=0.6,hi;q=0.4, x-request-id=29015a1e-d74b-4c4a-9ebb-92cd8a570e14, x-forwarded-for=<IP_ADDRESS>, x-forwarded-proto=https, x-forwarded-port=443, via=1.1 vegur, connect-time=1, x-request-start=1489477239406, total-route-time=0, content-length=29, mobile_no=9908640059
2017-03-14T07:40:39.435640+00:00 app[web.1]: warn: Making outgoing webhook request without webhookKey being set!
2017-03-14T07:40:39.478430+00:00 app[web.1]: error: Failed running cloud function sendotp for user undefined with:
2017-03-14T07:40:39.478433+00:00 app[web.1]:   Input: {"mobile_no":"9908640059"}
2017-03-14T07:40:39.478435+00:00 app[web.1]:   Error: {"code":141,"message":"unauthorized"} functionName=sendotp, code=141, message=unauthorized, mobile_no=9908640059, user=undefined
2017-03-14T07:40:39.482549+00:00 app[web.1]: error: Error generating response. ParseError { code: 141, message: 'unauthorized' } code=141, message=unauthorized
2017-03-14T07:40:39.485998+00:00 app[web.1]: error: unauthorized code=141, message=unauthorized

I can see the masterKey is set when you make the request to /api/functions/sendotp ? And that request originates from the client? You should remove the masterKey, it should NEVER be set on clients.

No change in response even if I don't send master key from the client.

if I don't send master key from the client.

You should remove the  masterKey from the clients, that's all.
Also you're using an external server for your web hooks right? If your cloud code is deployed with your parse-server you don't need to set those up. Are you able to create a simple function correctly? What kind of checks are you performing on the webhooks side? Are you using a module to wrap your Parse.Cloud.define ?
AFAICS, parse-server is creating the web hook request correctly, and your heroku server is 'responding' but I don't see any logs, so I'm not even sure your function is run.

Yeah actually deleting webhooks worked :) thank you.


Add Parametric Lap joint

As discussed with Davide, we will implement a lap joint that is controlled by 8 control points.
The setting out of the joint is similar to the NP joint.
We will keep a list of the corner intersection point in the joint attributes, including a centre point (plane).
Points will be kept in WCF.
Side and mid planes (3 on each side) can be derived from these points and center plane.
The control points will be derived from an offset from these side planes in a 2.5D manner.
Each control point have 2 unique parameters (16 in total) and share the depth parameter.

As furthered explored with Davide, the previous expectation of having 8 points are not enough to deal with many situations.
One such example is that for some joints, 8 points cannot be used to describe a rotated (relationship of the two-beam changed) but is a geometrical identical joint. See the crossed out one:

The conclusion is that it is easier to manage a joint with 4 polylines drawn. Each of them describing two sides of the lap joint. The two sides are joined at a specific height.

Some strings for testing:
[50.,[[[0.,0.],[0.,0.2],[1.,0.2],[1.,0.],],[[0.,0.],[1.,0.],],[[0.,0.],[0.,0.2],[1.,0.2],[1.,0.],],[[0.,0.],[1.,0.],],]]

[50.,[[[0.,0.],[0.2,0.],[0.2,0.2],[0.8,0.2],[0.8,0.],[1.,0.],],[[0.,0.],[0.,0.2],[1.,0.2],[1.,0.],],[[0.,0.],[0.2,0.],[0.2,0.2],[0.8,0.2],[0.8,0.],[1.,0.],],[[0.,0.],[0.,0.2],[1.,0.2],[1.,0.],],]]

[50.,[[[0.,0.],[0.038,0.192],[1.,0.],],[[0.,0.],[1.,0.],],[[0.,0.],[0.963,0.193],[1.,0.],],[[0.,0.],[0.193,0.037],[0.808,0.038],[1.,0.],],]]

[50.,[[[0.,0.],[1.,0.],],[[0.,0.],[0.5,0.2],[1.,0.],],[[0.,0.],[1.,0.],],[[0.,0.],[0.5,0.2],[1.,0.],],]]

[50.,[[[0.,0.],[0.2,0.2],[0.8,0.2],[1.,0.],],[[0.,0.],[1.,0.],],[[0.,0.],[0.2,0.2],[0.8,0.2],[1.,0.],],[[0.,0.],[1.,0.],],]]

closed via e00fb8b496a3b2be505e5443b6f5063246dcaf09


Prioritize branches above non-semver tags

Sort ordering needs to change to prefer branches above non-semver tags. In another lang community, that might not be sane, but the strong tendency towards the 'default branch' established by go get means it's what makes sense here.

The need to do this is also noted in #65


Fix memory leak of volatile fs caused by empty directories

Depends on

https://github.com/Nitrokey/trussed/pull/46

No significant changes.

Insignifcant changes

metric
value

change

binary-size-nk3am
1,505,330
:red_circle:
+2,536 (+0.17%)

binary-size-nk3am-test
1,883,610
:red_circle:
+3,097 (+0.16%)

binary-size-nk3xn
541,100
:red_circle:
+800 (+0.15%)

binary-size-nk3xn-test
541,204
:red_circle:
+808 (+0.15%)

binary-size-nkpk
715,434
:red_circle:
+2,393 (+0.34%)


Quantity Selector

Closes #113
Preview: https://deploy-preview-860--boosted.netlify.app/docs/5.1/forms/quantity-selector/
DoD
Development

[x] Should match specs (eg. either the Web UI Kit or any pattern from the WAI — or both…)
[x] Docs added:

including the "Sass" part using scss-docs shortcode
in Cheatsheet / Cheatsheet RTL examples
in /about/overview/#custom-components if it is a new Orange custom component
in /gettings-started/introduction/#components if it is a new Orange custom component that requires JavaScript (and Popper)
in /customize/overview#csps-and-embedded-svgs if it is a new Orange custom component that includes embedded SVGs in our CSS
in /forms/validation/?#supported-elements if it is a new Orange custom component that is a form control

[x] Check (and fix) RTL version
[x] Run linters
[x] Run compilers
[x] Tests added for JS-side
[x] Run tests
[x] Cross-browser test:

Firefox ESR
Latest Edge, Chrome, Firefox, Safari
iOS Safari
Chrome & Firefox on Android

[ ] Clean up the branch using rebase -i
[ ] Commited with feat(…): … message
Mention it in Migration Guide (if back-from-v4): renamed variables, changes in markup requirement,  etc.

Reviews

[x] Code review
[x] A11y review
[ ] Design review

@MewenLeHo Something's missing in what you've force-pushed:
Error: Error building site: TOCSS: failed to transform "scss/docs.scss" (text/x-scss): SCSS processing failed: file "/home/ju/Orange-Boosted-Bootstrap/scss/_variables.scss", line 1849, col 43: Undefined variable: "$add-icon"

All the jobs are :red_circle:

Hi, Here are some comments on the visual design of the Quantity Selector component.
On the standard size version:

The plus and minus symbols need to move down 1px
The counter number needs to move up 1px

On the small size version:

The plus and minus symbols need to move down 1px
The plus and minus symbols on the small version should use specific icons sized for this version of the component. Please refer to the Solaris icon library. Smaller versions of the icons are called: Form Plus small and Form Minus small

For both versions:

For the disabled state, please remove the light grey colour behind the number. Just use the number on a white background with the font colour set to #CCCCCC.
When the counter number is at '0' (zero), grey out the minus symbol (disabled state colour #CCCCCC).

Thanks

Hi @Franco-Riccitelli,
Thanks a lot for your suggestions.

On the standard size version

The plus and minus icons need to move down 1px
We are aware of this issue but it's not specific to this new component that's why we will address this problem in another PR.

The counter number needs to move up 1px
Could you please explain why? This input is simply using the standard '.form-control' class which is the same everywhere in Boosted so we don't understand why we should make this modification and why we should do it only here and not everywhere.

On the small size version:

The plus and minus icons need to move down 1px
Same anwser.

The plus and minus icons on the small version should use specific icons sized for this version of the component. Please refer to the Solaris icon library. Smaller versions of the icons are called: Form Plus small and Form Minus small
No. We tried to used Solaris icons at the early stages of development but they were not suitable because of the white spaces around some of them.
We had a disccusion abou it in Mattermost and @CyriaqueBillard even said that 'the issue here is that in our design for that component, the icons are not the Solaris ones'.
Current icons were sent by @CyriaqueBillard because after that.

For both versions:

For the disabled state, please remove the light grey colour behind the number. Just use the number on a white background with the font colour set to #CCCCCC.
When the counter number is at '0' (zero), disable the minus icon (disabled state colour #CCCCCC).
Not sure about that and @julien-deramond  is ok with me.
The input disabled statre should be consistend all over Boosted: https://boosted.orange.com/docs/5.1/forms/form-control/#disabled
In addition, with a white background and #CCCCCC text color the constrast will be too low and we might have an accessibility issue.

After the meeting with @CyriaqueBillard, @Franco-Riccitelli, @PigeardSylvain and @Aniort, we agreed that:

We don't have to restrain user to enter no more than 3 numbers in the input or to make the input strechable. Instead I will add a warning in the documentation to inform users that this component is suitable for incremental and decremental display of small numbers and that we do not recommand to use it with large numbers, i.e. numbers with more than 3-4 digits.

We need to change the disabled state of the component according to the recommandation made by @Franco-Riccitelli, there is no accessibility issue with this new version according to @Aniort and @PigeardSylvain.

The counter number needs to move up 1px: @Franco-Riccitelli will investigate because this issue is probably general and therefore should be applied to the whole framework. If true, we will address it in a specific pull request.

The plus and minus icons should be 1px lower: this issue is general and not specific to this new componenet so we will address it in separate and more generic pull request (https://github.com/Orange-OpenSource/Orange-Boosted-Bootstrap/pull/1020)

Following the call yesterday, I have checked the vertical positioning of the input text in text boxes that have an overall height of 40px. The reference for this was the Form Control page on Boosted:
https://boosted.orange.com/docs/5.1/forms/form-control/#disabled
I checked these using the Chrome browser on a Mac and PC.
The test showed that all the 40px high text boxes on this Boosted page include the input text at the correct vertical position, so no change is needed to those elements.
The Quantity Selector counter number however is 1px lower than the other Boosted form elements. See attached image.

A new version is now live, including all fixes requested by @Franco-Riccitelli and @CyriaqueBillard : https://deploy-preview-860--boosted.netlify.app/docs/5.1/forms/quantity-selector/
Could you please check if everything is ok for you?

The positioning of the plus and minus icons on both the default and small versions are good.
The small version is now using the correct plus and minus icons.
The vertical positioning of the number counter on both versions are also now correct. Thanks

The focus may indeed be suddenly lost when a button become disabled.
A solution may be to programmatically sets focus() to the most appropriate location inside the component:
Should we set the focus to the second button when the first one become disabled?
Any opinion @julien-deramond and @louismaximepiton?

Here is my review, I got several questions :

[ ]  Does QS need an example with Label ?

[ ]  Minor issue with keyboard users : Minus button isn't accessible when :

there was a '0' in the input
User enter for example a '5' and press tab directly

[ ]  Same issue as above but replace '5' and '0' -> Minus button is accessible but shouldn't


yml install not working...

Thank you for taking the time to create this. Great work !
I tried to install the container using the yml file on qnap nas container station and also through portainer stacks...
The install failed :
errorlog: the patch file is only an update and does not contain full dsm install files!
The run command works ok!
Maybe take a look ?

No, just ran the published yml...

I'm from NL !
The logs have gone as I reinstalled using the 'run' command...
The download works fine...
The error appeared after the extraction starts and it than enter in a loop and keeps downloading the pat file 64551.
That is all the info I can provide right now...
Bye

@intraxit what CPU arch is your QNAP?
I tried (yes I know it can not work due to arch) it on the raspberry pi 4 (ARM) and I got "same" error:
dsm  | ❯ Install: Extracting downloaded image...
dsm  | /run/install.sh: line 95: /run/extract/syno_extract_system_patch: cannot execute binary file: Exec format error
dsm  | ❯ ERROR: Invalid PAT file: File is an update pack which contains no OS image.
dsm exited with code 0

@kroese TIP: maybe add the CPU architecture check and warning or immediate stop the container

Yep that's the error!
I'm running qnap qts on ryzen 5500u laptop in kvm vm arch x64, so should be fine!
Remember, I was able to install the container using the run command and it is working ok.
So maybe docker reports wrong arch on ryzen chip if using yml ?

@kroese no, just run docker compose up
According to: Multi-platform images it is automatic.
"When running an image with multi-platform support, docker automatically selects the image that matches your OS and architecture."

@kroese  Afaik I am on the right platform (amd64), but somehow yml thinks apperantly something else..
@kissi7 Could you install kvm/qemu on raspberry pi and try it in a vm ? Interesting experiment imo...

@kroese I take it back and apologize! I see I used your Dockerfile and built virtual-dsm container directly on RPi4 locally. The build process took arm arch and than I was able to start it up by docker compose up (yes I change image in docker-compose.yml to use local one). I'm sorry I confused you and you are right it should not start by default at all on different arch.

@kroese
Have you tried to run the yml in portainer or only in docker compose ?
Arm chips will become much faster very soon (Apple M2) !

As mentioned the log files have disappeared as I deleted the container and reinstalled using the 'run' command. I will try again tmw and post the log files if possible...

installed today using portainer stack and same error...
logs:
❯ Starting Virtual DSM for Docker v4.0...
❯ Install: Downloading installer...
❯ Install: Downloading DSM_VirtualDSM_64551.pat...
0K ........ ........ ........ ........  9% 8.42M 36s
32768K ........ ........ ........ ........ 18% 8.48M 32s
65536K ........ ........ ........ ........ 28% 8.50M 29s
98304K ........ ........ ........ ........ 37% 8.02M 25s
131072K ........ ........ ........ ........ 47% 8.46M 21s
163840K ........ ........ ........ ........ 56% 8.06M 18s
196608K ........ ........ ........ ........ 66% 8.09M 14s
229376K ........ ........ ........ ........ 75% 8.56M 10s
262144K ........ ........ ........ ........ 85% 8.02M 6s
294912K ........ ........ ........ ........ 94% 8.53M 2s
327680K ........ ........ ..               100% 8.45M=41s
❯ Install: Extracting downloaded image...
❯ ERROR: Invalid PAT file: File is an update pack which contains no OS image.
❯ Starting Virtual DSM for Docker v4.0...
❯ Install: Downloading DSM_VirtualDSM_64551.pat...
0K ........ ........ ........ ........  9% 8.25M 37s
32768K ........ ........ ........ ........ 18% 8.45M 33s
65536K ........ ........ ........ ........ 28% 8.48M 29s
98304K ........ ........ ........ ........ 37% 8.07M 25s
131072K ........ ........ ........ ........ 47% 8.51M 21s
163840K ........ ........ ........ ........ 56% 8.49M 17s
196608K ........ ........ ........ ........ 66% 8.02M 14s
229376K ........ ........ ........ ........ 75% 8.50M 10s
262144K ........ ........ ........ ........ 85% 8.46M 6s
294912K ........ ........ ........ ........ 94% 8.05M 2s
327680K ........ ........ ..               100% 8.46M=41s
❯ Install: Extracting downloaded image...
❯ ERROR: Invalid PAT file: File is an update pack which contains no OS image.
❯ Starting Virtual DSM for Docker v4.0...
❯ Install: Downloading DSM_VirtualDSM_64551.pat...
0K ........ ........ ........ ........  9% 8.22M 37s
32768K ........ ........ ........ ........ 18% 8.47M 33s
65536K ........ ........ ........ ........ 28% 8.43M 29s
98304K ........ ........ ........ ........ 37% 8.53M 25s
131072K ........ ........ ........ ........ 47% 8.50M 21s
163840K ........ ........ ........ ........ 56% 8.49M 17s
196608K ........ ........ ........ ........ 66% 7.96M 14s
229376K ........ ........ ........ ........ 75% 8.53M 10s
262144K ........ ........ ........ ........ 85% 8.45M 6s
294912K ........ ........ .....
no specific error as to why the syno_extract_system_patch is not working...

There is no reason code displayed in the log as it is looping (see above log).

same story on 4.03:
❯ Starting Virtual DSM for Docker v4.03...
❯ Install: Downloading DSM_VirtualDSM_64561.pat...
0K ........ ........ ........ ........  9% 8.43M 36s
32768K ........ ........ ........ ........ 18% 8.46M 32s
65536K ........ ........ ........ ........ 28% 8.45M 29s
98304K ........ ........ ........ ........ 37% 8.49M 25s
131072K ........ ........ ........ ........ 47% 8.53M 21s
163840K ........ ........ ........ ........ 56% 8.06M 17s
196608K ........ ........ ........ ........ 66% 8.47M 14s
229376K ........ ........ ........ ........ 75% 8.49M 10s
262144K ........ ........ ........ ........ 85% 8.01M 6s
294912K ........ ........ ........ ........ 94% 8.52M 2s
327680K ........ ........ .                100% 8.51M=40s
❯ Install: Extracting downloaded image...
❯ ERROR: Failed to extract PAT file, reason 255
❯ Starting Virtual DSM for Docker v4.03...
❯ Install: Downloading DSM_VirtualDSM_64561.pat...
0K ........ ........ ........ ........  9% 8.10M 38s
32768K ........ ........ ........ ........ 18% 8.58M 33s
65536K ........ ........ ........ ........ 28% 8.03M 29s
98304K ........ ........ ........ ........ 37% 8.53M 25s
131072K ........ ........ ........ ........ 47% 8.47M 21s
163840K ........ ........ ........ ........ 56% 8.08M 18s
196608K ........ ........ ........ ........ 66% 8.44M 14s
229376K ........ ........ ........ ........ 75% 8.54M 10s
262144K ........ ..
exit code 137...
If I were to guess this is probably a permissions problem...
The /storage directory is mapped to /opt/dsm on the host and has a suid bit for the group administrators.
It yml fails to extract the pat file in that directory or has no permissions to run anyway... Just my guess!

If I remove the volumes: directive the yml install works ok !
So indeed a permissions error...
If I add env. variables and leave the /storage mapping:
UID=0
GID=0
0:0 is the admin user on qnap!
Extraction again failed...
What user are you using in the container to run the extraction?
Now there is probably a permission problem inside the container...confusing!

If I connect the /storage volume to the exposed volume in qnap container station everything works as expected...
so:
volumes:
- /opt/dsm:/storage
NOT working on qnap container station!

The way this works on qnap is exposing a volume through qnap container station. That volume has the right permissions set for docker to be able to read/write from the container. Connecting that specific volume to the /storage directory in the container solves the permission problems.


Error on creating empty bot + cannot port older project over

Describe the bug
After installation of new version, am now unable to convert older projects to new version.
Am also unable to create a new chatbot from scratch, error message as shown below:
Error calling npm to fetch template. Please ensure that node and npm are installed and available on your system. Full error message: Command failed with ENOENT: npm root -g --prefix C:\Users\Me\AppData\Roaming\BotFrameworkComposer.yo-repository --loglevel error spawnSync cmd.exe ENOENT
Version

Browser

[ ] Electron distribution
[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

OS

[ ] macOS
[x] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:
After updating:

Go to 'Home'
Click on 'Create new' and create an empty bot
See error

Expected behavior
Should have created an empty bot when starting from scratch.
Screenshots

Additional context
Am unsure if nodejs is the cause of issue: have tried to install latest version 14.17 but it does not seem to show in the versions listed in "About"

@Jin825 Could you confirm if npm is in your PATH evironment variable?

The following PowerShell should give you a quick answer:
$env:path.split(";") | select-string node

For me, this gives something like:
C:\Program Files\nodejs\

Hi All, yes I got the same result as @cwhitten.
Personally have reinstalled node and input in npm into env variables before posting error

Thank you @Jin825
We're still looking into this. Stand by.

@Jin825 A couple things you can try to help us debug this issue.

Could you try doing a global install of any npm package using the npm -g syntax in you command line.
Could you try the above command that is erroring out directly in the command line and see if there is any difference between running as admin vs running as a user.

@Jin825 Are you able to do any of the above as suggested by Srinaath?

Hi @Jin825 Just checking in again. Please let me know if you still need assistance. And if so; what are the answers to the above questions.

Closing due to lack of response. If you are still having issues and need assistance, please try the above suggestions and let us know the findings.


Smart indent should be aligning arguments

x <- function(a<ENTER>

should position caret under 'a'

#1285

While it does achieve what was described in the example above, I feel it's not quite correct and need to allow more flexibility, like in RS.

It should be aligning based on the column of the previous argument.  Right now it's always aligning with column+1 of the open paren.
When there's no previous argument and you press ENTER, it should indent one level relative to the first char of the function name.

In RS it is easy to do this:
res <- this.function.name.is.somewhat.long(
  'this long message would go over 80 cols if aligned with open paren',
  'other parameter'
)

But trying to do the same in RTVS is infuriating.
Examples (where indentation is 2 spaces):
action:
res <- myfunc(<ENTER>

expected:
res <- myfunc(
  <caret>

actual:
res <- myfunc(
              <caret>

action:
res <- myfunc(1,<ENTER>

expected & actual (already correct):
res <- myfunc(1,
              <caret>

action:

res <- myfunc(
1,

expected:

res <- myfunc(
1,

actual:

myfunc(
1,

)

We used to handle user indent, but user found it improper and rather have arguments aligned. However, we can technically combine smart indent and block indent (i.e. that is what RS does based on the previous line).

If they're typing a new call, user indent will equate to having the arguments aligned (in a way that user explicitly asked for by the placement of the first argument), no?

RS:
x <- function(a,<ENTER>
                     <caret>


State is cleared on fetch actions

Summary
The state previously was not cleared on fetch actions causing an incorrect state should the fetch action fail, leading to 500 errors.  Mostly notable when switching between local and remote IDPs.
Description for the changelog
State is now cleared on fetch actions
Other info
V2 only, Vue specific

Codecov Report

Merging #181 (5d75913) into v2-development (2c80489) will increase coverage by 1.68%.
The diff coverage is n/a.

@@                Coverage Diff                 @@
##           v2-development     #181      +/-   ##
==================================================
+ Coverage           97.46%   99.15%   +1.68%
==================================================
  Files                  60       28      -32
  Lines                 828      471     -357
  Branches               13        0      -13
==================================================
- Hits                  807      467     -340
+ Misses                 21        4      -17

Impacted Files
Coverage Δ

src/views/Repository.vue

src/store/modules/provider.js

src/store/modules/loader.js

src/views/Branch.vue

src/views/Home.vue

src/views/ThreatmodelEdit.vue

src/views/Dashboard.vue

src/store/index.js

src/store/actions/provider.js

src/components/Navbar.vue

... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c80489...5d75913. Read the comment docs.


java.nio.channels.AsynchronousCloseException while stopping the identity server instance

Describe the issue:
An error is being thrown while stopping the Identity server instance.
[2023-08-16 15:20:20,731] [] ERROR {org.apache.tomcat.util.net.Acceptor} - Socket accept failed java.nio.channels.AsynchronousCloseException
	at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.end(ServerSocketChannelImpl.java:269)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:288)
	at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:548)
	at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:79)
	at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
	at java.base/java.lang.Thread.run(Thread.java:829)

How to reproduce:

Get a latest identity server pack
Start the server with the relevant script command
If the server started successfully, execute the command to stop it.
The above mentioned error will be visible in the terminal logs.

Expected behavior:
No errors should be visible in the terminal logs.
Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: IS 6.2.0-alpha4-SNAPSHOT
Java versions: jdk11 & jdk17

Optional Fields
Related issues:

Suggested labels:

Analysis

Recreated the issue for the following setup. (Note: Once in a while, the issue is not reproducing.)

IS Version: 6.2.0 alpha 5 SNAPSHOT
OS: macos Ventura 13.5
Database: Default

Furthermore, the exception is thrown by the shutdown method of EclipseStarter[1] which is called by the CarbonServerManager[2].

[1] - https://help.eclipse.org/latest/nftopic/org.eclipse.platform.doc.isv/reference/api/org/eclipse/core/runtime/adaptor/EclipseStarter.html
[2] - https://github.com/wso2/carbon-kernel/blob/2aa7489f5800f5a4a3f96f62e3c4250bce4075d1/core/org.wso2.carbon.core/src/main/java/org/wso2/carbon/core/init/CarbonServerManager.java#L865

hi @dhaura
thanks for the thorough analysis. Need to verify if this is an OS-specific issue or not. Can you add your findings with the linux testing you've done?
@melanisilva can you please help with testing this on a windows setup?

Analysis Ctd

The error is thrown from here [1] and logged from here [2].
Error is NOT reproducible in IS 6.1.0, 6.2.0-alpha3. (on mac)
Error reproducible in IS 6.2.0-alpha4, 6.2.0-alpha5-SNAPSHOT. (on mac)
An extra log can be seen at the startup (compared to “issue not reproducible pack” startups):

[2023-08-28 17:11:54,567] []  INFO {org.apache.tomcat.util.net.NioEndpoint.certificate} - Connector [https-jsse-nio-9443], TLS virtual host [_default_], certificate type [UNDEFINED] configured from [/Users/dhaurapathirana/Documents/WSO2-IS/wso2is-6.2.0-alpha5-SNAPSHOT-Aug-28/repository/resources/security/wso2carbon.jks] using alias [wso2carbon] and with trust store [/Users/dhaurapathirana/Documents/WSO2-IS/wso2is-6.2.0-alpha5-SNAPSHOT-Aug-28/repository/resources/security/client-truststore.jks]

Probable Causes

Inside the shutdown method [3], the error is thrown while calling framework.waitForStop(0). Therefore, maybe framework.getState() is not updated in time.

[1] - https://github.com/apache/tomcat/blob/79900c9eef9aac2e892af73f23d9b16337deac27/java/org/apache/tomcat/util/net/Acceptor.java#L129
[2] - https://github.com/apache/tomcat/blob/79900c9eef9aac2e892af73f23d9b16337deac27/java/org/apache/tomcat/util/net/Acceptor.java#L171
[3] - https://help.eclipse.org/latest/nftopic/org.eclipse.platform.doc.isv/reference/api/org/eclipse/core/runtime/adaptor/EclipseStarter.html

hi @dhaura thanks for the thorough analysis. Need to verify if this is an OS-specific issue or not. Can you add your findings with the linux testing you've done? @melanisilva can you please help with testing this on a windows setup?

Sure , Will check.

An extra log can be seen at the startup (compared to “issue not reproducible pack” startups)

The logCertificate [1] method is hit by Error Reproducible IS packs but NOT by other Error Not Reproducible IS packs
[1] - https://github.com/apache/tomcat/blob/497f18240318e623ae608f8d4bdcbb366e2eab54/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L375

Analysis Ctd

A Tomcat version bump [1] for was carried out from v9.0.58.wso2v1 to v9.0.74.wso2v1 in IS and with that, this error is introduced.
Root cause for this is that a socket where Tomcat Acceptor is waiting [2], is getting closed by some other process. Thus, the AsynchronousCloseException is thrown.
Furthermore, this error happens intermittently.

[1] - https://github.com/wso2/carbon-kernel/pull/3611
[2] - https://github.com/apache/tomcat/blob/79900c9eef9aac2e892af73f23d9b16337deac27/java/org/apache/tomcat/util/net/Acceptor.java#L129

I'm putting this issue on hold since this is not a blocking issue for the release. Reducing the priority as well.

This was not reproducible on Linux. Currently only reproducible on MacOS. Hence it is likely to be an OS issue.

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


⚠ Jenkins Plugins Artifactory has degraded performance

In 44826bd, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1233 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 3ed181b after 18 minutes.


Implicit declaration in C99 error when packing a rails project using Ruby 2.4.10

Hey @pmq20 first of all, I'm really glad you decided to work on a library like this as it's something I've been looking for for a long time.
I'm having an issue compiling a base project made by just installing ruby 2.4.10 via rvm, generating a base rails project (rails new testproject), and packing that project using rubyc -r .. Same issue in ruby 2.5.
Relevant log lines:
squash_dir.c:479:2: error: implicit declaration of function 'free' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        free(path0);
        ^
1 error generated.
make: *** [squash_dir.o] Error 1
Failed running [{"CI"=>"true", "ENCLOSE_IO_USE_ORIGINAL_RUBY"=>"1", "CFLAGS"=>" -fPIC -O3 -fno-fast-math -ggdb3 -Os -fdata-sections -ffunction-sections -pipe  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/zlib  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/openssl/include  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/gdbm/build/include  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/yaml/build/include  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/libffi/build/lib/libffi-3.2.1/include  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/ncurses/build/include  -I/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/readline/build/include ", "LDFLAGS"=>" -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/zlib /var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/zlib/libz.a  -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/openssl  -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/gdbm/build/lib  -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/yaml/build/lib  -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/libffi/build/lib  -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/ncurses/build/lib  -L/var/folders/ts/g6q5nn2x3sv4xnz3f7wzyd840000gn/T/rubyc/readline/build/lib ", "ENCLOSE_IO_RUBYC_1ST_PASS"=>"1", "ENCLOSE_IO_RUBYC_2ND_PASS"=>nil}, "make -j4 -j1"]

More logs at https://pastebin.com/VZGqqtst
I'm guessing this is some sort of issue with my system setup but I'm not really sure where to start. Any ideas? Should I be using another ruby version?

Are you able to solve this issue @dennislysenko? I am also having the same issue. Can you please post the answer if you got the solution?

With help of @CAMOBAP on macOS this error disappeared with:
env CC="xcrun clang -mmacosx-version-min=10.10 -Wno-implicit-function-declaration" bin/rubyc --clean-tmp bin/rubyc -o rubyc-darwin-x64

I'm getting a similar issue trying to build an executable gem (see issue #170). Would something like this solution help you think?


Read cluster name from pharos-config configmap

This PR will improve license assign command in several ways:

it fixes cluster id in assign request payload: cluster_id -> cluster-id
it removes duplicated [LICENSE KEY] from help output
it reads cluster name from pharos-config configmap
it enhances output messages

Fixes #1143

@kke PTAL

@kke PTAL one more time.