Can't Update Fastlane to Latest Version

New Issue Checklist

[ ] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
I was trying to install a plugin for Cordova support and got the following error:
bash-3.2$ fastlane add_plugin cordova
Ignoring nokogiri-<IP_ADDRESS> because its extensions are not built.  Try: gem pristine nokogiri --version <IP_ADDRESS>

/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/user_interaction.rb:334: warning: constant ::TimeoutError is deprecated
[13:22:32]: -------------------------------------------------
[13:22:32]: --- Step: Verifying required fastlane version ---
[13:22:32]: -------------------------------------------------
[13:22:32]: fastlane version valid
[13:22:32]: ------------------------------
[13:22:32]: --- Step: default_platform ---
[13:22:32]: ------------------------------
[13:22:32]: Variable Dump:
[13:22:32]: {:DEFAULT_PLATFORM=>:ios}
[13:22:32]: Could not find lane 'add_plugin cordova'. Available lanes: ios test, ios beta, ios deploy

+------+-------------------------------------+-------------+
|                     fastlane summary                     |
+------+-------------------------------------+-------------+
| Step | Action                              | Time (in s) |
+------+-------------------------------------+-------------+
| 1    | Verifying required fastlane version | 0           |
| 2    | default_platform                    | 0           |
+------+-------------------------------------+-------------+

[13:22:32]: fastlane finished with errors
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/runner.rb:31:in `execute': Could not find lane 'add_plugin cordova'. Available lanes: ios test, ios beta, ios deploy (RuntimeError)
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/lane_manager.rb:43:in `cruise_lane'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/command_line_handler.rb:31:in `handle'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:36:in `block (2 levels) in run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/command.rb:153:in `run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/runner.rb:428:in `run_active_command'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/runner.rb:68:in `run!'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/delegates.rb:15:in `run!'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:127:in `run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:133:in `<top (required)>'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane:23:in `load'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane:23:in `<main>'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/ruby_executable_hooks:15:in `eval'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/ruby_executable_hooks:15:in `<main>'

So then I was looking at https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Plugins.md
and decided that I should upgrade to the latest version of fastlane.
bash-3.2$ fastlane -v
fastlane 1.29.2

bash-3.2$ fastlane update_fastlane
Ignoring nokogiri-<IP_ADDRESS> because its extensions are not built.  Try: gem pristine nokogiri --version <IP_ADDRESS>
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/user_interaction.rb:334: warning: constant ::TimeoutError is deprecated
[13:26:06]: -------------------------------------------------
[13:26:06]: --- Step: Verifying required fastlane version ---
[13:26:06]: -------------------------------------------------
[13:26:06]: fastlane version valid
[13:26:06]: ------------------------------
[13:26:06]: --- Step: default_platform ---
[13:26:06]: ------------------------------
[13:26:06]: Variable Dump:
[13:26:06]: {:DEFAULT_PLATFORM=>:ios}
[13:26:06]: Could not find lane 'ios update_fastlane'. Available lanes: ios test, ios beta, ios deploy

+------+-------------------------------------+-------------+
|                     fastlane summary                     |
+------+-------------------------------------+-------------+
| Step | Action                              | Time (in s) |
+------+-------------------------------------+-------------+
| 1    | Verifying required fastlane version | 0           |
| 2    | default_platform                    | 0           |
+------+-------------------------------------+-------------+

[13:26:06]: fastlane finished with errors
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/runner.rb:31:in `execute': Could not find lane 'ios update_fastlane'. Available lanes: ios test, ios beta, ios deploy (RuntimeError)
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/lane_manager.rb:43:in `cruise_lane'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/command_line_handler.rb:31:in `handle'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:36:in `block (2 levels) in run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/command.rb:153:in `run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/runner.rb:428:in `run_active_command'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/runner.rb:68:in `run!'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/delegates.rb:15:in `run!'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:127:in `run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:133:in `<top (required)>'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane:23:in `load'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane:23:in `<main>'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/ruby_executable_hooks:15:in `eval'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/ruby_executable_hooks:15:in `<main>'

bash-3.2$ gem update fastlane
Updating installed gems
Nothing to update

None of these steps got me to the latest version.
Complete output when running fastlane, including the stack trace and command used

You can use: --capture_output as the last commandline argument to get that collected for you

bash-3.2$ fastlane
Ignoring nokogiri-<IP_ADDRESS> because its extensions are not built.  Try: gem pristine nokogiri --version <IP_ADDRESS>
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/user_interaction.rb:334: warning: constant ::TimeoutError is deprecated
[13:27:39]: -------------------------------------------------
[13:27:39]: --- Step: Verifying required fastlane version ---
[13:27:39]: -------------------------------------------------
[13:27:39]: fastlane version valid
[13:27:39]: ------------------------------
[13:27:39]: --- Step: default_platform ---
[13:27:39]: ------------------------------
[13:27:39]: You must provide a lane to drive. Available lanes:
1) ios test
2) ios beta
3) ios deploy

Environment
Please run fastlane env and copy the output below. This will help us help you :+1:
If you used --capture_output option please remove this block - as it is already included there.
bash-3.2$ fastlane env
Ignoring nokogiri-<IP_ADDRESS> because its extensions are not built.  Try: gem pristine nokogiri --version <IP_ADDRESS>
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/user_interaction.rb:334: warning: constant ::TimeoutError is deprecated
[13:28:08]: -------------------------------------------------
[13:28:08]: --- Step: Verifying required fastlane version ---
[13:28:08]: -------------------------------------------------
[13:28:08]: fastlane version valid
[13:28:08]: ------------------------------
[13:28:08]: --- Step: default_platform ---
[13:28:08]: ------------------------------
[13:28:08]: Variable Dump:
[13:28:08]: {:DEFAULT_PLATFORM=>:ios}
[13:28:08]: Could not find lane 'ios env'. Available lanes: ios test, ios beta, ios deploy

+------+-------------------------------------+-------------+
|                     fastlane summary                     |
+------+-------------------------------------+-------------+
| Step | Action                              | Time (in s) |
+------+-------------------------------------+-------------+
| 1    | Verifying required fastlane version | 0           |
| 2    | default_platform                    | 0           |
+------+-------------------------------------+-------------+

[13:28:08]: fastlane finished with errors
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/runner.rb:31:in `execute': Could not find lane 'ios env'. Available lanes: ios test, ios beta, ios deploy (RuntimeError)
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/lane_manager.rb:43:in `cruise_lane'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/lib/fastlane/command_line_handler.rb:31:in `handle'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:36:in `block (2 levels) in run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/command.rb:153:in `run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/runner.rb:428:in `run_active_command'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/runner.rb:68:in `run!'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/commander-4.3.5/lib/commander/delegates.rb:15:in `run!'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:127:in `run'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/gems/fastlane-1.29.2/bin/fastlane:133:in `<top (required)>'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane:23:in `load'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane:23:in `<main>'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/ruby_executable_hooks:15:in `eval'
        from /Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/ruby_executable_hooks:15:in `<main>'

Additional information to help:
bash-3.2$ which ruby
/Users/zack/.rvm/rubies/ruby-2.3.1/bin/ruby

bash-3.2$ which fastlane
/Users/zack/.rvm/gems/ruby-2.3.1@bidwrangler_phonegap/bin/fastlane

You're using a really old version of fastlane, please make sure to update using [sudo] gem update fastlane, or if you have a Gemfile, use bundle update

I think that worked! thanks a bunch


ZeRO 3 NVMe Offload?

Firstly, thank you for making the weights open to all!
Now, how to do inference/fine-tuning with ZeRO 3 NVMe/CPU Offload, since this project is based on Megatron project, that shouldn't be much difficult to implement?

You can use ZeRO Offload while finetuning your model.
You can extend our version of Megatron by adding CPU/NVME offload for inference, it is not very hard. However, GPU wil have to download entire model (200 GB) for each inferred token. It takes additional 8 sec for each inferred token in best configuration =(

@MichaelEk can't you generate multiple tokens in parallel for different string prefixes? E.g. forward with a batch of 4 symbols. That way one can remove the bottleneck of streaming weights to GPU memory.

@MichaelEk Thanks for replying.
Actually I wasn't able to implement the offload while inference, that's why I created an issue here. It would be nice if an updated script could be provided.😅

@MichaelEk can't you generate multiple tokens in parallel for different string prefixes? E.g. forward with a batch of 4 sentences. That way one can remove the bottleneck of streaming weights to GPU memory if their need permits such use.

Yes, actually you can. If you have a lot input prefixes, you can significantly reduce the the total time spent loading weights. However, the total time of one generation will be too long to use it, for example, in chatbot.

You can use ZeRO Offload while finetuning your model. You can extend our version of Megatron by adding CPU/NVME offload for inference, it is not very hard.

Thanks for the hint. Was able to run on RTX 3070 TI :smiley:

can't you generate multiple tokens in parallel for different string prefixes?
Yes, actually you can. If you have a lot input prefixes, you can significantly reduce the the total time spent loading weights.

Can you tell a little more about this? I feel that now I need to do this, but I do not know where to start.

@pskucherov do you have a fork that's ready to pick up and start from for a single GPU?

@pskucherov do you have a fork that's ready to pick up and start from for a single GPU?

No, I don't have a working code yet. But it will work soon.
Running on nvme in this config is useless. On CPU RAM it is more realistic, although it is still very long.
In any case, it's all in a private repository.

@pskucherov Being able to run on RTX 3070 Ti is really great.
If you could share some code how to "ZeRO Offload" that would be nice!

Huggingface accelerate seems to be the solution. Closing issue.


Consider decoupling dotnet publish-iis from project.json to enable cross publishing

Currently dotnet publish-iis needs dotnet publish to run first. and the default is a post publish entry in project.json.
It would be nice to decouple dotnet publish-iis  from the project.json
For IIS
dotnet publish-iis -o  \\server\approot
(implicit content entry for web.config)
This would run dotnet publish followed by the current dotnet publish-iis
I realise this can be achieved by running the tools one after the other but I cant see a scenario where I would want to run dotnet publish-iis without dotnet publish first.
For Kestrel standalone (dev Linux Mac etc)
dotnet publish -o c:\localserver\approot
(never publishes web.config regardless of content entry in project.json)
using the same project.json
UseIISIntegration is already intelligent enough to work out if it is on IIS from the envvariables.
That way I can maintain one project for IIS, OSX, Linux

never publishes web.config regardless of content entry in project.json

What's in the content section in project.json is being copied blindly. This is done by dotnet cli and I don't think there is a way to prevent this based on the name of the file.
Postpublish scripts can actually know what platform the project was published for by reading the %publish:Runtime% variable. What you could do is to write a script that checks the target platform and delete the web.config file. I don't think we will add a functionality like this to the publish-iis tool. Having web.config in the publish output is not pretty but it does no cause any harm.
One question I have though is whether it is a real scenario where you create a web app and want to target both Windows and Linux. I thought that when you start working on a web you decide what production environment you want to run your application in and you optimize for it (in this case - you either run publish-iis or not). Because if you want to build for Windows and Linux then you should most likely write web.config for windows and nginx configuration for Linux.

@moozzyk
Publishing to dev and production might be a common scenario. where you might use one project.json.
I think it would be good if dotnet publish-iis was a one step command incorporating dotnet publish.  It just seems more natural to me.  Otherwise you  have to explain that dotnet publish-iis doesn't publish the project but only the web.config.  It seems a misnomer.
Or it may even be better to just forget about the publish-iisand just have a web.config template in vstudio/vscode that creates the webconfig and just use normal publish.

What you could do is to write a script that checks the target platform and delete the web.config file.

... or do what I did: Clone the tool and hack away. It's easy to mod to one's liking. I hacked it to set the default for log output to true and name the log file prefix to the application name in 15 minutes! :smile:

It seems a misnomer.

:+1: It isn't named according to what the tool actually does, which would be something like "configure-web.config," "configure-for-iis," or something along those lines.

dotnet publish-iis -o \\server\approot  -c release -f nettcoreapp1.0
To me that makes more sense and easier to explain to users.


PoC Optional labels

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 71.317%

Totals

Change from base Build<PHONE_NUMBER>:
0.007%

Covered Lines:
1932

Relevant Lines:
2490

💛  - Coveralls


Adjusting the dimension (height) of the items

Is there a way I can adjust the items so that the thumbnails are enlarge on the height to a 320px height to 290px width?
Regards.

Did you try to max-width, max-height them with css?
Owl will apply 100% width on img items, so maxing them should solve this.

Thanks I was able to figure out that the size of the image depends on teh hight/width ratio and solve the issue I had.  I was looking for a set pixelage but is more of a percentage thing.


Finding the nearest integer to the square root of a number

Finding the nearest integer to the square root of a number

Can anyone help what went wrong here?

I have adeed one more program to compute GCD of 2 numbers also

This looks great @RishiPrakash! I'm just going to change up some formatting and merge your code. Thanks for your contributions!

Thanks again @RishiPrakash!

It's always great to see to see your pull request merged 😄 Thanks a lot buddy @Kyle-P , looking forward to add more ideas and more programs over the next few weekends


Visual indicator for comment states

Should we have some sort of visual indicator for whether a post / comment has been saved or not? It's hard to tell at a glance. It would help with feedback when the user saves or unsaves a post as well.

[ ] Saved
[ ] Collapsed

The star icon gets filled on submissions already

Ahhh right. I guess just comments then?

Merging #294

@colejd this would be pretty easy to do now that we can have image attachments, and the cell title is separate from the body

This should be relatively easy to do now that we can have text image attachments @colejd


Update definition.yml

To enable entities to be viewable in entity explorer
Relevant information
Based on advise on why the entitites are not showing up in explorer. Requesting to change alertable to true.
Checklist

[x] I've read the guidelines and understand the acceptance criteria.
[x] The value of the attribute marked as identifier will be unique and valid.
[x] I've confirmed that my entity type wasn't already defined. If it is I'm providing an
explanation above.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @gvassallo. I just signed the CLA agreement.


two-fer: drop test case with multiple inputs

Closes #160

I thought it wouldn't close the other PR, either ... but it looks like it did :D


Add the ability to create custom fields for a specific project

At the moment, custom fields are created at the organisation. It would be good to be able to create custom fields that are tied to a specific project only.

Linked to #4384


Styles not working on latest Wordpress version (beta 5)

Bug Report
I've been testing the latest version of Wordpress (beta 5) but when it comes to test the create-guten-block toolkit with this release, I notice that, in the front-end, the stylesheet isn't loaded and therefore that block styles are not rendered.
I've followed the very same instructions to create a Gutenberg block and also I've tested the example block that comes with it and this behavior persist.
I wonder if it has something to do with the enqueue function or maybe something else.
I've been poking around in the init.php file in order to get this to work with no luck.
However, the toolkit works fine with the Gutenberg plugin and it renders the styles for both, editor and front-end
Expected Behavior
It is expected that the toolkit render the styles for both editor and front-end. Just like it does when it is used with gutenberg's plugin.
Actual Behavior
The stylesheet is not loaded in the front-end. Seems like the enqueue function is not working properly.
This is how it looks in the editor:

And this is how it looks in the front-end

I've verified and double checked the class and the fact that the plugin stylesheet is not loaded in the front-end

Steps to Reproduce
You only need a fresh installation of the latest release of Wordpress 5.0 beta 5 which can be downloaded from this link: https://wordpress.org/news/2018/11/wordpress-5-0-beta-5/
and create a new block with create-guten-block toolkit.
(Write your steps here:)

Create a block with npx create-guten-block my-block

Activate the resulting plugin in the plugin's manager

Create a new post and insert the new block

Save / Publish the post and go to front-ed to see if your block is styled as expected

node -v: 8.10.0

npm -v: 6.4.1

yarn --version (if you use Yarn):

npm ls cgb-scripts (if you haven’t ejected): 1.11.1

Then, specify:

Operating system: Mac OSX High Sierra

Possible Solution

Reproducible Demo
Since I'm working on a local environment, I can't share a Reproducible Demo.

Changing the dependency from wp-blocks to wp-editor in the enqueue function should make the css load again:
wp_enqueue_style(
	'adw_blocks-cgb-style-css', // Handle.
	plugins_url( 'dist/blocks.style.build.css', dirname( __FILE__ ) ), // Block style CSS.
	array( 'wp-editor' ) // Dependency to include the CSS after it.
	// filemtime( plugin_dir_path( __DIR__ ) . 'dist/blocks.style.build.css' ) // Version: filemtime — Gets file modification time.
);

Seems like there were some changes in Gutenberg components that moved dependencies between wp.blocks and wp.editor.

@mario-neuhold that seems to be true. Can you send a PR for that?

@ahmadawais I'm trying to figure out why that changed, to see if the adaptions are neede somewhere else too. Happy to provide a PR later. Or do you prefer a quick PR just for this piece of code?

@mario-neuhold Actually, I'm away focused on launching my VSCode Power User Course — so can use all the help atm. There was also #106 see if this is related to that. I'd like for you to investigate and make a PR, the way you suggested. 👍

I have found that just removing the dependency altogether also works and the styles display as they should on the front side. I am just wondering if there is any risk with that?

Same here, All my plugins are broken.
I will make a PR, but first I need to ask to someone close to the project, like Riad Benguela to confirm me this :
For the main style :
array( 'wp-editor' )
for admin only style :
array( 'wp-blocks', 'wp-i18n', 'wp-element', 'wp-editor' )
for admin js:
array( 'wp-edit-blocks' )
I still don't know why the editor styles are loaded in front (just for logged user though) but it adds unnecessary stylesheet for now.

Hello, I just tried with this new fix... However, I am still having the same issue. This didn't seem to fix it for my site. Is anyone else having this problem still?


fix(babel): fix babelrc config for ReactNative

upgrade React Native to 0.35
fix .babelrc config for React Native
support yarn ^0.16

It's work

Sorry to be slow to respond to this, @yutin1987, and thanks a lot for the PR. There is a CI failure on the TodoMVC job under Node 4. Mind taking a look into that?

Closing due to lack of activity. Feel free to comment again if you'd like this re-opened.


consul.watch + consul.event.list seems to be broken

This does not seem to work very consistently.
While using consul watch -type event echo "event" first every time an even is received, doing the same with node-consul results in inconsistent behavior.
Sometimes the change event will fire, other times it will not, if I manually trigger 10 events, generally node-consul will fire change on the first, and then it could do it on the next or after I've fired 10 events and then change will be fired and contain all the events I triggered.

Further context, it seems like the watch routine goes into a death spin, basically under a certain set of conditions, the _run function starts getting called thousands of times and when that happens, it no longer seems to be able to grab new events.
I've seen it where it never exits the loop, and other times when it does, if it does manage to exit the loop then that is when I see it emit change with all the new events.

It appears the the x-consul-index values can be vastly different. According to my local tests, during this behavior I'm seeing the index, prevIndex lengths end up being different, and somehow the recorded prevIndex ends up being less than x-consul-index, which causes the death spin of requests, like in the 2 ms range, normally on a local consul instance when the indexes are incrementing then requests usually take like 5s.

Example, some how newIndex is getting set to<PHONE_NUMBER>91290200 while the old index is<PHONE_NUMBER>486594000, there is a zero missing on the end, not really sure how this is happening, it is very odd, then subsequent request will be something like x-consul-index<PHONE_NUMBER>91290247 which is missing a 0 on the end.

it seems that if I do a simple diff, return index !== prevIndex things seem to work better.

I can open a PR but this seems to fix my issues and with a slight alteration tests pass https://github.com/ekristen/node-consul/commit/4056a5eca4ce92e7674ef0f44ebd035867660ba1

Super weird, but it looks like that's what upstream does:
https://github.com/hashicorp/consul/blob/master/watch/plan.go#L77-L93
I wonder why the deep compare is there, don't love that at all.

Published in version 0.23.0
Thanks for the report and fix!


PHP contribution front page get started

Description 📜
I am creating issue for index of php backend development and adding folders to it for starting contribution.

Domain of Contribution 📊

[x] Backend Dev (Java, .NET, Python, PHP)

/assign


Question about "splashscreen" activity

Hello, I have one question. Im compiling against API 24 in android and my minimum API version is 16. I followed your getting started guide, but my app is still crashing. It says that default firebase app is not initialized. I'm using simple trick with other activity as Splash screen for application (and it's set as main launcher). Main activity, where xamarin PCL project is initialized is launched from there and I added FirebasePushNotificationManager.ProcessIntent(Intent); just after LoadApplication in that activity. Also I have created MainApplication class and filled it with code. Even so it still crashes. Is this caused because of my splash activity, or am I doing something wrong?
 I should read closed issues first. It's now opening properly, but says SERVICE_NOT_AVAILABLE

I leave this as "fix". In my case it was all because my google-services.json was somehow corrupted. I downloaded it again and everything works fine. Awesome plugin, does exactly what I need :D


Use RelativeVirtualFile more aggressively in tools

Many APIs still carry metadata about the relative path of files. Since the introduction of RelativeVirtualFile we can make heavier use of it to streamline API usage. Notably:

[ ] Remove relative paths from ResolutionInfo

This can only be done in 1.0.0 since making these changes in a binary compatible manner is very hard, mainly because it spans multiple classes.

/cc @ochrons

This is now only relevant to the jsDependencies plugin that will moved out of the core (#2841). I'll leave this open until it is open in the new repo.

Migrated to scala-js/jsdependencies#2


[iOS] LayoutAnimation does not animate Text

Hello,
I'm creating a component that should animate the position of a Text element.
It seems that LayoutAnimation does not animate the Text frame unless it is wrapped in a View element.
I'm using absolute positioning together with top and left attributes in the CSS

@nicklockwood - known issue? If so, I can document this limitation.

@brentvatne I believe it's deliberate. Animating text looks pretty weird, so we disabled it.
(The technical reason is that text doesn't animate like you'd expect, with the text re-wrapping each frame - instead, because text is rendered as an image internally, what happens is that the rendered text image is stretched to fit the new frame, and then immediately wraps to the new layout after the animation completes.)

I guess we might be able to animate the position of the frame, but not its size. I don't know if that would work any better in practice though.

because text is rendered as an image internally
Is this something RN or iOS side? (I know that UILabels are backed by textures)

Just to give some context, I'm running some experiments to understand how RN fits some pieces of UI we have and I also tried to animate a Text component. More specifically I tried to animate the position, not the size, which may make sense sometimes.
As far as I can see there are three ways to handle this:

document the behaviour
RN triggers a warning in dev mode, so the Text doesn't animate but developers know the reason
RN animates the position but not the size (probably either point 1 or 2 is still needed in this case though)

Is this something RN or iOS side? (I know that UILabels are backed by textures)

It's just the way iOS does it. Drawing text is expensive, and is done mostly on the CPU not the GPU, so it's not something that can easily be done in real time, which is why iOS does it by pre-drawing the text into a texture. We could try to force it to redraw the text every frame when animating, but the performance would probably be horrible.

Most likely yes.
Moreover I think that no one will ever animate the size of a label. Changing the position, instead, can be useful (even if not really common).
The only thing is that it may be weird that LayoutAnimation behaves differently for different components. If not specified in the documentation (or maybe using a warning), someone may think that there is something broken in his code (as I did). I only discovered that by looking at an example I found online and by checking and testing differences between this example and my code

I agree with the above. I just ran into the same issue trying to animate Text. layout animation could use some more documentation

We updated a bunch of docs including Animated. Since this is "working as intended" and just a docs thing I'm going to close this issue, feel free to reopen if you have a specific complaint about the new animation docs though.

I would like to have LayoutAnimation work with iOS text. If the performance is a problem maybe enable it with a flag.

"Moreover I think that no one will ever animate the size of a label"
Hm, I think that's actually what I was trying to do yesterday! :-) Worked OK in iOS but the text was clipped on Android going from a smaller text word to a longer one.


Fix #75

Fixed issue #75  - ability to use DataView for edges.
Added DataView example

Thanks @bel0v!


x11rdp, how to always launch a new session

Hey guys, I have just installed x11rdp on ubuntu 14.04
What happens is every time I remote in and login I get connected to the same existing session.  I want to make a kiosk user with a shared username and password that will always launch a new session and never connect to an existing one.  I looked through the .ini files and it seems it will always resume the session if the geometry and username are the same.  Any way around this?  Thanks!

@jsorg71 can you make this behavior configurable ?

I think it is.
see
Policy - session allocation policy
in sesman.ini

Policy=UBDC in sesman.ini can do that.

Probably you don't want to preserve sessions which never be reconnected.
Set KillDisconnected to yes, DisconnectedTimeLimit to an integer value what you want.

that did it metalefty, thanks!
The UBDC is confusing as I thought it meant to maintain session if those items were the same.

@ScottCar close?

Yes, you can close.  This is working great.


Overflow issues when opening the Upload widget.

Hi guys, I have been working on upload widget on my admin panel, so my panel has his viewport height fixed as 100vh and width as 100vw and  I have set the overflow to hidden at the body. Problem is when I open the widget the screen automatically overflows and my layout is broken. Refreshing the browser doesn't change anything and I have to navigate to different to page to fix it.
Issue Type (Can be multiple)
UI Issue
Steps to reproduce
Can be reproduce my setting Viewport height and width to 100vh and 100vw respectively and setting overflow to hidden in the body and having a div inside of the body with overflow-y as auto and height: 100%. This Div should contain Upload widget.
Error screenshots
Before opening the widget

After opening the widget

As you can see in the background, the layout disappears and I cant figure out why its happening
Browsers (if issue relates to UI, else ignore)
Can be reproduced in all the browsers
Versions and Libraries (fill in the version numbers)
Latest version of all.js which is included in the script.

Hi @nerdophile, Can you please open a support ticket at: https://support.cloudinary.com/hc/en-us/requests/new, and include your code and a link to your webpage so we can further assist? As this issue is not related to the js SDK I will go ahead and close this thread. Thanks.


OnlineManager subscription doesn’t execute when going offline

Describe the bug
I just noticed that the built-in onlineManager instance emits a notification only when going online, but not when going offline.
To Reproduce
React.useEffect(
  () =>
    onlineManager.subscribe(() => {
      // Logs only `{ online: true }`, no matter what happens
      // Cannot detect going offline
      console.log({ online: onlineManager.isOnline() });
    }),
  [],
);

Expected behavior
OnlineManager should listen to the "offline" event of browsers, besides just "online".
Desktop (please complete the following information):

OS: macOS 11.1
Browser: Firefox
Version: 84.0.2 (64-bit)

Additional context
It seems like the default listener in OnlineManager doesn’t listen for "offline" events: https://github.com/tannerlinsley/react-query/blob/ac342e237ae9fab6759f2d7be616662da6c16225/src/core/onlineManager.ts#L51-L63

:tada: This issue has been resolved in version 3.5.13 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 3.5.13 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


How to keep the background image size?

The generated image size automatically adapts to characters, but I want to keep the original size of the background. How to keep the background size?  How to change the generated image background size?

Background of generated image is randomly cropped from bg image. You can change image height   https://github.com/oh-my-ocr/text_renderer/blob/01375e44d9417ac163d424fbae775c62f57bafd7/text_renderer/config/init.py#L146

Thank you for the answer. But I want to keep the original bg image like this picture. Can I change the size of cropped area?

https://github.com/oh-my-ocr/text_renderer/blob/01375e44d9417ac163d424fbae775c62f57bafd7/text_renderer/render.py#L205
Now I can change the cropped area, but the generated image will be fuzzy if I enlarge the cropped area. How can I keep the image clear while the cropped area is big? I want to make the effect like the pictures.


Fixed grid view height for tall icons

Fixed a bug that caused tall icons to go over the borders and in turn hide the file name.
Thanks @xPoppyx for noticing this👍
Before:

After:

:shipit:


Strategy for avoding plain text passwords in environment?

Is it possible to avoid plain-text passwords in the "Manage Environments" section of Insomnia?

Make the tokens non-readable?
Would it be possible to maybe reference the environment on the machine?
Maybe using a .env file?
Integrate with 1Pasword manager - or a different password manager?

Great question!

Would it be possible to maybe reference the environment on the machine?

You can install insomnia-plugin-system-env to achieve this. Plugins to the rescue! 🎉
In addition, there are a couple more options that you have, ordered in increasing effort:

Private environments, while these are plain-text, a private environment will never be exported or synced
Use a prompt template tag
Custom plugin to integrate with env file or a password manager

Prompt
Use a prompt template tag with the "mask text" and "default to last value" options selected

There are several options in the template tag modal to control persistence of the value. You may choose to set a storage key, which will clear the value from storage once the app is closed. Direct downside with the prompt: it will prompt each time the value is needed.
Custom plugin
Additionally, while there are currently no plugin APIs for manipulating an Insomnia environment itself, it is possible to create a plugin that will fetch a variable from a specified file on the filesystem or from a password manager. This plugin can be exposed via custom template tags. It should be fairly straightforward to make one, similar in principle to the system-env plugin. 👀
PS: we have discussed directly supporting system environment variables within the last week. I like this option in particular because it would allow for CI workflows with the CLI (inso) in a seamless and secure manner. The CLI currently doesn't cater for plugins so it can't use the one I just linked, yet. 😢)

Thanks for the answer. A custom plugin that would get the secrets from a .env file or a password manager would be epic. What needs to be done for the exposure of a plugin API to manipulate the environment?

Just to clarify, a plugin that reads from an .env file or a password manager does not need the exposure of a plugin API to manipulate the environment. It would just operate as a standalone template tag, to fetch a variable using x plugin.
However, off the top of my head, exposing an API to actually manipulate an Insomnia environment should also be trivial. If you'd like to use this API feel free to propose a design of how you might use it and we can work towards adding it. But for the above plugins^ you should be able to make do without it.
For instance, the .env file path can be an Insomnia environment variable, if you have different env files for local vs staging vs production.

For instance, the .env file path can be an Insomnia environment variable, if you have different env files for local vs staging vs production.

My biggest dream  😂  - because in our team we all share an .env file, needed for our code project to connect to services. Would be  🔥  if Insomnia hooks into this. When I find the time, I will give it a try 😅

You've given me inspiration to build something as well as I can absolutely see a use case for it, having used .env.local, .env.staging files in the past.
I do think this belongs in a plugin rather than the application itself, as there is a use case for standard Insomnia environments as well. Will ping here if I end up patching something together. 😄

@alexbjorlig there is a comment in the linked issue above, referring to a plugin that can consume env files! 🎉 https://github.com/Kong/insomnia/issues/1883#issuecomment-596269972

For instance, the .env file path can be an Insomnia environment variable, if you have different env files for local vs staging vs production.

My biggest dream 😂 - because in our team we all share an .env file, needed for our code project to connect to services. Would be 🔥 if Insomnia hooks into this. When I find the time, I will give it a try 😅

@alexbjorlig https://www.npmjs.com/package/insomnia-plugin-dotenv is now publish. Please give it a test run and let me know of any  bugs or enhancements.
Kind Regards

Thanks for putting together a plugin - however it's the first Insomnia plugin I tried. Where do I provide the path for the env file?

Thanks for putting together a plugin - however it's the first Insomnia plugin I tried. Where do I provide the path for the env file?

In the template screen:

Select dotenv from the dropdown.
Click to choose .env, your system dialog will open up to enable you to select a file
Enter the variable name you need to retrieve.
Live Preview should now show you the value assigned to that variable
Click done.
Your screen should like the below.

That works - and pretty impressive this is possible with just 35 lines of code - good job 🥳
However, if I understand this correctly the "limitations" of the environment API in insomnia means that if I have 31 secrets I have to define 31*3=93 environment variables - because a plugin can't create an environment on my behalf?
(I'm not saying it's a showstopper, it's just error-prone since my teammates will have to the same?)

@bongosway amazing! I'm excited to try this out. 💯
@alexbjorlig technically yes. You'll notice that in the template tag modal, there is a cog on the right hand side, meaning the entry can itself come from an Insomnia environment variable.
What you can do, is map each variable one to one against your env file in the base insomnia environment. Then specify an envFilePath in the three sub environments, eg env.local, env.staging, env.production. Lastly, when consuming the dotenv plugin, reference the envFilePath variable instead of hard-coding it. There might be a gotcha with how base and sub environments work, though.
Give it a go, I'll also put an example workspace together and share it with you tomorrow!

@bongosway amazing! I'm excited to try this out. 💯
@alexbjorlig technically yes. You'll notice that in the template tag modal, there is a cog on the right hand side, meaning the entry can itself come from an Insomnia environment variable.
What you can do, is map each variable one to one against your env file in the base insomnia environment. Then specify an envFilePath in the three sub environments, eg env.local, env.staging, env.production. Lastly, when consuming the dotenv plugin, reference the envFilePath variable instead of hard-coding it. There might be a gotcha with how base and sub environments work, though.
Give it a go, I'll also put an example workspace together and share it with you tomorrow!

Thanks! It's already much better than hard-coding the secrets multiple places

Here is an example workspace, but note, need @bongosway to review this PR to support it. I found a case that didn't work during testing, and created a PR against the plugin.

Note, you can create the dev/stage/prod sub environments as private environments, so that the machine-specific file-path will not sync for each user. 😄


[FEATURE] Make tournaments be stopped unconditionally

Hello this is a feature request! It would be nice if somehow, tournament creators can stop a tournament unconditionally. Or maybe just the programmers because it's really annoying to wait for the tournament to stop to see if the notifications work. Or maybe this feature will be useful in some other ways.

@JoshMatthew Thanks for the feature suggestion. I'm curious about the specifics of the feature request. Do you need to be able to programmatically complete a tournament early or do you just want to be able to just trigger the tournament end callback when you test the game?

@JoshMatthew Thanks for the feature suggestion. I'm curious about the specifics of the feature request. Do you need to be able to programmatically complete a tournament early or do you just want to be able to just trigger the tournament end callback when you test the game?

Yeah maybe an api? That when called will trigger the end tournament callback? So just also in development, it's gonna be easier to develop.

Maybe a button in the Nakama Console on the leaderboard/tournament that emulates the behaviour?

Maybe a button in the Nakama Console on the leaderboard/tournament that emulates the behaviour?

Yeah that would be convenient

@mofirouz I think it would also be great (though is off topic to this request) to have an entry in the configuration view that shows the current coordinator node for leaderboard/tournament reset hooks.


A11y: View & Download buttons are missing aria-label

Describe the task
When we removed the tooltips from the View & Download buttons on the Find Properties page, we removed the content inside of the buttons so now when screenreaders land on the buttons all it says is "button."
Add aria-label="View" and aria-label="Download" to these buttons.
Acceptance Criteria

[ ] Turn on Voiceover or other screenreader of your choice
[ ] Navigate to the Find Properties page & the view/download buttons.
[ ] When you navigate to the buttons, you will hear "View button" / "Download button"

I'd like to be assigned this please.

Hi @bacitracin just wanted to make sure you saw this request. Thanks!

@AZBL Whoops sorry missed this. Let me know if you have any questions!

Hi @bacitracin, a few questions in regards to this. Mostly I just want to make sure I'm targeting the right things because I'm not sure I am.
-The View Button isn't present on the staging site.
-In SidePanelControlBar.tsx the Download ThemeButton does have aria-label=Download and the screen reader voiceover does say "Download Button". On the main site my screen reader said "View Button" for that as well.
-I did notice the filter button doesn't have an aria-label and it just says "Filter".
Thanks - Alex

@AZBL Hey there! I'm checking the staging environment, and it looks like this ticket isn't relevant anymore. The Filter button doesn't need an aria-label because the visible text is available and read out by the screenreader.
Sorry for the confusion, the code is moving fast these days


Don't crawl external URLS

Hi Pascal
I'm sure this topic has been already solved but after searching tips on google I couldn't find the solution so I have to ask your support :)
My need is simple: I want Norconex to discard ALL external links.
so, if I'm crawling the website www.mydomain.com, who has links pointing to www.anotherdomain.com or anotherdomain2.com, I want norconex to stay and crawl only www.mydomain.com.
doing some searches I saw that the "stayOnDomain" doesn't help, and I  found tips using the tag
        <linkExtractors>             <extractor ignoreExternalLinks="true" />         </linkExtractors>
but seems to not work as expected.
can you help me on this?
the version used is : NORCONEX 2.8.1
thanks
Mauro

stayOnDomain should definitely do it.  What suggests it does not?  Can you provide the relevant portion of your logs, as well are your config?

Hi Pascal
thanks for the quick feedback.
below the relevant portion. let me know if it's enough or you need the full config:
..... <crawlers>     <crawler id="mywebsiteid">         <startURLs stayOnDomain="true" stayOnPort="false" stayOnProtocol="false">             <url>http://mydomain.com</url>         </startURLs>         ......     </crawler> </crawlers> .....
"mydomain.com" has external links that are indexed into Elastic Search because commited by Norconex but I don't understand why.
if it may be useful, what I noticed is that "mydomain.com" is on HTTPS while the external links are on HTTP. maybe is the stayOnProtocol parameter that creates the conflict for this specific case?

Hi Pascal
I think I found where "the bug" is nested.
after lot of tests, I found out that in the sitemap.xml (which is generated by the Framework we are using) there was ONE external link to another domain and I assume this was the "entry point" for the crawler to start crawling the ENTIRE external website.
I quickly applied a patch to remove this entry and crawled again "mydomain.com" and this time it worked fine (only mydomain,com was indexed in Elastic Search)
So it appears that the paramenter "stayOnDomain" is not considered when the crawler reads the URLs from the Sitemap.xml.
can this be possible?
I let this open waiting for your comment but feel free to close it after you reply.
Mauro

All start URLs are considered "onDomain". The sitemap entries are equivalent to start URLs in this case.
If you want the crawler to only index what is in the sitemap, you can set "maxDepth" to zero.   Alternatively, you can combine the stayOnDomain with a regex filter to just eliminate that extra one.


Delete selected Yale resources from Production

The following five resources were created in error, and we would like to requested that they be deleted:
https://trellis.sinopia.io/repository/yale/4fe4c107-ff30-453f-8f3d-5c33a03fccd1
https://trellis.sinopia.io/repository/yale/cd81f169-0459-4153-83a1-71af87afb215
https://trellis.sinopia.io/repository/yale/18f878b4-cb42-4b21-89fe-23fcfa8a4ee7
https://trellis.sinopia.io/repository/yale/c4512503-72da-4acb-9a0a-1a74eec52ad3
https://trellis.sinopia.io/repository/yale/75d2c86b-e94f-436d-807c-29ee4dcd9847

Done, deleted from production.


chore: remove inactive browserstack wrapper package

No longer uses the "inactive" browserstack wrapper package from npm, which was just responsible for creating the readyfile.

We can create the readyfile ourself and get rid of the npm package.

@hansl can you take a look at this?

Just fix the timeout issue and you'll get my LGTM. Good work!

@hansl Done :)

1 small nit and LGTM! Thanks

@hansl Good point -Fixed. Thanks again for your great comments!


Nightly Build Failure 2024-04-11

Overview
Everything except the creation of the Zenodo sandbox data release archive succeeded. That step fails with a Pydanic validation error when attempting to create a _NewRecord from the JSON that comes from a Zenodo response.  It looks like it's getting a server error instead of the expected response, so my guess is this is a problem that originates with Zenodo.
This error first appeared on 2024-04-11, but recurred the next time the nightly builds ran, on 2024-04-14.
Next steps

Test accessing the Zenodo Sandbox with ZENODO_SANDBOX_TOKEN_PUBLISH for some other purpose. Does that work? If we're able to make new raw data archives then presumably yes, and the most recent test archives are from 2024-04-13.
Try making the same request to the Zenodo sandbox in a notebook or with curl and see what the full response is that's getting fed into the _NewRecord class and failing. Reproducible, but not terribly helpful.

import requests
import os
import json

token = os.environ["ZENODO_SANDBOX_TOKEN_PUBLISH"]
dude = requests.request(
    method="GET",
    url="https://sandbox.zenodo.org/api/records/5563",
    headers={"Authorization": f"Bearer {token}"},
    timeout=5,
)
print(json.dumps(dude.json(), indent=4))

{
    "error_id": "35b0d52720334872a037327c26439814",
    "message": "The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.",
    "status": 500
}

However, trying the same thing on the production server seems to work fine, so this seems to be an issue with the Zenodo sandbox server.
import requests
import os
import json

#token = os.environ["ZENODO_SANDBOX_TOKEN_PUBLISH"]
#record_id = 5563
#base_url = "https://sandbox.zenodo.org/api"

token = os.environ["ZENODO_TOKEN_PUBLISH"]
record_id = 3653158
base_url = "https://zenodo.org/api"

dude = requests.request(
    method="GET",
    url=f"{base_url}/records/{record_id}",
    headers={"Authorization": f"Bearer {token}"},
    timeout=5,
)
print(json.dumps(dude.json(), indent=4))

{
    "created": "2024-02-26T17:59:12.569661+00:00",
    "modified": "2024-02-26T17:59:13.853168+00:00",
    "id": 10708669,
    "conceptrecid": "3653158",
    "doi": "10.5281/zenodo.10708669",
    "conceptdoi": "10.5281/zenodo.3653158",
    "doi_url": "https://doi.org/10.5281/zenodo.10708669",
    "metadata": {
        "title": "Public Utility Data Liberation Project (PUDL) Data Release",
        "doi": "10.5281/zenodo.10708669",
        "publication_date": "2024-02-26",
        "description": "
PUDL v2024.2.6 Data Release
\n
The main impetus behind this release is the quarterly update of some of ourcore datasets with preliminary data for 2023Q4. The EIA Form 860 – Annual Electric Generator Report, EPA Hourly Continuous Emission Monitoring System (CEMS), and bulk EIA API data are all up to date through the end of 2023, while the EIA Form 923 – Power Plant Operations Report lags a month behind and is currently only available through November, 2023. We also addressed several issues we found in our initial release automation process that will make it easier for us to do more frequent releases, like this one!
\n
We’re also for the first time publishing the full historical time series of of generator data available in the EIA860M, rather than just using the most recent release to update the EIA860 outputs. This enables tracking of how planned fossil plant retirement dates have evolved over time.
\n
There are also updates to our data validation system, a new version of Pandas, and experimental Parquet outputs. See below for the details.
\n\n\n
New Data Coverage
\n\n\n\n
\n
Add EIA860M data through December 2023 #3313, #3367.
\n
Add 2023 Q4 of CEMS data. See #3315, #3379.
\n
Add EIA923 monthly data through November 2023 #3314, #3398, #3422.
\n
Create a new table core_eia860m__changelog_generators which tracks the evolution of all generator data reported in the EIA860M, in particular the stated retirement dates. see issue #3330 and PR #3331. Previously only the most recent month of reported EIA860M data was available within the PUDL DB.
\n
\n\n\n\n
Release Infrastructure
\n\n\n\n
\n
Use the same logic to merge version tags into the stable branch as we are using to merge the nightly build tags into the nightly branch. See PR #3347
\n
Automatically place a temporary object hold on all versioned data releases that we publish to GCS, to ensure that they can’t be accidentally deleted. See issue #3400 and PR #3421.
\n
\n\n\n\n
Schema Changes
\n\n\n\n
Restored the individual FERC Form 1 plant output tables, providing direct access to denormalized versions of the specific plant types via:
\n
\n
out_ferc1__yearly_steam_plants_sched402
\n
out_ferc1__yearly_small_plants_sched410
\n
out_ferc1__yearly_hydroelectric_plants_sched406
\n
out_ferc1__yearly_pumped_storage_plants_sched408
\n
\n
See issue #3416 & PR #3417
\n\n\n\n
Data Validation with Pandera
\n\n\n\n
We’ve started integrating pandera dataframe schemas and checks with dagster asset checks to validate data while our ETL pipeline is running instead of only after all the data has been produced. Initially we are using the various database schema checks that are generated by our metadata, but the goal is to migrate all of our data validation tests into this framework over time, and to start using it to encode any new data validations immediately. See issues #941, #1572, #3318, #3412 and PR #3282.
\n\n\n\n
Pandas 2.2
\n\n\n\n
We’ve updated to Pandas 2.2, which has a number of changes and deprecations. See PRs #3272, #3410.
\n
\n
Changes in how merge results are sorted impacted the assignment of unit_id_pudl values, so any hard-coded values that dependent on the previous assignments will likely be incorrect now. We had to update a number of tests and FERC1-EIA record linkage training data to account for this change.
\n
Pandas is also deprecating the use of the AS frequency alias, in favor of YS, so many references to the old alias have been updated.
\n
We’ve switched to using the calamine engine for reading Excel files, which is much faster than the old openpyxl library.
\n
\n\n\n\n
Parquet Outputs
\n\n
The ETL now outputs PyArrow Parquet files for all tables that are written to the PUDL DB. The Parquet outputs are used as the interim storage for the ETL, rather than reading all tables out of the SQLite DB. We aren’t publicly distributing the Parquet outputs yet, but are giving them a test run with some existing users. See #3102 #3296, #3399.
\n
Other PUDL v2024.2.6 Resources
\n
\n
PUDL v2024.2.6 Data Dictionary
\n
PUDL v2024.2.6 Documentation
\n
PUDL in the AWS Open Data Registry
\n
PUDL v2024.2.6 in a free, public AWS S3 bucket: s3://pudl.catalyst.coop/v2024.2.6/
\n
PUDL v2024.2.6 in a requester-pays GCS bucket: gs://pudl.catalyst.coop/v2024.2.6/
\n
Zenodo archive of the PUDL GitHub repo for this release
\n
PUDL v2024.2.6 release on GitHub
\n
PUDL v2024.2.6 package in the Python Package Index (PyPI)
\n
\n
Contact Us
\n
If you're using PUDL, we would love to hear from you! Even if it's just a note to let us know that you exist, and how you're using the software or data. Here's a bunch of different ways to get in touch:
\n
\n
Follow us on GitHub
\n
Use the PUDL Github issue tracker to let us know about any bugs or data issues you encounter
\n
GitHub Discussions is where we provide user support.
\n
Watch our GitHub Project to see what we're working on.
\n
Email us at <a href=\"mailto:hello<EMAIL_ADDRESS>for private communications.
\n
On Mastodon: <EMAIL_ADDRESS>BlueSky: <a<EMAIL_ADDRESS>Twitter: @CatalystCoop
\n
Play with our data and notebooks on Kaggle
\n
Combine our data with ML models on HuggingFace
\n
Learn more about us on our website: https://catalyst.coop
\n
Subscribe to our announcements list for email updates.
\n
",
        "access_right": "open",
        "creators": [
            {
                "name": "Selvans, Zane A.",
                "affiliation": "Catalyst Cooperative",
                "orcid": "0000-0002-9961-7208"
            },
            {
                "name": "Gosnell, Christina M.",
                "affiliation": "Catalyst Cooperative",
                "orcid": "0009-0004-2979-6142"
            },
            {
                "name": "Sharpe, Austen",
                "affiliation": "Catalyst Cooperative"
            },
            {
                "name": "Norman, Bennett",
                "affiliation": "Catalyst Cooperative"
            },
            {
                "name": "Bush, Trenton",
                "affiliation": "Catalyst Cooperative"
            },
            {
                "name": "Schira, Zach",
                "affiliation": "Catalyst Cooperative"
            },
            {
                "name": "Lamb, Katherine",
                "affiliation": "Catalyst Cooperative"
            },
            {
                "name": "Xia, Dazhong",
                "affiliation": "Catalyst Cooperative"
            },
            {
                "name": "Belfer, Ella",
                "affiliation": "Catalyst Cooperative"
            }
        ],
        "keywords": [
            "utility",
            "energy",
            "climate",
            "sqlite",
            "parquet",
            "electricity",
            "emissions",
            "epa",
            "eia",
            "ferc",
            "coal",
            "natural gas",
            "regulation",
            "policy",
            "data",
            "eia923",
            "eia860",
            "eia861",
            "ferc form 1"
        ],
        "version": "v2024.2.6",
        "language": "eng",
        "resource_type": {
            "title": "Dataset",
            "type": "dataset"
        },
        "license": {
            "id": "cc-by-4.0"
        },
        "communities": [
            {
                "id": "catalyst-cooperative"
            }
        ],
        "relations": {
            "version": [
                {
                    "index": 8,
                    "is_last": true,
                    "parent": {
                        "pid_type": "recid",
                        "pid_value": "3653158"
                    }
                }
            ]
        }
    },
    "title": "Public Utility Data Liberation Project (PUDL) Data Release",
    "links": {
        "self": "https://zenodo.org/api/records/10708669",
        "self_html": "https://zenodo.org/records/10708669",
        "self_doi": "https://zenodo.org/doi/10.5281/zenodo.10708669",
        "doi": "https://doi.org/10.5281/zenodo.10708669",
        "parent": "https://zenodo.org/api/records/3653158",
        "parent_html": "https://zenodo.org/records/3653158",
        "parent_doi": "https://zenodo.org/doi/10.5281/zenodo.3653158",
        "self_iiif_manifest": "https://zenodo.org/api/iiif/record:10708669/manifest",
        "self_iiif_sequence": "https://zenodo.org/api/iiif/record:10708669/sequence/default",
        "files": "https://zenodo.org/api/records/10708669/files",
        "media_files": "https://zenodo.org/api/records/10708669/media-files",
        "archive": "https://zenodo.org/api/records/10708669/files-archive",
        "archive_media": "https://zenodo.org/api/records/10708669/media-files-archive",
        "latest": "https://zenodo.org/api/records/10708669/versions/latest",
        "latest_html": "https://zenodo.org/records/10708669/latest",
        "draft": "https://zenodo.org/api/records/10708669/draft",
        "versions": "https://zenodo.org/api/records/10708669/versions",
        "access_links": "https://zenodo.org/api/records/10708669/access/links",
        "access_users": "https://zenodo.org/api/records/10708669/access/users",
        "access_request": "https://zenodo.org/api/records/10708669/access/request",
        "access": "https://zenodo.org/api/records/10708669/access",
        "reserve_doi": "https://zenodo.org/api/records/10708669/draft/pids/doi",
        "communities": "https://zenodo.org/api/records/10708669/communities",
        "communities-suggestions": "https://zenodo.org/api/records/10708669/communities-suggestions",
        "requests": "https://zenodo.org/api/records/10708669/requests"
    },
    "updated": "2024-02-26T17:59:13.853168+00:00",
    "recid": "10708669",
    "revision": 4,
    "files": [
        {
            "id": "66aad64a-6960-44bd-9813-c7fa642665f0",
            "key": "ferc714_xbrl_taxonomy_metadata.json",
            "size": 192370,
            "checksum": "md5:9fef935f9a970839319ada082d6c9672",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc714_xbrl_taxonomy_metadata.json/content"
            }
        },
        {
            "id": "dc7dc9b7-610f-4c17-914d-8595ea676e97",
            "key": "ferc714_xbrl_datapackage.json",
            "size": 59809,
            "checksum": "md5:698167bc11fb964f472be9759ab4b7ea",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc714_xbrl_datapackage.json/content"
            }
        },
        {
            "id": "fae3beb1-809d-4ec7-b1fc-b94c77ffa9ad",
            "key": "ferc2_xbrl.sqlite.gz",
            "size": 13769037,
            "checksum": "md5:b28739737424b6beb040daffd1636f26",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc2_xbrl.sqlite.gz/content"
            }
        },
        {
            "id": "d940d97b-02da-4f4b-bcb2-5b59a1fff9e5",
            "key": "ferc714_xbrl.sqlite.gz",
            "size": 102148923,
            "checksum": "md5:6d9a7e9f62a4c931db99c48b3a59990c",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc714_xbrl.sqlite.gz/content"
            }
        },
        {
            "id": "464c4e51-8816-46b4-87d5-6f483f0e3dcf",
            "key": "ferc6_dbf.sqlite.gz",
            "size": 43933844,
            "checksum": "md5:f38244976b151849f5cc124d70ed303f",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc6_dbf.sqlite.gz/content"
            }
        },
        {
            "id": "e50f96f8-f8a4-41d8-9944-bcc0e10082b7",
            "key": "ferc2_xbrl_datapackage.json",
            "size": 1976390,
            "checksum": "md5:7b98f5e4773179265771c0149fff5186",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc2_xbrl_datapackage.json/content"
            }
        },
        {
            "id": "432ebe57-4efb-4c36-95d1-8f76d929ecb3",
            "key": "ferc6_xbrl_taxonomy_metadata.json",
            "size": 2874928,
            "checksum": "md5:2c85dae41667448ea32066726f76ae5e",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc6_xbrl_taxonomy_metadata.json/content"
            }
        },
        {
            "id": "98eb5984-e9ce-48ba-a3f2-f1d1596ccd3e",
            "key": "2024-02-26-0437-a9b4e3659-v2024.2.6-pudl-etl.log",
            "size": 5966490,
            "checksum": "md5:5aa7d02b91599dc71c3bac68ebaab961",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/2024-02-26-0437-a9b4e3659-v2024.2.6-pudl-etl.log/content"
            }
        },
        {
            "id": "ea5ea289-75e2-4b22-b647-343162adb821",
            "key": "ferc1_xbrl_taxonomy_metadata.json",
            "size": 7277778,
            "checksum": "md5:026ad62c418e5e8aab4a85e6a68d628a",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc1_xbrl_taxonomy_metadata.json/content"
            }
        },
        {
            "id": "32e9b53f-bc95-494b-8ccc-5d7133ddeea7",
            "key": "ferc2_dbf.sqlite.gz",
            "size": 74501858,
            "checksum": "md5:b2d6613b96d7491ac5a26c8368256f4b",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc2_dbf.sqlite.gz/content"
            }
        },
        {
            "id": "09f22c0c-518b-4d70-8749-e7946d9cd6ad",
            "key": "core_epacems__hourly_emissions.parquet",
            "size":<PHONE_NUMBER>,
            "checksum": "md5:04044a97321ca020a0859a5973ff4827",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/core_epacems__hourly_emissions.parquet/content"
            }
        },
        {
            "id": "1bb2a3db-7c6e-4cd7-86b8-3afcbdc690c2",
            "key": "pudl.sqlite.gz",
            "size":<PHONE_NUMBER>,
            "checksum": "md5:e4f52aa31f5296d24e05f4aaf5f47246",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/pudl.sqlite.gz/content"
            }
        },
        {
            "id": "0528c1d2-43fb-4a51-817f-e2ec0b6bea33",
            "key": "ferc6_xbrl_datapackage.json",
            "size": 1068696,
            "checksum": "md5:f2a77356d745ab2213c721496c82f9e3",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc6_xbrl_datapackage.json/content"
            }
        },
        {
            "id": "cc059e03-26dd-4c0e-b5ca-361c30e505f0",
            "key": "ferc60_xbrl.sqlite.gz",
            "size": 2301917,
            "checksum": "md5:412a72389b3ab2861b2e055a38aab12c",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc60_xbrl.sqlite.gz/content"
            }
        },
        {
            "id": "ab63b205-8150-4598-86f9-01c3f1485122",
            "key": "ferc1_dbf.sqlite.gz",
            "size": 275514824,
            "checksum": "md5:191824673c778ce8a5c4e051577ad556",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc1_dbf.sqlite.gz/content"
            }
        },
        {
            "id": "5596d31c-b7ab-438c-bbfe-e28054279675",
            "key": "ferc1_xbrl.sqlite.gz",
            "size": 97227925,
            "checksum": "md5:d4642e586d0a2ed855ee9481e2607523",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc1_xbrl.sqlite.gz/content"
            }
        },
        {
            "id": "02415c77-1ce4-4337-852f-3af1eca9a778",
            "key": "ferc60_dbf.sqlite.gz",
            "size": 2882516,
            "checksum": "md5:360fafcc84d8e19acf188f7baba7b5aa",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc60_dbf.sqlite.gz/content"
            }
        },
        {
            "id": "51cadedf-9c0f-4d51-822c-68211c0d8bd0",
            "key": "ferc1_xbrl_datapackage.json",
            "size": 1727987,
            "checksum": "md5:82064ef14c0d6da8b87e0360493bec94",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc1_xbrl_datapackage.json/content"
            }
        },
        {
            "id": "a96f1be1-d5d3-4321-9cea-9520baa9b7f6",
            "key": "ferc60_xbrl_taxonomy_metadata.json",
            "size": 1856822,
            "checksum": "md5:0cc4fca785314082b8f514bae888600e",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc60_xbrl_taxonomy_metadata.json/content"
            }
        },
        {
            "id": "03c86613-a4dd-4b1d-8061-ad4f44a8fc58",
            "key": "ferc6_xbrl.sqlite.gz",
            "size": 10588865,
            "checksum": "md5:5643d56b2fa8cff5e8e888fc4a483da9",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc6_xbrl.sqlite.gz/content"
            }
        },
        {
            "id": "f33f680f-4f71-4dfb-bbe3-567d64c9c3da",
            "key": "censusdp1tract.sqlite.gz",
            "size": 506670400,
            "checksum": "md5:12d5709d09e9020ad59c88d94709fb29",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/censusdp1tract.sqlite.gz/content"
            }
        },
        {
            "id": "ffc35da9-e182-44e3-9c1c-dd44ecc427d1",
            "key": "ferc2_xbrl_taxonomy_metadata.json",
            "size": 7105520,
            "checksum": "md5:fbbd750509118029d7d675e573f6ad5a",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc2_xbrl_taxonomy_metadata.json/content"
            }
        },
        {
            "id": "0691988c-0dd5-4174-8dc9-c10f8b73f886",
            "key": "ferc60_xbrl_datapackage.json",
            "size": 748860,
            "checksum": "md5:d2a6dec21e482fa4bde44b930e774975",
            "links": {
                "self": "https://zenodo.org/api/records/10708669/files/ferc60_xbrl_datapackage.json/content"
            }
        }
    ],
    "owners": [
        {
            "id": 90379
        }
    ],
    "status": "published",
    "stats": {
        "downloads": 2515,
        "unique_downloads": 1904,
        "views": 7966,
        "unique_views": 6996,
        "version_downloads": 90,
        "version_unique_downloads": 89,
        "version_unique_views": 102,
        "version_views": 112
    },
    "state": "done",
    "submitted": true
}

Verify that everything is fixed!
Once you've applied any necessary fixes, make sure that the nightly build outputs are all in their right places.
- [x] [S3 distribution bucket](https://s3.console.aws.amazon.com/s3/buckets/pudl.catalyst.coop?region=us-west-2&bucketType=general&prefix=nightly/&showversions=false) was updated at the expected time
- [x] [GCP distribution bucket](https://console.cloud.google.com/storage/browser/pudl.catalyst.coop/nightly;tab=objects?project=catalyst-cooperative-pudl) was updated at the expected time
- [x] [GCP internal bucket](https://console.cloud.google.com/storage/browser/builds.catalyst.coop) was updated at the expected time
- [x] [Datasette PUDL version](https://data.catalyst.coop/pudl/core_pudl__codes_datasources) points at the same hash as [nightly](https://github.com/catalyst-cooperative/pudl/tree/nightly)
- [ ] [Zenodo sandbox record](https://sandbox.zenodo.org/doi/10.5072/zenodo.5563) was updated to the record number in the logs (search for `zenodo_data_release.py` and `Draft` in the logs, to see what the new record number should be!)

Relevant logs
gsutil cp gs://builds.catalyst.coop/2024-04-11-0602-d37d630ee-main/2024-04-11-0602-d37d630ee-main-pudl-etl.log .

Creating a new PUDL data release on Zenodo.
2024-04-11 14:52:16,848: INFO - Using Zenodo token: XXXXXXXXXX (zenodo_data_release.py:88)
2024-04-11 14:52:16,848: INFO - Getting new version for 5563 (zenodo_data_release.py:240)
Traceback (most recent call last):
  File "/home/mambauser/pudl/devtools/zenodo/zenodo_data_release.py", line 408, in <module>
    pudl_zenodo_data_release()
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/pudl/devtools/zenodo/zenodo_data_release.py", line 395, in pudl_zenodo_data_release
    .get_empty_draft()
     ^^^^^^^^^^^^^^^^^
  File "/home/mambauser/pudl/devtools/zenodo/zenodo_data_release.py", line 241, in get_empty_draft
    latest_record = self.zenodo_client.get_record(self.record_id)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/pudl/devtools/zenodo/zenodo_data_release.py", line 135, in get_record
    return _NewRecord(**response.json())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/pydantic/main.py", line 171, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for _NewRecord
id
  Field required [type=missing, input_value={'error_id': '2a8f64a27fa...cation.', 'status': 500}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.6/v/missing
files
  Field required [type=missing, input_value={'error_id': '2a8f64a27fa...cation.', 'status': 500}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.6/v/missing

I get the same behavior very quickly when running the data release script locally:
./zenodo_data_release.py --env sandbox --publish --source-dir gs://pudl.catalyst.coop/nightly/

2024-04-14 10:31:35,667: INFO - Using Zenodo token: XXXXXXXXXXX (zenodo_data_release.py:88)
2024-04-14 10:31:35,667: INFO - Getting new version for 5563 (zenodo_data_release.py:240)
Traceback (most recent call last):
  File "/Users/zane/code/catalyst/pudl/devtools/zenodo/./zenodo_data_release.py", line 408, in <module>
    pudl_zenodo_data_release()
  File "/Users/zane/miniforge3/envs/pudl-dev/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/zane/miniforge3/envs/pudl-dev/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/zane/miniforge3/envs/pudl-dev/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/zane/miniforge3/envs/pudl-dev/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/zane/code/catalyst/pudl/devtools/zenodo/./zenodo_data_release.py", line 395, in pudl_zenodo_data_release
    .get_empty_draft()
     ^^^^^^^^^^^^^^^^^
  File "/Users/zane/code/catalyst/pudl/devtools/zenodo/./zenodo_data_release.py", line 241, in get_empty_draft
    latest_record = self.zenodo_client.get_record(self.record_id)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/zane/code/catalyst/pudl/devtools/zenodo/./zenodo_data_release.py", line 135, in get_record
    return _NewRecord(**response.json())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/zane/miniforge3/envs/pudl-dev/lib/python3.12/site-packages/pydantic/main.py", line 175, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for _NewRecord
id
  Field required [type=missing, input_value={'error_id': '45cff1fda13...cation.', 'status': 500}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing
files
  Field required [type=missing, input_value={'error_id': '45cff1fda13...cation.', 'status': 500}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing
[1]    80374 exit 1     ./zenodo_data_release.py --env sandbox --publish --source-dir

In our meeting today we agreed to first stop "successfully publishes to zenodo sandbox" from blocking "we mark a nightly build as successful." This should still let all the data get published, we just won't be able to test the Zenodo interaction until we try to do a manual data release. That's frustrating and fragile but not the end of the world.
Then we will, at some point, think about how to make our data publication process more robust and less dependent on the vagaries of Zenodo's sandbox environment.

@jdangerx who did we deputize to ping the Zenodo folks about this new and exciting behavior?

Nobody yet, are you game to explain the specifics of the failure? If not I'm happy to dig through the logs to provide as good of a bug report for them as I can.

All that's required to reproduce the failure is:
import requests
import os
import json

token = os.environ["ZENODO_SANDBOX_TOKEN_PUBLISH"]
dude = requests.request(
    method="GET",
    url="https://sandbox.zenodo.org/api/records/5563",
    headers={"Authorization": f"Bearer {token}"},
    timeout=5,
)
print(json.dumps(dude.json(), indent=4))

But trying it again now... it works! I guess they heard I was going to send an email and fixed it.
I'll close this issue in the morning if the nightly builds pass.

Zenodo appears to have fixed this problem without intervention from us, as the nightly builds succeeded tonight, so I'll close the issue.


For Aspire preview.2, some more updates are needed to the setup instructions using VS / CLI with aspire

Issue description
In the .NET Aspire setup and tooling doc, we need to make some more updates to the instructions for Preview 2.
Setup using CLI

[ ] During validation, we have been doing dotnet workload update first and then run dotnet workload install aspire. But the documentation asks users to run dotnet workload install aspire first and then run dotnet workload update.

Setup using VS

[ ] In the VS installer steps, after the step Modify to install the .NET Aspire component, we should additionally ask the user to run dotnet workload update and dotnet workload install aspire from a Command prompt. With VS 17.9 Preview 2, the installer will continue to include only the preview.1 version of aspire. So, users need to use the CLI to get the latest preview.2 version of aspire.

[ ] In the Add orchestration projects section, the screenshot of the Add .NET Aspire Orchestrator Support dialog needs to be updated. The Framework dropdown in this dialog has been removed in preview.2.

Couple of screenshots need to be updated

[ ] Aspire Sample Dashboard screenshot should be updated. With preview.2, the Projects, Containers and Executables tabs are combined into one section.
[ ] [Minor nit] In NuGet Package manager for Add Aspire Component, the packages show preview.1 instead of preview.2. I'm not sure if we plan to keep updating this screenshot for each preview.

@joperezr for my comment above for the 'Setup using CLI' part, is there a particular order that you recommend for those commands?

Theoretically the right order is dotnet workload update and then dotnet workload install aspire. Reason is that dotnet workload update will go and update all of the baseline workload manifests you have inside the SDK (a workload manifest is not the workload itself, but simply a "reference" to it which describes which things compose it and what needs to be downloaded if the workload is to be installed), and once those manifests are on their latest version, then you can run dotnet workload install aspire which will then go and download the packs and install them.
That said, I believe in most cases when running dotnet workload  install aspire that would also go and internally call update if needed, so it is possible that install alone would be sufficient. That said, this depends on when was the last time that the SDK in the background did a advertising manifest update, so the safest thing is to run update first and then install.

Preview 2 packages will show in nuget.org after we ship it. To see them in the package manager before we release, you just need to add the package source: https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json

Preview 2 packages will show in nuget.org after we ship it. To see them in the package manager before we release, you just need to add the package source: https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json

Thanks @joperezr. I think we'll put up with the old version numbers.
To resolve some conflicts I've closed the original PR and created this new one: #208
Should be ready to merge.


cyber_launch bug fix:for launch multiple dag files

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.wangbowen04 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Feat new method is directory recursively containing

Check List:

Fixes #1740
Unit tests : YES
Javadoc with a code example (on API only) : YES

Hello,
As discussed on the #1740 issue, here is a first version for FileAssertion.isDirectoryRecursivelyContaining(String syntaxAndPattern) and its internal functions.
I wait for your feedback (when you have some time sure :-) ).
Note : I have the feeling that some modified files were not formatted with the last code formatter, so I have stuck to reformat only code that I added.

Hello, thanks for that review. I would fix these points that week end probably.

@ebundy hope you are ok, are you still active on this PR? No problem if you don't, I can take over as I have a bit of time for the next two weeks.

Hello @joel-costigliola,
Thanks for asking. Sorry, I have had much, much work these last days, so I have chosen to rest away from my personal PC :)
To come back to your question, yes I will still be active on the PR if you allow me : )
I will very probably go on that week end.

@joel-costigliola
I have push the changes. Sorry, I have used the "--amend" flag when I commited.
A comparison  of your upstream/master with my feat_new_method_isDirectoryRecursivelyContaining branch would be maybe clearer.

no problem @ebundy this is your PR you do as you see fit.

@joel-costigliola It seems having a "bug" during the clean process of the TempDirectory feature of JUnit5 , which makes the CI Windows build to fail :
	Suppressed: java.nio.file.DirectoryNotEmptyException: C:\Users\RUNNER~1\AppData\Local\Temp\junit855669844843959767\root\foo 		at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:271) 		at java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105) 		at java.base/java.nio.file.Files.delete(Files.java:1141) 		at org.junit.jupiter.engine.extension.TempDirectory$CloseablePath$1.deleteAndContinue(TempDirectory.java:206) 		at org.junit.jupiter.engine.extension.TempDirectory$CloseablePath$1.postVisitDirectory(TempDirectory.java:201) 		at org.junit.jupiter.engine.extension.TempDirectory$CloseablePath$1.postVisitDirectory(TempDirectory.java:192) 		at java.base/java.nio.file.Files.walkFileTree(Files.java:2742) 		at java.base/java.nio.file.Files.walkFileTree(Files.java:2796) 		at org.junit.jupiter.engine.extension.TempDirectory$CloseablePath.deleteAllFilesAndDirectories(TempDirectory.java:192) 		at org.junit.jupiter.engine.extension.TempDirectory$CloseablePath.close(TempDirectory.java:180)
According to the TempDirectory logic, the directory should have been empty (since files are removed first) when Files.delete(path); is invoked on the directory created during the unit test but actually it is not. The issue is that Files.delete(path) doesn't guarantee that a file is removed, more particularly if the file is locked by the OS.
From my side, I was not completely up-to-date according to the AssertJ master.
Maybe a side effect introduced since the last CI build of my branch (about 1 month ago).
I tried to rollback to an old pom.xml version but no success : same behavior on the CI Windows build.
So as a last try, I have created another branch pointing to an old commit(54a7e42c6da8de38e2c2c7990bca44f0819379ca, about 1 month ago) and I applied only the changes for my PR feature (https://github.com/joel-costigliola/assertj-core/pull/1831).
And what is the result ?
No CI Windows build failure. So there is well something introduced recently enough that makes the Windows build to fail but that is not only a JUnit 5 version change since I tried to tweak that without any success.
I think that I will stop for now.
If you have time don't hesitate to look at that. Otherwise I will probably try to fix the issue the next week end.

I'll have a look at it @ebundy, thanks.

I'll have a look at it @ebundy, thanks.

Forget my last try : the testing branch with a old commit and only my changes for the feature (https://github.com/joel-costigliola/assertj-core/pull/1831/checks?check_run_id=582341888).
It worked yesterday only because the test class that fails was not added in the test source code.
I have added it right now and it has exactly the same behavior : failure during the cleaning of the directory.
I wonder whether the issue doesn't come from the Windows VM. Maybe a recent change ?
Ideally I would like to try to execute the CI run executed with my push from a month ago that was succeeded on the Windows CI build. If that doesn't work any longer, the problem has very probably its origin in the CI conf. But can we perform that ?

You can with a bit of git manipulation, create another branch, squash all the PR commits into one and rebase the branch to the one month old commit, push and that will trigger the build.
makes sense?

@joel-costigliola
With a Windows OS, I reproduced the issue. Bug fixed (caused by stream created by Files.walk() not closed)

Awesome @ebundy!

@ebundy the code looks good to me, the last piece of work is to add the isDirectoryRecursivelyContaining(Predicate) assertion.
I'd like to release 3.16.0 at the end of the month, let me know if you think you can finish it, if not I'm happy to take over - but rest assured you'll get the credits for the whole feature ;-)

@joel-costigliola
Fine ! Let's go to the overloaded one (isDirectoryRecursivelyContaining(Predicate) ) so !
About that I think that it will be straight enough to implement with the current  valid code.
What I may propose you is that I implement it the next week end, then I push that as a pull request. Then you could take over that by making the last adjustments that you deem required.
Credits ? No, the only real credit is for you : you took so much of your personal time to develop that have-to library. Big thanks.
Contributing even a little to AssertJ is a real pleasure.

Fine by me, thanks for the kind words.

Hello @joel-costigliola,
I have add the overloaded one : isDirectoryRecursivelyContaining(Predicate<File>).
I also fixed some things in the isDirectoryRecursivelyContaining(String) javadoc.

Integrated thanks a lot for your work and time @ebundy!

Hello @joel-costigliola,
You are welcome. Thanks to you.


Additional Resources Page

This page would need to look like the dashboard.ejs, quizzes.ejs and lecture.ejs pages and a link to it would need to be added to it's own section at the bottom of the sidebar.
It would include links to resources where students can learn more about SQL.

The PR was merged into the main branch.


Update stage 2 & 3 solutions

Changes based on https://github.com/codecrafters-io/build-your-own-redis/pull/16#discussion_r933343005.

Instead of replying without reading the client's input, we're checking whether the command contains "ping"
We handle io.EOF when reading from a client

Hey @marcosnils! Could you take a look at this please?
The changes are only to stage 2 & 3, the other diffs for stage 4 & 5 are just ripple effects.


Codegen language selection is not working properly

From the README:
xata-codegen generate [path to xata directory] -o generated/client.ts

If you do that in main, the result is a generated/client.ts.ts file. Note the duplicated extension
If you do
xata-codegen generate [path to xata directory] -o generated/client.js

The result is a generated/client.js.ts extension.
Previously the language was deducted from the extension. But now there's an optional --lang parameter that if it's not used, the extension the .ts extension is always added regardless of the extension the user provides. I haven't checked what happens if you use the param and also put an extension in the -o option.
My suggestion is to remove the --lang option and deduct the language from the extension.
cc @TejasQ
It's easy to reproduce the probem by runnning npm run build-example

Oh, okay. I thought language was intended to be configurable via parameter instead of inferred from the output param since it's its own parameter here:
https://github.com/xataio/client-ts/blob/d2d0c2d1f93f029672acf9e0534a84220aa99bb8/codegen/src/codegen.ts#L95
PR incoming...

The language was calculated from the output file and the output file was not modified
https://github.com/xataio/client-ts/blob/d2d0c2d1f93f029672acf9e0534a84220aa99bb8/codegen/src/index.ts#L22-L23


Is there a reason why WorkflowStatusListener does not have an onWorkflowFailed() method?

Our users want us to email them when a workflow status changes to FAILED. WorkflowStatusListener has an onWorkflowCompleted() method and an onWorkflowTerminated() method. Was it intentional not to provide an onWorkflowFailed() method in there? Is this something I should try to add?

The onWorkflowTerminated() method is invoked even during workflow failures. The idea was to add listeners for success and failure cases and this sufficed the requirement.

Oh sorry. I should have tested it. When I did searches on the code I only saw onWorkflowTerminated() called in the terminateWorkflow method of WorkflowExecutor so I didn't think it was called on just a FAILED workflow. I just tested it and it works fine. Thanks!


Updating jsftp and its .raw api

Resolves https://github.com/Robert-W/grunt-ftp-push/issues/50
References:
https://github.com/sergi/jsftp/issues/226
https://github.com/sergi/jsftp/commit/35a8edd6e1132898590d585a415c521df000cb58

Thanks, looks good to me.

Thanks! I don't see this tied to a specific milestone. Is this going to be in 1.21, or something to that effect?

This will be tied to 1.2.1. Unfortunately I do not have the time to actively maintain this project anymore so there are no upcoming milestones and if any interested parties want to take ownership of this project I am more than willing to add owners/collaborators.  I try to keep an eye out for any PR's that look ready to go and merge them as soon as I can but that's about all that is going into this.
I can cut a release for this tonight or tomorrow night for version 1.2.1.

Thanks Robert, totally understand. I still find it very handy and use it frequently, I'd be happy to become a collaborator on it and help maintain it.
Otherwise, tonight is plenty quick for a release 👍 . Thanks a bunch!

Ok sounds good, I just added you as a collaborator. Thanks for contributing back.


Use ant -diagnostics instead of realpath to get ANT_HOME

In the makefile of stf, this parts of code is trying to define the ANT_HOME
      ANT_BINDIR:=$(dir $(firstword $(shell $(WHICH) ant$(BAT) 2>$(NULL))))
      $(warning $(WHICH) ant$(BAT) 2>$(NULL) returned)
      $(warning $(ANT_BINDIR))
      ifneq (,$(ANT_BINDIR))
        ANT_BINDIR:=$(realpath $(ANT_BINDIR)$(D)ant$(BAT))
        ANT_HOME:=$(abspath $(ANT_BINDIR)$(D)..$(D)..)
        $(warning ANT_BINDIR set to $(ANT_BINDIR))
        $(warning ANT_HOME set to $(ANT_HOME))
        $(warning Found $(ANT_BINDIR), will start build with $(ANT_LAUNCHER).  Run make configure to install the required ant version 1.10.1 or follow the prereq install instructions in build$(D)build.md)

However, this part of code doesn't work as expected on RHEL7
RHEL7:
> makefile:174: ANT_HOME not set, looking in /tmp/svttemp/git/stf/../../systemtest_prereqs/apache-ant-1.10.1
> makefile:178: Cannot find /tmp/svttemp/git/stf/../../systemtest_prereqs/apache-ant-1.10.1/lib/ant-launcher.jar, looking for ant on the PATH
> makefile:194: which ant 2>/dev/null returned
> makefile:195: /usr/bin/
> makefile:199: ANT_BINDIR set to /usr/bin/ant
> makefile:200: ANT_HOME set to /usr
> makefile:201: Found /usr/bin/ant, will start build with /usr/lib/ant-launcher.jar.  Run make configure to install the required ant version 1.10.1 or follow the prereq install instructions in build/build.md

$ make -version
GNU Make 3.82
Built for x86_64-redhat-linux-gnu
Copyright (C) 2010  Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Ubuntu16:
> makefile:174: ANT_HOME not set, looking in /tmp/svttemp/git/stf/../../systemtest_prereqs/apache-ant-1.10.1
> makefile:178: Cannot find /tmp/svttemp/git/stf/../../systemtest_prereqs/apache-ant-1.10.1/lib/ant-launcher.jar, looking for ant on the PATH
> makefile:194: which ant 2>/dev/null returned
> makefile:195: /usr/bin/
> makefile:199: ANT_BINDIR set to /usr/share/ant/bin/ant
> makefile:200: ANT_HOME set to /usr/share/ant
> makefile:201: Found /usr/share/ant/bin/ant, will start build with /usr/share/ant/lib/ant-launcher.jar.  Run make configure to install the required ant version 1.10.1 or follow the prereq install instructions in build/build.md

GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Instead of using realpath in the makefile, is using ant -diagnostics a proper approach to get the correct ant.home? For example
root@113fe7939c9b:~/systemtest_prereqs# ant -diagnostics | grep "ant.home" -m 1 --max-count=1 | cut -d ':' -f 2
 /usr/share/ant
root@113fe7939c9b:~/systemtest_prereqs#

There should be a same way on Windows to parse the ant -diagnostics result to get the real ant.home

FYI @sxa555

A fair comment and that sounds like a reasonable approach - that code predates me being involved with the project so I'll tag in @lumpfish :-)

@TianyuZuo - what is it you are saying is not working?  Are you saying that on RHEL /tmp/svttemp/git/stf/../../systemtest_prereqs/apache-ant-1.10.1/lib/ant-launcher.jar does actually exist?  Since the code is trying to find ant I can't see how running ant is going to help.  What if ant is not on the path?

Not exactly.
What I was trying to talk about is the Ant in the system which was initially used to run the top level build.xml, not the stf-downloaded Ant.
If the Ant in the system was not successfully setup, the code snippet in the makefile may fail to find the realpath of system installed Ant. Taking that RHEL machine as an example, the Ant was installed in /usr/share/ant/bin/ant but the makefile can only find its symlink /usr/bin/ant. If the ant -diagnostics was used, the ant.home would be printed out and that was correctly pointing to /usr/share/ant.
If Ant was not on the path, I don't think which ant would give back a correct path either. So this suggestion is assuming that system can find a usable Ant on Path at least, the same as the previous makefile.

Looking at the gmake doc: https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html realpath is supported to follow symlinks.  Do you still have access to the machine where this is not working?  I see you have included make -version output - are you saying there is an issue with realpath in make 3.82?


Update to layouts/app.vue

Current coverage is 62.46% (diff: 100%)

Merging #56 into master will not change coverage

@@             master        #56   diff @@
==========================================
  Files            10         10
  Lines           389        389
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            243        243
  Misses          146        146
  Partials          0          0

Powered by Codecov. Last update 803b21b...d7fb956

Thank you @johnlindquist
Great job about your videos 👏


Update Composer dependencies (2021-11-18-00-04)

Visual regression test passed!
View the visual regression test report