Jupyter Scheduler 2.10.0 Source Distribution tar built too large causing PyPI upload failure

Description
Jupyter Scheduler 2.10.0 Source Distribution initial upload failed due to PyPI source distributions size limits (~150 Mb) due to tar build being drastically larger in size vs before jupyter-releaser introduction.
Built distribution upload went through, npm upload did not as it's later in the script.
How to reproduce

See CI failure: https://github.com/jupyter-server/jupyter-scheduler/actions/runs/11808986203/job/32898512565#step:5:372
Try building jupyter scheduler PyPI source distribution locally with jupyter-releaser build-python, see its size (>100 Mb)

Expected behavior

✅ Source distribution of the normal size is available in PyPI, @jupyterlab/scheduler 2.10.0 is released at npm.
Workflow does not fail, source distribution of the reasonable size is built and uploaded to PyPI, upload to npm happens.

For reference, here is the relevant log excerpt:
WARNING  Error during upload. Retry with the --verbose option for more details.
ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
         File too large. Limit for project 'jupyter-scheduler' is 100 MB. See
         https://pypi.org/help/#file-size-limit for more information.

Jupyter Scheduler 2.10.0 npm package is now available at https://www.npmjs.com/package/@jupyterlab/scheduler, Source Distribution is now available at PyPI https://pypi.org/project/jupyter-scheduler/2.10.0/#files.
Let's use this issue to track the need to understand why Jupyter Scheduler 2.10.0 Source Distribution tar was built too large causing PyPI upload failure and to prevent it happening in the next release.
jupyter_releaser issue on the topic: https://github.com/jupyter-server/jupyter_releaser/issues/592

Try building jupyter scheduler PyPI source distribution locally with jupyter-releaser build-python, see its size (>100 Mb)

Out of curiosity, do you know why it produces so big a distribution? From a quick look it seems that you might be missing:
[tool.jupyter-releaser.hooks]
before-build-python = ["jlpm clean:all"]

in the pyproject.toml but that's just a guess.

@krassowski no. I've created https://github.com/jupyter-server/jupyter_releaser/issues/592 in jupyter_releaser repo to surface the problem and potentially get some insight from jupyter_releaser contributors.

Do you have the contents of the package built locally with jupyter-releaser build-python?

@krassowski yes, here it is https://www.dropbox.com/scl/fi/51y8zhsjeqx2jmsyg9mll/jupyter_scheduler-2.10.0.tar.gz?rlkey=9v5gafpayncj7831zfr4q4o00&st=svnlkhgx&dl=0 (153.9 Mb)

It looks like it includes .yarn and node_modules directories which I am sure is responsible for a large portion of the size. It obviously should not be included. Also see https://github.com/jupyter-server/jupyter_releaser/issues/592#issuecomment-2478372873.
I think in addition jlpm clean:all you should also add:
[tool.hatch.build.targets.sdist]
artifacts = ["jupyter_scheduler/labextension"]
exclude = [".github", "binder"]

so binder directory gets excluded.
That said, I already see jupyter_scheduler/labextension in the tarball you shared and it, along node_modules should have been excluded by hatch because it is in your .gitignore.
So why does it include things from the git repo?
In the logs of check-release action (https://github.com/jupyter-server/jupyter-scheduler/actions/runs/11809051201/job/32898683727) I see that the releaser is reading configuration from package.json rather than from pyproject.toml. I wonder if this could be related:
build-python

--------------------------------------------------
Using default value for dist_dir: 'dist'
Using default value for python_packages: '['.']'
Using default value for help: 'False'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Running hooks for before-build-python
jupyter-releaser configuration loaded from package.json.

Also, that one does include the clean hook:
https://github.com/jupyter-server/jupyter-scheduler/blob/14c44518f9d48eb40eb6c20e063c275c84358698/package.json#L132-L143
Interesting. It looks like it did not use hatch at all?

None of that helps yet: https://github.com/jupyter-server/jupyter-scheduler/pull/561
I went ahead and triggered a new check-release run on an unrelated project just to see if this is not a regression in the ecosystem (rather than a misconfiguration). Compare older run on variable inspector with the run triggered today and both result in 1.53 MB
of artifacts, so I do not think that this is a system-wide issue, but just a problem with configuration.

I tried aligning the scheduler config with other repos using releaser in https://github.com/jupyter-server/jupyter-scheduler/pull/561 but nothing helped.
The thing is that jupyter-releaser does not do anything bespoke, it just runs pipx run build (here). It should not result in anything different from python -m build as used by the build action:
https://github.com/jupyter-server/jupyter-scheduler/blob/14c44518f9d48eb40eb6c20e063c275c84358698/.github/workflows/build.yml#L57

Running pipx run build locally does not produce such a large tarball for me, just 3.6 MB.


contextify for Windows, MSVC2010 proj and binary

https://github.com/Benvie/contextify
EDIT: recent files Feb 26 2012
https://github.com/downloads/Benvie/contextify/contextify-0.1.1-node0.6-win32.zip
https://github.com/downloads/Benvie/contextify/contextify-0.1.1-node0.7-win32.zip

interesting, perhaps we should consider bundling .node's for windows (x64/x86).

To build it you need the header files for v8 and node, node.lib for the version you're linking, and MSVC compiler which can be gotten for free via MSVC++ Express, but few people will have that and be thrilled to install a not insignificant dev environment to get it.

Hi, how important is contextify? Is it possible to replace it with a pure javascript part? We would like to add jsdom as a dependencie to etherpad lite for doing html imports. But we have a no compile policy to ensure everything works on windows

Node is adding Isolates in 0.7 which solves this.

Also with windows you can just include the prebuilt .node file which should work on I think any Vista or Win 7. One benefit to Windows is that binaries usually just work.

Do we have a prebuild .node file?

https://github.com/Benvie/contextify/blob/master/build/Release/contextify.node

@Benvie using your .node with 0.6.3  I get  the following message:

require("contextify")
Internal Contextify ERROR: Make sure Contextify is built with your current Node
version.
...

@albertosantini Are you sure you have contextify.node in correct dir? It should be in build/Release ...
E.g. d:\programovani\zdrojaky\javascript\node_modules\contextify\build\Release\contextify.node

@bubersson Yes, I checked it. I think it has been built for 0.6.1 (and not for 0.6.3).

@albertosantini It works for me... on 64bit Win7. I'm not sure if this could be related to 64 vs 32bit though.
D:\programovani\zdrojaky\javascript\node_modules>node -v
v0.6.3
D:\programovani\zdrojaky\javascript\node_modules>node

require("contextify");
[Function]

@bubersson Thanks for checking. Well... cloning Benvie's fork in node_modules dir, it works nice. Before I copied only the contextify/build/Release and contextify/lib dirs. Thanks for your time.

I'll create a new binary for 0.6.4 which should come out soon, but there shouldn't be any compatabity breakers so far for the binary compiled for 0.6.1.

I've got two windows 7 64 bit computers, both running node 0.6.7, one of them runs the binary correctly and the other fails. I've tried installing Visual C++ 2010 to get MSVCR100.DLL, but it didn't help. I've also tried installing node 0.6.6 instead, but that didn't help.
Is there any way to find out why it's failing?

I used a tool called dependency walker.  It shows what dlls are needed.
You could try using rundll on the dll too.
On Jan 17, 2012 10:48 PM, "Peter West" <
<EMAIL_ADDRESS>wrote:

I've got two windows 7 64 bit computers, both running node 0.6.7, one of
them runs the binary correctly and the other fails. I've tried installing
Visual C++ 2010 to get MSVCR100.DLL, but it didn't help. I've also tried
installing node 0.6.6 instead, but that didn't help.
Is there any way to find out why it's failing?

Reply to this email directly or view it on GitHub:
https://github.com/tmpvar/jsdom/issues/360#issuecomment-3528794

II would have assumed that installing msvc express would do the trick, but maybe not. What is technically needed though is the msvc redistributable. http://www.microsoft.com/download/en/details.aspx?id=5555. It's just that installing msvc should provide it, but worth a try

Just installed the x86 version and it works now.

i have the dll, but contextify still says 'Internal Contextify ERROR: Make sure Contextify is built with your current Node'

@Benvie, please, can you provide the binary for contextify 0.1, required by jsdom 0.2.12? Thanks.

https://github.com/downloads/Benvie/contextify/contextify-0.1.1-win32.zip
Let me know if that works. It was compiled against Node Master so I'm not sure how it will fair against 0.6.x. If it doesn't work I'll make another go at it.

https://github.com/downloads/Benvie/contextify/contextify-0.1.1-node0.6-win32.zip
https://github.com/downloads/Benvie/contextify/contextify-0.1.1-node0.7-win32.zip

Can you please explain on how to install those binaries to allow jsdom be installed via npm ?

You'll have to do npm install contextify -f to make it not remove the
directory. Then you copy the .node file into I think build/Release will
work. (make the folders if needed)
On Wed, Feb 29, 2012 at 4:53 PM, Evgeny Shepelyuk <
<EMAIL_ADDRESS>
wrote:

Can you please explain on how to install those binaries to allow jsdom be
installed via npm ?

Reply to this email directly or view it on GitHub:
https://github.com/tmpvar/jsdom/issues/360#issuecomment-4247647

You'll have to do npm install contextify -f to make it not remove the directory when it inevitably fails to build on Windows. Then you copy the .node file into I think build/Release will work. (make the folders if needed)

FYI if you guys compile with node-gyp or the latest node source code it will produce a .node file that does not depend on MSVRT, as that gets statically linked now. The upstream contextify already has a bindings.gyp file and is already compatible with node-gyp.
Also, for @Benvie and any other people attempting to distribute binaries, only changes in the minor node version need to be recompiled. i.e. 0.6.x has binary stability.
Cheers!

Ooh good to know. I thought that was the case but hadn't seen confirmation.

Hello, after some playing around directories i managed to get jsdom working on windows. thank you very much
Later will try to build with node-gyp for my x64 PC

Hi again.
Although i was able to run jsdom on Windows, using node-gyp and binaries provided above, i could do it only for local installation. Is it possible to do the same tricks for global modules installation for windows ?

Try putting the .node in C:\Program Files (x86)\nodejs (AKA the place where node.exe is).

Unfortunately not working. But nevermind, i'll do some workarounds to solve my problems regarding global installation of contextify and jsdom modules


Disabled Windows Defender

Hi!
After creating a Windows 10 template VM, Windows Security shows this message under Virus & threat protection:
"Your Virus & threat protection is managed by your organization.
No active antivirus provider. Your device is vulnerable."
Under Security providers, it says:
"Microsoft Defender Antivirus is turned off."
Is this intended behaviour? How can i revert this? I have looked for modified group policies and did not find any.
Thank you

Found out there was a regristry key called DisableAntiSpyware under Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender.
Deleting this key reactivated windows defender.

This is done intentionally in optimize.bat. Windows Defender is a resource hog and fake security. If you're on Qubes OS, I assume you value the key real security principle employed by Qubes OS over that: security through isolation
I've contributed to Wikipedia why anti-malware software ultimately doesn't work and won't ever work: https://en.wikipedia.org/wiki/Security_through_obscurity#Criticism

One of the largest proponents of security through obscurity commonly seen today is anti-malware software. What has followed is an infinite cat-and-mouse game of attackers finding new ways to avoid detection and defenders coming up with increasingly contrived but secret signatures to flag on.[10]

Anti-malware software will only stop garden variety malware, script kiddies, and technically uninformed bad guys. Then give you false positives along with a false sense of security. That's it.


bug: undefined method `has_key?' after upgrading to Jekyll 3.5

$ jekyll serve --trace
NoMethodError: undefined method `has_key?' for #<Liquid::Context:0x0055e25ad300f8>
  /var/lib/gems/2.3.0/gems/jekyll-gist-1.4.0/lib/jekyll-gist/gist_tag.rb:17:in `render'
  ...

In Liquid 4.0 has_key has been renamed to key
Fixed in #41

Same here, this bug prevents from using Jekyll 3.5 for now as deploy fails 😢

Generating...
3:57:55 PM: Liquid Exception: undefined method has_key?' for #<Liquid::Context:0x0000000478d648> in /opt/build/repo/_posts/2016-09-18-utiliser-des-plugins-jekyll-avec-github-pages.md 3:57:55 PM: jekyll 3.5.0 | Error: undefined method has_key?' for #Liquid::Context:0x0000000478d648

v1.4.1 was just released!


Symbol not found: __ZN10tensorflow12OpDefBuilder4AttrESs

Despite following instructions on similar issues, this error persists. Has anyone been able to resolve it so far?
python3 run_demo.py
Using TensorFlow backend.
Traceback (most recent call last):
  File "run_demo.py", line 27, in <module>
    from crfrnn_model import get_crfrnn_model_def
  File "./src/crfrnn_model.py", line 28, in <module>
    from crfrnn_layer import CrfRnnLayer
  File "./src/crfrnn_layer.py", line 28, in <module>
    import high_dim_filter_loader
  File "./src/high_dim_filter_loader.py", line 28, in <module>
    custom_module = tf.load_op_library(os.path.join(os.path.dirname(__file__), 'cpp', 'high_dim_filter.so'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 60, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(./src/cpp/high_dim_filter.so, 6): Symbol not found: __ZN10tensorflow12OpDefBuilder4AttrESs
  Referenced from: ./src/cpp/high_dim_filter.so
  Expected in: flat namespace
in ./src/cpp/high_dim_filter.so

Hi Q
This is an error because arising because you copied highdimfilter in a python environment different from the one used for running the code.
Why do you use python3 instead of python2.7 ?
In either cas, you should do make under the virtualenv used to make run demo.py.

Hi Matthias
I just changed my virtualenv to python2.7 and it produced the same error. I printed out the versions of all the other packages, that might hold some clues.
requirements_q.txt

Hi Q
You have to "make clean" in the cpp folder et redo "make" under the same virtualenv you use for running python otherwise, tf won't load the new op. Did you do this ?

Indeed, I was in the virtualenv while I used "make clean" to remove the .so file and compiled a new one.  Unfortunately tf isn't able to locate the new op...

Hi Q
What is your platform ? Desktop or server ?
Perhaps the same problem is pointed there : https://github.com/sadeepj/crfasrnn_keras/issues/35

It occurs on two different machines (desktop & laptop). They're both OSX, one High Sierra and the other Mojave. I haven't tried on a server yet.

Hi Q,
Did you fix your problem ?
Trying this within the src/cpp/ folder ?
TF_CFLAGS=( $(python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_compile_flags()))') )
TF_LFLAGS=( $(python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_link_flags()))') )
g++ -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0 -shared -fPIC -I/users/p19039/orlando/.virtualenvs/r-tensorflow/lib/python2.7/site-packages/tensorflow/include -O2  -o high_dim_filter.so high_dim_filter.cc modified_permutohedral.cc

Unfortunately I've not been able to, also not with previous versions of tensorflow.
Running your command produces the error:
high_dim_filter.cc:25:42: fatal error: tensorflow/core/framework/op.h: No such file or directory
 #include "tensorflow/core/framework/op.h"

This is on a Mac, not a Linux.

What about with TF version 1.2 ?

@qvandenberg Have you fixed that problem? I meet the same problem too. I tried several versions mentioned here, but none works.

It's fixed now, thank you for your efforts. The problem turned out to be the C++ compiler. It worked only when I used C++ 4.2.1:
Apple LLVM version 7.0.0 (clang-700.1.76)
It's stored under /usr/bin instead of /usr/local/bin where I keep most of my binaries. That caused the import problem.

@qvandenberg Got it! Thanks a lot!

Hello! When I run the sh.compile.sh command, it compiles successfully, but when I run train.py, I get an error: undefined symbol:_ZTIN10tensorflow8opkernelE. I checked a lot of information on the Internet and did not solve it. Hope you help me see what is the problem. Thanks again! ! !
ubuntu :16.04 ; nvidia-driver-440;cuda:10.0; cudnn:7.5.0; tensorflow-gpu:1.13.1; gcc/g++:5.4.0;

@matthiasmace


[web] Fix layout for container and main-view

This PR aims to fix a dislocation in main-view:

The following steps can use to reproduce the bug (verified in Chrome 47.0.2526.106 on Ubuntu 14.04.4):

Open the flow detail panel
Scroll the flow table

Thanks for the PR!
Things work just fine for me (Chrome 48.0.2564.116 and Firefox 43, Win 10).
What's the motivation for changing the box-sizing? Why do you modify the event log flex?

@mhils Things works again with the latest version of Chrome and Firefox, likely to be an issue for flex in specific version of Chrome.
And the event log, again, was broken on my browser and it works now :smile:
As for the box-sizing, I found it confusing when I debugging the issue. Though it should not be included it here, I think.
Seems good to be closed.

There are some good reasons for border-box: http://www.paulirish.com/2012/box-sizing-border-box-ftw/ - I personally like it a lot and I feel it makes dev easier.


[New game]: the elementalist

🎮 Game Request
A simple match-3 for you to become THE Elementalist!
Point down the features
Gain levels by matching 3 or more elements.
Select program in which you are contributing
GSSoC23
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @S-ishita !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

Hey @S-ishita ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


make install fails following Quickstart for Go-based Operators

When following the quickstart guide using operator-sdk v1.3 (modified for the operator I'm trying to build), I get the following error from make install:
        * accumulateFile error: "accumulating resources from 'bases/operator.shipwright.io_shipwrightbuilds.yaml': evalsymlink failure on '/var/home/adkaplan/go/src/github.com/shipwright-io/operator/config/crd/bases/operator.shipwright.io_shipwrightbuilds.yaml' : lstat /var/home/adkaplan/go/src/github.com/shipwright-io/operator/config/crd/bases/operator.shipwright.io_shipwrightbuilds.yaml: no such file or directory"
        * loader.New error: "error loading bases/operator.shipwright.io_shipwrightbuilds.yaml with git: url lacks host: bases/operator.shipwright.io_shipwrightbuilds.yaml, dir: evalsymlink failure on '/var/home/adkaplan/go/src/github.com/shipwright-io/operator/config/crd/bases/operator.shipwright.io_shipwrightbuilds.yaml' : lstat /var/home/adkaplan/go/src/github.com/shipwright-io/operator/config/crd/bases/operator.shipwright.io_shipwrightbuilds.yaml: no such file or directory, get: invalid source string: bases/operator.shipwright.io_shipwrightbuilds.yaml"

I see a file in confg/crc/bases called operator.shipwright-io_shipwrightbuildren.yaml instead.
Steps to reproduce

Init repo with operator-sdk:

$ operator-sdk init --domain=shipwright.io --repo=github.com/shipwright-io/operator

Create an API

$ operator-sdk create api --group operator --version v1alpha1 --kind ShipwrightBuild --resource=true --controller=true

Create the image and push via make docker-build and make docker-push
Run make install

@adambkaplan I was able to recreate the problem. It seems to happen when make manifests is run. The operator.shipwright-io_shipwrightbuildren.yaml file is being created instead of operator.shipwright-io_shipwrightbuilds.yaml as if something was doing bad pluralization.
The workaround I was able to do was:

Rename the file mv operator.shipwright.io_shipwrightbuildren.yaml operator.shipwright.io_shipwrightbuilds.yaml
Edi the file to reference builds: vi config/crd/bases/operator.shipwright.io_shipwrightbuilds.yaml  change all references to buildren to builds.
Run make install

@adambkaplan I was able to recreate the problem. It seems to happen when make manifests is run. The operator.shipwright-io_shipwrightbuildren.yaml file is being created instead of operator.shipwright-io_shipwrightbuilds.yaml as if something was doing bad pluralization.
The workaround I was able to do was:

Rename the file mv operator.shipwright.io_shipwrightbuildren.yaml operator.shipwright.io_shipwrightbuilds.yaml
Edi the file to reference builds: vi config/crd/bases/operator.shipwright.io_shipwrightbuilds.yaml  change all references to buildren to builds.
Run make install

I created a repo with 3 commits showing the 3 steps I've taken to recreate the issue:
https://github.com/jmrodri/shipwright/commit/149fafbc2bc2871f9519da16bd5aa88d659a0462
The third commit: make manifests is causing the problem.

I created a repo with 3 commits showing the 3 steps I've taken to recreate the issue:
https://github.com/jmrodri/shipwright/commit/149fafbc2bc2871f9519da16bd5aa88d659a0462
The third commit: make manifests is causing the problem.

We're using controller-gen v0.4.1 which depends on gobuffalo/flect v0.2.0 for pluralization logic. gobuffalo/flect v0.2.0 has a problem:
[jesusr@transam flecttest]$ ./flecttest
builds
shipwrightbuildren

With gobuffalo/flect v0.2.2, it is fixed.
[jesusr@transam flecttest]$ ./flecttest
builds
shipwrightbuilds

We're using controller-gen v0.4.1 which depends on gobuffalo/flect v0.2.0 for pluralization logic. gobuffalo/flect v0.2.0 has a problem:
[jesusr@transam flecttest]$ ./flecttest
builds
shipwrightbuildren

With gobuffalo/flect v0.2.2, it is fixed.
[jesusr@transam flecttest]$ ./flecttest
builds
shipwrightbuilds

Looks like the fix was done after the v0.4.1 release https://github.com/kubernetes-sigs/controller-tools/pull/522

Looks like the fix was done after the v0.4.1 release https://github.com/kubernetes-sigs/controller-tools/pull/522

It will be fixed when we update/align SDK with upstream: WIP https://github.com/operator-framework/operator-sdk/pull/4402

It will be fixed when we update/align SDK with upstream: WIP https://github.com/operator-framework/operator-sdk/pull/4402

@adambkaplan what plugin version are you using (the layout field in your PROJECT file)? #4402 won't actually fix this unless you are using the proper version of controller-gen (v0.4.1) which is only supported by the go/v3 plugin (introduced in operator-sdk v1.3.0). v0.4.0+ creatse v1 CRDs and webhooks by default, so you can use v0.4.1 if you add crdVersions={v1beta1} to your Makefile and webhookVersions={v1beta1} to your api/<version>/<kind>_webhook.go markers. Really https://github.com/kubernetes-sigs/controller-tools/pull/522 should be backported to v0.3.0.
@camilamacedo86 can you remove the Closes #4453 from #4402?
Since this is fixed upstream I'm going to close this issue.

Actually re-opening this to make sure this is fixed, since you mentioned operator-sdk v1.3 was used.

/assign @jmrodri
/kind bug


Remove unnecessary null-checks for exchange contexts.

The unit tests for this code no longer call it without an exchange
context, so we can remove the workarounds.
Fixes https://github.com/project-chip/connectedhomeip/issues/8031
Problem
Null-checks working around bugs in tests that have since been fixed.
Change overview
Remove the unnecessary null-checks
Testing
Ran the tests, they pass.

/rebase


leave a group function

I write in php
$response = $api->leaveChat();
but console says
PHP Fatal error:  Uncaught Error: Call to undefined method Zelenin\Telegram\Bot\Api::leaveChat()
Can you tell me the correct way to use this function?

Do you have this function ai Api.php?

my fault, sorry. i need update Api.php. thanks


Feature/log level info

:rocket: deployment of gp-redirect succeeded (http://gp-redirect-pr-18.dev.beta.nhschoices.net)


Adding a Moving Average

Hi! Thanks for this. Is there any way to add a moving average to a zero line cross?

The purpose of this EA is testing C1 indicators.
To add MA to your indicator you need ask to a developer to do this.


Versioning Enhance Test Suite Part 1

Added tests cases are blue. If there's anything marked N/A you think shouldn't be please comment.
This PR is just for tests in versioning.test.ts. There are other tests in incompatible-versioning.test.ts that I will tackle in a follow-up PR.
Pertains to #1406.

Gentle ping on this PR.


Remove dotnet-trace ps from memory leak tutorial

Summary
dotnet-counters has a ps command.

closing and reopening for a fresh build


macOS Guest Clipboard Erased When App Copied From Closes

Describe the issue
In macOS running natively, I can select text in an app, close the app, then paste the text into another app.
In UTM, if I select text in an app, then close the app, the clipboard is instantly erased.
(This is a pain, as my long-accustom workflow is to open an app+file, grab required text, and close the app to automagically return to the app in which I am working, then paste the text. Now, I have to open an app+file, grab required text, manually switch back to the app in which I am working, paste the text, switch back to the app from which I grabbed text, and close it. In other words, what was a 4 step process now becomes a 6 step process, slowing work by somewhere between a third and a half!)
Configuration

UTM Version: 4.6.3
Host macOS Version:  14.7.1
Guest macOS Version: 14.7.1 or 15.1.1 or 15.2 (basically, all 13.x or later versions which I have tested)
Mac Chip (Intel, M1, ...): M2 Max 96GB

Crash log
None.
Debug log
No relevant entries in either host or guest logs.
Upload VM
N/A

I cannot reproduce this with either guest->host or host->guest.
What I tried:

Open Notes on machine A
Type in some text and select it
Cmd+C
Cmd+Q to exit Notes
Open Notes on machine B
Press Cmd+V
The text shows up as expected

I tried with with guest A/host B as well as host A/guest B
Please provide steps to reproduce the issue on your side.


[Question] Add a new resource.

is the parent resource necessary? can i create a new top resource?

The backend recently modified the code for parameter verification, causing some bugs here. The main problem is that the frontend verification, backend verification and database verification are inconsistent. I tried to fix it all, but I don't know which is correct.

I think the parent menu is not required. so you can fix the backend verification.

Database.

Backend.

Frontend.

okay, I will fix this bug according to the frontend verification.

This is frontend bug, not backend.


Fesh Install Packaging certificate errors (1.0.21149.1 Latest Stable)

Below is the error statement and not possible to complete the install so users must go to MSI's in Alpha release (Win10 PRO 21H2):
ERROR:
This app package is not signed with a trusted certificate. Contact your system administrator or the app developer to obtain a new certificate or app package with trusted certificates. The root certificate and all immediate certificates of the signature in the app package must be trusted (0x800B010A)

hello @jimmy6DOF, I do not suopport v1 anymore. sorry.


New trailer format

DO NOT MERGE YET! (it's probably ok, which means it's working on the Simulator, but fails swapping on real hw which could be a Mynewt issue, not related to Mcuboot).

We also need to update the padding code in imgtool.py. We can remove the --align flag, and simplify the padding to use this new format.

I'll wait for this one to be merged and send another PR over that one or update the existing PR if it's not merged by then

Will that cross the 1.0 release boundary?

I'll wait for this one to be merged and send another PR over that one or update the existing PR if it's not merged by then

Will that cross the 1.0 release boundary?

We had pretty much decided on moving forward leaving the API stuff out of 1.0 on our last meeting but if people (on Zephyr ML) are good with adding the API as is I think we can add it and update the ecosystem later (eg Mynewt's patches need to be officially added/merged, and so on...)

We had pretty much decided on moving forward leaving the API stuff out of 1.0 on our last meeting but if people (on Zephyr ML) are good with adding the API as is I think we can add it and update the ecosystem later (eg Mynewt's patches need to be officially added/merged, and so on...)

Sorry, just to confirm what you mean, then: this patch would be part of mcuboot 1.0, and if it becomes API later, that's post-1.0, right?

We also need to update the padding code in imgtool.py. We can remove the --align flag, and simplify the padding to use this new format.

The status bytes still depend on "write size" so I haven't removed the --align flag. Should I assume always 8 byte write size? That could error if the padding is small enough, I think...

We had pretty much decided on moving forward leaving the API stuff out of 1.0 on our last meeting but if people (on Zephyr ML) are good with adding the API as is I think we can add it and update the ecosystem later (eg Mynewt's patches need to be officially added/merged, and so on...)

Sorry, just to confirm what you mean, then: this patch would be part of mcuboot 1.0, and if it becomes API later, that's post-1.0, right?

What I meant was that this PR here (#76) is for 1.0. PR #61 and any related changes might be postponed.

Sorry, just to confirm what you mean, then: this patch would be part of mcuboot 1.0, and if it becomes API later, that's post-1.0, right?
What I meant was that this PR here (#76) is for 1.0. PR #61 and any related changes might be postponed.

OK, so mcuboot 1.0 users will all just assume this format. Then we need a major version bump if we're going to change it later. And I guess that means the absence of the magic values which signal mcuboot is passing a parameter to the child means we can assume 1.0 for now, and tell it apart from 2.0 later, right? Just trying to understand how to maintain compatibility across major releases, including 1.0.

I thinking trying to have an API into the bootloader that does flash operations is going to be difficult. Some of the flash drivers may want interrupts or clocks, and it would be difficult to manage in the context of a new OS running. We could come up with something that returned information about it.
But, my main goal here was to define this is such a way that it never needs to change. To mark an upgrade, you write 16 magic bytes at the end of a partition. The image_ok is defined here, but everything before that in the trailer could change, and OTA shouldn't need to know about it.

But, my main goal here was to define this is such a way that it never needs to change.

I too hope that we never have to change this :). Just trying to make sure there's a path if we ever have to -- which, with a major version change, it seems like we would.
Not sure what you mean about the bootloader doing flash operations, but yes, I had in mind having a query about the layout, not a call into a flash driver (for the reasons you describe).

After a discussion with [redacted] today, I think the things we'll need to communicate are:

Partitions, at least the partitions the bootloader cares about (the slots and scratch).
Version information.
Separate version information about the trailer, so know how it should be marked.
How mcuboot is configured (swap, upgrade only, run slot)
Which slot was run (needed for OTA with dual slot solution).

Things I'm less sure about:

Other partitions
sector map and write alignment (the drive really should know these)
???

So, what's the verdict? Do we merge this PR and track updates at #61 or is anything else to do here?

So, what's the verdict? Do we merge this PR and track updates at #61 or is anything else to do here?

So you mean you want to create the header with exported declarations in #61?

So you mean you want to create the header with exported declarations in #61?

I'm feeling a little lost here! What does the exported declarations have to do with the on disk trailer format? Also not sure why @d3zd3z added the same comment here and on #61 about the discussion with [redacted], but those suggestions all seem to be related only to what the API will export, aka #61.

Yeah, we don't need to worry about the API in this change.
I think it is fine to merge this, at least with the imgtool change to go with it. This is separate from the API, which isn't even fully defined.

So you mean you want to create the header with exported declarations in #61?

I'm feeling a little lost here! What does the exported declarations have to do with the on disk trailer format?

My original comment related to the addition of struct image_trailer to boot/bootutil/include/bootutil/bootutil.h.
That part of the patch has a comment which reads "This is not actually used by mcuboot's code but can be used by apps".
I originally said that, if there wasn't going to be an OS/bootloader ioctl() style API for it (and you've since made it clear that having that API is post-1.0, and this patch series is for 1.0), then it seemed like a good idea to keep code that has users outside of mcuboot itself in a separate place than bootutil.h, because mixing up "internals" with exported definitions seems like asking for trouble. It will make the compiler flags for other applications pretty ugly at the least, and risks leaking internal definitions that others will come to rely on, etc.
You responded:

Creating an include with only "exported" stuff seems like a good idea.

So my turn to feel a little lost, since that comment and that struct are still in bootutil.h. Will that be moved to another place, are you going to keep it in bootutil.h?

So my turn to feel a little lost, since that comment and that struct are still in bootutil.h. Will that be moved to another place, are you going to keep it in bootutil.h?

Ah, that clarifies everything! I think we can add it now and I'll update the ticket for PR #61 mentioning that a new "exported functions" header needs to be created and the trailer struct moved there. Would that be OK?

Ah, that clarifies everything! I think we can add it now and I'll update the ticket for PR #61 mentioning that a new "exported functions" header needs to be created and the trailer struct moved there. Would that be OK?

Just to be clear, I'm not against merging this, just saying that I think that struct isn't so useful for external applications until it's in its own header. If you're going to merge as-is then I think that the comment which says it's not for mcuboot is a little strange, but that's not a big deal. If you're going to update this to make a separate header for that struct definition then awesome.


WIP es2022 features: escodegen update

Have to wait until
eslint/espree#486
eslint/eslint-scope#69
will be finished

eslint/eslint-scope#69 was released in v6.0.0
eslint/espree#486 was released in v8.0.0


[LIVY-870] Adding changes to provide Spark config using --properties-file

What changes were proposed in this pull request?
Currently for batch jobs, individual Spark configs are added as --conf parameter in Spark submit command line. If there are many Spark configurations added in Livy APIs, it might increase the length of command line and in some cases might cause issues due to limit on command line length (Windows has 8191 character length limit on the command line length.).
In this PR, modifying the behavior to merge all the configurations into a temporary properties file and adding that as --properties-file opt in spark-submit command
JIRA: https://issues.apache.org/jira/browse/LIVY-870
How was this patch tested?
Verified Spark job submission in a local Yarn cluster. Checked in the logs that all configurations are merged to a properties file which is added as an opt in spark-submit command. Please see the logs below.
21/10/12 12:01:15 INFO SparkProcessBuilder: Merged Spark config and generated file: D:\Data\Livy0.8\sparkConfig7641984478423208311.properties
21/10/12 12:01:15 INFO SparkProcessBuilder: Running 'D:/Spark/\bin\spark-submit.cmd' '--name' 'SparkBatchJobTest-14' '--class' 'WordCountJob' '--properties-file' 'D:\Data\Livy0.8\sparkConfig7641984478423208311.properties' 'file:///D:/Spark/examples/jars/default_artifact.jar' 'D:/Test.txt'

@akshatb1 would you please rebase to latest master to fix the CI failure?

@akshatb1 would you please rebase to latest master to fix the CI failure?

Thanks @leesf. I have rebased the branch.


Do not assume minion_id is hostname/fqdn

With this change we:

Rename hostname to fqdn. Feels more correct.
Do not assume on the salt part that the minion_id is the hostname/fqdn
Explicitly save minion_id as a column, as well as fqdn as another column
We will use minion_id internally to talk to salt, but show hostname, and
probably other properties [that we can cache in the very near future as
columns] like number of cores, memory, disk...

unfortunately the e2e-tests with this patch, plus a fix (https://github.com/kubic-project/e2e-tests/pull/31) of the hostname check that failed here in this run still fail at the cluster info. the nodes are still not visible in the kubernetes api.
{
    "metadata": {
        "selfLink": "/api/v1/nodes",
        "resourceVersion": "76"
    },
    "items": []
}

do you think there is still something missing here for that PR or is it a separate issue? i am not 100% sure if it's related

I'd try to reuse our existing migration now that we can to keep things more clean the maximum time we can.
Salt also needs to be updated (so this PR doesn't go alone) for the e2e tests to work: https://github.com/kubic-project/salt/pull/40.
However that PR (https://github.com/kubic-project/salt/pull/40) is a WIP and I will finish it this morning. The problem is that we need to blindly merge one or the other for the e2e tests to work for the other one.

The problem is that we need to blindly merge one or the other for the e2e tests to work for the other one.

@ereslibre ok, just saw the salt PR as well. at any time i can run the e2e-tests with both PR's, but it won't be visible in the status this PR, but i guess that's ok. so just tell me when you think you're done with the salt part, and i'll run the tests again with both PR's


Receiving a fair amount of unsuccessful requests

not sure why but overall I seem to only be able to upload nine or so images in a promise.all and then it starts to get flaky and return 400 messages from Imgur, any ideas?

@CMarshall92 Thanks for the issue!
I didn't notice the GitHub notification and was late replying to it.
Oh, I see.
This error message is returned when invalid file is received by "Imgur".
Is it possible that you are uploading an image file (png, jpge, jpg) that is not a real image?
For example, if the uploaded image file is different from the entity, it may return 400.
I was able to reproduce the same error by uploading a text file with the "jpg" extension.
I thought it would be better to add a process to determine if a file is valid before it is uploaded, so that the cause of the error can be determined.
I will manage that in a separate issue.

@CMarshall92 Today I found the "imgur" help page on uploading.
What files can I upload? What is the size limit? - imgur help
If your uploaded image files are valid, maybe you have exceeded the maximum size of the file.
Also, there seems to be an upload limit of 50 images per hour.

This issue has not been updated for a long time and will be closed.
If you have any more problems, please reopen the issue.
Thank you.


difference between this and express-hbs ?

Sorry for the obvious question - what's the difference between this and https://github.com/barc/express-hbs?

I don't know. This is a different module.

Looks to me like they just copied the code here and repackaged it.


how to find member nationality

is there any option to find members nationality?

@milad145 yes. msg.from.language_code.

@GingerPlusPlus
hi dear GingerPlusPlus,
language_code comes wrong,
when you create a telegram account , telegram ask your country in the first to input phone number,
and i want to get than country
is it possible?

@milad145 Afaik, language_code is based on the device someone is using. I noticed my language_code changing when using different device. I know no way of knowing country of phone number. Assume that language_code is correct, but give your users opportunity to override it if it's not.

thanks

Hi. please send for me your telegram ID.(@youseftear1)
On Tuesday, August 8, 2017 10:54 AM, milad145<EMAIL_ADDRESS>wrote:

Closed #395.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@GingerPlusPlus
hi,
sorry again,
can you look this issue
#394


Remove incorrect grid attributes

Met Office specific grid attributes should be inherited from the target grid after regridding.  If not present on the target grid, these attributes should be removed as they are no longer correct.  This PR removes incorrect grid attributes as part of the regridding algorithm.
Testing:

[x] Ran tests and they passed OK
[x] Added new tests for the new feature(s)

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

See the complete overview on Codacy

@cgsandford this is currently failing pycodestyles so probably needs a bit of work

@fionaRust pycodestyle was being unreasonable in this instance!  I asked Ben F about this and he was happy for me to skip the error, so this now checks out.


JobSet v0.3.2 patch release

Fixes to be included in patch release:

[X] #403
[x] #393
[x] #395
[x] #404
[x] #409

Maybe we need to add https://github.com/kubernetes-sigs/jobset/pull/403 also

Maybe we need to add #403 also
The PRs are all failing for the same reason. So thinking this PR could help get the CI pass again.

Good catch, thanks! Will cherry pick that one first.

Manually tested release-0.3 branch with cherry picks and confirmed things work as expected:

foreground deletion results in faster failure recovery time
validation in #409 work

Before making the patch release, I am tempted to add some conditional logic to the validation logic added in #409, to return a custom, clear error if the generated pod name is longer than 63 chars, rather than the default error message (i.e., if the error returned by "IsDNS1035Label" contains "must be no more than 63 characters" then return a custom error message, otherwise return the error returned by the validation function).
Current error message example:
Error from server (Forbidden): error when creating "examples/2x2x2/danielvm-maxtext.yaml": admission webhook "vjobset.kb.io" denied the request: must be no more than 63 characters (notice it doesn't mention WHAT is longer than 63 characters, which may confuse the user.
Proposed error message: Error from server (Forbidden): error when creating "examples/2x2x2/danielvm-maxtext.yaml": admission webhook "vjobset.kb.io" denied the request: JobSet name must be shorter; pod names generated for this JobSet will exceed 63 characters.
What do you think? @ahg-g @kannon92

+1 to making the error message clearer.
Please also revisit the messages that we return when failing validation for follower pods because of not yet scheduled leader, perhaps we can make those clearer too.

+1 to making the error message clearer.
Please also revisit the messages that we return when failing validation for follower pods because of not yet scheduled leader, perhaps we can make those clearer too.

Yeah those seem to confuse users, I updated the error message to include the leader pod name and changed it to INFO instead of ERROR so users don't think something is wrong: https://github.com/kubernetes-sigs/jobset/pull/421

Published patch release v0.3.2 today https://github.com/kubernetes-sigs/jobset/releases/tag/v0.3.2


[T2A3][T14-A2]Tan Min Rong Samson

we hope you can appreciate the need for good SLAP after this "inlining" exercise


[BUG] - repeated certificates stripped from Conway transaction

Internal/External
Internal
Area
Other   Any other topic (Delegation, Ranking, ...).
Summary
In Babbage transaction, the order of certificates is retained, even when the certificates are repeated. I can eg. build a tx that contain following certs in this order

stake address registration cert
deregistration cert for the stake address above
stake address registration cert from 1.
deregistration cert from 2.
stake address registration cert from 1.

The transaction can be submitted and the stake address is registered.
Certs in Babbage tx:
    "certificates": [
        {
            "stake address registration": {
                "keyHash": "e24c2f4946e87eae96ace9c362b3c6a0d40a0c3a04592a591824180a"
            }
        },
        {
            "stake address deregistration": {
                "keyHash": "e24c2f4946e87eae96ace9c362b3c6a0d40a0c3a04592a591824180a"
            }
        },
        {
            "stake address registration": {
                "keyHash": "e24c2f4946e87eae96ace9c362b3c6a0d40a0c3a04592a591824180a"
            }
        },
        {
            "stake address deregistration": {
                "keyHash": "e24c2f4946e87eae96ace9c362b3c6a0d40a0c3a04592a591824180a"
            }
        },
        {
            "stake address registration": {
                "keyHash": "e24c2f4946e87eae96ace9c362b3c6a0d40a0c3a04592a591824180a"
            }
        }
    ],

However in Conway tx, the repeated certs are stripped. Therefore when submitting the tx with certificates order listed above, the stake address is not registered.
Certs in Conway tx:
    "certificates": [
        {
            "Stake address registration": {
                "stake credential": {
                    "keyHash": "4f2792103fe76cb96a0c5fb89dcd3f36441a2f9c697dc783ed915039"
                }
            }
        },
        {
            "Stake address deregistration": {
                "stake credential": {
                    "keyHash": "4f2792103fe76cb96a0c5fb89dcd3f36441a2f9c697dc783ed915039"
                }
            }
        }
    ],

Steps to reproduce
Steps to reproduce the behavior:

build a tx

cardano-cli conway transaction build --tx-in "57e67b2775839788da91936645f98caac34a157a2f01dfc3e5334659160758cd#0" --certificate-file "test_addr_registration_certificate_order[build-submit_cli]_ci0_ykz_addr0_stake_reg.cert" --certificate-file "test_addr_registration_certificate_order[build-submit_cli]_ci0_ykz_addr0_stake_dereg.cert" --certificate-file "test_addr_registration_certificate_order[build-submit_cli]_ci0_ykz_addr0_stake_reg.cert" --certificate-file "test_addr_registration_certificate_order[build-submit_cli]_ci0_ykz_addr0_stake_dereg.cert" --certificate-file "test_addr_registration_certificate_order[build-submit_cli]_ci0_ykz_addr0_stake_reg.cert" --change-address addr_test1qz8el0prsndp9y00tvv4aykmjhrteg450e3f72v6n8rxl6ymn7p2shwt24atpgw7fzshqdjjl7c9cfr3skrkjq5n098sf5dk3h --witness-override 2 --out-file "test_addr_registration_certificate_order[build-submit_cli]_ci0_ykz_tx.body" --testnet-magic 42

sign and submit the tx, see that the stake address was not registered
View the tx, see that repeated certificates were stripped

Expected behavior
A clear and concise description of what you expected to happen.
System info (please complete the following information):

OS Name: Fedora 37
Node / CLI version:

cardano-node 8.9.0 - linux-x86_64 - ghc-8.10
git rev 0d98405a60d57e1c8e13406d51cce0e34356bd64

Screenshots and attachments
Conway and Babbage transactions: tx_certificates_issue.tar.gz

This is a dup of https://github.com/IntersectMBO/cardano-cli/issues/645

The representation of certificates in ledger has changed between Babbage and Conway. In Babbage it's a StrictSeq allowing for duplicate certificates to be stored in the transaction body. In Conway it is OSet which stores only unique certificates. This needs

This needs to be fixed in ledger.

The change to OSet is the fix in ledger, because duplicate certificates in the same transaction are not supported.
Hopefully we'll have enough time in ledger to change for Conway the functionality of "being stripped" to "deserialization failure". This is conceptually a pretty hard task due to annotator, but we are aware of it and have some plans on how to tackle it