ChatGLM2 Support

An implementation of ChatGLM 2 based on vLLM. This implementation adapts PagedAttentionWithRoPE and ParallelLinear layers for model inference.

@GoHomeToMacDonal  I clone your repo and pip install -e . lanch a test.py
from vllm import LLM, SamplingParams

def build_prompt(prompt):
    return "[Round 1]\n\n问：{}\n\n答：".format(prompt)

content = "你好"
prompts = [
    content,
    "晚上睡不着应该怎么办"
]
prompts = [build_prompt(item) for item in prompts]
# edit this model_url
model_url = '/chatglm2-6b'

sampling_params = SamplingParams(temperature=0.8, top_p=0.95, max_tokens=8192)
llm = LLM(model=model_url, gpu_memory_utilization=0.98, dtype="float16", trust_remote_code=True)
outputs = llm.generate(prompts, sampling_params)

# Print the outputs.
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

get the error:
Traceback (most recent call last):
  File "/data1/nfs15/nfs/bigdata/zhanglei/ai-platform/faster-infer/model-serving/pkg/scripts/test_chatglm.py", line 16, in <module>
    llm = LLM(model=model_url, gpu_memory_utilization=0.98, dtype="float16", trust_remote_code=True)
  File "/data1/nfs15/nfs/bigdata/zhanglei/ai-platform/faster-infer/model-serving/pkg/chatglm2-vllm/vllm/entrypoints/llm.py", line 93, in __init__
    self.llm_engine = LLMEngine.from_engine_args(engine_args)
  File "/data1/nfs15/nfs/bigdata/zhanglei/ai-platform/faster-infer/model-serving/pkg/chatglm2-vllm/vllm/engine/llm_engine.py", line 231, in from_engine_args
    engine = cls(*engine_configs,
  File "/data1/nfs15/nfs/bigdata/zhanglei/ai-platform/faster-infer/model-serving/pkg/chatglm2-vllm/vllm/engine/llm_engine.py", line 96, in __init__
    self._verify_args()
  File "/data1/nfs15/nfs/bigdata/zhanglei/ai-platform/faster-infer/model-serving/pkg/chatglm2-vllm/vllm/engine/llm_engine.py", line 187, in _verify_args
    self.model_config.verify_with_parallel_config(self.parallel_config)
  File "/data1/nfs15/nfs/bigdata/zhanglei/ai-platform/faster-infer/model-serving/pkg/chatglm2-vllm/vllm/config.py", line 128, in verify_with_parallel_config
    total_num_hidden_layers = self.hf_config.num_hidden_layers
  File "/data1/nfs15/nfs/bigdata/zhanglei/conda/envs/ke-vllm/lib/python3.10/site-packages/transformers/configuration_utils.py", line 261, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'ChatGLMConfig' object has no attribute 'num_hidden_layers'

is need a special transformers version ?

Thank you for the contribution, unfortunately this PR seems to have some merge conflict and ChatGLM3 also came out. Feel free to coordinate the contribution here if you have bandwidth!
https://github.com/vllm-project/vllm/issues/1552

@simon-mo Hi, we have resolved the code conflict, and it can be directly merged into the main branch.
As chatglm3 does not change the model structure, this implementation can be directly adopted to chatglm3. Below is the testing code:
from vllm import LLM, SamplingParams

prompts = ["""<|system|>
You are ChatGLM3, a large language model trained by Zhipu.AI. Follow the user's instructions carefully. Respond using markdown.
<|user|>
Hello
<|assistant|>
"""]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

llm = LLM(model="/home/skim/.cache/modelscope/hub/ZhipuAI/chatglm3-6b", trust_remote_code=True)

outputs = llm.generate(prompts, sampling_params)

# Print the outputs.
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

The output will be Hello! How can I assist you today?

hi ,can u tell me how to use it , I still have this problem: AttributeError: 'ChatGLMConfig' object has no attribute 'num_hidden_layers', Currently, I have updated to the latest version of VLLM

This is problem is caused by old version of transformers. I suggest upgrading both your transformers package and ChatGLM model to the recent versions.

This is problem is caused by old version of transformers. I suggest upgrading both your transformers package and ChatGLM model to the recent versions.

Yes, I have tried，transformers==4.35.0

This is problem is caused by old version of transformers. I suggest upgrading both your transformers package and ChatGLM model to the recent versions.

Yes, I have tried，transformers==4.35.0

Please provide more information of installed packages, and I will try to reproduce your problem later.

@GoHomeToMacDonal If you use other prompts, it shows big difference between the native ones.. Seems this PR is not that reliable.

@GoHomeToMacDonal If you use other prompts, it shows big difference between the native model.. Did you try more examples?
this is one example, seems it stopped after meeting some token

I guess you used the default max_tokens=16 in the sampling parameters. I suggest to set max_tokens to a larger value, e.g., 1024. For more details, please refer to vllm/sampling_params.py.
In addition, as ChatGLM3 added some special tokens, e.g., <|system|>, use ChatGLMTokenizer.build_chat_input build the input token ids and feed them into vLLM will generate more stable results.

@GoHomeToMacDonal If you use other prompts, it shows big difference between the native model.. Did you try more examples?
this is one example, seems it stopped after meeting some token

I guess you used the default max_tokens=16 in the sampling parameters. I suggest to set max_tokens to a larger value, e.g., 1024. For more details, please refer to vllm/sampling_params.py.
In addition, as ChatGLM3 added some special tokens, e.g., <|system|>, use ChatGLMTokenizer.build_chat_input build the input token ids and feed them into vLLM will generate more stable results.

@GoHomeToMacDonal It is the max_tokens setting issue. adding max_tokens works as expected. It's my first time using openai wrapper, thanks for the advice. BTW, I plan to use https://github.com/lm-sys/FastChat/pull/2622 to build the conv template, I did some test and the result looks equivalent to what ChatGLMTokenizer.build_chat_input generates.

build the input token ids and feed them into vLLM will generate more stable results.
Just curious, does vllm provides the token interface?

Hello, I am using ChatGLM2 but it seems sometimes the output is not aligned with huggingface version. Could anyone help to take a look at #1670 ?

The latest version already supports GLM. Can GLM3 support official tool calls and other functions? Does it support dialogue function?

The latest version already supports GLM. Can GLM3 support official tool calls and other functions? Does it support the dialogue function?

You need to implement the corresponding code for function calls and prompt building. The vllm library focuses on model inference, i.e., it can substitute ChatGLMForConditionalGeneration.generate to llm.generate. I suggest building prompts based on the official ChatGLM 3 repository, and replacing model inference functions, e.g., ChatGLMForConditionalGeneration.chat, ChatGLMForConditionalGeneration.stream_chat with vLLM.

@GoHomeToMacDonal This is still not supported for the Chatglm2-6b-32k version ,I have a message for the issue #1725


Markdown slow parsing

VSCode Version: Insiders
OS Version: macOS

Steps to Reproduce:
It took Marked.js 2 seconds to parse below content. I use spaces everywhere so no tabs.
"node              node
/  \              /  \
a   b    <----   a    b
                    /
                   z"

Does this issue occur when all extensions are disabled?: Yes

Fixed by c6b1114


Expose catalogue questionnaire license

Now that https://github.com/impactasaurus/soq-api/issues/8 is complete, add the license attribute to the data exposed for catalogue questionnaires

 Library of standard question sets


feat: no_std compat ✨

Add no_std compatibility
Pull Request type
Please check the type of change your PR introduces:

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Testing
[ ] Other (please describe):

What is the current behavior?
Doesn't compile in a no_std envirronement.
Issue Number: 7
What is the new behavior?
Works in a no_std envirronement.
Does this introduce a breaking change?

[ ] Yes
[x] No

All good. However 2 small things would need to be changed for the CI to pass :

Use rust nightly mode in order to use feature(alloc_error_handler) in lib.rs
Run tests with cargo test --lib test, see why

Let me know if that would be fine @abdelhamidbakhta


Escaping of ghc-options with spaces

Consider the following cabal file:
name:           foo
version:        <IP_ADDRESS>
build-type:     Simple
cabal-version:  >= 1.10

executable foo
  main-is: Main.hs
  build-depends: base
  default-language: Haskell2010
  ghc-options: -threaded -O2 -rtsopts "-with-rtsopts=-N -T"

Using hpack-convert, I end up with:
name: foo
version: '<IP_ADDRESS>'
license: UnspecifiedLicense
ghc-options:
- -threaded
- -O2
- -rtsopts
- -with-rtsopts=-N -T
dependencies:
- base
executables:
  foo:
    main: Main.hs

Which results in the following in the generated cabal file:
  ghc-options: -threaded -O2 -rtsopts -with-rtsopts=-N -T

Unfortunately, this does not build:
ghc: unrecognised flag: -T
did you mean one of:
  -F
  -v
  -j

I can work around this by putting the following in my package.yaml instead of the list of options:
ghc-options: -threaded -O2 -rtsopts "-with-rtsopts=-N -T"

However, it would be nice to keep the list approach, and have hpack detect when there's a space in an option and automatically wrap it in quotes.

Sounds a lot like #182. I think fixing this in hpack (#238) makes more sense than requiring people to muck around with YAML syntax and GHC argument parsing.

@snoyberg thanks for opening this issue.

I think fixing this in hpack (#238) makes more sense than requiring people to muck around with YAML syntax and GHC argument parsing.

Yes, I agree.  If there is a way to address this in hpack I'm all for it.  However, just quoting everything may do more harm than good.  So let's first collect all the facts.
Specifically:

Is -with-rtsopts the only affected GHC option?

Currently, the following is valid in package.yaml:
ghc-options: -Wall -Werror

If we quote all options that contain spaces the above will become invalid.

We could consider to only quote options that were given as a YAML list, e.g.
ghc-options:
  - -with-rtsopts=-N -T

we would rendered as `ghc-options: "-with-rtsopts=-N -T", while
ghc-options: -Wall -Werror

we would continue to render as ghc-options: -Wall -Werror.
This is easy to do, however, we would loose the property that a scalar and a singleton list have the same semantics.

Could #217 help with this?  Say we only quote when option were given as a hash, e.g.
ghc-options:
  with-rtsopts: -N -T

If this whole story is really only relevant for -with-rtsopts, we could consider to introduce a separate field for that, e.g.
ghc-options: -Wall
rts-options: -N -T

would render to ghc-options: -Wall "-with-rtsopts=-N -T".

I'm only aware of it affecting with-rtsopts, but I have no idea if that's really the only one. I had been thinking of this going in the direction of (3), with different behavior for scalars and lists. This seems to be a pretty common pattern in config files (Docker does similar things), and even mirrors the behavior of some similar API calls in Haskell (e.g., the difference between proc and shell in the process package).
All this said, I have no strong opinions at all on the outcome here, and definitely don't want to put any pressure on a fix: my current workaround is working fine for me for now.

I've only seen this problem with -with-rtsopts=..., but GHC has so many options it's hard to say. Something like -optF=... could also be a problem?
I think the cases of a single string and an array of strings should be handled separately as far as this is concerned. A single string means literally "pass this as-is to GHC". An array means "each of this is an atomic option".
Yup, I agree.
If hashes are implemented for this, I imagine they would behave a lot like lists. It might be a little tricky because #182 notes that "-with-rtsopts=-N -T" is fine but -with-rtsopts="-N -T" doesn't work. So only quoting the value is a no-go for this particular case.
I'm weakly -1 on adding a custom options for -with-rtsopts.

Semi-related: I've mostly given up on specify -with-rtsopts in my package descriptions. I set -threaded and -rtsopts; if users want to set +RTS -N, they can, but I'll stick with GHC's defaults.

As for (3), seems that this won't work neither.  Let's consider:
ghc-options:
  - -main-is Foo.run

If we now quote this and generate ghc-options: "-main-is Foo.run" that won't be accepted by GHC.
At this point I'm tempted to consider special casing -with-rtsopt, either with magic or by emitting a warning if we see -with-rtsopt unquoted.

As I understand it, all of the following are valid ways of specifying -with-rtsopts in a cabal file:
a) ghc-options: "-with-rtsopts -N -T"
b) ghc-options: "-with-rtsopts=-N -T"
c) ghc-options: -with-rtsopts "-N -T"
d) ghc-options: -with-rtsopts=-N
e) ghc-options: -with-rtsopts -N

In package.yaml c), d) and e) work as well. a) and b) require double quoting.
ghc-options: "-with-rtsopts -N -T" # invalid
ghc-options: "-with-rtsopts=-N -T" # invalid

# needs double quoting
ghc-options: '"-with-rtsopts -N -T"' # valid
ghc-options: '"-with-rtsopts=-N -T"' # valid

One form of magic we could introduce is

When a GHC option starts with -with-rtsopts we quote it when rendering, e.g.

ghc-options: -with-rtsopts -N -T # valid

Which has the unfortunate downside that some currently valid package specifications will become invalid, e.g.
ghc-options: -with-rtsopts -N -Wall # currently valid, but invalid if we introduce that magic behavior

We could try to mitigate this by only applying magic when GHC options are given as a list, e.g.
ghc-options: -with-rtsopts -N -T # invalid

ghc-options:
 - -with-rtsopts -N -T # valid

ghc-options: -with-rtsopts -N -Wall # valid

ghc-options:
 - -with-rtsopts -N -Wall # invalid

Not very excited!  Opinions?

Hmm, we're in a weird situation here because the YAML array syntax implies "each of these is exactly one argument" but stuff like -main-is Main.hs breaks that. If we really followed that guideline, we'd end up with this:
ghc-options:
  - -main-is
  - Main.hs
  - -with-rtsopts
  - -N -T

And then quoting arguments with spaces in them would be no problem.
I'm not sure what the right way forward is. At this point, I don't mind quoting and dealing with YAML syntax oddities. I think I might prefer it to magic, but I don't hold that opinion very strongly.

I'd like to +1 predictability and going with YAML syntax here. Too many corner cases otherwise and I'm sure one would turn around and bite us eventually. One direct change to make here would be to add a note to the docs about this case, and how to properly escape the input (which, unless I'm misreading, can be achieved today, even if it's ugly?).
Maybe the more hpackish way forward would be to reduce the need to directly deal with options-as-strings at all by adding structure? Something like this:
ghc-options:
  main-is: Main.hs
  threaded: true
  optimisation: 2
  warnings:
    - all
    - error
    - no-name-shadowing
  rts-options:
    threads: true
    collect-statistics: true

This is a bit like #217, but hpack will learn about all these weird and wonderful GHC options and how to Do The Right Thing for them, rather than being a generic mechanism. ghc-options should still be specifyable as a list of strings as an escape hatch, too.

Ok, to sum things up, as I understand it non of us is excited about introducing any magic behavior here, so let's exclude that.  What we are left with are the following options:

Close as wontfix
Add a dedicated rts-options field
Add hash syntax for GHC options

Even though (3) is nice, I would want to exclude it as well for the following reasons:

first and foremost, it's a non-trivial effort to implement and maintain; even if somebody else does the initial implementation I'm not ready to take responsibility to maintain it in the future; if there is strong demand we could revisit once we are at zero open issues and PRs
and secondly a more minor point, it does not allow to copy & paste options from the command line into package.yaml and vice versa

I'm ok with (1) but the fact that we already have more that one issue on this clearly indicates that this is a stumbling block.  So maybe we should do something about it.
(2) would look like this, we add support for an rts-options field which we know how to quote correctly.  In addition we check if ghc-options contains the substring -with-rtsopts and if so, we aggressively warn and suggest to the user to specify them as rts-options: ... instead.  A warning could look like this:
WARNING: The field "ghc-options" specifies "-with-rtsopts".  This is fragile and hence not
recommended. Please consider specifying them in the field "rts-options" instead.

For more information see: <URL to the relevant section in the README>

To silence this warning do ....

Concerns regarding (2):

Is it too invasive, especially for existing package configurations?
In theory, -with-rtsopts could be part of some valid GHC option (e.g. it could be the name of a custom preprocessor or unlit).  If that happens the user would get a warning even though nothing is wrong.  I think it's very very unlikely that this will lead to issues in practice, and we can mitigate it further by taking more context into account when doing the substring matching (say make sure that we only have one of  , ", = at the word boundaries).  In addition, I guess some way to silence the warning would be a good idea.  But still, I wanted to mention it.

Before making a final call on either (1) or (2) I would be deeply grateful for an other round of feedback from you guys.
Thanks and sorry for the lengthiness!

I am happy with how things are now, but I think option 2 (a dedicated field for RTS options) is the best way forward to prevent confusion. It would give first class support to RTS options and keep ghc-options as an escape hatch. My only concern is that this might lead to an explosion of new fields (as @quasicomputational hinted at). That's not necessarily a problem, though. It's just more stuff to code, test, document, and maintain.


compile html5

Describe your problem here.

lime test html5 -release
source/CreditsState.hx:15: characters 8-22 : You cannot access the sys package while targeting js (for sys.FileSystem)
source/MainMenuState.hx:220: characters 42-54 :     referenced here

pause
Press any key to continue . . .

Are you modding a build from source or with Lua?
Source
What is your build target?
HTML5
Did you edit anything in this build? If so, mention or summarize your changes.
No response

Fixed

Im having this bug, here is the error.
source/WeekData.hx:11: characters 8-19 : You cannot access the sys package while targeting js (for sys.io.File)
source/editors/WeekEditorState.hx:34: characters 8-16 : ... referenced here
source/editors/MasterEditorMenu.hx:119: characters 37-52 : ... referenced here


msgpackrpc.error

Hi Aqeel,
I'm getting this error after a few hundred iterations. Hitting r and backspace doesn't seem to resolve it.
I've attached the setting.json file with this.
Cheers

settings.txt

This usually happens when the connection between the Unreal Engine and AirSim is compromised. The python code tries to fetch the RGB frame or Depth map, and is not able to because of poor connectivity. I did take into account a few of the error catch to have a work around for this problem and it seemed to be working effectively.
How often do you get this error? also, can you share the results of the following print command?
https://github.com/aqeelanwar/PEDRA/blob/eb4e69aa3650ea4a96ee89104007ceecf7fb715f/algorithms/DeepQLearning.py#L451-L452

It happens every time I run the sim,
Here is the output

Incorporate the print commands in the python code. I have shared the link to the file above that will show you the placement of these commands in the DeepQLearning.py file.
Once you have incorporated the command, share the results of those print commands.

My existing DeepQLearning.py file seems to have these print commands already present.
On a side note, do you mean run DeepQLearning.py by starting the simulation using "python main.py" which then fetches the DeepQLearning file?, Sorry, I'm relatively new to python so don't understand these things sometimes..

No need to apologize.
Yes, try running the simulation through main.py. Those try and catch statement will print the error type as to what caused the problem. The error message will help us better understand the problem and how to solve it.

Thanks Aqeel,
So I haven't had a repeat of the problem overnight, Simulation has passed 300k iterations with no issues. I suspect the problem could've been caused by my network card going to sleep!?. I wasn't able to change the power settings in windows so I ran a parallel steam download overnight to see if keeping the card in active use helped, which it has seemed to.
300k iterations and not error yet

heres the time out error I got after 500k iterations and after incorporating the print command.

On a side note - Can the engine detect when the drone has successfully flown the env with crashing and save that model? Am i then able to load this pretrained model into another env? Lastly in inference mode, I'm assuming the control inputs are saved, how do I load them into "training mode"?
Thanks in advance, and for helping me out this far :)  🥇

It depends on the nature of the algorithm you are trying to run. If you are running REINFORCE algorithm then every time the enviroment crashes you can load the previously saved model to begin the simulation again.
What do you mean by control inputs? The controls are output of the deep network in the inference mode. Based on what training learned, the deep network predicts the best output.

This usually happens when the connection between the Unreal Engine and AirSim is compromised. The python code tries to fetch the RGB frame or Depth map, and is not able to because of poor connectivity. I did take into account a few of the error catch to have a work around for this problem and it seemed to be working effectively.
How often do you get this error? also, can you share the results of the following print command?
https://github.com/aqeelanwar/PEDRA/blob/eb4e69aa3650ea4a96ee89104007ceecf7fb715f/algorithms/DeepQLearning.py#L451-L452

Same error but ı have no 452 lines of code in my DeepQLearning.py file.


Please consider fixing issues detected by repository checker

Notification from ioBroker Check and Service Bot
Dear adapter developer,
I'm the ioBroker Check and Service Bot. I'm an automated tool processing routine tasks for the ioBroker infrastructure. I have recently checked the repository for your adapter bosch-ebike for common errors and appropiate suggestions to keep this adapter up to date.
This check is based the current head revisions (master / main  branch) of the adapter repository
Please see the result of the check below.
ioBroker.bosch-ebike
   -
ERRORS:

[ ] :heavy_exclamation_mark: [E029] Node.js 16 is required as minimum, node.js 18 is recommended. Please adapt "{'engines' : { 'node' >= '14.0.0' } }" at package.json.
[ ] :heavy_exclamation_mark: [E033] @iobroker/adapter-core 3.0.6 specified. 3.1.4 is required as minimum, 3.1.6 is recommended. Please update dependencies at package.json
[ ] :heavy_exclamation_mark: [E036] @iobroker/testing 4.1.1 specified. 4.1.3 is required as minimum,  4.1.3 is recommended. Please update devDependencies at package.json
[ ] :heavy_exclamation_mark: [E162] js-controller 3.2.0 listed as dependency but 4.0.24 is required as minimum, 5.0.19 is recommended. Please update dependency at io-package.json.
[ ] :heavy_exclamation_mark: [E405] Icon must be in the following path: https://raw.githubusercontent.com/TA2k/ioBroker.bosch-ebike/main/; correct sources-dist.json
[ ] :heavy_exclamation_mark: [E407] Meta URL must be equal to https://raw.githubusercontent.com/TA2k/ioBroker.bosch-ebike/main/io-package.json; correct sources-dist.json
[ ] :heavy_exclamation_mark: [E605] No actual year found in copyright. Please add "Copyright (c) 2024 TA2k<EMAIL_ADDRESS>at the end of README.md
[ ] :heavy_exclamation_mark: [E606] Current adapter version 0.1.7 not found in README.md
[ ] :heavy_exclamation_mark: [E701] No actual year found in LICENSE. Please add "Copyright (c) 2024 TA2k<EMAIL_ADDRESS>at the start of LICENSE
[ ] :heavy_exclamation_mark: [E145] Missing mandatory translation into de of some "common.news" in io-package.json.

WARNINGS:

[ ] :eyes: [W040] "keywords" within package.json should contain "ioBroker"
[ ] :eyes: [W105] Missing suggested translation into uk of "common.titleLang" in io-package.json.
[ ] :eyes: [W109] Missing suggested translation into uk of "common.desc" in io-package.json.
[ ] :eyes: [W115] "common.tier" is required in io-package.json. Please check https://github.com/ioBroker/ioBroker.docs/blob/master/docs/en/dev/objectsschema.md#adapter.
[ ] :eyes: [W145] Missing suggested translation into ru,pt,nl,fr,it,es,pl,uk,zh-cn of some "common.news" in io-package.json.
[ ] :eyes: [W181] "common.license" in io-package.json is deprecated. Please define object "common.licenseInformation"
[ ] :eyes: [W184] "common.main" is deprecated and ignored. Please remove from io-package.json. Use "main" at package.json instead.
[ ] :eyes: [W184] "common.materialize" is deprecated for admin >= 5 at io-package.json. Please use property "adminUI".
[ ] :eyes: [W515] Why did you decide to disable i18n support?

:thumbsup: No suggestions found
Please review issues reported and consider fixing them as soon as appropiate.
Errors reported by repository checker should be fixed as soon as possible. Some of them require a new release to be considered as fixed. Please note that errors reported by checker might be considered as blocking point for future updates at stable repository.
Warnings reported by repository checker should be reviewed. While some warnings can be ignored due to good reasons or a dedicated decision of the developer, most warnings should be fixed as soon as appropiate.
You may start a new check or force the creation of a new issue at any time by adding the following comment to this issue:
@iobroker-bot recheck
@iobroker-bot recreate
Please note that I (and the server at GitHub) have always plenty of work to do. So it may last up to 30 minutes until you see a reaction. I will drop a comment here as soon as I start processing.
Feel free to contact me (@iobroker-bot) if you have any questions or feel that an issue is incorrectly flagged.
And THANKS A LOT for maintaining this adapter from me and all users.
Let's work together for the best user experience.
your
ioBroker Check and Service Bot
@mcm1957 for evidence
Last update at Wed, 11 Sep 2024 03:49:59 GMT

Issue closed due to RECREATE request. Folloow up issue #22 has been created.
This issue can be closed.
your
ioBroker Check and Service Bot


Handling actions on classes.

I'm using mobx-sync in a React application to persist state for an offline mode. One issue I am facing is that when persisting a store that has an array of objects where those objects have actions for modifying their state, when initialized from storage the JS objects in the store know nothing about the methods that were on the object when it was first added. I can see what, and why the problem is occurring, but at a loss to what would be the best way to address it.
So if I have a ProfileStore that contains a set of Options and ProfileStore has actions to add an option, that is fine. But if Option has an action to update state, after loading that store and it's Options, the loaded Option throws "... is not a function" exceptions. Obviously the array of Option state is not "typed" to the Option class that is defined. I'm left wondering if there is a way to deserialize them in a way that is recognized as that type, or a best practice to alter the structure so that the actions don't rely on the state. I would like options to live under a profile store rather than trying to juggle separate stores for loosely coupled objects.

You can use custom formatter to stringify/parse your set of Options. For example:
/*!
 * Copyright 2019 yangjunbao<EMAIL_ADDRESS>All rights reserved.
 * @since 2019-05-22 10:20:06
 */

import { format, parseStore } from 'mobx-sync';

class ProfileStore {
  @format(
    (data: Array<Pick<Option, 'label' | 'value'>>) =>
      new Set(data.map((item) => new Option(item))),
    (value: Set<Option>) => Array.from(value),
  )
  options = new Set<Option>();
}

class Option {
  label: string;
  value: string;

  constructor(config: Pick<Option, 'label' | 'value'>) {
    this.label = config.label;
    this.value = config.value;
  }

  someAction() {
    return 'hello world';
  }
}

const profile = new ProfileStore();

profile.options.add(new Option({ label: 'label', value: 'value' }));

const stateValue = JSON.stringify(profile);

console.log(stateValue);

const newProfile = new ProfileStore();

parseStore(newProfile, JSON.parse(stateValue), false);

console.log(
  newProfile.options.size,
  newProfile.options
    .values()
    .next()
    .value!.someAction(),
);

The upon code will output the following stdout:
{"options":[{"label":"label","value":"value"}]}
1 'hello world'

Oh man, I feel silly, I hadn't seen that in the docs when I'd just been in there around the async vs. sync behavior. I'd been bouncing between Mobx and sync docs. Thanks! My knowledge of OO as it is applied (and re-applied) in JavaScript is unreliable at best.  I don't recognize them when they slap me in the face.  :)


Migrate server module tests to JUnit 5, part 7

This PR continues migration of the server module to the new JUnit 5 test structure.
The following packages were migrated:

stand
storage

@armiol PTAL.


(doc) change to os.environ.get

super simple doc change to fix accidental use of os.environ as a method

Please change this in the README file, not just the docs folder. README is the One True Doc.

Coverage remained the same at 80.13% when pulling 13f7db8405e268f5b3f7c665812b19b7f4ee0be8 on scoates:fix-env-get-doc into 65aa468fa925c0d90157b84dffe5159325b196fc on Miserlou:dev.

Done. Didn't even notice it there. Good catch.

Coverage remained the same at 80.13% when pulling 19b6880d576ef97c2ab1cca5e27889fce051b53c on scoates:fix-env-get-doc into 65aa468fa925c0d90157b84dffe5159325b196fc on Miserlou:dev.


Inconsistancy of triggering change event on the very first init across components.

For example, in MdRadioGroup, it only fires a change event if this isn't the first time the value is ever set. Same applies to TabGroup
However in MdButtonToggleGroup, it fires a change event even if it is the first time. Same applies to SlideToggle

If a component fires a change event upon initialization it is a bug.

I just took a look at it for the md-slide-toggle - I was not able to recognize any events at initialization.
Is it possible that you use an older version of Material 2?

change is emitted before click on the MdButtonToggle
http://plnkr.co/edit/LiCpyMHS1Ok98ZVWvoXs?p=preview

@colinfang I did not reference that issue to be closed, because I thought there are also issues with the TabGroup?

Just tested the selectChange and focusChange output and everything seemed to work as expected.


Unable to access "resources" file in lab setup; also getting error in step 3-3. Verify Data Pipeline.

Unable to access "resources" file in lab setup
Necessary files as per requirements :

Missing "resources" file while actual deployment :

Getting error while running below command
Command :
python3 gen_kinesis_data.py --region-name us-west-2 --service-name kinesis --stream-name retail-trans

Error :
[ec2-user@ip-10-0-5-70 ~]$ python3 gen_kinesis_data.py --region-name us-west-2 --service-name kinesis --stream-name retail-trans
Traceback (most recent call last):
File "gen_kinesis_data.py", line 31, in
class CustomDatetimeProvider(BaseProvider):
File "gen_kinesis_data.py", line 32, in CustomDatetimeProvider
class Meta:
File "gen_kinesis_data.py", line 34, in Meta
name: typing.Final[str] = "custom_datetime"
AttributeError: module 'typing' has no attribute 'Final'

Hi IftesamShk
Thanks for reporting errors

Unable to access "resources" file in lab setup

I updated the synthentic data generator script. So, the new script doesn't require resources any more.

Getting error while running below command: python3 gen_kinesis_data.py --region-name us-west-2 --service-name kinesis --stream-name retail-trans

I fixed the error. Let's pull new codes and try it again.

fix


v3.4.0 Bug "WARNING|2023-08-03 23:15:52.135333| sshnpd |Failed to establish connection - exit code 255"

Describe the bug
I'm only able to reproduce this bug on my VPS.
SSHNP:
./sshnp -f @client_0 -t @server_0 -h @rv_am -d vps -s id_ed25519.pub -v
...
Remote sshnpd error: Failed to establish connection - exit code 255 (use --local-port to specify unused port)

SSHNPD:
./sshnpd -a @server_0 -m @client_0 -d vps -s -u -v
...
WARNING|2023-08-03 23:15:52.135333| sshnpd |Failed to establish connection - exit code 255

Steps to reproduce

Use installer script (curl...) on Device and Client from installer guide
SSHNPD is automatically started on Device (VPS Linuxx64) because of installer script
Use SSHNP on Client (Mac Arm64)
Get the exit code 255

Expected behavior
Works just fine in my two docker containers when I test v3.4.0 as it is now
Screenshots
Client (sshnp) - Mac Arm64 M2 (Sanoma 14.0)
jeremytubongbanua@Jeremys-M2-Air bin % ls
at_activate	sshnp		sshnp@server_0	sshrv
jeremytubongbanua@Jeremys-M2-Air bin % ./sshnp
Version : 3.4.0
-k, --key-file             Sending atSign's atKeys file if not in ~/.atsign/keys/
-f, --from (mandatory)     Sending (a.k.a. client) atSign
-t, --to (mandatory)       Receiving device atSign
-d, --device               Receiving device name
                           (defaults to "default")
-h, --host (mandatory)     atSign of sshrvd daemon or FQDN/IP address to connect back to
-p, --port                 TCP port to connect back to (only required if --host specified a FQDN/IP)
                           (defaults to "22")
-l, --local-port           Reverse ssh port to listen on, on your local machine, by sshnp default finds a spare port
                           (defaults to "0")
-s, --ssh-public-key       Public key file from ~/.ssh to be appended to authorized_hosts on the remote device
                           (defaults to "false")
-o, --local-ssh-options    Add these commands to the local ssh command
-v, --[no-]verbose         More logging
-r, --[no-]rsa             Use RSA 4096 keys rather than the default ED25519 keys
-u, --remote-user-name     username to use in the ssh session on the remote host
    --config-file          Read args from a config file
                           Mandatory args are not required if already supplied in the config file

Invalid argument(s): Option from is mandatory.
jeremytubongbanua@Jeremys-M2-Air bin % ./sshnp -f @client_0 -t @server_0 -h @rv_am -d vps -s id_ed25519.pub -v
INFO|2023-08-03 19:15:48.677560|AtClientManager|setCurrentAtSign called with atSign @client_0
INFO|2023-08-03 19:15:48.677631|AtClientManager|Switching atSigns from null to @client_0
INFO|2023-08-03 19:15:48.678576|HiveBase|commit_log_d75a51c1003c996d4592b57407ea6b3f21a2e1163891d3612cb54a41aa363a01 initialized successfully
INFO|2023-08-03 19:15:48.679465|HiveBase|d75a51c1003c996d4592b57407ea6b3f21a2e1163891d3612cb54a41aa363a01 initialized successfully
INFO|2023-08-03 19:15:48.679493|AtClientCommitLogCompaction (@client_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-03 19:15:48.680152|AtClientManager|setCurrentAtSign complete
INFO|2023-08-03 19:15:48.680165|AtLookup|Creating new connection
INFO|2023-08-03 19:15:49.007991|AtLookup|New connection created OK
INFO|2023-08-03 19:15:49.148701|AtLookup|auth success
INFO|2023-08-03 19:15:49.203843| sshnp |Subscribing to notifications on 08cfec9c-45f0-4aec-a089-c630b2406d3e.vps.sshnp@
INFO|2023-08-03 19:15:49.525238|AbstractAtKeyEncryption (@client_0)|Encrypted shared symmetric key for @client_0 not found in local storage
INFO|2023-08-03 19:15:49.525294|AbstractAtKeyEncryption (@client_0)|Deleting @rv_am:shared_key@client_0 from LocalSecondary
INFO|2023-08-03 19:15:49.525454|AbstractAtKeyEncryption (@client_0)|Fetching shared symmetric key for @client_0 from atServer
INFO|2023-08-03 19:15:49.556308|Monitor (@client_0)|monitor started for @client_0 with last notification time: null
INFO|2023-08-03 19:15:49.582291|AbstractAtKeyEncryption (@client_0)|Retrieved my encrypted copy of shared symmetric key for @rv_am from atServer - saving to local storage
INFO|2023-08-03 19:15:49.590993|AbstractAtKeyEncryption (@client_0)|'Their' copy of shared symmetric key for @rv_am not found in local storage - will check atServer
INFO|2023-08-03 19:15:49.639716|AbstractAtKeyEncryption (@client_0)|Found 'their' copy of shared symmetric key for @rv_am in atServer - saving to local storage
INFO|2023-08-03 19:15:50.310675| sshnp |SUCCESS:id: c41eb13e-18de-4105-9601-faf53206d6e3 status: NotificationStatusEnum.delivered @rv_am:vps.sshrvd@client_0
INFO|2023-08-03 19:15:50.493121|AbstractAtKeyEncryption (@client_0)|Encrypted shared symmetric key for @client_0 not found in local storage
INFO|2023-08-03 19:15:50.493196|AbstractAtKeyEncryption (@client_0)|Deleting @server_0:shared_key@client_0 from LocalSecondary
INFO|2023-08-03 19:15:50.493378|AbstractAtKeyEncryption (@client_0)|Fetching shared symmetric key for @client_0 from atServer
INFO|2023-08-03 19:15:50.540406|AbstractAtKeyEncryption (@client_0)|Retrieved my encrypted copy of shared symmetric key for @server_0 from atServer - saving to local storage
INFO|2023-08-03 19:15:50.548459|AbstractAtKeyEncryption (@client_0)|'Their' copy of shared symmetric key for @server_0 not found in local storage - will check atServer
INFO|2023-08-03 19:15:50.596336|AbstractAtKeyEncryption (@client_0)|Found 'their' copy of shared symmetric key for @server_0 in atServer - saving to local storage
INFO|2023-08-03 19:15:51.264113| sshnp |SUCCESS:id: 1b3bd292-cb13-474a-b46c-a53b881d5eef status: NotificationStatusEnum.delivered
INFO|2023-08-03 19:15:51.930424| sshnp |SUCCESS:id: d1c3987c-3769-4ce4-9169-745db31c3d12 status: NotificationStatusEnum.delivered
INFO|2023-08-03 19:15:52.249759| sshnp |Received 08cfec9c-45f0-4aec-a089-c630b2406d3e notification
Remote sshnpd error: Failed to establish connection - exit code 255 (use --local-port to specify unused port)
INFO|2023-08-03 19:15:52.583145| sshnp |SUCCESS:id: 8b01a6f3-3bcd-4864-9ae8-e23868ed1b93 status: NotificationStatusEnum.delivered 57271 33141 jeremytubongbanua <IP_ADDRESS> 08cfec9c-45f0-4aec-a089-c630b2406d3e
INFO|2023-08-03 19:15:52.583218| sshnp |Tidying up files

Device (VPS Linux Arm64)
jeremyvps@debian:~/.local/bin$ ./sshnpd -a @server_0 -m @client_0 -d vps -s -u -v
INFO|2023-08-03 23:15:14.979536|AtClientManager|setCurrentAtSign called with atSign @server_0
INFO|2023-08-03 23:15:14.979701|AtClientManager|Switching atSigns from null to @server_0
INFO|2023-08-03 23:15:14.980509|HiveBase|commit_log_055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-03 23:15:14.981591|HiveBase|055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-03 23:15:14.983428|AtClientCommitLogCompaction (@server_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-03 23:15:14.984333|AtClientManager|setCurrentAtSign complete
INFO|2023-08-03 23:15:14.984371|AtLookup|Creating new connection
INFO|2023-08-03 23:15:15.446247|AtLookup|New connection created OK
INFO|2023-08-03 23:15:15.560977|AtLookup|auth success
INFO|2023-08-03 23:15:15.898065| sshnpd |Starting connectivity listener
INFO|2023-08-03 23:15:15.898354| sshnpd |Subscribing to vps\.sshnp@
INFO|2023-08-03 23:15:15.898396| sshnpd |Done
WARNING|2023-08-03 23:15:15.900662| sshnpd |connection available
INFO|2023-08-03 23:15:16.205073|Monitor (@server_0)|monitor started for @server_0 with last notification time: null
INFO|2023-08-03 23:15:50.799746| sshnpd |Received: privatekey
INFO|2023-08-03 23:15:50.800469| sshnpd |Private Key received from @client_0 notification id : 1b3bd292-cb13-474a-b46c-a53b881d5eef
INFO|2023-08-03 23:15:51.465807| sshnpd |Received: sshpublickey
INFO|2023-08-03 23:15:51.465877| sshnpd |ssh Public Key received from @client_0 notification id : d1c3987c-3769-4ce4-9169-745db31c3d12
INFO|2023-08-03 23:15:52.126821| sshnpd |Received: sshd
INFO|2023-08-03 23:15:52.126891| sshnpd |ssh callback request received from @client_0 notification id : 8b01a6f3-3bcd-4864-9ae8-e23868ed1b93
INFO|2023-08-03 23:15:52.126929| sshnpd |Starting ssh session for jeremytubongbanua to <IP_ADDRESS> on port 33141 using localhost:57271 on <IP_ADDRESS>
SHOUT|2023-08-03 23:15:52.126941| sshnpd |Starting ssh session using hostSsh (/usr/bin/ssh) from: @client_0 session: 08cfec9c-45f0-4aec-a089-c630b2406d3e
INFO|2023-08-03 23:15:52.130109| sshnpd |08cfec9c-45f0-4aec-a089-c630b2406d3e | Executing /usr/bin/ssh<EMAIL_ADDRESS>-p 33141 -i /tmp/.aeff58c0-1054-454a-8d04-a9e98d638c63 -R 57271:localhost:22 -o LogLevel=VERBOSE -t -t -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -o BatchMode=yes -o ExitOnForwardFailure=yes -o ForkAfterAuthentication=yes sleep 15
INFO|2023-08-03 23:15:52.134912| sshnpd |08cfec9c-45f0-4aec-a089-c630b2406d3e | sshStdErr | command-line: line 0: Bad configuration option: forkafterauthentication

SHOUT|2023-08-03 23:15:52.135285| sshnpd |08cfec9c-45f0-4aec-a089-c630b2406d3e | Exit code 255 from /usr/bin/ssh<EMAIL_ADDRESS>-p 33141 -i /tmp/.aeff58c0-1054-454a-8d04-a9e98d638c63 -R 57271:localhost:22 -o LogLevel=VERBOSE -t -t -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -o BatchMode=yes -o ExitOnForwardFailure=yes -o ForkAfterAuthentication=yes sleep 15
WARNING|2023-08-03 23:15:52.135333| sshnpd |Failed to establish connection - exit code 255
INFO|2023-08-03 23:15:52.763217| sshnpd |SUCCESS:id: 3c376b2a-5e69-4854-ab1e-3724d2c2c2b5 status: NotificationStatusEnum.delivered for: 08cfec9c-45f0-4aec-a089-c630b2406d3e with value: Failed to establish connection - exit code 255 (use --local-port to specify unused port)

Smartphones
Client is MacOS Sanoma 14.0 M2 (Arm Chip) MacOS Arm64
Device is an IONOS VPS running Linux x64
Were you using an atApplication when the bug was found?
No response
Additional context
When I use docker containers, everything works just fine. So this means that trunk is working fine.
I've tested v3.3.0 client and device as well..
So my VPS (as sshnpd) does not like v3.4.0 for some reason

Important detail: Device (VPS Linux Arm64) running Debian 10

The fix to this bug was running SSHNPD with the --ssh-client=pure-dart flag
jeremyvps@debian:~/.local/bin$ ./sshnpd -a @server_0 -m @client_0 -d e2e -s -u -v --ssh-client=pure-dart

Logs of it working:
SSHNP
jeremytubongbanua@Jeremys-M2-Air bin % ./sshnp -f @client_0 -t @server_0 -d e2e -s id_ed25519.pub -v -h @rv_am
INFO|2023-08-03 20:01:04.923882|AtClientManager|setCurrentAtSign called with atSign @client_0
INFO|2023-08-03 20:01:04.923946|AtClientManager|Switching atSigns from null to @client_0
INFO|2023-08-03 20:01:04.925659|HiveBase|commit_log_d75a51c1003c996d4592b57407ea6b3f21a2e1163891d3612cb54a41aa363a01 initialized successfully
INFO|2023-08-03 20:01:04.926818|HiveBase|d75a51c1003c996d4592b57407ea6b3f21a2e1163891d3612cb54a41aa363a01 initialized successfully
INFO|2023-08-03 20:01:04.926850|AtClientCommitLogCompaction (@client_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-03 20:01:04.927503|AtClientManager|setCurrentAtSign complete
INFO|2023-08-03 20:01:04.927519|AtLookup|Creating new connection
INFO|2023-08-03 20:01:05.254824|AtLookup|New connection created OK
INFO|2023-08-03 20:01:05.369083|AtLookup|auth success
INFO|2023-08-03 20:01:05.418566| sshnp |Subscribing to notifications on 5245e3a6-c8f6-4f69-92be-74ef3285387d.e2e.sshnp@
INFO|2023-08-03 20:01:05.681990|AbstractAtKeyEncryption (@client_0)|Encrypted shared symmetric key for @client_0 not found in local storage
INFO|2023-08-03 20:01:05.682043|AbstractAtKeyEncryption (@client_0)|Deleting @rv_am:shared_key@client_0 from LocalSecondary
INFO|2023-08-03 20:01:05.682291|AbstractAtKeyEncryption (@client_0)|Fetching shared symmetric key for @client_0 from atServer
INFO|2023-08-03 20:01:05.737836|AbstractAtKeyEncryption (@client_0)|Retrieved my encrypted copy of shared symmetric key for @rv_am from atServer - saving to local storage
INFO|2023-08-03 20:01:05.743224|AbstractAtKeyEncryption (@client_0)|'Their' copy of shared symmetric key for @rv_am not found in local storage - will check atServer
INFO|2023-08-03 20:01:05.749753|Monitor (@client_0)|monitor started for @client_0 with last notification time: null
INFO|2023-08-03 20:01:05.800831|AbstractAtKeyEncryption (@client_0)|Found 'their' copy of shared symmetric key for @rv_am in atServer - saving to local storage
INFO|2023-08-03 20:01:06.467464| sshnp |SUCCESS:id: fbff867f-195b-4c65-b85d-db7a4333532b status: NotificationStatusEnum.delivered @rv_am:e2e.sshrvd@client_0
INFO|2023-08-03 20:01:06.652914|AbstractAtKeyEncryption (@client_0)|Encrypted shared symmetric key for @client_0 not found in local storage
INFO|2023-08-03 20:01:06.652964|AbstractAtKeyEncryption (@client_0)|Deleting @server_0:shared_key@client_0 from LocalSecondary
INFO|2023-08-03 20:01:06.653072|AbstractAtKeyEncryption (@client_0)|Fetching shared symmetric key for @client_0 from atServer
INFO|2023-08-03 20:01:06.713546|AbstractAtKeyEncryption (@client_0)|Retrieved my encrypted copy of shared symmetric key for @server_0 from atServer - saving to local storage
INFO|2023-08-03 20:01:06.719041|AbstractAtKeyEncryption (@client_0)|'Their' copy of shared symmetric key for @server_0 not found in local storage - will check atServer
INFO|2023-08-03 20:01:06.765747|AbstractAtKeyEncryption (@client_0)|Found 'their' copy of shared symmetric key for @server_0 in atServer - saving to local storage
INFO|2023-08-03 20:01:07.422481| sshnp |SUCCESS:id: cec47b31-3b59-445a-8afc-e6ca6d80dd0a status: NotificationStatusEnum.delivered
INFO|2023-08-03 20:01:08.086856| sshnp |SUCCESS:id: 700fd594-1753-4491-8482-c603aab51750 status: NotificationStatusEnum.delivered
INFO|2023-08-03 20:01:08.770126| sshnp |SUCCESS:id: adba83d0-25c5-4ecd-800b-328a90c77e89 status: NotificationStatusEnum.delivered 58745 41143 jeremytubongbanua <IP_ADDRESS> 5245e3a6-c8f6-4f69-92be-74ef3285387d
INFO|2023-08-03 20:01:09.388435| sshnp |Received 5245e3a6-c8f6-4f69-92be-74ef3285387d notification
INFO|2023-08-03 20:01:09.388497| sshnp |Session 5245e3a6-c8f6-4f69-92be-74ef3285387d connected successfully
INFO|2023-08-03 20:01:09.388591| sshnp |Tidying up files
ssh -p 58745 jeremyvps@localhost -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -i /Users/jeremytubongbanua/.ssh/id_ed25519
jeremytubongbanua@Jeremys-M2-Air bin % ssh -p 58745 jeremyvps@localhost -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -i /Users/jeremytubongbanua/.ssh/id_ed25519
Warning: Permanently added '[localhost]:58745' (ED25519) to the list of known hosts.
Linux debian 4.19.0-21-cloud-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug  3 23:53:08 2023 from <IP_ADDRESS>
jeremyvps@debian:~$ logout
Connection to localhost closed.
jeremytubongbanua@Jeremys-M2-Air bin %

SSHNPD:
jeremyvps@debian:~/.local/bin$ ./sshnpd -a @server_0 -m @client_0 -d e2e -s -u -v --ssh-client=pure-dart
INFO|2023-08-04 00:00:57.728873|AtClientManager|setCurrentAtSign called with atSign @server_0
INFO|2023-08-04 00:00:57.729037|AtClientManager|Switching atSigns from null to @server_0
INFO|2023-08-04 00:00:57.730114|HiveBase|commit_log_055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-04 00:00:57.730994|HiveBase|055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-04 00:00:57.732774|AtClientCommitLogCompaction (@server_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-04 00:00:57.733722|AtClientManager|setCurrentAtSign complete
INFO|2023-08-04 00:00:57.733768|AtLookup|Creating new connection
INFO|2023-08-04 00:00:58.196159|AtLookup|New connection created OK
INFO|2023-08-04 00:00:58.310970|AtLookup|auth success
INFO|2023-08-04 00:00:58.645065| sshnpd |Starting connectivity listener
INFO|2023-08-04 00:00:58.645351| sshnpd |Subscribing to e2e\.sshnp@
INFO|2023-08-04 00:00:58.645396| sshnpd |Done
WARNING|2023-08-04 00:00:58.647606| sshnpd |connection available
INFO|2023-08-04 00:00:58.931012|Monitor (@server_0)|monitor started for @server_0 with last notification time: null
INFO|2023-08-04 00:01:07.019815| sshnpd |Received: privatekey
INFO|2023-08-04 00:01:07.019870| sshnpd |Private Key received from @client_0 notification id : cec47b31-3b59-445a-8afc-e6ca6d80dd0a
INFO|2023-08-04 00:01:07.676175| sshnpd |Received: sshpublickey
INFO|2023-08-04 00:01:07.676232| sshnpd |ssh Public Key received from @client_0 notification id : 700fd594-1753-4491-8482-c603aab51750
INFO|2023-08-04 00:01:08.347895| sshnpd |Received: sshd
INFO|2023-08-04 00:01:08.347947| sshnpd |ssh callback request received from @client_0 notification id : adba83d0-25c5-4ecd-800b-328a90c77e89
INFO|2023-08-04 00:01:08.347993| sshnpd |Starting ssh session for jeremytubongbanua to <IP_ADDRESS> on port 41143 using localhost:58745 on <IP_ADDRESS>
SHOUT|2023-08-04 00:01:08.348011| sshnpd |Starting ssh session using pureDart (pure-dart) from: @client_0 session: 5245e3a6-c8f6-4f69-92be-74ef3285387d
INFO|2023-08-04 00:01:09.314805| sshnpd | sshnpd connected notification sent to:from "@client_0:5245e3a6-c8f6-4f69-92be-74ef3285387d.e2e.sshnp@server_0
INFO|2023-08-04 00:01:09.931175| sshnpd |SUCCESS:id: afbf2626-64ae-40a3-9442-b67448a7654b status: NotificationStatusEnum.delivered for: 5245e3a6-c8f6-4f69-92be-74ef3285387d with value: connected
SHOUT|2023-08-04 00:01:24.315313| sshnpd |5245e3a6-c8f6-4f69-92be-74ef3285387d | ssh session complete

I suspect the option we use is in a newer version of ssh.
Try also upgrading the os and see if it persists.
sudo apt update
sudo apt upgrade

SSHNPD on Debian 12 Linux x64 works
SSHNPD
jeremyvps@localhost:~/.local/bin$ ./sshnpd -a @server_0 -m @client_0 -s -u -v -d e2e
INFO|2023-08-04 00:15:00.404931|AtClientManager|setCurrentAtSign called with atSign @server_0
INFO|2023-08-04 00:15:00.405291|AtClientManager|Switching atSigns from null to @server_0
INFO|2023-08-04 00:15:00.406059|HiveBase|commit_log_055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-04 00:15:00.407023|HiveBase|055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-04 00:15:00.408584|AtClientCommitLogCompaction (@server_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-04 00:15:00.409928|AtClientManager|setCurrentAtSign complete
INFO|2023-08-04 00:15:00.410057|AtLookup|Creating new connection
INFO|2023-08-04 00:15:00.738197|AtLookup|New connection created OK
INFO|2023-08-04 00:15:00.852924|AtLookup|auth success
INFO|2023-08-04 00:15:01.152077| sshnpd |Starting connectivity listener
INFO|2023-08-04 00:15:01.152547| sshnpd |Subscribing to e2e\.sshnp@
INFO|2023-08-04 00:15:01.152673| sshnpd |Done
WARNING|2023-08-04 00:15:01.155494| sshnpd |connection available
INFO|2023-08-04 00:15:01.410185|Monitor (@server_0)|monitor started for @server_0 with last notification time: null
INFO|2023-08-04 00:15:23.310843| sshnpd |Received: privatekey
INFO|2023-08-04 00:15:23.311079| sshnpd |Private Key received from @client_0 notification id : ad1400d7-e1e4-44b6-818f-855e036c710e
INFO|2023-08-04 00:15:23.965582| sshnpd |Received: sshpublickey
INFO|2023-08-04 00:15:23.965807| sshnpd |ssh Public Key received from @client_0 notification id : 7e60e0cf-b711-4155-b6fe-f66733bdb4c0
INFO|2023-08-04 00:15:24.594506| sshnpd |Received: sshd
INFO|2023-08-04 00:15:24.594567| sshnpd |ssh callback request received from @client_0 notification id : 7e33be6d-9340-43a7-bb78-07964007c982
INFO|2023-08-04 00:15:24.594615| sshnpd |Starting ssh session for jeremytubongbanua to <IP_ADDRESS> on port 44289 using localhost:59097 on <IP_ADDRESS>
SHOUT|2023-08-04 00:15:24.594633| sshnpd |Starting ssh session using hostSsh (/usr/bin/ssh) from: @client_0 session: 4c23cf9b-5cf6-42fa-9daa-165676550823
INFO|2023-08-04 00:15:24.597543| sshnpd |4c23cf9b-5cf6-42fa-9daa-165676550823 | Executing /usr/bin/ssh<EMAIL_ADDRESS>-p 44289 -i /tmp/.9bb96174-2ad4-4a29-b822-005ede3ceff5 -R 59097:localhost:22 -o LogLevel=VERBOSE -t -t -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -o BatchMode=yes -o ExitOnForwardFailure=yes -o ForkAfterAuthentication=yes sleep 15
INFO|2023-08-04 00:15:25.136798| sshnpd |4c23cf9b-5cf6-42fa-9daa-165676550823 | sshStdErr | Warning: Permanently added '[<IP_ADDRESS>]:44289' (ED25519) to the list of known hosts.

INFO|2023-08-04 00:15:25.869579| sshnpd |4c23cf9b-5cf6-42fa-9daa-165676550823 | sshStdErr | Authenticated to <IP_ADDRESS> ([<IP_ADDRESS>]:44289) using "publickey".

INFO|2023-08-04 00:15:26.054092| sshnpd | sshnpd connected notification sent to:from "@client_0:4c23cf9b-5cf6-42fa-9daa-165676550823.e2e.sshnp@server_0
INFO|2023-08-04 00:15:26.356215| sshnpd |4c23cf9b-5cf6-42fa-9daa-165676550823 | sshStdOut | ssh session complete

INFO|2023-08-04 00:15:26.729137| sshnpd |SUCCESS:id: a5217cdd-31a7-4a9c-84aa-d052bd48e170 status: NotificationStatusEnum.delivered for: 4c23cf9b-5cf6-42fa-9daa-165676550823 with value: connected

I will try Debian 10 again without doing apt upgrade and apt update

@cconstab tried doing sudo apt-get update && sudo apt-get upgrade and still getting the same error
Debian 10
SSHNPD
jeremyvps@debian:~/.local/bin$ sudo apt-get update && sudo apt-get upgrade
...

jeremyvps@debian:~/.local/bin$ ./sshnpd -a @server_0 -m @client_0 -d e2e -s -u -v
INFO|2023-08-04 00:25:49.259380|AtClientManager|setCurrentAtSign called with atSign @server_0
INFO|2023-08-04 00:25:49.259525|AtClientManager|Switching atSigns from null to @server_0
INFO|2023-08-04 00:25:49.260527|HiveBase|commit_log_055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-04 00:25:49.261319|HiveBase|055719601711c762ce6bc501676e84468ea43ee6a20f32649e09e7cf951c9526 initialized successfully
INFO|2023-08-04 00:25:49.262239|AtClientCommitLogCompaction (@server_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-04 00:25:49.263061|AtClientManager|setCurrentAtSign complete
INFO|2023-08-04 00:25:49.263093|AtLookup|Creating new connection
INFO|2023-08-04 00:25:49.741647|AtLookup|New connection created OK
INFO|2023-08-04 00:25:49.856447|AtLookup|auth success
INFO|2023-08-04 00:25:50.271444| sshnpd |Starting connectivity listener
INFO|2023-08-04 00:25:50.271783| sshnpd |Subscribing to e2e\.sshnp@
INFO|2023-08-04 00:25:50.271838| sshnpd |Done
WARNING|2023-08-04 00:25:50.274064| sshnpd |connection available
INFO|2023-08-04 00:25:50.563484|Monitor (@server_0)|monitor started for @server_0 with last notification time: null
INFO|2023-08-04 00:25:53.488257| sshnpd |Received: privatekey
INFO|2023-08-04 00:25:53.488321| sshnpd |Private Key received from @client_0 notification id : 381e6a9f-eafb-42e4-b61a-a46927ffd422
INFO|2023-08-04 00:25:54.262124| sshnpd |Received: sshpublickey
INFO|2023-08-04 00:25:54.262193| sshnpd |ssh Public Key received from @client_0 notification id : 4439edf2-a975-491e-ba5c-b89a82c0a89e
INFO|2023-08-04 00:25:54.898681| sshnpd |Received: sshd
INFO|2023-08-04 00:25:54.898878| sshnpd |ssh callback request received from @client_0 notification id : ce0e876f-fff1-4019-a6d0-630343ffeb69
INFO|2023-08-04 00:25:54.898977| sshnpd |Starting ssh session for jeremytubongbanua to <IP_ADDRESS> on port 33273 using localhost:59304 on <IP_ADDRESS>
SHOUT|2023-08-04 00:25:54.899050| sshnpd |Starting ssh session using hostSsh (/usr/bin/ssh) from: @client_0 session: 35cfd932-98b9-43f1-a942-a7396bc3ab08
INFO|2023-08-04 00:25:54.902001| sshnpd |35cfd932-98b9-43f1-a942-a7396bc3ab08 | Executing /usr/bin/ssh<EMAIL_ADDRESS>-p 33273 -i /tmp/.3e1d14f2-0fff-4fef-b5bc-562ff96d7083 -R 59304:localhost:22 -o LogLevel=VERBOSE -t -t -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -o BatchMode=yes -o ExitOnForwardFailure=yes -o ForkAfterAuthentication=yes sleep 15
INFO|2023-08-04 00:25:54.905800| sshnpd |35cfd932-98b9-43f1-a942-a7396bc3ab08 | sshStdErr | command-line: line 0: Bad configuration option: forkafterauthentication

SHOUT|2023-08-04 00:25:54.906259| sshnpd |35cfd932-98b9-43f1-a942-a7396bc3ab08 | Exit code 255 from /usr/bin/ssh<EMAIL_ADDRESS>-p 33273 -i /tmp/.3e1d14f2-0fff-4fef-b5bc-562ff96d7083 -R 59304:localhost:22 -o LogLevel=VERBOSE -t -t -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes -o BatchMode=yes -o ExitOnForwardFailure=yes -o ForkAfterAuthentication=yes sleep 15
WARNING|2023-08-04 00:25:54.906299| sshnpd |Failed to establish connection - exit code 255
INFO|2023-08-04 00:25:55.579558| sshnpd |SUCCESS:id: 058af9bf-4b0d-4197-b3ef-03c219142a2e status: NotificationStatusEnum.delivered for: 35cfd932-98b9-43f1-a942-a7396bc3ab08 with value: Failed to establish connection - exit code 255 (use --local-port to specify unused port)

SSHNP
jeremytubongbanua@Jeremys-M2-Air bin % ./sshnp -f @client_0 -t @server_0 -s id_ed25519.pub -h @rv_am -v -d e2e
INFO|2023-08-03 20:25:51.117096|AtClientManager|setCurrentAtSign called with atSign @client_0
INFO|2023-08-03 20:25:51.117150|AtClientManager|Switching atSigns from null to @client_0
INFO|2023-08-03 20:25:51.118105|HiveBase|commit_log_d75a51c1003c996d4592b57407ea6b3f21a2e1163891d3612cb54a41aa363a01 initialized successfully
INFO|2023-08-03 20:25:51.119140|HiveBase|d75a51c1003c996d4592b57407ea6b3f21a2e1163891d3612cb54a41aa363a01 initialized successfully
INFO|2023-08-03 20:25:51.119171|AtClientCommitLogCompaction (@client_0)|Starting commit log compaction job running for every 11 minute(s)
INFO|2023-08-03 20:25:51.119777|AtClientManager|setCurrentAtSign complete
INFO|2023-08-03 20:25:51.119790|AtLookup|Creating new connection
INFO|2023-08-03 20:25:51.469885|AtLookup|New connection created OK
INFO|2023-08-03 20:25:51.608707|AtLookup|auth success
INFO|2023-08-03 20:25:51.676059| sshnp |Subscribing to notifications on 35cfd932-98b9-43f1-a942-a7396bc3ab08.e2e.sshnp@
INFO|2023-08-03 20:25:52.028396|AbstractAtKeyEncryption (@client_0)|Encrypted shared symmetric key for @client_0 not found in local storage
INFO|2023-08-03 20:25:52.028565|AbstractAtKeyEncryption (@client_0)|Deleting @rv_am:shared_key@client_0 from LocalSecondary
INFO|2023-08-03 20:25:52.028877|AbstractAtKeyEncryption (@client_0)|Fetching shared symmetric key for @client_0 from atServer
INFO|2023-08-03 20:25:52.032794|Monitor (@client_0)|monitor started for @client_0 with last notification time: null
INFO|2023-08-03 20:25:52.087535|AbstractAtKeyEncryption (@client_0)|Retrieved my encrypted copy of shared symmetric key for @rv_am from atServer - saving to local storage
INFO|2023-08-03 20:25:52.096125|AbstractAtKeyEncryption (@client_0)|'Their' copy of shared symmetric key for @rv_am not found in local storage - will check atServer
INFO|2023-08-03 20:25:52.167652|AbstractAtKeyEncryption (@client_0)|Found 'their' copy of shared symmetric key for @rv_am in atServer - saving to local storage
INFO|2023-08-03 20:25:52.851344| sshnp |SUCCESS:id: 7c73cb09-b5f8-43ea-98f2-a21f34f8b859 status: NotificationStatusEnum.delivered @rv_am:e2e.sshrvd@client_0
INFO|2023-08-03 20:25:53.029865|AbstractAtKeyEncryption (@client_0)|Encrypted shared symmetric key for @client_0 not found in local storage
INFO|2023-08-03 20:25:53.029984|AbstractAtKeyEncryption (@client_0)|Deleting @server_0:shared_key@client_0 from LocalSecondary
INFO|2023-08-03 20:25:53.030244|AbstractAtKeyEncryption (@client_0)|Fetching shared symmetric key for @client_0 from atServer
INFO|2023-08-03 20:25:53.099977|AbstractAtKeyEncryption (@client_0)|Retrieved my encrypted copy of shared symmetric key for @server_0 from atServer - saving to local storage
INFO|2023-08-03 20:25:53.108795|AbstractAtKeyEncryption (@client_0)|'Their' copy of shared symmetric key for @server_0 not found in local storage - will check atServer
INFO|2023-08-03 20:25:53.168048|AbstractAtKeyEncryption (@client_0)|Found 'their' copy of shared symmetric key for @server_0 in atServer - saving to local storage
INFO|2023-08-03 20:25:53.899316| sshnp |SUCCESS:id: 381e6a9f-eafb-42e4-b61a-a46927ffd422 status: NotificationStatusEnum.delivered
INFO|2023-08-03 20:25:54.600380| sshnp |SUCCESS:id: 4439edf2-a975-491e-ba5c-b89a82c0a89e status: NotificationStatusEnum.delivered
INFO|2023-08-03 20:25:55.066179| sshnp |Received 35cfd932-98b9-43f1-a942-a7396bc3ab08 notification
Remote sshnpd error: Failed to establish connection - exit code 255 (use --local-port to specify unused port)
INFO|2023-08-03 20:25:55.200615| sshnp |Received 35cfd932-98b9-43f1-a942-a7396bc3ab08 notification
Remote sshnpd error: Failed to establish connection - exit code 255 (use --local-port to specify unused port)
INFO|2023-08-03 20:25:55.303854| sshnp |SUCCESS:id: ce0e876f-fff1-4019-a6d0-630343ffeb69 status: NotificationStatusEnum.delivered 59304 33273 jeremytubongbanua <IP_ADDRESS> 35cfd932-98b9-43f1-a942-a7396bc3ab08
INFO|2023-08-03 20:25:55.303957| sshnp |Tidying up files

@cconstab --ssh-client=pure-dart fixes it still in Debian 10..
The two fixes right now:

Use Debian 10 but --ssh-client=pure-dart
Use Debian 12

I suspect the option we use is in a newer version of ssh.
Try also upgrading the os and see if it persists.
sudo apt update
sudo apt upgrade

Thanks so much for the research!

Debian 10 is still maintained, and we will no doubt encounter customer situations that try to use older distros that are officially out of maintenance.
What version of OpenSSH do you get from the package manager?

Meanwhile, ill add a startup check to verify that the available ssh command on the host supports what we need and, if not, prints a helpful error message explaining and instructing to use the pure-dart flag

@cpswan 👍

Looking through the openssh release notes, the ssh client's ForkAfterAuthentication flag was added in openssh version 8.7 released on 2021-08-20
The version of openssh supported on debian 10 is, I think, version 7.9, released on 2018-10-19 and as best I can tell, of the flags that sshnpd currently uses, only the ForkAfterAuthentication flag was added after that version
@JeremyTubongbanua can you confirm the version please by running ssh -V on your Debian 10 host?
However, looking more into command-line flags, looks like the capability of forking after authentication was also in version 7.9 by using the -f flag - indeed, reading man 5 ssh_config, setting ForkAfterAuthentication=yes has the same behaviour as using the -f flag. So if we use -f then we can support Debian 10 also
So I think there are three things to do

Replace -o ForkAfterAuthentication=yes with -f in the args that sshnpd passes when it spawns the ssh, and verify this works on Debian 10
Check the actual minimum version we require. I believe it is version 7.6, released 2017-10-03, which is when the accept-new setting was made available for the StrictHostKeyChecking option
Have sshnpd, when running in --ssh-client=/usr/bin/ssh mode (which will remain the default), check the available version, and fall back to using pure-dart mode if /usr/bin/ssh is unavailable, or is not openssh at or above version 7.6

@cpswan make sense to you?

@gkc all makes sense to me.
Some other thoughts...

OpenSSH is a regular offender for security vulnerabilities. We should probably warn when we find a known vulnerable daemon.
OpenSSH isn't the default ssh daemon in some distros (e.g. OpenWRT uses Dropbear). We should figure out if Dropbear has the capabilities we need (probably not) and we can also advise that customers can install OpenSSH in their OpenWRT systems (space permitting - things are often tight).

I'll note here that the thing that gets me putting OpenSSH onto OpenWRT systems is the lack of -D for SOCKS proxy in Dropbear. This will likely be a consideration for some of the web/api things we might do.

After I ssh'd into a fresh installation of Debian 10
root@debian:~# apt list openssh
Listing... Done

@cpswan
apt-get update && apt-get install -y sudo sudo wget curl openssh-server nano vim iproute2 nmap tmux
root@debian:~# apt list openssh-server
Listing... Done
openssh-server/oldoldstable,now 1:7.9p1-10+deb10u2 amd64 [installed]
N: There is 1 additional version. Please use the '-a' switch to see it

root@debian:~# apt list -a openssh-server
Listing... Done
openssh-server/buster-backports 1:8.4p1-2~bpo10+1 amd64
openssh-server/oldoldstable,now 1:7.9p1-10+deb10u2 amd64 [installed]
openssh-server/oldoldstable 1:7.9p1-10+deb10u1 amd64

@gkc
root@debian:~# ssh -V
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1n  15 Mar 2022

@JeremyTubongbanua could you pick this up this sprint?

I will note that this is not P0 because this only occurs where we have the need to run a double ssh (i.e. the daemon starts an ssh to the client host) and within a week or so we will support direct ssh as the default option when using an rvd

We can revisit this ticket after the 4.0.0 launch

Your choice of ssh server should be entirely your own. So if we want to support different clients, they should have to specify via an ssh-client flag, similar to how sshnp does it. I don't think this is urgent,  moving to P3


Installing node-sass with the binding blocked by corporate proxy

Hi,
I'm trying to install node-sass via NPM. Sadly I'm behind a corporate proxy and when it tries to download the windows binding from https://github.com/sass/node-sass/releases/download/v3.4.2/win32-ia32-14_binding.node, it can't because the proxy will block the file for being a binary.
Is there any option to provide this file from somewhere else? I think it should be no problem to get this file once in some other way and put it into some directory, but I have no idea how to make npm look for it there. Is this possible?
Cheers and thanks in advance

You can config npm to use your proxy settings.
http://jjasonclark.com/how-to-setup-node-behind-web-proxy/

I know, the proxy settings are not the issue, npm is talking fine to the internet.
However, the proxy also acts as an anti-virus and will block "malicious" files (which basically means everything it considers an executable). That's why I'm asking how to only provide this specific file differently.

As documented on the homepage of this project there options to configure where the binary is downloaded from.

ok, clearly need to work on my rtfm skills. thank you, that works!

# in ~/.npmrc
registry=http://registry.npmjs.org/
proxy=http://localhost:8088/
http-proxy=http://localhost:8088
https-proxy=http://localhost:8088/
strict-ssl=false
unsafe-perm=true

# https://medium.com/@mallim/how-to-use-node-sass-in-a-closed-environment-859880720f2a
# OR sudo npm config set sass-binary-site=https://npm.taobao.org/mirrors/node-sass --global
npm install --save-dev node-sass --sass-binary-site=https://npm.taobao.org/mirrors/node-sass


[BUG] functools.cached_property does not work on cdef classes

Describe the bug
The cached_property decorator fails when used inside a cdef class.
Code to reproduce the behaviour:
from functools import cached_property
cdef class A:
    @cached_property
    def x(self):
        return 1

a = A()
a.x

Output:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
...
----> 1 a.x

File ...
    963     return self
    964 if self.attrname is None:
--> 965     raise TypeError(
    966         "Cannot use cached_property instance without calling __set_name__ on it.")
    967 try:
    968     cache = instance.__dict__

TypeError: Cannot use cached_property instance without calling __set_name__ on it.

Expected behaviour
The above snippet should generate a working (cached) property.
According to the docs, __set_name__ is only called on descriptors if they are created as part of the class definition. I suspect that the reason for this issue is that Cython is actually translating the above code into the equivalent of
from functools import cached_property
class A:
    def x(self):
        return 1

A.x = cached_property(A.x)

a = A()
a.x

which reproduces the issue in pure Python. The above snippet may be fixed by manually calling A.x.__set_name__(A, "x"), so I would guess that line needs to be added into the Cython handling of the cached property as well somewhere.
OS
Linux
Python version
3.8.16 (and 3.10.2)
Cython version
3.0.0b3 (and 0.29.36)
Additional context
No response

Since the decorator accesses the object's __dict__, it also won't work for that reason. It doesn't really provide a performance benefit for Cython code, since a regular property would have the same performance.

That's a good point. You could fix that limitation as documented here.
Why wouldn't it provide a performance benefit for Cython code? Wouldn't caching the value allow you to elide the function call and any logic in the property (for something less trivial than my example above)? Or am I misinterpreting your statement?

The decorator does allow you to elide the function call, but only from the Python perspective. Every attribute lookup is still a function call, it just means it only uses the builtin interpreter logic, so it's at C speeds. But for C extension classes, all attributes are accessed only through properties (since they're stored as part of the struct), so there's nothing to skip. All that cdef public ... does is auto-generate the implementation for you.

Hmm I think you're focusing on a different piece of this than I am. Let me illustrate with an example, please correct me if you see a point where I'm misunderstanding.
Say I have a class:
class A:
    @cached_property
    def one(self):
        return expensive_function_that_returns_one()

The one descriptor will be defined on the class, not on each instance. The first time I do a = A(); a.one it will effectively do a.one = A.one(a). Then, future accesses to a.one will be able to actually see that a.__dict__['one'] == 1 and bypass calls to A.one.
Now let's switch that to a Cython class with a raw property:
cdef class A:
    @property
    def one(self):
        return expensive_function_that_returns_one()

Accessing the attribute A.one automatically happens at C speeds because it's a reserved slot in the struct rather than a Python dictionary access. So yes, removing this overhead in Cython is meaningless because it's equivalent.
The actual function call one is still going to happen at Python speeds because it's a Python function. AFAIK there is no way around this because you can't make cdef functions properties.
The expensive_function_that_returns_one doesn't actually have to be an expensive function, all it needs is to be a Python (not C) function in order to introduce the usual overheads (unboxing PyObjects, etc) associated with a Python function call instead of a C function call.

If cached_property worked in Cython, then we could remove the overhead of both the second and third bullet points, right?

That would all be true normally yes, except that Cython handles @property decorators specially. It detects you using it (along with the standard<EMAIL_ADDRESS>variations), then puts them in the tp_getset list in the type definition. That generates getset_descriptor objects, which store function pointers for the getter/setter. Those have fixed signatures, so there's no argument parsing overhead or anything, but for the setter there's type-checks/unboxing for the value of course.
Definitely calling expensive_function_that_returns_one() repeatedly would be wasteful, what I was meaning was to do the traditional caching in your code. Possibly Cython could just detect cached_property and generating that code automatically, but it might be a little awkward since you then have the same name referring to both the descriptor and the struct member. Any access in your Cython code wouldn't be able to bypass the property.

I think you're right in your diagnosis of what the problem is. It probably also wouldn't be too hard to add the __set_name__ calls and make this kind of work.
It's probably work emphasising that cdef classes aren't regular Python classes - you're trading flexibility and features for speed. There's all kinds of stuff you can do in the body of Python classes that won't work in cdef classes (pretty much anything that relies on order of execution, putting function definitions in if statements, etc.). That's just because functions and properties are statically defined at compile-time rather than dynamically at runtime (mostly).
Cython definitely tries to get regular Python-style classes to behave as close as possible to Python behaviour. But I don't think we worry too much if cdef classes have to deviate a bit.
Which isn't to say that it won't get fixed if it can be, but it isn't completely obvious to me that this should work.

That would all be true normally yes, except that Cython handles @property decorators specially.

OK yeah I was wondering if that was the case. I don't know the Cython internals terribly well unfortunately, thanks for the info.

Possibly Cython could just detect cached_property and generating that code automatically

That's what I was imagining could be done. Whether it's worthwhile or not is a different question, and as I said above I'm totally fine if Cython decides not to fix this. It would be nice to include a better error message if that's the decision, though.

It's probably work emphasising that cdef classes aren't regular Python classes - you're trading flexibility and features for speed.

Yup, totally fair. I was expecting this decorator to work, but not for any particularly good reason. Its implementation is certainly antithetical to the way Cython classes work given that it's exploiting the instance's __dict__ and the relationship between descriptors on classes and instance attributes.

I think the rootcause of the problem is that Cython for cdef classes does not call __set_name__ as described in documentation [1]:

Automatically called at the time the owning class owner is created.

Simple example not involving @cached_property based on descriptor documentation [2]
# descriptor.py file
import logging

logging.basicConfig(level=logging.INFO)

class LoggedAccess:

    def __set_name__(self, owner, name):
        self.public_name = name
        self.private_name = '_' + name

    def __get__(self, obj, objtype=None):
        value = getattr(obj, self.private_name)
        logging.info('Accessing %r giving %r', self.public_name, value)
        return value

    def __set__(self, obj, value):
        logging.info('Updating %r to %r', self.public_name, value)
        setattr(obj, self.private_name, value)

# example.pyx file
from descriptor import LoggedAccess

cdef class Person:

    cdef public object _name
    cdef public object _age
    name = LoggedAccess()
    age = LoggedAccess()

    def __init__(self, name, age):
        self.name = name
        self.age = age

Compiling and using example.pyx will trigger an error:
>>> from example import Person
>>> p = Person('John', 5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "example.pyx", line 11, in example.Person.__init__
    self.name = name
  File "/home/matus/dev/cython/test/example.py", line 17, in __set__
    logging.info('Updating %r to %r', self.public_name, value)
                                      ^^^^^^^^^^^^^^^^
AttributeError: 'LoggedAccess' object has no attribute 'public_name'

In contrast same example in python works:
from descriptor import LoggedAccess

class Person:

    name = LoggedAccess()
    age = LoggedAccess()

    def __init__(self, name, age):
        self.name = name
        self.age = age

[1] https://docs.python.org/3/reference/datamodel.html#object.set_name
[2] https://docs.python.org/3/howto/descriptor.html

OK I was able to fix example above by adjusting __pyx_pymod_exec_pyxpyx as follows:
   /* "pyxpyx.pyx":7
  *     cdef public object _name
  *     cdef public object _age
  *     name = LoggedAccess()             # <<<<<<<<<<<<<<
  *     age = LoggedAccess()
  *·
  */
   __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_LoggedAccess); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 7, __pyx_L1_error)
   __Pyx_GOTREF(__pyx_t_3);

   __pyx_t_2 = __Pyx_PyObject_CallNoArg(__pyx_t_3); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 7, __pyx_L1_error)
   __Pyx_GOTREF(__pyx_t_2);
   if (PyObject_HasAttrString(__pyx_t_2, "__set_name__")) {
       PyObject_CallMethod(__pyx_t_2, "__set_name__", "Os", (PyObject *)__pyx_ptype_6pyxpyx_Person, __pyx_k_name);
   }
   __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
   if (__Pyx_SetItemOnTypeDict((PyObject *)__pyx_ptype_6pyxpyx_Person, __pyx_n_s_name, __pyx_t_2) < 0) __PYX_ERR(0, 7, __pyx_L1_error)
   __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;

   /* "pyxpyx.pyx":8
  *     cdef public object _age
  *     name = LoggedAccess()
  *     age = LoggedAccess()             # <<<<<<<<<<<<<<
  *·
  *     def __init__(self, name, age):
  */
   __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_LoggedAccess); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 8, __pyx_L1_error)
   __Pyx_GOTREF(__pyx_t_2);
   __pyx_t_3 = __Pyx_PyObject_CallNoArg(__pyx_t_2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 8, __pyx_L1_error)
   if (PyObject_HasAttrString(__pyx_t_3, "__set_name__")) {
       PyObject_CallMethod(__pyx_t_3, "__set_name__", "Os", (PyObject *)__pyx_ptype_6pyxpyx_Person, __pyx_k_age);
   }
   __Pyx_GOTREF(__pyx_t_3);
   __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
   if (__Pyx_SetItemOnTypeDict((PyObject *)__pyx_ptype_6pyxpyx_Person, __pyx_n_s_age, __pyx_t_3) < 0) __PYX_ERR(0, 8, __pyx_L1_error)
   __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;

The change is adding lines - they ensures that __set_name__ method is called:
   if (PyObject_HasAttrString(__pyx_t_3, "__set_name__")) {
       PyObject_CallMethod(__pyx_t_3, "__set_name__", "Os", (PyObject *)__pyx_ptype_6pyxpyx_Person, __pyx_k_age);
   }

and
   if (PyObject_HasAttrString(__pyx_t_3, "__set_name__")) {
       PyObject_CallMethod(__pyx_t_3, "__set_name__", "Os", (PyObject *)__pyx_ptype_6pyxpyx_Person, __pyx_k_age);
   }


Discuss the workflow and potential tasks

I had a first look at the data, @FuXiming and @nh36. They are fine now, but there remain some problems, I had to correct entries, where rhymeids were not given in the same number as there are words in the line. But this is now fixed, or should be double checked. So now, the next steps should be:

[ ] extract rhyme words, so we can make a list of OCBS readings for them (no need to do this in the rhymes themselves)
[ ] set up the workflow with versions and the like (@FuXiming, you can upload files to the raw/ folder. To do so, you should check this, go to the raw/ folder and see how to upload a file. This is most convenient for me then, please use NO white space in file names, use underscore instead)
[ ] we want a rhyme network, although this won't tell us much, I am afraid (so maybe we can drop it)
[ ] we want a concordance of all words in the text, just so that we can collect them

Hey @LinguList, thanks for looking through things. Columns R:1 and R:2 list out all the rhyme words, happy to pull those into a file and place in raw/ if that's convenient. I also have OCBS for them (or proximate guesses...) in the Alignment column, but can include that in the rhyme word list file too.  Thanks again!

Also @LinguList, I am going to delete the old files in Code, and add in the latest two versions + new documentation.


bug(flatpak): cannot open mimetype

Bug description
While testing https://github.com/containers/podman-desktop/pull/9804, I noticed that on linux, we were not handling the mimeTypes properly. As I was not able to open properly Podman Desktop.

reading some documentations

https://github.com/electron-userland/electron-builder/blob/3cca3c304b939e7b8f4377f91b77a8a6966fd6f3/packages/app-builder-lib/src/options/linuxOptions.ts#L84C12-L84C21
https://github.com/electron-userland/electron-builder/issues/4035

We are missing the mimeTypes in our .electron-builder.config.cjs configuration.
Operating system
Fedora 40
Installation Method
Flathub (Linux)
Version
next (development version)
Steps to reproduce

Have podman desktop install through flatpak
Try to open using gio mime podman-desktop:extension/redhat.ai-lab
assert No default applications for “podman-desktop:extension/redhat.ai-lab”

Relevant log output
No response
Additional context
WIP branch with non-working fix https://github.com/axel7083/podman-desktop/tree/fix/flatpak-mimetypes
Weird problem when adding logs on
https://github.com/containers/podman-desktop/blob/1adfefa5129858f83f2cb636ded4f6fbc8223722/packages/main/src/index.ts#L114-L118
main ↪ second-instance podman-desktop,--allow-file-access-from-files,podman-desktop:extension/podman-desktop.minikube /home/axel7083 [object Object]
main ↪ additionalData {"argv":[]}

cc @benoitf @jeffmaury is this problem critial enough to include as unplanned in the sprint, or should we delay to the next one?

next one it's not a regression

Assigning myself for next-sprint

Moving to next sprint, this is very very hard to test properly, and I don't want to write hacky tests, hopefully we can do https://github.com/podman-desktop/podman-desktop/issues/10042 first to ease future testing


OpenFoam10

There are problems compiling the code on OpenFoam10

It seems that there are problems compiling the code on OpenFOAM9

it works up to OpenFoam6...


Fix #1074, Refactor CFE_TIME_RegisterSynchCallback

Describe the contribution
Fix #1074

One return point
Eliminates "possible uninitialized variable" static analysis warning

Testing performed
Standard build, unit test
Expected behavior changes
None except eliminates static analysis warning
System(s) tested on

Hardware: cFS Dev Server
OS: Ubuntu 18.04
Versions: Bundle main + this commit

Additional context
Partially just a conversation starter - do we want to avoid (invalid) warnings w/ similar refactors? Alternatively we could just ignore/repress the warnings.
Third party code
None
Contributor Info - All information REQUIRED for consideration of pull request
Jacob Hageman - NASA/GSFC

CCB:2021-01-13 APPROVED
Replaces multiple returns with an "if tree" fixes some static code analysis complaints.

CCB:2021-01-13 APPROVED
Replaces multiple returns with an "if tree" fixes some static code analysis complaints.


Is it possible to print a local file?

Is it possible to print a local PDF file?
example:
string LocalFileName = "C:\Users\work\AppData\Local\Temp\fileExample.pdf";
PrintingService.Print(LocalFileName, showModal: true);
I tried, but without success.

That won't be possible, however:
Read the file using an[<InputFile/>](https://learn.microsoft.com/en-us/aspnet/core/blazor/file-uploads?view=aspnetcore-8.0)  Component and convert it to a BASE64 then print it.


bug: Titles Not Available in Course Content

Describe the bug
Titles Not Available in Course Content
To Reproduce
Steps to reproduce the behavior:
1.go to https://app.100xdevs.com/courses/3
2. titles is not there
Expected behavior
i should be seeing titles
Screenshots or GIFs
If applicable, add screenshots to help explain your problem.

Info (please complete the following information):

Browser chrome

Additional context
Add any other context about the problem here.

This bug has been resolved, updated built is not deployed yet.
Maybe can be closed in favour of #1632  @devsargam.


Support running lium sync without --version option

Unlike the original repo sync command, lium sync cannot be executed without specifying a version.

Discussed offline with @hikalium :
There is a problem that if running repo sync without specifying the version, the build may not pass.  Therefore, implement it to print a hint message instead of executing repo sync when executed without the --version option.
The hint message indicates the need for the version option and suggests possible versions.


This PR applies to the part 1 in issue # 37, ensure install works

capabilities: Basic Install doesn't change yet?
/lgtm


handle institution move from IL view

Think of a way to handle someone moving from one inst to another from the IL page.

One option:

On old inst page, add a button for "move to another inst."  This goes to a page to select the new inst, which generates an approval request.
Move is not completed until new inst admin approves it.
Email should be sent to old inst admin to confirm person has been moved.


Carthage could not build CleanroomASL 2.0.

Hi.
CleanroomASL and CleanroomLogger are not build with Carthage.
carthage version is 0.18.1
Xcode version 8.0(8A218a)
This is error output.
$ carthage update CleanroomASL --platform iOS
*** Checking out CleanroomASL at "2.0.0"
*** xcodebuild output can be found in /var/folders/mj/5zhy3sh11h18pdwrb4v04b300000gn/T/carthage-xcodebuild.sdGbmO.log
A shell task (/usr/bin/xcrun xcodebuild -project /Users/taiki/Desktop/clone/HandsPOS.swift3/Carthage/Checkouts/CleanroomASL/CleanroomASL.xcodeproj -scheme CleanroomASL-OSX -configuration Release CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= CARTHAGE=YES clean -showBuildSettings) failed with exit code 66:
xcodebuild: error: Scheme CleanroomASL-OSX is not currently configured for the clean action.

I change Cartfile.resolve commit hash to below 2cases.

13481a1747e8e2f9a92ecd0d087be7d352a637ce then checkout and build are success.
0c70c07a17c42ed870f2769aa09b0123800e8836 is failed.

So I thinking this commit(update for universal framework) is breaking carthage build.
Maybe also CleanroomLogger.

Thanks for the report. Something seems a little strange with how Carthage is behaving here. First, you're specifying --platform iOS but Carthage is attempting to build using the CleanroomASL-OSX scheme.
Secondly, while previous CleanroomASL releases used platform-specific schemes, the new release has a single build scheme: CleanroomASL, which is what Carthage should be using to build.
I'll look into it on my end, but I'm wondering if Carthage has an old copy of the repo cached. It shouldn't be trying to build with CleanroomASL-OSX with the universal framework version. So would you mind trying a couple things in the meantime?
First, you can delete the repo from Carthage's cache:
rm -rf ~/Library/Caches/org.carthage.CarthageKit/dependencies/Cleanroom*

and also from your project's Carthage/Checkouts directory. Then, try again. It may work.
If it doesn't, there is also some sort of Carthage cache database that may need cleanup:
rm -rf ~/Library/Caches/carthage

Please let me know if any of these remedies work. Thanks!

As a test, I created a new directory and put in the Cartfile:
github "emaloney/CleanroomASL" ~> 2.0.0

Then, I did:
carthage update

and got the result:
*** Fetching CleanroomASL
*** Checking out CleanroomASL at "2.0.0"
*** xcodebuild output can be found in /var/folders/gn/gt76_ktx7w1010xg4v68p3txynz8ws/T/carthage-xcodebuild.NpizXz.log
*** Building scheme "CleanroomASL" in CleanroomASL.xcodeproj

Success!
Then, I cleared everything out with:
rm -rf Carthage Cartfile.resolved

and tried:
carthage update --platform iOS

and got the same result:
*** Fetching CleanroomASL
*** Checking out CleanroomASL at "2.0.0"
*** xcodebuild output can be found in /var/folders/gn/gt76_ktx7w1010xg4v68p3txynz8ws/T/carthage-xcodebuild.PnLwUy.log
*** Building scheme "CleanroomASL" in CleanroomASL.xcodeproj

Finally, I tried putting both CleanroomASL and CleanroomLogger in the Cartfile:
github "emaloney/CleanroomASL" ~> 2.0.0
github "emaloney/CleanroomLogger" ~> 3.0.0

and then did:
rm -rf Carthage Cartfile.resolved
carthage update --platform iOS

Success again:
*** Fetching CleanroomLogger
*** Fetching CleanroomASL
*** Checking out CleanroomASL at "2.0.0"
*** Checking out CleanroomLogger at "3.0.0"
*** xcodebuild output can be found in /var/folders/gn/gt76_ktx7w1010xg4v68p3txynz8ws/T/carthage-xcodebuild.ESw7Ry.log
*** Building scheme "CleanroomASL" in CleanroomASL.xcodeproj
*** Building scheme "CleanroomLogger" in CleanroomLogger.xcworkspace

Given this, I'm guessing the issue is local to your machine, but it might also be the result of a bug in Carthage if the problem is with stale caches not being updated.
If there's any other info you can provide about your installation that would help us file a bug report, I'd like to make sure we follow up by opening a Carthage issue. For example:

The version of Carthage you're using
The contents of your Cartfile (or at least how CleanroomASL and CleanroomLogger are listed if you're not able to share it entirely)
The version of Xcode you're using
The version of macOS your machine is running

Hi, thanks for your quickly reply.
I just try on my home mac(Sierra), then result is success!
Both mac have same version of carthage(0.18.1) and Xcode8(8A218a), but build failed macOS is El Capitan.
$ cat Cartfile
github "emaloney/CleanroomLogger" ~> 3.0.0
$ carthage update --platform iOS
*** Fetching CleanroomLogger
*** Fetching CleanroomASL
*** Checking out CleanroomASL at "2.0.0"
*** Checking out CleanroomLogger at "3.0.0"
*** xcodebuild output can be found in /var/folders/5t/h_x9sc_13756x03977z5l58r0000gn/T/carthage-xcodebuild.N0wCbl.log
*** Building scheme "CleanroomASL" in CleanroomASL.xcodeproj
*** Building scheme "CleanroomLogger" in CleanroomLogger.xcworkspace

So this issue is only my business mac(El Capitan) as you indicated, I'll try again on business mac at tomorrow and report here.
Cause of issue is probably that I often switch Xcode version (xcode-select) to build carthage for Swift 3(Xcode 8) project and Swift 2.2(Xcode 7.3.1) project.

Hi.
I remove carthage cache and already checkout and build files.
rm -rf ~/Library/Caches/org.carthage.CarthageKit/dependencies/Cleanroom*
rm -fr Carthage/Build/iOS/Cleanroom*
rm -fr Carthage/Checkout/Cleanroom*

Thus carthage update is succeeded, perhaps cause is cache of carthage.
Thanks for your support.


[HOTFIX] Changes in selecting the carbonindex files

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

[ ] Any interfaces changed?

[ ] Any backward compatibility impacted?

[ ] Document update required?

[ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/5053/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/6063/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/4903/

Please provide description in this PR


Improve rates error handling

In the spirit of:

https://github.com/ofek/bit/pull/68
https://github.com/sporestack/bitcash/commit/a8fdba2aa986f0dacd030a4e76955b03d05a6e0d

Thanks mate! Your in-line comment is also noted. Will do...

 Thanks for merging.


python3Packages: don't depend on setup hooks' bash eval

This came up in #335110 and then in #347194.
The python ecosystem currently uses bash eval in some setup hooks - and of course some packages depend on that. Mostly in pytestFlagsArray, but also in setupPyBuildFlags and setupPyGlobalFlags.
This PR deals with all those cases, replacing it with more adequate solutions. This will allow us to replace pytestFlagsArray with a structuredAttrs-supporting pytestFlags alternative more easily later on.
@ShamrockLee @SomeoneSerge @emilazy @philiptaron
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

external overrides are exactly that, right?

My thinking too. Overlays can access lib.version but there's little to no control over propagated stuff, including the hooks.

Uhm, isn't that the case with every change to stdenv / any hook etc

Yes. I'd reckon it's OK because we're rather conservative about touching stdenv and hooks. Which I reckon we are at least partly because we don't know how people might be (ab)using nixpkgs "internals" out-of-tree. We could know a tad more if we advertised a more explicit interface, e.g. versioning

@ofborg build picard
@ofborg build chipsec
@ofborg build fichub-cli
@ofborg build python311Packages.macaddress build python312Packages.macaddress
@ofborg build python311Packages.iso421 python312Packages.iso4217
@ofborg build python311Packages.certbot-dns-inwx python312Packages.certbot-dns-inwx
@ofborg build python311Packages.qcodes python312Packages.qcodes
@ofborg build conan
@ofborg build rountersploit
@ofborg build python311Packages.certbot-dns-cloudflare python312Packages.certbot-dns-cloudflare
@ofborg build python311Packages.certbot-dns-google python312Packages.certbot-dns-google
@ofborg build python311Packages.certbot-dns-ovh python312Packages.certbot-dns-ovh
@ofborg build python311Packages.certbot-dns-rfc2136 python312Packages.certbot-dns-rfc2136
@ofborg build python311Packages.certbot-dns-route53 python312Packages.certbot-dns-route53
@ofborg build python311Packages.certbot python312Packages.certbot
@ofborg build python311Packages.psycopg python312Packages.psycopg
@ofborg build python311Packages.dtw-python python312Packages.dtw-python
@ofborg build python311Packages.numcodecs python312Packages.numcodecs
@ofborg build python311Packages.bottleneck python312Packages.bottleneck
@ofborg build python311Packages.vapoursynth python312Packages.vapoursynth
@ofborg build python311Packages.pytest-xdist python312Packages.pytest-xdist
@ofborg build python311Packages.psutil python312Packages.psutil
Test pytest-xdist
@ofborg build dosage

Oops! I didn't mean to DoS our CI :sweat:

but you explicitly asked for DoS‐age!

python3Packages.certbot-dns-{cloudflare,ovh,rfc2136,route53} fails to build on master, complaining
DeprecationWarning: CSR support in pyOpenSSL is deprecated. You should use ...

I filed an issue in Nixpkgs (#354054). Such issue might also need to be upstreamed.

We must check if they failed to build on master or staging since they failed after this change.

Yeah, I didn't want to rebase while all the ofborg stuff was running, so might have been after where the branch is currently on.

Oops! I switched to the wrong commit. Now I'm waiting for the correct one to build.

Since this is causing a lot of rebuilds and thus is hard to test, I am thinking of splitting it up in two pieces - one going to master with everything that causes fewer rebuilds, and one part staying here, causing the big number of rebuilds, but possibly easier to review alone.

Sounds great!

That is.. unless someone is already convinced that this PR is good to go? Then I'd just rebase over the trivial conflict in numcodecs.

These changes looks good for me. Considering those extra flags and pre/post steps contain package-specific tricks to make them build, it's still safer to at least test if they build successfully.

LGTM!
We could also re-target it to the master branch if the number of
rebuilds are low enough.
On 2024-11-15 04:23, Wolfgang Walther wrote:

@ShamrockLee [1] After the other PRs have been merged, shall we continue with this one? I think this should be good to go, do you agree?
--
Reply to this email directly, view it on GitHub [2], or unsubscribe [3].
You are receiving this because you were mentioned.Message ID: @.***>

Links:
[1] https://github.com/ShamrockLee
[2] https://github.com/NixOS/nixpkgs/pull/352976#issuecomment-2477339153
[3]
https://github.com/notifications/unsubscribe-auth/AKQF2MZNNKVU7IU33A5ZB3D2AUBCXAVCNFSM6AAAAABRA5CBEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZXGMZTSMJVGM

@wolfgangwalther I re-built from the PR, and the broken packages now build. More specifically, it fixes https://github.com/NixOS/nixpkgs/issues/354054.
So, in any case, I will at least backport the required fixes for certbot. Although it would probably be preferable to simply add a backport label to the PR (which doesn't break anything nor update anything, it just unbreaks a few builds).

@aviallon My two cents: We could consider backporting changes that don't add additional working directory changes. As for other changes with additional pushd and popd, they should be backported only to fix something already broken.