Consumption metrics faster after restart

Right now we start to collect consumption metrics every N minutes, and calculate synthetic size every M minutes for all tenants in sequence (order unspecified). In default config N == M, so there is a change that on first rounds after the restart:

upload may have some synthetic sizes
upload may will have some synthetic sizes
upload may will have most synthetic sizes
upload should have most if not all synthetic sizes

However, we now need due to external requirements reporting every N instead of possibly multiples of N. There have also been ideas of setting up an SLI for how often we upload the metrics -- I think the interesting case is "how often we upload these per tenant", not "how often a POST request goes out".

On #3542 I noticed that we also leak a bit of metrics for any detached tenants, as we never clear them. The problems with the current approach of listing all tenants and iterating them is that we lack the ability to clean things up once complete, or react to for example synthetic size calculation completing.
I think we should instead organize metrics sending so that:

task to buffer and collected metrics is spawned
each tenant gets a process collecting the metrics on a tick, including synthetic size

per tenant caching
synthetic size has a global rate limit already
synthetic size could be "ongoing" while other metrics are collected, and when it completes, we would just send it like other metrics to be buffered and sent
on tenant deactivating, we would exit the task
sending cached metrics also on a tick

the top-level consumption metrics task

spawning up new tasks for appeared tenants

I think this design would allow us fine grained control over metrics buffering (send everything at 1min, or at buffer size, more constant load on receiver), easy to see path to use the reqwest_retry using retry mechanism, etc.
The per-tenant tasks would be able to measure if we are keeping up with the goal of collecting all metrics every 5min, and have no leaks.
Pre-requisites:

event mechanism to note on when a tenant has been activated

Do we currently have an invariant on consumption metrics collection needing to happen near the same wall clock time, or could it be a per-tenant wall clock time?
As in, after a tenant is created, a clock is started, on N minutes we collect and send the first metrics over for buffering, after that L minutes of buffering, we'd POST them on.
On restart, a lot of clocks would be started at the same time because all local tenants are activated in the range of 2-15s.

Removed the accidentially added tech_design_rfc. At least I don't recall what I was thinking when selecting that.

With best effort persistent cache added in the work towards #5323 I think the problem has now shifted from getting the metrics faster on startup towards something else, if any. Closing this for now.


[rvalue] Add a conversion to bool for RValue.

This lets you conditionally trampoline RValue results using if statements, e.g.:
if (RValue rv = foo()) {
  return rv;
}

I wrote this originally as part of an experiment. Seems generally useful.

@swift-ci smoke test and merge


Fixed #29892 -- Added explicit Selenium wait in admin autocomplete tests.

@jdufresne @codingjoe This is the kind of thing I'm aiming at for the selenium tests.

Avoid the implicit wait, since...
Our use-case is quite simple: we know the expected state before/after, and we don't have to deal with difficult timing questions, so...
We end up with faster (and more precise?) tests if we can control the transitions more tightly and then use straight asserts. (A failure would tell us something, I think.)

What do you both think? Thanks!
ticket-29892

buildbot, test on selenium

buildbot, test on selenium

buildbot, test on selenium.


[ZEPPELIN-2414] Memory leak under scoped mode of SparkInterpreter caused by inapproprately setting Thread.contextClassLoader

What is this PR for?
Fix memory leak under scoped mode of SparkInterpreter which is caused by inapproprately setting Thread.contextClassLoader.
What type of PR is it?
Bug Fix
Todos
Rest Thread.contextClassLoader in Methods:

[ ] - public void open()
[ ] - private Results.Result interpret(String line)
[ ] - public List completion(String buf, int cursor)

What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2414
How should this be tested?
Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

@Jongyoul Lee Ready to review. Please help.

Thanks @hammertank for the contribution.
What do you think wrap interpreter using ClassLoaderInterpreter instead of modifying individual interpreter?
Interpreter is being instantiated from RemoteInterpreterServer, so this is place where ClassLoaderInterpreter can wrap Interpreter. Like Interpreter factory does

@Leemoonsoo Changed the code as you suggested. But I have no idea how to write a testcase to validate the modification.

#2262  has been merged. so now, could you try include commit only related to this contributions?

@Leemoonsoo re-commit as required. Please review again.
Jenkins did not get the final travis-ci results successfully. Is there a way to trigger Jenkins to fetch again without a push?

You can close/reopen the issue to trigger Jenkins

Found "ClassNotFoundException" after applying this fix. This is caused by SparkContext using the first TranslatingClassLoader to deserialize TaskResult.
Sorry to say that I haven't figure out a solution yet. :stuck_out_tongue_closed_eyes:


Make doi non-required

As of now, there is no automated way to create DOIs automatically for each new version (where version can be a tagged or released version, or a commit) before the actual release.
doi is still a worthwhile field to have, in case this changes at some point, or the field is updated manually after releases.

doi is already non-required in the current specs.

... and in the current schema. Closing.


Refactor tests into xclim as an installed module

Pull Request Checklist:

[x] This PR addresses an already opened issue (for bug fixes / features)

This PR fixes #601 (and self-fixes #615)

[x] Tests for the changes have been added (for bug fixes / features)
[x] Documentation has been added / updated (for bug fixes / features)
[x] HISTORY.rst has been updated (with summary of main changes)
[x] bumpversion (minor / major / patch) has been called on this branch
[ ] Tags have been pushed (git push --tags)

What kind of change does this PR introduce?

This refactors all tests, test utilities, and fixtures into the xclim.testing module. This involved a lot of changes to imports between modules. Individual tests are not callable but json testdata is now installed and accessible using imports from xclim (import xclim.testing.tests.data).
The makefile recipe for testing now also reflects the new structure. Same goes for tox (I am still chasing down a few bugs for tox but it's coming along).
I also opted to port the RavenPy advancements for open_dataset here (see: https://github.com/CSHS-CWRA/RavenPy/blob/master/ravenpy/tutorial.py).

Does this PR introduce a breaking change?
Most definitely. This is a huge breaking change in terms of structure, but behaviours are still very similar. End users should not notice the difference (excepting the larger size of the installation package).

Other information:

Oof, what a mess. This will be squashed before merging.

Oof, what a mess. This will be squashed before merging.

All tests pass. LGTM.


Approve ERC20 token transfer stuck!

Description
I followed the tutorial here: https://polygon-edge.netlify.app/docs/additional-features/chainbridge/setup-erc20-transfer/
I successfully register resource ID,  mint erc20 token, check balance but when I make an approve command below, it stucks there even I tried to set fast gasPrice (at that time it is around 2 GWei https://gasstation-mumbai.matic.today/v2 )
cb-sol-cli erc20 approve
--url https://polygon-mumbai.g.alchemy.com/v2/GHr4Koyt1fr4ngb7S6FN7PEKHNDogBLr
--privateKey [ADMIN_ACCOUNT_KEY]
--erc20Address "0x2988C95C4fda85b0Ec4fC4ed86c643e7bac42fB3"
--recipient "0xB65926162B2aB8E15279347f7Bc1aF77b5eCD81f"
--amount 500
--gasPrice<PHONE_NUMBER>
You guys can check all transactions at the address below ( associated with ADMIN_ACCOUNT_KEY above ): https://mumbai.polygonscan.com/address/0xE5D77C6682c4d96EA5aA371bd8c0e9fD308f545a
The transaction for set resource and mint token are successful in the transaction detail page
This is my contract deploy info:
cb-sol-cli deploy --all --chainId 99 \
  --url https://polygon-mumbai.g.alchemy.com/v2/GHr4Koyt1fr4ngb7S6FN7PEKHNDogBLr \
  --privateKey [ADMIN_ACCOUNT_PRIVATE_KEY] \
  --relayers 0xE5D77C6682c4d96EA5aA371bd8c0e9fD308f545a \
  --relayerThreshold 1 \
  --gasPrice<PHONE_NUMBER> \
  --fee 0.000000005925192625 \
  --gasLimit 6000000

Url:        https://polygon-mumbai.g.alchemy.com/v2/GHr4Koyt1fr4ngb7S6FN7PEKHNDogBLr
Deployer:   0xE5D77C6682c4d96EA5aA371bd8c0e9fD308f545a
Gas Limit:   6000000
Gas Price:<PHONE_NUMBER>
Deploy Cost: 0.058606365427019322

Options
=======
Chain Id:    99
Threshold:   1
Relayers:    0xE5D77C6682c4d96EA5aA371bd8c0e9fD308f545a
Bridge Fee:  0.000000005925192625
Expiry:      100

Contract Addresses
================================================================
Bridge:             0xcCf71922B0e6Dd5951cC640D6Ff43EdafdcB5Dbd
----------------------------------------------------------------
Erc20 Handler:      0xB65926162B2aB8E15279347f7Bc1aF77b5eCD81f
----------------------------------------------------------------
Erc721 Handler:     0x943441e37964Cc07494Ee6F6E16a3dd7cB6704dD
----------------------------------------------------------------
Generic Handler:    0xc8CBc3de1FE5609eF4184E12318963b1615792F6
----------------------------------------------------------------
Erc20:              0x2988C95C4fda85b0Ec4fC4ed86c643e7bac42fB3
----------------------------------------------------------------
Erc721:             0xFFed684ACDfba5D35517d88B0c559576a6342F06
----------------------------------------------------------------
Centrifuge Asset:   Not Deployed
----------------------------------------------------------------
WETC:               Not Deployed
==========================================

My first question is I already set a new account for deploying a contract, registering resource id, and mint token .... but the approve transferring transaction is still stuck (I wait for 10 mins with a high gas fee already). If every time transaction is stuck, we need to set up a new account and deploy everything again then I think it is not effective so do we need a way to cancel all stuck transactions?
The second question I want to ask is about the mint concept in the transfer erc20 token tutorial:
I wonder why we need to mint a token every time if we already have it? The current process for transferring is:
1. Register resourceId -> 2. Mint token -> 3. Approve transferring -> 4. Deposit token

Let's say I have HARC token in Mumbai testnet below:
https://mumbai.polygonscan.com/token/0x7146b43ce5241c5b668084f79aa97de61aa152bd?a=0x2E3E8f108CadF1aE9f1813AA37b501Ec12b6aE59
If I want to transfer that token to polygon edge local then is it correct that I just need to register resourceID with target contract is 0x7146B43cE5241c5b668084f79aa97dE61Aa152BD and then deposit token to polygon edge chain without needed to mint or approve the transferring?
Your environment

MacOS 12.4
0.4.0
hash https://github.com/0xPolygon/polygon-edge/commit/8b85d687bc964109cd7935ef62a1e503e47a2fe9

Steps to reproduce
Expected behaviour
The approval transaction should not be stuck
Actual behaviour
The approval transaction is stuck

@harry-nguyen22 Hello, Thank you for raising the issue.

Another way to remove stuch transaction is to send a transaction with same nonce but higher gas price. I'm not sure why it's stucked except for gas price issue. From my side, it worked when I tried yesterday.

You don't have to register resource ID and mint token every time. In the example with the ERC20 contract by ChainBridge, user doesn't have initial balance and it needs to call mint. You don't have to call mint if you have enough balance.

@Kourin1996 I tried with new accounts and every steps run successfully except the approve step (even I set high gas price). Did you try with Alchemy RPC ? Because Infura requires me to set Billing information before using polygon Mumbai testnet so I don't use it.  Using https://rpc-mumbai.matic.today always return 404 error ( I think due to rate limit)

@harry-nguyen22 When I tried it, I used Alchemy RPC API. Does your ERC20 contract implements approve method? Actually you don't have to use cb-sol-cli to approve. All you need to do is to approve transfer from your account by ERC20Handler.

@Kourin1996 I cleaned up everything and create new account and setup all stuff again and finally I can approved the ammout and deposit successfully 😁. Here is my address in Mumbai: https://mumbai.polygonscan.com/address/0xd255E1Bd52Bf5c9cF223b32B86a2616431Dff0F0
You can see that the mint, approve and deposit transaction (1 fail and 2 successfully) are successful but somehow the recipient account in my local edge chain still see 0.0 balance.
Does resourceId from Mumbai chain and localhost chain should be the same value? And we only need to start a relayer only when we need to make a deposit command, right ?

Yes, you need to set the same resource ID for the token you want to bridge in both chain.
And you need to start relayer before you send deposit tx. Please make sure ERC20Handler in the destination chain has enough balance to send the recipient account tokens in case of Lock/Release mode in the destination chain. If you've set burn/mint mode, ERC20Handler needs to be able to mint ERC20 token. Please check document for more details.

@Kourin1996 Thank for your answer. I don't set any specific mode so I think the default mode still lock/ release. Below is my contract info in destination chain (localhost chain)
================================================================
Url:        http://localhost:10002
Deployer:   0xB83f0058833BD87a9EddB9Ca896fE25fA2Adb657
Gas Limit:   8000000
Gas Price:   20000000
Deploy Cost: 0.0002906526

Options
=======
Chain Id:    100
Threshold:   1
Relayers:    0xB83f0058833BD87a9EddB9Ca896fE25fA2Adb657
Bridge Fee:  0
Expiry:      100

Contract Addresses
================================================================
Bridge:             0xca67123ffe12583E4C3DCa9533FBab05F69e6459
----------------------------------------------------------------
Erc20 Handler:      0xDeA74FCB008d43D1Bf1698FB0f20c957C7B13185
----------------------------------------------------------------
Erc721 Handler:     0x6253E277Ce043Cbdb3739eE8Cc16A62579DC4108
----------------------------------------------------------------
Generic Handler:    0x484CCb6f99ee125989da50E75298f161bA591Ecb
----------------------------------------------------------------
Erc20:              0x29371E3050bB4851f7530dFF128d127cA3524544
----------------------------------------------------------------
Erc721:             0x6c416573b8DCF66bd61A41De20352eFb64766607
----------------------------------------------------------------
Centrifuge Asset:   Not Deployed
----------------------------------------------------------------
WETC:               Not Deployed
================================================================

You mean the address 0xDeA74FCB008d43D1Bf1698FB0f20c957C7B13185 should have enough balance to be able to transfer the token to the recipient account? I tried to use my local test account ( 890 ETH ) to send 10 eth to that address but the transaction failed. I check activity logs but it just said "encounter error"? I can use metamask to transfer the ETH ( get from primine account ) normally without any issues.

@Kourin1996 I got your point. So the handler only accepted the token from the erc20 contract it registered before. So we need to mint it first and then transfer it to the handler address. I can make a deposit successful! Thank you very much.


Chain.fillet() should originate

Chain.fillet does not work when parent models have different origins

Also it should not attempt to fillet arcs which are members of the same Bézier curve


updates

I think this PR might've ended up in the wrong repo... regardless, I must respectfully decline. :sweat_smile:


add x509/http pkg and rework

add a new x509 pkg following Go's api as usual
add http pkg with a dummy OpenSSL RoundTripper
/cc @vbatts
Signed-off-by: Antonio Murdaca<EMAIL_ADDRESS>

reworked the repo a bit to have crypto/ and http where I'm currently working

still failing on that weird rand compile issue
On Mon, Oct 12, 2015 at 8:40 AM, Antonio Murdaca<EMAIL_ADDRESS>wrote:

reworked the repo a bit to have crypto/ and http where I'm currently
working
—
Reply to this email directly or view it on GitHub
https://github.com/shanemhansen/gossl/pull/14#issuecomment-147384698.

:( couldn't found the issue there yet, still digging, I have a bunch of local changes I should push here though

@vbatts @shanemhansen can we merge this as is for now since I'm already importing it in docker to back it up with openssl and the diff is becoming pretty big? No broken changes has been made here, it's just a clean setup following go's crypto pkg

I'm still researching the failures in tests tho

Bug should be fixed now -- pls Travis -- so I'd say this is good to go now :)

alright fixed! :tada:

w00t!


Unable to create a new database.

I want a method to create a new database, just like the following one(https://github.com/mapbox/node-sqlite3/wiki/API#new-sqlite3databasefilename-mode-callback).
new sqlite3.Database(filename, [mode], [callback])

It doesn't seem I can do it. There is no such API.

Well, I found a way to work around it, just create a new empty file.
 if (!await pathExists(dbPath)) {
    await createFile((dbPath);
}
const db = await sqlite.open(dbPath);


privacy.php and terms.php are unprofessional

The text on these pages could be worded much better.

You said you found a tool to fix this - why don't you go ahead and do that yourself.


Perlless profile conflicts with apparmor

Describe the bug
When enabling apparmor, eg. through the hardened profile, it installs an apparmor profile for perl. This triggers the forbiddenDependenciesRegex check for the string perl, making it impossible to use both.
I do appreciate the idea of actually making sure that there is no perl available in the resulting closure, but I fear that the current check may be a bit to ambitious to be practical.
Notify maintainers
@nikstur

Maybe it would be more to the point to check for a major version number, eg. perl-(5|6) ?

for app armor specifically the regex could also be adjusted too (?<!abstractions\/)perl as a negative lookbehind, or we could look specifically for bin/perl instead


SCM is broken in GOCART2G GEOSgcm

I'm going to CC in a lot of people because I just try and keep the SCM running. I have no idea how to use it scientifically.
cc: @narnold1 @wmputman @amdasilva @pcolarco @sdrabenh

The SCM currently uses GOCART.data as AERO_PROVIDER. However, since https://github.com/GEOS-ESM/GEOSgcm_App/pull/264 removed the option for GOCART.data and did not introduce a GOCART2G.data, I no longer have any template on how one should set up for a data-driven GOCART run. When you run you get:
 ERROR: cannot find export AERO
pe=00000 FAIL at line=07054    MAPL_Generic.F90                         <unknown error>
pe=00000 FAIL at line=00638    MAPL_Generic.F90                         <status=-1>
pe=00000 FAIL at line=00556    MAPL_Generic.F90                         <status=-1>
PHYSICS::SetServices                          1494

As a blind attempt, I did try to just to set the AERO_PROVIDER to GOCART2G and edited GEOS_ChemGridComp.rc to look like the full model does by enabling GOCART, GOCART2G, ACHEM, HEMCO. Note that I'm not even sure one should do this for the SCM but... 🤷🏼
Doing this, (and adding a symlink to ExtData/chemistry) at least one case of the SCM fails because of missing data. I tried the arm_97jul case and it fell apart in ExtData:
   EXTDATA: ---- 00080: pSO2_OCS
   EXTDATA: *******************************************************

pe=00000 FAIL at line=02143    ExtDataGridCompMod.F90                   <From ExtData/chemistry/MERRA2GMI/v0.0.0/L72/MERRA2_GMI.tavg24_3d_dac_Nv.x576_y361_t12.%y4.nc4 could not find time no extrapolation>
pe=00000 FAIL at line=01005    ExtDataGridCompMod.F90                   <status=1>
pe=00000 FAIL at line=01799    MAPL_Generic.F90                         <status=1>
pe=00000 FAIL at line=00767    MAPL_CapGridComp.F90                     <status=1>

The issue is, I think (I'll name drop @bena-nasa), that the MERRA2_GMI files are from 1999 on, but arm_97jul (as it says) is in 1997. So ExtData fails. It looks like many cases are not after 1999:
$ fd cap_restart -X cat | sort
19921101 000000
19921101 060000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19930715 000000
19970618 000000
19970618 000000
19970618 233000
19970618 233000
19980501 000000
19980501 000000
19980507 210000
19980507 210000

At this point, a scientist on @GEOS-ESM/aerosol-team or someone needs to figure this out. Issues I see at least are:

Can you use GOCART2G with SCM or do you need to use data-driven?

If full GOCART2G is possible, it looks like we might not have enough data to run at least some cases.
If data-driven GOCART2G, well, someone needs to provide an example of how to set one up since gcm_setup can no longer provide that.

No matter what, someone then needs to make sure the model isn't outputting garbage.

Once all this is known, I can work on fixing up scm_setup, the input files, etc. to try and get it to work, but until then, the SCM is broken.

@mathomp4 I don't know what to tell you. Haven't run the SCM in 15+ years and don't know who is using it. I would defer to @amdasilva to prioritize this, but not clear I'm even capable of helping (let alone being able to do something on a useful time scale to you).

@pcolarco the issue @mathomp4 mentioned is also true for any gcm run prior to 1999. We need ExtData/chemistry/MERRA2GMI/v0.0.0/L72/MERRA2_GMI.tavg24_3d_dac_Nv.x576_y361_t12.*.nc4 files backfilled to 1979 in order to run the normal gcm. Do you have those files somewhere that Rob can copy?

OK, I misunderstood the issue. I should be able to produce those files. They derive from MERRA-2 GMI which goes back to 1980. I will "pad" to 1979 with the 1980 values and bring up to 1999. This will take some time as the files are large to produce.

@pcolarco Thanks. Hopefully that's all that's needed for it to run.
Well, maybe not run scientifically correct, but doesn't crash correct which is what I care about. 😄

I have pushed the MERRA2_GMI oxidant fields to $SHARE/dasilva/fvInput/MERRA2GMI/L72. They now span 1979 - 2025. Notes:
1979 is just a copy of 1980 since that is the first year of the MERRA2GMI run.
2020 and later are just copies of 2019 since that is the last year of the MERRA2GMI run.
I will ask Arlindo and Ravi to move around and copy onto higher vertical resolution files.

This is now "fixed" in #302 . Note that the SCM could be outputting garbage, but it runs.


Empty log and error window in web user interface

Always showing loader and not load any information.

Сan you give an answer?

Сan you give an answer?

@morbillius did you find a solution to this? I know that logs are stored in projects/log-dir, but the html filebox tells me it's searching for it in get-log/data/ which doesn't exist.


pkgs.dzgui is missing

Hi there, I am unable to activate dzgui using programs.dzgui.enable = true, and the only way it worked was by adding this flake as an overlay and then use the dzgui package itself.
I am on NixOS unstable.
If you need additional infos, don't hesitate to request them.

Hi, thanks for opening the issue.
Could you please commit the configuration where you are encountering this problem (to a separate branch in your configuration repo, probably), and share it here?
I'm trying to figure out what could be causing this problem, and made some changes that may or may not fix it for you. If you could check if the problem has been fixed, I'd be grateful. :smiley:

Hey man, thanks, your commit seemed to fix it!


Home Screen is blank

Home screen doesn't show any message if no skill is created

Hey i would like to work on this issue under hacktoberfest  if you can assign it to me

@tejaswi755 assigned


Add ipython/bpython support to flask shell

@mitsuhiko

It's a nice idea and it came up before, but this should be in a separate package that registers a setuptools entrypoint that hooks into the Flask cli :)


bug: Recipe builds are failing on signing step

hey all after following the manual setup directions I'm getting the following error back from actions
    DOCKER_METADATA_OUTPUT_BAKE_FILE: /tmp/docker-actions-toolkit-dOlNP5/docker-metadata-action-bake.json     TAGS: sha256:8a80680feaecbd9226ec7894aff4f6bdd449e3a565e4f76cc03ba79b1927c99e     COSIGN_EXPERIMENTAL: false     COSIGN_PRIVATE_KEY: *** 653 cosign.key Error: signing [ghcr.io/j3rr1ck/Gnos_Linux_Nvidia@sha256:8a80680feaecbd9226ec7894aff4f6bdd449e3a565e4f76cc03ba79b1927c99e]: getting signer: reading key: unsupported pem type: ENCRYPTED SIGSTORE PRIVATE KEY main.go:74: error during command execution: signing [ghcr.io/j3rr1ck/Gnos_Linux_Nvidia@sha256:8a80680feaecbd9226ec7894aff4f6bdd449e3a565e4f76cc03ba79b1927c99e]: getting signer: reading key: unsupported pem type: ENCRYPTED SIGSTORE PRIVATE KEY Error: Process completed with exit code 1.
I tried swapping in both passworded and non-passworded keys just to see if that was the problem but the error is consistent across both
am I missing something obvious here?

I am also having issues with the signing. At first, my keys were coming out encrypted, which the docs say won't work. You can tell they are encrypted when the .key starts with BEGIN ENCRYPTED SIGSTORE PRIVATE KEY.
I, somehow, got an unencrypted key pair, but I still can't sign the image. The error I am getting is:
Error: signing [ghcr.io/johnrellim/imageName@hash getting signer: reading key: unsupported pem type: SIGSTORE PRIVATE KEY
main.go:74: error during command execution: signing [ghcr.io/johnrellim/image@hash getting signer: reading key: unsupported pem type: SIGSTORE PRIVATE KEY

Are there any settings within Cosign that need to be adjusted? Is there an issue with the new versions? I reinstalled it recently and am currently using  on this version of Cosign:
GitVersion:    v2.0.1
GitCommit:     8faaee4d2b5f65678eb0831a8a3d5990a0271d3a
GitTreeState:  clean
BuildDate:     2023-04-06T19:10:33Z

Any ideas?

I'm also getting an error with the signing step.

COSIGN_PRIVATE_KEY: ***
653 cosign.key
Error: signing [ghcr.io/vaparetia/custom-gnome-theme@sha256:3f02d5cd6fd2de60d4736b6a67a054f1a0926cf9e41adfbce344c42846e67579]: getting signer: reading key: unsupported pem type: ENCRYPTED SIGSTORE PRIVATE KEY
main.go:74: error during command execution: signing [ghcr.io/vaparetia/custom-gnome-theme@sha256:3f02d5cd6fd2de60d4736b6a67a054f1a0926cf9e41adfbce344c42846e67579]: getting signer: reading key: unsupported pem type: ENCRYPTED SIGSTORE PRIVATE KEY
Error: Process completed with exit code 1.

Not entirely sure how to remedy this issue.
I'm a noob so, I'm basically useless lol :laughing:

I ran into this problem too and it seems cosign has changed the format of the key file but the signing (github task?) doesn't support it yet.
I worked it around by editing my key replacing ENCRYPTED SIGSTORE PRIVATE KEY with ENCRYPTED COSIGN PRIVATE KEY (the old format) and uploading the key again.

I can confirm that making the above change to the SIGNING_SECRET works.
Another possible workaround would be to use an earlier version of Cosign than 2.0.1.

I can confirm that making the above change to the SIGNING_SECRET works.
Another possible workaround would be to use an earlier version of Cosign than 2.0.1.

Changing to Cosign 2.0.0 worked like a charm.

I ran into this problem too and it seems cosign has changed the format of the key file but the signing (github task?) doesn't support it yet.
I worked it around by editing my key replacing ENCRYPTED SIGSTORE PRIVATE KEY with ENCRYPTED COSIGN PRIVATE KEY (the old format) and uploading the key again.

That didn't work for me unfortunately, but installing cosign 2.0.0 and resigning did fix my issue.
All, Thanks for pointing me in the right direction!

I was trying to generate my key on Windows 10 using Powershell. When it prompts to press Enter instead of typing a password I was doing just that, following the instruction on the site, but I believe that Windows 10 powershell passes a CRLF return character instead of a LF character. Maybe that affected the password entering step. Another thing to note is that git prompted me that it would replace LF to CRLF characters on my cosign.pub next time it touched it ( I believe that would be on the git-push ). When I moved the signing steps on my Linux machine, git stopped complaining for CRLF characters and I managed to finally pass signing the build.
To sum what I am trying to say. Either cosing on windows (I used cosign 2.0.1) has issues or the CRLF LF difference comes at play.

I didn't even think somebody would try this on Windows... I'd recommend WSL for most dev things, just in case.

I marked this as help wanted, because I am not that familiar with the whole cosign & GHA part of things. If you can report with information not available here or provide more clarity, please comment, but don't just +1 (you could react to this comment or smth).
I encourage you to manually regenerate the cosign keys if this problem happens to you, and only comment afterward if the issue is not fixed.


Substreams GUI improvement

Currently, substreams GUI allows navigating from the current block to adjacent blocks only. It would be great to add a feature that allows jumping to any processed block.

@sahra-karakoc I merged your issue with other GUI improvements: https://github.com/streamingfast/substreams/issues/150


feat: add docker build

I've added a Dockerfile to build the exporter. I've also added a github action that is executed when you push a new tag. The image is pushed to latest and tag on github registry. You can find an exemple on my fork for the result.

I like that you have added this, I was looking at doing this myself but put it off, looks good.
Some thoughts.
We can use environment variables instead of CMD with default MODE=http.
It does not make sense to use the socket mode in a Docker container.
I can see this being used in a Kubernetes workflow as a sidecar container in a pod that points to localhost of another kea-dhcp container.
We could skip using CMD and directly call "kea-exporter" from the ENTRYPOINT and configure with env-vars instead.
Example Dockerfile
FROM python:3.12-slim

WORKDIR /usr/src/app

ENV MODE="http"
ENV ADDRESS="<IP_ADDRESS>"
ENV PORT=9547
ENV TARGET="http://<IP_ADDRESS>:8080"

COPY . .

RUN pip install --no-cache-dir -e .

EXPOSE 9547/tcp

ENTRYPOINT ["kea-exporter"]

Hello,
Thank you for your feedback. My initial intention was to allow the user to choose between socket and http mode since there are no compatibility issues, but I can change this.
I will also use this in a Kubernetes environment but not as a sidecar, as I'm operating within a hybrid setup with virtual machines.
I'm not seeing a scenario where you would need to alter the entrypoint in this configuration. However, if you do find a need for it, I can add a default environment variable and incorporate it into the entrypoint, like so:
ENV PORT=9547
ENTRYPOINT ["kea-exporter", "-p", "$PORT"]

The -a <IP_ADDRESS> option can be overridden, as the Prometheus start_http_server function doesn't support dual-stack binding with ::, and in a Kubernetes environment where only IPv6 is used, you would need to override this parameter.
Regards,

Thats true, you could have a shared volume and run socket mode between containers.
We should leave every option open for the user to choose.👍

What do you think about this layout @M0NsTeRRR?
You can still add optional command line arguments or use env-vars at runtime.
As click.option specifies the default port in cli.py so we do not actually need to specify it with the entrypoint.
FROM python:3.12-slim

WORKDIR /usr/src/app

COPY . .

RUN pip install --no-cache-dir -e .

EXPOSE 9547/tcp

ENTRYPOINT ["kea-exporter"]

I've added suggested change, i've removed also the /tcp as it's the default behavior for EXPOSE

Sure, we have some more labels to add if we want to follow opencontainers specification.
I also need to look about USER property as this container doesn't need root permission to run.

I've added default labels and the image is run with a non root user.
Regarding org.opencontainers.image.authors label, I think @mweinelt should put their name here as I'm only a contributor :)

Thanks for working on this. Can you also update the README.md (maybe just above "Features") with information relating to the resulting image? And can we make sure the resulting image will also be built for arm64?

I think @mweinelt should put their name here

Feel free to add "Martin Weinelt<EMAIL_ADDRESS>into the metadata as needed.

Thanks for working on this. Can you also update the README.md (maybe just above "Features") with information relating to the resulting image?
Sure :)

And can we make sure the resulting image will also be built for arm64?
Good catch, I completely forgot ARM users, will look into in

We should be good. I let you review it @mweinelt

Needs a rebase.

Should be good

Hello,
Still waiting feedback regarding the 3 conversations, so we can merge it and tag 0.6.0 :).
Regards,

Sorry for the late reply. None of the things are realdealbreakers to me and were just little things that popped up in my head.
Carry on :)

Hello,
No worries, thanks for your review. I was actually also waiting for feedback from @mweinelt, regarding the ongoing discussion and what they would like to do. 😊

I released v0.6.0 earlier today and the CI run seems to have completed successfully.
https://github.com/mweinelt/kea-exporter/actions/runs/8347134319

@mweinelt package is not publicy available.
docker pull ghcr.io/mweinelt/kea-exporter
Using default tag: latest
Error response from daemon: Head "https://ghcr.io/v2/mweinelt/kea-exporter/manifests/latest": unauthorized

I think you have disabled Inherit access from source repository  in https://github.com/settings/packages
I think you can configure it here https://github.com/mweinelt/kea-exporter/pkgs/container/kea-exporter

For some reason the package is private. I have inherited access from source repo enabled.

If you have change the global settings, it's apply only to new repository : This setting will be applied to new Container, npm, rubygems and NuGet packages.
I still have a 401

Wow, did not expect visibilty to be a danger zone setting.

Oh okay, lot of menu without explanations on github documentation ^^.
It works now ! :)


[Feature] Improved structured text UI

Implements the new structure UI that is discussed in GN-4247
Some comments to hopefully help in working further on this.
This only works with the changes made in PR editor #788.
Locally you can test this branch by pulling the latest ember-rdfa-editor code (master) and running:

npm link in editor ember-rdfa-editor repo
npm link @lblod/ember-rdfa-editor in ember-rdfa-editor-lblod-plugins (this repo)

code structure
nodes are defined in addon/plugins/structured-blocks-plugin/nodes with addon/plugins/structured-blocks-plugin/nodes/config.ts containing the definition of the structure and everything that the nodes have in common. The other files have specific information for the node itself.
Everything uses an ember-node. These ember components that are loaded in are defined in addon/components/structured-blocks-plugin.
The main used component is structured-block.
UI
The UI is not build with a grid, but instead line per line. Note: a grid like structure might be better. The hard part is structuring in a way that hover and shown-hover things have influence on the correct parts.
Every line is: space for hovered add button - space for prefix (e.g. Hoofdstuk 1. - space for input field (currently text field)  - space for hovered remove button.
Empty divs with the same classes are used to create the same indentation. See sub-row that basically implements this structure. With sub-row you can add a row that is indented to the left/right.
This definitely needs some rework, but with usage of sub-row the overhead is really small and for now it is quite easy to use.
CSS
AU classes are used as much as possible, but often they need some extra changes to follow the (better) figma example.
Sometimes a div might contain a class that isn't needed.
smelly code
Because of refactoring (and focusing on POC), some code might be unused or have a weird structure. E.g. empty divs (although sometimes these have influence on the UI...)
Todos

[ ] check if the design is good enough

[ ] change the input boxes for title and such to text boxes (multiline) or directly to embedded prosemirror

note that currently the text in input boxes is saved in attrs. If using embedded prosemirror, an extra node is needed for the embedded to save its info to. -> definitely possible, but not a trivial change.

[ ] Text inside structure_content (the box under an article) should be added as prosemirror textnodes, but this is not the case. So it doesn't really get "saved" to prosemirror.

[x] The grey-box hover works, but the add and remove buttons are always show for the current hovered node and all its children. Some custom hover logic is needed here.

[ ] the other views are broken (at this point of testing, link plugin also seems to not work anymore). Don't merge this before fixing these influences on the other pages/plugins.

possible extras (not limited to this)

[ ] change the UI properties (showRemoveBorder, addBlockHover) to be tracked properties in ember component, instead of saving them in attrs. Note: simple change, the important part is checking that this doesn't break in edge cases. (see PR editor #788)
[ ] draggable functionality
[ ] auto numbering of everything (e.g. Artikel 1., Artikel 2. etc)

this is running our ci all the time, so ill close the pr but leave up the branch


Add sub pages under "About"

Frequently asked questions (internal - methodology/faqs or external https://www.whitehouse.gov/wp-content/uploads/2022/02/CEQ-CEJST-QandA.pdf TBD)

About
Frequently asked questions
Engagement calendar

@vim-usds you will need two svgs for the FAQ page. attaching them here.

@vim-usds vim needs these graphics for the FAQ page


Code-completion should recognize enum-like classes and constructor like functions.

The current code completion suggests EnumClass.foo in an EnumClass typed context and Foo.constructorName in a Foo typed context. For all other static members of a class, you have to first write the ClassName before code completion will suggest the static members.
I suggest expanding this favoritism:

suggesting any static final variable declared on SomeType with a type that is a sub-type of SomeType in a SomeType typed context, and
suggest any static method on SomeType with a return type that is a sub-type of SomeType in a SomeType typed context.

That will treat any final static value (or even just the const ones) on a class with that class's type as a canonical instance of the class, and treat any static method on a class returning that class's type as a factory method.
This would allow user written enum classes to match language level enums in discoverability and usability, and it would allow allow factory functions to be recognized without them needing to be factory constructors.
It will also, for example, mean that double x =  will suggest double.nan and double.parse, which are both classical examples of double expression you might want to complete with. Parse functions currently have worse discoverability than constructors (say, for Uri), even if they are likely to be the correct function to call in many cases.

What about static final or const fields of type int, double, num, String, bool for enum-like classes of those types?

@zoechi I don't see how to find those. The hint provided above is the context type, so you can check that exact type for constants or factories. If the context type is int, I have no clue where to look for constants of type int (there are likely many).
If we had type aliases with static members, say;
typedef MyAliasForInt = int {
   static const foo = 1;
   static const bar = -1;
}

then we might be able to complete those in a MyAliasForInt context. There is no MyAliasForInt type at the language level (it's just an alias for int), but static tools may be clever enough to make the distinction.

@lrhn If it is only about autocompletion, then an annotation could do that
@Enum<int>()
abstract class MyEnum {
  const int foo = 1;
  const int bar = 2;

  const bool otherConstMember = false;

  MyEnum._();
}

MyE should show the suggestions
MyEnum.foo
MyEnum.bar

This would make writing of the already verbose old-style enums even more cumbersome though.
But writing usually is less of an issue than discoverability and readability.
Type alias looks good, but I'd still prefer having the full capabilities of Dart classes available for enums.
The ideal situation in my opinion would be if
enum  Color {
  red, green, blue
}

would lead to a class like
abstract class Color implements Enum {
  static const int red = Color(0);
  static const int green = Color(1);
  static const int blue = Color(2);

  static const values = [red, green, blue];

  // value could be `index` but might be weird if other types then `int` should be supported here
  final int value;
  const Color._(this.value);

  factory Color.fromValue(int value) => values.firstWhere((v) => e.value == value); // or a switch/case for performance

  @override
  String toString() => '${super.toString()} - $value';
}

and would allow to add everything that a normal class would allow
except what would prevent the class from being used as enum
like non-const constructors and non-final fields.
enum  Color {
  red = Color(2, 'Red'),
  green = Color(5, 'Green'),
  blue = Color(9, 'Blue');

  // additional static members
  static const lightColors = [red, green];

  // additional field
  final String name;

  // a different type for `value` (index in current Dart enums) might be useful as well not sure
  // final double value;

  // custom default enum constructor
  // `const` could be implicit
  const Color._(this.value, this.name);

  // additional factory constructor
  factory Color.fromName(String name) => values.firstWhere((v) => e.name == name);

  // additional getter or method
  String get nameAndValue => '$name - $value'

  // custom toString
  @override
  String toString() => 'Color $name (value)';
}

For issues discussed in https://github.com/dart-lang/language/issues/50
(I don't know if I fully understand the related requirements though)
If int values should be used instead of instance it could be Color.blue.value
and foo(Color.fromValue(5)) (referring to https://github.com/dart-lang/language/issues/50#issuecomment-430763406)
I don't know if there are plans to add support for custom casts to Dart that would allow
void foo(Color a) {
  print(a.name);
}

void main() {
  foo(5);
}

to make conversion between int and Color implicit.

What I would like to add to this, if we could annotate classes with enum, is to provide for type checking Like enums do.
If I have a class
class Colors {
  static const String red = "red";
  static const String blue = "blue";
}

I would like to run a check to see if the value I'm encountering was from my Colors class, even though it evaluates to a String. Currently, I have to assert that it's a String, which will pass if the user passes "Yellow" as a parameter, even thought that's not part of my enum class

Something like this? I have no idea about any of the technical details about implementation, but being able to extend an enum would be great. https://pub.dartlang.org/documentation/enums/latest/enums/Enum-class.html

I think this is partially done in #40699. If the servers "Suggestion Sets" are enabled and the class is in another file, the static members are suggested via the suggestion sets:

It doesn't work if they're in the same file (see MyColor2) or imported when suggestion sets are disabled yet though.

With b253c4a004187f0c33082d17618e509956d048fb (added for the new / current completion protocol) we suggest any static fields.

The fields don't have to be declared in the same class as the context type, not A.foo above. Instead we use the type of the field to give it higher relevance if if it matches the context type, so it works mostly automatically.
If you know the name of the constant, you can start typing it.

What is still missing.

Maybe suggesting static methods. I suspect that there are not so many of them to significantly increase the number of suggestions. And I like the consistency with (factory) constructors.
Filtering not only by the name of the constant, but also by the name of the class that contains it. So, typing dou^ would keep only constants from double.
It looks that currently we give local variables so much relevance that this pushes type matching suggestions down. At least in some cases this feel unfortunate, e.g. below, see how much I had to scroll and skip local variables to get to the alignment constants. I will probably need to gather some statistics to decide.

Maybe suggesting static methods.

It would be good to get some data here to see how many additional suggestions we're likely to introduce.

Filtering not only by the name of the constant ...

I'd love to see that. I don't know whether clients currently allow for that, though.

It looks that currently we give local variables so much relevance that this pushes type matching suggestions down.

The context type (type matching) should also be used for local variables, so I would have expected that to improve the situation. (And maybe it does, just not by enough.)
But I agree that we need to do some statistical analysis. The current relevance scores are based on very old-style code and need to be updated.

Filtering not only by the name of the constant, but also by the name of the class that contains it. So, typing dou^ would keep only constants from double.

I'm not sure if I understand this correctly, do you mean filtering for things where either the class or constant names contain the prefix (so typing dou^ would keep both constants from double and constants containing dou from other classes)? Otherwise it seems like typing "m" (as in your middle screenshot) would no longer work?

I'd love to see that. I don't know whether clients currently allow for that, though.

VS Code will always apply a fuzzy search over the "filterText" (which defaults to the label if we don't set it.. usually we set it to the label minus any signature/details). That means if you type "abc" it would match "A.bc" but we can't tell it that it must match entirely in one side of the dot (this is something I've suggested before - for ex. having filterText accept an array, but sadly nothing has come of it!)

@alexander-doroshko Is this something we could also do in IntelliJ?

Is this something we could also do in IntelliJ?

Do you mean showing one text in the popup and using another text for filtering? Yes, IntelliJ Platfom API supports it. Here's a nice example from JavaScript:

That's cool!
But I think Konstantin's thought was about being able to suggest something like double.maxFinite in such a way that it would match whether the user has started to type double or maxFinite.

We have alternatives how to filter MyClass.someField:

Filter only by the text after ., so you can write soF. This is what we do now.
Filter only by the text before ., so you can write MyC.
Filter by both separately, so you can write either MyC or soF.
Filter by both merged, full text, so you can write MyF.

The option (4) seem to be most flexible.
One issue with this, is that fuzzy matching implementation that we have in DAS, requires that the first letter must match. So, if we match soF against MyClass.someField, it will always fail. Maybe DAS should re-try with (1)? Or maybe we should make fuzzy matcher not to fail if the first letter does not match.
But this also depends on what clients will do for fuzzy matching.
It does not matter what DAS will do, if the client will have different rules and filter out suggestions.

It does not matter what DAS will do, if the client will have different rules and filter out suggestions.

That's not entirely true. :-) If DAS filters out a suggestion then the client can't decide to not filter it out.
So I think on the server side the right behavior is to take into account how the clients will filter. Hence my interest in knowing how the clients will behave.

Just BTW, as far as I remember, both IntelliJ and the Analysis Server already support all 3 different strings: to show in the popup, to use for filtering, and to insert if selected.

suggest something like double.maxFinite in such a way that it would match whether the user has started to type double or maxFinite

We have alternatives how to filter MyClass.someField...

I think IntelliJ already supports all 4 alternatives. Here are screenshots (with mocked result from DAS, I changed CompletionSuggestion.getCompletion() to "MyClass.someField")

If DAS filters out a suggestion then the client can't decide to not filter it out.

true :)


Implementing Init

Hi!  Any reason to leave init?(coder) unimplemented? Would this:
public required init?(coder aDecoder: NSCoder) {
    super.init(coder: aDecoder)
}

cause any problems?

Just not using storyboards to call this. If you are calling from storyboards and there needs to be additions, both in code and readme, feel free to implement them and send a pull request.

The solution by @paperkettle works like a charm

Feel free to send a pull request

Fixed this in here:
https://github.com/goktugyil/EZSwipeController/commit/18427b8500d856c84db2880554fda34cfb9a09ee


fix: UI: Correct useEffect() re-rendering during file import.

Fixes: #3781
@igarashitm - please try your import with this change.  I could not reproduce your situation but corrected useEffect rerendering triggering which should catch your issue.

Seems it contains a leftover from dependabot PRs in yarn.lock, let me clean it up first.

Nope it doesn't work - even reload didn't help this time.

Hmm, it's inconsistent... after it reproduced, I closed the browser tab, restarted the stasndalone with mvn clean spring-boot:run, then opened a new tab, then it worked. Then yet again, I closed the browser tab, restarted the standalone, opened a new tab, then the issue reproduced again.

Will close this PR for now since it seems to have no effect (but does appear to be more efficient).  I went to my local atlasmap/standalone and mvn clean spring-boot:run followed by http://localhost:8585/ in a separate browser window in various sequences and can not get the schema source to fail to load (still using the fix here).  I also tried the fully standalone java -jar ~/git-clone/atlasmap/standalone/target/atlasmap-standalone.jar as well as the standard storybook atlasmap instance - also unable to get it to fail for me.  Perhaps a browser instance issue? Version 90.0.4430.212 (Official Build) (64-bit)


Export Resultset hotkey ?

Hi,
It seems the Export Resultset command is not available for hotkey customization.
Am I not looking for the proper name or is it really not available ?
Would you add it ?

Thanks,

Agreed, this should be a real customizable command.

Perfect, thanks.


#195: race condition fix

Some parts of the code neglected the fact that plugins are added during runtime.
The stream- and plugin-registry now use RWMutexes (as they act like caches) for improved read performance.

Coverage decreased (-0.09%) to 64.964% when pulling ad741e873ca01eaa5f8a744a409d0c140cf77f4b on fix/195_race_condition into e2ca53cd703523b5132acc0f2cf4e407be156974 on master.


Missing libraries for compile some projects

Hi,
I've realized that there are some missing libraries to be linked at the time to compile the next projects:

beam-node
node_1_test
node_test
wallet_db_test
offline
beam-wallet

libcrypto.lib (from OpenSSL) and Crypt32.lib should be added to the linker for them to compile and link properly.
EDIT: I guess the problem comes from https://github.com/beam-mw/beam/blob/master/sqlite/CMakeLists.txt#L32. I'm setting the path for libssl and libcrypto instead of using the recommended openSSL sources you guys mention in the readme:

Hi,
We are using these prebuilt binaries for OpenSSL https://slproweb.com/products/Win32OpenSSL.html (especially Win64 OpenSSL v1.1.0i) to build the project, and CMake finds it if you set OPENSSL_ROOT_DIR system variable.
Where did you get your OpenSSL lib?
Thank you.

Please try to remove all these dlls and add QT5 and Boost paths to the Path Environment Variable
...\libs\qt511\5.11.1\msvc2017_64\bin
...\libs\boost_1_68_0\lib64-msvc-14.1
Also, could you please start Beam using the debugger in Visual Studio to understand the problem?
Thank you.

I would suggest that, for Windows builds, it would be best to have a documentation similar to one here:
bitcoin msvc build readme
I apologize if there already is similar page and I haven't seen it.
I am still working on building various dependencies, not knowing yet if they should built as static libs or no - that's why I think that instructions above would be useful, that is, something like:

PS >.\vcpkg install boost:x64-windows-static     libevent:x64-windows-static
openssl:x64-windows-static     zeromq:x64-windows-static
berkeleydb:x64-windows-static     secp256k1:x64-windows-static
leveldb:x64-windows-static

Good thing is that with the latest VS2017 we don't need to generate MSVC vxproj and sln files, CMake should be suffincient. Still working on it, however.

At this point I have built all programs and libraries that don't require QT5, using VS2017 CMake support and building all the dependencies via vcpkg.
Here are the steps:

clone and build vcpkg
build boost using vcpkg
build openssl using vcpkg
make the results visible to Beam CMake  via VS2017 CMakeSettingsjson file - needs this to be added:
"variables": [
{
"name": "CMAKE_TOOLCHAIN_FILE",
"value": "\scripts\buildsystems\vcpkg.cmake"
}
]

Wallet project needed Windows bcrypt library (actually boost that it uses requires it) and was failing with unresolved externals. I had to add this to wallet CMakeLists.txt:
if(MSVC)
target_link_libraries(wallet bcrypt)
endif()
Issue is mentioned here: bcrypt issue
Unfortunately vcpkg builds QT version 5.9 and this is giving me troubles building QT dependent projects.
Still working on it.

@Toninoso Vcpkg recently updated to Qt 5.11.2 which might help fix some of the issues.


Pi 3B+ & 4B cannot send SPI write over 128 bytes

Hello, I am using the pigpio C interface to drive an OLED, and cannot send transfers of over 128 bytes using the spi_write command. Apart from this spi_write works, as well as all other GPIO pin use for D/C and RES flags.
I can confirm this is not a hardware / OLED issue as I have a logic analyzer hooked up and increasing the transfer amount over 128 bytes results in nothing being sent.
Initialisation of the pigpio daemon:
 initNewGPIO = pigpio_start(NULL, NULL); // Initialise pigpio
Opening SPI interface:
portNewWrite = spi_open(initNewGPIO, 0, 1000000, 0); // Channel, Baud, Flags
Regular call of the spi_write function (preceded by setting D/C flag and filling buffer with desired data):
spi_write(initNewGPIO, portNewWrite, data, count); // where data[] is a char* array of 8192 bytes, and count is a sizeof(data)
The previous initialisation and calls have no output on the SPI lines at all, as can be seen in the screenshot below -- the initialisation happens, followed by no activity on the MOSI line. Anything above 128 bytes has this effect, whereas 128 bytes outputs normally.
I am relatively new to pigpio and spi-dev and would appreciate any troubleshooting tips to be able to better understand what's going on here. Is this an issue with my baud rate, or the SPI mode that I am using?
Please let me know if I should provide additional information to make the issue more clear.
Thanks,
H

There are no D/C, RES flags.  I don't understand how you are using the library.
spi_flags consists of the least significant 22 bits.

21 20 19 18 17 16 15 14 13 12 11 10  9  8  7  6  5  4  3  2  1  0
 b  b  b  b  b  b  R  T  n  n  n  n  W  A u2 u1 u0 p2 p1 p0  m  m

I took a screenshot which didn't show sending the RES flags. I am using the pigpiod_if2 daemon to open SPI port 0, and manually setting RES and DC flags when sending the data buffer.

So when you say flags (RES and DC) you mean pins?  I'm not familiar with that terminology associated with SPI.  The standard pins are SCK, MOSI, MISO and CE.  I'm still confused.

Yes I mean pins! I manually set them via GPIO commands. Yep, however I am only using SCK, CE, and MISO. You can see in the screenshot that these standard pins are functioning as expected, as I am able to use the OLED without issue.
You could be right that the flags are the issue, I need to have another look.

Perhaps showing a more complete listing of your script will help.

First, the framebuffer is declared, and the SPI initialisation function is run.
char *fb = (char *) calloc(8192, sizeof(char));

void NHD_initSPI()
{
    initNewGPIO = pigpio_start(NULL, NULL); // Initialise pigpio

    if (initNewGPIO < 0)
    {
        printf("Failed to initialise GPIO for OLED! [NHD]\n");
        printf("PIGPIO error code: %d\n", initNewGPIO);
    }
    else
    {
        printf("Initialised GPIO for OLED [NHD]\n");
    }

    //set_mode(initNewGPIO, BUSY, PI_INPUT); // SET 24 (BUSY) to input
    set_mode(initNewGPIO, RES, PI_OUTPUT); // SET 25 (RESET) to output
    set_mode(initNewGPIO, DC, PI_OUTPUT); // SET 24 (D/C) to output
    set_mode(initNewGPIO, CS, PI_OUTPUT); // SET 8 (CS) to output
    set_mode(initNewGPIO, MOSI, PI_OUTPUT); // SET 10 (MOSI) to output
    set_mode(initNewGPIO, CLK, PI_OUTPUT); // SET 11 (CLK) to output

    set_pull_up_down(initNewGPIO, DC, PI_PUD_UP);
    set_pull_up_down(initNewGPIO, RES, PI_PUD_UP);

    portNewWrite = spi_open(initNewGPIO, 0, 1000000, 0); // Channel, Baud, Flags
    printf("Opened SPI on Channel 0 [NHD] \n");
}

Then we can write a single pixel into the framebuffer (not to screen yet). Each pixel on the SSD1322 is a nibble, so we right to the left or right nibbles based on if it is even or odd.
void NHD_FB_pixel(uint8_t x_virtual, uint8_t y)
{
    // Convert x from a virtual address to a physical address
    // uint8_t x_physical = x_virtual >> 1;

    // [0, 1] [2, 3] [4, 5]  ----> Virtual Address space
    //    |      |      |
    //    v      v      v
    //    0      1      2    ----> Physical Address space

    // Check if the virtual address is odd or even (thanks Jack)
    if (x_virtual & 0x01U) {
        // If the virtual address is odd, set the right nibble
        fb[(y * BUFFER_WIDTH) + x_virtual >> 1U]  |= 0x0FU;
    }
    else {
        // If the virtual address is even, set the left nibble
        fb[(y * BUFFER_WIDTH) + x_virtual >> 1U]  |= 0xF0U;
    }
}

Then we can render the contents of the framebuffer to the screen by calling NHD_FB_render(), which calls SSD13222_writeBuffer(char data[], uint16_t count).
void NHD_FB_render()
{
    SSD1322_setColumns(SSD1322_COLS_MIN, SSD1322_COLS_MAX);
    SSD1322_setRows(SSD1322_ROWS_MIN, SSD1322_ROWS_MAX);
    NHD_stream();

    unsigned int i;

    //SSD1322_writeBuffer(fb, sizeof(fb));

    for (i = 0; i < 64; i++)
    {
        char linebuffer[128];

        for (int fill = 0; fill < 128; fill++)
        {
            //linebuffer[fill] = 0xFF;
            linebuffer[fill] = fb[(i * BUFFER_WIDTH/2) + fill];
        }
        SSD1322_writeBuffer(linebuffer, 128);
    }
}

void SSD1322_writeBuffer(char data[], uint16_t count)
{
    //gpio_write(initNewGPIO, DC, 1);
    spi_write(initNewGPIO, portNewWrite, data, count);
}

I expect to be able to simply use spi_write(initNewGPIO, portNewWrite, framebuffer, 8192) instead of writing my buffer line by line as shown in NHD_FB_render and SSD1322_writeBuffer.

Ok, thanks.  I can ignore DC, RES!
One thing that catches my eye is that you don't need to configure the SPI pins.  pigpio does this for you.  It shouldn't be a problem though configuring them twice.  I'll need to verify the source to make sure.
8192 is a maximum limit, I think, without consulting the source. Have you tried 4096, 1024, etc?
I'm going off line now and won't be able to look at this again for couple of days.

I've tried anything over 128 -- 256, 512, 1024, etc.
I can't get any of those working, and am really confused by where this cap could be :/


TinyMCE not all textareas when using ng-repeat.

I've run into an issue with TinyMCE on using it in an ng-repeat.
When my application starts, it loads some data from a service. The ng-repeat is bound to an array from the service. When it runs, some of the textareas aren't rendered while others are. There does not seem to be a pattern, as I can't consistently make it do the same behavior on the same textareas every time the app runs.  Notice the last two textareas in the screenshot....

However if I add items to an array one at a time using something like a button, it works without a problem.

Going to close this issue as it is a support question. Feel free to continue discussion and we can help if there is enough information and time.
A plunker with a live demonstration of the issue would be helpful.

I think this may be a bug. I had a very similar issue using the latest version (0.0.17), I solved it be reverting to 0.0.15

I am using tinyMCE 4.7.5. I am unable to resolve this issue. Can anyone suggest me some solution?


ipaddr thinks that <IP_ADDRESS>/50000000 is a valid CIDR

Calling ipaddr.parseCIDR('<IP_ADDRESS>/50000000') should throw an exception but does not.

Thanks!!! :+1: :)


Use Fare package for regular expression generation

@moodmosaic I've noticed that you have extracted Xeger to a separate NuGet package: https://github.com/moodmosaic/Fare.
In AutoFixture we have a copy of that project. Does it make sense to actually depend on your package rather than have a copy-paste inside our code?
However, in order to achieve that we need you to add .NET Standard support and probably sign the assembly, as it we are signed. I saw that you are reluctant to sign it, however we could probably trigger Fare 2.0, so that change is not so breaking.. Also we could cheat a bit and sign everything targeting .NET Framework v4.5+ 😊

Does it make sense to actually depend on your package rather than have a copy-paste inside our code?

Fare should be the same to what we have inside AutoFixture, plus some minor changes here and there based on feedback I got from its users.
If you wish, you can take a dependency on it, and delete the appropriate files from AutoFixture. Once you do that, AutoFixture will depend on Fare, instead of having no other dependencies.
I don't think we should worry about anything, as I no longer work actively in Fare, but if that makes you feel safer, I can add you as a collaborator.
If you lean towards taking Fare as a dependency, you'd have to do what it takes (support .NET Standard, etc.) as I'm currently busy with other stuff and don't have the bandwidth to make changes there.

AutoFixture will depend on Fare, instead of having no other dependencies.

Well, that is not the case - we already have dependencies for the .NET Standard framework. See v4.
Today AutoFixture is distributed as a NuGet package, so it doesn't provide us with any benefits to not have external dependencies. That's why I don't think we should worry about that and it's not a goal to not depend on anything.

If you lean towards taking Fare as a dependency, you'd have to do what it takes

Got it 😉 I'm not sure I also have much free time for this now, but will see whether I have it in future.

Ideally I prefer to depend on other packages rather than keep code inside out project. In this particular case I understand that real benefits are not so great, except the more clear code organization. So if you don't mind, let's keep this as a potential improvement, but do not prioritize it.

Today AutoFixture is distributed as a NuGet package, so it doesn't provide us with any benefits to not have external dependencies.

Yes, the .NET Standard version depends on NETStandard.Library and System.ComponentModel.Annotations. That's fine, as these packages are part of the .NET itself, and they were inside the BCL previously.
However, I'd strongly suggest to avoid taking on dependencies on other packages, (say, Foo) as we don't have control on what their authors are doing.
Now, Fare can be an exception, as it was created for AutoFixture, and it's maintained by people who are also involved with AutoFixture.

Got your point. Well, totally makes sense. I believe that's always the rule that you should depend on third-party products only if that provides large benefit. Otherwise, dept of support might make that non-profitable. I felt that already for our glue libraries, where we should follow up each their release to ensure we support it.

I don't really mind what we do outside the core AutoFixture package (e.g. in the Glue Libraries). But for the core package, I'd really like to depend on as less stuff as possible, preferably 0 packages other than those maintained by us (and Microsoft, for .NET Standard).

Closed via #972


Add margin top to a field that is preceded by another field

Will resolve #223

  Fields should apply their own margins

  Fields should apply their own margins


Better example GIF

The GIF currently in use is not ideal in that it should show a better use case, also its execution and in general is too small.

Closed with feat/better_gif


Bug Report: game crashing when song is loaded

when i opened my custom song as soon as the voice says "go!" the game crashes,i swear i did everything right

Can you take a picture or video

I think You need to add the settings folder copy the one from the test song folder to your folder and rename it to your-song-settings.json [https://github.com/CamtheKirby/VsDave-Modable/wiki/How-2-Add-Settings](settings json tutorial)

Uploading 2024-08-17 18-45-02.mp4…

https://github.com/user-attachments/assets/e9b138f4-aca2-4b86-af04-16a3df8ef52c

Can you show me the folder the chart json is in

also this might happen because the chart might not be compatible but I'm not sure yet

i fixed it,i had to put the settings thing

also theres another bug i found:whenever i press the arrow keys with no notes coming and i have no ghost tapping,the game crashes

https://github.com/user-attachments/assets/6b3ea7a5-8104-40c6-b432-b256531cc015

OK I'll work on that when I feel like working on this engine again

okay


didn't have any blocks in api /blocks

Hello!  Have a many questions. Firstly a problem with pool blocks. Epic node was found a couple of blocks, but pool didn't have it on the api json. Also revenue value of miner page is zero. Have a question with that. Which name should be in miner? MinerName.RigName or MinerName only? When I set a name without .rig I have a workers changed randomly for each worker in the /miner api. When I'm setting a .rig I have a separate page for each rig. But I have many rigs so I should set a paying password for each? Could you update a html and js files for compatibility with new /blocks api?

I have failed test code on their new daemon on windows, because the daemon flash exit without any message, so I have put the updated code in https://github.com/maoxs2/open-grin-pool/tree/untest branch.
Then let me explain some details.
Payment module is half-done which dones't support autosend.
There are 3 parameters for the miner: login, pass, and agent. Login is the MinerName. Pass is the payment password, ("x" is also treated as no password), which is saved in plaintext in redis so maintainer can manually update it. Agent is treat as the rig name, and if it is empty, it will be granted with a random name in pool agent = "NoNameMiner" + strconv.FormatInt(rand.Int63(), 10).
This repo will be intergrated into https://github.com/mining-pool and follow new api standard, so it wont be significantly updated.

Solved and Tested #38


Reintroduce buffered buffereless designs

this should close #257. @julianhoever can you please check again?

closes #244

@glencoe Yes. I will check that now. If it solves that issue can I merge this PR or is there more to do?


Create Dense Urban ODD Scenarios

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description
Scenario Slayts: Slayts
Scenario Documentation: Google-sheets
Development Suite: Evaluator Suite
Reference Videos
Sample AWSIM Test Failures: AWSIM Test Failures
NTUCampus (Thanks to Daniel-san from TierIV)
This issue addresses the need to create realistic scenarios for dense urban Operational Design Domains (ODDs) involving Vulnerable Road Users (VRUs) such as pedestrians, bicycles, and motorcycles. The main task is to check the capability of the scenario_simulator to implement these scenarios, create sample scenarios with defined trajectories for NPCs, and provide the necessary documentation and virtual maps.
Purpose
The purpose of this issue is to enhance the simulation capabilities of Autoware by providing detailed and realistic scenarios involving VRUs in dense urban environments. These scenarios are crucial for testing and validating the safety and effectiveness of autonomous driving systems in complex urban settings.
Possible approaches

Evaluate the existing capabilities of the scenario_simulator to determine if it can simulate the required urban scenarios with VRUs. If not, identify the gaps and create detailed issues to address them.
Develop sample scenarios that allow NPCs to move along defined trajectories, simulating real-world interactions with VRUs.
Document all the scenarios comprehensively, outlining their design and expected outcomes.
Create virtual maps as necessary to support the scenarios, ensuring that they reflect typical dense urban conditions.
Implement the scenarios and integrate them into the AWF project on evaluation.ci for continuous integration and testing.

Definition of done

[ ] Determine the capability of scenario_simulator for urban VRU scenarios.
[ ] Create and document sample scenarios with defined NPC trajectories.
[ ] Prepare detailed scenario documentation.
[ ] Develop virtual maps required for the scenarios.
[ ] Upload and integrate scenarios into the AWF project on evaluation.ci.

The first test for the currently created scenarios: success rate - 18/58

@mitsudome-r @xmfcx I think we can close this issue. All the items in definition of done over.

The scenarios are added to Web.Auto and are ready to start investigation on failed scenarios.


fix: Only retry on transient errors for OSS artifact driver

This is related to https://github.com/argoproj/argo-workflows/issues/5220 but with a focus on OSS artifact driver.
Signed-off-by: terrytangyuan<EMAIL_ADDRESS>Checklist:

[x] My organization is added to USERS.md.

We probably don't want features like this to be specific on only one kind of driver and create an inconsistent design

We probably don't want features like this to be specific on only one kind of driver and create an inconsistent design

The transient errors are driver specific though and currently the user experience is bad if non-transient errors keep getting retried.


Remove Array.find_last()

The String method equivalent was removed in https://github.com/godotengine/godot/pull/40092, in favour of rfind(). The latter is much better as it allows defining a starting index like find().
Furthermore, the Packed Arrays do not have find_last(), only rfind().
Finally, the method isn't even used by the engine, it would seem...
So, what can we say. Oops.

Actually, hold on. I updated the PR to include find_last in the project converter. Since rfind() accepts an optional secondary argument, one to one conversion should still be viable.

Thanks!


.github: Update the golangci-lint version

Signed-off-by: timflannagan<EMAIL_ADDRESS>
Description of the change:
Update the .github/workflow/sanity.yaml GHA workflow and bump the golangci-lint to a version that works with Go 1.17.
Motivation for the change:
The sanity check is permafailing right now. See https://github.com/golangci/golangci-lint/issues/2374 for more information.
Reviewer Checklist

[ ] Implementation matches the proposed design, or proposal is updated to match implementation
[ ] Sufficient unit test coverage
[ ] Sufficient end-to-end test coverage
[ ] Docs updated or added to /doc
[ ] Commit messages sensible and descriptive
[ ] Tests marked as [FLAKE] are truly flaky
[ ] Tests that remove the [FLAKE] tag are no longer flaky

/lgtm


Glow not supported warning on opening Compatibility project with no glow

Godot version
4.2.rc2
System information
macOS 14.1
Issue description
If you create a new Mobile renderer project and then change the project to Compatibility renderer you get a warning when loading the project in the editor:
servers/rendering/storage/environment_storage.cpp:387 - Glow is not supported when using the GL Compatibility backend yet. Support will be added in a future release. (User)
Even if you don't have any environments using glow in the project and only view 2d UI scenes. This only happens if you create the project as non-compatibility, so I'm guessing it's then adding some default environment somewhere with glow enabled that is generating this warning when loaded, even if this is not used in the game or any scenes.
Maybe it's the editor preview environment that is loaded with glow enabled?
Steps to reproduce

Create a new Mobile renderer project
Change it to Compatibility renderer
Open the project and get the warning above, even though no glow is used in project

Minimal reproduction project
NoGlow.zip

Is the glow option enabled in the environment settings in the 3D editor? It defaults to enabled when non-compatibility and is saved, so it is likely the cause of this

Is the glow option enabled in the environment settings in the 3D editor?

Yes, this seems like the cause. However, the project doesn't have any 3d scenes, so I've not used the 3d editor. Despite this it seems that for each scene loaded this default 3d editor environment is loaded, even though they are not 3d scenes and the 3d editor is not open. I can open the 3d editor for a scene and open the settings for the environment and toggle off glow and save to toggle it off for this scene, which removed the warning if I open only that scene. But that doesn't seem to affect other scenes, so opening any other scene will get the warning again. The change doesn't seem to be saved in the scene file as I don't see any change in the file for this preview environment change, so I'm guessing the override for this scene and what defaults to use are saved somewhere else?

There should be a way to edit the default preview environment used in the editor for every new scene. It would solve this and other problems.  And I don't want to set a runtime default environment in the project.

Glow was implemented by https://github.com/godotengine/godot/pull/87360, so this warning is no longer printed. Closing.


extract 7z archive part from 7z installers

@BillyONeal This code can separate the 7z archive from the self extracting PortableGit installer (and probably more). The extracted 7z archive can then be passed to cmake -E tar without having to use 7z to extract the installer silently.

Do we want this now that we have determined that we always need 7z anyway?

Do we want this now that we have determined that we always need 7z anyway?

as far as I know vcpkg downloads the console version of 7z which cannot extract executables which I learned in https://github.com/microsoft/vcpkg/pull/23659 where i tried to use the console version but had to switch to the full version.


Evolution of java script

Title Of Blog Post : Evolution of JavaScript
Short Description of what is expected : An Evolutionary post about JS covering all the major milestones
Category :hacktoberfest
Tags :hacktoberfest, good first issue, JavaScript, technology
Checklist

[ .]  Added Image For banner
[ .]  Markdown file in the _posts directory following the naming convention
[ .]  Include Front Matter at the beginning of your Markdown file with metadata
[ .]  Testing Locally. This is very Important.

This Pull request closes #Issue

Hey please review and merge my PR

Hey @shreyanshig , Your PR is appreciated.  your post file is not correctly formatted according to the jekyll font matter please refer to contributon guidlines for correct format

this isn't the format for front matter
do git pull in your local correct errors and test manually then create PR.

Also please link the issue here


Ticketing - PPH Link logo in top-left sometimes disappears.

This may have something to do with cert?

This issue is being closed due to that it hasn't since come up


use published service hostname for db/account jobs

also runs a check inside database.sh / account.sh to ensure the command succeeded

/lgtm


Delete virtual machine with the highest instance ID

It is possible that I am wrong, but the statement in subject is incorrect.
Older instances are deleted first and those instances have lower instance ID.
The incorrect statement appears in the explanation of the "Scale-in policy"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1f1d9e2d-9e2a-b137-efbc-b20c27b2ac12
Version Independent ID: 92d71abf-71aa-bc2d-21a9-78bc1a4dcac4
Content: Use custom scale-in policies with Azure virtual machine scale sets - Azure Virtual Machine Scale Sets
Content Source: articles/virtual-machine-scale-sets/virtual-machine-scale-sets-scale-in-policy.md
Service: virtual-machine-scale-sets
Sub-service: autoscale
GitHub Login: @ju-shim
Microsoft Alias: jushiman

@mpalmier74 Thanks. We will verify it and get back.

@mpalmier74 Thank you for your query!!!
The default scale in policy uses the following order of operations:

Balance virtual machines across availability zones (if the scale set is deployed in zonal configuration)
Balance virtual machines across fault domains (best effort)
Delete virtual machine with the highest instance ID

Then you can change the scale in policy to oldest which will delete the VMs that are the oldest. We don’t call out that the VMs that are old have a high or low instance ID. Rather the default policy will remove the VMs with the highest instance ID before removing VMs with a lower Instance ID. This is also because when scaling in, the default behavior is to remove the instances that were added and not remove the oldest. Really, all the instances are identical so it does not really matter if you remove old or new ones.
Hope the provided information is helpful.
We will now close this issue. If there are further questions regarding this, please tag me in a comment. I will reopen it and we will continue the discussion.


Feature Request: Updating Existing TOC

I noticed that if I create a TOC for a Bear note and then make changes to the note headers, if I rerun the script it does not update the TOC and says the Note already has headers.
I took a peek at the code and I think the key place in the code is here https://github.com/alexander-lee/markdown-github-bear-toc/blob/06037316dac6c8b3e16e3ea29a5142fca65c02bb/markdowntoc/markdowntoc.py#L210-L212
I propose to have logic in that if statement to delete the TOC and then continue recreating it so that if there are any changes they get accounted for.

I've seen other mechanisms where there is an explicit TOC marker that the user can insert in the place they want the TOC to be placed. Then on re-run that same marker (in a comment) is located and then deleting the existing links/TOC is considered safe.
I'm not using this tool, but considering options (my current use case is html generation so currently experimenting with TOC generation tweaks in the markdown to html phase). If I end up using this, I'll likely need/want this functionality (and would implementing it myself).


Windows subsystem support

Rendered
[mbrubeck: edited to link to final rendered version]

cc @retep998

I haven't tried to do this myself, but the RFC is implying that it requires special compiler flags.  If this is a use case we want to support, then perhaps Cargo should also gain something for it.
There might be a similar concern with mac.  I'm not sure what the difference between a console app and a nonconsole app is over there, or even what the proper names are.  But it might be worth covering them both at the same time and with the same Cargo option.  If this is also a concern, designing it to hit both with as few changes as possible could be helpful.  Even if we need a separate RFC to finish it.
One of the things I want to do with Rust is Windows desktop apps. I had assumed this was trivially possible and that I could just kill the console at any time, but I guess I should have checked.

Right now you can change the subsystem by passing -Clink-args="/SUBSYSTEM:WINDOWS" either via cargo rustc or RUSTFLAGS. It's not ideal, but to have a proper way to configure that would need to also support choosing the target NT version as well.

@camlorn AFAIK, there's no equivalent for OSX or linux, all executables are the same there, the only difference between a GUI app and a console app are code differences.

@Diggsey
Just verified. The only difference is whether or not it's in an app bundle.  Source.
@retep998
Perhaps subsystems should be handled by letting us specify an oldest Windows version and having Rustc or Cargo automatically determine what it should be.  I've looked into the subsystem stuff before and came away vowing to never touch it because of poor documentation.  It's possible being able to configure the subsystem has uses I don't know about, but it feels like an option that would exist just to let me get it wrong.
I would personally vote for defaulting such an option to Windows 7, with Vista and XP requiring you to explicitly set it.

@camlorn Right, but the executable does not need to be linked specially - it is sufficient to simply place it in the correct location in the application bundle. I see this as being a problem for one of the various packaging plugins for cargo to solve.

Thanks for the PR @Diggsey! I've got a few questions, but mostly along the lines of clarification.
Could you provide a detailed explanation of what WinMain would look like and what the compiler generates (the compiler generates main, so I assume it'd also generate WinMain). The signature looks like it has some information like hInstance and nCmdShow which we'd probably want to make sure are accessible through at least some means, right?
Also, this would in theory be a breaking change for anyone who explicitly has a WinMain function in their crate today I believe, so this may want to explicitly mention that and perhaps try to find a few crates that either do this or confirm that it's quite rare.
Being relatively unfamiliar with this space as well, I wouldn't mind expanding the alternatives a bit more in detail. For example I'm not sure what a #[cfg] attribute would look like here. Also is it true that object files need to be compiled with a particular subsystem in mind? If that's true then we need to consider how we ship the standard library, but I thought that this only had to do with linker flags at the very end. Along those lines, it's not clear to me at least why dependencies would need to be recompiled.
Also, how would a subsystem function help? Doesn't compile-time generation need to be different as opposed to something at runtime? It seems like libraries could discover whether they're part of an application or not, but it's also not clear to me how it'd solve the problem at hand.

Could you provide a detailed explanation of what WinMain would look like and what the compiler generates

extern "system" WinMain(_: HINSTANCE, _: HINSTANCE, _: LPSTR, _: c_int) -> c_int {
    // call Rust's std::rt entry point
    return exit_code;
}

None of the arguments matter and can all be obtained via other means.

perhaps try to find a few crates that either do this or confirm that it's quite rare.

https://github.com/search?q=WinMain+language%3Arust&type=Code&utf8=✓

Also is it true that object files need to be compiled with a particular subsystem in mind

Nope, the only thing that matters is choosing the subsystem at link time. Since any non-console app can still be run from within a console, it is up to the application to determine whether stdout/stderr are valid at runtime and redirect logging appropriately.

@alexcrichton Hopefully these changes address your comments.

Doesn't compile-time generation need to be different as opposed to something at runtime?

No, the aim is for the entire compilation process to be completely identical up until link time. If libraries want to have different code-gen, they should do so via the existing feature or cfg systems.
It's not all that different from other linker options such as what icon to use for the produced executable: it greatly simplifies things if it's independent of the compilation process.

One major difference is that console programs need to use WriteConsole and UTF-16 to do console I/O correctly.

@DemiMarie It's trivial to figure out whether the handle you get via GetStdHandle is a console handle or a pipe (libstd already does this). It's also easy to determine whether writing to that handle results in an error, so you can fallback to logging to a file. There's simply no need to know which subsystem you were linked for.

I guess without the WinMain you can't get access to the hInstance which is probably why the Window with glutin is always so disconnected from the Main Process in the Task Manager?!

@CryZe That problem is probably due to the fact that you have the console subsystem.
You have to pass the hInstance when creating a window, however it's mostly for corner cases when DLLs want to register window classes. Glutin passes null for this parameter, which means "the main app".

@tomaka It doesn't, either something is exporting WinMain (maybe one of your dependencies, it would be useful to know if that's the case!) or the linker arguments are not getting through to the linker.

I guess without the WinMain you can't get access to the hInstance which is probably why the Window with glutin is always so disconnected from the Main Process in the Task Manager?!

GetModuleHandle(NULL). There's your hinstance.

I've removed some of the "hypothetical" alternatives at @alexcrichton's suggestion, and I've fleshed out a possible design for the "command-line configurable" alternative.
I initially went with the "dual symbol" solution as the main proposal because I thought it would be a quick way to get this support into rustc, with room for later improvement. That may still be the case, but if there's no objection to the specific design I went with for the alternative, it may be better to jump straight to that.
The main problem I see is that it adds extra command-line surface for everyone, but it's only going to be relevant when targeting windows, and I can imagine there may be some objection to that.

@tomaka Ah, you may have been using the GNU toolchain. This is more of a problem for MSVC. I've updated the RFC accordingly.

@Diggsey The /SUBSYSTEM flag can also be used to specify the minimum NT version the binary should run on. How would you specify that with this RFC, or how is there room left for this in a future RFC?

The duplicate symbol solution which this RFC proposes does not affect linker flags at all, and the version numbers specified for the subsystem do not affect which entry point is used, so I don't think there's any overlap at all there.
The alternative which adds a subsystem config to rustc and cargo does have some overlap: as proposed, rustc would not understand a version number, and so would not pass it through to the linker, but it would still be possible to manually pass in the option as we do now. I don't believe it restricts us in future: we could backwards compatibly extend the set of possible values for subsystem to include a version number, or we could add a separate option controlling the version.

Thanks for the update @Diggsey! This to me like it's ready for FCP, I'll bring it up at the tools triage this week.

I think you should update the RFC text to include the linker options required for both GCC and MSVC.
Also, this is going to block Servo moving to the native Windows toolchain, which we arein the process of doing. Is there a workaround?

I think some tooling is also needed so that this is easily controllable from rustc and Cargo. Having to use RUSTFLAGS to accomplish this is not very ergonomic. Overriding the linker in Cargo is also not a good option as unless the linker program is a native executable it will be run by cmd.exe which has 25% of the command line length that CreateProcess does. Servo hits this limit if you use fake-ld.cmd to try and add this link args that way.
So I suggest rustc be able to emit correct link args even if it can't parse them. I guess that puts me in the option 1 camp with the addition of supporting --crate-subsystem.

cc @vvuk @larsbergstrom

The tools team got a chance to discuss this yesterday, but some points were brought up that made us think that this isn't ready for FCP just yet. @vadimcn brought up that we may be able to sidestep the WinMain symbol entirely by forcing the entry point as mainCRTStartup instead of WinMainCRTStartup for applications. It's thought that the two CRT entry points do the exact same initialization, and then just delegate to different entry points (would be good to confirm!). In that case, we can simply always pass the entry point for executables as mainCRTStartup, and then all we need to do is to pass the subsystem somehow.
I personally have come to feel that we need to address passing the linker argument as well. Figuring out how to pass linker arguments can be relatively arcane and difficult sometimes, and it's best if this kind of information is encoded into the crate anyway. @brson brought up the idea of perhaps having:
#![windows_subsystem = "windows"]

That way crates could encode this information (as it's probably local to a crate anyway), it's clearly windows-specific (just ignored elsewhere), is extensible to other values if we need, and can also be controlled with #[cfg_attr].
Thoughts on that strategy?

Ah and a downside of that approach is now we're accumulating "cargo culted knowledge" that whenever you write a windows application you just have to blindly apply that attribute to your crate (probably without understanding it). It'd be "best" if you could, for example, link to the "windows application" crate which does all the right things for you, but seems like it's more difficult to get working.

The mainCRTStartup option is alternative 2 in the RFC, and alternative 1 proposes a way to specify the linker arguments through rustc and cargo.
A crate attribute could work, although it would be somewhat magical. It could potentially be done via an external crate, but libstd would have to add and then stabilise a few new features for that to happen. I also feel like this is a primitive enough feature that it should be possible without external crates.
I can rewrite the RFC to make one of these alternatives the primary proposal if one in particular is preferred.

I just tested and passing -Clink-args="/subsystem:windows /entry:mainCRTStartup" to rustc appears to work correctly. The process is spawned without a console.

@retep998 That certainly makes things easier. It looks like this will at least unblock Servo's msvc port.

Unfortunately, going this route for RUSTFLAGS for servo is going to cause problems -- it's going to mean that all build_script_build.exe's built by cargo are going to have the windows subsystem set, and I think aren't going to get consoles for stdout/stderr.  Have yet to confirm that.
Also, I can't figure out how to pass anything with a space to -Clink-args via RUSTFLAGS; setting RUSTFLAGS to either "-C link-args='/subsystem:windows /entry:mainCRTstartup'" or with inner "s causes a "multiple input filenames" error.  It would be helpful if -C link-args were to append, instead of replace (e.g. behave like -Wl,... for gcc).

I didn't consider the build.rs case. You're right that this will cause them to get built as /subsystem:windows too. I don't know whether that is a blocker or just a pain for debugging the build system.

@vvuk When Cargo runs a build script it specifies stdout/stderr/stdin for the build script. The Windows subsystem does not override stdout/stderr/stdin set from the parent process, it only changes the default of whether a console is spawned if there isn't already a console. Therefore even if the build script was compiled for the windows subsystem, it would still work just fine.

@vvuk For the space issue, pass it as -Clink-args="/subsystem:windows" -Clink-args="/entry:mainCR TStartup" so they can be interpreted as two separate arguments just fine.

@Diggsey what do you think about updating the RFC according to what was mentioned earlier? I think that should solve the motivation, plus some of the problems servo has been running into as well?

@vvuk When Cargo runs a build script it specifies stdout/stderr/stdin for the build script. The Windows subsystem does not override stdout/stderr/stdin set from the parent process, it only changes the default of whether a console is spawned if there isn't already a console.

Windows subsystem binaries don't get a console at all -- a process would need to AttachConsole(ATTACH_PARENT_PROCESS) to get access to the parent's stdout/err/in if it's flagged as subsystem:windows.  Console processes attach to parent, or make a new one if needed.  So things like build.rs compiled with subsystem:windows won't print any output, so cargo won't see any cargo:... directives, nor see errors etc.

@vvuk Having a console has little to do with stdout/stderr/stdin. If the parent process explicitly sets the stdout/stderr/stdin of the child process, then the child process can read and write to them without issue regardless of whether there is a console attached.

I'll retest, but it did not work for me -- cargo did not capture any of the output, at least in my local builds. Now that I think about it, it does work on appveyor, so...

[build] rustflags = ["-Clink-args=/subsystem:windows /entry:mainCRTStartup"]
Tested building some stuff which relies on build scripts with that and the build scripts ran fine and the output was a valid program. Granted the cmake invocations popped up new console windows which can be a bit annoying, but it works fine otherwise. So long as you don't need to pass -Clink-args for anything else.

@retep998 Popping up console windows during build makes my computer basically unusable when compiling Servo. I'd say that's a non-starter.

I've rewritten the RFC based on the feedback.

I don't see why you need to build build scripts with a specific subsystem or with the console disabled; why can't they just build as they are now?
I can't immediately think of something that a build script might want to check for that would change if the subsystem changed.  As far as I can figure, the important thing from the perspective of build scripts is that it's build with MSVC, not that it happens to not have a console or that the binary of the build script itself runs on XP (you never distribute that piece).

@camlorn The problem is that when using RUSTFLAGS, it will affect both the program and any build scripts. This is exactly one of the issues this RFC is intended to solve: you often want your program to be compiled for the windows subsystem, but you don't want that to affect the build scripts.

@Diggsey
Ah, I see. I kind of had it backwards.  I read the comments as saying that this RFC would introduce that problem, not fix it.

This looks great to me, thanks @Diggsey!

并没有理解如何隐藏后台窗体。
Help!There is something wrong in my program.

Help！How can I hide the DOS form?
请问，我应该如何隐藏后台窗体？

@3442853561 This repository is for RFCs, proposals for the Rust programmer language. This is not the right place to ask questions.
Secondly, your question is vague and cannot be answer as is. Please be specific, and ask it at either https://reddit.com/r/rust or https://stackoverflow.com.

This wouldn't involve libstd support, but for the benefit of applications using libcore, could this support the "NATIVE" subsystem as well?

@joshtriplett the NATIVE subsystem is not for simple no_std applications. It is for kernel mode drivers which is an area that nobody has even attempted with Rust yet, so there is basically no information on what the right way forward is yet. There's nothing stopping the proposed attribute from eventually supporting that, but it'll have to be part of a larger effort to figure out how to write Windows kernel drivers in Rust.

@retep998 I wasn't suggesting that the RFC needed to define how that would work.  I just wondered if a Rust crate producing a Windows driver could use this mechanism to declare the correct target subsystem, assuming it doesn't use std and otherwise meets the assumptions of a Windows driver, or if such a crate would still have to use linker options directly.

@joshtriplett At the moment this RFC doesn't specify any subsystems other than windows or console. To make a driver you have to pass other linker args anyway such as /DRIVER, so it shouldn't be too hard to pass along /SUBSYSTEM:NATIVE manually as well.

@rfcbot fcp merge
Discussion I think has reached a close at this point on the RFC, and having discussed it in a tools team meeting I believe our concerns are addressed as well. I propose we merge!

@rfcbot reviewed

@rfcbot reviewed

🔔 This RFC is now entering its week-long final comment period for merging :bell:

Ok, looks like nothing new came up during FCP, so merging! Thanks again for the RFC @Diggsey!
Tracking issue: https://github.com/rust-lang/rust/issues/37499

I find it strange that panic! behavior isn't mentioned in this RFC. Currently it just exits with code 3 and prints nothing. I understand that in general you can't expect any console output in GUI executable, but the fact that the panic behavior isn't specified in this case is odd.
What would be the next steps to address such an issue? Issue in the https://github.com/rust-lang/rust repository?

@pravic Panic behavior is probably better handled with custom panic hooks.