fix: encode URLs correctly (fix #15298)

Description
Fixes #15298. The problem was that special characters were not encoded correctly in URLs created for assets. This was also a problem for vite-imagetools when there were spaces or other special characters in the filename.
Additional context

What is the purpose of this pull request?

[x] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[x] Read the Contributing Guidelines, especially the Pull Request Guidelines.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Update the corresponding documentation if needed.
[x] Ideally, include relevant tests that fail without this PR but pass with it.

I mentioned we had this problem here https://github.com/vitejs/vite/pull/15246#pullrequestreview-1767673646. I think it is a good idea to do this, but I don't know if we should merge this one in a patch. We always had this issue and it would be good to see how all the encoding affects performance. I added it to the 5.1 milestone for now.

Yes, I forgot that 🙈Am 2023. 12. 12. um 18:49 schrieb Ben McCann @.***>:
@benmccann commented on this pull request.
In packages/vite/src/node/plugins/importAnalysis.ts:

@@ -401,7 +401,8 @@ export function importAnalysisPlugin(config: ResolvedConfig): Plugin {
url = joinUrlSegments(base, url)
}

   return [url, resolved.id]

   // only encode if the url can't be parsed

is this comment left over from your earlier changes?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

I agree that correctness is more important than performance here. I still think we should merge this one as part of the next minor. We discussed with the team and decided to start the beta for 5.1 after the holidays. This change could introduce subtle bugs, for example, I think this changed line isn't right https://github.com/vitejs/vite/pull/15311/files#diff-f2f744fef86a2c562dd5142240912f7a2d28404fac536740a2424daf628aa609R409. That normalized URL is later used in both warmup and to update the module graph. In both cases, the URL should be decoded. So we need to move that later on, and only encode when we write a URL into the code.

You mean the line in packages/vite/src/node/plugins/importAnalysis.ts? That's the only line I was not 100% sure, but without the change it throws for some URLs.

I think the encoding may need to be moved here instead https://github.com/pzerelles/vite/blob/f45775c9515cf9a3d3c9ffc448b871dfc78efe84/packages/vite/src/node/plugins/importAnalysis.ts#L602C17-L602C17

Thanks so much for providing the right location. I probably didn't understand the code good enough. The original problem is also fixed when moving the encode to that location.

To set expectations, we may start work in 5.1 beta in January. My take is that we should merge it then. I'll be off for a few weeks, then do a review. Other maintainers may get to this one sooner though and I'm fine with them deciding to move forward if performance and test cases look enough to cover this one. We should have a test covering an id that wouldn't work with the encoding as it was before the last commit. I think a file called module%.js should already trigger the issue.

Yes. There is also another thing. Right now I encode only placeholders VITE_ASSET{id}_ and not VITE_PUBLIC_ASSET{id}_, because I found no documentation about it and when it is used.

Instead of adding a new test I extended the existing unicode url test to include a % character.
That test was wrong before anyway because in dev mode it expected an unencoded url.

If performance is a concern, I did some tests and I found that browsers can generally handle un-encoded imports too. It'll encode them automatically when requesting. Even if the import string has a mix of encoded and unencoded strings, e.g. import './test%20テother.js', the browser can specifically encode the "テ" and not the %20 as %2520 (double-encoding). So we can probably get away with encoding % as %25 only instead of encodeUri.

@pzerelles we discussed about this in today's team meeting and decided to move forward with the change, but using a helper for performance reasons that will only transform % into %25 instead of a full encoding as this should be enough. Let us know if you'd like to modify this PR to do so, if not, we can work in another PR.

But this will not fix urls in general and libraries like vite-imagetools or images in general will still be wrong if the files contain spaces or other special characters.

@pzerelles would you create a minimal repro for the issue you describe with image-tools?

Of course: https://github.com/pzerelles/vite-bug-repro
The second test logo will not be loaded, because vite-imagetools (currently) writes the transformed image to disk with url-encoded filename (which is also wrong), but the web server will look for a file with the url decoded.
The main reason for vite-imagetools doing this is because vite does not url-encode asset urls when they are written to html or js output (I assume). I already have a PR in the vite-imagetools repository to remove this behavior and output files unencoded, and maintainers have signaled to me that it will be accepted if vite can provide the url-encoding, which in their opinion is also the correct place to do it.

Based on the repro, is your main concern of this inconsistency on this HTML during vite preview?

The strings come from this source code:
import testLogo from "./logo 10.png";
import testLogoScaled from "./logo 10.png?w=64&format=webp";

document.querySelector<HTMLDivElement>("#app")!.innerHTML = `

`

So from what I can tell, we only need to fix the URL string from asset imports. We don't have to handle the encoding of imports to fix vite-imagetools? I think it's good to fix the inconsistency for asset imports specifically, which means using encodeURI on this filename. filename itself shouldn't be pre-encoded because it's not a url string. (implied by its name)
But for others, it looks like vite-imagetools should not be writing files in encoded characters in the first place and it should work. If I manually edit the name after build, it's also working fine.

Based on the repro, is your main concern of this inconsistency on this HTML during vite preview?

The strings come from this source code:
import testLogo from "./logo 10.png";
import testLogoScaled from "./logo 10.png?w=64&format=webp";

document.querySelector<HTMLDivElement>("#app")!.innerHTML = `

`

So from what I can tell, we only need to fix the URL string from asset imports. We don't have to handle the encoding of import specifiers to fix vite-imagetools? I think it's good to fix the inconsistency for asset imports specifically, which means using encodeURI on this filename. filename itself shouldn't be pre-encoded because it's not a url string. (implied by its name)
But for others, it looks like vite-imagetools should not be writing files in encoded characters in the first place and it should work. If I manually edit the name after build, it's also working fine.

That is correct, vite-imagetools should not be writing files in encoded form and I have a PR waiting there to fix that, too.
If vite can correctly encode the URL string from asset imports, that would be the solution and I think that is what my PR here is about.

I think this PR is doing more than expected to fix the issue for vite-imagetools though. We only need to encode the filename as linked before, but this PR is also:

Encoding the filename passed to renderBuiltUrl
Encoding import specifiers (import-analysis)

Which I don't think are needed for vite-imagetools. no2 is sort-of a real (but different) issue though which we suggested only encoding the %. I don't quite understand how no2 affects vite-imagetools otherwise since it shouldn't be a concern for plugins. Vite is only normalizing the import specifiers to bridge them back to the server when it's requested by the client/browser.

I guess what I'm proposing is that, we can scope this PR down by

Only using encodeURI on the filename here.
For import analysis, only handle % encoding for import specifiers.

But isn't renderBuiltUrl used to insert the URL into HTML during SSR? If yes, that needs to be encoded as well.

The consumer who uses renderBuiltUrl will need to encode the filename themselves in this case. In my mind, "filename" is un-encoded, and the returned built "url" is encoded. Unless we rename the filename as url so it's less confusing. Our docs also suggested usage without encoding 🤔  Maybe @patak-dev have some thoughts on this.

To be honest, I wasn't thinking about encoding when designing the renderBuiltUrl API or the docs as this wasn't in the table at that point. From what I see from the usage in the ecosystem, everyone is adding a base or wrapping the filename in a function call and then returning it without encoding.
What if we keep filename as is, and start encoding (only replacing %) on our side to the output of renderBuiltUrl? We have many places in our API were we use url but we mean an unencoded url.

But isn't that the distinction, filename or url. I agree that the filename should not be url-encoded, but the url should be if it is represented as string.

But isn't that the distinction, filename or url. I agree that the filename should not be url-encoded, but the url should be if it is represented as string.

We have two definitions for URL internally:

Browser URL: these are the ones your browser will see. They are wrapped when needed during dev with /@id/, \0 is replaced by __x00__, for example. They also have queries like t and import. And these should be encoded (at least the %).
Server URL: these are the ones we pass to transformRequest, resolveId, the moduleGraph APIs, etc. These are unwrapped, no /@id/, the can start with \0 (they don't have queries t and import, but they can have a direct query for css), these should not be encoded. These are the URLs you use when authoring source code too.
id: these are resolved Server URLs

I think we need to properly document this in the docs, but it is tricky. We should first find good names for these.

From a developer standpoint, I import an image and get back a string to use as src in my img tag. The RFC says that it should be encoded. If that is not how it should work, everyone will need to url-encode the imported url separately.

I'm proposing moving the encoding from renderBuiltUrl(encode(filename)) to encode(renderBuiltUrl(filename)) inside Vite, so it should be the same for vite-imagetools, no?

Hmm, maybe I'm also looking at the issue wrongly. @patak-dev I think I agree that perhaps we should leave renderBuiltUrl filename un-encoded, and also leave its returned url un-encoded so we don't have to change that.
If we do these encoding/decoding at a low-level (e.g. the renderBuiltUrl primitive) that many higher-level APIs relies on, it makes it hard to reason the encoding state of the URL. For simplicity and from what I observe in the codebase so far, I think what we should do is that:

For every logic that deals with URLs internally, it must be decoded.
At the start of the logic, if the URL is encoded, we decode it.
At the end of the logic, if the URL needs to be encoded, then we encode the (surely) decoded URL.

Examples
Examples of no2:
https://github.com/vitejs/vite/blob/6c4bf266a0bcae8512f6daf99dff57a73ae7bcf6/packages/vite/src/node/plugins/css.ts#L327-L328
https://github.com/vitejs/vite/blob/6c4bf266a0bcae8512f6daf99dff57a73ae7bcf6/packages/vite/src/node/server/middlewares/static.ts#L121-L122
Examples of no3:
https://github.com/vitejs/vite/blob/6c4bf266a0bcae8512f6daf99dff57a73ae7bcf6/packages/vite/src/client/overlay.ts#L238-L243
https://github.com/vitejs/vite/blob/6c4bf266a0bcae8512f6daf99dff57a73ae7bcf6/packages/vite/src/node/server/middlewares/static.ts#L196-L198

If the internals keep flipping between encoded and decoded, I think it'll be hard to maintain over-time.

So I guess my new suggestion now is to update this part (which is the end of the logic) and encode it 🤔
https://github.com/vitejs/vite/blob/6c4bf266a0bcae8512f6daf99dff57a73ae7bcf6/packages/vite/src/node/plugins/asset.ts#L209
@patak-dev I think for this case, we might need to do a full encode since we don't know how it's going to be used. We could only do the partial encode for import specifiers I believe since we have control over that.

Moving this to 5.2 since we're releasing 5.1 soon and the PR needs an update to finish up.

So I guess my new suggestion now is to update this part (which is the end of the logic) and encode it 🤔
https://github.com/vitejs/vite/blob/6c4bf266a0bcae8512f6daf99dff57a73ae7bcf6/packages/vite/src/node/plugins/asset.ts#L209
@patak-dev I think for this case, we might need to do a full encode since we don't know how it's going to be used. We could only do the partial encode for import specifiers I believe since we have control over that.

I changed the PR to this and it works for dev server. But in build mode, the URLs are unencoded and the original Repro from #15298 fails with preview.

I added the URL encode again in toOutputFilePathInJS, but after renderBuiltUrl. Everything is working then, only the mysterious URL malformed during tests, that does not make the tests fail.

I went ahead and rebased this, and fixed the URI malformed issue. The issue long exists in Vite, only when you reference an asset with % in the filename.
I also removed the encoding in toAssetPathFromHtml. I searched around the codebase and make sure every part where we render the URL as a string (which is the end of the lifecycle of a url string), to call encodeURI. (or partialEncodeURI if only % encode is needed). Among the places are nearby:

toAssetPathFromHtml
toAssetPathFromCss
toAssetPathFromHtml
assetUrlRE
publicAssetUrlRE
cssUrlAssetRE
workerAssetUrlRE

Hoping this change doesn't blow everything up and should be more accurate.

@bluwy Thank you very much. I wouldn't have found all those places probably. Will try if it solves all issues I had.

/ecosystem-ci run


Don’t require relaunching to :PlugInstall

Would it be possible to remove the requirement to relaunch Vim in order to :PlugInstall new plugin entries?

Hmm, could :PlugInstall automatically :source my vimrc configuration for us?

Consider doing something like:
command! MyPlugInstall source $MYVIMRC | PlugInstall


Client Remains Connected on Certain Movie Queries

Description
Using httpie or curl the client seems to stay connected and require and extra "enter" key or ctrl-c in order to end the request.  Using part of the predicates seem to return as expected.
Failing Request
http localhost:4120/api/movies?q=matrix&genre=action&year=2003
Successful But Similar Requests

http localhost:4120/api/movies?year=2003
http localhost:4120/api/movies?genre=action
http localhost:4120/api/movies?q=matrix

Expected Behavior
All queries listed above should complete once executed and require no additional input to close the client.
Reproduce

Start running the app
In the terminal window issue the Failing Request
See it hung
Press "Enter" to close

Additional Context
My guess is that the errors in the tests described in #283 are related to the same problem since they were timeouts on movie queries.

The issue seems to happen when '&' is present. This can be avoided by using single quotes after http: http 'localhost:4120/api/movies?q=matrix&genre=action&year=2003' or escaping the & symbol : http localhost:4120/api/movies?q=matrix\&genre=action\&year=2003

Going to close this issue since there's a resolution/workaround for the problem.


Don't use document in Worker

Based on https://github.com/mlc-ai/web-llm/issues/124 & https://github.com/mlc-ai/web-llm/issues/162 I thought we could fallback to globalThis.location.origin when there is no document like in a Web Worker.

Hopefully this PR works. I did it via "Create PR" in VS Code and I don't like the way it auto made/named it. Sorry for that.


Support for limiting outgoing traffic

Added options for limiting outgoing traffic to allow traffic only to a
whitelisted subset of domains/IPs.
The default functionality did not change.

After this PR added a few things to testing:

cat the resulting squid.conf https://github.com/CSC-IT-Center-for-Science/ansible-role-squid/commit/8f5ad0fdeb929229ad2f59a62afc7726264befba
define {{ internal_net }} to (hopefully) test more of the template https://github.com/CSC-IT-Center-for-Science/ansible-role-squid/commit/fe0133b1aeb840d874c26949b8929cc60cb46251


Get-DbaMemoryCondition: convert to bigint when calculating the NotificationTime

Get-DbaMemoryCondition: convert to bigint when calculating the NotificationTime

Type of Change

[x] Bug fix (non-breaking change, fixes #7238 )
[ ] New feature (non-breaking change, adds functionality, fixes # )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[x] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose

Minor change to ensure the calculation of NotificationTime does not result in an arithmetic error as reported in #7238.
Commands to test

See the integration test.

test failure unrelated

How do I get the fix for this?

@SicilianNajdorf, typically a new release of dbatools will be created after multiple pull requests have been merged. The new released version will be visible in the PowerShell Gallery under the Version History: https://www.powershellgallery.com/packages/dbatools
More details are here: https://dbatools.io/download/

Thanks
How do I get the new release?


[Code Engine/Knative] Execution concurrency

Config:
I can run lithop successfully with config https://github.com/lithops-cloud/lithops/issues/391

Benchmark

 python3 flops_benchmark.py --loopcount=5 --matn=4096 --workers=100 --memory=512 --outdir=ce_shard --name=100
 python3 flops_benchmark.py --loopcount=5 --matn=4096 --workers=1000 --memory=512 --outdir=ce_shard --name=1000

However, by default both benchmark run with workers=100，as I can see Knative accepted 100 req/s so only scale up around 100 pods.

I noticed max worker for Code Engine is 1200 https://github.com/lithops-cloud/lithops/blob/master/lithops/serverless/backends/code_engine/config.py#L28
and default worker for Knative is 100 https://github.com/lithops-cloud/lithops/blob/master/lithops/serverless/backends/knative/config.py#L30

Expected

If workers=1000, we need lithops client to send out 1000 req/s concurrently.

Currently, I need to specify workers in the config file

lithops:
    #storage: ibm_cos
    storage_bucket: lithops-serving
    #executor: serverless
    #rabbitmq_monitor: <True/False>
    #data_cleaner: <True/False>
    workers: 1000

From what I observed,  1000 POST are sent out in 6m21s for 1000 execution, max 446/s POST in parallel at the very begining.

@gilv @JosepSampe Are these all in your expectation? Thanks!

@zhanggbj Yes, this is the expected behaviour in knative. By default, lithops uses a threadpool of 500 threads (max) to perform the invocations. At the same time, in knative invocations are synchronous, i.e., lithops has to wait one invocation to finish to invoke another one.  It is for this reason you see a max level of concurrency at ~500 in the last plot. You can increase (or decrease) this number by using the invoke_pool_threads parameter in the map() call: https://github.com/lithops-cloud/lithops/blob/master/docs/api-details.md#executormap
It is important to note that this behaviour only happens with knative. In the rest of available compute backends invocations are asynchronous, so lithops would invoke the 1000 calls immediately without waiting for their completion.

@zhanggbj @JosepSampe I guess it's resolved...closing this for now

Hi @JosepSampe @gilv ,
One more question about the threads, is there anything I can configure the threads numbers from 500 to 1000, just got some time and I would like to know the 1000 execution result? 500 thread is a limiation for us (Code Engine or Knative serving)
If no config for now, do you mean I need to hack the code here ? https://github.com/lithops-cloud/lithops/blob/cdb692d99b4bfbfcf161d0c92640ced2dedd6ee8/lithops/executors.py#L206

Hi @JosepSampe @gilv ,
One more question about the threads, is there anything I can configure the threads numbers from 500 to 1000, just got some time and I would like to know the 1000 execution result? 500 thread is a limiation for us (Code Engine or Knative serving)
If no config for now, do you mean I need to hack the code here ? https://github.com/lithops-cloud/lithops/blob/cdb692d99b4bfbfcf161d0c92640ced2dedd6ee8/lithops/executors.py#L206

@zhanggbj Yes, we can add this value as a parameter in the configuration file. Meanwhile you can test different values of invoke_pool_threads by simply setting it in the map() call (no need to modify lithops source code):
fexec = lithops.FunctionExecutor()
fexec.map(my_map_function, iterdata, invoke_pool_threads=1000)

@zhanggbj Yes, we can add this value as a parameter in the configuration file. Meanwhile you can test different values of invoke_pool_threads by simply setting it in the map() call (no need to modify lithops source code):
fexec = lithops.FunctionExecutor()
fexec.map(my_map_function, iterdata, invoke_pool_threads=1000)

@JosepSampe thanks for your reply, i've opened a new issue https://github.com/lithops-cloud/lithops/issues/521.

@JosepSampe thanks for your reply, i've opened a new issue https://github.com/lithops-cloud/lithops/issues/521.


More on S3, storage, and IAM

Many thanks to @max-zanko for good suggestions.

@ThanosBaskous review? @max-zanko review welcome as well!

A lot of great changes in this PR @jlevy! I found a few small mistakes, had a couple of nits, and a few suggestions - only one or two of those major.

Awesome. Thanks for such a careful review. Will correct tmrw.

Thanks @max-zanko. I think I got all your suggestions too.


Make a playable game, no stylings

TODO

[x] Add an APIGW showing the unstyled hat game pages I have so far
[x] Put the APIGW behind CloudFront, as the default behavior. POSTs allowed.
[x] itest for every page that's done so far
[x] Room code backend processing
[x] itest the prod domain name in prod, requires a new CloudFormation output
[x] Get a failing test for the leader's auto-populating roster
[ ] Have the response to the POST to room-code send them to /room-code/

I should use /room in place of /room-code. You're creating a room, and you should wind up at /room/ABCDEF.

This did something to the DOM: aws --profile hat-game-prod apigatewaymanagementapi post-to-connection --data $(echo -n '
p2
' | base64) --connection-id ${CONNECTION_ID} --endpoint-url https://qu3vq8riw2.execute-api.us-east-1.amazonaws.com/ws-roster.

Not sure why messages show up partially as comments.


test(core): Add wishlist related tests

What/Why?

moved add/delete wishlist test from account.spec.ts to separate file wishlist.spec.ts
test with adding product to wishlist
test with creating new wishlist from drawer
test to make sure guest user cannot use wishlist
test to verify Favorites wishlist is default and cannot be deleted

Testing
See regression test run in CI (there is separate ticket to fix button test in visual regression)

🍹 I would also suggest we cleanup the accounts at the end of the test wherever we are creating them.

@avattipalli this is already done automatically for each test as a part of account fixture, see core/tests/fixtures/index.ts
await accountFactory.cleanup();


[EH] drop 3.5

addressing: #17236

NOTE: check if uamqp drops 3.5

uamqp won't drop 3.5 support in the 1.3.0 release, will drop afterwards.


Merge multiple .nyc_output from different path and report nothing

I run several tests in different sub directories, such as
workdir/repo1
workdir/repo2
workdir/repo3
......

there's a .nyc_output and coverage folder in every repo directory, and I want to merge these coverage results together. As https://github.com/istanbuljs/nyc/issues/1129 said, firstly merge these output together
nyc merge workdir/repo1/.nyc_output workdir/.nyc_output/repo1.json
nyc merge workdir/repo2/.nyc_output workdir/.nyc_output/repo2.json
...

Then I run nyc report in workdir and it shows
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

my nyc version: 14.1.1

You have provided none of the required information as specified in https://raw.githubusercontent.com/istanbuljs/nyc/master/.github/ISSUE_TEMPLATE.md.  Without that information it is not possible to help you.

@coreyfarrell
Link to bug demonstration repository
https://github.com/yjjnls/nyc-test
Expected Behavior
run sh test.sh
it should merge the two .nyc_output and report a merged result
Observed Behavior
report nothing
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

Environment Information
  System:
    OS: macOS Mojave 10.14.5
    CPU: (4) x64 Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
    Memory: 46.21 MB / 8.00 GB
  Binaries:
    Node: 11.15.0 - /usr/local/bin/node
    npm: 6.7.0 - /usr/local/bin/npm

Please create a package.json in the root directory of https://github.com/yjjnls/nyc-test.  This needs to include:
{
  "nyc": {
    "extension": [".ts", ".tsx"]
  }
}

nyc 14 and above run include/exclude/extension matching during reporting.  nyc 15 is currently in pre-release and has added .ts and .tsx the default extension list.  FWIW I recommend also configuring an nyc dependency in your root package.json to use the same version as the repositories you link.  Using the script as you have to install latest means that you get new semver-major releases without warning.

that works, thank you

@coreyfarrell I added the package.json and it reports all the result in the submodules. But the merged result is different from the one in the submodule.
For example, the root nyc report shows that
RsaCryptoSuite.ts           |    70.77 |    81.48 |       72 |    76.79 |... 16,117,118,119 |

while in the did-auth-jose, I ran npx nyc report, the result
RsaCryptoSuite.ts       |    74.36 |      100 |       80 |    74.36 |... 08,109,110,111 |

and the coverage html in the root shows that the red block is uncovered, that;s incorrect.

Please try the following in your test.sh:
git submodule update --init

cd did-auth-jose
npm install
npm test
npx nyc merge .nyc_output ../.nyc_output/did-auth-jose.json
cd ../hub-sdk-js
npm install
npm test
npx nyc merge .nyc_output ../.nyc_output/hub-sdk-js.json
cd ..

npx nyc report

Running nyc merge from the directory of each project potentially has a different result than running it from outside the project directory, one specific thought is that running nyc merge from the root directory prevents it from finding files in did-auth-jose/node_modules/.cache/nyc.

@coreyfarrell geat work


Delegate instantiation of interfaces

As far as I understood ExtendedXmlSerializer de/serializes interface properties by emitting/reading the actual implementation type from/to an xmlattribute. I don't want this behaviour because I want the implementing types to be independent on both sides of the wire. That means they implement the same interface but are not related otherwise. Instead I would rather like to instruct ExtendedXmlSerializer to use a factory supplied by me that knows about instatiating interface instances. Something like the following pseudocode:
public interface IDomainObject
{
    // interface members ...
}
public interface IDomainObjectFactory
{
    T Create<T>();
    object Create(System.Type interfaceType);
}

Then I would like to configure the serializer in the following way:
IExtendedXmlSerializer serializer = new ConfigurationContainer().Type<IDomainObject>()
    .InstantiateWith(() => new DomainObjectFactory().Create<IDomainObject>()) // unfortunately there is no such method
    .Create();

My question is whether there is any advice how I could implement something like that. Or any pointers at all where to look if I want to completly customize object instantiation.

Wow that is an interesting and powerful scenario, @taxophobia!  Thank you for writing in and asking about it.
For some context with ExtendedXmlSerializer, it was primarily meant for application state, for games, or really other applications that needed some internal memory easily stored as an object graph in the XML format.  Data exchange over the wire while (hopefully 😁) supported was not a focus.
So with that stated, we might be able to achieve what you're looking for via custom serializer.  It's not perfect as you would have to apply it on a type-by-type basis, but it's a starting point that we could explore.
I have created a basic prototype/example for you to check out here:
https://github.com/ExtendedXmlSerializer/home/blob/fb8c97867db70f7acf0197d9c7a7642ccec1f00a/test/ExtendedXmlSerializer.Tests.ReportedIssues/Issue440Tests.cs#L13-L62
Essentially, when an interface-type gets written, the full type name is written out.  And when it's read back in, the type is fetched via the written inner-content and activated.  You could replace that with a call to your IDomainObjectFactory.Create.
Again, not perfect, but a starting point we could further explore.  Please let me know how all of this sits with you and we can take it from there. 👍

Thank you very much for the example. It works for the usecase of serializing/deserializing a single instance without nested properties. For nested properties I think i should be able to adopt the  example of the AnswerToEverything extension from the wiki. Basically doing the instantiation and then relying on the default serializer to handle the properties. I will tinker a bit more and post code or more questions later.

Cool... good to hear you found a direction, @taxophobia.  A new release was posted to NuGet today so going through the issues here.  Sounds like this is sufficiently resolved for now.  If you do find that you have another question around this issue please do post a comment and I will re-open to further explore it with you.  Closing for now.


Cannot parse window function with named windows

Describe the bug
The PostgreSQL and MySQL databases allow users to give an alias for a window specification which can be after the OVER clause instead of the window specification. This allows users to avoid redundantly specifying the window in multiple places in a query. The parser is unable to parse queries containing the WINDOW clause.
References

WINDOW Clause - PostgreSQL documentation
Window Function Calls - PostgreSQL documentation
Window Functions - PostgreSQL tutorial
Named Windows - MySQL documentation

To Reproduce
Steps to reproduce the behavior:

Example SQL

SELECT type, day, MIN(days_since_previous) days_since_previous FROM (
    SELECT
        type,
        day,
        (COUNT(*) OVER win = 2) valid,
        (last_value(day) OVER win - first_value(day) OVER win) as days_since_previous
    FROM weather
    WINDOW win AS (PARTITION BY type ORDER BY day ROWS BETWEEN 1 PRECEDING AND CURRENT ROW)

Parsing this SQL using JSqlParser with this statements

        try {
            Statement st = CCJSqlParserUtil.parse("SELECT type, day, MIN(days_since_previous) days_since_previous FROM (\n" +
                    "    SELECT \n" +
                    "        type, \n" +
                    "        day,\n" +
                    "        (COUNT(*) OVER win = 2) valid,\n" +
                    "        (last_value(day) OVER win - first_value(day) OVER win) as days_since_previous  \n" +
                    "    FROM weather\n" +
                    "    WINDOW win AS (PARTITION BY type ORDER BY day ROWS BETWEEN 1 PRECEDING AND CURRENT ROW)\n"
            );
        } catch (JSQLParserException e) {
            e.printStackTrace();
        }

Exception

net.sf.jsqlparser.JSQLParserException
	at net.sf.jsqlparser.parser.CCJSqlParserUtil.parse(CCJSqlParserUtil.java:51)
...
Caused by: net.sf.jsqlparser.parser.ParseException: Encountered unexpected token: "OVER" "OVER"
    at line 5, column 19.

Was expecting one of:

    "&"
    ")"
    "."
    "::"
    "<<"
    ">>"
    "COLLATE"
    "["
    "^"
    "|"
...

Expected behavior
It should possible to parse a query with a WINDOW clause.
System

Database you are using: PostgreSQL, MySQL
Java Version 11.0.3
JSqlParser version: 3.0

a closing bracket is missing at the end
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW is the problematic/unsupported part.

The rest works fine with JSQLParser 4.7


Possible typo in "Understand variable syntax"

Link to doc: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#understand-variable-syntax
In the third paragraph, second line states "The template expression value does not change because all template expression variables get processed at runtime before tasks run."
However, template expression variables are processed at compile time, not runtime. Is this a possible typo?
For example, even the first line of the second paragraph says "In a pipeline, template expression variables (${{ variables.var }}) get processed at compile time, before runtime starts."

Thanks @soham-dongargaonkar! I will fix that shortly.


AL0603: Implicit Conversion

Description
Hi, I've just downloaded the latest AL Language from docker and I've found out this warning.
I have a simple question: How can it be handled? is there some c# cast or something like that?
if MyEnum in [0, 1, 2, 5, 6] then
this will throw this warning but the enums aren't they an integer-based?
Can I do something like that (or like C#)?
if MyEnum in (Enum)[0, 1, 2, 5, 6] then

You could achieve this by doing MyEnum.AsInteger() in [0, 1, 2, 5, 6]
But this rule generates another problem.
If you do that:
table 50000 "MyTable"
{

    fields
    {
        field(1; DocType; Enum "Sales Document Type")
        {

        }
    }

    local procedure MyProcedure()
    begin
        TestField(DocType, DocType::Order);
    end;
}

Then the TestField emits the warning on the second argument. There should be either an overload of TestField that supports Enums, or the rule should test the datatype between the field and the value in the specific case of a TestField.

@nicolassaleron I can understand the design issue with this - there are a few things that aren't as slick as I'd like, such as the fact that TestField on enum fields require you to now explicitly cast the Enum value to int.
SomeTable.TestField(SomeEnumField, SomeEnumField::SomeValue) raises the above error.
SomeTable.TestField(SomeEnumField, SomeEnumField::SomeValue.AsInteger()) fixes that, but it doesn't feel very slick. Admittedly it's a small price to pay for this feature.
I suppose since AL doesn't support generics/inheritance/interfaces, there's no real way to overload TestField to make it compatible with enum types, but I'm sure some compiler magic could be put in place, after all, it's .NET in the background which can support these features.

@charlespockert I completely agree with you and AsInteger is just a workaround to get rid of this problem without deactivating the rule for other interesting cases.

After downloading symbols" version 16.xx there were about 400 + similar error  , how to resolve it ?
manually replace by appending **.AsInteger()** at the end. or any other way to replace at one-go ? or any other option ? pls advise . thanks.

After downloading symbols" version 16.xx there were about 400 + similar error , how to resolve it ? manually replace by appending .AsInteger() at the end. or any other way to replace at one-go ? or any other option ? pls advise . thanks.

Depends, you can use AsInteger in cases where your option doesn't always match the enumeration (assuming that it's the intended behaviour).
Otherwise the best option is to replace references to option with the enum type in your table fields/code.
Enum is compatible with option at the schema level so there are no issues with moving from option->enum when deploying to production as far as I understand.

Noted. Thank you very much.

Charles , sorry, what is "AsInteger" ? How to write it ?

How do I fix this in 2020 w 1? AsInteger is not available.
Here's my line of code:
SLO.SetQtyToShip(rec."Document No.", rec."Document Type", true);

It looks like this can be closed, as the original question was answered.
For those wondering about how to use AsInteger(), it is just RecordVar."Enum Field Name".AsInteger() or EnumVar.AsInteger(). If you can't do that, you must not have an Enum, maybe an Option instead. Then you have a different problem.

For those wondering how to assign from an integer this is how it is done:
Level := Enum::Level.FromInteger(OrdinalValue); (Example from Kaufmanns page)


GPU utilization is 0 most of the time

nvidia-smi gives 0% percentage GPU-Util most of the time(in offline mode). Am I doing something wrong? Using the same config file as in [solution-3]. CPU usage is high most of the time.

Hmm hard to tell what is happenning but I'd play with the batch_size_train/inference for starters


how to change pitch and how to list all the available speaker ids

for  "hi"  model can you please give an example on how to change pitch and how to list all the available speakers for hi model

Any update on this?

List speaker ids -
tts --model_path "{modelDir}{checkpointName}" --config_path "{modelDir}config.json" --list_language_idxs --list_speaker_idxs

Try changing DEFAULT_SAMPLING_RATE to 18000, it worked for Telugu.

how can we change the default sampling rate? @ru2saig @kdcyberdude


Add multicolumn index to entities table on the subgraph and entity columns

Resolves #724

Needing this is a bit surprising giving that those fields are part of the primary key. This answer explains that this exact type of index would be created for composite primary keys, by order of declaration. So it seems we got the order of the primary key wrong, we have PRIMARY KEY (id, subgraph, entity) when PRIMARY KEY (subgraph, entity, id) would be better suit our use case.
So should we keep the primary key as-is and add this extra index, or should we do a migration for the primary so we don't need to add an index?

True, we are now. IIRC though, the current plan for supporting interfaces will involve querying entities by id + subgraph, because at the time of the query you don't know which exact entity type the results we have. At that point, we'll need the (id, subgraph, entity) primary key index again. (@leodasvacas, please correct me if I'm wrong).

I agree that if we are always including subraph and entity in the query filter it makes sense to simply alter the primary key index to be (subgraph, entity, id) rather than add a new index.
However, I am not completely sold on the base assumption that the query performance will be the same (just because we haven't benchmarked it yet.)  I'll do some quick performance testing.

@Jannis I haven't implemented that yet, but I believe the query will look like:
SELECT 1 FROM entities WHERE entites.subgraph = 'subgraph_id' AND entities.id = 'id' AND (entites.entity = 'interface_implementor1' OR entites.entity = 'interface_implementor2' ...)
So it filters by subgraph and id, but also by multiple entities. I don't know what would be the query plan here, so don't know if an index by (subgraph, id) or by (subgraph, entity, id) would be prefered by postgres, but I believe either would work fine.

To understand the query plans used better, I've executed a few queries on the production database.
Individual entity query
SELECT 1 FROM entities WHERE subgraph = 'subgraph_id' AND entity = 'foo' AND id = 'bar';

uses the subgraph_entity_idx btree (subgraph, entity) index.
Collection query
SELECT * FROM entities WHERE subgraph = 'subgraph_id' AND entity = 'foo';

also uses this index.
Interface query
SELECT 1 FROM entities
WHERE subgraph = 'subgraph_id'
  AND id = 'id'
  AND entity IN ('interface_implementor1', 'interface_implementor2' ...);

also uses this index.
My conclusion: I think we indeed only need the (subgraph, entity, id) index. @fordN @lutter Can we alter the primary key index with a migration or would it require a fresh entities table?

Updated the migration to rebuild the primary key index in the desired column order. Note that this will have to reindex the entities table and might therefore take some time in production.

Rebased on lastest master and merged.

In case anybody wonders about the failed tests: I was a little too quick merging this, and hte branches that Travis pulls to run its tests don't exist anymore, causing the failures.


search is missing packages

Some packages seem not to be listed, for example searching for oauth2 doesn't list the oauth2 package wich does exist.
reproduce:
rebar3 hex search oauth2
p1_oauth2: 0.6.1
```

vs:

![image](https://cloud.githubusercontent.com/assets/119093/13120663/cddcf328-d57c-11e5-9c33-3e1f68164ccf.png)

Haven't you asked about this before :). Rebar3 only displays packages it can build.

Ohhhh nope didn't ask about that before must have been someone else, sorry.


Can't update effects-ng. Could not resolve dependency error

Which @ngneat/effects-* package(s) are the source of the bug?
effects-ng
Is this a regression?
Yes
Description
I wanted to update effects-ng from v2.0.0 to the latest.
But npm install shows  Could not resolve dependency error.
I found a similar error but looks like it's not fixed #45.
Please provide a link to a minimal reproduction of the bug
No response
Please provide the exception or error you saw
npm ERR! While resolving<EMAIL_ADDRESS>npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/@angular/router
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR! node_modules/@ngneat/effects-ng
npm ERR<EMAIL_ADDRESS>from the root project

### Please provide the environment you discovered this bug in

```true
"@angular/core": "^15.1.5",
    "@angular/forms": "^15.1.5",
    "@angular/platform-browser": "^15.1.5",
    "@angular/platform-browser-dynamic": "^15.1.5",
    "@angular/router": "^15.1.5",

### Anything else?

_No response_

### Do you want to create a pull request?

No

@NetanelBasal ping you once more since there is new similar issue.

Done


Remove unused external functions and local external calls

Local external calls are fairly simple: it's when we use the function in the module via the external reference. Previously, with hotcode upgrades that might have made sense, but currently it doesn't, maybe even adds a tiny bit of overhead on each call. A couple of these were cleaned up in chttpd module.
Unused externals are functions which are exported but not used anywhere.
In some cases we didn't even use the function at all, like in the case of stream_init/1,2 or rexi:stream/1,2,3 we can remove their bodies as well.
Since stream_init is removed, update the init_stream as per https://github.com/apache/couchdb/issues/5122, where we noticed that the newer init_stream doesn't actually update the timeout metric, so before cleaning up stream_init move that to the init_stream.

This cleanup was prompted by investigating https://github.com/apache/couchdb/issues/5122 were we noticed that we have whole functions in rexi which are not used. When diving to debug an issue having dead code adds significant overhead needing to weed through and understand logic which is never actually run.
To automate the lookups and get an initial report used the excellent xref that comes with Erlang/OTP.  I used something like this as a starting point: https://gist.github.com/nickva/0de5c227249b66650a90a24d0a422273

Could we have an automated check for unused vars/functions on every CI run?

@big-r81 unlike missing functions, unused external are bit more complicated. Some modules have external functions that are useful for debugging, some are used in tests, some modules don't set up their behavior declarations properly (or xref doesn't handle behaviors well) so it there is a longer list to go through. I just went manually through the report and searched for usage by hand.


Bonnie/fix typo

Pull Request Check Sheet

[ ] Have you read and followed our Contribution Guidelines?
[ ] Have you made an issue in the OpenNMS issue tracker?If so, you should:

update the title of this PR to be of the format: ${JIRA-ISSUE-NUMBER}: subject of pull request
update the JIRA link at the bottom of this comment to refer to the real issue number
prefix your commit messages with the issue number, if possible

[ ] Have you made a comment in that issue which points back to this PR?
[ ] Have you updated the JIRA link at the bottom of this comment to link to your issue?
[ ] If this is a new feature, is there documentation?
[ ] If this is a new feature or substantial change, are there tests that cover it?

Pull Request Process
One or more reviewers should be assigned to each PR.
If you know that a particular person is subject matter expert in the area your PR affects, feel free to assign one or more reviewers when you create this PR, otherwise reviewers will be assigned for you.
Once the reviewer(s) accept the PR and the branch passes continuous integration in Bamboo, the PR is eligible for merge.
At that time, if you have commit access (are an OpenNMS Group employee or a member of the Order of the Green Polo) you are welcome to merge the PR.
Otherwise, a reviewer can merge it for you.
Thanks for taking time to contribute!
External References

JIRA (Issue Tracker): http://issues.opennms.org/browse/${JIRA-ISSUE-NUMBER}
Continuous Integration: Bamboo, CircleCI

Doc edits look good. The .DS_Store files should be removed from the tree though.

Doc edits look good. The .DS_Store files should be removed from the tree though.

Googled how to remove them from the Mac repo, and I think I've done that, but less clear on how to remove them from the git.


feat(transfer): add back-icon and forward-icon slots

3142

 Quality gate: Passed
4 Issues

 4 Code issues

Details


Convert Basetools user manual RTFs to Wiki pages. (Bugzilla Bug 1625)

This issue was created automatically with bugzilla2github
Bugzilla Bug 1625
Date: 2019-03-14T22:19:13+00:00
From: BobCF
To:   BobCF
CC:   edk2+bugs+int+994+563148131503455288, yonghong.zhu
Last updated: 2019-03-20T23:07:53+00:00

Comment 6725
Date: 2019-03-14 22:19:13 +0000
From: BobCF

Industry Specification: ---
Releases to Fix: EDK II Master
Target OS: ---

Convert the following RTFs under BaseTools/UserManuals to Wiki pages.

Build_Utility_Man_Page.rtf
LzmaCompress_Utility_Man_Page.rtf
EfiRom_Utility_Man_Page.rtf                         PatchPcdValue_Utility_Man_Page.rtf
GenCrc32_Utility_Man_Page.rtf
GenDepex_Utility_Man_Page.rtf
SplitFile_Utility_Man_Page.rtf
GenFds_Utility_Man_Page.rtf
TargetTool_Utility_Man_Page.rtf
GenFfs_Utility_Man_Page.rtf
TianoCompress_Utility_Man_Page.rtf
GenFv_Utility_Man_Page.rtf
Trim_Utility_Man_Page.rtf
GenFw_Utility_Man_Page.rtf
UtilityManPage_template.rtf
GenPatchPcdTable_Utility_Man_Page.rtf
VfrCompiler_Utility_Man_Page.rtf
GenSec_Utility_Man_Page.rtf
VolInfo_Utility_Man_Page.rtf
Intel_UEFI_Packaging_Tool_Man_Page.rtf

Change tool name from <toolname>.exe to <toolname> for python tools, Since we do not generate .exe file for python tools.

Update the User Manuals if its content is not align with the current tool behavior.

Comment 6803
Date: 2019-03-20 23:07:53 +0000
From: Yonghong Zhu <<yonghong.zhu>>
Bob will work on it.


Adding a non-detected window

Many times when I open up a new application, the new window isn't tiled with the rest of the open ones. It doesn't show on chunkc tiling::query --desktop windows. I have to restart chunkwm for the window to get detected.
Unless there's some command I can do to get it detected that I just don't know about.

It might be helpful if you could try to see wh exact steps cause this to happen. Since I've never heard of this, I think most users don't experience this. Is it a specific app? Does it only happen under certain conditions? For instance, when many windows are open? Etc.

I'm using an iMac on 10.12.6 with a 2.7GHz i5, 16GB RAM and a terrible 5200rpm HDD. Apps do take a while to startup.
Running chunkwm 0.2.36 with About 9 tiles across 6 workspaces, split between two monitors.
For example, I just launched Excel, and it didn't tile. Constantly opening and closing Workbooks made them never tile, they just floated as normal non-chunkwm windows, but with the regular blue border around them.
That said, when I closed Excel through Cmd+Q and reopened it, everything tiled properly, and had no issues from there on out. Further Excel restarts didn't have any issues.
Same exact behavior on OneNote, Xcode but not Sublime Text or Visual Studio Code. Seems to be apps on the heavier side, but only on their first run.

This most likely happens because the accessibility api hits a timeout during application startup, for applications that are slow to start.
See https://github.com/koekeishiya/chunkwm/blob/master/src/core/state.cpp#L287
If you enable logging in your config file and set the log-level to warn, you can see if this is indeed the issue you are having:
chunkc core::log_level warn

If you are not running chunkwm from a terminal, you can also specify the output log-file:
chunkc core::log_file ~/.chunkwm_log

You can then view the log when running chunkwm by opening a terminal: tail -f ~/.chunkwm_log

Yeah that's exactly it.
2018-02-08 11:13:29 WARN : 99951:Microsoft Excel could not register window notifications!!!

I do have this issue with one or two applications myself, and it is very annoying. I'm just not sure how to best resolve this type of problem..

I guess my Mac Mini is too slow so this issue occurs every time when I start Firefox / Chrome the first time after system booted.

fixed at master.


[Feature Request] Use 2 different values for 1 app

I'm sure that

[X] This issue is still present in the current beta version of this adapter
[X] There is no other (open) issue with the same topic (use the search!)
[X] This issue is not described in the adapter documentation / FAQ (read the docs!)

Version of awtrix-light firmware
0.96
The problem
My battery system for solar power has two different objects in ioBroker:
One item for charging (packInputPower), one for discharging (outputPackPower).
The batteries can either be charged or discharged - not both at the same time.
It would be great to have ONE app - but either show a charging animation together with the value of the charging-object - or shows a discharging animation with the other value of the discharging-object.
If this is not possible, perhaps two apps, but only one is shown that is active.
iobroker.current.log (in debug mode!)
No response
Version of nodejs
20.17.0
Version of ioBroker js-controller
6.0.11
Version of adapter
1.3.0

You can do both yourself with the current ioBroker on-board means:

If you want to display two apps alternately, use a script to switch their data points awtrix-light.[instance].apps.[appname].visible back and forth appropriately when changing between loading and unloading.

If you want to use both display types in a joint expert app, you can use a script to set the data points awtrix-light.[instance].apps.[appname].icon (and also awtrix-light.[instance].apps.[appname].text if necessary) appropriately when changing between loading and unloading.

I used a Blockly script now and it seems to work.
But the sort order is mixed up now.
It doesn't match the order I set in the adapter and doesn't match the order A>Z in objects.

Have you already tried "Is it possible to re-order apps?" in https://github.com/klein0r/ioBroker.awtrix-light/blob/master/docs/en/README.md?

Yes, I already moved them up and down. It is set as mentioned above "set in adapter".

invalid jsonConfig: Please create an extra issue for this topic.

Done.

Nevermind, found the "Position" input field :) Works now as expected :)

Since the adapter has an option for an alternative icon why no option for an alternative value?
So the question was placed perfect! 👍🏻 No reason to be the "guard" 😉
I already have fixed it with an additional script.

I'm not a guard, I try to be a friendly helper - for users and for developers.

As @hsteinme already said, it is possible to achieve this by using any logic adapter.

It would be great to have ONE app - but either show a charging animation together with the value of the charging-object - or shows a discharging animation with the other value of the discharging-object.

Just create a new state in 0_userdata.0 and calculate the value e.g. with the javascript adapter based on the other states


What is the key property for ?

Hello,
I see that vnode.js extracts key from data.key
and in snabbdom.js, key is used for checking if same vnode
function sameVnode(vnode1, vnode2) {
  return vnode1.key === vnode2.key && vnode1.sel === vnode2.sel;
}

I don't really understand the use case for key? In what scenarios would I want to set data.key in my h constructs? ~~Thanks

It is for arrays of children. The index may have changed (new sort, splice or shift) but the node is the same. key is used to assign a unique id within the array, independent of the index.

Yes. If you reorder element then the key property is used to figure out which element moved where.

Thanks!
I'm wondering: if i assign unique keys to sortable children, does this also
mean that snabbdom can more efficiently handle patching when they are
sorted?
Le 28 mars 2016 08:21, "Simon Friis Vindum"<EMAIL_ADDRESS>a
écrit :

Yes. If you reorder element then the key property is used to figure out
which element moved where.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/paldepind/snabbdom/issues/92#issuecomment-202362321

Thanks @paldepind @hville -- I looked at the source code a little bit more and this makes sense now with your commentary


Added .options method for simplified option setting

Adds a .options method to all objects making it simpler to set options in a flattened way. Two formats are supported:
img.options(width=600, cmap='viridis')

and the more explicit:
img.options({'Image': dict(width=600, cmap='viridis')})

which allows usage on complex nested objects.

[x] Addresses https://github.com/ioam/holoviews/issues/222, https://github.com/ioam/holoviews/issues/803, https://github.com/ioam/holoviews/issues/2287
[x] Add unit tests
[x] Add docstrings
[x] Add section in getting started and user guides
[x] Move logic into separate utility
[x] Add DynamicMap.options

I'm very happy with this PR now, the .options method is really convenient and because it has to actually resolve the difference between different kinds of options its also much stricter than other approaches, which should result in a better user experience.
The main question I have now where to document it (I want a section in both the getting started and user guide) and what our guidance on using it is. Should we use it throughout the docs? @jbednar  @jlstevens

Ready to review.

I think we should use this approach consistently throughout the docs, certainly in all places currently using .opts, but also replacing magics in many cases.

I haven't looked at the changes to the docs (i.e notebooks yet) but I have looked at the code and made a few comments. It mostly looks fine and I am happy the bulk of the new functionality is implemented as a classmethod on hv.opts.

Happy to review once the merge conflicts are resolved. I doubt I'll have many new comments to add once that is done.

Other than our ongoing discussion (which will become another PR), I am happy to merge once all the tests are green (though I think another PR that is about to be merged will touch the test data).

Making good progress on getting the validation unified so +1 on merging asap. This PR shouldn't touch test data so I'll try to get it passing without waiting on the other PR.

Also finding a number of bugs in the existing option validation, so refactoring this is definitely a good idea. How do you feel about dropping the skip_invalid option? I feel like it should either warn or error, although I'm happy to be persuaded otherwise until we have a way of specifying an explicit backend in the opts magic.

How do you feel about dropping the skip_invalid option?

Fine by me. If I remember correctly, that was the first approach we had when we started supporting multiple backends. I do think it is less relevant now.

Ready to merge now, refactoring will happen in https://github.com/ioam/holoviews/pull/2354.

As the tests are passing, I'll go ahead and merge.


Multiply plain repeatedly and scale out of bound error

Hi,
I am trying to multiply a ciphertext by a plaintext repeatedly. When I use the python version Pyfhel, I am able to do repeated multiplications without much variation in noise margins. However, when I use the C++ version of SEAL, I run out of scale quickly resulting in "scale out of bounds" error. I am following the ckks_basics.cpp and use the same parameters as in the example file. Any help would be appreciated.
Thanks

First, allow me to clarify a few things.

Pyfhel is not maintained by us. We have no more knowledge of it than you.
Scale has nothing to do with noise. "Scale out of bounds" can appear even if noise is extremely low. Although repeated multiplication of a ciphertext by a plaintext will slowly increase the noise, it is not the reason why you see "scale out of bounds".
"Scale out of bounds" error specifically means that the scale of a ciphertext or plaintext is larger than the product of all elements in coeff_modulus. If you perform multiplications without rescaling, you can quickly see this error. The more rescaling you perform, the less elements will be left in coeff_modulus. Even if you managed to have the same scale in a ciphertext after every multiplication and rescale, eventually the coeff_modulus can be too small to accommodate another multiplication.

Could you please post your source code (together with output log) so that I can reproduce the error and look into it?

Also please note that Pyfhel uses a really old version of SEAL where the parameters (specifically coeff_modulus) have a different meaning than what they do today. Therefore, the encryption parameters you use with Pyfhel will not translate directly to a modern version of SEAL, which might be causing the problem you are seeing.

Thanks for the response. I am trying to multiply an encrypted 2 with a plaintext 2 repeatedly. Below is my code and the corresponding output. Please let me know if you need any more information
int main()
{
EncryptionParameters parms(scheme_type::CKKS);
size_t poly_modulus_degree = 16384;
parms.set_poly_modulus_degree(poly_modulus_degree);
parms.set_coeff_modulus(CoeffModulus::Create(
poly_modulus_degree, { 60, 40, 40, 60 }));
double scale = pow(2.0, 40);

auto context = SEALContext::Create(parms);

CKKSEncoder encoder(context);
size_t slot_count = encoder.slot_count();
cout << "Number of slots: " << slot_count << endl;

std::cout << "Generating keys ..." << std::endl;
KeyGenerator keygen(context);
PublicKey public_key = keygen.public_key();
SecretKey secret_key = keygen.secret_key();
RelinKeys relin_keys = keygen.relin_keys();
GaloisKeys gal_keys = keygen.galois_keys();

Encryptor encryptor(context, public_key);
Evaluator evaluator(context);
Decryptor decryptor(context, secret_key);

Plaintext inputPtxt;
encoder.encode(2, scale, inputPtxt);
Ciphertext inputCtxt,inputCtxt1;
encryptor.encrypt(inputPtxt,inputCtxt);

for(int itr=0;itr<10;itr++)
{

	cout << "Scale Before multiplication = " << log2(inputCtxt.scale()) << " bits"<< endl;
	evaluator.multiply_plain_inplace(inputCtxt,inputPtxt);
	evaluator.relinearize_inplace(inputCtxt, relin_keys);
	evaluator.rescale_to_next_inplace(inputCtxt);
	cout << "Scale After multiplication = " << log2(inputCtxt.scale()) << " bits"<< endl;

	inputCtxt.scale() = pow(2.0, 40);
	inputPtxt.scale() = pow(2.0, 40);
	parms_id_type last_parms_id = inputCtxt.parms_id();
	evaluator.mod_switch_to_inplace(inputPtxt, last_parms_id);

	// Decrypt the result
	Plaintext decrypted_data;
	vector<double> result;

	decryptor.decrypt(inputCtxt, decrypted_data);
	vector<double> result1;
	encoder.decode(decrypted_data,result1);
	cout << "After " << itr+1 << " multiplications " << result1[0] << endl;
}
return 0;

}
The output is:
Number of slots: 8192
Generating keys ...
Scale Before multiplication = 40 bits
Scale After multiplication = 40 bits
After 1 multiplications 4.00001
Scale Before multiplication = 40 bits
Scale After multiplication = 40 bits
After 2 multiplications 8.00004
Scale Before multiplication = 40 bits
terminate called after throwing an instance of 'std::invalid_argument'
what():  scale out of bounds

@nnarendra007
It seems you are using the ckks scheme，Make sure you have enough moduli （say > 10 in your case)

As @fionser, there is no way your code could work because you have so few coeff_modulus primes. Each sequential call to rescale_to_next_inplace removes one prime from the parameters, so clearly you won't be able to do that 10 times when you have only 4 primes (3 data primes). Check out 3_levels.cpp and 4_ckks_basics.cpp in native/examples/.

Thank you for the responses. I was able to fix the code and get it running for 10 iterations. However, is there any documentation about how to select the primes for the coefficient modulus? I need to multiply the plaintext with ciphertext for about 50-60 iterations.
Note: I was able to run it for 50 iterations on the python version which probably sets the parameters automatically without we specifying it.
Thanks again for the replies

Hmm, well, 50-60 iterations of what you have there is really costly and would require enormous parameters. Maybe Pyfhel just sets a really large coeff_modulus without any parameter security checks that would be enabled by default in modern versions of SEAL?
Maybe the computation could be organized in a lower-depth kind of way, but the code snippet you posted doesn't make sense to me, so I'm not sure if this is the case. Can you explain what you are trying to do?

I basically need to multiply an input ciphertext with a chain of plaintexts. For Eg: I have an input ciphertext encrypting a value 2.0. That needs to be multiplied by a chain of plaintexts encoding an array say [2.0,3.0,4.0,5.0,........]. The length of the array will be big say 50 as i mentioned in previous post. Now the input ciphertext gets multiplied by 2.0 in first iteration, 3.0 in next iteration and so on to finally get my resultant ciphertext after going through the array. My question was to see if we can have the number of primes equal to the array length so that I can do the multiplications successfully

Are the plaintext coefficients always integer scalars, or are you hoping to multiply different CKKS batching slots with different -- possibly non-integer -- values? If you truly hope to multiply with non-integer vectors, and you can't multiply the plaintexts with each other beforehand, you'll need at least poly_modulus_degree == 65536 to have a secure implementation, which is not great from a performance perspective.

Yes. That is exactly what I would like to do. Thank you for all the clarifications. I would then need to rethink on my problem formulation. Thank you all for the responses.


Terminal Crash on Resize (urxvt)

Hello again. I'm running urxvt on AwesomeWM. After installing Fish_Line I get a crash any time I try to resize more than a few pixels.

Sorry for my late awnser.
I've tested fishline and fish in different environement, apparently under evilvte (minimal vte term) I can reproduce it but under Sakura everything works great.
The error message I get seems to come from fish (it stays on the screen for 1 second before exiting).
Can you check if this is a fishline specific bug or a fish one. Just change you prompt to something like
echo $$ and do the same thing.
Thanks for reporting.

Definitely a fishline bug. Seems to be related to the "N" segment. I don't have time tonight, but I'll run some tests tomorrow.

I could not reproduce your bug on URXVT/evilvte if the powerline fonts are not loaded, it seems weird, it might be because I am loading the special charactere without their espace sentence (I will change this in the future release).
Also, after checking some of the fish issues I found that you are not the only person to suffer from this and it is a prompt related bug.
If you can make more testing I will be glad to fix it for this prompt.

I reported your problem to the fish-shell project (here). It might be fix in the next minor version as shown by the issue.
If you get more info, you can post it here or in the fish project's issue.
Thanks

I upgraded to from the stable release of fish to the git version and my issue seems to be resolved.

Glad your bug is fixed.
Issue closed because this is a fish-shell bug.


Lets-Chat behind nginx reverse Proxy

Hi @all,
did someone installed Lets-Chat behind an nginx reverse Proxy and can share the configuration with me. My problem at the moment is that the official URL is for example https://server.com/letschat/ and then the client tries to connect to https://server.com/login and this connect fails.
Thanks a lot
Regards
Manuel

In my environment, this configuration works correctly.
 location /letschat/ {
   proxy_pass http://<IP_ADDRESS>:5000/;
   proxy_redirect /  /letschat/;
   proxy_redirect default;
   proxy_http_version 1.1;
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection "upgrade";
   proxy_set_header Host $host;
 }

That works.
Thanks a lot.

thank you @nomeaning777


Refresh error - blank page when click the reload button

On each page, when reloading the page with the Reload button on Top Bar or on each tab, the result is the same:
Uncaught DOMException: Failed to execute 'insertBefore' on 'Node': The node before which the new node is to be inserted is not a child of this node.
    at insert (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:8191:16)
    at move (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:5123:9)
    at move (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:5096:7)
    at activeBranch.transition.afterLeave (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:2338:15)
    at performRemove (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:5210:20)
    at el._leaveCb (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:2850:9)
    at finishLeave (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:7792:13)
    at resolve2 (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:7822:30)
    at whenTransitionEnds (http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:7903:12)
    at http://localhost:8001/naive/node_modules/.vite/deps/chunk-V6FUCMFK.js?v=ae91ebc1:7830:11

Only when click CTRL+F5 the page will display.

I don't know what's going on over there with your sample tests okay


Option to install latest versions of PHP/Tomcat/etc.

Would be nice to have vars that allow version picking.

Tomcat is currently limited to centos officially approved version. I'm ok with this.
PHP now has vars in vagrantfile:
PHP_VERSION = '5.6' # Available options: 5.4, 5.5, 5.6, 7.0, 7.1, 7.2
PHP_MEMORY_LIMIT = 256
PHP_TIMEZONE = 'America/Los_Angeles'
PHP_MAX_EXECUTION_TIME = 60

Commit coming soon.


Upgrade from PHP 5.5.9 to 5.6: undefined method Closure::country

Hi,
I am running ubuntu 14.04.5 which used php5.5.9 as the default PHP version. Lets call this SERVER 1. Recently I have upgraded to PHP5.6, and I get the following error:
PHP Fatal error:  Call to undefined method Closure::country() in /home/domain/public_html/boxbilling/bb-library/Box/TwigExtensions.php on line 101

The line 101 is
$record = $this->di['geoip']->country($value);

I am also running another ubuntu 14.04.5 server ( let's call it SERVER 2), and I also upgraded php from 5.5.9  to 5.6 in the same way, but when I copy the boxbilling files and database from the previous server to this server, and I see it works fine, no error happens.
When I compared the list of installed packages of the 2 servers, here is the list of packages that is in SERVER 2 and not in SERVER 1
libtsan0
rsync
sudo

Then in SERVER 1, I tried installing libtsan0 (even though I don't know what it is)
apt-get install libtsan0

I got the following error:
Unable to locate package libtsan0

I guess my SERVER 1 doesn't have some components neccesary for boxbilling to work in PHP5.6. (When I switch it back to 5.5.9, it works again!)
Could someone here help me troubleshoot the problem? I want it to work in PHP5.6 and in SERVER 1.
Thank you very much!

Try upgrading to PHP 7.0. I use that with BoxBilling and it works just fine.


enabling pelican-bootstrap3 affects 'codehilite' extension in MD_EXTENSIONS

If I have pelican-bootstrap3 enabled, syntax highlighting works fine as long as I don't define the MD_EXTENSIONS variable myself in pelicanconf.py
However, if I set the variable to:
MD_EXTENSIONS = ['codehilite', 'extra']

the output is not highlighted. Perhaps it can't find the appropriate css, as the language identifiers are being stripped out, e.g.
:::python
def hello():
print "Hello, World!"
becomes
def hello():
print "Hello, World!"
there just is not any syntax highlighting.

Hi!
First of all, thanks for your interest in this theme! It's with heavy heart that I have decided that I can no longer maintain this theme due to lack of time and changing interests. To makes sure the theme lives on however and not goes to waste, I've donated it to the Pelican community. It can now be found in the official pelican-themes repository. I'd suggest you re-submit your issue  there.
Sorry for the inconvenience!


⚠ Bofrost.fr has degraded performance

In 6442b81, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 845 ms

Resolved: Bofrost.fr performance has improved in bec257f after 6 minutes.


Added Pull Up Resistor on CP2102 RST Pin

As per the datasheet Typical Connection Diagrams, a 1kΩ pull-up on the RST is recommended

Solved manually in current version. Thanks for the pull!


add cromwell utility scripts

This PR adds some utility scripts I've been using lately for working with cromwell - Mainly after getting metadata and outputs and processing them into e.g. timing data or into separate output lists.
Some of this might overlap with what cromwellInteract does - In that case it might be that I should use that instead, so feel free to point that out. I didn't however see if there's functionality for listing and flattening outputs from an output json, or for extracting task-specific data from a one or multiple metadata jsons, so created this.

stale, not merged


GSSOC'23: Plant Disease Detection

Is your feature request related to a problem? Please describe.
Yes .
Farmers who grown plant/vegetables are facing lot of economical Loss every year because of various Disease that can happen to a plant/vegetables . In our problem i have found 38 different types of disease of 15 plants .  If a farmer can detect these early and apply appropriate treatment it can save lot of waste and prevent the economical loss .
Describe the solution you'd like
In this Project , i have build an end to end machine learning Project in Agriculture Domain to solve the problem of Plants disease. In this Project i have build a web application using React.js which will deployed to the cloud and anywhere form the world can acess this applicaiton . All they just need to do give the image of the leaves of plant and the application will tell you disease of the plant with accuracy . Behind the scenes it will be using deep learning model and CNN .
datasets used : https://www.kaggle.com/arjuntejaswi/plant-village
Describe alternatives you've considered
No response
Additional context
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

please assing me this if it's found relevent .

https://github.com/The-Data-Alchemists-Manipal/MindWave/pull/518
@khusheekapoor i have completed the raised issued . Please take a look and LGTM! . and please assign the label of gssoc'23

oohk from next time will take care of it !
Thanks for informing


fix(security): vulnerabilities found in fabric-all-in-one

List of vulnerabilities found in fabric-all-in-one image during Azure Container scan.

VULNERABILITY ID
PACKAGE NAME
SEVERITY

CVE-2021-36159
apk-tools
CRITICAL

CVE-2021-30139
apk-tools
HIGH

CVE-2022-28391
busybox
CRITICAL

CVE-2021-28831
busybox
HIGH

CVE-2021-42378
busybox
HIGH

CVE-2021-42382
ssl_client
HIGH

CVE-2021-42383
ssl_client
HIGH

CVE-2021-42384
ssl_client
HIGH

CVE-2021-42385
ssl_client
HIGH

CVE-2021-42386
ssl_client
HIGH

CVE-2022-1271
xz
HIGH

CVE-2022-1271
xz-libs
HIGH

CVE-2018-25032
zlib
HIGH

Marking as P3 because the Fabric AIO image is not meant to be used in production.

Hi @petermetz can you assign this to me? Thank you


Issues with lots of themes installation

I have tried several themes, and I encountered lot of issues with them.
Often, after installation, there is simply blank page displayed. Also, sometimes the documentation on how to modify texts and use the theme is not available at all and we have to guess things.
Examples of themes:

clean-blog: index.md page is not displayed, only the theme logo, cannot use it.
bulma: blank page after installation. Intallation steps unclear. Cannot install it.
dimension: no explanation on how to change background image or add pages and contents. Bug was created 3 years ago without any solution provided.

And that is the first ones I used so it is very disappointing. It is sad, because this doesn't seem to be a Pico CMS issue, only the theme providers issues. However this hurts Pico CMS as people try a few themes and after 30 minutes that it doesn't work the let Pico CMS down..
I suggest that all themes would be tested again, with latest Pico CMS version, from scratch with latest themes versions, and correct the mistakes that come up. Everything needs to be working out of the box, as people are new to Pico and don't know how to dig into each theme to fix them so it works.
Installation instructions should be extremely clear and not complicated, telling what directory must be copied where, it is a pain to understand and figure this out, for all themes. Also instructions on how to modify pages contents should be available, else it is not worth publishing a theme that causes so much issues. Theme publishers should be contacted or else the theme should simply be removed.
That will certainly make Pico CMS much more attractive and usable for newcomers.

I've addressed much of this on https://github.com/picocms/Pico/issues/592, but, if there is a specific theme you really like, I'd be happy to help you get going with it.
Unfortunately, most of the themes are just not things that are ready to go out-of-the-box, and/or have limited documentation provided.  Sometimes, a little technical know-how is needed to diagnose and correct issues you may encounter.  We try to make Pico as user-friendly as we can, but we're a small team.
Many of these themes were contributed once, then not touched again.  If we were to enforce stricter guidelines and/or remove themes that weren't 100% perfectly documented, we'd have very little remaining to show in our gallery.
While I agree that the current situation isn't ideal, the alternative of simply not having any themes isn't much better.
Pico's original intention was kind of that you'd just create your own theme and it would provide you the framework to turn your Markdown into a webpage.
While the themes gallery is a good showcase of what others have created, there will always be some amount of web development or administration left to the end user.
While we'd love to have more "Ready-to-go" options in our gallery (and this is something that may happen in the future, as I noted on https://github.com/picocms/Pico/issues/592), it's really up to the community.  And just like our Pico team, our community is really kind of small.

As explained by @mayamcdougall, we're a very small team and we just don't have the time to officially support 3rd-party themes and to do quality assurance. In the end we have to decide whether we want a place for themes, or not. Pico is no one-click CMS (even though it actually is from a technical point of view, but that's kinda nitpicky :laughing:), it rather targets web developers which seek for a very simple, yet powerful alternative to create their own website. Thus we require some basic knowledge in the used technologies (like Markdown, YAML, Twig and web development in general, not necessarily PHP).
Most 3rd-party plugins and themes were made by people for their own websites, thus documentation never was their top priority - and here we are now. As explained by Maya, we can't just fix the issues with these themes, since we simply don't have permission to do so. Most issues are well known and have been reported to the developers already, but a lot of those projects are kinda abandoned. Just to make this clear, this is fine! All of this is open source, people are sacrificing their spare time, we simply have no entitlements here. We can only choose whether to include them in our list, or not. For now we have decided to rather include them, even though this is no final decision. More opinions are very welcome.
Without any question, this situation is suboptimal. Any help is very much appreciated @gelavat! :heart:

Thanks both for your interesting explanations and inputs.

In my opinion removing completely the themes would not be an option. Pico CMS with the default theme is very ugly. It is ok as we know that we can install other themes, but without those themes it would not be usable nowadays, at least I wouldn't have used it.
Also, with the themes that are currently available, the most beautiful ones are really nice once they work!
I am already in contact with some of those themes to improve the documentation on how to install and customize the content. E.g. https://github.com/unicate/pico-bulma/issues/3
These two parts are very important because Pico noobs like me really need it. We don't know the structure, the programming, etc... although now I start to get it more precisely, you have to see that newcomers are just willing to install Pico CMS and then quickly the best theme they find. And of course, as now it is very difficult to make many themes work, they will get stuck if they choose the bad one and not try further and simply stop using Pico. However usually there are just a few little things to correct and hence it seems that proper documentation will really make things done the right way. It doesn't need to be a full automatic installation, what is available now is very fine, we understand that it needs to be installed manually and this is ok.
So I would do those recommendations that could be very helpful:
You could add a requirement for the themes providers that the installation and customization sections in their Readme.md file are the most accurate possible. They should test on a new directory their github content, following the Installation section of their readme.md file, and it must work out of the box with the pictures shown and everything like in their demo page, or very close to it. Also a customization section must exist, so that people know much quicker how to change the content as they need.
With this, I am pretty sure that it will solve the issue completely. Of course, removing a theme if those sections are not working would be the last option, to be avoided, but at least they get the guidelines and are motivated to do it. That should be stated somewhere in your theme creation guidelines.
In parallel to this, I would recommend those things:

Create a 'top five' themes section in your themes list, that would be the best design and usable themes, in big. This will help people to quickly select one for their need. Of course others might still be good too. And these top five themes would be strongly watched by your team, to ensure that they really work out of the box, following the readme.md and the installation and customization sections. With three it is already much easier to maintain. Here you would need to provide Pull requests to the theme publisher in case something goes wrong, which will help them correct it quickly. usually honestly it is very little things, but that end up with a blank page or so...
Also, if they get an issue request from your team, they will take the modification request more seriously and hence implement it quicker.
Do a quick basic installation verification of each themes available from time to time. It could be done by your team or by helpers, reporting to you what theme is not working, and trying to help. So you would get a list of 'not so compatible' themes, mentioning what Pico version is related. This could be noted on your site, so people know if it is well compatible or not. But the theme would not be removed. It would be a quite low priority of work, maybe just once a year or so.

So this is enforcing things a bit but not too much, with quite a low overhead.
I am sure it would help consistency, robustness and adoption of Pico CMS.
Thanks for that great tool!

Thank you very much for your input @gelavat :+1: We'll definitely take this into consideration, these are very good suggestions, but I'm afraid our time is too limited to actually make this happen. Maybe you wanna help us and undertake some responsibility here? Your help with reporting these issues to the respective developers and possibly also providing them with some suggestions on how they could improve their docs (you did struggle with them, so your input is way more helpful than ours could ever be) - possibly with a ready-to-merge pull request - is very welcome! :heart:

Pico CMS with the default theme is very ugly.

Don't let @PhrozenByte see that, lol. 🙈
But no, the default theme isn't meant to look like a modern, professional website.  It's just a template you can use as a starting place for your own theme or to learn how Pico's theming works.  It's simple on purpose. 😉
Anyway, the biggest hurdle with any of these ideas is time. @PhrozenByte and I both volunteer our time to keep Pico maintained, with anything programming-related being on @PhrozenByte (I'm actually not a PHP developer. 😅).  I try to handle the community and website tasks that way he doesn't have to worry about them.
Your ideas are valid.  I don't think we get enough theme submissions for it to be practical to implement a "Top Five" guideline though (or to spend the time organizing, making the criteria, documenting the guidelines, managing and maintaining the "top five" themes themselves, etc).  We get a new theme every couple months at best.
But, I do plan to implement a "Ready to Go" tag or label in the not-too-distant future.
And if I had more time, I'd pump out a few more themes because I enjoy porting them.  I'm not much of a designer myself, but there's a lot of free HTML themes out there.  If you look, many of the more modern themes in the gallery will say "Ported by/Original by" in their description.
My most recent effort was porting Stellar from HTML5 UP.  If you want an example of what a "Ready to Go" theme might look like, Stellar is what I have in mind.
Most of the others I've ported were done a couple years ago.  With Stellar, I had a bit of programming experience under my belt, and as such, took a much more programatic approach to its functionality.
Where some of the older ports were as basic as "Customize the strings on your web page by filling out these YML properties", Stellar's port allows a deeper level of customization without ever touching the Twig template.
I don't have a demo site for Stellar, since it's just a port, but you can compare HTML5 UP's demo site with the site for my iOS app, CheriPom, which is what I ported it for.
(Also, @PhrozenByte, slight plug for my own work. I haven't shared it before now because it was never relevant to Pico, but I've totally been waiting for somewhere to work that in. 😅)
The documentation for Stellar is a bit long and hopefully covers everything you'd need to know.  I haven't exactly had to follow it myself though, so there could always be some things I missed.  If you'd like to try it out in your Pico adventures, I wouldn't mind hearing your feedback on what worked for you and what didn't. 😉
As I said, in the future I'd like to do more of these all-in-one style themes.
Hopefully it doesn't sound like I'm shooting down too many of your ideas here.  I can't stress enough that they're not bad ideas, they just don't fit our current situation that well.  We do appreciate the input though, and technically, we could probably use the extra kick to get motivated on some of this stuff again. 😂

Hello @PhrozenByte . Thanks for your inputs.

Maybe you wanna help us and undertake some responsibility here?
Well, maybe why not. I won't come often, just from time to time. But it could help.
I provided already a PR for the clean-blog with installation instructions update and a customization section, as per your suggestion.
It has been accepted and now I think it works out of the box, I am very happy.

I think this is the solution. Also in case the themes repo owners do not answer, I think that we could just fork them and provide the fork address, so we can continue to improve the doc at least, or even more the theme in itself. Whenever they want to update that to their base repo we can then switch back to their repo, what do you think?
This would simply be a change in the repo address shown on the themes page. One would be the original and the other link would be the "current stable" version.
Personally I like the idea of the 4-5 best themes, that we would watch more thoroughly for compatibility (to avoid too much work, but providing a working set of templates to the user so he doesn't need to try and fail). Whenever an update is made on these 4-5 themes, we would check that it is still valid and the installation procedure is still working.
However I also like @mayamcdougall 's suggestion with the "Ready-to-go" tag. That is a good idea too, but it would mean a lot more of tests and validation as it could be for all themes the case, and it can be broken at any time by udpates.
Hence I think that both solutions would be good to implement and not of too much hard work.
The hard work is a bit for the compatibility test and installation procedure update, but that's basically all, the rest is quite simple.
Instead of the "Ready-to-go" tag, that will not be valid anymore once there is an update to the theme potentially, or to pico, I would rather suggest to put this kind of tag:
"Last verified: " and put the date of validation (there can be serveral ones, so not always dependent on the theme version or commit). It should always be only with the latest pico version so that it makes it clear. Also this is the configuration that newcomers will always have.
This way the user knows that it has been verified at a certain point. He can also tell us or the repo maintainer that it doesn't seem to be valid anymore if he encounters issues.
What about all that?
To continue this way, I would suggest this: I will try to fix some other themes. Then you could do with Maya a selection of the ones you prefer and that we know works with the installation instructions.
Then there should just be a highlight of those themes with like a green frame or so, and 10% bigger, with a "top and supported" tag on them on the themes page.
That would basically be all there is to do.

Thanks @mayamcdougall I get it. Great work for the stellar theme. I didn't try it but it seems very nice, congratulations for the porting!
I think that an example website is important to have for all themes, else it is difficult for users to go for it without trying.
Maybe I can provide some hosting of that theme, of course out of the box version only, I would have to see.

Well, the reason I didn't do a demo site was that there'd be no difference out-of-the-box to the original theme's demo site.
The changes are all on the backend, leveraging Pico to turn what was once a single static HTML template into a dynamically generated website.
I didn't design the theme, nor do I want to take the credit for its design.  That's why I reference the original demo site (and of course the practical aspect of not having to host something that would be identical to the end user anyway).

Anyway, to clarify a little on the "Ready-to-go tag", what I mean is to have a Category of theme titled "Ready-to-go" that would be at the top of the gallery with  the existing All, General Purpose,  Single Page, Portfolio, and Blog-Style filter buttons.
In addition to this, I may add some kind of badge to the image or description of the theme.  I'm leaning toward putting it in the description because I'd like to save the idea of an image badge (or border) for recommended/top themes (also, putting it in the description wouldn't actually require any changes to the existing site).  Too many items overlaid on the thumbnail would feel cluttered, so I think one overlay at most (on select items only) would be the goal here.
As far as other style changes go... the Gallery itself is a bit difficult to work with.  The gallery is run by it's own, self-contained javascript applet that makes a lot of realtime changes to the page DOM.
If I remember correctly, it was a real pain to style things without it looking broken. 🤔
Things like a green border might be easy enough, but I don't think making items bigger is happening (at least, without the whole gallery looking broken).
And we could technically make a "top row" of themes by simply capitalizing the .md files of the ones we want at the top.  It's case-sensitive alphabetical.  That's how I've got the Default theme at the top right now. 😂
Also, I originally had some reservations about putting any themes "on top" in the past because I didn't want the Gallery to be biased.  This goes especially so for not wanting to provide preferential treatment to my own themes (which do tend to go the extra mile with documentation and try to provide a good user experience).
But, since adding the wiki themes to the site, it has felt a bit more cluttered. 🤔
(@PhrozenByte, I'm wondering if in the future we should consider dropping the current gallery in favor of something paginated.  It's visually impressive (or at least, it used to be 😂), but I'm not sure if it's practical for such a large number of entries.  I'll add this to my research list on top of finding a list-based replacement for plugins.)
I'm going to put in a little work on the gallery tonight and see if I can come up with changes that I like.

 I personally don't like the idea of a "Last Verified" tag.  It seems unnecessary, seems like extra work (something we're trying to cut down on, as we've discussed in our discussion about revising the themes and plugins submission process), and just not of much benefit to the end user.
Pico's implementation has been stable for a long time.  In general, a theme that worked years ago still works today (even if their documentation might be lacking or outdated).  Also, with most of the themes being untouched, there just aren't any changes to "verify" on either side (Pico's or the theme's).
Basically, I see it being something that would just fall to the wayside, and then end up looking bad.  It'll look good to see Last Verified: <Last week's date> on a theme, but pretty soon that'll become Last Verified: <Last year's date>.  And while neither Pico or the Theme has changed (significantly) in that time, without taking the extra effort to make sure nothing has broken, the date would just drift further into the past.
Idk, "Last Verified" doesn't sound like a lot of work at face value... but when you think about how a "Last Verified" date is only useful if it's recent, it starts to become a much bigger maintenance headache.
And again, that's just my opinion on the matter.  And again like before, it's not a bad idea in general, I just don't feel like it fits our circumstance. 😕

Just wanna leave a :heart: here, I totally agree with everything that has been mentioned/suggested/noted here and I feel like that this discussion is going into the right direction! :+1:

(@PhrozenByte, I'm wondering if in the future we should consider dropping the current gallery in favor of something paginated. It's visually impressive (or at least, it used to be joy), but I'm not sure if it's practical for such a large number of entries. I'll add this to my research list on top of finding a list-based replacement for plugins.)

I agree. Feel free to replace it with something else, the website's source code (JS/CSS/HTML) is very clumsy in general. Maybe it's a good time to finally switch to Pico, too?

Maybe it's a good time to finally switch to Pico, too?

🤯*
(Head exploding emoji.  I tend to use Apple devices for everything these days, so I legit have no idea what emoji character support looks like these days on Linux and Android. Let me know if I'm ever just posting a bunch of "missing character boxes" for you. 😂)
So, "Switching the website to Pico" sounds like it needs it's own Issue.  Some things I'd be wondering would be:

Last I knew, GitHub Pages couldn't do anything but Jekyll, and that's why we hadn't switched. Has that changed?
Assuming that hasn't changed, who's going to host it? (I'd be more than willing to, and I'm sure you'd be too.)
And what kind of security issues would that bring? (security Maintenance issues is really what I mean. Managing credentials/keys, etc).
This is probably a given, but continuing to use the picocms.github.io would be preferred.  I'd bet we could easily configure a setup that automatically pulls from it every couple minutes if there's changes. (I mean... I know I could.  Assuming you'd be more than capable of that too. 😉)
Actually, given the right setup that automatically pulls the repo, security credentials would be a non-issue.  Just have whoever's hosting it maintain the server (as they would be anyway), and make all changes through git.  If the server is just doing a pull, no credentials would be needed.
Finally, there's the issue of theming.  Would we want to port the existing theme or go with something new?  I'm leaning toward new that way we'd get a nice refresh to go along with bragging about the website finally running on Pico itself.

If you want to make a new issue to discuss this, I can copy this comment over to it.  Otherwise it can just be some things to keep in the back of our minds for later.

Switching to Pico basically means that we'd use Pico as a static website generator. We'd set up a GitHub Action that installs Pico on the CI server and creates a static copy of the website using wget -r. These static HTML pages will be commited to the picocms.github.io repo then. No hosting required. The CI scripts are no big deal at all, it's rather about switching from Liquid (Jekyll's template engine) to Twig.
About the theme: I don't really have any preference here. I doesn't look bad, I kinda like it, but it definitely needs improvement - both visually (the theme isn't mobile-ready in general, just take the font size) and from a technical point of view (the CSS/JS is very clumsy).
Yes, a new issue would be a good idea.

That sounds a little clunky, lol.
I've never done anything with CI stuff, so I'll take your word for it that it'd be a simple setup. 👍
I assume there'd be like a separate "statically generated" branch or something to hold the output files without contaminating the main site repo? (Kind of like the older days where the site was in the gh-pages branch on the main repo).
I'm torn on the theme as well.  Honestly, I hate making big changes to it because it feels held together in places with "gum and paper clips".  I also don't really like that it's essential a bunch of old leftover bits of a commercial theme.  I'm sure we've got tons of extra cruft in there that isn't actually used by anything anymore (or never was).
Since moving to Pico would require a rewrite either way, I think I'd prefer to just port a new theme.  Something with a similar layout and structure to the existing theme, that way as much of the site as possible could be a 1:1 reimplementation.  Extras like the gallery could always be patched on top of whatever theme we choose (license permitting), or replaced with newer implementations.
With this route, we'd also have a fully open theme in the end that we could put in the theme gallery.  It'd also be a bit of a flex to say "Not only does the site run Pico, here's the theme for your own usage." (Probably with a different default color scheme though, that way anyone else using it doesn't start with a carbon-copy of Pico's site, lol). 😉

Just closed this tab accidentally and lost a whole post. 🤦🏻♀
Oh well, I should get more the the point for once.
Gallery is Isotope, licensed Commercial/GPLv3.  We could use it as long as the new theme was GPLv3, not MIT.
Fancybox (the image viewer) used to be MIT and is now fully commercial.  Not sure what version we use.
I'll probably be replacing these if we do a new theme.
</comment> 😩


Need to re-run GloSC everytime Steam restarts

Everytime that Steam restarts or my system reboots, GloSC no longer launches the specified game. GloSC hooks to the Steam overlay properly but no longer launches the game.
I have to reopen GloSC and click "Add all to Steam" to get it working properly again.
I'm running Windows 10 64-bit version 1709.

Did you use the installer?
Or the zipped version?
GloSC_GameLauncher.exe has to run at system startup (as admin).
Proper way to do that would be via windows task scheduler.
You can also execute this on commandline:
"schtasks.exe /create /f /tn "GloSC_GameLauncher" /tr "[PATH_TO]\GloSC_GameLauncher.exe" /sc onlogon /rl highest"
See: https://github.com/Alia5/GloSC/blob/master/Build_Install.md

The task scheduler command is working. The game now executes with the overlay after a restart. Thank you.