Module compilation error with NGINX 1.21.5

After:
git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git
wget http://nginx.org/download/nginx-1.21.5.tar.gz
tar zxf nginx-1.21.5.tar.gz
cd nginx-1.21.5
./configure --with-compat --add-dynamic-module=../ModSecurity-nginx
make modules
I can see this error:
make -f objs/Makefile modules
make[1]: Entering directory '/root/nginx-1.21.5'
cc -c -fPIC -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g  -I src/core -I src/event -I src/event/modules -I src/os/unix -I /usr/local/modsecurity/include -I objs -I src/http -I src/http/modules
-o objs/addon/src/ngx_http_modsecurity_module.o
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c: In function 'ngx_http_modsecurity_pcre_malloc_init':
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:78:9: error: 'pcre_malloc' undeclared (first use in this function); did you mean 'old_pcre_malloc'?
78 |     if (pcre_malloc != ngx_http_modsec_pcre_malloc) {
|         ^~~~~~~~~~~
|         old_pcre_malloc
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:78:9: note: each undeclared identifier is reported only once for each function it appears in
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:82:25: error: 'pcre_free' undeclared (first use in this function); did you mean 'pkey_free'?
82 |         old_pcre_free = pcre_free;
|                         ^~~~~~~~~
|                         pkey_free
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c: In function 'ngx_http_modsecurity_pcre_malloc_done':
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:102:9: error: 'pcre_malloc' undeclared (first use in this function); did you mean 'old_pcre_malloc'?
102 |         pcre_malloc = old_pcre_malloc;
|         ^~~~~~~~~~~
|         old_pcre_malloc
../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:103:9: error: 'pcre_free' undeclared (first use in this function); did you mean 'pkey_free'?
103 |         pcre_free = old_pcre_free;
|         ^~~~~~~~~
|         pkey_free
make[1]: *** [objs/Makefile:1232: objs/addon/src/ngx_http_modsecurity_module.o] Error 1

Have the same issue, looks like it's related to a change in nginx (now nginx is built with the PCRE2 library by default).

@simonepittis @monitmanllc please check this - https://github.com/SpiderLabs/ModSecurity-nginx/pull/260
Long story short: use --without-pcre2 configure argument when building ModSecurity-nginx connector module. PCRE2 support must be added to the library (libmodsecurity) and then to the connector. Applying just the connector's PR will lead to enormous memory leaks in regex processing.

@defanator Thank you for the info appreciate it, I have used --without-pcre2 and compiled as expected. Have a good afternoon.

yes, works!
@defanator  thanks for the help.

@defanator Thank you for the info appreciate it, I have used --without-pcre2 and compiled as expected. Have a good afternoon.

How is it working after compiling with the --without-pcre2 flag? Are you having any of the reported memory allocation problems? What version of Nginx are you compiling with? Thanks for the help and info.

@defanator Thank you for the info appreciate it, I have used --without-pcre2 and compiled as expected. Have a good afternoon.

Are you getting any problems with memory allocation or leaks after compiling and testing?

@Danrancan tested now without "--without-pcre2" and works fine. no performance issue and no memory issue. :-)

@Danrancan tested now without "--without-pcre2" and works fine. no performance issue and no memory issue. :-)

Thank you very much @simonepittis! This is exactly what I needed to know!

@Danrancan tested now without "--without-pcre2" and works fine. no performance issue and no memory issue. :-)

@simonepittis
Wait, I just re-read this. You say you tested it WITHOUT "--without-pcre2". Was that a mistake? or do you actually mean that you did NOT use the "--without-pcre2" arguement, and it is still working? If so, that implies there was an update that I don't know about correct? Otherwise, i believe you made a grammatacal mistake and meant to say.. "tested now with "--without-pcre2" and works fine." Could you please clarify?


ability to ignore whitespace

It would be great to have the ability to ignore whitespace, in some cases this is usefull as I had two files that were very similiar with a few differences I needed to discover but they were slightly tabbed differently so all lines were red due to tabbing only.

hmmm ... i'll have alook into how this can be done.


configure sam build --watch support

I use aws-sam-webpack-plugin with  webpack to build the files.
webpack.config.js
const AwsSamPlugin = require("aws-sam-webpack-plugin");
const awsSamPlugin = new AwsSamPlugin();
const path = require('path');

module.exports = {
    //entry: './app.js',
    entry: () => awsSamPlugin.entry(),
    output: {
      path: path.resolve("."),
      filename: (chunkData) => awsSamPlugin.filename(chunkData),
      libraryTarget: 'commonjs'
    },
    target: 'node',
    mode: 'production',

      // Add the AWS SAM Webpack plugin
    plugins: [awsSamPlugin]

}

instead of using sam buld I use npm run build
in package.json
....
 "scripts": {
    "test": "mocha tests/unit/",
    "build": "webpack --watch "
  },
.....

although webpack  rebuild the changes, it does not appear when sam local  start-api
is running,
I am looking help for configure to appear changes

@asankagit Are you saying that changes to the functions aren't appearing or that changes to your SAM project aren't appearing?

yes  I have configured my webpack.config.js and package.js as above

@asankagit
Maybe you can achieve this by using two terminals.
## 1st terminal
npm run build
## 2nd terminal
sam local start-api

@asankagit
One solution could be to use something like concurrently like I've used on my boilerplate
The reason for the initial npm run build is to ensure the container doesn't start up with old files or an empty directory.

@asankagit
One solution could be to use something like concurrently like I've used on my boilerplate
The reason for the initial npm run build is to ensure the container doesn't start up with old files or an empty directory.

Webpack watch does seem to work for me. The slowness makes it hard to tell if latest changes are completed or still a in progress. Sometimes I could see a build complete just as I get ready to run, but it was actually a previous build that had not finished but did not have latest changes.
I found it more efficient to manually run the build than keep wondering if I needed to wait longer for the "latest" change.


Include defense player stats in player stats releases

I wanted to make a PR to ffscrapr to help getting the ff_scoringhistory() for IDP.
Since @tanho63 uses the static player stat files it'd be an easy and cool way to have the defense player stats file in https://github.com/nflverse/nflverse-data/releases.
With the aggregate_game_stats_def() it should be possible to store them.
Love you all ❤

Done.


Visual Studio 2013 Extension - v0.1.1433.0

I'm loving this project. I was able to start a new grammar that works fine in the Visualizer application, but I can't seem to get it to recognize my files in VS. The included .njson and .ncs grammars work fine.
I am running Nitra v0.1.1433.0 and have installed the extension in VS2013 update 4. Are there any known issues / workthroughs for editing new grammars in VS?

Look for NitraGlobalConfig.xml file in Nitra installation diretory. You have to add section for your own language
<Language Name="MyLang" FileExtensions=".abc" Grammar="Parser.Class.Name" StartRule="StartRuleName">
      <Module>C:\mylang\My.Lang.dll</Module>
</Language>

Unfortunately there is not support for environment variables in config.
Now we are completely redesign custom languages integration with VS, so this config will be dropped soon.

Sadly, I tried that and it still didn't work.
<Language Name="Perl" FileExtensions=".npl" Grammar="PerlNitraGrammar" StartRule="CompilationUnit">
    <Module>C:\code\githome\PerlParser\PerlNitraGrammar\bin\Debug\PerlNitraGrammar.dll</Module>
</Language>

At present we are working on a new version of VS integration. Wait a bit and we will publish it.


Feature request: Add a close mapping for the bookmarks window

It would be nice to also have a close command for the bookmarks window in addition to the toggle one.

It would be nice to also have a close command for the bookmarks window in addition to the toggle one.

You can use the function require"bookmarks".close_bookmarks() to do your own mapping.


Prettier formatting for ConvertTo-Json output. #2736

This change standardizes JSON output to example given, as well as
codemaid and online lint tools.
Sample object used for testing:
@{
foo = @{
first = 'a'
second = 'bbbbbbbb'
}
barbarbarbar = @{
first = 'a'
second = 'bbbbbbbb'
NestedArray = @(
'Test3'
'Test4'
'Test5'
3
4
)
NestedObject = @{
MoreObject = 'AnotherObject'
TestBool = $true
}
}
array = @(
'Thing1'
'Thing2'
)
dan = 15
} | ConvertTo-Json

Hi @kittholland, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

We should have a test for pretty print.

I will work on the pretty print test soon.

@daxian-dbw the CI breaks are related to https://github.com/PowerShell/PowerShell/pull/2806, how to proceed?

@SteveL-MSFT I just restarted the Travis CI build.

Thanks for rerunning Travis-CI, I see my tests are failing in that environment. I will have to build out a non windows environment for more testing.
There are also some here-string based tests that are labeled -pending$isCoreCLR and are tagged in a Feature describe block. I think I will need to rework any here-string based tests to accomodate the new formatting.

Closing and reopening to restart CI

Hi @kittholland, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

Can we retry the travis CI? I have successful test results on ubuntu 14.04.

Closing and reopening to restart CI

Hi @kittholland, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

Due to differences in newline characters between platforms, I think doing comparisons on here-strings will have to be very carefully handled if it is possible.
I receive a different string hashcode on the output of ConvertTo-Json on linux vs. windows for example.

@SteveL-MSFT and @JamesWTruher This looks like a forgotten PR that is a candidate for merging. Can you please take a second look at it?


Can's set the priority in the official Android App

Hi,
It would be nice if I can set also the priority, because I use several remotes. One of them has the priority 1. So I can't overwrite that.
Cheers #

https://github.com/b2un0/kodi-script-hyperion/issues/1

brindosch: He is talking about Android, not Kodi ;-)

I am talking too :)

I think that is answered :)


Water broken in Babel 2022oct13

As far as I can tell, this is because there are now two PubChem entries for water:

https://pubchem.ncbi.nlm.nih.gov/compound/962 (type biolink:SmallMolecule)
https://pubchem.ncbi.nlm.nih.gov/compound/22247451 (type: biolink:MolecularMixture)

Since the two PubChems are of different types, this triggers this piece of logic from chemical.py:
https://github.com/TranslatorSRI/Babel/blob/cc3db38233fbd5ea6a124d65dc91306941ac5493/src/createcompendia/chemicals.py#L529-L538
And water is changed from a biolink:SmallMolecule (in Babel 2022sep13) to biolink:ChemicalEntity (in Babel 2022oct13).
This should give us UNII:059QF0KO0R as the preferred ID; however, for some reason, Babel 2022oct13 gives us HMDB:HMDB0002111, even though HMDB isn't even on the list of prefixes for ChemicalEntity. So there's probably a second bug lurking in here somewhere.

Chris thinks that ChemicalEntity prefixes are hardcoded in Babel somewhere.

It turns out that there are 8,178 examples of this happening in Babel 2022oct13 of between 2 and 19 identifiers, all of which are of type {'biolink:SmallMolecule', 'biolink:MolecularMixture'}. (Here is a badly formatted file with all the identifiers). All but 9 of these have an InChIKey, and so appear to be grouped by an InChIKey collision.
Possible solutions:

Some of these collisions (but not all) could be resolved by removing InChIKey IDs.
We could force mixtures that evaluate as {'biolink:SmallMolecule', 'biolink:MolecularMixture'} to biolink:MolecularMixture, but that would include water.
We could tweak the custom prefixes for chemicals so that water goes back from HMDB:HMDB0002111 to PUBCHEM.COMPOUND:962 (see node.py code snippet below)

https://github.com/TranslatorSRI/Babel/blob/075b32cf8e323ee0c9007eaf715a4741ae382dd5/src/node.py#L85-L109
Some example identifiers
The nine cliques without InChIKeys are:

Nickel boride (PUBCHEM.COMPOUND:82791 and Nickel boride (Ni3B) PUBCHEM.COMPOUND:6336855), InChIKey collision:

{'UMLS:C0628420', 'CAS:12007-02-2', 'CAS:12619-90-8', 'CAS:12007-00-0', 'PUBCHEM.COMPOUND:82791', 'MESH:C049893', 'PUBCHEM.COMPOUND:6336855'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:166138 and PUBCHEM.COMPOUND:166137, no InChIKey collision:

frozenset({'PUBCHEM.COMPOUND:166138', 'PUBCHEM.COMPOUND:166137', 'MESH:C407145', 'CAS:70236-51-0', 'UMLS:C0913820'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:9572776 and PUBCHEM.COMPOUND:54609060, no InChIKey collision:

frozenset({'CAS:5401-69-4', 'PUBCHEM.COMPOUND:9572776', 'PUBCHEM.COMPOUND:54609060'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:12272670 and PUBCHEM.COMPOUND:9573653, no InChIKey collision:

frozenset({'MESH:C006076', 'CAS:50471-02-8', 'CAS:52237-22-6', 'PUBCHEM.COMPOUND:12272670', 'PUBCHEM.COMPOUND:9573653', 'UMLS:C0070490'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:163409081 and PUBCHEM.COMPOUND:163358784, no InChIKey collision:

frozenset({'PUBCHEM.COMPOUND:163409081', 'UNII:83BU3492RP', 'PUBCHEM.COMPOUND:163358784', 'CAS:2647442-13-3', 'MESH:C000722705', 'PUBCHEM.COMPOUND:162625114'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:16132438 and PUBCHEM.COMPOUND:118984450, no InChIKey collision:

frozenset({'UMLS:C0293359', 'CHEMBL.COMPOUND:CHEMBL1201538', 'MESH:D061268', 'DrugCentral:5002', 'PUBCHEM.COMPOUND:16132438', 'UMLS:C5389884', 'DRUGBANK:DB00046', 'PUBCHEM.COMPOUND:118984450', 'UMLS:C0982221', 'CAS:133107-64-9'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:156597127 and PUBCHEM.COMPOUND:75972517, no InChIKey collision:

frozenset({'PUBCHEM.COMPOUND:156597127', 'PUBCHEM.COMPOUND:75972517', 'CAS:1323151-45-6'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:16132344 and PUBCHEM.COMPOUND:121494108, no InChIKey collision:

frozenset({'CAS:79804-71-0', 'PUBCHEM.COMPOUND:16132344', 'CAS:121249-14-7', 'PUBCHEM.COMPOUND:121494108', 'MESH:C085671'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

PUBCHEM.COMPOUND:16235945 and PUBCHEM.COMPOUND:10313238, no InChIKey collision:

frozenset({'CAS:17417-06-0', 'PUBCHEM.COMPOUND:16235945', 'PUBCHEM.COMPOUND:10313238'}) 	(2)	{'biolink:SmallMolecule', 'biolink:MolecularMixture'}

The two cliques with 19 identifiers that have this problem are Pralidoxime/Pralidoxime chloride/Pralidoxime cation, (Z)-/Pralidoxime methyl sulfate (PUBCHEM.COMPOUND:4884, PUBCHEM.COMPOUND:135398747, PUBCHEM.COMPOUND:13548464 and PUBCHEM.COMPOUND:135573307):

{'KEGG.COMPOUND:C07400', 'INCHIKEY:JBKPUQTUERUYQE-UHFFFAOYSA-O', 'CAS:45765-69-3', 'CHEMBL.COMPOUND:CHEMBL1420', 'UNII:P7MU9UTP52', 'GTOPDB:7597', 'HMDB:HMDB0014871', 'PUBCHEM.COMPOUND:135398747', 'MESH:C028797', 'CHEBI:8354', 'PUBCHEM.COMPOUND:135484646', 'DrugCentral:2231', 'UMLS:C0071808', 'DRUGBANK:DB00733', 'PUBCHEM.COMPOUND:4884', 'GTOPDB:4652', 'CAS:6735-59-7', 'PUBCHEM.COMPOUND:135573307', 'UNII:YV274KB8PY', 'CAS:25615-00-3'}

and Penicillin G potassium salt/Penicillin G potassium/Penicillin G (PUBCHEM.COMPOUND:102090456, PUBCHEM.COMPOUND:23664709 and PUBCHEM.COMPOUND:5904):

{'PUBCHEM.COMPOUND:102090456', 'INCHIKEY:JGSARLDLIJGVTE-MBNYWOFBSA-N', 'UMLS:C0030842', 'UMLS:C0030827', 'PUBCHEM.COMPOUND:23664709', 'CHEBI:18208', 'CHEBI:17334', 'DrugCentral:2082', 'CHEMBL.COMPOUND:CHEMBL29', 'KEGG.COMPOUND:C00395', 'UNII:Q42T66VG0C', 'GTOPDB:4796', 'KEGG.COMPOUND:C05551', 'DRUGBANK:DB01053', 'PUBCHEM.COMPOUND:5904', 'HMDB:HMDB0015186', 'MESH:D010406', 'MESH:D010400', 'CAS:61-33-6', 'CAS:113-98-4'}

This appears to fundamentally be a problem with InChiKey, which simply fails to recognize the difference between different compounds sometimes.
But Inchikeys are the basis of a big part of our concordance strategy, so I'm not exited about getting rid of them overall.   I'm thinking something along line lines of when inchikeys evaluate to {small molecule, molecular mixture} that we remove the inchi keys in that case and just merge by canonical smiles?  but I'm not sure we have smiles from all the sources...

Result of discussion on Nov 9, 2022:

Water should be a small molecule! But that's not an easy fix, and raises issues with how we handle conflation down the line.
[ ] Would like to get a new Babel -- maybe for a stopgap we modify the chemical entity prefixes so we get them in the order that everything else is in.

@cbizon I'm guessing that InChIKeys are important because people query them as identifiers? If not, I think we could solve some of these problems by replacing InChIKeys with InChIs -- we'd have more identifiers, but hopefully then water (small molecule) and water (molecular mixture) would be separated.

Are the Inchi's different for water and hydron?   My understanding is is that inchikey is basically a hash of the inchis, so it's possible that two different inchis hash to the same key, but I'd be a little surprised.
The Inchi's (and by extension keys) are meant to be strings that map to a structure.  The InChi (not the key) is a representation of the structure, so if you have the InChi you can draw the structure.  That makes it at least a de-facto identifier, since it's a string and it's 1:1 with the entity (the structure).
The InChiKey specifically is important for us because it is the key that is being used by EBI's UniChem to merge chemical cliques, and that is the nucleus of our approach in Babel.

Are the Inchi's different for water and hydron? My understanding is is that inchikey is basically a hash of the inchis, so it's possible that two different inchis hash to the same key, but I'd be a little surprised.

You are correct -- water and hydron both have the same InChIKey and the same InChI! I was confusing that term with canonical SMILES, which is where they differ (O and [H+].[OH-] respectively). So what I really meant to say was "what if we do all our chemical conflating with canonical SMILES?" But as long as we have to use InChiKeys anywhere, the water and hydron clique will be recombined, so I don't think that buys us anything.
One other approach we considered during last week's call was to separate just the PubChem entry with the MolecularMixture as a separate thing, so we would have one SmallMolecule clique consisting of water+all the other identifiers (including any hydron identifiers from other sources) and one MolecularMixture clique consisting of just PUBCHEM hydron. We could then refine this over time by looking for other identifiers that we can recognize as molecular mixtures (whether by checking if their canonical SMILES has a '.' or looking something up in their record) and moving those into MolecularMixture. Cliques that have multiple PubChem types but not SmallMolecule+MolecularMixture specifically will revert to ChemicalEntity as they do currently.
Do you think that approach might be a good short-to-medium term fix for this issue?

I think it's a decent pragmatic approach, recognizing that it's not 100% what we want.    It's worth the tradeoff in my opinion.

I think it's a decent pragmatic approach, recognizing that it's not 100% what we want. It's worth the tradeoff in my opinion.

I've implemented this in Babel now (PR #91 in progress). In most cases only a single PubChem compound is separated out as a MolecularMixture, but sometimes there are more than one, such as in the following example:

WARNING:root:Found a clique that that contains PUBCHEM types ({'biolink:SmallMolecule', 'biolink:MolecularMixture'}). This clique will be split into a biolink:MolecularMixture ({'PUBCHEM.COMPOUND:11149677', 'PUBCHEM.COMPOUND:22836554', 'PUBCHEM.COMPOUND:4066245', 'PUBCHEM.COMPOUND:5373419'}) and a biolink:SmallMolecule ({'CAS:21679-31-2', 'UMLS:C0244965', 'CAS:17272-66-1', 'MESH:C049529', 'INCHIKEY:POILWHVDKZOXJZ-ARJAWSKDSA-M', 'PUBCHEM.COMPOUND:5460482'})

I will see how quickly I can get this Babel output up on an experimental instance and then we can play around with it some more.

Nice!

@cbizon Now up at https://nodenormalization-dev.apps.renci.org/docs and water is fixed again! This doesn't include the fix to #68, so the identifier prefixes are a little random for ChemicalEntity, but SmallMolecule and MolecularMixture both seem to be working well.

Fixed in PR #91.


List files in zip only from currentFolder folder

Currently listFiles() returns array of all files within zip. It would be handy to have method that takes currentFolder into consideration.
I tried to use $files = $zipper->folder('sql')->listFiles();  but realized after looking into code that it is not possible.

I could make PR for it but if it is a correct thing to do. It caused side effects for existing code.
as there could be code with
...
$zipper->folder('sql');
...
doSomethingInteresting();
...
$allFilesInZip = $zipper->listFiles();
doSomethingWithAllFiles();
....

resolved with #82 now I can do same with $zipper->listFiles('/^sql\//i);


Fix paths double encoding

QUrl::setPath used by default 'DecodedMode' of the input path
http://doc.qt.io/qt-5/qurl.html#setPath
unlike setQuery setFragment setUserInfo
When the path already contains the URL-encoded characters they will be encoded again

Thank you.


Provide specialized version of the default config file

Each community of users may want to have different default config file, especially regarding interaction style, up direction, coloring.
It would be nice to provide default config file for each of the basic usage of F3D, eg:

3D Printing
Gamedev
Animation
Sciviz

Ideally, for the NSIS windows installer, a dedicated screen would let the user choose the default config to install.

Will be possible once #399 is merged

So this is no trivial to implement. Should we do it or is it a bad idea @Meakk ?

to put in context of #374

I think it's a good idea!

added medical to the list

I'm still unsure this is a very good idea as config files are proving already challenging to use for most users. moving back anyway

@Meakk I'm thinking of closing this one. What do you think ?

Closing this without feebdbacks


Add new index.html file

Add index.html file

can't merge the add-index branch to master without an approved review


[ExtraMetadataLoader] Logo drop shadow options

Extension
ExtraMetadataLoader
Describe the solution you'd like
Allow the drop shadow applied to the logos to be customized. Including color, radius, direction, offset and enabling/disabling it.

Same as in https://github.com/darklinkpower/PlayniteExtensionsCollection/issues/60, partially done in a commit I can't upload. I added options for blur radius, shadow depth, shadow direction and opacity. I'll take a look later for how to add a color picker.

Partially done in https://github.com/darklinkpower/PlayniteExtensionsCollection/commit/6150e8854ff54b76462247d03c2fcf933ab975b6.


Not responding trying to close an connected ssh tag

Environment
Windows build number: [版本 10.0.18362.239]
Windows Terminal version (if applicable):Windows Terminal (Preview) Version: 0.2.1831.0 (Microsoft Store build)

Any other software?
OpenSSH client: OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

Steps to reproduce

1.Start an ssh session in a tab(current or new would do) with RSA certificate identification.
2.Close the tab while the ssh is still connected.
Expected behavior

The tab should be closed.
Actual behavior

The Terminal stops responding and then the typical Windows messagebox for

unresponding apps will show.

This looks like a dupe of #1445 to me

This looks like a dupe of #1445 to me

Well, the behaviors aren't identical.
The problem you mentioned concerns closing the old tab, but in this case it's closing the current tab.
And a subtle feature in the case is that a ssh session logged in with password can be correctly closed, but a ssh session with RSA certificate will cause the problem.
Maybe they share the same defected code actually, but I'm not totally inside of this.
Anyway thanks and hope you go smooth debugging！


Fix hang for complex output

This may be related to #68
It has been found that some of the regex are not optimised, and in the case of large outputs (like a print statement) will result in an exponential usage of CPU and memory.

@danyf90 @fortmarek Hi 👋 We've also hit this problem while using xcbeautify with Mint, and I'm glad it's already fixed!
I'd appreciate if you publish a new version for it when you find a time. Thanks!

hey @akkyie 👋
Released as part of the 0.13.0 version 🚀

@fortmarek Thanks a lot! 🥳


treewide: Fix appimageTools breakages, use pname instead of name

Rebased version of #273719.
Includes suggested changed from eclairevoyant review.
Includes additional fixes for new packages since #273719.
Closes #273719
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

the fixes has been merged back into https://github.com/NixOS/nixpkgs/pull/271071


Support Implied Profiles in RepositoryValidatingRuleBuilder

The Core libraries now support the concept of "implied profiles", which indicates that a resource should undergo validation against this "implied profile" whether it is present in the incoming resource's profile section or not. This request is to add the capability to our RepositoryValidatingRuleBuilder to provide the ability to make use of this new feature in the core libraries.
A potential API could look like:
 	ruleBuilder
				.forResourcesOfType("Patient")
                                .impliedProfile("http://www.hl7.org/fhir/us/core/StructureDefinition-us-core-patient.html")
				.requireAtLeastProfile("http://www.hl7.org/fhir/us/core/StructureDefinition-us-core-patient.html");

Which would cause any incoming Patients to be validated against US Core, whether or not they indicate that they support the profile in their meta.profile section.

It seems we could use the rulebuilder to populate the Profile Mapping Extension accordingly, which signals validation expectations to the client.

Chatting with some people, it seems it would also be useful to have the capability statement introspect these rules so that we can populate the resources' profile and supportedProfile sections, as seen here: https://hl7.org.au/fhir/core/0.4.0-preview/declaring-conformance.html

@tadgh just following up to see if there has been any progress on this issue (i.e. a PR) and if it is likely to be included in the next Nov release?

This has not yet been picked up for development. As it stands it is unlikely to make it into the November release.


Deleting vertex

Hi guys,
I'm currently struggling with trying to delete a vertex from my graph using gremlinrestclient. I tried executing queries as usual bot have no luck finding the right syntax. I tried g.V(123).remove(), g.V(123).delete() and various other notations but wasn't able to successfully delete a vertex from my graph.
Do you have any ideas on how to solve this problem?

From the traversal source g, you can use the drop() step to use elements and properties (more info in the docs) [1]. Try this:
>>> from gremlinrestclient import GremlinRestClient
>>> c = GremlinRestClient()
>>> c.execute("graph.addVertex('name', 'tspecht')")
Response(status_code=200, data=[{u'properties': {u'name': [{u'id': u'sw-39c-2dh', u'value': u'tspecht'}]}, u'type': u'vertex', u'id': 4224, u'label': u'vertex'}], message=u'', metadata={})

>>> c.execute("g.V(4224).drop()")
Response(status_code=200, data=[], message=u'', metadata={})

>>> c.execute("g.V()")
Response(status_code=200, data=[], message=u'', metadata={})

Does this solve your issue?

http://tinkerpop.apache.org/docs/3.1.0-incubating/#drop-step

thanks for the explanation @davebshow I just now realise that this question was rather a gremlin question and had nothing to do with gremlinrestclient, so thanks a lot for answering it anyways! :)


Fixes #2405 0.13 migration failure

Serialized MarathonTask entities are now deserialized using the old format.

This is a breaking change. Can you please add a note to the changelog? Thanks!

Tested that with tasks created with 0.11. Migration worked. :+1:
If you add the changelog entry, I will merge this.

Thanks for the review – I added a change log entry, please review that one as well.

Thanks!


Convert major/minor id from stat command from hex to decimal

Updates the partition device checking to use decimal values for device major/minor id.  Also checks the device type to account for the fact that some block and character devices share device ids.

Quick test using a VM installed with virt-install --name igntest --vcpus 2 --memory 8192 --disk path=/var/lib/libvirt/images/ign.qcow2 --os-variant ol8.10 --boot uefi.  This VM uses /dev/vda as the root device, whcih has a major device id of 251, which also happens to conflict with the real time clock.
[root@ocne16929 ~]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    251:0    0   15G  0 disk
|-vda1 251:1    0  244M  0 part /boot/efi
|-vda2 251:2    0  488M  0 part /boot
`-vda3 251:3    0 14.3G  0 part /sysroot

[root@ocne16929 ~]# ls -l /dev | grep 251
crw-------. 1 root     root    251,     0 Nov 14 19:31 rtc0
brw-rw----. 1 root     disk    251,     0 Nov 14 19:31 vda
brw-rw----. 1 root     disk    251,     1 Nov 14 19:31 vda1
brw-rw----. 1 root     disk    251,     2 Nov 14 19:31 vda2
brw-rw----. 1 root     disk    251,     3 Nov 14 19:31 vda3

[root@ocne16929 ~]# journalctl -u grow-rootfs.service | cat
...
Nov 14 19:31:20 localhost.localdomain grow-rootfs[956]: ++ stat -L -c %F /dev/rtc0
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + [[ character special file != \b\l\o\c\k\ \s\p\e\c\i\a\l\ \f\i\l\e ]]
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + continue
...
Nov 14 19:31:20 localhost.localdomain grow-rootfs[1044]: ++ stat -L -c %F /dev/vda
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + [[ block special file != \b\l\o\c\k\ \s\p\e\c\i\a\l\ \f\i\l\e ]]
Nov 14 19:31:20 localhost.localdomain grow-rootfs[1045]: ++ stat -L -c %t /dev/vda
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + MAJOR_ID=fb
Nov 14 19:31:20 localhost.localdomain grow-rootfs[1046]: ++ stat -L -c %T /dev/vda
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + MINOR_ID=0
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + MAJOR_ID=251
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + MINOR_ID=0
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + '[' 251:0 = 251:0 ']'
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + DISK=/dev/vda
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + break
Nov 14 19:31:20 localhost.localdomain grow-rootfs[1047]: ++ realpath ./dev/disk/by-label/root
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + ROOT=/dev/vda3
Nov 14 19:31:20 localhost.localdomain grow-rootfs[878]: + echo '/dev/vda 3 /dev/vda3'
Nov 14 19:31:20 localhost.localdomain grow-rootfs[877]: + DEVICES='/dev/vda 3 /dev/vda3'
Nov 14 19:31:20 localhost.localdomain grow-rootfs[877]: + read -r -a ROOT_DEVICES
Nov 14 19:31:20 localhost.localdomain grow-rootfs[877]: + DISK=/dev/vda
Nov 14 19:31:20 localhost.localdomain grow-rootfs[877]: + PARTITION=3
Nov 14 19:31:20 localhost.localdomain grow-rootfs[877]: + ROOT=/dev/vda3
Nov 14 19:31:20 localhost.localdomain grow-rootfs[877]: + mount /dev/vda3 /sysroot
Nov 14 19:31:21 localhost.localdomain grow-rootfs[877]: + growpart /dev/vda 3
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1061]: NOCHANGE: partition 3 could only be grown by 2015 [fudge=2048]
Nov 14 19:31:21 localhost.localdomain grow-rootfs[877]: + true
Nov 14 19:31:21 localhost.localdomain grow-rootfs[877]: + xfs_growfs /dev/vda3
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]: meta-data=/dev/vda3              isize=512    agcount=4, agsize=935936 blks
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]:          =                       sectsz=512   attr=2, projid32bit=1
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]:          =                       crc=1        finobt=1, sparse=1, rmapbt=0
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]:          =                       reflink=1    bigtime=1 inobtcount=1
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]: data     =                       bsize=4096   blocks=3743744, imaxpct=25
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]:          =                       sunit=0      swidth=0 blks
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]: naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]: log      =internal log           bsize=4096   blocks=16384, version=2
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]:          =                       sectsz=512   sunit=0 blks, lazy-count=1
Nov 14 19:31:21 localhost.localdomain grow-rootfs[1116]: realtime =none                   extsz=4096   blocks=0, rtextents=0
...


A cascade of constraints?

One way out of the problem of over-constraining the openWEMI vocabulary would be to create a cascade of related (probably sub-classed) vocabularies where the properties have additional constraints. Something like:
Least constrained
Five classes and 3 properties (or 6 with inverses) with good definitions but no domains or ranges defined. (Can WEMI still be sub-classed to Endeavor?). Being somewhat informal, this could use the schema.org domainIncludes and rangeIncludes which are helpful but not standard.
More constrained
Properties are given domains, but are defined as owl annotation properties so they can be used with strings. Either the property names or the vocabulary name must be different (as in dc1.1 and dcterms). This solution does not embody the structure that we have defined.
Most constrained
Properties are given domains and ranges. All properties are therefore owl object properties.
Note that if multiple levels of openWEMI are defined they could be mixed-n-matched in instance data.
This intersects with #43 and if one of these solutions is chosen, then we have to decide if adding specific properties makes sense within that solution. It is also possible that additional properties could be added along with additional constraints. I haven't thought that far.

My vote would be for the "most constrained", but dropping the OWL semantics (owl:ObjectProperty could be replaced with rdf:Property, and owl:unionOf could be replaced with rdfs:range openwemi:Endeavour). This eliminates some of the logical overhead of OWL, and is closer to the approach of high-level vocabularies like DC [1].
I think setting up the range as openwemi:Endeavour would provide the most flexibility (equivalent to Work or Expression or Manifestation or Item), and a resource could still be inferred to be of a more specific type through the domain (if A openwemi:expresses B, then we can infer that A is an openwemi:Expression, with the same being true for the inverse). This also supports a resource belonging to more than one class (type), i.e. both a openwemi:Work and openwemi:Expression (see El Diablo from the comics example).
Additional constraints or more distinct semantics, additional properties (as mentioned in #43) with specific domains/ranges (e.g. openwemi:instantiatesWork), etc. could be added in an extended openWEMI vocabulary (e.g. openWEMI-XL). This extension might also be a good place to define other relationships (isPart/hasPart, etc.) if necessary (see #18).
I think annotation properties are more for metadata describing the vocabulary components, like rdfs:comment, rdfs:label, etc.
[1] https://www.dublincore.org/specifications/dublin-core/dcmi-terms/dublin_core_terms.ttl

June 14 meeting decided to define domains and ranges for the properties.


feat: Add dark mode #2

Completed the feature Dark mode, issue - #2

@Rahul-Palamarthi can you share a pic ?

@divyanshudhruv yes I will share the pic

Ok please

@Rahul-Palamarthi can you share a pic ?

https://github.com/divyanshudhruv/Minifolio/assets/87606332/81d5bb30-d856-4a90-a955-e2a7502707da

Cool

Will merge after 6 oct

Is the button responsive?

Is the button responsive?

The size of the button remains same (fine for mobile view also)

Idk why but the terminal typing text is not visible in dark mode

I can fix it

If you deploy your forked repo, it would  be very easy for me to view changes. So please deploy the changed branch

Idk why but the terminal typing text is not visible in dark mode

I have fixed it and also made a PR
https://github.com/divyanshudhruv/Minifolio/assets/87606332/86bc50de-fd9c-4325-bf1a-405b14825ed8

If you deploy your forked repo, it would be very easy for me to view changes. So please deploy the changed branch

@divyanshudhruv  should I deploy to gh-pages ?

Yes yes, then share me the link, I will see then merge

I think you should remove Your Name from the header.

I think you should remove Your Name from the header.

i mean there should be a placeholder so that the header styles will work properly (since using flex inside the header so that the dark-mode button will be at the end of the header)

Yes yes, then share me the link, I will see then merge

yes I will share the link

Yes yes, then share me the link, I will see then merge

yes I will share the link

Ok, fast

Yes yes, then share me the link, I will see then merge

yes I will share the link

Ok, fast

@divyanshudhruv, Here is the link to the gh-pages from my forked repo

Hello, please resolve some conflicts in the files

Ok, btw can you publish the forked repo in gh page ? So that I can see the code? Also ping me and share me the link

Can you remove the border from the button and set the background as #f0f0f0 ? And also increase the size some.

Yes

Ok please do

Ok please do

@divyanshudhruv, I have update the code and made a PR, You can see the changes in the gh-pages of my forked repo

Ok, can I do some minor fixes in the code ? Then I will merge it

Ok, can I do some minor fixes in the code ? Then I will merge it

@divyanshudhruv I will be working on it 🍁

Ok, can I do some minor fixes in the code ? Then I will merge it

@divyanshudhruv I have completed the merge conflicts 🚀

Ok so see the current portfolio and there is a back to top button. Just make the dark button as same like that. I mean the width,height the background color, the font color. And the hover style you can do on your own.

Ok so see the current portfolio and there is a back to top button. Just make the dark button as same like that. I mean the width,height the background color, the font color. And the hover style you can do on your own.

sure

@Rahul-Palamarthi please resolve the conflicts

@divyanshudhruv sure

Added the hacktoberfest-accepted label, will merge soon


Webpack 4 hot rebuild time twice as slow

Originally posted this issue on webpack-dev-server's repo, but was told nearly all of HMR is handled by Webpack itself, so I am moving that issue over here.

Operating System: Mac OS High Sierra
Node Version: 8.10.0
NPM Version: 5.6.0
webpack Version: 4.1.1
webpack-dev-server Version: 3.1.1

[x] This is a bug
[ ] This is a modification request

Code
Note that the webpack file I posted isn't exactly what we have, but anything removed was just company info, me merging a common webpack file with our dev one to produce a single code snippet, and removing some unrelated/redundant info (e.g. multiple templates with similar content).
  // webpack.config.js
const webpack = require('webpack');
const merge = require('webpack-merge');
const config = require('./webpack.common.config.js');
const Jarvis = require('webpack-jarvis');

module.exports = merge(config, {
  mode: 'development',
  devtool: 'cheap-module-eval-source-map',
  output: {
    publicPath: '/',
    filename: '[name]-[hash].js',
    crossOriginLoading: 'anonymous',
    chunkFilename: '[name]-[hash].js'
  },
  optimization: {
    noEmitOnErrors: true,
    namedModules: true,
  },
  plugins: [
    new CopyWebpackPlugin([{
      from: 'app/assets/images/favicon/',
      to: 'favicon/',
    }]),
    new CopyWebpackPlugin([{
      from: 'app/assets/images/square/logo.png',
      to: 'logo.png',
    }]),
    new ExtractTextPlugin({
      filename: '[name]-styles-[contenthash].css',
      allChunks: true,
    }),
    new DeterministicHash(),
    new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/),
    new HtmlWebpackPlugin({
      template: 'templates/index.ejs',
      filename: 'index.html',
      inject: false,
      chunks: ['common', 'main'],
      appleTouchIconSizes: [57, 72, 114, 120, 144, 152],
    }),
    new Jarvis({port: 7003}),
    new webpack.HotModuleReplacementPlugin(),
    new webpack.DefinePlugin({
      _DEVELOPMENT_: true,
    })
  ],
  module: {
    rules: [
      {
        test: /\.(otf|ttf|woff|woff2|png|jpg|svg|mp3|wav|aff|dic)$/,
        use: [
          {
            loader: 'file-loader',
            options: {
              name: 'assets/[name]-[sha1:hash:hex:8].[ext]'
            },
          },
        ]
      },
      {
        test: /\.s?css$/,
        use: ExtractTextPlugin.extract({
          fallback: 'style-loader',
          use: [
            'css-loader',
            {
              loader: 'postcss-loader',
              options: {
                plugins: () => [
                  require('precss'),
                  require('autoprefixer'),
                ]
              }
            },
            'sass-loader'
          ]
        }),
      },
      {
        test: /\.jsx?$/,
        use: ['babel-loader'],
      },
    ]
  },
  entry: {
    main: [
      'babel-polyfill',
      'react-hot-loader/patch',
      'webpack/hot/only-dev-server',
      'webpack-dev-server/client?https://<IP_ADDRESS>:7001',
      './app/main.js',
    ],
  }
});

  // dev-server.js
const path = require('path');
const fs = require('fs');
const webpack = require('webpack');
const WebpackDevServer = require('webpack-dev-server');
const config = require('./webpack.devconfig');
const ssl = require('./devssl');

new WebpackDevServer(webpack(config), {
  publicPath: config.output.publicPath,
  headers: {'Access-Control-Allow-Origin': '*'},
  hot: true,
  https: true,
  clientLogLevel: 'error',
  cert: ssl.cert,
  overlay: true,
  key: ssl.key,
  historyApiFallback: true,
  disableHostCheck: true,
  watchOptions: {
    ignored: /\/node_modules\/.*/,
  },
  stats: {
    assets: false,
    cached: false,
    cachedAssets: false,
    children: false,
    chunks: false,
    chunkModules: false,
    chunkOrigins: false,
    colors: true,
    depth: false,
    entrypoints: true,
    excludeAssets: /app\/assets/,
    hash: false,
    maxModules: 15,
    modules: false,
    performance: true,
    reasons: false,
    source: false,
    timings: true,
    version: false,
    warnings: true,
  },
}).listen(7001, '<IP_ADDRESS>', function(err, result) {
  console.log(`Serving chunks at path ${config.output.publicPath}`);
});

Expected Behavior
Similar, if not faster HMR
Actual Behavior
Hot reload time increased from ~5s to ~10s when upgrading from Webpack 3 to Webpack 4.
For Bugs; How can we reproduce the behavior?
Unsure if it reproduces in every case, but in our case we have a large webapp that spits out multiple entry points, with sass processing, along with a lot of other webpack features.

Hot reload time increased from ~5s to ~10s when upgrading from Webpack 3 to Webpack 4.

Have you tried your Webpack 4 changes in an older version of node, say 8.9.4? For us it makes it fast again.

cc @bmeurer node 8.10 includes a V8 6.2 upgrade.

Interesting. Any idea what it could be related to? 6.2 is a pretty massive update (performance-wise).
cc @hashseed @psmarshall @mathiasbynens

Would you be willing to bisect the V8 change that caused this regression? I have some instructions here.

For reference, I did some benchmarks comparing various nodes with various webpack versions.  I don't claim to be an expert, I just thought it may be helpful to have more information.
For my test, I just added and removed the string 'test' to a React component 10 times, saving between each addition/removal. These numbers should be taken with a grain of salt since when ugprading from Webpack 3 to Webpack 4, we had to change our process slightly, however the overarching trend seems clear. Node 8.10.0 definitely seems to be impacting the rebuild time, though Webpack 3->4 seems to be a larger cause of the increase.
Webpack 3 - Node 8.9.4
Build - 60602ms
Rebuild - 9564ms 4041ms 4401ms 4062ms 4196ms 4902ms 3922ms 3507ms 4388ms 4217ms
Rebuild Average - 4720ms

Webpack 3 - Node 8.10.0
Build - 58165ms
Rebuild - 9493ms 5112ms 5314ms 4965ms 4754ms 5050ms 5165ms 5059ms 5094ms 5096ms
Rebuild Average - 5510.2ms

Webpack 4 - Node 8.9.4
Build - 52564ms
Rebuild - 8294ms 8235ms 5260ms 5094ms 7926ms 8109ms 7979ms 8086ms 7721ms 7927ms
Rebuild Average - 7463.1ms

Webpack 4 - Node 8.10.0
Build - 47680ms
Rebuild - 9977ms 8344ms 8518ms 8205ms 8648ms 8291ms 8320ms 8320ms 7580ms 8227ms
Rebuild Average - 8443ms

@Sojborg let try adding this line to the config:
"mode": process.env.NODE_ENV === "production" ? "production" : "development",

WARNING in configuration

The 'mode' option has not been set, webpack will fallback to 'production' for this value. Set 'mode' option to 'development' or 'production' to enable defaults for each environment.

You can also set it to 'none' to disable any default behavior. Learn more: https://webpack.js.org/concepts/mode/

Shouldn't you use mini-css-extract-plugin instead of ExtractTextPlugin? 🤔

@drewjenkins Can you create minimum reproducible test repo with same config for webpack@3 and webpack@4 when you have increase hmr time?

/cc @Sojborg @drewjenkins please read comment above

I switched off of ExtractTextPlugin to mini-css-extract-plugin with no noticeable performance gain.  @evilebottnawi I'm not sure if I would be able to reproduce the increase with a minimal test repo since I'm assuming the issue appears worse when you have a repo with a lot of code in it like our main UI repo (hundreds of thousands of lines/styles/etc).

@p3x-robot I'm assuming @birkir is using https://github.com/webpack-contrib/thread-loader - which it looks like poolTimeout is the option you'd set to infinity

Same issues with our project. What helped:

follow recommendations from Speeding Up Webpack, Typescript Incremental Builds by 7x

swap thread-loader with happypack, latest beta version supports webpack v4
integrate hard-source-webpack-plugin, subsequent builds are 2 faster

There is a bug in hard-source plugin when using mini-css-extract-plugin I've filed that keeps from adopting on my project https://github.com/mzgoddard/hard-source-webpack-plugin/issues/301

I'm having a similar experience. Just went through an upgrade from 3 to 4, and went from a custom config to an ejected CRA config.

The project consists of ~3800 modules.
With 3 our incremental rebuilds were between 3-4 seconds, consistently.

After upgrading and using the CRA config I ended up with rebuilds between 8-12 seconds. Panic mode set in, started disabling all I could disable, and tried as many tweaks as I could think of, including but not limited to:

disabling splitChunks
disabling sourcemaps (this has no observable impact!?)
configure thread-loader with poolTimeout: Infinity
disable thread-loader alltogether (general rebuild time didn't go down but they were more consistently in the lower 8-9 range after this)
configure cache-loader (had to disable this again on babel-loader as it seemed to have a negative impact when combined with babel's caching)
combine cache-loader with thread-loader
get rid of cache-loader & thread-loader and use happypack instead (still ~8 seconds)
explicitly turn on cache, module.unsafeCache & module.cacheWithContext (but I'm guessing they're on by default)
turn off pathinfo
disable CRA's tap hook for custom messages

None of these changes made any considerable impact, even when combined. I have the feeling there's some kind of bottleneck but I'm at a loss what to try next.
I tried running the webpack.debug.ProfilingPlugin but this keeps throwing on an incremental build (Error: Session is not connected), not sure how to work around this. The initial build time gets logged but doesn't interest me.
The current config is this one: https://gist.github.com/pleunv/6979a32fd8d77ea2dbc7a833437e0b4e
Any help/insight would be massively appreciated :)

Same drama here :(

So I guess what you're saying is to pull all of the node_modules out of the main dependency graph and put them in a separate entry point during dev.
I'm curious if there would be a way to more easily automate or or configure this behavior... @sokra, any insight in this by any chance? :) This is something the CommonsChunkPlugin used to be able to do with a filter I guess, but this probably didn't have the same effect as setting it up as a separate entry point.

mmmm sounds like a solution but not the solution, I think there must be something of more automatic. With webpack 3 everything was going pretty well and it was fast

I can also confirm that.
webpack 3: 4 sec. HMR
webpack 4: 10-20 sec. HMR
(node 8.9.4)

Incase this helps anyone else, I was getting crazy slow hot reloads times after starting a new project based on a previous architecture and upgrading it to webpack 4.
Reading through this thread I tried to downgrade to an older version of webpack and realized that it was actually a performance degradation in flow 0.57.1.  See https://github.com/facebook/flow/issues/5858
I've successfully upgraded to webpack 4.8.1.   I'm getting reload times of ~2 seconds on a 4k line project based on material-ui/react-router.  With flow 0.57.1 it was close to ~30 seconds or the initial startup time.

I just did the following things, which made the rebuild process n times faster.

Use node v8.9.4
nvm install 8.9.4

Replace awesome-typescript-loader with ts-loader, then turn on transpileOnly and experimentalWatchApi
module: {
  rules: [
  {
    test: /\.tsx?$/,
    use: [{
      loader: "ts-loader",
      options: {
        transpileOnly: true,
        experimentalWatchApi: true,
      }
    }],
  }
  ...
}

Turn on development mode and disable pathinfo
const config = {
  output: {
    ...
    pathinfo: false,
  },
  mode: 'development',
  ...
}

Most of the ideas come from https://medium.com/@kenneth_chau/speeding-up-webpack-typescript-incremental-builds-by-7x-3912ba4c1d15, an awesome article.

I'm not using TS and I've tried all of those things, none made a meaningful impact. Could someone help me profile my particular case? Still can't get the ProfilingPlugin to work for a rebuild.

@pleunv I was able to find where my build was slow by removing plugins until it was fast.
In my case it was compiling .less files that were slow. I added cache-loader after css-loader and it fixed the problem.
https://gist.github.com/cdeutsch/361fc44b545eb4a832212324d5f298d8
Then a week later, upgrading from<EMAIL_ADDRESS>to<EMAIL_ADDRESS>(possibly combined with<EMAIL_ADDRESS>made the cache-loader unnecessary.
No idea if it's related to the rest of the reported problems.
@p3x-robot try run tsc on it's own outside Webpack. I assumed it was Typescript that was slow, but it turned out to be .less which was being compiled as part of Typescript file edits because I was importing them as a CSS Module.

@cdeutsch Gave that a try just now, disabled all non-critical plugins and removed all loaders except for babel-loader and a catch-all file-loader. Still couldn't get it below ~7 seconds, which is really weird. I did notice that webpack seems to be doing more work than necessary... it emits every single chunk every rebuild, while in 3 it only touches the chunks affected by the change. Anything I could be doing wrong here?

I'm sorry but could we stay on-topic here please?

wow, I upgraded from node 8 to node 10 and rebuild time went down from 10s to 2s 🤯
(like before with webpack 3)

new HappyPack({
      id: 'js',
      threads: 4,
      // https://github.com/babel/babel-loader#options
      loaders: ['babel-loader?cacheDirectory'],
    }),

cacheDirectory on babel-loader improved a lot for us

I was able to make some improvements by switching to node 10, switching from ExtractTextPlugin to mini-css-extract-plugin, using fast-sass-loader instead of sass-loader, and using happy-pack.  Managed to get rebuild time from ~5s in WP3 to ~2.5s in WP4

Some versions of node 8 have perf regressions

Same here, first time using Webpack since February, and I thought the -w watch was broken, but it's just building slow.

hot rebulding is compiling all files instead of just the one modified
check my issue with react-hot-loader https://github.com/gaearon/react-hot-loader/issues/1026
when I changed to parcel, it just worked

Using hard-source-webpack-plugin and node 10 helped a lot here...

Probably you are building it wrong for developmetn dont forget to set with new webpack mode to development
instructions: https://webpack.js.org/concepts/mode/

I did two things that hugely improved my incremental build times:

The aforementioned trick around output: { ..., pathInfo: false }, helps
Moving my dev sourcemaps from devtool: 'source-map' to 'cheap-module-eval-source-map' increased build time by 10x (!!!)

Confirmed that @voxmatt method did help me reduce from 80s to 20ish s. HUGE improvement... But sametime, pretty surprised webpack got such a poor performance just because its loading dev with source-map.

Could someone else who is experiencing this try editing webpack/lib/Stats.js and change the default for showReasons to false:
// const showReasons = optionOrLocalFallback(options.reasons, !forToString);
const showReasons = optionOrLocalFallback(options.reasons, false);

I don't know why or if it's necessary, but it seems like stats.toJson is called a few times for me and reason calculation is very slow. Note that even if I set reasons: false it is still often called w/ options as {} or undefined

Related, if you're using html-webpack-plugin, set templateParameters:
new HtmlWebpackPlugin({
  // ..
  templateParameters: (compilation, assets, options) => ({
    compilation,
    webpackConfig: compilation.options,
    htmlWebpackPlugin: {
      files: assets,
      options,
    },
  }),
}

This leaves out a call to getStats().toJson() without options:
    webpack: compilation.getStats().toJson(),

Adding that dropped my rebuild from 13s to 5s (still too slow... still working...)

Related, if you're using html-webpack-plugin, set templateParameters:

That issue has been fixed in the upcoming html-webpack-plugin v4 release (a pre-release is available -<EMAIL_ADDRESS>- see jantimon/html-webpack-plugin#953 :-)

@edmorley Thank you! I just discovered that as well. It's much much faster.

I experienced the same issue after upgrading,
for me i was using this command to build
node ./node_modules/webpack/bin/webpack.js --config xxx
after replacing it with
./node_modules/.bin/webpack --config xxx
it's back to normal now!

I am currently upgrading Webpack 3 (https://github.com/WikiEducationFoundation/WikiEduDashboard/blob/master/gulp/tasks/webpack.js) and here is my PR (https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/1981).
The Production version is fine. But the Development version is considerably slower in Webpack 4 when it was nearly instantaneous in Webpack 3.
Can anybody look into what should be changed? The config files are all in the above links.

Guys.
I think key solution of this issue, devtool option.
I change 'inline-source-map' into 'cheap-module-eval-source-map' on devtool option, and
speed is faster from 15s to 2s.
thanks @voxmatt
you save my time.

Thanks @searene!  thewebpack mode is very important option about webpack speed

Setting mode to develop really increases the reload speed  but I need to use localhost as I have an environment map as follows:-
let appEnv;

switch (process.env.NODE_ENV) {
  case "development":
    appEnv = {
      //... dev environ config
    };
    break;
  case "production":
    appEnv = {
      //... prod environ config
    };
    break;
  default:
    appEnv = {
      //... local environ config
    };
}

export default appEnv;

Specifying development as mode was reloading the changes really fast but the functionality was breaking, as development app environment was returned instead of local.
Added following to solve the problem:-
    new webpack.DefinePlugin({
      "process.env": {
        NODE_ENV: JSON.stringify("localhost")
      }
    }),

@voxmatt Oh man, you've saved my life, thanks a lot!

what is the proper version of html-webpack-plugin to fix this?

@sibelius see if https://github.com/styleguidist/mini-html-webpack-plugin helps with it!

I want to close this issue, for webpack@4 we did everything we could, also almost all loaders and plugins received updates and performance improvements, webpack@5 has more improvements and very fast, anyway if you faced with performance again (in webpack@5 too) please open new issue with reproducible test repo, thanks for using webpack, feel free to feedback

Late to the party but I recently run into a similar issue. We end up finding that WEBPACK_DEV_SERVER env variable is no longer set in the new dev server (https://github.com/webpack/webpack-cli/issues/1973). We use the variable to gate whether to run optimization and other production only configs. The performance is recovered after this is fixed.

Same issues with our project. What helped:

follow recommendations from Speeding Up Webpack, Typescript Incremental Builds by 7x

swap thread-loader with happypack, latest beta version supports webpack v4, use 4 threads (3 is a default)
integrate hard-source-webpack-plugin, subsequent builds are 2 faster

Scrolling through all replies, tried out many of them. This is the most effective one.  Our project hot loader speed has decreased from 10s to just 200ms. Crazy!

This works for me
"mode": process.env.NODE_ENV === "production" ? "production" : "development",


[Test] Accounts management

create/delete/modify account
address qr code modal
account sync (eth)
account sync (btc)
Use reduxTracker to ensure actions are fulfilled and no error snackbars showed

stop at third party wallet connect

unable to add Coinbase account due to SMS text verification upon login


positive/negative bar

@kimmobrunfeldt I have a sort of oddball issue.
I am trying to create a progress bar that starts in the center of a bar, and moves either left (up to -100) or right (up to +100). The standard line will start on the right and move left if a negative value is given when it is animated.
Do you have any suggestions for how I might go about this?

Interesting question. Right now the only possible solutions I can come up with would be to use two Line progressbars.
I tried to test if you could do it like this:
1. Draw line right (L command)
              ----------->

2. Move back to start  (M command)
|             ----------->

3. Draw line to center (L command)
----------->---------->

4. Finished path
---------->----------->
 -0.5 - 0    0 - 0.5

But it didn't work. Check this JSfiddle https://jsfiddle.net/7rsdpc59/1/

I actually got it working! I just had to keep the path as a continuous line.
Now the line is formed like this:
1. Draw line right (L command)
             ------------>

2. Draw line to start  (L command)
<-------------------------

3. Draw line to center (L command)
------------>

4. Finished path
<----------->------------>
 -0.25 - 0    0 - 0.25

Now you can animate between -0.25 and 0.25 to achieve the effect.

Closing the issue, solution above.

Hello guys,
I develop an example for this case.
It could be an alternative for this purpose and show negative values using two progressing bars.
I want to share my solution with us. Js Fiddle
Regards.


Update repository info

Suggest updating the repository info such that https://pypi.org/project/cobs and docs point directly to the current source location.

Thanks for your contribution! I appreciate it. I didn't notice that this info was still showing the old repositories.


The conversation with Randall video embed is broken

https://genevaers.com/uncategorized/2021/03/28/genevaers-developers-a-conversation-with-randall-ness/

Fixed by re-editing the version of the video on GenevaTV and uploaded to the site


Introduce remove_stopping_executable with Docker

This allows to run the code in isolation and from the command line.

Thanks for this helpful tool!

Update: Trying to run this again today and now it fails with

docker: Error response from daemon: error while creating mount source path '/host_mnt/Volumes/Untitled/DCIM/2024-11-29 Tobias PuntaVelo/100GOPRO': mkdir /host_mnt/Volumes/Untitled: file exists.

Which I don't understand or can fix.
So this is not merge ready, yet…


use build

Replace sdist by build

tested via https://github.com/jburel/omero-figure/actions/runs/9233345157


Draft: Use docker only in tests

Please let's NOT do this - make the container binary switchable / configurable / autodetected.
Instead of this you could just update the readme to to install podman-docker package or hell, run 'alias docker=podman'

Personally, I don't think the change is necessary – previously, the podman was used for the httpd's image, docker for tomcat. However, with the latest changes (9096b14f1c94be6f08aebe4e642a7e267ade67f6, e4422b6dd9c6d7255cf2d6f0ce422b127c7c3709) the tests are broken and we should address it. (The second commit suggest that we want to move to podman, hence this PR.)

Please let's NOT do this - make the container binary switchable / configurable / autodetected.
Instead of this you could just update the readme to to install podman-docker package or hell, run 'alias docker=podman'

Personally, I don't think the change is necessary – previously, the podman was used for the httpd's image, docker for tomcat. However, with the latest changes (9096b14, e4422b6) the tests are broken and we should address it. (The second commit suggest that we want to move to podman, hence this PR.)

Yeah, these went in without a pull request - it looks like @jfclere accidentally pushed directly to the branch so this was missed in review.

I changed the podman to docker as agreed. Also, I made a few changes to tests. It seems that the previously introduced changes are causing issues when tests are re-run; one has to be sure that everything from the previous run is cleaned up (tests are now faster so it seems like a reasonable trade off until we rework the tests in the future).


MySQL cant handle " around column names

As we force DBImport to use more and more custom SQL for sqoop, some databases failes with "org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.io.LongWritable". Reason is because it uses a custom SQL for the sqoop query and the column names have " around them. This creates a value instead of reading that column.

Added a check to verify what database type requires " and only add them in case the source requires it.


Jekyll serve jacks up the cpu to 100% on MacOS

I don't know why but for some reason this jekyll site jacks up the CPU usage. A normal Jekyll build takes 8 seconds while on my own site (github.com/httpsterio/rautalanka) it takes under a second. I don't know if it's some gems you have installed or why, but it's almost unusable for me.

Ok solved,
the exclude on line 45 in _config.yml gets overridden by the excludes at the bottom of the config.
node_modules was the issue that jacked everything up while building and serving the site. I'd remove the excludes from line 45 and add node_modules and .git to the exclude at the end of the config.

@httpsterio interesting stuff! Never happened to me but thanks for reporting it! See https://github.com/myJekyll/jasper2/commit/77c99b500612b5cdab24756dd39309f3afd70fce
Cheers!
F:)


Dashboard authentication is not working ?

I'm trying to improve access control by uncommenting dashboard auth settings in /opt/graphite/webapp/graphite/local_settings.py
###############################
# Authorization for Dashboard #
###############################
DASHBOARD_REQUIRE_AUTHENTICATION = True
DASHBOARD_REQUIRE_EDIT_GROUP = 'dashboard-editors-group'
DASHBOARD_REQUIRE_PERMISSIONS = True

I'm working with the embedded authentication ( using mysql as de database backend), and any authentication related log is written on the /opt/graphite/storage/log/webapp/*.log files.
What is what I'm doing wrong ?

I am seeing the same issue: Graphite-web dashboard authentication simply does NOT work. Every user which can access the dashboard via a web browser can modify/save dashboard without the need to login.
OS: Ubuntu 14.04
Graphite-web: 0.9.12

Is there any solution for this issue?
I really like Graphite and want to use it, but I cannot allow access to my systems data without authentication.

You can use some external dashboard with authorization (e.g. Grafana http://grafana.org ) and limit Graphite access only from that Grafana host (localhost, if installing Grafana on the same host as Graphite)


Unstable result

I have trained this repo with 13+ hours of Korean speech, about 1600 epochs.
The synthesized results are not stable,  sometime the model generates meaningless audio (with some .
like this:
111.zip
(it should stop at first  or 2nd second, but it does not)
Sometime it could not synthesize the sentence completely like this:
222.zip
Could someone please tell me what is the problem here? I think the problem is at attention alignment, but don't know how to fix this.
Any suggestion is highly appreciated.
Thank you.

Check this one out for same problem.
Also, trimming and adding some silence with preprocessing audio files help a lot.

Check this one out for the same problem.
Also, trimming and adding some silence in the end with preprocessing audio files helps a lot.

Thank you so much, sir.


fix(stack-props): make acm, zoneId and zoneName mandatory

make acm, zoneId and zoneName mandatory

Now cdk diff with no context variables will return warning annotation messages like this but will not stop the deployment.

@guan840912  Ready now!


pyqir 0.10.0 update not compatible

Using pyqir 0.10.0 generates the attributes in a different way. This shows the following error when submitting qir:
"1000: Compile error: -1 Expected the QIR file to have qubit count specified but no annotation was found"

solved

Solved


[Feature] AndroidX support

Since Android development is migrating to androidx - should we implement those stubs and maybe moxy-androidx? Would like to participate :smiley:

See #204


Pylance Django Intelesence

Environment data

Language Server version: v2023.5.20
OS and version: Windows 11
Python version: 3.7.12 pyenv

Code Snippet
VSCode settings.json
"python.languageServer": "Pylance",

Repro Steps

Create Django model and call model.objects.create method

Expected behavior
When I use Jedi as an LSP, I get the following result, I would like the same from Pylance.

Actual behavior

Also when going to the definition I get stub files instead of the source code.

unfortunately, this is something pylance can't handle in current form. jedi could handle this since it actually run code to access those information, but pylance will only statically inspect code and this kind of thing is an area it can't handle. this kind of behavior can only be supported by tools that actually run code such as jedi or notebook
we go to stub when you go to def on either object or create since the object is dynamically added to the Package model (https://github.com/django/django/blob/main/django/db/models/base.py#L419) and it doesn't actually exist in code (or I should say it exists in code but not in a way tool can figure out without running the code)
so stub is one of workaround python tool uses in such situation. but for this particular case, there is another problem, the objects, code dynamically injected, can be different manager based on concrete type so stub has to use the common denominator which is BaseManager
and that is why you see the catch all signature of **kwargs
in future, when we add django specific feature we might be able to figure out how to support this kind of dynamic behavior, but for now, I think you best bet is staying with tools that actually run code such as jedi or notebook if you want this kind of behavior.


Missing an expected key: 'NSPrivacyCollectedDataTypes' in Amplify_InternalAmplifyCredentials.bundle/PrivacyInfo.xcprivacy

Describe the bug
I'm using Amplify in my new app and getting an invalid binary error when I upload it to the App Store for review.
Privacy report has the following error: "Missing an expected key: 'NSPrivacyCollectedDataTypes' in Amplify_InternalAmplifyCredentials.bundle/PrivacyInfo.xcprivacy"
It seems similar to this issue: https://github.com/aws-amplify/amplify-swift/issues/3552. However, 3552 is marked as fixed already.
Any thoughts?
Steps To Reproduce
Steps to reproduce the behavior:
1. Add Amplify to your project
2. Archive the bundle for distribution.
3. Generate Privacy Report for the Archive.
4. See the report

Expected behavior
Clean privacy report
Amplify Framework Version
2.35.5
Amplify Categories
Auth
Dependency manager
Swift PM
Swift version
5.10
CLI version
n/a
Xcode version
Xcode 15.4 (15F31d)
Relevant log output
<details>
<summary>Log Messages</summary>

INSERT LOG MESSAGES HERE

```
Is this a regression?
Yes
Regression additional context
There is a similar bug that marked as fixed: https://github.com/aws-amplify/amplify-swift/issues/3552
Platforms
iOS
OS Version
iOS 17.5 / MacOS 14.5
Device
iPhone 15 pro
Specific to simulators
No response
Additional context
No response

Could you tell me how to do it?


create mock install middleware named "acceptMockInstall"

Hi @zhanghao-zhoushan,
Please create a middleware named "acceptOverwrite" and add unit test for it.
The behavior of the middleware:
When ctx.options.mockInstall is truthy, set ctx.mockInstall to true. When ctx.options.mockInstall is falthy, do nothing.
Pass through the next callbacks. Do not block or terminate middleware passthrough process.
The test cases of the middleware:
when ctx.options.mockInstall is truthy, alters context's mockInstall value.
when ctx.options.mockInstall is truthy, pass through middleware process.
when ctx.options.mockInstall is falsy, do nothing.
when ctx.options.mockInstall is falsy, pass through middleware process.

The implementation is similar to #18.

👍👍Well Done! @zhanghao-zhoushan


[Feature Request]: Support Adoptium JVM

What would you like to happen?
Do you plan to support Adoptium JVM from Eclipse Foundation in the future?
Asking cause of having trouble to see the hop-gui with the latest Temurin JRE with version 11.0.22+7
(hop-gui is working with 11.0.20,8)
and we have to use Adoptium JVM as default.
Also have seen in your superb docs here that it is currently unsupported -  https://hop.apache.org//manual/latest/supported-jvms.html
Issue Priority
Priority: 3
Issue Component
Component: Hop Gui

We are unsure what is causing the Adoptium/Temurin JVM issues.
The page you are referring to was created after we received multiple reports of issues with this specific JVM.
As far as I know, the baseline for all these JVM's is the same (OpenJDK) so it was a big surprise to us that the UI does not load on Temurin (especially with the fact that we use SWT which is also an Eclipse project).

I'm using Temurin on Windows/Linux from the time being without any problem. Currently I'm on jdk 11.0.21+9. @seltner I will try the version you are suggesting and let you know

Hi @hansva ,
thx for the info - and also for me a good info that you crated that page after having problems with Adoptium/Temurin JVM
and not the other way round that you already planned from the start to not support this specific JVM :rofl:

Windows 11 with Temurin JDK (the version that is failing on your side).
C:\java\hop\hop>echo off
===[Environment Settings - hop-gui.bat]===================================

Java identified as "C:\Program Files\Eclipse Adoptium\jdk-<IP_ADDRESS>-hotspot\bin\java"

HOP_OPTIONS=-Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=GUI -DHOP_AUTO_CREATE_CONFIG=Y

Command to start Hop will be:
"C:\Program Files\Eclipse Adoptium\jdk-<IP_ADDRESS>-hotspot\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\* -Djava.library.path=lib\core;lib\beam -Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=GUI -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.ui.hopgui.HopGui

===[Starting Hop]=========================================================
2024/02/05 10:42:51 - Hop - Projects enabled
2024/02/05 10:42:51 - Hop - Enabling project : 'default'

I can confirm it doesn't work with the latest version
openjdk version "11.0.22" 2024-01-16
OpenJDK Runtime Environment Temurin-11.0.22+7 (build 11.0.22+7)
OpenJDK 64-Bit Server VM Temurin-11.0.22+7 (build 11.0.22+7, mixed mode)

I think is a MacOS only issue unfortunately :-(

It's a MacOS only issue
https://github.com/eclipse-platform/eclipse.platform.swt/issues/1012

The problem you are seeing is caused by the fact that the latest Temurin JDKs for Mac are built using macOS SDK 14. The Oracle JDK is built using an earlier SDK. If Oracle and others ever build their Mac JDKs against macOS SDK 14 or later you'll see the same problem.
See https://github.com/eclipse-platform/eclipse.platform.swt/issues/1012#issuecomment-1915168980

It seems a fix has been applied to SWT to handle this problem.
One small issue on our side... we are stuck on the current SWT version as the newer version requires us to start the Java upgrade. New SWT releases only work on java 17 and higher.