ldaps issue with any node js versions

For some reason I am not able to authenticate securely with AD. I am able to do ldap but not ldaps.
I see two type of errors:

TypeError: Cannot read property 'on' of undefined

Uncaught Exception: unable to get local issuer certificate

I am currently trying Node v8

'use strict';
var AuthStrategy = require('../auth-strategy.js'),
user_profile = require('../user-profile.js'),
passport = require('passport'),
config = require('../../config/auth-strategy-config.json'),
logger = require('../../logger').logger,
ldapConfig = config.active_directory_ldap,
ldap = require('ldapjs'),
ldapurl = 'ldap://' + ldapConfig.host + ':' + ldapConfig.port,
opts = {
base: null,
filter: null,
scope: null,
attributes: null
};
var fs = require('fs');
var tls = require('tls');
var ldap = require('ldapjs');
server: {
url: ldapurl,
searchBase: ldapConfig.userbaseCtxDN,
searchFilter: ldapConfig.userbaseFilter,
adminDn: ldapConfig.bindDN,
adminPassword: ldapConfig.bindCredentials,
searchAttributes: ['mail', ldapConfig.userAttributeID, ldapConfig.displayNameAttr, ldapConfig.securityRolesAttr],
tlsOptions: {
key: fs.readFileSync('../../JBoss/Eo/server/config/aeDCs/eo.net.key'),
cert: fs.readFileSync('../../JBoss/Eo/server/config/aeDCs/eo.net.crt'),
ca: [
fs.readFileSync('../../JBoss/Eo/server/config/aeDCs/dc.crt')
]
}
}
I was even asked to try the following which I did
https://github.com/felixrieseberg/windows-build-tools
Still cannot authenticate

This almost looked like my issue but it didnt help
https://github.com/joyent/node-ldapjs/issues/229

Did you verify AD is properly setup for LDAPS and verify what port it was setup with? (Default is 636)
I only ask because many IT admins do not setup LDAPS by default.

So this application is broken into two. One is a thick client and the other is a web client.
I was able to get the thick client to conenct with LDAPS by installing the DC certs on the server manually. Then I changed the url from ldap://dc/ to ldaps://dc/. That worked and so easy.
For the Web Client its not so easy. With all the Nodejs and JavaScript stuff. What is confusing is that I am able to connect with ldap and not ldaps on the webclient. I have done telnet on port 636 and also connected sucessfully with LDAP Admin to the DCs on 636 using TLS/SSL. I believe the issue is in the ldaps/nodejs on the app.

Is it a self-signed cert and if so is it in a local CA that the Node server is configured with? If not I believe you can by pass it using the Node TLS API for testing purposes. https://nodejs.org/api/tls.html#tls_tls_connect_options_callback
You should be able to use tlsOptions to send it the settings needed to bypass the CA check or you can send it a pem file. Which is shown in the link above.
Should be something like:
const client = ldap.createClient({
    url: 'ldaps://<IP_ADDRESS>',
    tlsOptions: {
        rejectUnauthorized: false
    }
  });

My certs are all from our internal CA. So both the server and the DC certs are from our company CA. I have imported my server's cert on the client and made sure SSL 3.0, TLS 1.0, TLS 1.1, and TLS 1.2 are all active in the browser.
So connection should look like this for the handshake Client->Server->DC
If I bypass the CA check, will the app still process LDAPS connection? I ask because we can do LDAP currently but our organization doenst like passing credentials in cleartext. If I can bypass the CA and still process the transaction with LDAPS (port 636) then we will like to try that option.
I get the following errror from the code above
const client = ldap.createClient({
^^^^^^
SyntaxError: Unexpected identifier
Thanks for your reply

Are you making the request from the client / browser or from a Node backend?
Browser app (Like React) -> Nodejs Server -> AD
Browser app (Like React) -> AD
Bypassing the CA check will still allow secure connections, but will not validate the certificate to prevent man in the middle attacks. Think of it like clicking the continue anyway button on browsers when accessing a website that is using a self signed certificate.

Browser app (Chrome) -> Nodejs Server -> AD
That sounds like an option I will like to try. The intranet site is already using port 443 signed by our local CA. The app will never be used on the internet (outside our domain). The only issue is the LDAPS and our security team don't like 389 so if I can bypass the CA check it would help. I wonder if the check is trying to go online to verify.....
/
I got this from our vendor but I wasnt really sure why I needed it but I suspect it had something to do with rebuilding the sqlite3, that the app was complaining about. Also to help register the cert online. We are certainly not experts in this area but we are willing to learn and not give up on trying to make port 636 work. So beware that this might not make sense....
https://github.com/felixrieseberg/windows-build-tools
Python
VC++ Build tools 2015 (freely available from MS)
Make scratch directory, for example
c:\server
Dealing with proxy issue
2.1) Change your NPM configuration to set proxies
npm config edit
proxy=http://:80/ https-proxy=http://:80/
2.2) Allow node to use additional GE certificates Create file e.g. C:\certs\GEcerts.pem
as text file combining GE_External_Certificate1.pem,GE_External_Certificate2.pem
https://nodejs.org/api/cli.html#cli_node_extra_ca_certs_file
2.3) Install node-pre-gyp
set NODE_EXTRA_CA_CERTS=C:\certs\GEcerts.pem https://nodejs.org/api/cli.html#cli_node_extra_ca_certs_file
npm install node-pre-gyp –g
2.4) Either
1.4.1) directly install Python 2.7.x and VC++ Build tools 2015
OR
1.4.2) let npm do it for you.
From Windows PowerShell (Administrator) session
cd c:\server
set-location env:
new-item -path . -name NODE_EXTRA_CA_CERTS -value C:\certs\GEcerts.pem
This is the Powershell equivalent of set NODE_EXTRA_CA_CERTS=C:\certs\GEcerts.pem https://nodejs.org/api/cli.html#cli_node_extra_ca_certs_file
npm install --global --production windows-build-tools https://github.com/felixrieseberg/windows-build-tools
1.5) set PYTHON=...path to python.exe
1.6) From "server" folder
npm install sqlite3 --build-from-source
/
I suspect one of the command lines above goes online to register the local Cert. If I can bypass it that will help.

Unfortunately NodeJS has a set list of CAs and does not use the servers certificate store. You will need to either load in the CA, which it looks like you were trying, or do the CA check bypass. There is some information and even a commit talking about using the servers certificate store, but I have never used it and unsure if it's actually functional.
Next time I'm on-site at my office that has a Windows domain I'll try making a connection using a self signed certificate to see if it goes through smoothly. I don't see why it wouldn't, but I have never used a self signed certificate with this library.
You have probably already double checked these, but do you have port 636 open on the firewall and in this code block are you updating the ldapConfig.port variable to 636 when you change ldap to ldaps?
ldapurl = 'ldap://' + ldapConfig.host + ':' + ldapConfig.port,
The url parse method in ldapjs looks for ldap or ldaps and if no port is passed it will default it to 389 or 636. I normally don't pass it a port unless I'm working with someone who is using a strange port for their LDAP server.

Yes that was what I suspected was happening. I suspected I needed to open firewall to make the registration happen. Our servers are hardened so I need to get through all that later.
Yes I have port 636 opened from the server to the AD. My TELNET works and also LDAP ADMIN works on port 636 using SSL/TLS. The issue might be java script and maybe node area within the app.
I am passing in the following variables:
ldapurl = 'ldap://' + ldapConfig.host + ':' + ldapConfig.port,
and can see that npm is clearly showing:
ldapurl = ldaps://ServerName:636
so ldapurl = ldap://ServerName:389 works just fine.
For now, are you able to help me bypass the CA check and I will deal with the online registry later.

I tried your suggestion above and npm doesn't like it.
const client = ldap.createClient({
url: 'ldaps://<IP_ADDRESS>',
tlsOptions: {
rejectUnauthorized: false
}
});
It crashes because I am not a JS programmer. I am not doing something right. Can I list tlsOptions twice?
I already have it in my code block above that is handling all the cert stuff.

No just replace your current TLS options with this one:
tlsOptions: {
    rejectUnauthorized: false
}

If the code you provided above is right it should look something like this:
'use strict';
var AuthStrategy = require('../auth-strategy.js'),
    user_profile = require('../user-profile.js'),
    passport = require('passport'),
    config = require('../../config/auth-strategy-config.json'),
    logger = require('../../logger').logger,
    ldapConfig = config.active_directory_ldap,
    ldap = require('ldapjs'),
    ldapurl = 'ldap://' + ldapConfig.host + ':' + ldapConfig.port,
    opts = {
        base: null,
        filter: null,
        scope: null,
        attributes: null
    };

var ldap = require('ldapjs');

server: {
    url: ldapurl,
    searchBase: ldapConfig.userbaseCtxDN,
    searchFilter: ldapConfig.userbaseFilter,
    adminDn: ldapConfig.bindDN,
    adminPassword: ldapConfig.bindCredentials,
    searchAttributes: ['mail', ldapConfig.userAttributeID, ldapConfig.displayNameAttr, ldapConfig.securityRolesAttr],
    tlsOptions: {
        rejectUnauthorized: false
    }
}

Ok so npm didnt complain during startup. I went to the login page and tried signing in then got the following:
``

ERROR   Fri Apr 13 2018 10:48:34 - TypeError: Cannot read property 'on' of undefined
at setupSocket (D:\JBoss\Eo\server\node_modules\ldapjs\lib\client\client.js:111:15)
at Client._connect (D:\JBoss\Eo\server\node_modules\ldapjs\lib\client\client.js:742:3)
at new Client (D:\JBoss\Eo\server\node_modules\ldapjs\lib\client\client.js:247:22)
at Object.createClient (D:\JBoss\Eo\server\node_modules\ldapjs\lib\client\index.js:60:12)
at new LdapAuth (D:\JBoss\Eo\server\node_modules\passport-ldapauth\node_modules\ldapauth-fork\lib\ldapauth.js:129:28)
at Strategy.handleAuthentication (D:\JBoss\Eo\server\node_modules\passport-ldapauth\lib\passport-ldapauth\strategy.js:
156:10)
at Strategy.authenticate (D:\JBoss\Eo\server\node_modules\passport-ldapauth\lib\passport-ldapauth\strategy.js:200:33)
at attempt (D:\JBoss\Eo\server\node_modules\passport\lib\middleware\authenticate.js:341:16)
at authenticate (D:\JBoss\Eo\server\node_modules\passport\lib\middleware\authenticate.js:342:7)
at Layer.handle [as handle_request] (D:\JBoss\Eo\server\node_modules\express\lib\router\layer.js:82:5)

Well at least the certificate error is gone now. :+1:
What version of ldapjs is in your package.json file?

Based on the error you have and looking at previouse issues it looks like you likely have an old version of ldapjs installed. See #289
If this is the case I would try updating the library, over the time I have used this library I haven't noticed any breaking changes. If it does break something alternatively you could downgrade to the older version of NodeJS if the rest of your app isn't using any of the newer features/API in NodeJS.

I see   "version": "0.7.1",

I added ldapjs to my dependences because I dont see it there per #289 and it still didn't work.
Is my ldapjs too old?
I also have node version 8.11.1

Can you past what your package.json dependencies looks like here?

Yes sure.
"dependencies": {
"asn1": "0.2.1",
"assert-plus": "0.1.5",
"bunyan": "0.22.1",
"nopt": "2.1.1",
"pooling": "0.4.6",
"dtrace-provider": "0.2.8",
"ldapjs": "mcavage/node-ldapjs#acc1ca8f4314fd9d67561feabc8ce4c235076a5e"
},
I just added this yesterday per case #289
"ldapjs": "mcavage/node-ldapjs#acc1ca8f4314fd9d67561feabc8ce4c235076a5e"

The above line from that case is old, the newest version of ldapjs is what you want and will resolve the issue in the case.
The latest version is 1.0.2, I would recommend using the npm update to update your packages. It normally will do all the dirty work for you. https://docs.npmjs.com/cli/update
However I believe you said you did not have any entry for ldapjs in your package.json before this, is that correct? If that is true it must be a dependency of some other library or manually imported.

Yes. Here is what it looked like:
"dependencies": {
"asn1": "0.2.1",
"assert-plus": "0.1.5",
"bunyan": "0.22.1",
"nopt": "2.1.1",
"pooling": "0.4.6",
"dtrace-provider": "0.2.8",
}
I ran npm update and it flashed something and went back to the prompt. Not sure if am suppose to see anything.

Is this an app you or someone in your organization built? Do you know if all the dependencies were installed using npm or did they give you all the node_modules manually?

No our vendor developed the app or maybe third party. I know I have been given ..\server\node_modules\ldapjs folder before to update just to try out things to fix the ldaps issue.
I also see another package.json further up above the folders.
Right now, I see npm is stuck after npm update
SyntaxError: Error parsing D:\JBoss\Eo\server\node_modules\ldapjs\package.json: Unexpected token } in JSON
at position 1232
at JSON.parse ()
at readPackage (module.js:126:52)
at tryPackage (module.js:136:13)
at Function.Module._findPath (module.js:218:20)
at Function.Module._resolveFilename (module.js:545:25)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object. (D:\JBoss\Eo\server\auth\impl\ldap-ad-auth-strategy.js:8:10)
at Module._compile (module.js:652:30)

You should have a package.json further up in your app, likely in the server\ folder. Is that the package.json you pasted above?
It should look something like this:
yourapp\node_modules
yourapp\package.json

The package.json inside ldapjs folder belongs to the ldapjs library and you typically do not want to touch it.
For reference here is the location of the ldapjs package.json that you wouldn't want to change:
server\node_modules\ldapjs\package.json

No I pasted the one inside of ldapjs folder.

Can you past the one from in server\ folder?

Perfect, okay so here is what I suggest you try doing.

Make a backup of your package.json, you can copy past it to anywhere.
Delete the ldapjs folder inside node_modules server\node_modules\ldapjs
Edit server\package.json and change "ldapjs": "^0.7.1", to "ldapjs": "^1.0.2",
While in the server directory run npm install

Would npm install need internet access or is it local to the server?

It needs internet access, so if the server doesn't have internet you will need to do it on another computer and manually copy the folder.
Give me a minute and I can post a temporary link to download it if you'd like. Just make sure you update your package.json so if anyone needs to install the app again in the future it will have the latest version.

It looks like the dependencies change between versions, I'd copying your app to a thumb drive and installing the dependencies and copying it back to the server.

Update the server\package.json as mentioned above
Backup the node_modules folder, just in case they gave you any custom ones not in the package.json
Delete server\package.json
Copy app to thumb drive
On a computer with NodeJS and internet change to the app directory and run npm install
Copy app back to the server

Ok just to be clear
Step 1 to backup Server\package.json
Step 3 to delete the file package.json from its location
Doing all the steps and I will report back.

I see the updated Server\node_modeules. There are now 138 items in there. The old one was only 47.
But I do not see ldaps folder in there.

Assuming the developers added everything to your package correctly it should automatically install everything needed after you run npm install. If it needed to compile any modules from source you should have gotten an error.
Did you get any errors when you ran npm install or when you tried to run the app after copying it back to the server?

Ok I spoke too soon. There are many errors. I guess am missing ``
`c:\Server>
c:\Server>c:\Server>npm install
'c:\Server' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>npm WARN deprecated<EMAIL_ADDRESS>Jade has been renamed to pug, please install the latest version of pug instead of jade
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN deprecated<EMAIL_ADDRESS>please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>
c:\Server>><EMAIL_ADDRESS>install c:\Server\node_modules\bcrypt
'install' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>> node-gyp rebuild
'rebuild' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>
c:\Server>'node-gyp' is not recognized as an internal or external command,
''node-gyp'' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>operable program or batch file.
'operable' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>
c:\Server>><EMAIL_ADDRESS>install c:\Server\node_modules\ldapjs Ori\node_modules\dtrace-provider
'install' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>> node-gyp rebuild
'rebuild' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>
c:\Server>'node-gyp' is not recognized as an internal or external command,
''node-gyp'' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>operable program or batch file.
'operable' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>
c:\Server>><EMAIL_ADDRESS>install c:\Server\node_modules\dtrace-provider
'install' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>> node-gyp rebuild || node suppress-error.js
'rebuild' is not recognized as an internal or external command,
operable program or batch file.
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\suppress-error.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>
c:\Server>'node-gyp' is not recognized as an internal or external command,
''node-gyp'' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>operable program or batch file.
'operable' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>npm notice created a lockfile as package-lock.json. You should commit this file.
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules\bcrypt):
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>install: node-gyp rebuild
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules\ldapjs Ori\node_modules\dtrace-provider):
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>install: node-gyp rebuild
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
module.js:549
throw err;
^
Error: Cannot find module 'c:\Server\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
c:\Server>
c:\Server>added 91 packages, removed 460 packages, updated 3 packages and moved 23 packages in 19.655s
'added' is not recognized as an internal or external command,
operable program or batch file.
c:\Server>`

The npm install command is deleting my "c:\Server\node_modules\npm" off the system.

Can you copy past the commands you are running along with what directory you are in?
You should just be in the app root folder and running npm install.
For example using powershell:
PS C:\Users\username> cd /app
PS C:\app> npm install
You should now have a app\node_modules folder and assuming you are using the latest NodeJS a app\package-lock.json file.

Yes sir it went a little further this time. I had to unistall node from c:\server which was were I initially had it installed and nothing seems to work there. After several tries, I decided to allow node install to the default c:\program files\nodejs and that worked.
Right now I see this error:
C:\Program Files\nodejs\node_modules\bcrypt>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node_modules\node-gyp\bin\node-gyp.js" rebuild ) 'node' is not recognized as an internal or external command, operable program or batch file. npm WARN<EMAIL_ADDRESS>had bundled packages that do not match the required version(s).  Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      C:\Users\giwahea\AppData\Roaming\npm-cache_logs\2018-04-18T20_43_09_361Z-debug.log
C:\Program Files\nodejs>`

I normally don't dev too much on Windows, but I believe I remember seeing that error before related to the C++ compiler with Visual Studio. Do you have the C++ compiler installed?
If you want you can send me your full package.json file. If so I can download all the dependencies for you and just send them to you in a zip file. If you don't want to share it here I can send you an invite to a privet repo on github or send you a Dropbox link.

I think it worked this time. How do I know if  ldapjs is updated?
I see the following:
C:\Program Files\nodejs>node -v
v8.11.1
C:\Program Files\nodejs>npm -v
5.8.0
C:\Program Files\nodejs>npm.cmd -v
5.8.0
C:\Program Files\nodejs>
npm used to be 5.6.0

I sent you an invite to a GitHub privet repo.
The easiest way to update the server is to just copy the NodeJS installer to the server and run it. I wouldn't really worry about that though, pretty much anything in the major version should have no compatibility issues.
One question you could ask the develoeprs is what version of NodeJS did they develop on. It's possible they developed on an old version of NodeJS and that may be why they used an outdated version of ldapjs.

The app was developed on nodejs v0.10.36

I aready installed v8.11.1 and v9.8.0. How do I get NODE_MODULE_VERSION 59? The upgrade to v9.8.0 is asking for  server\node_modules\sqlite3\lib\binding\node-v59-win32-x64\node_sqlite3.node
``

You want to downgrade, not upgrade NodeJS. They developed on a version from early 2015.

Uninstall all versions of NodeJS on the server
Download the NodeJS installer (v0.10.36) https://nodejs.org/download/release/v0.10.36/node.exe
Copy the v0.10.36 installer to your server
Install NodeJS v0.10.36
Deploy your app again using all the original files the developers sent you

Alternatively if you want to upgrade NodeJS you can keep trying to get the package working with the new updates, but you will need to get the ldapjs package updated. This does pose a risk that something they developed could be incompatible with the new version of ldapjs and or the newer NodeJS version you are using. Since you didn't develop the app I'd suggest downgrading, because we don't know what is in their app code.

Yes the application works with v0.10.36 and v8.11.1.
We are having issues doing LDAPS. We have upgraded the ldapjs to 1.0.2 and the app lunches.
The reject option you mension is what I am trying to do so I dont have to go on the internet to check CA.

Looking at the NodeJS API docs that TLS option is still supported in v0.10.36, so you should be able to still use that setting with the older version of NodeJS. This should also avoid the incompatibility issue of new NodeJS and old ldapjs. Which is what caused this error:
ERROR Fri Apr 13 2018 10:48:34 - TypeError: Cannot read property 'on' of undefined
You most likely need to do one of the following:

Use NodeJS 0.10 + ldapjs v0.7.1 and add tlsOptions: { rejectUnauthorized: false }
Use NodeJS v8 + ldapjs v1.0.2 and add tlsOptions: { rejectUnauthorized: false }

Ok making sure I have the above

AWESOME MR TASTYPACKETS.......YOU ARE THE MAN.
Thanks for not giving up on me....

Yes sir. It was a little painful but we did it.
The app was delievered with NodeJS 0.10 + ldapjs v0.7.1 a lot of the ldaps mouldes were missing.
Like you said....I went ahead and used your recommendation. So NodeJS v8 / ldapjs v1.0.2. Plus lots of dependences...
•Use NodeJS v8 + ldapjs v1.0.2 and add tlsOptions: { rejectUnauthorized: false}
and add tlsOptions: { rejectUnauthorized: false }
•Use NodeJS v8 + ldapjs v1.0.2 and add tlsOptions: { rejectUnauthorized: false}

Awesome, glad we were able to get it sorted out!

Can you mark this issue closed?

Yes I can.
Sent from Yahoo Mail on Android
On Fri, May 11, 2018 at 12:34 PM, Ezekiel<EMAIL_ADDRESS>wrote:
Can you mark this issue closed?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks again.

What version of Node and ldapjs are you using?
Did you add the option to ignore certificate verification?
tlsOptions: { rejectUnauthorized: false}

Node : 6.13.0
LdapJS: 0.7.1
Yes, I verified with changing tlsOptions and getting rid of the 2nd error, but still, I am seeing the 1st error.

If for some reason you can not upgrade ldapjs, you'll likely need to downgrade your version of Node installed on the server.

The second issue you are having is due to breaking changes between the Node and ldapjs versions. These are old and I'd encourage you to upgrade things for security, however if this was a 3rd party app and you are not comfortable updating the code I'd downgrade the Node version.
To downgrade Node you simply install an older version on your server. I belive further up in this thread I posted a link to the old version of Node this person's developers used.
If it's Windows simply uninstall Node and reinstall the correct executable for the version you need. If this is Linux I'd recommend using NVM to install the version you need.
Do you still have contact with the original author? Perhaps they can tell you what version of Node they developed on. If not based on that ldapjs version I'd guess it's a pretty old version of Node, like the one discussed earlier in this thread.
Sorry I'm on my cell phone in bed, if you haven't figured it out by tomorrow I'll write up some better directions when I'm at a full keyboard.

Thanks for the prompt response.
I haven't downgraded/upgraded the version..
Just tried by giving certificates in tlsOptions

one error was fixed
Next found that close socket is returning unset in client.js, so I modified client.js file to set closeSocket value with "socket" directly as shown below.

No idea what is happened inside, but the issue was resolved and getting authentication passed.
Can you suggest, is it a correct fix?.

Are you working on NVP or EOWeb application?
Sent from Yahoo Mail on Android
On Thu, Nov 8, 2018 at 2:20 AM<EMAIL_ADDRESS>wrote:
Thanks for the prompt response.
I haven't downgraded/upgraded the version..
Just tried by giving certificates in tlsOptions
one error was fixed
Next found that close socket is returning unset in client.js, so I modified client.js file to set closeSocket value with "socket" directly as shown below.
No idea what is happened inside, but the issue was resolved and getting authentication passed.
Can you suggest, is it a correct fix?.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

Yes,
On Sat, 10 Nov 2018 at 7:51 AM, eshovogiwah<EMAIL_ADDRESS>wrote:

Are you working on NVP or EOWeb application?
Sent from Yahoo Mail on Android
On Thu, Nov 8, 2018 at 2:20 AM<EMAIL_ADDRESS>wrote:
Thanks for the prompt response.
I haven't downgraded/upgraded the version..
Just tried by giving certificates in tlsOptions
one error was fixed
Next found that close socket is returning unset in client.js, so I
modified client.js file to set closeSocket value with "socket" directly as
shown below.
No idea what is happened inside, but the issue was resolved and getting
authentication passed.
Can you suggest, is it a correct fix?.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/joyent/node-ldapjs/issues/474#issuecomment-437491445,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Aqx1xAb6JqVsnAZXcezzE6b8xUwN3dt-ks5uterrgaJpZM4TPENR
.

Yes, nvp

Sorry I never replied to you.
I'm on my cell phone again, but I'll try my best to type this out.
In your ActiveDirectoryLdapAuth function I see you use fs read sync. While this is correct every time this function calls it'll do a synchronous read to the file system which isn't normally a good idea in Node. It's probably better to import this CA on startup of your application.
For the second part that should work,in newer version of Node. However I don't know your full code or the old ldapjs, so I can't gurentee 100% you won't have an issue. You also could just delete closeSocket and rename the event listener to socket.on, because you are just making a pointer to that.
Anyway long story short, looks like it'll work okay just not really what I'd normally do or recommend. You should probably have this app looked at / upgraded.
Never heard of NVP.

I tried everything I could to avoid upgrading and ultimately I had to. I followed Ezekiel's steps and he got us fixed. We have both NVP and EOWeb in our environment.
GE Network Viewer - NVPGE Electric Office Web - EOWeb
Thanks,Eshovo
Sent from Yahoo Mail on Android
On Fri, Nov 9, 2018 at 3:59 PM, Ezekiel<EMAIL_ADDRESS>wrote:
Sorry I never replied to you.
I'm on my cell phone again, but I'll try my best to type this out.
In your ActiveDirectoryLdapAuth function I see you use fs read sync. While this is correct every time this function calls it'll do a synchronous read to the file system which isn't normally a good idea in Node. It's probably better to import this CA on startup of your application.
For the second part that should work,in newer version of Node. However I don't know your full code or the old ldapjs, so I can't gurentee 100% you won't have an issue. You also could just delete closeSocket and rename the event listener to socket.on, because you are just making a pointer to that.
Anyway long story short, looks like it'll work okay just not really what I'd normally do or recommend. You should probably have this app looked at / upgraded.
Never heard of NVP.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

Wow that's interesting, does GE not update this software or is it just an older version? It seems strange for them to be using an outdated code base.

Mr. Tastypackets,
They like to move to the next version of the application which may not have the fix. I think you have to report it as either a bug or enhancement request to get it in the next release of the application.
Change is not easy to do. So I like to fix the version I have because of all the custom changes we have included and getting approvals to implement in prod.
For this user,
Take a look at Tastypacket's steps from April 16th 2018 above. If you are trying to use ldaps, get a separate machine, install NodeJS v8 from https://nodejs.org/en/
Past your dependencies from server\node_modules\ldapjs\package.json here.
Your goal would be to get nodejs v8 and ldapjs v1.0.2 (run npm install or npm update from cmd). Take notes of all your directories.
Run node -vRun npm -v

Thanks for your inputs .. I will try your input and let you know the result.

⚠ This issue has been locked due to age. If you have encountered a recent
problem that seems to be covered by this issue, please open a new issue.
Please include a minimal reproducible example
when opening a new issue.


Remove Indian Cities

Api is dead and unreachable
Thank you for taking the time to work on a Pull Request for this project!
[x ] Your submissions are formatted according to the guidelines in the contributing guide
[x ] Your additions are ordered alphabetically
[x ] Your submission has a useful description
[x ] The description does not end with punctuation
[x ] Each table column should be padded with one space on either side
[x ] You have searched the repository for any relevant issues or pull requests
[x ] Any category you are creating has the minimum requirement of 3 items
[x ] All changes have been squashed into a single commit

Thanks for the contribution @Sutar210599, but this has already been removed 😅


When I try to use Environment Menu python for both 2022 and 2019, visual studio crashes

This issue has been moved from a ticket on Developer Community.

[severity:I'm unable to use this version]
When I try to use Environment Menu python for both 2022 and 2019, visual studio crashes. It was working fine but I THINK (not sure) it tried to install Python 3.10 out of nowhere, and then everything stopped working. and doesn't work.

Original Comments
Feedback Bot on 3/29/2022, 03:06 AM:
(private comment, text removed)

Original Solutions
(no solutions)

I have a similar problem in VS 2022 Pro & Community versions. After installing Python 3.10, I open the Python Environment window and it crashes.

Sounds like a duplicate of #6944

Closing stale issue and in favor of #6944.


cast page-break values from iso8601

This fixes the error I was having locally where a page break value that was a date wasn't being automatically casted to a date. It's a bit hacky to reach into the __changeset__/0 from the primary table (not to mention that there's no guarantee that we're ordering by a column on the primary table), but I made it as optimistic as possible: the active branch of do_cast_type/2 should be hit only very rarely. We can expand that function in the future if we see an error popping up for datetimes etc

I added some more clauses for dates, times, datetimes, and naivedatetimes. We could probably add a clause for custom ecto types too but I'm not gonna be that ambitious on this PR just in case this change blows up prod


TorrentCreatorEventArgs -- OverallCompletion seems not showing the right progress?

TorrentCreatorEventArgs -- OverallCompletion seems not to show the right progress, it seems still to show the piece hash progress?

Is this happening with the 2.x release of MonoTorrent?

Is this happening with the 2.x release of MonoTorrent?

Yep

Ah, I think I see the issue. The mechanism used by monotorrent 2.x to hash data in parallel with reading from disk was to partition the list of files. When the partitioning strategy is used, it looks like incorrect values will be reported for the Overall progress.
It's probably not worth trying to fix this as the code was rewritten to take advantage of better threading primitives as part of the monotorrent 3.0 release series.
The two simplest options are:

I can patch 2.x to disabling partitioning. This will have some impact on performance, but will also make this emit correct information.
Could you update to a 3.x pre-release?

For me , I'll try the 3.x pre-release, it's up to you whether you would fix this issue in 2.x.
Thx!


vue 3/ vue next

What problem does this feature solve?
Implement vue 3 support - I know this is probably happening in the background but there does not seem to be an issue/milestone which people can follow and I though it would be helpful to create one.
I've been playing and converting my projects to vue next to understand and learn it and I'm very happy with everything, but the main stumbling block is debugging without devtools....
What does the proposed API look like?
Some support for vue next, even if just rudimentary...

Any news on this? I don't think it's meaningful to start new project on vue2 at this point, but the support in devtools is essential for any development.

Hi @mika76 and @thefoxie! To check out progress on the Vue 3 DevTools, I'd recommend checking out the next branch of the repo!

Thanks @bencodezen - I actually created this issue before the first preview release with the hope of getting some info. Very little info coming out about devtools. I'm sure it's being worked on but it's be nice to see some movement.

I was about to use vue3 for a new upcoming project but then I realised current version of devtool doen't detect vue3 app? Any chance I use newer version of devtools with vue3 support?

Hey @avimehenwal! You can actually use a local version of the DevTools by checking out the next branch. Let me know if you have troubles getting setup!

fwiw: v6.0.0-beta.2 includes a Firefox XPI working fine on my end :clap: – as for Chrome the instructions mentioned by @bencodezen work great :-)

Note to use v6.0.0-beta.2 you must add<EMAIL_ADDRESS>as a dependency as documented in release v6.0.0-beta.1. Spinning my wheels for a bit on that one. :)

Note that with v6.0.0-beta.3<EMAIL_ADDRESS>seems to work for me in firefox while the rc.1 causes it to not be able to detect the components. YMMV.

Note that with v6.0.0-beta.3<EMAIL_ADDRESS>seems to work for me in firefox while the rc.1 causes it to not be able to detect the components. YMMV.

git clone git@github.com:vuejs/vue-devtools.git
cd vue-devtools
git checkout next
yarn
yarn build:watch
and then throw error:
(node:11088) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
$ lerna run build --scope @vue-devtools/app-backend* --scope @vue/devtools-api && lerna run build:watch --stream --no-sort --concurrency 99
lerna notice cli v3.22.1
lerna info run Ran npm script 'build' in '@vue/devtools-api' in 6.6s:
$ rimraf lib && yarn build:esm && yarn build:cjs
$ tsc --module es2015 --outDir lib/esm
$ tsc --module commonjs --outDir lib/cjs -d
lerna ERR! yarn run build exited 1 in '@vue-devtools/app-backend-api'
lerna ERR! yarn run build stdout:
$ rimraf lib && yarn ts
$ tsc -d -outDir lib
src/api.ts(1,41): error TS2307: Cannot find module '@vue-devtools/shared-utils' or its corresponding type declarations.
src/backend-context.ts(1,24): error TS2307: Cannot find module '@vue-devtools/shared-utils' or its corresponding type declarations.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

lerna ERR! yarn run build stderr:
(node:28420) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.                                                                                f
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:23856) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Bu fer.from() methods instead.ffer.from() methods instead.                                                                                f
(Use `node --trace-deprecation ...` to show where the warning was created)                                                                                fer.from() methods instead.
error Command failed with exit code 2.
error Command failed with exit code 1.

To debug Vue 3, use the version 6 beta of the devtools. Thanks!


ComfyUI Failed Installation

I was getting a "from diffusers.loaders.autoencoder import FromOriginalVAEMixin" compilation error after pulling via ComfyUI custom node manager, so I followed the README instructions and git cloned the repo. When following the next steps, I ran install.py:
 Installing collected packages: torch
   Attempting uninstall: torch
     Found existing installation: torch 2.3.1+cu118
     Uninstalling torch-2.3.1+cu118:
       Successfully uninstalled torch-2.3.1+cu118
[!]   WARNING: The scripts convert-caffe2-to-onnx.exe, convert-onnx-to-caffe2.exe and torchrun.exe are installed in 'C:\ComfyUI_windows_portable\python_embeded\Scripts' which is not on PATH.
[!]   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
[!] ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
[!] streamdiffusion 0.1.1 requires diffusers==0.24.0, but you have diffusers 0.28.2 which is incompatible.
[!] torchaudio 2.3.1 requires torch==2.3.1, but you have torch 2.3.0 which is incompatible.
 Successfully installed torch-2.3.0
 Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
 Requirement already satisfied: kornia>=0.6.9 in c:\comfyui_windows_portable\python_embeded\lib\site-packages (0.7.2)
 Requirement already satisfied: xformers>=0.0.20 in c:\comfyui_windows_portable\python_embeded\lib\site-packages (0.0.26.post1)
PS C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\EasyAnimate>

It should come as no surprise that run_nvidia_gpu.bat will now result in a "Torch not compiled with CUDA" error and a startup failed.
Any ideas?

Your PC has already installed  torch-2.3.1+cu118 (cuda-11.8 compiled), but running install.py to install torch-2.3.0 (not compiled CUDA) that replaced torch-2.3.1+cu118.

torch-2.3.1+cu118 is OK for torch>=2.2.0 in requirement.txt
you could try to roll-back install your  torch-2.3.1+cu118, and run comfyui without running install.py.

I forgot to hit "Refresh" facepalm
Thank you for the support!


wiremock version 2.11 is incompatible with java 11

With 2.11 wire mock tests hanging and mvn can't execute in thread mode.
Update to 2.27.2(newest version)

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[x] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?
Build cs with tests and multithreading in my env.

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_multiplication_x: centos8 :heavy_check_mark: debian. SL-JID 323

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 330

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

No unit tests failures observed in packaging or Travis's job#1


Some Application not able to start up properly

Hello,
I am using shinyproxy operator on GKE cluster. The version is 2.0.0.
I reached the status that I have some apps (example app) working. However, I have an application that works in the docker swarm cluster but does not work on this GKE cluster.
Here are the logs from the Shinyprox-operator pod
92de-5d8e24b3bfa1 specId=shiny_admin_dashboard] Container unresponsive, trying again (18/20): http://<IP_ADDRESS>:3838
2024-04-05 20:05:40.606  WARN 1 --- [pool-3-thread-6] e.o.shinyproxy.ShinyProxyTestStrategy<EMAIL_ADDRESS>proxyId=4b60f433-64c7-4188-92de-5d8e24b3bfa1 specId=shiny_admin_dashboard] Container unresponsive, trying again (19/20): http://<IP_ADDRESS>:3838
2024-04-05 20:05:40.680  WARN 1 --- [pool-3-thread-6] e.o.containerproxy.service.ProxyService  : [user=xxxxxx proxyId=4b60f433-64c7-4188-92de-5d8e24b3bfa1 specId=shiny_admin_dashboard] Container did not respond in time

The logs from the application pod show the application starts properly.
The browser shows the following

Could someone please show me how to troubleshoot this error?
Thanks,
Robin

found the settings containerWaitTime.  In operator the parameters follows Camelcase other than Snakecase.


fix: wrong vsis3 path logged

Motivation
The vsis3 path is logged in order to facilitate the user to investigate an issue on the output tile by running gdal directly with the path copied from logs.
The path is currently logged wrongly: when the script is ran on Argo Workflows, the standardised path is already a S3 path (as target is S3 scratch bucket).
The result with the current implementation gives a path like /vsis3/linz-workflows-scratch/2024-02/14-test-northland-2018-2020-dsm-1mtfvvz/test-northland-2018-2020-dsm-1mtfvvz-standardise-validate-1780775412/s3:/linz-workflows-scratch/2024-02/14-test-northland-2018-2020-dsm-1mtfvvz/flat/AX27_10000_0105.tiff
Modification
Change the implementation to use get_vfs_path which replaces s3:// by /vsis3/.
Checklist

[x] Tests updated
[x] Docs updated
[x] Issue linked in Title

The code LGTM, but is the example in the PR description right? There's a "s3:" in there.

Yes that was the issue

unrelated, but how well does this path logic work in a windows environment? I am assuming we have just out of scopped windows though.

This path logic is triggered only when using Argo, which has a target path (where standardised files are saved) of a S3 bucket.
If the path is a local path, we don't modify it


Props type mismatch

The lib offers a compare prop but it is not defined in types. Due to this it throws a type error while using in typescript.

Hi @zubair1103, this typescript definition is created by community. It was possible before as compare prop was added. I am planing to take a time to this library but not right now. If you have time you can contribute patch to the repo with types, or some another solution. Thanks


Possible bug in d3 v3 --> v4 changelog?

In the section on d3-chord (link: https://github.com/d3/d3/blob/master/CHANGES.md#chords-d3-chord), it states:

For consistency with arc.padAngle, chord.padding has also been renamed to ribbon.padAngle.

Do you mean that "chord.padding" has been renamed to "chord.padAngle" instead?

No; d3.svg.chord was replaced with d3.ribbon, so chord.padding is replaced with ribbon.padAngle.


haxelib post submit version

After submitting the "format 3.2.0" library, I got the following message:
Version 3.2.0-alpha (id#3173) added

The alpha should not be there (most likely a default value that is not reset)

I'm only getting to know haxelib code base, but I think this is caused by 6d41c431e4d3056d4a8269c4aedc46acb408cafe - the @:nullable metadata is not supported by standard Haxe RecordMacros, so it thinks that it's not nullable and probably is converted to 0 and thus first constructor of the Preview enum. I guess that RecordMacros now supports SNull<SEnum<...>>, so I fixed it in 4548df2d601547d8c97aa4eb63b822d0a2855d21 and it looks good now. @back2dos please confirm/review

This is probably related: https://github.com/HaxeFoundation/haxe/commit/784efa790c11d17d604955b5221c530ced2a8804


Simulated (GAZEBO) vehicle becomes unstable when trying to control a gimbal via the ActuatorControls message

The bug
Initially raised this in the sitl repo but was advised to also raise here. Simulated MAV (GAZEBO) becomes unstable when trying to control a gimbal via the ActuatorControls message (in Mavros, which in turn publishes SET_ACTUATOR_CONTROL_TARGET on mavlink). This has also been reported for LED outputs (not sure if this was also simulated or not).
To Reproduce
Steps to reproduce the behavior:

roslaunch px4 mavros_posix_sitl.launch vehicle:=typhoon_h480
Arm the vehicle and take off
Publish /mavros/actuator_control ROS message at 10hz (group_mix=2)
The UAV becomes unstable (particularly if non zero setpoints are sent to the controls field of the ActuatorControls topic).

If this process is repeated with a lower publish rate (1 hz works) the vehicle remains stable.
Drone:

Simulated typhoon h480

It seems that all offboard setpoints are ignored when the actuator setpoints are set:
https://github.com/PX4/Firmware/blob/f9324fb76a4af1e2cfe8813e1a7af5d65c2b1ed1/src/modules/mavlink/mavlink_receiver.cpp#L1000-L1011

Thanks for the report @jannsta1. I think the Typhoon H480 gimbal in SITL needs some care overall. It's not really working properly in general.

OK thanks @julianoes - if these updates are unlikely to be worked on for a while I could have a stab at them? (more than happy to leave it to the professionals if its already in the pipeline! :)). If help desired would you be able to outline any other required updates?
The only other thing I've noticed while making use of the model is that the yaw control seems a bit sluggish and/or if you fly outward in a direction from the origin and then turn around and fly back to the starting point there appears to be a large steady state error. Increasing the PID gains seems to help but I also wonder if using previous imu state for the yaw setpoint could result in some drift over time.
Also, from @Jaeyoung-Lim's comment above. it seems like the symptoms described by this issue are not actually contained in the gimbal controller gazebo plugin but are associated with the mavlink receiver. If this is the case then the repair might be a bit beyond me.

if these updates are unlikely to be worked on for a while I could have a stab at them?

I don't think anyone is currently looking into this :disappointed:, so feel free.

@jannsta1 Have your SITL simulation ever crashed when sending actuator setpoints? Sending actuator setpoints over mavros seem to make the SITL crash

@Jaeyoung-Lim - it was just the vehicle that was crashing... not gazebo itself. Although I was killing the simulation fairly soon after the loss of control.
Let me know if you want me to try anything in particular (e.g. send a particular message on a control group at a certain rate).
I wonder if you have any suggestions on fixing the central issue? assuming you have identified the root cause (which seems reasonable), a couple of ideas spring to mind:

only set the ignore_thrust / body rate... etc flags when the control group is 0, but I guess its not guaranteed that the main motors would be using control group zero.
For gimbal control it might be better to use the MAV_CMD_DO_MOUNT_CONTROL message anyway so I could try to create a new mavros plugin for this.

@jannsta1
I have tried disabling the ignore flag when the group_mlx=2 However, this still results in erratic behavior of the setpoints.
For context: I am currently sending setpoint body_rates and thrust commands that make the drone follow a trajectory, while sending actuator_control topics with group_mix=2 to control the gimbal

@Jaeyoung-Lim - could it be that another module is also publishing to the same actuator_controls topic at the same time? Having a look at the topic factor graph my interpretation is that the mavlink module would subscribe to actuator_controls_XX topics rather than publish to them?

Indeed, there is a process that is resetting the gimbal control values. I made a small fix so that at least I can control the gimbal when I stream actuator setpoints at 100Hz.
The problems that I can see is two fold.

The gimbal control topics are reset to 0 for whatever reason resulting in oscillatory behavior for low rate setpoints

The actuator outputs in SITL are as the following.

Sending actuator_setpoints(group_mix=2) at 10Hz

Sending actuator_setpoints(group_mix=2) at 100Hz

You can see that the values are reset to 0 when the actuator_setpoints (group_mix=2) are sent at a lower rate

The other setpoints: in my case the body_rate and thrust setpoints get disturbed by the actuator setpoints.

Sending actuator_setpoints(group_mix=2) at 100Hz

Sending actuator_setpoints(group_mix=2) at 100Hz

Suppressing the outputs from output_rc.cpp by commenting out the following fixes the problem.
https://github.com/PX4/Firmware/blob/5ba56c8c2cb7b48b6987ac06a503fd8841eb9925/src/modules/vmount/output_rc.cpp#L74-L84
log: https://review.px4.io/plot_app?log=b47fbcf5-babb-402e-9fdd-017e7b09c179

The pink line shows the gimbal setpoint being set continously and not being set to zero as shown in previous examples.

Thanks @Jaeyoung-Lim!
Just a quick point, I was stopping vmount (just by appending vmount stop to the file 6011_typhoon_h480.post) maybe this will prevent having to do what otherwise looks like a complicated fix?

@jannsta1 Thanks. You are right. Stopping vmount solved the problem of resetting the values to 0.

@Jaeyoung-Lim A bit off topic but have you had other problems with running the typhoon model? As of this week, if I donwload the latest repositories and try to arm/launch the typhoon_h480 model I get messages like  Preflight Fail: Compass #0 uncalibrated. All other tested models (iris_xx) seem to work fine and I've tried rolling back to different commits of sitl_gazebo and Firmware but don't seem to be able to get a working version. If needs be I will raise a separate issue for it / look into root cause but since the sitl_gazebo issues section doesn't seem to be monitored very actively it would be great to get a sanity check from someone!

@jannsta1 have you tried make clean in order to make sure the param file is deleted?

Thanks for the idea, I probably wasn't doing make clean as often as I should. Just been through the procedure of cloning the firmware again and now if I start gazebo with roslaunch px4 mavros_posix_sitl.launch vehicle:=typhoon_h480 verbose:=true I can takeoff with pxh> commander takeoff which I don't think I could do before. Also, now when I arm and send setpoints for a first time the vehicle doesn't take off. On a second attempt I get a different error - Critical navigation failure! Check sensor calibration. I guess there is something wrong with my ros node that generates setpoints but strange that this has worked fine up until now and still seems to work OK with other vehicles.

Hang on, if you're using roslaunch, you need to do this to clean the params:
rm ~/.ros/eeprom/parameters

This is a known issue and really frustrating actually, see: https://github.com/PX4/Firmware/issues/10686

Noted! But despite this I find that I can't control the typhoon_h480 model after the following Firmware commit: https://github.com/PX4/Firmware/commit/43e3fc707de823faa19b16091e5ee4a900e3c446.
The previous commit https://github.com/PX4/Firmware/commit/4a4d323a970225ec27e9cd4a409e7a287ad57ff6 and prior work fine.
I wonder if could confirm if you observe the same @julianoes or @Jaeyoung-Lim?

@jannsta Could you post this on discuss or create a separate issue for being unable to fly? It is unrelated to the issue that should be discussed here.

logged bug report https://github.com/PX4/Firmware/issues/12221 to document what I think might be a confounding issue.

@jannsta1 The MountControl plugin in mavros is now merged! You can control the gimbal continuosly while controlling the vehicle offboard.

Nice one! Will be next week before I try to integrate this but this is certainly enough to close this issue.


Add vellum

Stencil is now built on top of Vellum.
Fixed Alert component to work with Vellum
Cleaned up styles in fixtures because we are using Vellum styles now

Status: Opened for Review
Reviewers: @kpeatt @jeffkamo @ry5n @avelinet
JIRA: ?

Updated button component from vellum button width fix

@cole-sanderson Made some updates here but this is looking good. GtM from me!

@kpeatt looks good.

:+1:

NOTE: PR-110 dependent on this branch. Please don't remove it untill Tabs branch is merged to master.


Splatfacto a pointcloud for initialization not from COLMAP

Describe the bug
I am trying to train using splatfacto method and use pointcloud exported from metashape for initialization. I've added to the transform.json  "ply_file_path" and exported pointcloud as .ply file.  I have all the time the same error: RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
I have tried in different ways to export/convert .ply file. Using for convertion cloudcompare and pdal library.
When I use ns-process-data images where the pointclod is made by colmap, there is no problem. The header of .ply from colmap and .ply file exported from metashape (after convertion) looks the same. When I copy only the pointcloud made by colmap and train dataset from metashepe it works (of course the pointcloud has different coord that cameras).
When using pointcloud in .ply format made not from COLMAP (tested metashape, cloudcompare, pdal, notepad) after 0-100 steps error occurs:
Traceback (most recent call last):
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\admin\miniconda3\envs\nerfstudio\Scripts\ns-train.exe\__main__.py", line 7, in <module>
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\scripts\train.py", line 262, in entrypoint
    main(
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\scripts\train.py", line 247, in main
    launch(
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\scripts\train.py", line 189, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\scripts\train.py", line 100, in train_loop
    trainer.train()
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\engine\trainer.py", line 250, in train
    loss, loss_dict, metrics_dict = self.train_iteration(step)
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\utils\profiler.py", line 112, in inner
    out = func(*args, **kwargs)
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\engine\trainer.py", line 473, in train_iteration
    self.grad_scaler.scale(loss).backward()  # type: ignore
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\torch\_tensor.py", line 492, in backward
    torch.autograd.backward(
  File "C:\Users\admin\miniconda3\envs\nerfstudio\lib\site-packages\torch\autograd\__init__.py", line 251, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Two point cloud are attached: data.zip

yes,  initializing gaussians randomly works normally.
I paused training and looks like coordinates of the point cloud are swapped, rotated and scaled. I wonder what matrix parameters and where to use it to rotate pointcloud and why transform.json has rotated cameras?

transform.json can sometimes automatically apply a dataparser transform to shift and scale the poses, which is saved as 'applied_transform' in the json. Maybe if this pose is not identity, you might need to transform the points by the same matrix? I'm not too familiar with the dataparser transform code though, you might want to dig into the code around this, starting here

thank you, unfortunately in tested dataset in transforms.json there is no applied_transform. Looks like the problem with the pointcloud initialization is due to the coordinates not the format/exporter.

Hi! same issue here. I tried using colmap without initial pointcloud and it worked well. However, when using either polycam or spectacularAI generated rgb and depth, this issue will appear. Any solutions?

Hi!
I face the same issue but with images from a known pointcloud.
With random initialization, the train script fails with this error.

I do not remember seeing this particular error, but the latest released Spectacular AI SDK does not yet produce the expected sparse_pc.ply file. This will be fixed in the next SDK release later this week. As a work-around, it is possible to use the latest development version as described here https://github.com/SpectacularAI/sdk-examples/issues/126#issuecomment-1934172987
It may also be relevant for this issue that when Nerfstudio (after promting the user) tries to create sparse_pc.ply from a colmap/ folder, this operation does not look correct. The resulting point cloud has a coordinate system flip (at least the Z-axis was flipped). The incorrect point cloud often kind-of works too, but the splatting converges a lot slower.
If we simply write the same point cloud that we would have written as .csv/.txt to mimic COLMAP output, but now as ASCII PLY, the Splatfacto method seems to be working fine (see here).
We also have a separate repo for tools that can convert between a few different popular point cloud formats (including COLMAP txt -> PLY), which may also be helpful debugging this https://github.com/SpectacularAI/point-cloud-tools.

Hi. I had the same error but in my case ply headers were different: double and uchar from Metashape, and float and uint8 from Colmap. I simply changed the header of the metashape ply and it worked.

Same error here, and it persists when fixing the types in the PLY header.

Improving compatibility with other SfM softwares is not an easy work. Different softwares maintain different convention of coordinates, if the poses feed to nerfstudio are in different coordinates than the point cloud, unexpected error would occur.

I am checking this issue at the moment because I want to use the pointcloud from metashape and my impression is, that due to unknown transformations, sometimes there are cameras with no points in view. This then could cause the crash to happen.
I have switched the axis of the ply file in all possible combinations and was not able to get the correct orientation. However, in two combinations, the system didn't crash and continued training with wrong starting points (like in random initialization)

Hello, in metashape, first of all, you need align up you scene, and place all of it above z=0 (very important to avoid training crashes)!
Then, export cameras.xml and tie cloud or dense cloud.
Run nerfstudio dataset conversion from metashape.
Before training, you need to flip ply file.
I'm using this python/open3D :
import open3d as o3d import numpy as np cloud = o3d.io.read_point_cloud("spc.ply") R1 = cloud.get_rotation_matrix_from_xyz((-np.pi / -2, 0, 0)) cloud.rotate(R1, center=(0, 0, 0)) R2 = cloud.get_rotation_matrix_from_xyz((0, 0, np.pi / 2)) cloud.rotate(R2, center=(0, 0, 0))
Then add this line to transforms.json before "camera_model" :
"ply_file_path": "spc_fliped.ply", "camera_model": "OPENCV",
You can toogle visibility of initial cloud of nerfstudio webUI 😉
Wish you the best renders !

What do you mean by 'align up the scene'?
Regarding the python code to update the ply file, I think that this should get integrated into the nerfstudio-metashape importer. Maybe also the alignment can be fixed this way? Ideally, the nerfstudio importer would provide an option to define a metashape-ply-file and would also convert and move it to the bundle folder.

In fact, I just want to clarify that after importing and aligning the images, it's important to position the object or scene above the horizon line and logically in its natural orientation. I've noticed that nerfstudio often crashes when camera orientations are pointed towards an object below the horizon line (Z=0).
I believe this 'positioning 'step should remain manual in order to have a coherent scene in Metashape. However, if we start adjusting the position of the point cloud in converter with Open3D, we will inevitably have to adjust the cameras as well~~
On the other hand, it would be wise to offer the option to export from Metashape, in addition to the cameras.xml file, the point cloud (sparse/tie point) .ply file, to be processed (in 'ns-process-data metashape --data...' by making this corrective rotation in space, with the python script above for exemple.
I obtain much better results with metashape tie point cloud than with a nerfstudio random initialization. However, The dense point cloud often provides too much information, resulting in a visually less appealing outcome and much heavier splat scenes. For me, the advantage of  gaussian splatting is to have light and optimized scenes and still offer a photorealistic rendering on low-end devices.

Thank you very much for this help! I will contribute this to the metashape processing tool so everybody can simply take a ply-export from metashape

@pyjacques this code is now part of nerfstudio. You can simply add the optional parameter - -ply to ns-process-data metashape...

@simonbethke Whaoo, thank you for implementing this into nerfstudio ! I'm Happy to contribute too 😉

closing per https://github.com/nerfstudio-project/nerfstudio/pull/3122


problems using imagesloaded with browserify and jquery

Suppose you have a script with an isolated jquery (v2.1.3) that you require with browserify, and the page has another jquery (v.1.4.4), then if you use imagesloaded in the browserify script you get an error: TypeError: undefined is not a function, because imagesloaded is getting $ as the jquery 1.4.4 and not the required with browserify, which it should.

I'm afraid there is not an easy solution here. imagesLoaded will work with jQuery 1.7+. If you are using a duplicate jQuery v1.4.4, which is four years old, then it will break. Moreover, $.fn.imagesLoaded will not have promises, which is bad. So I'd rather imagesLoaded break, then try to degrade silently and not alert you that promises will be broken.
I would try to resolve that duplicate old jQuery, rather than catching this edge case with imagesLoaded.

@desandro: isn't the root problem that this library looks at the window's jquery object as opposed to either requiring it directly or taking it in as a parameter?  I'm just resolving this issue for a personal project now and those are the two solutions I'm aware of.

I need to update docs with example of using browserify + jquery + imagesLoaded. Similar to Masonry http://masonry.desandro.com/appendix.html#browserify

Ah - that's another solution.

I've added docs on Browserify + imagesLoaded http://imagesloaded.desandro.com/#browserify
imagesLoaded v3.2.0 adds the .makeJQueryPlugin() utility method.
var $ = require('jquery');
var imagesLoaded = require('imagesloaded');

// provide jQuery argument
imagesLoaded.makeJQueryPlugin( $ );
// now use .imagesLoaded() jQuery plugin
$('#container').imagesLoaded( function() {...});

I'm closing this issue as resolved. If you run into an issue with imagesLoaded, Browserify or makeJQueryPlugin, please open a new issue.


PMM-7671 - Fix failing

https://pmm.cd.percona.com/blue/organizations/jenkins/pmm2-ui-tests/detail/pmm2-ui-tests/8199/tests/ Fixed


feat: use sheet for washboard settings

https://github.com/wasmCloud/wasmCloud/assets/4237280/8ad57040-70bf-4b3b-b970-93634319fd70

Love it 😗🤌


Add InterfaceOperStatus deviation

This pull adds a deviation InterfaceOperStatus that is related to devices that generate Interface_OperStatus_DOWN instead of Interface_OperStatus_LOWER_LAYER_DOWN for an aggregated link.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 72.664%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1462

Relevant Lines:
2012

💛  - Coveralls

/gcbrun

/gcbrun


Security: rate-limit outbound messages, Credit: Equilibrium

TODO before starting
Find good rate-limits, and work out if the rates should be fixed or dynamic.
Motivation
Zebra's message handling code is complex, so it might be possible for it to get stuck in a request-response loop. To limit the impact of these loops, we should rate-limit Zebra's outbound messages to each peer.
A similar issue was reported by Niklas Long of Equilibrium.
Analysis
There are some tricky engineering tradeoffs here:

if the limit is too low, then Zebra is very slow to sync
if the limit is too high, we're not protected from denial of service
the best limit can be different for different message types, depending on how large or important they are

So we need to find good rate-limits, and work out if the rates should be fixed or dynamic.
Suggested Solution

[ ] Analyse the maximum message rate per peer that Zebra reaches in optimal syncing conditions (Testnet, local network)
[ ] Calculate a reasonable per-peer rate limit that won't slow down syncing too much
[ ] Add a rate limit layer to each Client's outbound message queue

Alternatives
We could restrict rate-limiting to identical messages. We'd need to store message contents, hashes, or types.
We could also rate limit inbound messages - they're currently limited via load-shedding.
We could make rate limits dynamic.
We could use different rate-limits for different message types, or for requests and responses.

Before we start this ticket, we need to find good rate-limits, and work out if the rates should be fixed or dynamic.

Network behaviour is good enough for now.


Add Support for Host-Image-Name

Please extend support for hosts as we do with images.

@ericgomes56 Postee is now a generic message routing module.
Once there will be a webhook support to send host scan results, then Postee will support it automaticall.
Closing the issue for now.


New version: NFLData v1.1.2

Registering package: NFLData
Repository: https://github.com/nflverse/NFLData.jl
Created by: @john-b-edwards
Version: v1.1.2
Commit: 3c24f5ee5a5587a75eafaddb2fa23a10bac1c35c
Reviewed by: @john-b-edwards
Reference: https://github.com/nflverse/NFLData.jl/commit/3c24f5ee5a5587a75eafaddb2fa23a10bac1c35c#commitcomment-147045024
Description: Read up-to-date NFL data into Julia as a DataFrame.

[merge approved]


Template extractor cannot export release pipeline

Hello, i get following error when exporting
Object reference not set to an instance of an object. at VstsDemoBuilder.Services.ExtractorService.<>c__DisplayClass30_2.b__2(KeyValuePair2 x) in D:\a\1\s\src\VstsDemoBuilder\Services\ExtractorService.cs:line 1250 at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext() at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source) at VstsDemoBuilder.Services.ExtractorService.GeneralizingGetReleaseDefinitions(ProjectConfigurations appConfig) in D:\a\1\s\src\VstsDemoBuilder\Services\ExtractorService.cs:line 1250
What can be the root cause?

Hi @zhjuve Thank you for reaching us. We will look into this.

This could be related to unexpected Json format. We have generalized the logic to export different kinds pipeline definitions. This was an unexpected and we couldn't reproduce. Will make a note of this and closing the issue.


Start hero section on website

The hero section on the website is starting to look sort of like what it should look like based on the spec. I fully fleshed out the color/font styling from zeplin, but did not do layout completely (if the graphics change we may want to change how this bit is laid out)
I also hacked IBM plex fonts in by copying some styles from the legacy site. It looks like we're missing some of the weights, so we'll have to add those and we should clean up my hack.

"Fixed" sounds like something is broken. Maybe "Constant size" ?


Code and Documentation Optimization

Added generics to container-type data structures.
Used final keyword for enum class variables.
Corrected spelling mistakes.
Removed unnecessary semicolons in Groovy scripts.
Optimized unused variables in Lua script 'for' loops to '_'.
Marked fields or methods as @VisibleForTesting to indicate they should only be called by test methods.
Annotated forcefully cast Java code with @SuppressWarnings("unchecked").
Extracted common methods to a utility class to reduce redundant code.
Implemented other code style enhancements.

@moqimoqidea Conflicting


Creating Rawtx and signing from multiple inputs

I am working on creating a script that will scrape a few blockchain websites such as blockchain.info, blockr.io, etc.. From scraping I am taking the transaction ID and the Vout to build a transaction that I can then sign. The code below code is intended to get the inputs list with the transaction id and Vout and convert it into a bitcoin-php transaction factory. Once converted it will be signed with one private key and then displayed to the user in order to sign with a final key.
With my current code I am getting an error saying "Catchable fatal error: Argument 1 passed to BitWasp\Bitcoin\Transaction\TransactionInput::__construct() must be an instance of BitWasp\Bitcoin\Transaction\OutPointInterface, string given, called in \php\bitcoin.php on line 80 and defined in \php\bitwasp\src\Transaction\TransactionInput.php on line 38"
Let me know if something is not clear, and thanks for any help. Below is most of the code I am using excluding the website scraping
...
function createinputlist($pubkeyone,$pubkeytwo,$privatekey, $inputlist){
$private = PrivateKeyFactory::fromWif($privatekey);
$redeemScript = ScriptFactory::scriptPubKey()->multisig(2, [PublicKeyFactory::fromHex($pubkeyone),PublicKeyFactory::fromHex($pubkeytwo),$private->getPublicKey()]);
$p2sh = ScriptFactory::create()->push($redeemScript->getBuffer())->getScript();
$inputs = array();
foreach($inputlist as $transaction){
$inputs[] = new TransactionInput($transaction['txid'], $transaction['vout'], $p2sh);
}
return $inputs;
}
function createrawtx($orderid, $escrowaddress, $escrowamount, $refundaddress, $refundamount){
$ecAdapter = Bitcoin::getEcAdapter();
require_once("market.php");
$depositinfo = getdepositinfo($orderid);
$pk1 = PrivateKeyFactory::fromWif($depositinfo['privatekey']);
$rs = new Script(Buffer::hex($depositinfo['redeemscript']));
$redeemScript = new \BitWasp\Bitcoin\Script\P2shScript($rs);
$os = $redeemScript->getOutputScript();
// One party wants to spend funds. He creates a transaction spending the funding tx to his address.
$spendTx = TransactionFactory::build()
->input(createinputlist($depositinfo['publickeyone'], $depositinfo['publickeytwo'], $depositinfo['privatekey'], getinputs($depositinfo['multisig'])))
->payToAddress($escrowamount, AddressFactory::fromString($escrowaddress))
->payToAddress($refundamount, AddressFactory::fromString($refundaddress))
->get();
$signer = TransactionFactory::sign($spendTx);
$signer
->sign(0, $pk1, $os, $redeemScript);
$signed = $signer->get();
return $signed->getHex();
}
...

$inputs[] = new TransactionInput($transaction['txid'], $transaction['vout'], $p2sh);
Check the source code for TransactionInput. It's a type mismatch error: you're passing a string instead of an OutPoint.
The examples might be a little out of date, but always refer to the code for what you should be passing. We make heavy use of type-hinting, which is meant to help you :)
I get quite a lot of questions about transaction signing, but it definitely works!

Yes the examples a bit out dated. I have a few working code examples that I could give you to update or add to your examples if you want.
I just looked it over, and it sounds like that line should be
$inputs[] = new TransactionInput(new OutPoint($transaction['txid'], $transaction['vout']), $p2sh);
Does that look right now?

Looks rght!
One thing, the txin.script, you don't have to put the PUSH in there. Just use the signer, which will regenerate the scriptSig for you.

okay, not sure that worked correctly. It needed to be Buffer::hex($transaction['txid']) but that seems to have fixed that problem.
Now I am getting an issue with the transaction builder. Is there a good example that shows building a transaction from multiple inputs?

Either loop across $inputs and call $builder->input(...), or use $builder->inputs() on your array.
Since you have many inputs, they all have to be signed. If you have two keys, call it twice for that input with different keys. It should handle partially signed transactions well enough.

My best advise would be to rely on the code, not the examples. A good IDE will serve you well!

I dont understand what you mean.
IDE related, any one you can recommend? Right now I vary between using Brackets and Visual Studio
To make it easier,  below is the code I have for looping through the transaction,
foreach($inputlist as $transaction){
$inputs[] = new TransactionInput(new OutPoint(Buffer::hex($transaction['txid']), $transaction['vout']), $p2sh);
}
After that loop, the $inputs[] is sent to the transactionfactory builder as seen below
$spendTx = TransactionFactory::build()
->input($inputs[])
->payToAddress($escrowamount, AddressFactory::fromString($escrowaddress))
->payToAddress($refundamount, AddressFactory::fromString($refundaddress))
->get();

I've really enjoyed moving to PhpStorm, but I haven't used those two. You can get 30-day evaluations, just wipe the appdata folder when it's over..
Just change the second line to this:
  ->inputs($inputs)
Hrm, you're making an escrow site? Don't keep private keys on the server first of all.. Second, you don't have to sign that redeem transaction, your customers do (unless there's a dispute). Anyways.. You're about to enter a rabbit hole, so, good luck keeping this secure!

I already made an escrow site, I guess you did some background checking into me ;) , tho I am doing improvements still, this is actually for a different project.
Yes the private keys are kept on a server encrypted. They do not hold any funds so to speak so I consider the risk minimal. The private key is used in a 2 of 3 where 1 person holds 2 keys and we hold one key. So even if we were hacked the best the hacker would get would be 1 key which is useless.
I was about to comment and tell you it ddn't work when I noticed the S in inputs. What a big difference one letter can make.
Well that worked, thanks for your help

The BitWasp project was initially a FOSS multisig marketplace, but ended up focusing on the library instead. Might revisit things when it supports a proper SPV client.

Yes, I am familar with the background. I was quite disappointed when it seemed like it died a while back and was part of the reason why I made my site.
It was quite hard making it due to the lack of native php libraries for bitcoin. I would say this is one of the only ones I am aware of. There are a few other smaller ones but most do not support everything. For example one might only create bitcoin addresses while another might only generate multi-signature address.

It worked well enough for ordering/payments, but lacked a whole heap of stuff you'd expect on a site (search, decent menu/layout, etc). There wasn't a lot of interest from developers, so once the bitcoin stuff was finished, it was hard to keep maintaining.
A lot of time was wasted trying to consolidate various libraries.. But eventually lead to this thing :)

Yeah, eventually id like to expand and build a marketplace similar to bitmit back in the good ole days.


Update it.po

Thanks, maybe try also @ https://github.com/berkeleybross/easy-gesture, its an actively maintained fork, 72 Commits ahead.


Reload tera templates in debug mode

Reload tera templates would bring a lot of convenience in debug mode.

Got the following error when running commands cargo install --debug --path . and zine build in the demo directory.
Error: Template 'article.jinja' not found

Yes, debug mode is mainly for the maintainer to verify the templates UI quickly without restarting zine serve in the development stage. Using it in other scenarios would unsurprisingly cause panic.


Document new endpoint, update ownerapi_endpoints.json & add notice about removed endpoint.

Changes:

Document set_cabin_overheat_protection endpoint, referenced in issue #701
Update ownerapi_endpoints.json to v4.19.0-1639 on par with the March 20th, 2023 App Update
Add notice to set_vehicle_name endpoint being removed from the endpoints file.

(Alternatively the docs for this can be removed, but it might return, up to the repo owner)

Notes:

Some new charging related endpoints which return invalid_command as of now.
Removal of one endpoint (mentioned above).
New energy endpoints.
More BFF changes.

Thanks!


[BUG] Delete ngram fw-bw-ppl

https://github.com/thu-coai/cotk/blob/master/cotk/metric/ngram_perplexity.py#L30
there is no such an average proposed and the meaning of the average is suspicious

Done


Server URL for TFS 2018

Server URL for TFS 2018 is https://{your_server}/
Agent fails to connect on
https://{your_server}/tfs
but connects fine without 'tfs' bit.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 23328b13-f079-ad21-5d7a-6c640f25391c
Version Independent ID: 4a39889d-fa00-ae2b-aa43-f358631b7b1a
Content: Deploy a build and release agent on Linux - Azure Pipelines
Content Source: docs/pipelines/agents/v2-linux.md
Product: devops
Technology: devops-cicd
GitHub Login: @andyjlewis
Microsoft Alias: alewis

In the "Authenticate with a personal access token (PAT)" section, which gets included from "_shared/v2/prepare-permissions.md", we should:

Say "Azure DevOps Server" instead of "Team Foundation Server" when moniker >= Server 2019.
Update the URL to be: https://{your_server}/ (as mentioned in #4066 as well)

Thanks,
--Neno


Brew packages installed on host are not available in container, even with host-exec

Describe the bug
When you install something via brew in the host, you cannot access it inside a container, even by using distrobox-host-exec. It is simply not available in the PATH. Other host commands work as expected.
What did you expect to happen?
distrobox-host-exec should be able to run brew commands that the host can run
Output of rpm-ostree status
State: idle
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bazzite:stable
                   Digest: sha256:03f4b166f57323b6228982bb8b78297d88f6483faf5dfe45982498cd15722c80
                  Version: 40.20240625.0 (2024-06-27T01:59:48Z)
            LocalPackages: mullvad-vpn-2024.3-1.x86_64
                   Pinned: yes

  ostree-image-signed:docker://ghcr.io/ublue-os/bazzite:stable
                   Digest: sha256:03f4b166f57323b6228982bb8b78297d88f6483faf5dfe45982498cd15722c80
                  Version: 40.20240625.0 (2024-06-27T01:59:48Z)

Hardware
No response
Extra information or context
> distrobox-host-exec which brew
which: no brew in (/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin)

brew by default is only added to PATH when the bash session is interactive, as default in /etc/profile.d/brew.sh:
#!/usr/bin/env bash
[[ -d /home/linuxbrew/.linuxbrew && $- == *i* ]] && eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"

Using bash with the --interactive flag solves the issue:

📦[zeglius@fedora ~]$ distrobox-host-exec bash -i -c 'which brew'

   Welcome to Bazzite 󰊴

  󱋩  bazzite-nvidia:stable

           COMMAND        │         DESCRIPTION
  ─────────────────────────┼──────────────────────────────
    ujust                  │ List all available commands
    ujust toggle-user-motd │ Toggle this banner on/off
    ujust changelog        │ View changelog
    fastfetch              │ View system information

  󰋼 Want to customize and theme your desktop? Tweak your desktop by following
  our guide https://universal-blue.discourse.group/docs?topic=574

  •  Report an issue https://github.com/ublue-os/bazzite/issues
  • 󰈙 Documentation http://docs.bazzite.gg/
  • 󰙯 Discord https://discord.bazzite.gg/
  • 󰫑 Mastodon https://fosstodon.org/@UniversalBlue

/home/linuxbrew/.linuxbrew/bin/brew

Another option is using the absolute path to brew binary (which is way better), you don't want scripts  to use interactive mode
distrobox-host-exec /home/linuxbrew/.linuxbrew/bin/brew

This is still bugged; many packages in brew assume that they are available after being called. For example this will not work in a container, because the atuin init script calls atuin itself.
distrobox-host-exec /home/linuxbrew/.linuxbrew/bin/atuin init fish | source

I'm finding it very difficult to set up my shell this way. Something as simple as this, my usual config (with these packages via brew), is seemingly arcane when in a container:
        atuin init fish | source
        starship init fish | source

Well yeah, the point of containers is to prevent being affected by host dependencies. My best advice is simply installing brew inside the container.
Other than that, I don't think anything else can be done without breaking container isolation.

Well yeah, the point of containers is to prevent being affected by host dependencies. My best advice is simply installing brew inside the container.
Other than that, I don't think anything else can be done without breaking container isolation.

The point of containers in this context is to prevent the container from affecting the host. The existence of distrobox-host-exec implies that the host ought to be able to affect the container, and it would be able to if these packages were installed at the image level or even the home level. It's just a quirk of the weird setup here that brew is a separate user instead of the binaries being located in ones own home. That decision is causing these downstream effects.
With Bazzite pushing heavily for both Brew and Distrobox, it doesn't make sense for this incompatibility to be intended. At least, the base images provided by ujust distrobox-assemble should include a transparent brew installation to facilitate this.

With Bazzite pushing heavily for both Brew and Distrobox

Not denying that, but I don't think it is intended to mix both. I would recommend installing these tools with the package manager the container image provides (apt if Ubuntu/debian, pacman for arch, etc).
Also there are some workarounds that might help, like symlinking host-spawn binary to ~/.local/bin/atuin or any other PATH directory, so it will call host atuin

With Bazzite pushing heavily for both Brew and Distrobox

Not denying that, but I don't think it is intended to mix both. I would recommend installing these tools with the package manager the container image provides (apt if Ubuntu/debian, pacman for arch, etc).
Also there are some workarounds that might help, like symlinking host-spawn binary to ~/.local/bin/atuin or any other PATH directory, so it will call host atuin.
You can check https://distrobox.it/posts/execute_commands_on_host/ to see which option fits your needs


Everything seems ok but no vhca device in test Pod.

Hi, I've met a problem and have no idea how to fix.  I have several nodes deployed with rdma sriov device plugin and the sriov cni, and everythin goes ok and pods can communicate with others via the vhca device whether the pods are launched on the same node or not. But one day, one the node goes bad, new pod launched on it fails to require a vhca device(the pod is launched normally and in Running phase), but everything seems ok.
I've checked the log as below:

checking the rdma/vhca resource on node:

# kubectl describe node <IP_ADDRESS>
...
Capacity:
 cpu:                48
 ephemeral-storage:  52399108Ki
 hugepages-1Gi:      0
 hugepages-2Mi:      0
 memory:             131747876Ki
 nvidia.com/gpu:     8
 pods:               110
 rdma/vhca:          8
Allocatable:
 cpu:                48
 ephemeral-storage:<PHONE_NUMBER>3
 hugepages-1Gi:      0
 hugepages-2Mi:      0
 memory:             131645476Ki
 nvidia.com/gpu:     8
 pods:               110
 rdma/vhca:          8
...

When creating a new pod on the node, I get log from rdma sriov device plugin:

2018/08/07 07:33:33 allocate request: &AllocateRequest{ContainerRequests:[&ContainerAllocateRequest{DevicesIDs:[16:5f:e4:4f:a7:28],}],}
2018/08/07 07:33:33 allocate response:  {[&ContainerAllocateResponse{Envs:map[string]string{},Mounts:[],Devices:[&DeviceSpec{ContainerPath:/dev/infiniband,HostPath:/dev/infiniband,Permissions:rwm,}],Annotations:map[string]string{},}]}

I use test-sriov-pod.yaml to create test pod, the pod can be lauched normally and in Running phase, but the network interface is not a vhca device and no vhca devices found with show_gids:

# ethtool -i eth0
driver: veth
version: 1.0
firmware-version:
expansion-rom-version:
bus-info:
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

# show_gids
DEV	PORT	INDEX	GID					IPv4  		VER	DEV
---	----	-----	---					------------  	---	---
n_gids_found=0

The sriov cni configuration is as below and it's the only cni on that node:

{
    "name": "mynet",
    "type": "sriov",
    "if0": "ens5f0",
    "ipam": {
        "type": "host-local",
        "subnet": "<IP_ADDRESS>/24
	"rangeStart": "<IP_ADDRESS>",
	"rangeEnd": "<IP_ADDRESS>",
        "routes": [
            { "dst": "<IP_ADDRESS>/0" }
        ],
        "gateway": "<IP_ADDRESS>"
    }
}

Besides, I found that all the vhca interface is in down status with command ip a, and I let them up with command ifconfig <eth-name> up manually, but nothing is changed.
Thanks for your help!

Hi @flymark2010, I see the output of ethtool -i eth0.
Driver is veth instead of mlx5.
It indicates that instead of sriov-cni some other cni provided the eth device. You might want to check if you have any other cni config file which is taking priority due to lexical ordering that you faced last time.

@flymark2010, veth is certainly not a mlx5 driver. So something went wrong there.
you can try to unload veth driver at that host using rmmod veth and see that this interface goes away from the Pod.


Indenting exports overwrites them and then undo doesn't work

I'm using the latest version of VS Code on OS X (Version 0.10.8 (0.10.8), db71ac615ddf9f33b133ff2536f5d33a77d4774e).
Replication steps:
Paste the following into a new whatever.ts file.
function foo() { }
function bar() { }
function baz() { }

export {
foo,
bar,
baz
};

Now highlight the properties of the export (the 3 lines saying foo, bar, baz) and press tab to indent them and it outputs the following and can't be undone using the undo command:
function foo() { }
function bar() { }
function baz() { }

export {
	az
};

The indenting seems to happen but where did my exports go?! :open_mouth:

I can replicate this (VS Code 0.10.8 (0.10.8), OSX 10.10.5)
Before updating I checked in 0.10.6, and the indentation resulted in the correct outcome.

duplicate https://github.com/Microsoft/vscode/issues/2829

closing as a duplicate of #2829. Thanks @SamVerschueren