Also automates script installation.
Based on https://github.com/jilleb/mib2-toolbox/issues/1#issuecomment-763540383 and https://github.com/jilleb/mib2-toolbox/issues/122
Full description and documentation updates TBD

Great work! I'll review and merg later today!

Great work! I'll review and merg later today!

I'd hold of merging until it's had a bit more testing - I haven't tested the commit I just pushed with the updated script, the instructions need updating  etc, confirm there's no bad side effects to using the DVD block etc.
I've also got the version set to 1 which means it'll need the SWDL download enabled and then manually force it to install, which will need instructions.
Alternatively we can set it to a higher number that'll likely auto-install correctly but users will need to turn manually force the DVD install if/when then install a firmware pack in future.

I'd hold of merging until it's had a bit more testing - I haven't tested the commit I just pushed with the updated script, the instructions need updating  etc, confirm there's no bad side effects to using the DVD block etc.
I've also got the version set to 1 which means it'll need the SWDL download enabled and then manually force it to install, which will need instructions.
Alternatively we can set it to a higher number that'll likely auto-install correctly but users will need to turn manually force the DVD install if/when then install a firmware pack in future.

Allright. I'll put it to a experimental branch then 😎

Allright. I'll put it to a experimental branch then 😎

I'd hold of merging until it's had a bit more testing - I haven't tested the commit I just pushed with the updated script, the instructions need updating etc, confirm there's no bad side effects to using the DVD block etc.
I've also got the version set to 1 which means it'll need the SWDL download enabled and then manually force it to install, which will need instructions.
Alternatively we can set it to a higher number that'll likely auto-install correctly but users will need to turn manually force the DVD install if/when then install a firmware pack in future.

I think we can fix this with my method from the RadioStationDB install. I defined an extra component. This way it well set another component in the update which isn't there in normal updates. So no SWDL needed.
Needs to add something like:
. . .
ConfigComponent = "DVD"
PacketName = "GreenEngineeringMenu"
ConfigCheckPath = "-"
ConfigFinalizePath = "/net/mmx/mnt/app/eso/hmi/engdefs"
IncludeModules = ",ESD,"

Will test myself.

I'd hold of merging until it's had a bit more testing - I haven't tested the commit I just pushed with the updated script, the instructions need updating etc, confirm there's no bad side effects to using the DVD block etc.
I've also got the version set to 1 which means it'll need the SWDL download enabled and then manually force it to install, which will need instructions.
Alternatively we can set it to a higher number that'll likely auto-install correctly but users will need to turn manually force the DVD install if/when then install a firmware pack in future.

I think we can fix this with my method from the RadioStationDB install. I defined an extra component. This way it well set another component in the update which isn't there in normal updates. So no SWDL needed.
Needs to add something like:
. . .
ConfigComponent = "DVD"
PacketName = "GreenEngineeringMenu"
ConfigCheckPath = "-"
ConfigFinalizePath = "/net/mmx/mnt/app/eso/hmi/engdefs"
IncludeModules = ",ESD,"

Will test myself.

Okay nevermind. DVD seems to work another way. you don't need to config component. Like you already done with path Toolbox this defines a new module. Somehow every module will list with current version 2. I don't know why but this shouldn't be a problem. If we set Toolbox as path like you done in your file we have a new module entry which only correspond to our toolbox installations.

Okay nevermind. DVD seems to work another way. you don't need to config component. Like you already done with path Toolbox this defines a new module. Somehow every module will list with current version 2. I don't know why but this shouldn't be a problem. If we set Toolbox as path like you done in your file we have a new module entry which only correspond to our toolbox installations.

Hmm final script isn't running for me. Gives me errorcode 113. Same es in my test before.

Hmm final script isn't running for me. Gives me errorcode 113. Same es in my test before.

Ok, I've set the install version to 3500 do it should now install without any manual software download stuff, it should basically just work.
If this does end up causing conflicts with official fw update we can either write instructions for how to manually install the official fw, or yeah as suggested add extra stuff to the final script to "undo" any DeviceRelease entry stored.

Ok, I've set the install version to 3500 do it should now install without any manual software download stuff, it should basically just work.
If this does end up causing conflicts with official fw update we can either write instructions for how to manually install the official fw, or yeah as suggested add extra stuff to the final script to "undo" any DeviceRelease entry stored.

Hmm final script isn't running for me. Gives me errorcode 113. Same es in my test before.

Darn, could you try from my first commit on this branch? I might have got the updated sha1 hashes wrong or something - I'm planning on writing a script to automate this.

Hmm final script isn't running for me. Gives me errorcode 113. Same es in my test before.

Darn, could you try from my first commit on this branch? I might have got the updated sha1 hashes wrong or something - I'm planning on writing a script to automate this.

Ok, I've set the install version to 3500 so it should now install without any manual software download stuff, it should basically just work.
If this does end up causing conflicts with official fw update we can either write instructions for how to manually install the official fw, or yeah as suggested add extra stuff to the final script to "undo" any DeviceRelease entry stored.

We don't need to include it. So no number will be set. completly delete this version info ;-)

Ok, I've set the install version to 3500 so it should now install without any manual software download stuff, it should basically just work.
If this does end up causing conflicts with official fw update we can either write instructions for how to manually install the official fw, or yeah as suggested add extra stuff to the final script to "undo" any DeviceRelease entry stored.

We don't need to include it. So no number will be set. completly delete this version info ;-)

Ok, I've set the install version to 3500 so it should now install without any manual software download stuff, it should basically just work.
If this does end up causing conflicts with official fw update we can either write instructions for how to manually install the official fw, or yeah as suggested add extra stuff to the final script to "undo" any DeviceRelease entry stored.

We don't need to include it. So no number will be set. completly delete this version info ;-)

Oh, if that works and installs by default that's even better! I'll try that next time I get some dev time in my car, as well as verifying the hashes.

Ok, I've set the install version to 3500 so it should now install without any manual software download stuff, it should basically just work.
If this does end up causing conflicts with official fw update we can either write instructions for how to manually install the official fw, or yeah as suggested add extra stuff to the final script to "undo" any DeviceRelease entry stored.

We don't need to include it. So no number will be set. completly delete this version info ;-)

Oh, if that works and installs by default that's even better! I'll try that next time I get some dev time in my car, as well as verifying the hashes.

Hmm I made some adjustments to the script which errors in 113. Now i went back to last commit and i get error 119 now in Unit common_toolsDir will try again first version again now.
Maybe get in touch with me via telegram, would unclutter the treads i think :D t.me/olli991

Hmm I made some adjustments to the script which errors in 113. Now i went back to last commit and i get error 119 now in Unit common_toolsDir will try again first version again now.
Maybe get in touch with me via telegram, would unclutter the treads i think :D t.me/olli991

I opend up a experimental test branch for the new installer:  https://github.com/jilleb/mib2-toolbox/tree/experimental-new-intall
I thought this would be much easier to compare everything without merging with master.
Current state:

The old get scripts method is still included for testing in de esd and therefore also the Phone dir.
I made a Install dir to make the structure even more clear, hopefully.
Finalscript is included. Currently it's commented out in the metainfo, because it gives me the error. For testing you could clone the branch and activate it again and check. To my understanding checksums should all be fine but i don't know... compare it to your working sctructure.

I opend up a experimental test branch for the new installer:  https://github.com/jilleb/mib2-toolbox/tree/experimental-new-intall
I thought this would be much easier to compare everything without merging with master.
Current state:

The old get scripts method is still included for testing in de esd and therefore also the Phone dir.
I made a Install dir to make the structure even more clear, hopefully.
Finalscript is included. Currently it's commented out in the metainfo, because it gives me the error. For testing you could clone the branch and activate it again and check. To my understanding checksums should all be fine but i don't know... compare it to your working sctructure.

@olli991 Could you check if this:
USB Drive.zip
works for you?

@olli991 Could you check if this:
USB Drive.zip
works for you?

It actually went trough without errors. Only thing i noticed is i have a NOK for RadioStationDB in the final Log. wtf? :D

It actually went trough without errors. Only thing i noticed is i have a NOK for RadioStationDB in the final Log. wtf? :D

It actually went trough without errors. Only thing i noticed is i have a NOK for RadioStationDB in the final Log. wtf? :D
i will investigate yours and mine version... so finalscript worked... should be possible to get this running even with my more edited version.

The NOK could also happen when it didn't update because the version on the unit was identical to the version that was already there.

It actually went trough without errors. Only thing i noticed is i have a NOK for RadioStationDB in the final Log. wtf? :D
i will investigate yours and mine version... so finalscript worked... should be possible to get this running even with my more edited version.

The NOK could also happen when it didn't update because the version on the unit was identical to the version that was already there.

There is absolutly no RadioStationDB Update even mentioned in the metainfo or shown via custom swdl before. that bothers me.

There is absolutly no RadioStationDB Update even mentioned in the metainfo or shown via custom swdl before. that bothers me.

Okay i found a possible flaw you finalscript has a blank row at the end mine not.
Tried it, didn't work. Checked the files with winmerge.... somhow mine is completly yellow even though content is literally the same... somting broken with my file i think... i now copied you scriptfile and edited to my commit.... let's see

Okay i found a possible flaw you finalscript has a blank row at the end mine not.
Tried it, didn't work.
Now everything works.
Now everything works. The door is open we can perform everthing with modified update! 🎉

@andrewleech did you already tested if we could also save the finalscript somwhere else? Will test it tomorrow if i can.
This way we could make the structure even more simplified.
We also need to know if mapcare can block this update.
A now working and restructured build can be found in the experimental branch. Would be good if we investigare more on this build, i think.

@andrewleech did you already tested if we could also save the finalscript somwhere else? Will test it tomorrow if i can.
This way we could make the structure even more simplified.
We also need to know if mapcare can block this update.
A now working and restructured build can be found in the experimental branch. Would be good if we investigare more on this build, i think.

 Great you found the difference!
This is a crucial finding too - as a default install of git on windows will download all text files with the line endings modified this way. We can/should add a .gitattributes file to the repo which tells git not to do this.
I haven't tried changing the dir of final script, but I'm sure it can be changed a bit. It'll just need to be somewhere with a hashes file, but it's probably easiest to keep it in its own folder so as to not need to update hashes too often.

 Great you found the difference!
This is a crucial finding too - as a default install of git on windows will download all text files with the line endings modified this way. We can/should add a .gitattributes file to the repo which tells git not to do this.
I haven't tried changing the dir of final script, but I'm sure it can be changed a bit. It'll just need to be somewhere with a hashes file, but it's probably easiest to keep it in its own folder so as to not need to update hashes too often.

Yeah it'll be interesting if the mapcare swap timeout blocks this, I kind of doubt it seeing as the navdb items are now blocked out from the installer.
If that does cause a problem we should just be able to switch to a firmware update as the donor file.

Yeah it'll be interesting if the mapcare swap timeout blocks this, I kind of doubt it seeing as the navdb items are now blocked out from the installer.
If that does cause a problem we should just be able to switch to a firmware update as the donor file.

I prepared metainfo but didn't test so far. Don't know if this could work
#FinalScript = "./common/tools/0/default/finalScript.sh"
FinalScript = "./Install/final/finalScript.sh"
FinalScriptChecksum = "4928fe9cd3b741104e2a85c47f4d16fc9ff16d76"
FinalScriptMaxTime = "30"
FinalScriptName = "Final Script"

[Install\final\dir]
FileSize = "797"
CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

#[common\tools\0\default\dir]
#FileSize = "797"
#CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

I prepared metainfo but didn't test so far. Don't know if this could work
#FinalScript = "./common/tools/0/default/finalScript.sh"
FinalScript = "./Install/final/finalScript.sh"
FinalScriptChecksum = "4928fe9cd3b741104e2a85c47f4d16fc9ff16d76"
FinalScriptMaxTime = "30"
FinalScriptName = "Final Script"

[Install\final\dir]
FileSize = "797"
CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

#[common\tools\0\default\dir]
#FileSize = "797"
#CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

So it turns out we don't need to reuse an existing config, I renamed my DVD block to MQB and it works perfectly.
I've got the folders cleaned up as suggested, works great.
I've added the gitattributes to enforce line endings, only to discover that metainfo2.txt does have to be CRLF, while the rest should be LF. The metainfo can't be parsed by the unit at all if its LF.
I still need to push up a fix for that last line ending need, but the rest of the branch is working perfectly. It shows the toolbox install, automatically installs fine without the manual download stuff, looks slick.
I've got a new video of the install process I'll publish too.

So it turns out we don't need to reuse an existing config, I renamed my DVD block to MQB and it works perfectly.
I've got the folders cleaned up as suggested, works great.
I've added the gitattributes to enforce line endings, only to discover that metainfo2.txt does have to be CRLF, while the rest should be LF. The metainfo can't be parsed by the unit at all if its LF.
I still need to push up a fix for that last line ending need, but the rest of the branch is working perfectly. It shows the toolbox install, automatically installs fine without the manual download stuff, looks slick.
I've got a new video of the install process I'll publish too.

I prepared metainfo but didn't test so far. Don't know if this could work
#FinalScript = "./common/tools/0/default/finalScript.sh"
FinalScript = "./Install/final/finalScript.sh"
FinalScriptChecksum = "4928fe9cd3b741104e2a85c47f4d16fc9ff16d76"
FinalScriptMaxTime = "30"
FinalScriptName = "Final Script"

[Install\final\dir]
FileSize = "797"
CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

#[common\tools\0\default\dir]
#FileSize = "797"
#CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

Nice so this idea with a completely new block actually worked. I suggest FinalScript works also in your commit from different source now. That's a really clean solution now.
Final steps:

Rework Readme and include your video there, if you want.
Make sure update works without mapcare.

I prepared metainfo but didn't test so far. Don't know if this could work
#FinalScript = "./common/tools/0/default/finalScript.sh"
FinalScript = "./Install/final/finalScript.sh"
FinalScriptChecksum = "4928fe9cd3b741104e2a85c47f4d16fc9ff16d76"
FinalScriptMaxTime = "30"
FinalScriptName = "Final Script"

[Install\final\dir]
FileSize = "797"
CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

#[common\tools\0\default\dir]
#FileSize = "797"
#CheckSum = "d995d9b1383b96e4e26380eb35b0af7487c4457e"

Nice so this idea with a completely new block actually worked. I suggest FinalScript works also in your commit from different source now. That's a really clean solution now.
Final steps:

Rework Readme and include your video there, if you want.
Make sure update works without mapcare.

Thanks for the py script review, I'll fix that!
I noticed your branch had other changes, but as I didn't know what they were for I wanted to keep my git commits/authorship with just the changes I've worked on for the installer.
If your other changes are to support the new installation method/paths it makes sense to merge them in, but other kinds of improvements should probably be in their own commits/branch?
On a positive note, I'm pretty sure I've got the line endings working correctly now.
I downloaded the repo as a zip from my branch (https://github.com/andrewleech/mib2-toolbox/archive/new_installer.zip) and loaded it on a clean SD and it installed correctly.
Video is just uploaded: https://youtu.be/-6A-4-3qR_0

Thanks for the py script review, I'll fix that!
I noticed your branch had other changes, but as I didn't know what they were for I wanted to keep my git commits/authorship with just the changes I've worked on for the installer.
If your other changes are to support the new installation method/paths it makes sense to merge them in, but other kinds of improvements should probably be in their own commits/branch?
On a positive note, I'm pretty sure I've got the line endings working correctly now.
I downloaded the repo as a zip from my branch (https://github.com/andrewleech/mib2-toolbox/archive/new_installer.zip) and loaded it on a clean SD and it installed correctly.
Video is just uploaded: https://youtu.be/-6A-4-3qR_0

I noticed your branch had other changes, but as I didn't know what they were for I wanted to keep my git commits/authorship with just the changes I've worked on for the installer.

I see. The other changes are added scripts, so improvments and additions to the toolbox.
Some changes are actually relevant to the update process i.e. the update_toolbox.sh. I will manually copy your changes to the test branch then, if this is ok for you. But I'm not sure if we ever push this pull request then 🤣

I noticed your branch had other changes, but as I didn't know what they were for I wanted to keep my git commits/authorship with just the changes I've worked on for the installer.

I see. The other changes are added scripts, so improvments and additions to the toolbox.
Some changes are actually relevant to the update process i.e. the update_toolbox.sh. I will manually copy your changes to the test branch then, if this is ok for you. But I'm not sure if we ever push this pull request then 🤣

@olli991 I've just merged / cherry picked the rest of your script fixes onto my branch, should be looking better now!

@olli991 I've just merged / cherry picked the rest of your script fixes onto my branch, should be looking better now!

I'll take a look at the Readme later. I'm open to suggestions whether you think it's worth linking in the video, not sure how much help it is? Might be worth extending it with some details about how to enable developer / GEM in the first place, as well as just what to download / copy onto SD (this wasn't obvious to me when I started here).

I'll take a look at the Readme later. I'm open to suggestions whether you think it's worth linking in the video, not sure how much help it is? Might be worth extending it with some details about how to enable developer / GEM in the first place, as well as just what to download / copy onto SD (this wasn't obvious to me when I started here).

Can we somehow change this pullreqest to merge into the experimental branch? So this way we can collaborate in the branch and can pull this into the master?! I'm just getting into the mechanics of git tbh.

Can we somehow change this pullreqest to merge into the experimental branch? So this way we can collaborate in the branch and can pull this into the master?! I'm just getting into the mechanics of git tbh.

The experimental branch looked like it had a mixture  / double up of commits, things weren't all that cleanly separated and most of my initial changes were kind of merged into your first commit (so not in my name anymore).
I generally work on keeping my branches as clean as possible. I think I can give you access to my branch though if that helps?

The experimental branch looked like it had a mixture  / double up of commits, things weren't all that cleanly separated and most of my initial changes were kind of merged into your first commit (so not in my name anymore).
I generally work on keeping my branches as clean as possible.  I just uploaded a local directory into a new branch. Local files where everything we've done so far. This way commits are gone.  I just uploaded a local directory into a new branch. Local files where everything we've done so far. This way commits are gone. I can't fix this anymore I think.

For documentation. We merged this pull request and my findings into the experimental branch. Further pullreqest on the new version will go into experimental till we merge experimental into master.
Further investigations:

I tried to shorten the paths in the metainfo even more. It didn't work. I tried to change [MQB\Toolbox\0\default\File] into  [MQB\Toolbox\File]. Update loads but didn't find any modules to update. Everythin on N/A.
I played arout with the naming and it is even possible to write for excample  [M.Q.B.\Toolbox\0\default\File]. Dots will be parsed and shown everywhere.

Only thing left to to is readme. And maybe a new screenshot from the grennmenu itself for the readme. Also some housekeeping could be done.

For documentation. We merged this pull request and my findings into the experimental branch. Further pullreqest on the new version will go into experimental till we merge experimental into master.
Further investigations:

I tried to shorten the paths in the metainfo even more. It didn't work. I tried to change [MQB\Toolbox\0\default\File] into  [MQB\Toolbox\File]. Update loads but didn't find any modules to update. Everythin on N/A.
I played arout with the naming and it is even possible to write for excample  [M.Q.B.\Toolbox\0\default\File]. Dots will be parsed and shown everywhere.

Only thing left to to is readme. And maybe a new screenshot from the grennmenu itself for the readme. Also some housekeeping could be done.

@olli991 shame shortening the path didn't work, there must be some enforcement to the format of it.
I did think last night, we could just rename our new Install folder to MQB then it'd align with metainfo and clean up the paths

@olli991 shame shortening the path didn't work, there must be some enforcement to the format of it.
I did think last night, we could just rename our new Install folder to MQB then it'd align with metainfo and clean up the paths

Indeed that would be right.

Indeed that would be right.

I think its much clearer if we have a install folder tbh.
I would leave it like this.
Only think we could think about is changing MQB to to somthing better. Because MQB stands for Modularer Quer Baukasten and thats literally a bit wide :D

I think its much clearer if we have a install folder tbh.
I would leave it like this.
Only think we could think about is changing MQB to to somthing better. Because MQB stands for Modularer Quer Baukasten and thats literally a bit wide :D

Could change MQB to Toolbox and also call the folder that - I think that'd be as good as Install myself.

Could change MQB to Toolbox and also call the folder that - I think that'd be as good as Install myself.

Could change MQB to Toolbox and also call the folder that - I think that'd be as good as Install myself.

metainfo will look like this than [Toolbox\Toolbox\0\default\File]. But we have to live with it then :D

Could change MQB to Toolbox and also call the folder that - I think that'd be as good as Install myself.

metainfo will look like this than [Toolbox\Toolbox\0\default\File]. But we have to live with it then :D

It's not a big deal. On my original toolbox I didn't really think about nice folder names and such 😅😅 just went with what worked.

It's not a big deal. On my original toolbox I didn't really think about nice folder names and such 😅😅 just went with what worked.

It's not a big deal. On my original toolbox I didn't really think about nice folder names and such 😅😅 just went with what worked.

I have a fable for also good looking and / or intuitiv stuff. It's more work right, but could make it much easier to understand for newsbies 😬

It's not a big deal. On my original toolbox I didn't really think about nice folder names and such 😅😅 just went with what worked.

I have a fable for also good looking and / or intuitiv stuff. It's more work right, but could make it much easier to understand for newsbies 😬

Yeah, I have to agree on that. Having clear naming decreases the amount of support needed 😅

Yeah, I have to agree on that. Having clear naming decreases the amount of support needed 😅

Maybe change MQB\Toolbox to Toolbox\GEM?

Maybe change MQB\Toolbox to Toolbox\GEM?

@olli991 I've re-targeting this PR to the experimental branch, that worked now.
I've added a commit with the renaming Install -> Toolbox and updated scripts to suit. I also found a bunch of path issues/discrepancies in the scripts as I went through them for this, so there's a few cleanups there.
I've also updated the readme, let me know what you all think!

@olli991 I've re-targeting this PR to the experimental branch, that worked now.
I've added a commit with the renaming Install -> Toolbox and updated scripts to suit. I also found a bunch of path issues/discrepancies in the scripts as I went through them for this, so there's a few cleanups there.
I've also updated the readme, let me know what you all think!

One more fix needed - the esd file needs it's line endings fixed too, it's currently broken / hash doesn't match

One more fix needed - the esd file needs it's line endings fixed too, it's currently broken / hash doesn't match

One more fix needed - the esd file needs it's line endings fixed too, it's currently broken / hash doesn't match

How could this happen 🤔

One more fix needed - the esd file needs it's line endings fixed too, it's currently broken / hash doesn't match

How could this happen 🤔

One more fix needed - the esd file needs it's line endings fixed too, it's currently broken / hash doesn't match

How could this happen 🤔

The esd file isn't listed in gitattributes yet, so when I read my update hash script or was running on local LF version, but a clean zip from GitHub had it as CRLF (or the other way around, don't remember for sure)

One more fix needed - the esd file needs it's line endings fixed too, it's currently broken / hash doesn't match

How could this happen 🤔

The esd file isn't listed in gitattributes yet, so when I read my update hash script or was running on local LF version, but a clean zip from GitHub had it as CRLF (or the other way around, don't remember for sure)

My local .esds are all crlf... maybe it doesn't matter?

My local .esds are all crlf... maybe it doesn't matter?

It doesn't matter if it's LF or CRLF as long as the hash matches - changing the line endings changes the hash.

It doesn't matter if it's LF or CRLF as long as the hash matches - changing the line endings changes the hash.

This final set of changes still need a live test. I'll try to do that tomorrow.

This final set of changes still need a live test. I'll try to do that tomorrow.

This final set of changes still need a live test. I'll try to do that tomorrow.

Can do also later. We'll see. I'll give it a run if i can this evening. If it works I'll merge.

This final set of changes still need a live test. I'll try to do that tomorrow.

Can do also later. We'll see. I'll give it a run if i can this evening. If it works I'll merge.

Ottimo lavoro! Rivedrò e fonderò più tardi oggi!

Ciao jilleb ti scrivo per dirti che ho provato ad installare sul mio discover mib2 pro Volkswagen il tuo Mib2-toolbox ma quando vado ad installarlo l'autoradio mi dice che non ha potuto leggere il file, sulla SD ho messo tutto come da guida ho attivato opzione sviluppatore sullautoradio ma niente.

Ottimo lavoro! Rivedrò e fonderò più tardi oggi!

Ciao jilleb ti scrivo per dirti che ho provato ad installare sul mio discover mib2 pro Volkswagen il tuo Mib2-toolbox ma quando vado ad installarlo l'autoradio mi dice che non ha potuto leggere il file, sulla SD ho messo tutto come da guida ho attivato opzione sviluppatore sullautoradio ma niente.

@enricogialletta first of: Please write in english.
Second: Which version did you use? Release 2.2B or a clone of the master branch?

@enricogialletta first of: Please write in english.
Second: Which version did you use? Release 2.2B or a clone of the master branch?

: clone del ramo master. all'interno del tool ci sono 8 cartelle e 4 file

: clone del ramo master. all'interno del tool ci sono 8 cartelle e 4 file

Wait for new release. We're working on it.

Wait for new release. We're working on it.

Confirmed the last build works in car!

Confirmed the last build works in car!

ok thank you so much as soon as you have the solution can you send me everything? Thanks again
Inviato da Libero Mail per Android martedì, 26 gennaio 2021, 06:17PM +01:00 da olli991<EMAIL_ADDRESS>:

Merged  #125 into experimental-new-intall.
—
You are receiving this because you were mentioned.
Reply to this email directly,  view it on GitHub , or  unsubscribe .

ok thank you so much as soon as you have the solution can you send me everything? Thanks again
Inviato da Libero Mail per Android martedì, 26 gennaio 2021, 06:17PM +01:00 da olli991<EMAIL_ADDRESS>:

Merged  #125 into experimental-new-intall.
—
You are receiving this because you were mentioned.
Reply to this email directly,  view it on GitHub , or  unsubscribe .

can you send me everything please?
Inviato da Libero Mail per Android martedì, 26 gennaio 2021, 08:03PM +01:00 da olli991<EMAIL_ADDRESS>:

Confirmed the last build works in car!
—
You are receiving this because you were mentioned.
Reply to this email directly,  view it on GitHub , or  unsubscribe .

can you send me everything please?
Inviato da Libero Mail per Android martedì, 26 gennaio 2021, 08:03PM +01:00 da olli991<EMAIL_ADDRESS>:

Confirmed the last build works in car!
—
You are receiving this because you were mentioned.
Reply to this email directly,  view it on GitHub , or  unsubscribe .

@enricogialletta if you subscribe to notifications on this issue you should get an update when it's all done and ready to use: https://github.com/jilleb/mib2-toolbox/issues/122

@enricogialletta if you subscribe to notifications on this issue you should get an update when it's all done and ready to use: https://github.com/jilleb/mib2-toolbox/issues/122

Closing this issue, since it's finished. I'll also close the signature and metainfo error related issues.

Closing this issue, since it's finished. I'll also close the signature and metainfo error related issues.

I managed to install the toolbox and on android auto now the car streem and AA mirror apps come out but when I put a YouTube video on the monitor you can hear only audio the video is not -- Inviato da Libero Mail per Android martedì, 26 gennaio 2021, 10:17PM +01:00 da Andrew Leech<EMAIL_ADDRESS>:
…
@enricogialletta if you subscribe to notifications on this issue you should get an update when it's all done and ready to use: #122 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

Sounds like a problem with the Carstream software, not with the toolbox.

I managed to install the toolbox and on android auto now the car streem and AA mirror apps come out but when I put a YouTube video on the monitor you can hear only audio the video is not -- Inviato da Libero Mail per Android martedì, 26 gennaio 2021, 10:17PM +01:00 da Andrew Leech<EMAIL_ADDRESS>:
…
@enricogialletta if you subscribe to notifications on this issue you should get an update when it's all done and ready to use: #122 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

Sounds like a problem with the Carstream software, not with the toolbox.


NIFI-4885: Granular component restrictions

NIFI-4885:

Introducing more granular restricted component access policies.
Current restricted components have been updated to use new granular restrictions.

After deleting a specific granular policy, the following message appears:
Showing effective policy inherited from Process Group restricted-components. Override this policy.
The text "restricted-components" in the message is hyperlinked.  If clicked you get a dialog that says:
Unable to locate group with id 'restricted-components'.
Unable to enter the selected group.

Thanks for the feedback @andrewmlim! I've pushed another commit addressing the issue.

@mcgilman
I tested your latest changes and they look good.  A couple minor things to address:

In the message "Only listing restriction specific users. Users with permission "regardless of restriction" not shown but are also allowed."  should add an "s" to the second instance of "restriction".  So the correct message reads:

Only listing restriction specific users. Users with permission "regardless of restrictions" not shown but are also allowed.

The following text is used in a tooltip and doc:

"Allows users to create/modify restricted components assuming otherwise sufficient permissions”
I thought this might be more clear:
“Allows users to create/modify restricted components assuming other permissions are sufficient”
-In the Admin and User Guides, change the content to:
Allows users to create/modify restricted components assuming other permissions are sufficient. The restricted components may indicate which specific permissions are required. Permissions can be granted for specific restrictions or be granted regardless of restrictions. If permission is granted regardless of restrictions, the user can create/modify all restricted components.

Also, maybe a separate issue, but wanted to note that the MoveHDFS processor is tagged as "restricted" in the Add Processor window, but it is not treated like other restricted components.

@mcgilman Looks good!  Thanks for including a fix for MoveHDFS.

Thanks @mcgilman! I am a +1 for the UI code.

@mcgilman this all looks good to me as well! Given positive review feedback from @andrewmlim and a +1 from @scottyaslan I am happy with the changes and think this is a great improvement on our security model. +1 merged to master. Thanks!


[Match] Manually encrypt and upload manually created certs?

New Issue Checklist

[ x ] Updated fastlane to the latest version
[ x ] I have read the Contribution Guidelines

Issue Description
Because of issue #5123 i'm wondering if there is a way to manually encrypt some certs to upload?
My specific scenario is that I need to re-generate the provisioning profiles in order to test on a new device but since apple's developer site is sporadic I can't get match to work however the site does occasionally work (very slowly).
Environment
fastlane version (run fastlane -v): fastlane 1.101.0
Do you use bundler to execute fastlane (i.e. bundle exec fastlane)? - No
Do you use a Ruby environment manager (e.g. chruby, rbenv, rvm)? - No

Hey @reallistic, you can run match in readonly mode to just decrypt and install all the certificates

@KrauseFx I was under the impression I needed to run match --force_for_new_devices in order for the new devices to be added to the provisioning profile and/or certs?

@reallistic Yes, that's correct, but you can't add new devices when the Developer Portal is down, sorry

@KrauseFx Thanks for the info.
Does this also mean it isn't possible to manually encrypt the repo to add new (or other) stuff?

@reallistic to add a new device, you'll have to access the developer portal, as it all happens on the server-side. You'll have to wait until Apple's servers are up again. Alternatively, you can manually add the device using the Apple Dev Portal website as as far I as know, just the Xcode API is down (which is what spaceship uses)

I should have further explained that some time ago I messed up my repo by manually adding a provisioning profile.
I've since reverted the commit which now allows readonly to work, but i'm wondering if it is possible for me to manually encrypt that provisioning profile so match --readonly can successfully pull it.
I completely understand the limitations of match without the dev portal.
What I now want to know is (as the title of this issue states) Can you manually encrypt and upload certs/provisioning profiles/other files/etc?
Perhaps by using a local pgp install or similar?

@reallistic Thanks for following up. Right now it's not possible. The reason is Match checks against Apple to see if the profile is valid when run. Thanks!

Thanks for the info.


tweets.2016-09-01 dataset

""" Creates a vocabulary from a tsv file.
"""

import codecs
import example_helper
from torchmoji.create_vocab import VocabBuilder
from torchmoji.word_generator import TweetWordGenerator

with codecs.open('../../twitterdata/tweets.2016-09-01', 'rU', 'utf-8') as stream:
    wg = TweetWordGenerator(stream)
    vb = VocabBuilder(wg)
    vb.count_all_words()
    vb.save_vocab()

In this code, in oder to create a vocabulary, you had been used '../../twitterdata/tweets.2016-09-01'
dataset. But where I will find this dataset? Please let me know.
Please share this dataset with my mail<EMAIL_ADDRESS>if it is possible.

Hi @rezwanh001,
I am also looking for the dataset. Any updates on this?
TIA

Hello,have you solved this problem?

@KingS770234358 , This issue is not solved yet.

@rezwanh001 as the huggingface mentioned in the readme file,the code in the 'script' folder are used to process the raw data in the folder ‘data'. I think 'tweets.2016-09-01' may be the result of processing.

Maybe you should run the script 'convert_all_datasets.py' in the 'script' folder.

@KingS770234358 I tried running that script. Ran into this error.
Converting Olympic
-- Generating ../data/Olympic/own_vocab.pickle
     done. Coverage: 0.030899113550021062
-- Generating ../data/Olympic/twitter_vocab.pickle
     done. Coverage: 0.8874630645842128
-- Generating ../data/Olympic/combined_vocab.pickle
Traceback (most recent call last):
  File "/Users/avij1/Desktop/imp_shit/torchMoji/scripts/convert_all_datasets.py", line 88, in <module>
    data = pickle.load(dataset, fix_imports=True,encoding='utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 6: invalid continuation byte
     done. Coverage: 0.8874630645842128
Converting PsychExp


Sometimes incorrectly parsing output

I have some code that is currently calling getMAC() about once a second. Every once in a while it will return 'eth0' instead of the MAC address. It doesn't happen very often compared to how often it's called, but it happens a few times a day. This is running on a Debian Linux machine, btw.

We're hitting this problem too! No clue why it seems to be doing this. The first time i tired it returned the interface name every time, but on the second try it returns the correct mac address... If you have any insights into this let me know and i'll be happy to open a PR. We'll probably keep investigating.

Figured it out! PR incoming ;)

https://github.com/nekuz0r/node-arp/pull/8


Migrate to v4 of RDF4J

We are still using v2, which is very outdated...

Done with bcc7e36875b37e7cac1060d854a2b834f1e62b66


What would be the mixin contrary to hide-text? [unhide-text]

When I use the span tag with glyphincons within labels or buttons, in many cases I want to hide the text with the mixin hide-text. But it also hides the icon, because it the same as text and to be a child element ..
Knowing that want to apply a kind of "unhide-text" in the span tag to restore the visibility of it.
Someone uses some class or technique for this?

I'd say the most robust way of achieving what I think you're trying to do is something like this:
<button ...> the hidden text</button>

Yeah. My intention is only to show the icon. This works, but the amount of span tags in html will increase as a class of reversion to hide-text would be ideal. I will try here.
Thanks!

as the :before pseudo element (which contains the actual glyph as generated content) still inherits the styles applied to its , I don't think there's a clean way to both hide the span's content AND to have its icon show - at least to my knowledge. also the same problem applies if you were trying to apply a hiding class to the <button> itself and then trying to somehow unhide just the .

Truth! If I finish, I decided not to use the hide-text to this case because it really has no way to reverse the icon visibility. So how am working on SASS, I created a specific class for these cases in my project:
.ico-only {
   text-indent: -9999px;
   span.pficon {
     text-indent: 0px;
     float: left;
   }
}

Mesmo assim obrigado @patrickhlauke !

:+1:


feat(healthcheck): add /api/health endpoint and node script to verify app

closes #99

:tada: This PR is included in version 1.17.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add authorizer functions

Yandex Cloud API Gateway support custom OpenAPI extension that enables you to define Authorizer function. It would be great to create those right from this plugin.

Hello! I hope you're doing well. I'm reaching out to inquire about the current status of this Issue. I understand that everyone's time is valuable and that this project might not be your top priority. However, if there's been any progress or if there's an estimated timeline for when this issue might be addressed, I'd greatly appreciate an update.

I personally use Terraform in my pet projects nowadays.
Terraform offers a broad infrastructure as code (IaC) capability, allowing you to manage not only cloud services but also other resources like DNS and SaaS products, providing a more unified approach to infrastructure management. It is cloud-agnostic, supporting multiple providers, which enables a single configuration to be used across different environments and avoids vendor lock-in, enhancing flexibility in deployment strategies. Additionally, Terraform's state management enables a more reliable and trackable infrastructure provisioning and scaling process, improving the predictability and stability of infrastructure changes over time.


rkt: support deferring of the signature download

as per the discussion on #573

thanks for the patch, I am slightly lukewarm (this function's getting too spaghettish, not your fault), going to sleep on it.

I totally agree: the nested conditionals are getting a little out of hand.

@jonboulle ping

@jzelinskie figured out what was nagging at me, comments inline

@jzelinskie so close! one more fix then lgtm.

@jonboulle thanks, rebase necessary?

@jzelinskie yes please squash. Lgtm

@jonboulle ok, just tested with quay staging and it all looks good! thanks a bunch


Reject non-string keys

As far as I understand, JSON only allows string keys. However, the current implementation of MapColumnConverter allows arbitrary types as map keys which potentially cause issues for JsonConverter. This PR adds a safeguard that rejects non-string map keys.

Thanks for the review. I’ll come up with a better way to handle these concerns mentioned above and revise code accordingly.

lgtm


albert: 0.26.6 -> 0.26.10

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for albert is: Fast and flexible keyboard launcher
meta.homepage for albert is: https://albertlauncher.github.io
meta.changelog for albert is: https://github.com/albertlauncher/albert/blob/v0.26.10/CHANGELOG.md
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.26.10 with grep in /nix/store/vcy8w13q4vdxzhfzfg4b6jg6zv9sdddn-albert-0.26.10
found 0.26.10 in filename of file in /nix/store/vcy8w13q4vdxzhfzfg4b6jg6zv9sdddn-albert-0.26.10

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

albert

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vcy8w13q4vdxzhfzfg4b6jg6zv9sdddn-albert-0.26.10 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A albert https://github.com/r-ryantm/nixpkgs/archive/3039cf4b6727f35a7e98ece4ca8d02568d3292c8.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/3039cf4b6727f35a7e98ece4ca8d02568d3292c8#albert

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vcy8w13q4vdxzhfzfg4b6jg6zv9sdddn-albert-0.26.10
ls -la /nix/store/vcy8w13q4vdxzhfzfg4b6jg6zv9sdddn-albert-0.26.10/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review

x86_64-linux

:white_check_mark: 1 package built:

albert

Maintainer pings
cc @ericsagnes @Synthetica9 @eljamm for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

@ofborg build albert


Fix gene_type in search results (renamed to Biotype)

Fixes #164

Does this need a reindex in dev, stage, prod?

Yes, for sure. Along with the disease changes -- it'd be great to reindex.

Index has been started.

Index is now finished.


Generate step hangs

After running the command on macos
./scripts/generate.sh \
  -i https://raw.githubusercontent.com/alpacahq/bkdocs/master/assets/openapi.yaml \
  -p client \
  -o generated \
  -n api.services.alpaca.broker.client

I get the output
Storing intermediate outputs in /Users/brapaport/udon/fastapi_client/tmp.BUD4aF2bG; it will be removed if generation is successful
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

But it appears to hang forever.

M1 Mac, you might need to edit scripts/util/openapi-generate.sh to account for ARM running X86 docker image. I know this issue from something else, dont have M1 to test myself.


Can't open .md preview in full tab any more?

Issue Type: Bug
VS Code just updated and now my .md file previews show up in a tiny panel to the right instead of in a full-width tab. I can't find a way to make them go back to the full-width tabs (like my editor tabs).
VS Code version: Code 1.38.1 (b37e54c98e1a74ba89e03073e5a3761284e3ffb0, 2019-09-11T13:31:32.854Z)
OS version: Darwin x64 18.7.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz (8 x 2200)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_deferred_display_list: disabled_offskia_renderer: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (0.02GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (14)

Extension
Author (truncated)
Version

vscode-sidebar
Acr
1.1.2

rtf
ale
2.0.2

vscode-markdownlint
Dav
0.30.2

docs-article-templates
doc
0.2.7

docs-authoring-pack
doc
0.1.2

docs-markdown
doc
0.2.17

docs-preview
doc
0.3.29

docs-yaml
doc
0.2.4

restructuredtext
lex
114.0.0

python
ms-
2019.9.34911

team
ms-
1.149.2

vscode-yaml
red
0.5.2

rewrap
stk
1.9.1

code-spell-checker
str
1.7.18

Ohhh I figured it out. The button that looks like the "preview" button is now the "side preview button" and there's another button that I didn't recognize for the tab preview. Took forever to find it though - nothing with those words in the menus. In retrospect, I'm not sure why I was cueing off the magnifying glass in the button - that's not really relevant to preview, but maybe it was in the old button for some odd reason.

Thanks for following up. Closing since this sounds related to icon update


Correct spelling

Looks like just a merge conflict needs to be fixed then it can be merged.

Resolved!

@hugovk , would you mind reaching out to us in slack or via email? We'd like to send a small token of appreciation but aren't sure where to send it. You can find my email here.

@placer14 Emailed, thanks!


Error codes generated by plugins cannot be ignored in analysis_options

Trying to ignore an error code that was generated by a plugin results in an unrecognized_error_code error:

The validation is based on a list of error codes here:
https://github.com/dart-lang/sdk/blob/dd033a6ab7537083ece4fff6ba15093a29e91860/pkg/analyzer/lib/error/error.dart#L23
This doesn't seem to have any way to know about plugins error codes (I'm not sure plugins even need to declare them up-front?). It's not just the validation that's failing though, the error also isn't ignored either.

There are two distinct and interesting bugs here.
The first is the validation. Being able to validate that the error codes listed in the analysis options file are correct requires that we can know all of the valid error codes. We don't currently have any way to know about error codes from plugins. In order to continue to validate error codes I think we'll need a way to ask plugins for a list of the names of the error codes that it can produce.
The second is the filtering. The problem here is that we're not applying the filtering to the diagnostics produced by plugins. For that I think we'd need to update either NotificationManager to filter errors as they're being recorded (recordAnalysisErrors) or the places that call that method to filter them before sending them to the NotificationManager. Even though I kind of like it better, the former choice would result in some duplication of effort because we've already filtered the diagnostics produced by the analyzer in the LibraryAnalyzer. In either case, we'll end up recreating the information about which codes to ignore, which is a bit unfortunate.
@scheglov In case he has other thoughts.

Creating IgnoreInfo is probably inexpensive enough that we don't need to worry about it.
If necessary, we could cache it in FileState.
I think either place to filter errors is fine, as long as we have IgnoreInfo.


Kdf x963 ec changes

KDF used by CMS EC Key Agreement has been moved to EVP_KDF. This kdf is almost identical to the the SSKDF hash variant, So it has been implemented in the SSKDF code with its own method table.

Checklist

[x] documentation is added or updated
[x] tests are added or updated

Updated commit messages

Merged, thanks.


Wanted to work on forgot password functionality.

If someone wants to forget the password i can add the page for forget password and can make controller for forget password.
And the functionality is like that if someone wants to forget then he enter the email and then that email an otp is send and then that otp verify and then new password field is visible and then that password is stored in the database.

not planned as of now


Avoid warning under setopt warn_nested_var

Previously in a shell running with setopt warn_nested_var, the
following warning would be produced:

_zsh_highlight_main__precmd_hook:1: array parameter _zsh_highlight_main__command_type_cache set in enclosing scope in function _zsh_highlight_main__precmd_hook

The documentation for WARN_NESTED_VAR is:

Print a warning message when an existing parameter from an enclosing function scope, or  global, is set  in a function by an assignment or in  math context. […] Where possible, use of typeset -g to set the parameter suppresses the error, but note that this needs to be used every time the parameter is  set.

The CI failure appears unrelated to this PR's changes:
/work/highlighters/main/main-highlighter.zsh:1680: parse error near `}'

No, it's a genuine failure.  (Note it's on the correct line number and didn't happen on master.)  Older zsh's don't support the reserved word version of typeset, only the builtin version, so they take the empty parentheses as an empty set of glob qualifiers.  Does it work as two lines? — typeset -gA _zsh_highlight_main__command_type_cache; _zsh_highlight_main__command_type_cache=()
Once redrawhook is merged we should also merge #688 to put an end to this class of problem.

Blast, you're right! Unfortunately it does not suppress the warning if done as two lines. I figured this was a quick fix but it looks like this would be a problem for zsh < 5.1. There are probably workarounds but I am not very familiar with the older versions. Should I close this PR and just leave an issue about the warning?

Unfortunately it does not suppress the warning if done as two lines.

Ah, my bad.  I missed that the documentation actually warns about this.

There are probably workarounds but I am not very familiar with the older versions.

The main thing is that you can't use =(…) in the same line as typeset (or one of its aliases, e.g., local).  We also avoid relying on special parameters provided by the zsh/parameter module: for example, we do use ${options}, but we have a fallback codepath for when zsh/parameter isn't available.
Anyway, I just pushed a different fix.  Thanks for the report.


[Bug]: Outdated Ubuntu image in Terraform configuration file.

Issue:
I have encountered an issue with the Ubuntu image that is available in the Terraform configuration file. The current image is not available in the GCP registry. This causes errors with the infrastructure deployment.
Description:
The current Ubuntu image in the Terraform configuration file is outdated and needs to be replaced with version 20.04.
Expected Behavior:
I expect the Terraform configuration file to be updated to run smoothly on GCP.
Steps to Reproduce:
Open the Terraform configuration file.
Look for the Ubuntu image resource.
Note the version and release date of the image.
Current Behavior:
The current Ubuntu image in the Terraform configuration file is outdated and causes errors after using the command "deploystack install" in the terminal.
Proposed Solution:
I propose that the Ubuntu image in the Terraform configuration file be patched and updated to the 20.04 version available. This will ensure the infrastructure is up-to-date and will work with the official tutorial.
Additional Information:
I would greatly appreciate it if the necessary steps could be taken to address this issue and provide a patch for the Ubuntu image in the Terraform configuration file. Thank you for your attention to this matter.

I have created a pull request to solve this problem. Please don't hesitate to review it.

The issue is resolved, thank you, Ryan, for merging the pull request.


Unable to download movies from CBS All Access

Checklist

[x ] I'm reporting a broken site support
[x ] I've verified that I'm running youtube-dl version 2020.02.16
[x ] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] Command-line args: [u'-v', u'https://www.cbs.com/movies/star-trek-first-contact/iTGmLzm3gQN57pAqWamGAFpjatqCYgs_/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.01.24
[debug] Python version 2.7.12 (CPython) - Linux-4.4.0-18362-Microsoft-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 2.8.15, ffprobe 2.8.15
[debug] Proxy map: {}
[generic] iTGmLzm3gQN57pAqWamGAFpjatqCYgs_: Requesting header
WARNING: Falling back on generic information extractor.
[generic] iTGmLzm3gQN57pAqWamGAFpjatqCYgs_: Downloading webpage
[generic] iTGmLzm3gQN57pAqWamGAFpjatqCYgs_: Extracting information
ERROR: Unsupported URL: https://www.cbs.com/movies/star-trek-first-contact/iTGmLzm3gQN57pAqWamGAFpjatqCYgs_/
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/generic.py", line 2375, in _real_extract
doc = compat_etree_fromstring(webpage.encode('utf-8'))
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/compat.py", line 2551, in compat_etree_fromstring
doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/compat.py", line 2540, in _XML
parser.feed(text)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
raise err
ParseError: syntax error: line 1, column 0
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
ie_result = ie.extract(url)
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/generic.py", line 3350, in _real_extract
raise UnsupportedError(url)
UnsupportedError: Unsupported URL: https://www.cbs.com/movies/star-trek-first-contact/iTGmLzm3gQN57pAqWamGAFpjatqCYgs_/

Description

Have an account with CBS All Access (USA).  Passing my username and password as arguments when trying to download the movie.  The website does not require me to pass username and password as arguments when downloading TV shows.

I don't think it's CBS All Access per se, as I can download CBS All Access TV shows just fine, like Star Trek Picard. I think it's the new movies they have added, where the movie URLs have a different format than the TV show URLs:
https://www.cbs.com/shows/star-trek-picard/video/OByehBzRg7bOQxNbekLFzyoBLQ0Fwbv5/star-trek-picard-et-in-arcadia-ego-part-2/
https://www.cbs.com/movies/bon-voyage-charlie-brown-and-dont-come-back/vGYMDWBAZ_9R7_mW1fyMxbhtEpRA1w_h/
In the movie link, the title is in the middle of the link, while the tv show episode title is at the end of the link.
It may just need an extra check of the URL for movies and parse it differently.


Docker Release

SQL-121 Prepare for public Docker Release
Please review the docs page here: https://github.com/yetanalytics/lrsql/blob/docker_release/doc/docker.md
Note that the current image is pushed to yetanalytics/lrsql:latest so you can follow along with the doc instructions.

dig the pass-through env approach. just sidesteps everything. everything looks good but one question - is there a feasible path to PEM or keystore injection, or to authority config?

See https://github.com/yetanalytics/lrsql/blob/docker_release/doc/docker.md#customization


McJsonEscapeString (overzealous)

Hi!
Very-very thanks for this feature, but it's very overzealous as it also converts all accented characters, which isn't necessarily lucky since json is a human-readable/modifiable format (It is used by translators, for example).
So, I think this code uneccesary:
if ((Integer(c) <  32) or (Integer(c) > 126)) then
   Result := Result + ESCAPE + 'u' + IntToHex(Integer(c), 4)

But if you need this, I suggest more parameters, for example:
function McJsonEscapeString(const aStr: string; const aEscapeNonEnglishChars: boolean = False): string;
...
      begin
        if  (integer(c) < 32) or ((integer(c) > 126) and aEscapeNonEnglishChars) then
          Result := Result + ESCAPE + 'u' + IntToHex(integer(c), 4)
        else
          Result := Result + c;
      end;
...

Hi, new information, I searched and found these two functions:
https://www.freepascal.org/docs-html/fcl/fpjson/stringtojsonstring.html
https://www.freepascal.org/docs-html/fcl/fpjson/jsonstringtostring.html
These work exactly as I want and only convert the characters you really need, leaving the json completely human readable and editable (for the translators, in my case). It doesn't even convert the "/" character, or only if you specifically ask it to, see: strict parameter.
But since your function library supports several programming environments, you obviously cannot solve it with fpc functions. But this is a good example of the fact that only really necessary conversions are needed, the official freepascal function shows this.
Thank you!

I think I can improve McJsonEscapeString and McJsonUnEscapeString with this Strict parameter.

Thank you!
For me, it would be best if it worked exactly like the two functions from the fpjson unit.
So there should be a (easiest) level (json human read/write level) where you only work with escape as:
\ "  #8 #9 #10 #12 #13
If this is the Strict:= false; then for me its okay.
Just theory (preserve the capabilities of your code):
type TMCJsonEscapeLevels=(elNormal, elStrict, elForce)
Normal level escape see above.
Strict level escape (same as with above fpc unit), include it: /
Force level escape, when escape almost all (as works now your library).

Hi,
Just checking:

elNormal: \ "  #8 #9 #10 #12 #13 <-> \\ \" \b \t \n \f \r
elStrict: \ " / #8 #9 #10 #12 #13 <-> \\ \" \/ \b \t \n \f \r
elForce: \ " /  #8 #9 #10 #12 #13 #u000X <-> \\ \" \/ \b \t \n \f \r \u000X

I think I can do this using a new constructor as a way to pass a TMCJsonEscapeLevels parameter as RAII.
Also, I think I can automatically use McJsonEscapeString and McJsonUnEscapeString with AsString get/set in order to improve this functionality, as you requested before.
I'll try.

Thanks, that sounds good, just a note, these levels refer to the escape function, unescape can always be completely (force) like fpjson.JSONStringToString works, or it can be controlled, for example:
type TMCJsonEscapeLevels=(elNormal, elStrict, elForce);

function McJsonEscapeString   (const aStr: string; const aStrict: TMCJsonEscapeLevels = elNormal): string;
function McJsonUnEscapeString (const aStr: string; const aStrict: TMCJsonEscapeLevels = elForce): string;

Hi,
Right, I need this elNone level type as a backward compatibility (the user is responsible to escape strings).
Also, since UnEscape always do a complete verification, I'll leave McJsonUnEscapeString as it is now.
The AutoEscape property idea will be at first a new constructor, in order to permit these kind of examples (this make sense for you?):
var
  JNone, JNormal, JJson, JForce: TMcJsonItem;
begin
  JNone := TMcJsonItem.Create;              //elNone is the default
  JNone.AsJSON := '{"path":"C:\\Windows"}'; //or use McJsonEscapeString
  IsEqual := ( JNone['path'].AsString = 'C:\\Windows' );

  JNormal := TMcJsonItem.Create(elNormal);
  JNormal.AsJSON := '{"path":"C:\Windows"}'; //internally = '{"path":"C:\\Windows"}'
  IsEqual := ( JNormal['path'].AsString = 'C:\Windows' );

  JJson:= TMcJsonItem.Create(elNormal);
  JJson.AsJSON := '{"json":"{"1": 123}"}'; //internally = '{"json":"{\"1\":123}"}'
  IsEqual := ( JJson['json'].AsString = '{"1":123}' );

  JForce:= TMcJsonItem.Create(elForce);
  JForce.AsJSON := '{"force":"a#13bç"}'; //internally = '{"force":"a\nb\u00E7"}' considering CP1252
  IsEqual := ( JForce['json'].AsString = 'a#13bç' );
end;

AsJSON and AsString Getters and Setters are the focus here.
One problem about "auto escaping" is that all the chars position might change when there are insertions, so the error messages about parsing might have wrong at pos "%s" compared to the original string.
What I don't get yet is how see escaped strings within levels {elNormal, elStrict and elForce};
Revising all this subject I've realized that McJsonUnEscapeString is ignoring "invisible" chars like #13. I think the right behavior might be:
IsEqual := ( McJsonUnEscapeString('\b\t\r\f\n') = #8#9#10#12#13 );

Thank you for your work, I read it about 3 times, but I still don't fully understand it. :)
I wrote earlier that you need escape/unescape ONLY for storage (file). I understand that immediately storing the escape values in the structure can be problematic, but you see this better.
As I wrote before, what if escape/unescape was activated only when saving/loading?
Then, during saving, strings of type string would be escaped. When loading, the reverse would happen. In this way, the internal structure would basically not change(!), and later modifications could be simpler. (I don't know if your library currently knows whether a value is a string or not. Maybe that's why you linked it to AsString as getters/setters?)
But if you think that what you wrote can be solved completely safely, for example: "JForce:= TMcJsonItem.Create(elNormal);" with AsString getter/setter, that makes perfect sense to me. However you solve it, I think I'll be able to use it. If you release a test version, I will try it. Thank you!

Hi,
It is confusing, indeed (I was looking fpjson.pp).
I was thinking beyond just escape/unescape when saving/loading.
I'm trying to improve AsJSON and AsString getters and setters, what will cover saving/loading tasks too. Internally, all values are stored as strings (to keep the class simpler).
To sum up, I'm trying to make users' experience easier, like in the path example
Also with:
  JJson:= TMcJsonItem.Create(elNormal);
  JJson.AsJSON := '{"json":"{"i": 123}"}'; //internally = '{"json":"{\"i\":123}"}'
  IsEqual := ( JJson['json'].AsString = '{"i":123}' );

This example will also work if you are reading an unescaped fle:
  JJson:= TMcJsonItem.Create(elNormal);
  JJson.LoadFromFile('unescaped.json'); // UTF-8 content like {"json":"{"i": 123}"}
  IsEqual := ( JJson['json'].AsString = '{"i":123}' );

But, my main concern is that an unescaped JSON file with content like {"json":"{"i": 123}"} or {"path":"C:\Windows"} is also a INVALID JSON file. McJson will not write invalid files, so why it need to read them?
I think this leads me to focus efforts only on AsString like this:
  JNormal := TMcJsonItem.Create(elNormal);
  JNormal.Add('path1').AsString := 'C:\Windows'; //auto-escaped to '{"path1":"C:\\Windows"}'
  // or
  JNormal.S['path2'] := 'C:\Users'; //auto-escaped to '{"path2":"C:\\Users"}'
  IsEqual := ( JNormal['path1'].AsString = 'C:\Windows' );
  IsEqual := ( JNormal.S['path2'] = 'C:\Users' );

Could you share again an example of what file are you dealing with and how do you want to use McJson to read it?

Since your json library is suitable for handling any json file, I don't really understand the request, but I'll show you an example naturally. By giving suggestions, your library develops and learns more and more.
In the active project, I generate a json file completely from scratch to make it transparent and modifiable. But this is only file-level data (write/read) because I basically store (and works) the data in an internal structure:
type
   TCustomLanguageItemList = specialize TFPGMap<integer, TLanguageItem>;

The reason for this is, for example, that it is necessary to sort them in ascending order by key (I didn't see a sort option in your library).
Simplified (!) examples (simplified, truncated examples so that it is not too long and unnecessary for you. So if something seems wrong here, it works for me in the real code.):
About 20000 keys:
{
  "200": {
    "OrgSrc": "Value",
    "ModSrc": "Value",
    "OrgDst": "Value",
    "ModDst": "Value",
    "Info": ""}
}

var
  JsonItem, JSonDB: TMcJsonItem;

    JSonDB.LoadFromFile(aFileName)

    for i := 0 to JsonDB.Count - 1 do
    begin
      JsonItem := JsonDB.Items[i];

      // Process JsonItem...
    end;

uses fpjson,

procedure TLanguageItem.SetDataFromJsonItem(const aJsonItem: TMcJsonItem);
begin
  Src := JSONStringToString(aJsonItem.Values[ltSrc.NameStr].AsString);
  Dst := JSONStringToString(aJsonItem.Values[ltDst.NameStr].AsString);
end;

procedure TLanguageItem.SetDataToJsonItem(const aJsonItem: TMcJsonItem);
begin
  aJsonItem.Values[ltSrc.NameStr].AsString := StringToJSONString(Src);
  aJsonItem.Values[ltDst.NameStr].AsString := StringToJSONString(Dst);
end;

function TLanguageItemList.ImportFromJsonFile(const aFileName: string): boolean;
begin
  Result := False;

  Json := TMcJsonItem.Create;
  try
    if FileExistsUTF8(aFileName) then
      Json.LoadFromFile(aFileName)
    else
      exit;

    Self.Clear;

    for i := 0 to Json.Count - 1 do
    begin
      JsonItem := Json.Items[i];
      Index := StrToInt(JsonItem.Key);

      for lt in TLanguageTypes do
      begin
        Value := JSONStringToString(JsonItem.Values[lt.NameStr].AsString);

        // mac (you): I use core function, because it can be imported from three sources: json, text, binary.
        ImportCore(lt, Index, Value);
      end;
    end;

finally
    Json.Free;
  end;
end;

function TLanguageItemList.ExportToJsonFile(const aFileName: string): boolean;
begin
  Result := False;

  Json := TMcJsonItem.Create;
  try
    for i := 0 to Self.Count - 1 do
    begin
      IndexStr := IntToStr(Self.Data[i].Index);
      JsonID := Json.Add(IndexStr);

      for lt in TLanguageTypes do
      begin
        JsonID.Add(lt.NameStr).AsString := StringToJSONString(Self.Data[i].GetItem(lt));
      end;
    end;

    Json.SaveToFile(aFileName);

  finally
    Json.Free;
  end;
end;

Please, edit your previous post, focus into the JSON tiny example (related to a 20,000 keys file), in order to see the necessity of using JSONStringToString.

As you requested.

Hi, about 20,000 keys with texts, a lot of json forbidden characters are included, another project has many more keys and texts. I don't think I knew what you were asking exactly. If I don't use text-json conversion, the text will surely be wrong somewhere, which is why I used the json text conversion procedure.
I replaced all the StringToJSONString/JSONStringToString functions with yours (McJsonEscapeString (jetNormal) / McJsonUnEscapeString) and it works fine, thank you!
ps.: Just a note, I've been getting a message from your library when compiling for quite some time:
 pascal McJSON.pas(183,14) Warning: An inherited method is hidden by "ToString:AnsiString;"

Just a note again, as I know you pay attention to the speed, as I measure the time of the entire processing, it quickly became apparent that your json-text conversion functions are a little slower than those of the fpsjon unit, the total processing time (nothing changed apart from these functions) fpjson : 0.58 secs, with MCJson 0.87 secs.

Hi,
Thanks for the reply.
In order to fix the warning related to ToString in Lazarus you can change it to override. I can't do this because it will fire an error with older versions of Delphi. But thanks anyway. I'm also a "no warning" guy. ;)
About the performance, I'll take a second look at fpjson.pp functions and try to figure it out some point of optimization, but McJson*EscapeString functions are quite simple indeed compared to fpjson.pp.
Regards,

Thank you, its works for me:
function ToString: string; {$IFNDEF FPC} overload; {$ELSE} override; {$ENDIF} // Supress fpc compiler warning...

Thanks for everything!

Hi!
I was avoiding IFNDEF but this is a special case.
I'll revise my goals here.
Thanks.


implementation of get-predeployed-accounts function

Usage related changes

Implementation of get-predeployed-accounts function, which return an array of objects, containing the data of all  pre-deployed accounts (which are: address, initial_balance, private and public keys).

Development related changes

Checklist:

[x] I have formatted the code
[x] I have performed a self-review of the code
[x] I have rebased to the base branch
[x] I have documented the changes
[x] I have updated the tests
[x] I have created a PR to the plugin branch of starknet-hardhat-example.

Fix bug #125


[QA] Teleporting to 'Dialogic' World Leads to an Empty Scene

Build version:

not version specific

Issue Description:
In the EA Explorer, when teleporting to the world 'Dialogic', it does not load after the teleportation. A empty scene with 'QuestAPI critical error' modal appears. However, the world loads fine in the Old Explorer.
STR:

Launch the EA Explorer.
Teleport to the world using /world dialogic command.
Observe that the world does not fully load, showing the 'QuestAPI critical error'.
Open a browser and go to: Dialogic World.
Observe that the world loads as expected.

❌ Actual Result:
In the EA Explorer, the world fails to load and shows the 'QuestAPI critical error' modal.
https://github.com/user-attachments/assets/b3952d45-de5c-4c3f-ada7-96d7bcd300d7
✔ Expected Result:
The world should load successfully in the EA Explorer, similar to how it functions in the Old Explorer.
https://github.com/user-attachments/assets/7f6fd416-816d-4635-99e4-9e0df74c53c9
Repro Index:
10/10 attempts
Additional Notes:
Windows 11. 12th Gen Intel(R) Core(TM) i7-12650H 2.30 GHz. nvidia geforce RTX
Macbook M1 Pro Sonoma 14.5

Still happening on latest version (0.49.0)


add long click event on a certain day

Unable to add long click or touch listener on a particular day.

my fork can do this using OnDatePickedListener
https://github.com/Marchuck/material-calendarview
see branch feature/on-date-picked-listener

Duplicate of https://github.com/prolificinteractive/material-calendarview/issues/335. Closing


IA-2509: Upgrade material ui

Remove reference to material ui 4 and implement MUI 5
Related JIRA tickets : IA-2509
Self proofreading checklist

[x] Did I use eslint and black formatters
[x] Is my code clear enough and well documented
[x] Are my typescript files well typed
[ ] New translations have been added or updated if new strings have been introduced in the frontend
[ ] My migrations file are included
[ ] Are there enough tests
[ ] Documentation has been included (for new feature)

Changes

upgrade bluesquare-components to use MUI5
rewrite SnackBarContainer to a hook in TSX
replace all occurrences of @materila-ui
small css fixes

How to test
Cypress run
Delete your node modules or webpack container container / image
Run npm ci or docker-compose build
Make sure is working, try to go a bit everywhere
Notes
⚠ This is using a branch of bluesquare-components

Screencast from 2023-11-10 11-49-01.webm
Screencast from 2023-11-10 11-50-01.webm
@beygorghor  on these two pages there is a problem and I think is related to the upgrade

Screencast from 2023-11-10 11-49-01.webm Screencast from 2023-11-10 11-50-01.webm @beygorghor on these two pages there is a problem and I think is related to the upgrade

@hakifran this is fixed now

@beygorghor there is another style issue but this one is not blocking


Development: nautobot-server run server does not work using poetry run

Environment

Python version:
Nautobot version: 1.0.0a2

Steps to Reproduce

Install nautobot using poetry install
Try to run poetry run nautobot-server runserver <IP_ADDRESS>:8000

Expected Behavior
Dev server should start.

Observed Behavior
$ poetry run nautobot-server runserver <IP_ADDRESS>:8000 --insecure
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jathan/sandbox/src/nautobot/nautobot/core/cli.py", line 55, in main
    run_app(
  File "/Users/jathan/sandbox/src/nautobot/nautobot/core/runner/runner.py", line 268, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
    super().execute(*args, **options)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 96, in handle
    self.run(**options)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 103, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/utils/autoreload.py", line 620, in run_with_reloader
    exit_code = restart_with_reloader()
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/utils/autoreload.py", line 246, in restart_with_reloader
    args = get_child_arguments()
  File "/Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/lib/python3.9/site-packages/django/utils/autoreload.py", line 233, in get_child_arguments
    raise RuntimeError('Script %s does not exist.' % py_script)
RuntimeError: Script nautobot-server does not exist.

If I run poetry shell first to enter the virtualenv and then follow that with nautobot-server ... it works as expected.
$ poetry shell
Spawning shell within /Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9

$ . /Users/jathan/Library/Caches/pypoetry/virtualenvs/nautobot-b2ttWva6-py3.9/bin/activate

$ nautobot-server runserver <IP_ADDRESS>:8000 --insecure
Performing system checks...

System check identified no issues (0 silenced).
February 24, 2021 - 18:59:51
Django version 3.1.7, using settings 'nautobot_config'
Starting development server at http://<IP_ADDRESS>:6789/
Quit the server with CONTROL-C.
nautobot     INFO     Nautobot initialized!

Hmm this was not yet fixed. I think I must have tagged the wrong ticket.

The root cause for this is when it attempts to use the auto-reload functionality because it can't find the root path of the script and determine the files needed to reload on change.
If you pass the --noreload flag, it works:
$ poetry run nautobot-server runserver <IP_ADDRESS>:8000 --insecure --noreload
Performing system checks...

System check identified no issues (0 silenced).
March 19, 2021 - 21:52:21
Django version 3.1.7, using settings 'nautobot_config'
Starting development server at http://<IP_ADDRESS>:8000/
Quit the server with CONTROL-C.```

Related Poetry issue: https://github.com/python-poetry/poetry/issues/2435


Do not run bower after install automatically

This should be an user action, not something automatically run on install.
We're using ui-select in a browserify project, and we don't need/have bower installed. Furthermore it pollutes the development folder with an unused bower_components.

If someone sees this, please tell us if merging is an option.

@brianfeister can we please get this merged? If you want to keep the bower install for development purposes you can add it to prepublish, then it's installed when you do a local npm install.

Not even k-lange... NPM should not rely on bower, it's a malpractice on the long term ! As if apt-get was relying on yum or pacman...

Please please please can you finally merge that one ???

+10

💖

PRAISE THA LORD

:+1:


Antibody diffusion models

Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures
https://openreview.net/forum?id=jSorGn2Tjg
Generative Diffusion Models for Antibody Design, Docking, and Optimization
https://doi.org/10.1101/2<IP_ADDRESS>9190

Thanks @agitter , I'll go over WE


Drop all references to change_message from zuul vars

After a new update, the change_message is appearing in more fields. As
observed recently [1], if the first comment on the PR contains some mock
ansible, if that is saved into the zuul_vars, it'll crash when trying to
load them. This change drops the newly introduced vars 'build_refs' and
'buildset_refs', whihc contain the change_message.
[1] https://github.com/openstack-k8s-operators/ci-framework/pull/1879
As a pull request owner and reviewers, I checked that:

[x] Appropriate testing is done and actually running

/approve


some improvements and new features

I did some improvements and new features on the tail:

add a abstract SequenceQueue which control the element of queue was produce one by one: next will be process only if previous done. This also fix a bug of edge case.
add feature like tail -F in linux/unix, which can handle relink, rotate, rename.
can tail a file which not exists at first.
reserve file descriptor to avoid reopen file repeatedly
unwatch can return the current position. it's very useful for some case some application want to resume from last position after transient shutdown

Old PR. Archive.


Building from source "undefined reference to `cublasGemmStridedBatchedEx' "

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Linux Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary):source
TensorFlow version:1.15
Python version:3.5.12
Installed using virtualenv? pip? conda?: pip install
Bazel version (if compiling from source): 0.29.1
GCC/Compiler version (if compiling from source): GCC-5.4.0
CUDA/cuDNN version: CUDA 9.1.85 cuDNN 7.0.5
GPU model and memory: Tesla M4 4Gb

Describe the problem
Provide the exact sequence of commands / steps that you executed before running into the problem
You have bazel 0.29.1- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python
Found possible Python library paths:
/usr/lib/python3/dist-packages
/usr/local/lib/python3.5/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]
/usr/local/lib/python3.5/dist-packages
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: y
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Do you wish to build TensorFlow with TensorRT support? [y/N]: n
No TensorRT support will be enabled for TensorFlow.
Found CUDA 9.1 in:
/usr/local/cuda/lib64
/usr/local/cuda/include
Found cuDNN 7 in:
/usr/local/cuda/lib64
/usr/local/cuda/include
Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 5.2]: 5.2
Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl            # Build with MKL support.
--config=monolithic     # Config for mostly static monolithic build.
--config=ngraph         # Build with Intel nGraph support.
--config=numa           # Build with NUMA support.
--config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
--config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws          # Disable AWS S3 filesystem support.
--config=nogcp          # Disable GCP support.
--config=nohdfs         # Disable HDFS support.
--config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished
$bazel build --verbose_failures --action_env=LD_LIBRARY_PATH=/usr/local/cuda/lib64 --config=cuda --config=opt //tensorflow/tools/pip_package:build_pip_package --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --config=nonccl --config=nohdfs --config=mkl --config=monolithic --config=noaws --config=nogcp --config=monolithic
Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Target //tensorflow/tools/pip_package:build_pip_package failed to build
[19,101 / 21,282] checking cached actions
INFO: Elapsed time: 1270.016s, Critical Path: 336.18s
[19,101 / 21,282] checking cached actions
INFO: 10667 processes: 10667 local.
[19,101 / 21,282] checking cached actions
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
I just search all the issues about "undefined reference to `cublasGemmStridedBatchedEx'", but I believe mine is different.

hello, I think the reason of this issue has already been figured out. The CUDA version of my system is 9.1, however tensorflow v1.15 requires at least CUDA 10.  More details could be found in https://www.tensorflow.org/install/source#common_installation_problems
Now I wonder why I cannot build tensorflow 1.15 from source code in my system environment, yet I could use pip to install tensorflow 1.15 or 2.0 version?

Today I tried to build tensorflow 1.14 with CUDA 9.1 and CUDNN 7 and my build failed with same error:
ERROR: /opt/tensorflow/tensorflow/tensorflow/contrib/reduce_slice_ops/BUILD:50:1: Linking of rule '//tensorflow/contrib/reduce_slice_ops:gen_reduce_slice_ops_py_wrappers_cc' failed (Exit 1)
bazel-out/host/bin/_solib_local/_U_S_Stensorflow_Scontrib_Sreduce_Uslice_Uops_Cgen_Ureduce_Uslice_Uops_Upy_Uwrappers_Ucc___Utensorflow/libtensorflow_framework.so.1: undefined reference to `cublasGemmBatchedEx'
bazel-out/host/bin/_solib_local/_U_S_Stensorflow_Scontrib_Sreduce_Uslice_Uops_Cgen_Ureduce_Uslice_Uops_Upy_Uwrappers_Ucc___Utensorflow/libtensorflow_framework.so.1: undefined reference to `cublasGemmStridedBatchedEx'
collect2: error: ld returned 1 exit status
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 813.405s, Critical Path: 96.34s
INFO: 4817 processes: 4817 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

For anyone facing the same issue - I was using nvidia docker container with CUDA 9.1 and CUDNN 7. I had to add following step before building tensorflow:
echo "/usr/local/cuda-9.1/targets/x86_64-linux/lib" > /etc/ld.so.conf.d/cuda_targets.conf
ldconfig

For whatever reason /usr/local/cuda-9.1/targets/x86_64-linux/lib was not there.

Hello,
I can confirm the same behavior while attempting to build 1.14 with CUDA coming from either Ubuntu 18.04 or Debian 9 with backports.
Any help would be greatly appreciated,
Regards, Adam.

Hi There,
We are checking to see if you still need help on this, as you are using an older version of tensorflow which is officially considered end of life . We recommend that you upgrade to the latest 2.x version and let us know if the issue still persists in newer versions. Please open a new issue for any help you need against 2.x, and we will get you the right help.
This issue will be closed automatically 7 days from now. If you still need help with this issue, please provide us with more information.


[NNPA] Support 4D in zhigh.concat

Stickifications for 4D and NHWC are the same, so we can support zhigh.concat with mix of 4D and NHWC stickified tensors.
Signed-off-by: Tung D. Le<EMAIL_ADDRESS>

LGTM

Jenkins Linux amd64 Build #5382 [push] [NNPA] Support 4D in zhi... started at 20:15

Jenkins Linux s390x Build #5398 [push] [NNPA] Support 4D in zhi... started at 21:15

Jenkins Linux ppc64le Build #4519 [push] [NNPA] Support 4D in zhi... started at 21:19

Jenkins Linux ppc64le Build #4519 [push] [NNPA] Support 4D in zhi... passed after 1 hr 3 min

Jenkins Linux s390x Build #5398 [push] [NNPA] Support 4D in zhi... passed after 1 hr 3 min

Jenkins Linux amd64 Build #5382 [push] [NNPA] Support 4D in zhi... passed after 1 hr 13 min


Need example with unit testing

The examples on testing seem to suggest that the function being defined, say foo should be nested within it's corresponding unit test, say test_foo. This seems weird. I would have expected unit tests to be written either in a separate file or in a separate function, but nesting the definition within the test feels off. Can you please add a basic unit test to one of the examples for more clarity?

All that's required is moving the command definition above the test definition.
Marking this as beginner friendly. Please feel free to submit a PR!

Right, and then how do you execute all your defined tests in one go?

However the test framework you're using (unittest, pytest, etc.) does.

I'm assuming that the examples in question are these:
https://click.palletsprojects.com/en/7.x/testing/

@dposada looks like #1130 already addressed this issue

I went ahead and created #1330. Based on the original poster's confusion, I think it makes sense to give examples that show the source code in a separate module from the test code. Let me know what you think.