[DSIG] Mistake in Format 1:, d: delete "Zero out the file checksum in the 'head' table"

You can look in this thread, for example. https://lists.aau.at/pipermail/mpeg-otspec/2016-December/000837.html

a) update the spec to match the implementation (i.e. remove the line "Zero
out the file checksum in the head table.", as it is not actually
performed by signcode.exe + mssipotf.dll, as I and Hin-Tak have repeatedly
explained);

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: acf0601c-2dd5-c94c-8860-07e9def0c20c
Version Independent ID: c69a8340-3140-31cf-d27b-c0e2b235ccc6
Content: DSIG — Digital signature table specification - Typography
Content Source: typographydocs/opentype/spec/dsig.md
Product: typography
GitHub Login: @PeterCon
Microsoft Alias: PeterCon

Thank you for the feedback. This issue will be triage and addressed in the version of the OpenType specification.

@PeterCon It would be nice to understand what is the right thing here because "Zero
out the file checksum" should be done, no? Or is Authenticode not caunting on that part of binary that is that checksum?

@ValZapod : As you point out, this issue has been lingering for some time. I will endeavour to find some resolution to this soon. It may require getting input from several platform or tool vendors.

Or we can just do @HinTak, please could you comment how to word it so we can close it ASAP?

According to my notes and slides I gave a talk on 2016, the current official signing implementation (ie Microsoft's) does not zero out the checksum. So to adjust the spec to the De facto implementation, the 4th item "zero out the file checksum..." should be removed. (with perhaps a note saying it used to be mistakenly there).
About ttc header, the bulk of many old Microsoft shipped ttc also have a version 1 header rather than version 2. So that's another issue where the De facto signing implementation (does not reject v1 header) does not match the spec (which only states v2).

does not zero out the checksum

So it does not use those bytes of checksum in its calculations of RSA md5 signature?? How does it work? Because if it is not zeroed, and then rewritten, there is no way to check the signature as the file was modified (with the new checksum).

Huh? The checksums are per table. The signature is based on all the tables plus checksum fields (with valid content), rather than all the tables minus the checksum fields (with zero'ed content).
Thus the signature also certifies the checksums, rather than excluding them. Clear?

on all the tables

No, only head table. From spec:

Zero out the file checksum in the 'head' table

Does DSIG table itself has a checksum and if yes, how it is done?

You need to read the spec - the head table stores the checksum of all the other tables.

all the other tables

Including DSIG? But then it means checksum of checksums that is head table should be AGAIN recalculated and zeroed. Again, what it is used to calculate md5?

No, excluding DSIG - since it did not exist before signing. You can read the code in the DSIGInfo directory in FontVal (which contains an implementation of verifying signature), if you don't understand the spec.

So it also does not "Recalculate the checksum in the 'head' table."? Can you give a pull request maybe.

?? @ValZapod  Look: it was discussed at quite some length 4 years ago. I think the people involved understand the issue well enough.

Haha, well, you did not manage to modify the Microsoft docs in these 4 years or do an opensource implemenation and otf still have md5 hashes inside, pathetic, everybody is usg sha2 already. Again, you either zero out and recalculate or not zero out and not recalculate. Or 3, somehow md5 digest is not counting that checksum. What is it?

@PeterCon was in on the discussion 4 years ago, and he is on this now. I have nothing to add beyond what I already wrote, and therefore prefer not to respond to this thread.

@HinTak : Can you remind me where this was discussed 4 years ago? (Sorry, I was removed from work in this area two years ago and have lost track of such details.)

@PeterCon the beginning of this post seems to be part of a series of exchanges from an archive of the mpeg otspec formerly on yahoo lists. Somebody from Microsoft (at the bottom of https://lists.aau.at/pipermail/mpeg-otspec/2016-December/000837.html , and I believe more than one person too) did respend but it did not go anywhere further.
As I wrote earlier -
https://github.com/MicrosoftDocs/typography-issues/issues/455#issuecomment-679192579 - from my slides of a talk I gave in 2016, I would suggest removing the 4th clause "zero out the file checksum..." of the item group of 5, as this is how the actual de facto implementation behaves; also the spec says a v2 ttc header contains fields for DSIG, but many older ttc shipped by Microsoft does not do this (i.e. the ttc has a v2-like header where the version field is 1 instead of 2). The latter is more a laxness in the signing tool not refusing to sign a non-conformant font. So both of these issues are about the spec not agreeing with the signing tool.
However, as you know, the de facto verification implementation is also in that microsoft signing dll, it accepts what it signs :-).

I've marked this as a P1 issue and have asked MS to provide clarification.

@PeterCon yes, I think that the proper course of action: find somebody within Microsoft who has access to the source code of the signing/verifying opentype authenticode dll, have a look and verify this claim, and either fix the signing/verifying tool or fix the spec. At this stage, after so many fonts have been signed for almost two decades, I think it is the spec that needs to be fixed. However, the first step (hunt down the source within Microsoft and get a tech opinion on it) must be done by a Microsoft staffer - outside parties can only keep poking... And I have done my share of poking a few years ago so am a bit tired of poking MS people every few months ...

Can't we just deprecate DSIG table completely and mark it for future removal?

Can't we just deprecate DSIG table completely

Does the word stable API means anything to you? Wow. DSIG, even incorrect, is what is used to activate otf styles sets or variables or ligartures. Also we can only used signed files in bootloader. And we should use everything signed with SHA2.

@ValZapod I think this is valuable information. Can you clarify the applications or environments in which DSIG is a factor (i) for activating OT features or (ii) for a bootloader?
And when you say, "I always check" do you mean when you as an individual user download fonts from a site, or are you referring to some software product that retrieves and uses fonts from sites? And what is being used to verify the signature in this scenario?

BTW, I just checked, ISO/IEC 14496-22:2015(E) -- not 2019, I do not have it -- also says that Zero out the file checksum in the head table thing. You need to give that data to ISO/IEC as well, haha.

individual user download fonts from a site

No, when I download pirated fonts packs from our ru-board forum... There are other forums of course. For example, DG META SCIENCE (ala ITC Officina Super family) or my favourites non-free from Latex https://tex.stackexchange.com/questions/425098/which-opentype-math-fonts-are-available

for a bootloader?

Is not Windows Secure Boot EUFI bootloader uses ONLY signed fonts? And not with just a dummy signature. It is not like that in debian, it was discussed that this should be fixed. There is after all some bytecode in otfs as well.

for activating OT features

Hard to say. It is required in standards to activate OTF, so who knows, you can strip DSIG from font and play with it. Word does not care, for example. Adobe though, is prety sensitive about it, especially Illustrator.

I’ll take a look at this, but it may be sometime before I can get back with a response.

Does the word "stable API" mean anything to you?

Yes. I've been maintaining various C libraries that provide ABI/API compatibility indefinitely. I've maintained them for over a decade.

Wow. DSIG, even incorrect, is what is used to activate otf styles sets or variables or ligartures.

That's irrelevant.

Also we can only use signed files in bootloader. And we should use everything signed with SHA2. Ah, yes, do you wanna know how many vulns are there because of malformed otfs? Hundreds. Also, when I download fonts from some less legal places, I always check that signature is correct.

That's redherring. Just sign the font files externally using generic signing tools like you sign your kernel image or any other files. I'm sure your bootloader does not load and verify DSIG.

kernel image

Is a ELF. It has special place for signature and timestamp signature. PE is even more compicated with 2 styles of signatures and timestamp signatures. PDF is again different.  They do not ask to verify signature though or even check if it is there though.
Here you are https://typedrawers.com/discussion/1147/are-empty-dsig-tables-in-a-font-useless

DSIG, even incorrect, is what is used to activate otf styles sets or variables or ligartures.

Gong way back to initial WOFF2 discussions, our decision about removing DSIG from WOFF2 files was partially informed by this message from Sergey Malkin: https://lists.w3.org/Archives/Public/public-webfonts-wg/2014May/0025.html

I looked at the code for mssipotf.dll. The code looks pretty clear both in comments and code that the entire file is hashed to come up with the signature. The same hash function is used in both signing and verification. In the simplest case where the input font has no DSIG table the entire file input file is mapped and the hash is created. In the case where there is an existing DSIG table, a new internal font is created with the DSIG removed with the intention of creating exactly the same as original file so that can be hashed and eventually compared with an encrypted hash in the DSIG table in the case of verification or when signing that internal image has a new signature added based on the new hash.

The same hash function is used in both signing and verification.

It is literally impossible. Signing is rsa signature of md5 digest. And to verify you need to check the whole chain of 3 certificates.

entire file is hashed

We know that. The question is how the entire file is hashed.

The same internal function in mssipotf.dll is used to create the hash in both signing and verification of font files. The simplest case is when the font does not have an existing DSIG to be verified or to be replaced. This case only occurs when you have a previously unsigned font to be signed.
In this simplest case the entire file is passed into CryptHashData() https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-crypthashdata.
The SIP is built on an infrastructure in Microsoft Windows where various file types can be signed. There is build in support for many file types including .exe, .dll, etc. Font files were not originally built into the code so mssipotf.dll was created to extend the signing infrastructure to support .ttf and .ttc files.

CryptHashData() and CryptCreateHash() is just doing an md5. You will need to then sign with RSA that md5. And then maybe get a timestamp from timestamp server from Verisign, dunno (?). It is complicated. Now the point here is that the question is about The checksums of all tables are correct.

It does not appear that the code zeros out the checksum in the 'head' table prior to calling CryptHashData(). Once the hash is created and encrypted and added to the DSIG structure and added to the font then the checksum in the head table is recalculated. The reason this does not break the signature is when the signature is verified, an in memory image is created of the font without the DSIG and the checksum is recalculated before the data is hashed.
Prior to signing, the checksums are verified to make sure they are correct. Signing will be refused if those checks are not successful.

an in memory image is created of the font without the DSIG and the checksum is recalculated before the data is hashed

THANK YOU! What about usFlag? Is it added afyer or before the checksum is recalculated in memory image?

Are you referring bit 11 of the flags field of the head table? If so, I don't see any evidence that is touched during signing or verification.
If you are referring to flags in the DSIG, the DSIG table is "removed" prior to computing the checksum and hash for verification or signing.

Okay

the DSIG table is "removed" prior to computing

What about the header of DSIG? At what point it is added? It is obvious that length of the PKCS#7 packet canot be added before, but the Version number of the DSIG table (0x00000001), number of signatures in the table (0x0001) and usFlag (named as such in https://docs.microsoft.com/en-us/typography/opentype/otspec180/dsig but not in https://docs.microsoft.com/en-us/typography/opentype/spec/dsig, another error) can be added before and so is format of the signature.

The DSIG header is part of the DSIG and added to the font when the rest of the DSIG is added.

@PeterCon Well, now it is obvious that we need to change "Zero
out the file checksum in the head table." to "Recalculate and fill in file checksum in the head table for this stripped binary." and remove "Add the usFlag (reserved, set at 1 for now) to the stream of bytes" whatsover.

Oh, somebody just leaked all source code of windows XP (like 10 hours ago) and I am right now looking into ntgdi\fondrv\tt\mssipotf folder, in particular mssipotf.cpp.
What is this in function OTSIPObjectOTF::VerifyIndirectData?
        // Retrieve the flag field from the DSIG table.
        // These bytes will be used as part of the hash.
        usFlag = pDsigTable->GetFlag ();

        // Compute the hash value for the file and compare it to the hash
        // value in the SIP_INDIRECT_DATA structure.

@paullinnerud

It does not appear that the code zeros out the checksum in the 'head' table prior to calling CryptHashData().

Please clarify: do you mean the head.checksumAdjustment field?

You know you can just google it https://stackoverflow.com/questions/25727592/what-is-checksumadjustment-in-ttf-otf-head-table-used-for and all source code has leaked...

That may be the case. But as I've got many, many issues for the spec to work on in parallel, it makes much more sense to let Paul, who has already interpreted the source code, to clarify this for me than to spend my time ramping up on that code from scratch.

this for me than to spend my time ramping up on that code from scratch

I am prety sure that usFlag is used in Windows XP source code to calculate md5 while he said it is not (in sipobotf.cpp). I also see very strange stuff in signature.cpp that RSA part is not even verified in Windows XP. That is from looking into your leaked code for 5 minutes. I can spend more time. ;)

Thanks. In the meantime, I will continue addressing comments on axis value tables within the STAT table, specifications for an in-preparation extension to the COLR table, processing of GPOS/GSUB nested lookups, and start making sense of complex issues regarding the scale for the 'opsz' design axis and interaction with CSS.

@paullinnerud I think you basically confirmed my findings a few years ago (see earlier comment). I would suggest correcting the spec to match the de facto signing / verification implementation at this point in time, given so many fonts have been signed for two decades...

think you basically confirmed my findings a few years ago

No, we did not. We found out that binary is stripped from DSIG and then head.checksumAdjustment field is recalculated and from that bitstream the digest is done (using md5 in 2020, crazy, though Donald Chinn did say that the functions will need to be changed to support sha256 and others).

@ValZapod are looking at 20-year-old code. The new implementation in FontVal does all the newer digests, because newer version of the signing implementation signs with newer digest algorithms. sha1(or sha256? one of the sha* family) was quite popular, and used by Adobe's fonts a few years ago.
Anyway, please let the Microsoft folks respond. Please do not answer question (and wrongly...) on their behalf.

@ValZapod : I have a recollection that some changes (I don't know what) were made around the Windows 7 timeframe. So, I wouldn't necessarily assume that XP would provide definitive answers.

This commit https://github.com/HinTak/Font-Validator/commit/8d23f52b8e9dedb1c53ad147f9c4bdfd81bbfc7c was created in Apr 2016 and made public in July 2017. It says "support all current valid arguments to HashAlgorithm.Create() - apparently Adobe starts to use SHA256" . So somehow, I have received reports/requests 4 years ago from somebody else that sha256-signed fonts from Adobe are out there.

@peterCon Yes, I meant head.checksumAdjustment.
flags in the DSIG header indicates set to 0 however later it indicated set to 1. The constructor for the object in code sets to 1 and it's never set to anything else.

mssipotf.dll had a major update in 2009 and smaller update in 2012. The last change in 2012 was to support SHA2 signatures according to the checkin log.

The last change in 2012 was to support SHA2 signatures according to the checkin log.

Where? In RSA or in digest? I was talking about the digest.

mssipotf.dll had a major update in 2009 and smaller update in 2012. The last change in 2012 was to support SHA2 signatures according to the checkin log.

Cool - I think I have the 2009 dll's.

IIUC, this revision is what's needed. (The level 2 numbering is from GitHub's markdown, not quite what would be in the spec.) Confirmation from Paul and others requested:

If there is an existing DSIG table in the font,

Remove the DSIG table from font.
Remove the DSIG table entry from sfnt Table Directory.
Adjust table offsets as necessary.
4. Zero out the file checksum in the 'head' table.
5. Add the usFlag (reserved, set at 1 for now) to the stream of bytes

Hash the full stream of bytes using a secure one-way hash (such as MD5) to create the content digest.
Create the PKCS#7 signature block using the content digest.
Create a new DSIG table containing the signature block, and setting the flag field as desired.
Add the DSIG table to the font, adjusting table offsets as necessary.
Add a DSIG table entry to the sfnt Table Directory.
Recalculate the checksum in the 'head' table.

So,  this?

If there is an existing DSIG table in the font,

Remove the DSIG table from font.
Remove the DSIG table entry from sfnt Table Directory.
Adjust table offsets as necessary.
Zero out the file checksum in the 'head' table.Recalculate checksumAdjustment in the 'head' table.
5. Add the usFlag (reserved, set at 1 for now) to the stream of bytes

5 should also still be there. See https://github.com/shaswata56/WindowsXP/blob/ef9fc49453d1788eadf64ece58451605d854e05a/windows/core/ntgdi/fondrv/tt/mssipotf/sign/sign.cpp#L108 usFlag is assigned to pbDsig, then https://github.com/shaswata56/WindowsXP/blob/ef9fc49453d1788eadf64ece58451605d854e05a/windows/core/ntgdi/fondrv/tt/mssipotf/sign/sign.cpp#L117 function HashTTFfile processes &pbHash
In HashTTFfile function, here https://github.com/shaswata56/WindowsXP/blob/ef9fc49453d1788eadf64ece58451605d854e05a/windows/core/ntgdi/fondrv/tt/mssipotf/sign/httffile.cpp#L483 we and I quote "Pump the extra bytes associated with the DSIG table into the hash function".

So, this?

If there is an existing DSIG table in the font,

Remove the DSIG table from font.
Remove the DSIG table entry from sfnt Table Directory.
Adjust table offsets as necessary.
~Zero out the file checksum in the 'head' table.~Recalculate checksumAdjustment in the 'head' table.
~5. Add the usFlag (reserved, set at 1 for now) to the stream of bytes~

Yes, this is basically what I having been suggesting for several years - remove the 4th item. Not sure about the 5th item either way - I think it was a provision for multiple / chained signatures. I have not seen it done.

@PeterCon kind of beating a dead horse - https://github.com/MicrosoftDocs/typography-issues/issues/455#issuecomment-679192579 - I have been suggesting just removing the "zero out checksum..." item for years...

The step "Add the usFlag (reserved, set at 1 for now) to the stream of bytes" should still be there but the flag is added to the hash regardless of if there is existing DSIG or not. It's also added to hash after file data is added.

If there is an existing DSIG table in the font,
a. Remove DSIG table from font.
b.  Remove DSIG table entry from sfnt Table Directory.
c.  Adjust table offsets as necessary.
Hash the full stream of bytes using a secure one-way hash (such as MD5) to create the content digest.
Add the flags (uint16) (reserved, set at 1 for now) to the stream of bytes
Create the PKCS#7 signature block using the content digest.
Create a new DSIG table containing the signature block.
Add the DSIG table to the font, adjusting table offsets as necessary.
Add a DSIG table entry to the sfnt Table Directory.
Recalculate the checksum in the 'head' table.

basically what I having been suggesting for several years - remove the 4th item

You do understand it does not remove the 4th item, right?

Hash the full stream of bytes using a secure one-way hash (such as MD5) to create the content digest.
Add the flags (uint16) (reserved, set at 1 for now) to the stream of bytes

You need to change places for those 2. Hash happens after! What is the point to say add usFlag, if it done after the hash? Hash md5 digest is the only thing that is signed by RSA + timestamp, so those buytes will be lost and you can change that flag while preserving the signature 100%! That is simply wrong.

In the simplest case where the font does not already have a DSIG, the entire font image is sent to CryptHashData() and then the uint16 of the flag is passed to CryptHashData().

So the wording needs to change to make it explicit that both are part of the hash?

does not already have a DSIG, the entire font image is sent to CryptHashData() and then the uint16 of the flag is passed to CryptHashData().

Yes, what is your point? From https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-crypthashdata
hHash

Handle of the hash object.

pbData

A pointer to a buffer that contains the data to be added to the hash object.

needs to change to make it explicit that both are part of the hash?

Yes.

Re Add the flags (uint16) (reserved, set at 1 for now), the spec needs to be clear as to what flags refers to.
If it is the DSIG.flags field, then bit 1 is not reserved, and the current wording in the spec suggests that setting bit 1 is optional. Are we saying that setting bit 1 is required?

Something like this.

If there is an existing DSIG table in the font,
a. Remove DSIG table from font.
b.  Remove DSIG table entry from sfnt Table Directory.
c.  Adjust table offsets as necessary.
Add font data to stream of bytes.
Add the flags (uint16) (reserved, set at 1 for now) to the stream of bytes
Hash the full stream of bytes using a secure one-way hash (such as MD5) to create the content digest.
Create the PKCS#7 signature block using the content digest.
Create a new DSIG table containing the signature block.
Add the DSIG table to the font, adjusting table offsets as necessary.
Add a DSIG table entry to the sfnt Table Directory.
Recalculate the checksum in the 'head' table.

BTW, this area of the code has changed considerably from the original version.

The constructor for the object that manages flags field of DSIG sets it to 1 and it is never changed.

b.  Remove DSIG table entry from sfnt Table Directory.
c.  Adjust table offsets as necessary.

You also forgot about

The reason this does not break the signature is when the signature is verified, an in memory image is created of the font without the DSIG and the checksum is recalculated before the data is hashed.

Now:

object that manages flags field of DSIG sets it to 1 and it is never changed.

Because only 1 is allowed. "permission flags Bit 0: cannot be resigned Bits 1-7: Reserved (Set to 0)"
As I said,

usFlag (named as such in https://docs.microsoft.com/en-us/typography/opentype/otspec180/dsig but not in https://docs.microsoft.com/en-us/typography/opentype/spec/dsig, another error)

The constructor for the object that manages flags field of DSIG sets it to 1 and it is never changed.

But that is one implementation. For the DSIG table to work, is it always necessary for the flag to be set to 1? Or could a font have flags set to 0?

Or could a font have flags set to 0?

Why did you chage usFlag to just flags? Or who was it... It can, moreover they are copied here https://github.com/shaswata56/WindowsXP/blob/ef9fc49453d1788eadf64ece58451605d854e05a/windows/core/ntgdi/fondrv/tt/mssipotf/sign/sign.cpp#L97
Now he just said that it is only 1 in newer versions... so...

Why did you chage usFlag to just flags?

Field names in several tables have been revised in prior versions for consistency in naming conventions.

"If this bit is set to zero (0) the font may have a signature applied over the existing digital signature(s)."
That like with Nvidia drivers dlls, tney have 1 sha1 and one sha2 signature, both have 3 certificates of RSA PKI with 3 certificates for timestamp. That is 12 certificates, all different (they are crazy, I know).

@PeterCon I am okay with what @paullinnerud suggests. As for the value of usFlag, I checked that it was a placeholder for concatenating signatures (for future implementations), zero allowing others to counter-sign until the last person who signs, sets it to 1. After 20 years, no signing implementation offers the functionality of counter-signing, and no verification implementation checks counter-signing, so the usFlag is effectively a reserved value of 1. @paullinnerud is saying that the MS implementation behaves always to write a "last signature", and forbid further counter-signing.

This need to be there too "The reason this does not break the signature is when the signature is verified, an in memory image is created of the font without the DSIG and the checksum is recalculated before the data is hashed."

I am kind-of against adding any description about memory models as it is an implementation details, and does not need to go into a specification. Keep things simple.

It is time for a pull request.
Who is going to submit this info to ISO body?

Once I get clarity on the details, I will prepare a revision for the OT spec, and present the proposed revision here for review. Later it will get submitted to ISO for their consideration.

And it also seems that @paullinnerud is suggesting the verification side of MS implementation does not accept chained/concatenating/counter-signing.

OK that seems to suggest that bit 0 in the flags field should not be optional but is always expected to be set.

If there is an existing DSIG table in the font,
a. Remove DSIG table from font.
b. Remove DSIG table entry from sfnt Table Directory.
c. Adjust table offsets as necessary.
d. Recalculate checksumAdjustment in the 'head' table.
Add font data to stream of bytes.
Add the flags (uint16) (set at 1 for now as no crossign implementation) to the stream of bytes
Hash the full stream of bytes using a secure one-way hash (such as MD5 or SHA256) to create the content digest.
Create the PKCS#7 signature block using the content digest (block in many cases contains timestamping to make the signature valid before the decades long timestamping certificate will expire).
Create a new DSIG table containing the signature block.
Add the DSIG table to the font, adjusting table offsets as necessary.
Add a DSIG table entry to the sfnt Table Directory.
Recalculate checksumAdjustment in the 'head' table.

And it also seems that @paullinnerud is suggesting the verification side of MS implementation does not accept chained/concatenating/counter-signing.

OK that seems to suggest that bit 0 in the flags field should not be optional but is always expected to be set.

Yes, it is reserved as 1 for now, until such times as somebody wants to implement counter-signing, and needs to define how counter-signing should work. This is perhaps similar to how the higher unicode range bits in the OS/2 table - they takes on a default and reserved value of 0 until somebody defines what 1 means. In this case, "1" means "this is the final signature". Nobody has done a "intermediate signature" signing, and the verifying always expects a final signature.

About ttc header, the bulk of many old Microsoft shipped ttc also have a version 1 header rather than version 2. So that's another issue where the De facto signing implementation (does not reject v1 header) does not match the spec (which only states v2).

BTW, @HinTak, what does that mean? TTC is not supported by Windows, are they not? I understand you get rid of atmfd.dll, but TTC was never supported at all, not like with postscript fonts.

Windows have been shipping a few ttc's since about 20 years ago. And they are signed, too.

Windows have been shipping a few ttc's since about 20 years ago. And they are signed, too.

Oh, indeed. Press win and type C:\Windows\Fonts it will actually open the real thing that is under "The Interface". 18 ttc files. Then why they are not supported for instalation the last time I checked?

It's been possible to install TTCs in Windows since TTCs were first defined.

OK, I've drafted revisions. There are enough changes that I'm attaching a PDF for the entire chapter showing the revisions.
I did not mention timestamps since, IIUC, that would be a detail in the signature block covered by the PKCS #9 spec.
As the discussion brought up TTCs, I took a look at the TTC section toward the end and noticed it didn't give instructions about creating a DSIG table for a TTC. So, I listed what I think (IIUC) would be the appropriate adjustments to the steps discussed in the case of a TTC.
Please review.
dsig_184_draft_201001.pdf

Oh, and I also updated the references for PKCS #7/#9: the current links no longer work.

@PeterCon :

there is no need to remove "permission flags" in the description column of the header table . Removing it would look odd, given there is still a whole paragraph below about it.

I would suggest using one of the urls under ietf.org instead of rfc-editor.org for pkcs 7/9. Looks more official. Strangely I have those ftp files in my hard drive (I tend to do downloads with mirroring tools, to keep the original urls) so they were available as written in Sept 2015.

Argh, I see there is some confusion with the permission field in the signature header, and the two bytes which is appended to the end of the file stream during hashing. Both are 0x0001 and both are called *flags but have different meanings. One is a collection of permission bits, which nobody has bothered with implementing or verifying other possible values. The other is just two arbitrary bytes apend to the end of input data, to make the hashing more interesting (... ask your colleagues to explain why those two bytes and not something else...) . Perhaps saying "Append two bytes, 0x00 and 0x01 ..." is better than what is at item 2.

perhaps a reference to ttc header sub-section of Font collection section? I think that section also needs a little clarification: it says "if a digital signature is used, the DSIG table for the file must be the last table in the TTC file". What it means, is that, for TTC, the DSIG table is appended to the TTC font file after signing, and the TTC header's dsigOffset field has a value somewhat near the end of file (minus size of the DSIG table). i.e. For. Signed TTC, the DSIG table is always located at the very ending portion of file. You can try rewriting my words on this in a nicer way.

I have no strong opinion, one way or the other, on the other changes.

The "...last table..." part is "by content location", or "by physical location".

I would suggest using one of the urls under ietf.org instead of rfc-editor.org for pkcs 7/9. Looks more official.

See https://www.rfc-editor.org/ref-example/ for Internet Society guidance on referencing RFCs: URLs are of the form http://www.rfc-editor.org/info/rfc#.

there is no need to remove "permission flags" in the description column of the header table . Removing it would look odd, given there is still a whole paragraph below about it.

I missed that paragraph. If, in fact, bit 0 of the DSIG flags field is going to be required—which is what I was given to understand from the discussion—, then that paragraph is no longer applicable and should be removed.

Argh, I see there is some confusion with the permission field in the signature header, and the two bytes which is appended to the end of the file stream during hashing. Both are 0x0001 and both are called *flags but have different meanings.

As I said earlier, we need to be clear what "flags" being added to the stream refers to. Nobody gave me any clarification, so I took it to mean the same as the only "flags" clearly in the context, namely the DSIG flags field.
If the bytes added to the stream are just that, bytes, then they're not flags; they're just arbitrary bytes.
I appreciate clarification on these points. Thanks.

@PeterConstable sorry, I have switched off and blocked notifications from non-Microsoft staffers, and I also categorically refuse to read comments and observations on stolen source code. I see now that I am confused because there are no non-final signatures out there in the wild. A non-final signature is generated by hashing differently, with a different permission flag appended. In that case I have a history question, perhaps for @paullinnerud or @Gr3gH : what is or was the use-case / business case for a non-final signature? Given that font vendors almost always want to ship and release fonts as final-signed, if signed at all.

If the bytes added to the stream are just that, bytes, then they're not flags; they're just arbitrary bytes.

They are bytes of flags, previously usFlag.
BTW, I do not understand how counter-signing even is supposed to work. Changing the flag from 0 to 1 will change the first signature, will it not?

there is no need to remove "permission flags" in the description column of the header table

OK, I got further clarification from Paul: the "flags" appended to the stream are the same flags to be written to DSIG.flags in step 5, or (when validating a font's signature) that are read from the font's DSIG.
So, here's an updated draft in which I

restore the permission flags description
revise step 2 (flags value is being selected before the DSIG is created)
revise the paragraph after the steps to clarify the validation process wrt flags

dsig_184_draft_201002_diff-from-183.pdf

perhaps a reference to ttc header sub-section of Font collection section? I think that section also needs a little clarification:

OK, I should have done this before uploading the draft in the previous comment: I've added the following at the end of the section on signatures for TTCs:

See the TTC Header section of the OpenType Font File chapter for related information.

And in that other section of the OTFF chapter, I've revised this paragraph:

If a digital signature is used, the DSIG table for the file must be the last table in the TTC filelocated at the end of the TTC file, following any other font tables. Signatures in a TTC file are expected to be Format 1 signatures.

@PeterCon

OK, I got further clarification from Paul: the "flags" ...

revise the paragraph after the steps to clarify the validation process wrt flags

Thanks a lot for the clarification and revisions. It is all good to me now. I have not seen a non-final signature in the wild, but I'll know one what I see one in the future. It is also great that you have condensed my rambling on the ttc header into a neat and clear change.
FWIW, the code in
https://github.com/HinTak/Font-Validator/tree/master/DSIGInfo is an interface-compatible drop-in replacement to mssipotf (its dotnet exposure), somewhat incomplete, but cross-platform and works on linux/mac os too. It was written by just looking how shipping fonts' signatures look like, assuming officially released fonts have valid signatures. So I have not seen a non-final signature. When FontVal was open-sourced by Microsoft, mssipotf (which was always bundled with FontVal releases) was not opened. I don't look at source code I do not legally have. It is good to see details clarified.

With no further comments suggestion corrections needed, I'm assuming this is resolved, so closing.

Later it will get submitted to ISO for their consideration.

Any progress on ISO/IEC standard? Will it be in https://www.iso.org/standard/82487.html (AWI AMD 2)?

Any progress on ISO/IEC standard?

The work on AMD2 is currently underway. Yes, this update can definitely be part of the new amendment - I opened a new issue 45 to address it.


Vendoring on Windows replaces license files contents by a path for some packages that didn't change

Problem
I'm using:
$ cargo --version
cargo 1.68.2 (6feb7c9cf 2023-03-26)

And trying to update only the getrandom dependency to the latest version in Firefox:
cargo update -p getrandom
./mach vendor rust

That results in changes in some LICENSE files for packages not related to getrandom, where the contents of the file gets replaced by a path: third_party/rust/cssparser-macros/LICENSE now contains the text../LICENSE, and third_party/rust/wgpu-{core,hal,types}/LICENSE.{APACHE,MIT} now contain the text ../LICENSE.APACHE or ../LICENSE.MIT.
The .cargo-checksum.json files also reflect these change as though they were legit changes of contents on the files.
Analyzing what's going on in the system seems to point to the sync function in src/cargo/ops/vendor.rs as the culprit for deleting then writing the file.
What should happen
cssparser-macros and wgpu-{core,hal,types} should be unaffected by vendoring.
Steps
No response
Possible Solution(s)
No response
Notes
No response
Version
$ cargo version --verbose
cargo 1.68.2 (6feb7c9cf 2023-03-26)
release: 1.68.2
commit-hash: 6feb7c9cfc0c5604732dba75e4c3b2dbea38e8d8
commit-date: 2023-03-26
host: x86_64-pc-windows-msvc
libgit2: 1.5.0 (sys:0.16.0 vendored)
libcurl: 7.86.0-DEV (sys:0.4.59+curl-7.86.0 vendored ssl:Schannel)
os: Windows 10.0.22621 (Windows 10 Home) [64-bit]

Do you perhaps have core.symlinks=false in your git configuration? Do you have Windows Developer mode enabled? Do you have any cargo configuration files?

Thanks for your reply! With these keywords I was able to find this issue and this post, which in turn helped me solve my issue as follows:
git config --global core.symlinks true
rm -rf ~/.cargo/git/checkouts/*
./mach vendor rust

So indeed I had core.symlinks=false set globally, and I also needed to update the local value for core.symlinks in the already-checked-out repositories (for simplicity I just deleted them so that cargo would check them out again).


Can we destroy the current Parse object and recreate it?

Hi ,
Destroy the current Parse object and recreate is it possible because some time parse query not responding alway gives the server time out message. But I kill the app and restart it again it's working fine.
Thanks
Kiran

Hey @kirank-canwill, not sure whether you mean Parse class object or PFObject class?
Currently uninitializing and reinitializing Parse SDK is unsupported, and outside of tackling the problem of timeouts on Parse Queries, I am not sure on what would be a biggest use case here.

Thanks for your support. I've resolved this point. Please mark this as Closed.


BUG: <Unable to import 'triu' module>

Describe your issue.
Looking at the example code, unlike version 1.12.0, in version 1.13.0, ImportError: Cannot import name 'triu' from 'scipy.linalg'(c:\~~\__init__.py). Such a phenomenon occurred. .
Reproducing Code Example
import scipy.linalg as la

# example
m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]])

# triu useing
result = la.triu(m)
print(result)

Error message
ImportError: Cannot import name 'triu' from 'scipy.linalg'(c:\~~\__init__.py)

SciPy/NumPy/Python version and system information
scipy:1.13.0
numpy:1.26.4
python:3.10.14

triu is deprecated and removed from scipy.linalg. You need to use triu from numpy with
np.triu(m)

I don't think that there is any occurrence of la.triu in the docs - please let us know if I am mistaken @HP657 :)

Hi @lucascolley or @ilayn, I believe I am facing this issue. Having trouble following the outcome to this issue thread, it looks like this is closed but I believe it is an issue in the latest version of scipy.
Here are the steps to reproduce:
python -m venv temp_venv
source temp_venv/bin/activate
pip install scipy==1.13.0
python
Python 3.11.8 (main, Feb 13 2024, 21:57:06) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.linalg import get_blas_funcs, triu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'triu' from 'scipy.linalg' (<redacted>python3.11/site-packages/scipy/linalg/__init__.py)`

Thank you!

Hi @ds-mike See my comment above.

Oh thanks! It is a gensim issue that I had missed originally.


Report JIT-compiled functions as Python functions.

With this change the cProfiler module will report the source location for JIT-compiled functions as for ordinary Python functions.

(Note that in case of an exception we now call PyTrace_RETURN rather than PyTrace_C_EXCEPTION, as the latter would simply be ignored by the profiler.)

Is it the only reason? One could imagine a profiler not ignoring exceptions (though I don't think a profiler should care about it, indeed; the difference probably exists for regular tracing functions such as debuggers).

I added some docs, which are hopefully clear enough.

Looks good!  As soon as CI is happy, I will merge.

Current coverage is 87.90%

Merging #1799 into master will not affect coverage as of fd272c5

@@            master   #1799   diff @@
======================================
  Files          283     283
  Stmts        49500   49500
  Branches      5052    5052
  Methods
======================================
  Hit          43512   43512
  Partial        868     868
  Missed        5120    5120

Review entire Coverage Diff as of fd272c5

Powered by Codecov. Updated on successful CI builds.

ok, appveyor is temporarily broken, but I'll merge anyway


npm isn't picking up the readme

Oddly, https://www.npmjs.com/package/codemirror-copilot is blank. Should be fixable, probably by including README.md in files.

It's picking up README from here: https://github.com/asadm/codemirror-copilot/tree/main/packages/codemirror-copilot
We can just remove that and symlink root one there (or vice versa) 🤔

Should be fixed in #10, will test out with the next release.


Not correct

[Enter feedback here]
This document has incorrect Openssl documentation. (doesn't work that way)
I already had my root key and X509 cert, generated with the following command:
openssl req -x509 -newkey rsa:2048 -keyout root_private.pem -nodes -out root_cert.pem
Then, I needed to generate the verification cert...
Create verification key using command:-
openssl genrsa -out verification.key 2048
Create the verification cert using command:-
openssl req -new -key verification.key -out verification.csr
When creating the verification cert, I need to specify the verification code obtained (7A69A4702DA903A41C3A5BC5575A8E3F49BEC5E5BA2D4CE1) as the "Common Name" certificate field.
Now, just create the proof of possession certificate with the following command:
openssl x509 -req -in verification.csr -CA root_cert.pem -CAkey root_private.pem -CAcreateserial -out verificationCert.pem -days 1024 -sha256

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0308fed1-2c82-f02f-2745-931dc16fc538
Version Independent ID: 6be62d35-1075-8719-996e-d02bc1b7281d
Content: Tutorial - Prove Ownership of CA certificates in Azure IoT Hub
Content Source: articles/iot-hub/tutorial-x509-prove-possession.md
Service: iot-hub
GitHub Login: @v-gpettibone
Microsoft Alias: robinsh

@kushantp58 Thanks for the feedback! We will review and update accordingly.

@kushantp58 Thanks for the feedback!
I have assigned the issue to the content author for further review and update the doc as appropriate.

The documentation is correct as written.  The easy way to create a verification certificate is to create the key with the verification string embedded as the Common Name, get the CSR, and create the certificate. This has been tested multiple times, and it works.

Ok I'll follow your advice. I know you guys must have definitely tried it out multiple times...but it didn't work for me ... I'll  give it a try once more. Thanks for the help. In case of any issues I'll reach out

What issues are you running into? What is the error and when do you receive it?

Ok I tried this part again for openssl -  "If you are using OpenSSL to generate your certificates, you must first generate a private key and a certificate signing request (CSR)" using openssl req -new -key pop.key -out pop.csr  but before that you need to generate the pop.key

Maybe I've to assume that I already have a key...
Thanks for all the help... Greg and Team !!

I'm able to reproduce your error, but I don't know why it is happening. Let me investigate further

Hi Kushant. The documentation does appear to be wrong. If you have the time, please follow this 2-step process to get a CSR and let me know what you get.
$ openssl genpkey -out fd.key -algorithm RSA -pkeyopt rsa_keygen_bits:2048
$ openssl req -new -key fd.key -out fd.csr
see below
REDMOND+v-gpettibone@MININT-T5I1CK2 /cygdrive/c/cygwin64/home/v-gpettibone/testca
$ openssl genpkey -out fd.key -algorithm RSA -pkeyopt rsa_keygen_bits:2048
................+++++
............................................................................................................+++++
REDMOND+v-gpettibone@MININT-T5I1CK2 /cygdrive/c/cygwin64/home/v-gpettibone/testca
$ openssl req -new -key fd.key -out fd.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [XX]:.
State or Province Name (full name) []:.
Locality Name (eg, city) [Default City]:.
Organization Name (eg, company) [Default Company Ltd]:.
Organizational Unit Name (eg, section) []:.
Common Name (eg, your name or your server's hostname) []:BB0C656E69AF75E3FB3C8D922C1760C58C1DA5B05AAA9D0A
Email Address []:.
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
REDMOND+v-gpettibone@MININT-T5I1CK2 /cygdrive/c/cygwin64/home/v-gpettibone/testca
$ openssl req -text -in fd.csr -noout
Certificate Request:
Data:
Version: 1 (0x0)
Subject: CN = BB0C656E69AF75E3FB3C8D922C1760C58C1DA5B05AAA9D0A
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:c0:b8:be:1a:e1:65:1c:ab:2f:4d:3a:c4:c2:f7:
e4:12:32:b3:d2:c3:7d:32:ff:cd:94:6d:36:15:b5:
96:b4:c3:b4:89:34:6f:f8:f8:fa:c4:8d:cb:0d:35:
07:5a:8a:19:79:5b:1d:60:75:90:3f:53:b9:34:54:

Hey Greg,
I did try out the entire procedure and it does work, I would definitely recommend this procedure.
It works for me -

(Since I was using it as a subordinate certificate for IoThub instance, I had to output it into a .pem file which worked for me too and I finally have it verified).
Thanks for all the help.. Greg & Team
Regards,
Kushant

Hi Kushant, thanks much. I'll update the docs.


What is the correct folder structure for the dataset?

Hi,
As per the README.txt, I've setup my CLEVR dataset to look like this -
IODINE-master/data/CLEVR/
	images/
		train/
			CLEVR_train_xxxxxx.png
			CLEVR_train_xxxxxx.png
			.
			.

		test/
			.
			.
			.
		val/
			.
			.
			.
	questions/
	scenes/
	COPYRIGHT.txt
	LICENSE.txt
	README.txt

Is this the right folder structure? Because I'm getting the following error, which I think is probably because the png files are not accessible.
Traceback (most recent call last):

  File "/Volumes/GoogleDrive/My Drive/IS 800 - Deep Learning [Project]/IODINE-master/tools/train_net.py", line 90, in <module>
    train_net(cfg)

  File "/Volumes/GoogleDrive/My Drive/IS 800 - Deep Learning [Project]/IODINE-master/tools/train_net.py", line 83, in train_net
    evaluator=evaluator

  File "./lib/engine/train.py", line 48, in train
    for iter, data in enumerate(dataloader):

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 385, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]

  File "./lib/data/clevr.py", line 25, in __getitem__
    img = io.imread(img_path)[:, :, :3]

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/skimage/io/_io.py", line 48, in imread
    img = call_plugin('imread', fname, plugin=plugin, **plugin_args)

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/skimage/io/manage_plugins.py", line 210, in call_plugin
    return func(*args, **kwargs)

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/skimage/io/_plugins/imageio_plugin.py", line 10, in imread
    return np.asarray(imageio_imread(*args, **kwargs))

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/imageio/core/functions.py", line 265, in imread
    reader = read(uri, format, "i", **kwargs)

  File "/Users/akv/anaconda3/lib/python3.7/site-packages/imageio/core/functions.py", line 182, in get_reader
    "Could not find a format to read the specified file in %s mode" % modename

ValueError: Could not find a format to read the specified file in single-image mode

Added the 'mode' while searching for images.
Line 17 of \lib\data\clevr.py:
for file in os.scandir(os.path.join(root, 'images', mode)):


XMPP parser error when JWT auth token is enabled

Hi,
I made some test with the last gst-meet version and a Jitsi-Meet instance with JWT auth token enabled.
I can connect the gst-meet user on a running meeting if I add a valid JWT token on the wss url and it works well.
But if a new moderator joins the room, gst-meet failed to parse the presence message, and it leaves the room with this error :

2023-06-08T14:11:58.704482Z ERROR error=parse error: Either 'jid' or 'nick' attribute is required.
2023-06-08T14:11:58.704628Z ERROR fatal (in read loop): parse error: Either 'jid' or 'nick' attribute is required.

Here is the XMPP message that generate the error :
<presence xmlns='jabber:client'
<EMAIL_ADDRESS>          xml:lang='en'
<EMAIL_ADDRESS>    <stats-id>Aaliyah-Uo5</stats-id>
    <c hash='sha-1' ver='p1SSmeQ82gSAjXEw+FlBmWtBv2k=' node='https://jitsi.org/jitsi-meet' xmlns='http://jabber.org/protocol/caps'/>
    <features>
        <feature var='https://jitsi.org/meet/e2ee'/>
    </features>
    <SourceInfo>
        {}
    </SourceInfo>
    <jitsi_participant_codecType>
        vp8
    </jitsi_participant_codecType>
    <avatar-url>
        https://www.gravatar.com/avatar/XXXXXX
    </avatar-url>
    <email>
<EMAIL_ADDRESS>    </email>
    <nick xmlns='http://jabber.org/protocol/nick'>
        BOB
    </nick>
    <jitsi_participant_e2ee.idKey.curve25519>GtXeEe0YKDGaSJy+VcEjjfZ6+znz7B1HRU7l/Gp45y0</jitsi_participant_e2ee.idKey.curve25519>
    <jitsi_participant_e2ee.idKey.ed25519>bGZlraoA8K0AdWvx3+aaQuaVJRpfweG7tHJHQkYL4D8</jitsi_participant_e2ee.idKey.ed25519>
    <occupant-id id='yhfk8wkTJwdRwbXaxvDEzTNcuW1Kk/xE8XIPncpghdM=' xmlns='urn:xmpp:occupant-id:0'/>
    <x xmlns='http://jabber.org/protocol/muc#user'>
        <item<EMAIL_ADDRESS>affiliation='owner' role='moderator'>
            <actor<EMAIL_ADDRESS>nick='focus'/>
        </item>
    </x>
</presence>

After digging the gst-meet code, I think the error "parse error: Either 'jid' or 'nick' attribute is required." came from the rust xmp-parser  library on the actor element but I'm not sure.
Does anyone has already see this error with Gst-Meet ?

Yes, it seems to come from here: https://docs.rs/xmpp-parsers/latest/src/xmpp_parsers/muc/user.rs.html#107
xmpp-parsers validates that exactly one of jid or nick is set on the actor element. Since Jicofo is setting both, the error occurs.
xmpp-parsers seems to deviate from the spec here, which allows both to be set (https://xmpp.org/extensions/xep-0045.html#schemas-user), meaning it probably should not have been modelled as an enum.

Filed an issue upstream: https://gitlab.com/xmpp-rs/xmpp-rs/-/issues/88

I've implemented a workaround in https://github.com/avstack/gst-meet/commit/a42e069d282db43fc8ed088db79be1b3ec6a3557
If you can confirm that current master works for you, I'll make a release with the workaround.
I'll keep this issue open until it's fixed upstream and I can remove the workaround.

Hi @jbg,
Your workaround works fine.
I don't know if xmpp-rs will be fixed or if someone must ask prosody to change its implementation (see the Jitsi Forum post).
But now Gst-Meet works with my JItsi-Meet environment, so thank you.


Clipping glitch with Flow component

On initial load, and until a press occurs the flow component pieces are showing outside of the clip parent.
See here: https://github.com/briannhinton/framer-bucket/blob/master/flow-app.js
Let me know if it's an error on my part as I'm still learning the Framer library.

You’re correct. This is not a bug per-se, as usually the layers are already set up in a way that they are clipped by the device screen. In this case, using the fullscreen device, you’ll have to do a bit of extra plumbing yourself.
I think a good way to do it is setting the parent of each layer to the flow component in advance (which otherwise will happen when showNext is used), e.g. if you include layerB.parent = flow your example will work as expected.


[SPARK-10750] [ML] ML Param validate should print better error information

Currently when you set illegal value for params of array type (such as IntArrayParam, DoubleArrayParam, StringArrayParam), it will throw IllegalArgumentException but with incomprehensible error information.
Take VectorSlicer.setNames as an example:
val vectorSlicer = new VectorSlicer().setInputCol("features").setOutputCol("result")
// The value of setNames must be contain distinct elements, so the next line will throw exception.
vectorSlicer.setIndices(Array.empty).setNames(Array("f1", "f4", "f1"))

It will throw IllegalArgumentException as:
vectorSlicer_b3b4d1a10f43 parameter names given invalid value [Ljava.lang.String;@798256c5.
java.lang.IllegalArgumentException: vectorSlicer_b3b4d1a10f43 parameter names given invalid value [Ljava.lang.String;@798256c5.

We should distinguish the value of array type from primitive type at Param.validate(value: T), and we will get better error information.
vectorSlicer_3b744ea277b2 parameter names given invalid value [f1,f4,f1].
java.lang.IllegalArgumentException: vectorSlicer_3b744ea277b2 parameter names given invalid value [f1,f4,f1].

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42819/
Test FAILed.

Jenkins, test this please.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42820/
Test FAILed.

Jenkins, test this please.

Merged build triggered.

Merged build started.

Test build #42824 has started for   PR 8863 at commit d937e87.

Test build #42824 has finished for   PR 8863 at commit d937e87.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42824/
Test PASSed.

LGTM.  At some point, we could make a more general pretty-printing method to use here, but I think this is fine for now.  Merging with master.


Update chart job to the latest

JIRA link (if applicable)
Update chart job to the latest
Change description
Update chart job to the latest
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x] No

Braking change will need to be updated after duplicate user release


prepend-sudo

It was requested that there be a bind (in the input module) to prepend sudo to a command. This isn't a problem, but isn't something I would use. Does anyone have a suggestion for what key combination to bind this to (that doesn't clobber emacs binds)?

While I read that guys comment, personally i think sudo !! is perfectly fine (for forgotten sudo commands), I would rather be explicit than implicit with the key binding, writing out sudo makes me think about the command im running more than a key binding would... Maybe its just me, and being stuck in my old ways...

I use sudo !! as well, but as I said, I'm not opposed to adding it. If there are a few people that comment here within the week desiring it, I'll add it. Otherwise, I'm going to leave it be. Thanks for your input.

I have come to prefer double esc to prepend sudo -- oh-my-zsh's sudo plugin does this. Previous to that, I too just used sudo !!, and still do on systems that I can't push my dotfiles to. I :+1: having this functionality.

I will never bind it to <meta><meta>. I can see that causing problem for non-emacs users.

I've decided against adding this. I feel that sudo !! is sufficient. I forces you to look at your command (one press of enter will only perform expansion, not execution).


 Please see this Google Help Center article for details.

Many people over at Kodular got this problem too, see https://community.kodular.io/search?q=gcp keys exposed
Affects

[ ] Designer
[ ] Blocks editor
[ ] Companion
[x] Compiled apps
[ ] Buildserver
[ ] Other... (please describe)

Expected behavior

We should not have exposed keys in the platform and I don't want to have these kinds of warnings in the Google Play Console
Steps to reproduce

It appears to me that the Firebase and Fusiontables component is the cause for the warning. Fusiontables is deprecated anyways, so that's fine for me. I suspect that Google detects that App Inventor is using a deprecated authentification method for Firebase

I suspect that Google detects that App Inventor is using a deprecated authentification method for Firebase

This is the reason. I have verified this myself.


docs(checklist): add pr review checklist

Relates to ottoapi_docs otto-ec/ottoapi_orga#91
Adds checklist to review a pull request.

Maybe something like:

Is the change in line with our API guidelines?

Maybe something like:

Is the change in line with our API guidelines?

@DominiquePetrich1709 Thanks! Is this a suggestion for this API Guidelines checklist or for an ottoapi_docs review?

Maybe something like:

Is the change in line with our API guidelines?

@DominiquePetrich1709 Thanks! Is this a suggestion for this API Guidelines checklist or for an ottoapi_docs review?

@BirgitBader It's for the pr review checklist.


Support finding references at module in module.exports = or export in export =

Fixes https://github.com/Microsoft/vscode/issues/21507#issuecomment-433652554

@sheetalkamat Please re-review

Hey guys, trying to use a find occurrences/usages/references but it only shows me the usage in the same file, not elsewhere. How do make it works?

Hey guys, trying to use a find occurrences/usages/references but it only shows me the usage in the same file, not elsewhere. How do make it works?

Agreed, all I'm seeing is same-file references too.
e.g. const function1 = () => {} defined in events.js is referenced in index.js (to be called on an API call). Clicking Find All References on function1 in events.js does not list the reference `index.js.

Just got into this issue again from an old project using this kind of syntax:
const object = {
 methode: (),
 other: ()
}
module.exports = object;

// Or even this one
const object = {
 KEY1: 'dfdfd',
KEY2: 'sdddf'
}

module.exports = object;

In intellj, this kind of find usage is able to track which other file is using the export, that would be nice to have a way to track which file use the export :)

This problem still exists: https://github.com/microsoft/TypeScript/issues/44832


fix: foreign key column names

:tada: This PR is included in version 1.1.19 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Extended aliases

Expanded some of the aliases like directions. Now, for example, northeast will be matched by ne, northe, northea, and northeas as well as the original northeast.

Thanks for the PR! I have a typo-detection system in place already (using soundex) that I assumed would solve these use cases but that must be an incorrect assumption.
I'm too busy at the moment to properly address this but I'll get back to you in May :)

Looking at this PR again I think it would be better to use a trie data structure for parsing command lookups so that any unique prefix will execute that command. Then we wouldn't have to specify all these aliases manually ;)
I'll merge this anyway and it can be fixed at a later date if/when I start working on this project again.


[v0.6.0 Release Docs] Patterns (beta)

As part of Release Blog for v0.6.0, we have created this issue belonging to epic [#4127] in order to organize and ensure the smooth release of the blog. Feel free to leave a comment if you would like to take up this respective blog or any section of it.
Some of the Areas to be covered for Patterns :

[ ] What is a service mesh pattern?
[ ] What patterns are available ? Can I bring my own ?
[ ] Importing from GitHub.com
[ ] Can I customize a pattern ? How do I design one ?
[ ] mesheryctl pattern

Contributor Guides and Resources
Guides and Resources

v0.6.0 Docs
v0.6.0 Milestone
v0.6.0 RoadMap
v0.5.0 blog
:hammer_and_wrench: Meshery Build & Release Strategy
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Hi, I would like to work on this. Could you please tell how do I go ahead with it?

Heyy @Shatakshi0805 ! How's this going !? Feel free to reach out when you need help.

working on it. would make a pr soon.

@Shatakshi0805  Any updates? Please feel to reach out if you need some help or want to get the blog reviewed!

Hi, so sorry I forgot to inform that I got caught up in some other work. Maybe someone else can work on this issue.


How do I install a specific version of a lib

How can i install a specific version of freeglut?

Have you found/read the documentation?

@worm69, could you please reference above docs and try again?

THX @PhoebeHui @autoantwort i will try


Codeunit 22 - ""Item Jnl.-Post Line"" - Change PostInventoryToGL() to global

Could you change PostInventoryToGL() from local to global?

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Switch to Standard linter for all @probot repositories

[x] probot #219
[ ] settings
[ ] dco
[x] stale
[ ] github-app
[ ] create-probot-app
[ ] scheduler
[ ] no-response
[ ] autolabeler
[ ] autoresponder
[ ] metrics
[ ] owners

Stale was updated in https://github.com/probot/stale/pull/58.

🎉 gonna close this out since all the things have been linted except no-response which isn't really being actively maintained right now? Also I have an open PR there and linting merge conflicts are 😭 I'm gonna open an issue over there, so if it ever gets back up and running, we can lint then.

One more that needs migrated:

[ ] reminders


Fix NoAMethodError in FaradayServerError

Problem:
Error when Faraday::ClientError.new(nil)
Solution
added try to the response hash on FaradayServerError

For rubocop, rubocop -a ; rubocop --auto-gen-config.
Needs tests.
Needs changelog.
Thanks.

https://github.com/ashkan18/graphlient/issues/51

I do not agree with the original author on certain things so I decided to keep working on my own fork until the main branch is fixed. But in the meantime if any of you really need a fix on this soon, check out https://github.com/neroleung/graphlient.

I do not agree with the original author on certain things so I decided to keep working on my own fork until the main branch is fixed. But in the meantime if any of you really need a fix on this soon, check out https://github.com/neroleung/graphlient.

What are these things you don't agree with @neroleung. This library is written and maintained by a bunch of people, we're happy to discuss anything and help you contribute to the parent repo rather than fork away.

My apologies. My project has a tight deadline and whether this bug can be fixed quickly or not determines if I need to drop this gem in my project. I didn't expect quick responses and reviews like you all did in open source repos. So I got anxious. I'll make new PRs and fix things up in stages like you all suggested. Thanks~

Closed via https://github.com/ashkan18/graphlient/pull/68.


fix: istioctl operator remove fail when not specify reversion

Please provide a description of this PR:
Fix istioctl operator remove fail when not specify reversion
Fix https://github.com/istio/istio/issues/43659

@my-git9: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-istiodremote_istio
ae5fb1a659b3876ae66f1a0a790591ea0155a7f6
link
true
/test integ-security-istiodremote

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-security-istiodremote

/test integ-telemetry-mc

@my-git9: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-multicluster_istio
cc9bf00dd0095c356ad721bba94de47bf0605eb8
link
true
/test integ-security-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-security-multicluster

Can we get a release note for this, especially since there is an issue. Thanks.

Can we get a release note for this, especially since there is an issue. Thanks.

Can we get a release note for this, especially since there is an issue. Thanks.
Thank you for your review, there is a related release note in the previous repair PR, you can check the discussion above:
https://github.com/istio/istio/pull/43660#discussion_r1121070073
Do you think we should add this release note again？

Can we get a release note for this, especially since there is an issue. Thanks.

Thank you for your review, there is a related release note in the previous repair PR, you can check the discussion above:
https://github.com/istio/istio/pull/43660#discussion_r1121070073
Do you think we should add this release note again？

Do you think we should add this release note again？

Can we tweak the existing release note to include the reporting issue?

Do you think we should add this release note again？

Can we tweak the existing release note to include the reporting issue?

Updated.


Feature to add/remove titles from a users watchlist

Has there been any research on this topic? I would be willing to contribute, just wondering if it's already been discussed and has been dismissed or is it feasible to be implemented? As far as I can see, you're probably making the same API calls as the Netflix client does, so that should not be an issue. Unless there's no need for this feature.

@LBBO "adding / removing titles from viewing history" what is the current stage of this? that would be amazing!

@DarkSupremo Removing titles should be fairly simple, I might even get that done this weekend. Adding them to the viewing history is a whole other story, though!
As far as I know, the normal netflix UI offers no other way of doing that, than watching titles. I inspected the network activity while doing just that and every time I paused or resumed the video, it sent a request to a certain endpoint which, judging by its URL, does not belong to the shakti API. Those requests always had practically identical headers, however the bodies always contained three JSON objects, which seemed to contain Base64 encoded JSON data which seems to be partially encrypted as one property is named ciphertext and contains completely illegible data.
It might be possible to send these requests manually, but it would definitely take me a while to figure out, how. If you'd like, you're very welcome to help!

@genderquery looking forward to a bit of feedback: this my first PR on a node project and using Netflix and node is not something I do a lot off.
Let me know if this PR is acceptable or advise otherwise.
(In hind sight I've spotted a typo and perhaps hideViewingHistory is a more transparent name than clearViewhingHistory which implies the history is completely cleared for a movie or series when it's simply hidden from the user.)
@DarkSupremo hopefully this covers removing (a request to the same getViewingHistory()(//viewingactivity), but a POST instead of GET passing {"movieID":YOUR_MOVIE_ID,"seriesAll":hideWholeSeriesOrNot,"authURL":"YOUR_AUTH_URL_HERE"}) as the request body. I don't know how to add new things to the viewing history other watching something (e.g. /watch/80077368) but I'm hoping there are other ideas.

@LBBO FIY I've fixed my typo, added hideMovieViewingHistory/hideAllViewingHistory methods and updated the README. Hope it's all legit 😄

This is requested in the CLI project as well: https://github.com/LBBO/netflix-migrate/issues/32


BooleanLiteral is not a valid helper on line 2.

I use this add-on in an ember v3.24.0 project, but it throws an error:
Template Compiler Error (TemplateCompiler) in bts-client/components/layout/global-filters/section/template.hbs

`BooleanLiteral` is not a valid helper on line 2.

 @group.name)}} as |current|
>
  ...
</PowerSelectMultiple>

Does @autofocus={{false}} invalid?

Thanks for raising the issue.
@autofocus={{false}} this is a valid statement. Working on the fix.

Thanks for raising the issue.
@autofocus={{false}} this is a valid statement. Working on the fix.

Fixed in the latest release https://github.com/siva-sundar/ember-template-inspector/releases/tag/v0.1.2

Fixed in the latest release https://github.com/siva-sundar/ember-template-inspector/releases/tag/v0.1.2


Cosine Graph Error

I am trying to create a cosine graph using Manim but it was different from the actual graph.
Code:
from manim import *
from numpy import cos

class CreateGraph(GraphScene):
    def __init__(self, **kwargs):
        GraphScene.__init__(
            self,
            x_min=-5,
            x_max=5,
            y_min=-5,
            y_max=5,
            graph_origin=ORIGIN,
            axes_color=GREEN)

    def construct(self):
        # Create Graph
        self.setup_axes(animate=True)
        f1 = self.get_graph(lambda x: cos(1/x) , x_min=-4, x_max=4)

        # Construct the Figures
        self.play(ShowCreation(f1))
        self.wait(1)

Error:

Environment
OS System: Windows 10
manim version: master <V.6 Community Version>
python version: .3.7.0

I think you are using the community version of manim.
So you're supposed to ask this question on discord of ManimCommunity or open an issue in ManimCommunity/manim instead of here.


WEBAPP-476: Make app themable

This pull request belongs to an issue on our bugtracker.
You can find it there by looking for an issue with the key which is mentioned in the title of this pull request.
It starts with the keyword WEBAPP.

Wait for the merge until we discussed how/where we want to deploy this :)

@svenseeberg @Schedulaar If the requirements for the prototype are met we can merge this.


improved error message if security code does not have enough characters

Coverage increased (+6.5%) to 78.778% when pulling 56d7ae60a5af8cf929e68e80e312c3663181e4c7 on brean:patch-1 into 270f471f6ab1c4c1a5b9f444e4e49a8cedaae94f on ggravlingen:master.

Coverage increased (+6.5%) to 78.778% when pulling 56d7ae60a5af8cf929e68e80e312c3663181e4c7 on brean:patch-1 into 270f471f6ab1c4c1a5b9f444e4e49a8cedaae94f on ggravlingen:master.

Coverage increased (+6.5%) to 78.778% when pulling 56d7ae60a5af8cf929e68e80e312c3663181e4c7 on brean:patch-1 into 270f471f6ab1c4c1a5b9f444e4e49a8cedaae94f on ggravlingen:master.


Response file object (returned by InfoExtractor._request_webpage) may be closed for failed requests (matching expected_status) on Python 3.4.1+

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.08.04. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.08.04

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Since bpo-15002 (introduced in Python 3.4.1), HTTPErrors close their fp when the error's destroyed. The current implementation of InfoExtractor._request_webpage accommodates for expected_status by catching HTTPErrors and returning this fp. Unfortunately, this means subsequent reads against this file object by the caller are unreliable.
Fortunately, the number of extractors that make use of expected_status is small; as of 2018.08.04, it's just bbc, lynda, markiza, and twitch. One possible fix would be to return the error, and have the callers that use expected_status test for and access fp instead. Alternatively, we might be able to monkey patch our way into nixing the effects of urllib.response.addclosehook on those instances of HTTPError. I haven't looked into whether there's a cleaner approach.
Issue encountered whilst debugging reports of problems running @bato3's fix for #17116 on Python 3.7.

I don't see any problem here. By using expected_status you treat potentially failed outcomes as normal thus you should be ready for consequences like closed connection. It's the responsibility of a client code to check whether connection was closed or not in such cases.
Also if you want error then don't use expected_status and catch exception instead. The whole point of expected_status is to simplify code in cases when success and failure are both expressed in the same way. For example, when _download_json always returns JSON (e.g. for 404 and for 403) so that in 403 scenario you don't need to catch HTTPError, read output and parse it in client code.

@dstftw, we're not talking about a closed connection here, we're talking about the response body being inaccessible, which defeats the point of expected_status. The expectation is that if you call _download_webpage with expected_status, you'll get the response body back if the request was successful or if the response code matches expected_status. But this is not the case.


Why are invalid statements parsed without errors?

Hello, If I execute the following in the sqlite3 CLI -
sqlite> select 1,2 union select 3,4,5;

I receive an error, as expected -
Error: SELECTs to the left and right of UNION do not have the same number of result columns

why does sqlite-parser not recognize it as invalid?

That statement is valid, syntactically, but not valid semantically. sqlite-parser parser does not have these kinds of semantic validations in its current form.
If you want to check statements for these types of issues, you need a linting/validation tool that accepts the ASTs generated by this parser. Many semantic validations would also require knowledge of the schema of the database for the statements (e.g., wrong column data type).


Channel results from search show as "LIVE"

Probably looking for source and assuming they are live?

Fixed in https://github.com/lbryio/odysee-ios/commit/43c1e3dfc1e07739bfc4411e06a387ac17e4d5c1.


Update New-ApplicationAccessPolicy.md

Updates to reflect the additional number of policies now allowed, updates to clarify the policy types attribute.

Docs Build status updates of commit 807020a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-ApplicationAccessPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 7932865:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-ApplicationAccessPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Azure is missing multipart upload integration

Azure currently downloads files as one large part. We should craft a design for multipart transfers on Azure. I anticipate downloads will be easier but uploads may be tricky due to an S3 incompatible API.

Hey @antonzabreyko I’m using Zenhub in GitHub, click this link to join my workspace and see other features available in GitHub or download the Zenhub extension and sign up with your GitHub account.
Posted using Zenhub


can Swarmkit support service affinity or exclusion?

Hi, All,
Like the title.
When we experience swarm mode in docker 1.12. We found it is really important for us to  schedule service. Here we have some need that different services should be scheduled to different nodes.
In another word,  can swarmkit schedule a service ServiceA to node without ServiceB?
Thanks in Advance.

Swarmkit doesn't support affinity or anti-affinity yet. The constraints are on node and engine labels so far. See constraints in the doc. Current recommendation is to use node engine label to select nodes.
Adding (anti-)affinity is not a big task. But one of the problem with (anti-)affinity is that the evaluation process. If you put a constraint that ServiceA runs on nodes without ServiceB, when instances of ServiceB get rescheduled, they may be scheduled on nodes with ServiceA since ServiceB doesn't have this constraint. It would break the anti-affinity from ServiceA. We can re-evaluate every service on the node to decide if it fits or not, but they would result in heavy computation.
cc @aaronlehmann.

Thanks for your feedback. Maybe we shall find a better scenario or do a future plan in swarmkit. @dongluochen

Any update on this affinity in docker service create ? Now it's supporting or not ?

Did get the question during a conversation with a partner. They need to know when and how we plan to support this?

"Service aware affinity & anti-affinity" is also important for one of my customers.  They want to use it in production environment for resource and performance infrastructure optimization.
Lets say Service A replica(s) should run only on a nodes that Service B replica(s) is not running or the other way around, Service A replica(s) should run only on a nodes with running Service B replica(s).
That allows optimized resource usage and/or performance, for example:

A could be compute heavy and B storage IOps heavy, so better have them running on the same hosts.
Both, A and B, could be very Memory or CPU intensive so would make sense to schedule them running on a different nodes.

Should we expect to see soon "service aware affinity & anti-affinity" constrains supported in swarm kit/swarm mode?
Thanks.

I miss that feature too. Using constraints and managing docker hosts labels is too much overhead. Eventually I ended up running several services in one container, so I don't have to worry about scaling containers over the cluster.

simple workaround
https://github.com/keyolk/swarmkit/commit/212a1900ba4a085d8b8f0f0da56ec9e1ce1f8f2c
$ docker service create --constraint antiaffinity==test --name=one tomcat
$ docker service create --constraint antiaffinity==test --name=two tomcat

two's tasks never be assigned same node which has one's task.
also one's tasks never be assigned on same node.


Is it possible to allow just a subset of markdown?

title. In my case I want to disable images.

Does this work?
var marked = require("marked")
function MyRenderer() {
    marked.Renderer.apply(this, arguments)
}

MyRenderer.prototype = Object.create(marked.Renderer.prototype)

MyRenderer.prototype.image = function () {
    return "" // don't print images
}

See #420 or use #660

Closing to consolidate issues


located((By.XPATH, "//*[@class='icon-button "

====== WebDriver manager ======
Current firefox version is 91.0.2
Get LATEST driver version for 91.0.2
Driver [C:\Users\metapodcod\.wdm\drivers\geckodriver\win64\v0.29.1\geckodriver.exe] found in cache
Generating Max Profit For Stop Loss.
Loading script...

Traceback (most recent call last):
  File "TradeViewGUI.py", line 122, in optimize_longs_shorts
    if self.comboBox.currentIndex() == 0:
  File "OptimizeLongsShorts.py", line 17, in __init__

  File "OptimizeLongsShorts.py", line 37, in run_script
    # Making sure we are on inputs tab and resetting values to default settings.
  File "my_functions.py", line 32, in click_settings_button
    wait.until(EC.visibility_of_element_located((By.XPATH, "//*[@class='icon-button "
  File "selenium\webdriver\support\wait.py", line 80, in until
selenium.common.exceptions.TimeoutException: Message:

It looks bot wasn't able to click settings button when running so it
timed out. It could be possible that your profile wasn't automatically
logged in when bot opened up a new Firefox window.
On Mon, Sep 6, 2021, 12:46 PM Yusuf @.***> wrote:

====== WebDriver manager ======
Current firefox version is 91.0.2
Get LATEST driver version for 91.0.2
Driver [C:\Users\metapodcod.wdm\drivers\geckodriver\win64\v0.29.1\geckodriver.exe] found in cache
Generating Max Profit For Stop Loss.
Loading script...
Traceback (most recent call last):
File "TradeViewGUI.py", line 122, in optimize_longs_shorts
if self.comboBox.currentIndex() == 0:
File "OptimizeLongsShorts.py", line 17, in init
File "OptimizeLongsShorts.py", line 37, in run_script
# Making sure we are on inputs tab and resetting values to default settings.
File "my_functions.py", line 32, in click_settings_button
wait.until(EC.visibility_of_element_located((By.XPATH, @.***='icon-button "
File "selenium\webdriver\support\wait.py", line 80, in until
selenium.common.exceptions.TimeoutException: Message:
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/TreborNamor/TradingView-Machine-Learning-GUI/issues/20,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOHWMRNVT6ZRX3QEYGVLQ5LUAT45XANCNFSM5DQ3KR3A
.


Submitting / Updating samples on a weekly basis

Hi all,
(INAB-CERTH) is sequencing roughly 100-200 samples on a weekly basis, as part of our collaboration with the National Health Organization of Greece and in the context of a flagship project on genomic surveillance.
We deposit the raw sequences (fastq files, sequenced using Illumina and based on a slightly adapted ARCTIC protocol) to ENA under the Project ID: PRJEB44141.
Although we are already analyzing these samples using the computational infrastructure of INAB-CERTH, I would be very keen in having these samples analyzed also through the Galaxy workflow - ideally also performing a re-analysis of older samples in order to assess the impact of changes in the pipeline to the final sequence / lineage assignments.
My question is this: the direct URLs of all ~2000 paired-end reads is already here. Should I make a copy of the links using the provided template file, which I will be changing every week, or is there a different process in place?
Thank you again for setting this up, and for your time and efforts here!

@fpsom add those 2000 links would be a first great start. This repo here more or less gives you the possibility to trigger the entire stack (workflow runs, data export, dashboard update ...). If you are interested we could later integrate it into our automatic monitoring, as we do with CoG-UK. However, this assumes that your samples are produced in the same way, the metadata is complete and the automatic scripts can reason and aggregate the data based on ENA metadata.

@fpsom for optimal results: if you can do separate PRs per library prep or some other logical grouping of the 2000 samples, that'd be excellent. This way some of the variant reports make a bit more sense to look at.

@fpsom for optimal results: if you can do separate PRs per library prep or some other logical grouping of the 2000 samples, that'd be excellent. This way some of the variant reports make a bit more sense to look at.

Aside: it could also be multiple files in a single PR.

Thank you both! We are actually using the exact same library prep, and (hopefully) metadata are complete based on the ENA Data Hub expectations.
So, I'll go ahead and put in a  PR with the ~2000 samples, as a first step, and we can see how it goes from there.

@fpsom just checked the ENA metadata of the samples. Under library_strategy it says WGS for some samples and AMPLICONIC for others. Knowing if just some or all of these samples have been amplified (and if so with which primer scheme) would be rather crucial.

Hm. Thanks @wm75  for catching this - I'm 99.9% certain that this is an error on the metadata itself, as all these samples have been processed with the exact same protocol. However, I will do an internal check and update this issue accordingly.

I've just checked with the web-lab team - all of the samples have been created using AMPLICON as the library_strategy. I'll be updated the ENA metadata as well to actually reflect that, but this may take some time.

@fpsom awesome! Thanks for checking and correcting the records. I'll add all these samples here to our regular bot runs then. This way they end up in one account on Galaxy together with other national genome surveillance efforts.

Closing this issue and the PR cause it will be handled separately.

Thanks @wm75 ! Could you please tag me in the correct place, so that I know where I should be looking?

Sure. Your samples should get mentioned in the JSON file ftp://xfer13.crg.eu/gx-surveillance.json a day after they've been processed. In there you will find links to the corresponding analysis histories on usegalaxy.eu.
By that time (or with an adiitional day delay at most) key results files (BAMs, VCFs and consensus FASTAs) should appear on that same ftp server.
Expect this to be done by the end of the week, but I'll aalso keep an eye on the progress and let you know when everything's done.

That's fantastic, thanks!

@fpsom you should be able to find 1,840 of your samples now on the FTP server. In the JSON file the corresponding analysis batches have "study_accession" == "PRJEB44141". BAMs, VCFs and FASTAs should be available for all of them I hope.
Pangolin lineage assignments for them will become available soon in all_pangolin.tsv.gz on the same server.

We had some processing issues for 177 samples, which I need to investigate and rerun.
For an additional 62 samples of the project the ENA lists more than one run accession per sample. We excluded these cases for the moment because it wasn't clear to us, which of the runs for these samples we should use, or whether it would be best to pool the runs.

@wm75 thank you very much for the notice! I've just checked the JSON file and indeed I can see the different batches as you indicated.
I've also tried viewing the histories/reports so that I can access the files, but (as far as I can tell), an empty page loads instead. For example, I'm using the following link ("history_link": "https://usegalaxy.eu/histories/view?id=c9e92e502f51ebaf") from a batch with only 6 samples (copied below for convenience).
I'm sure that I'm doing something wrong from my side, so any suggestion would be very much appreciated.
  "fb7b89c3694df167": {
    "batch_id": "CERTH-25-05-2021-08:34:12",
    "variation": "https://usegalaxy.eu/histories/view?id=fb7b89c3694df167",
    "samples": [
      "ERR5960501",
      "ERR5960502",
      "ERR5960503",
      "ERR5960504",
      "ERR5960505",
      "ERR5960506"
    ],
    "time": "2021-07-31T12:32:13.341555",
    "report": {
      "history_link": "https://usegalaxy.eu/histories/view?id=c9e92e502f51ebaf",
      "datamonkey_link": "https://usegalaxy.eu/api/histories/c9e92e502f51ebaf/contents/11ac94870d0bb33a4cac297db07f88f9/display"
    },
    "consensus": "https://usegalaxy.eu/histories/view?id=339d5541ccfad9e0",
    "collection_dates": [
      "2021-02-26",
      "2021-02-26",
      "2021-02-26",
      "2021-02-26",
      "2021-02-26",
      "2021-02-26"
    ],
    "study_accession": "PRJEB44141"
  },

Oh, my fault it seems. For some reason the automatic publishing of these histories didn't work so they are not accessible except from within the .eu bot account.  Had forgotten to add PRJEB44141 to the projects for which we want to make data accessible. Should work now I hope.

Yes, that works perfectly now! Thank you for your help on this!

@fpsom pangolin assignments are now on the FTP server as well. Enjoy and if you have any feedback we would be happy to know :)


Add getter for last downstream header byte received

Commit Message: Add getter for last downstream header byte received
Additional Description: Used for latency measurements
Risk Level: WCPGW?
Testing:  Added unit test.
Docs Changes: None
Release Notes: None
Platform Specific Features: None

Also unit test :-)
/wait

I added a unit test. There appears to be an unrelated tcmalloc compile failure at HEAD, so I can't seem to run it locally, but let's see what the automatic checks on the github side does.
For reference, this is the unrelated error:
$ bazel test test/common/stream_info:stream_info_impl_test
INFO: Analyzed target //test/common/stream_info:stream_info_impl_test (1 packages loaded, 28844 targets configured).
INFO: Found 1 test target...
ERROR: /usr/local/google/home/habets/.cache/bazel/_bazel_habets/de688f6285cfa010ac620e8961bfaa1b/external/com_github_google_tcmalloc/tcmalloc/BUILD:296:11: Compiling tcmalloc/arena.cc failed: (Exit 1): gcc failed: error executing command (from target @com_github_google_tcmalloc//tcmalloc:common) /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++14' -MD -MF ... (remaining 35 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from external/com_github_google_tcmalloc/tcmalloc/internal/lifetime_tracker.h:21,
                 from external/com_github_google_tcmalloc/tcmalloc/huge_page_filler.h:32,
                 from external/com_github_google_tcmalloc/tcmalloc/huge_region.h:24,
                 from external/com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:26,
                 from external/com_github_google_tcmalloc/tcmalloc/page_allocator.h:25,
                 from external/com_github_google_tcmalloc/tcmalloc/static_vars.h:40,
                 from external/com_github_google_tcmalloc/tcmalloc/arena.cc:18:
external/com_github_google_tcmalloc/tcmalloc/internal/lifetime_predictions.h:179:20: error: declaration of 'const int tcmalloc::tcmalloc_internal::LifetimeDatabase::kMaxStackDepth' changes meaning of 'kMaxStackDepth' [-Wchanges-meaning]
  179 |   static const int kMaxStackDepth = 64;
      |                    ^~~~~~~~~~~~~~
external/com_github_google_tcmalloc/tcmalloc/internal/lifetime_predictions.h:107:18: note: used here to mean 'constexpr const int tcmalloc::tcmalloc_internal::kMaxStackDepth'
  107 |     void* stack_[kMaxStackDepth];
      |                  ^~~~~~~~~~~~~~
In file included from external/com_github_google_tcmalloc/tcmalloc/common.h:37,
                 from external/com_github_google_tcmalloc/tcmalloc/arena.h:23,
                 from external/com_github_google_tcmalloc/tcmalloc/arena.cc:15:
external/com_github_google_tcmalloc/tcmalloc/internal/logging.h:49:22: note: declared here
   49 | static constexpr int kMaxStackDepth = 64;
      |                      ^~~~~~~~~~~~~~
Target //test/common/stream_info:stream_info_impl_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 5.532s, Critical Path: 3.47s
INFO: 10 processes: 8 internal, 2 linux-sandbox.
FAILED: Build did NOT complete successfully
//test/common/stream_info:stream_info_impl_test                 FAILED TO BUILD

Executed 0 out of 1 test: 1 fails to build.

Risk Level: WCPGW?
famous last words =P

I added a unit test. There appears to be an unrelated tcmalloc compile failure at HEAD, so I can't seem to run it locally, but let's see what the automatic checks on the github side does.

Use clang instead or pass --define tcmalloc=gperftools in the build. I think it's some compiler weirdness with old gcc AFAIR.

 This one would probably resolve with libc++.

Both of those give me the same error as the CC=clang, FWIW.


Replace mustache.js with own fork

We've always used a custom implementation of mustache.js, however, I'd like to move our modified version into a separate repository to make maintaining it much easier.
https://github.com/tmplat-extension/tmplat-mustache
We should then depend on it within the extension.

Alternatively, I'm considering about simply replacing our custom implementation with pure mustache.js, however, this would be a massive breaking change and I doubt users would understand or be happy with it, so I'm simply considering it at this stage. It would make maintenance much easier.
That said; using the latest version of mustache.js is proving difficult. Mainly because it's much better structured than 0.7.2, which our custom implementation is currently based on. I'm tempted to just fork this version and use our file as-is.


Add release action

@mariomac the release action was missing, I took it from the console plugin repo with some minor changes to match the Makefile

Codecov Report

Merging #18 (87b7b14) into main (b6c7a34) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #18   +/-   ##
=======================================
  Coverage   43.03%   43.03%
=======================================
  Files           8        8
  Lines         488      488
=======================================
  Hits          210      210
  Misses        262      262
  Partials       16       16

Flag
Coverage Δ

unittests
43.03% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6c7a34...87b7b14. Read the comment docs.


🌱 Move hub side csr code to csr driver

Summary
Related issue(s)
Fixes #

@qiujian16 the integration test failed, please take a look.

/lgtm


Translation "ru_RU" not working at LoginSecurity 3.0.1

Hi, I had a problem installing the language in the LoginSecurity configuration file, after I indicated the name "ru_ru", nothing changed, and the plugin complained about the lack of this language. I will ask you to respond.
Using a Paper 1.14.4
([LoginSecurity] Can't find translation for ru_ru. Are you upt to date?)

Please use the latest dev build: https://ci.codemc.io/job/lenis0012/job/LoginSecurity/
Create a new issue if you are still experiencing issues.


Package displaying in browser inspector credentials of login

The package is displaying the credentials information when you run the Meteor.loginWithLinkedIn. This should not be displayed since its sensitive data.

It's holds them during handshake, and is accessible  only to your application and user who is authenticating at that time


[14.0.x] ISPN-14428 size() and keySet() are not reliable and return fuzzy results

https://issues.redhat.com/browse/ISPN-14428

CI died, rerunning.


fix(statseg): don't skip zero ErrorIndex entries

Each entry in statseg has an integer value.  Depending on the entry
type, it has a different meaning.  Some types interpret it as a pointer,
while others treat it as an offset.  Null pointer is bad while zero
offset is perfectly fine.
This patch updates govpp validation logic to allow zero values for
ErrorIndex entries.  This enables '/err/null/blackholed packets' counter
to show up.  Previously it didn't export.
Validation logic was cross-checked against the implementation in vpp
(vpp source code, src/vpp/app/vpp_get_stats.c, main function).

For context: vpp maintains counters in a shared memory segment, known as statseg. The segment is exposed so that it is possible to sample counters with extremely low overhead. Vpp itself provides a tool to examine the stat segment (vpp_get_stats). Not entirely unexpected, govpp provides facilities to examine the statseg.


Mindbell activity doesn't disappear

When the setting No sound during music is enabled and the bell rings while audio is playing the Mindbell activity appears and remains.

To be released with 3.2.2