Getting a match with children

I'm using the following code to loop through my sample images using the following code:
	const imageFile = 'eric_2.jpg';

   await faceapi.nets.ssdMobilenetv1.loadFromDisk('./models/ssd_mobilenetv1');
   await faceapi.nets.faceLandmark68Net.loadFromDisk('./models/face_landmark_68');
   await faceapi.nets.faceRecognitionNet.loadFromDisk('./models/face_recognition');
   await faceapi.nets.ageGenderNet.loadFromDisk('./models/age_gender_model');

      // This method returns the files in the specified path
   const files = await api.listFiles(`${__dirname}/images/`);
   const image1 = await canvas.loadImage(`./images/${imageFile}`);
   const detection1 = await faceapi.detectSingleFace(image1, new faceapi.SsdMobilenetv1Options()).withFaceLandmarks().withFaceDescriptor();

   for(const file of files){

		const image2 = await canvas.loadImage(`./images/${file}`);
		const detection2 = await faceapi.detectSingleFace(image2, new faceapi.SsdMobilenetv1Options()).withFaceLandmarks().withFaceDescriptor();
		const distance = await faceapi.euclideanDistance(detection1.descriptor, detection2.descriptor);

		var match = '=> not a match';
		if(distance < 0.7)
			match = '=> match';

		console.log(`Euclidean distance from image: ${imageFile} to ${file} is ${distance} ${match}`);
		console.log('');
	}
}

I take it back it seems like I get a match with everyone I have taken using my phone to my image
**Here is the code output of my kids that is saying its a match:
Euclidean distance from image: eric_1.jpg to dj.jpg is 0.4787967301362279 => match
Euclidean distance from image: eric_1.jpg to drew.jpg is 0.5003818530582609 => match**
Celebrities I'm testing that does not match:
Euclidean distance from image: eric_1.jpg to brad_pitt.jpg is 0.7968919207835512 => not a match
Euclidean distance from image: eric_1.jpg to cindy_crawford.jpg is 0.8775858144569341 => not a match
Euclidean distance from image: eric_1.jpg to kaley_cuoco_1.jpg is 0.8474952774223865 => not a match
Euclidean distance from image: eric_1.jpg to kaley_cuoco_2.jpg is 0.8581044739989803 => not a match
More pictures of me that matches.
Euclidean distance from image: eric_1.jpg to eric_1.jpg is 0 => match
Euclidean distance from image: eric_1.jpg to eric_2.jpg is 0.36870495014601334 => match
Euclidean distance from image: eric_1.jpg to eric_3.jpg is 0.680211514000396 => match
The images of me and my kids are 3456x4608 72dpi.  The images of the celebrities I just downloaded from the web 1435x2154 300dpi.
I downloaded the models from the URL https://github.com/justadudewhohacks/face-api.js-models
Am I doing something wrong?  This is my first attempt using this API.
Thanks

This can be disregarded.   After looking at the landmarks I created I noticed that my pictures were rotated 90degrees.  After rotating them back to properly show the right position it gives me the right results now.


GetEntitiesRecursive method stack overflow possible?

It looks like one of the methods, GetEntitiesRecursive, uses recursion, which could potentially cause a stack overflow. Maybe it should be turned into an iterative method?

Yes, a stack overflow exception is possible at the moment. I'm not sure how deep the directory structure will get to hit that limitation though. It might not be a problem in practice, but it is possible to create a directory structure of infinite depth. That would certainly hit the limit. Making it iterative would not help in those cases.
Best solution would be to add a maximum depth parameter to GetEntitiesRecursive. I'm not quite sure whether the function should result in an exception (MaximumDepthReachedException?) or whether it should skip directories at that depth. Throwing MaximumDepthReachedException would be the most straightforward solution, but it will block usage of GetEntitiesRecursive for infinite filesystems.

This can be easily rewritten using a Stack<> object to store the list, with a check on the stack count to limit depth.  Something like:
const int MaxSearchDepth = 1024;

public static IEnumerable<FileSystemPath> GetEntitiesRecursive(this IFileSystem fileSystem, FileSystemPath path)
{
    if (!path.IsDirectory())
        throw new ArgumentException("The specified path is not a directory.");
    Stack<FileSystemPath> stack = new Stack<FileSystemPath>();
    stack.Enqueue(path);

    while (stack.Any())
    {
        var curr = stack.Pop();
        foreach (var entity in fileSystem.GetEntities(curr))
        {
            yield return entity;
            if (entity.IsDirectory())
            {
                stack.Push(entity);
                if (stack.Count > MaxSearchDepth)
                    throw new Exception("Path search depth limit exceeded.");
            }
        }
    }

This ensures that there is no recursion and significantly reduces memory usage as there is only one instance of the enumerator class created no matter how deep you go.
The only issue is that it reverses the order of enumeration of folders at each level.  This can be resolved by sorting the entity list from GetEntities as an array:
var entities = fileSystem.GetEntities(curr).OrderBy(e => e.EntityName).ToArray();
foreach (var entity in entities)
    yield return entity;
foreach (var entity in entities.Where(e => e.IsDirectory).OrderByDescending(e => e.EntityName))
{
    stack.Push(entity);
    if (stack.Count > MaxSearchDepth)
        throw new Exception("Path search depth limit exceeded.");
}


IFC4X3_ADD2 instead of IFC4X3?

Hi,
When I set the Ifc version to IFC4X3 I get this message:
XMLSchemaValidationError: failed validating 'IFC4X3' with XsdEnumerationFacets(['IFC2X3', 'IFC4', 'IFC4X3_ADD2']): Reason: value must be one of ['IFC2X3', 'IFC4', 'IFC4X3_ADD2'] Schema component
Is IFC4X3_ADD2 not supported or is this just a matter of renaming IFC4x3 to IFC4x3_ADD2 as a choice?

fixed


Fix system name variant chars

Changes
This is quite a big PR so I will try to make a thorough description of the all changes:
Motivation
The motivation for creating this PR was the problems in several issues with the variant characters in member names, e.g. issue #701. The problem is complex, and there was still a problem using these characters in the code when I started fixing this: The regexp for validating member names were not correct and had tried to be fixed, but were still not correct. As a non-US user I was also hit by this problem.
The problem
The variant characters  '#', '@' and '$' - or rather the hex values 7B, 7C and 5B - are the ones causing the problems. They are allowed in IBM i object and member names - but the representation of these hex values change when the CCSID change! On my system running CCSID 277 (Danish) the variant characters are the special nordic letters 'Æ', 'Ø' and 'Å' (additional letters in our alphabet) instead of '#', '@' and '$'! So it does not work having these characters hardcoded in this extension - because they change according to the setup of the system the user connects to.
The solution
My solution was to use SQL to return the local representation of the variant characters - see the change here. The output from this is stored in a new IBM i object property called system_name_variant_chars.local. The corresponding values from CCSID 37 are store in another new property called system_name_variant_chars.american. The two properties are later used for converting names to local or american representation.
Parsing member names
Since the properties for the variant characters are part of the IBM i object and used when parsing member names and validating the names with a regular expression, I had to move the function parserMemberPath from the Tools.js source to the IBMi.js source - and I had to change all the references to this function. This was probably the biggest change of them all.
Difference in retrieving members via SQL and DSPFD
There was another problem: When members are retrieved by using SQL (via QSYS2.SYSPARTITIONSTAT), the file- and member names are returned in CCSID 37 (american), but when retrieved by DSPFD, they are in the locale of the connection! So another change was to have SQL translate the variant characters before returning the values.
Sorting the member list
Before my change, the members were sorted by SQL and by DSPFD - and the result were not in the order of my Danish alphabet. I added a sort function to the list of members returned by getMemberList - in the locale in VS Code...! I'm not sure this is the best solution, since the order is different from other tools like PDM and RDi. What do you think about this?
Sorting the object list
After fixing the order of the members I also added a sort to the object list from getObjectList - also in the VS Code locale.
Searching source files
Finally I fixed searching source files with variant characters in the name.
Conclusion
As stated in the beginning, this (to me) is a big change with code changes and source restructuring. I have tested the code to the best of my knowledge, but some errors may have slipped through anyway. So I hope you will give it a thorough look and test and give me some feedback. Let us have this problem fixed once and for all... 💣
Checklist

[x] have tested my change
[ ] updated relevant documentation
[ ] Remove any/all console.logs I added
[x] eslint is not complaining
[ ] have added myself to the contributors' list in CONTRIBUTING.md
[ ] for feature PRs: PR only includes one feature enhancement.

@chrjorgensen thanks for all your changes! I will get some testing done today and let you know how it goes.

@worksofliam I'm currently testing the change in an environment with QCCSID 65535 (difficult, requires a dedicated LPAR) - and it seems like there are still some problems converting variant chars. Let me work a little bit more on this PR to fix this...

@chrjorgensen 65535 will be the death of us. Thanks for letting me know!

@worksofliam Yeah, I know SQL is disabled when QCCSID is 65535. But even the DSPFD method returns american, when QCCSID is 65535 - even though the user has like 277 (Danish). What should the output be in this case? I would expect Danish characters, not American - what do you think?

@worksofliam I have now tested connections to systems with different QCCSID values - 037 (US), 277 (DAN) and 280 (ITA). And I have made some new changes to get it all right. Please have a look at these changes.
Btw, did you know that the user CCSID takes priority over the system CCSID? Even though the system value is 65535, there are in fact conversion done if the user has a CCSID other than 65535. So the code is not completely correct, it should test the CCSID on the user before testing the system value. I have also corrected this.
One thing still missing is the member text - in SQL it is returned as UCS2 but in DSPFD it is in CCSID 37...

@chrjorgensen That's some serious testing!

Btw, did you know that the user CCSID takes priority over the system CCSID?

In what respect? I think the program that runs our SQL does not respect job (user?) CCSID, but does respect the system QCCSID. I could be wrong. I'd have to recheck.. it's been a minute.
I am going to take a look. I wonder what we can actually do about the DSPFD returning 37 characters.

@chrjorgensen Okay, I've planned some time to take a look at this tonight. Is the only remaining issue member text when coming back as 37 from DSFPD?

@worksofliam Yes, I think the member text when using DSPFD is the only remaining issue (member text using SQL is correct, returned in CCSID 1200).

@chrjorgensen do you think I should go ahead and test now? Do you think this is test and merge ready?

Hi @edmundreinhardt
Me too - with the variant character problem for decades... 😆
The combined list of variant characters could be a solution - however, IMHO it may confuse the user when adding source file/member, what characters are allowed - and that the name can be returned differently to the list than specified by the user (because of conversion to CCSID 37 and back to the CCSID of the job).

@worksofliam Yes, please go ahead 😃

I was worried about that too, but we switch to this solution in RDi decades ago (saves dealing with the 65535 nightmare) and not one person of the many customers worldwide has ever complained.
 This is a very useful list, though!

@chrjorgensen sorry this hasn't been reviewed yet. Just having some time off before starting a new job.

@worksofliam No worry - we all need a break once in a while... 🌤 Looking forward to hear about your new job!
I also enjoyed the break, it made my mind fresher - and I got some new ideas for improvements! 😆

@chrjorgensen good evening! This is the plan: I have to get the next release out for the database changes, then this will be reviewed and merged.
Thanks!

@chrjorgensen I went ahead and merged the conflicts from a previous release. Will review now.

Hi @worksofliam
Your requested changes has been added - please re-review now.

Thank you @chrjorgensen :)


Add a REST call to adjust dsw config for the current server

Sometimes production servers can have a prefix for all the REST-API web apps.
IT's a setting for Web Gateway.
So, if the production server is example.com, then normally MDX2JSON will be at:
example.com/MDX2JSON
but with this setting it will sit at:
example.com/prefix/MDX2JSON
The command that returns the prefix:
w $system.CSP.GetConfig("CSPConfigName")
There is a config.json file in DSW root installation that this REST call should change in a following way:
{
"endpoints": {
"mdx2json": "prefix/MDX2JSON"
}
}

Let's name it
/MDX2JSON/ServerForDSWSetup

Closed in favor of DSW issue


Fix LICENSE in MANIFEST.in

The MANIFEST.in points to LICENSE instead of LICENSE.txt so the license file is not included in the source distribution.

Thanks!


don't ignore encoder failures in json::encode

first getting https://github.com/rust-lang/rust/issues/21470 fixed here, will PR to rust repository later
this is only a partial fix.
do you want the full fix in another PR or right here?

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @huonw (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see CONTRIBUTING.md for more information.

Thanks!


Timestamp can be fed to image in initialzation

Included the possibility for Image to read timestamps. There is no explicit check that they are in datetime.datetime format. Do you think that should be added? I am honestly not to familiar with this.

I think it looks good for now. Let's see if adaptions are needed when we start using this. My recommendation for these type of updates is to simulateneously use the functionality in a use case to see whether the final solution is working out. This leads to less iterations. This is so short, that I would hope that everything works out.

I agree, but I didn't bother writing the code for transcribing the image names to datetime as Benyamine told us that he already has a plan for it. I think/hope that this will not affect any current code, and we will quickly see if everything works out when Benyamine tries it out. You can merge if you think it is ok!


Crash in rayon_core::latch::TickleLatch::set

We're getting a SIGSEGV in rayon under load with cross-ThreadPool dispatch.
   3: <rayon_core::latch::TickleLatch<L> as rayon_core::latch::Latch>::set
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/latch.rs:195
      <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/job.rs:123
   4: rayon_core::job::JobRef::execute
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/job.rs:59
      rayon_core::registry::WorkerThread::execute
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:681
      rayon_core::registry::WorkerThread::wait_until_cold
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:665
   5: rayon_core::registry::WorkerThread::wait_until
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:639
      rayon_core::registry::main_loop
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:759
      rayon_core::registry::ThreadBuilder::run
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:56
   6: <rayon_core::registry::DefaultSpawn as rayon_core::registry::ThreadSpawn>::spawn::{{closure}}
             at root/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:101
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at rustc/49cae55760da0a43428eba73abcb659bb70cf2e4/src/libstd/sys_common/backtrace.rs:130

The faulting RIP is 11cd2c0 here:
00000000011cd2c0 <_ZN10rayon_core5sleep5Sleep6tickle17h2e956c4db5bbecdaE>:
 11cd2c0:       48 8b 07                mov    (%rdi),%rax
 11cd2c3:       48 85 c0                test   %rax,%rax
 11cd2c6:       74 05                   je     11cd2cd <_ZN10rayon_core5sleep5Sleep6tickle17h2e956c4db5bbecdaE+0xd>
 11cd2c8:       e9 03 00 00 00          jmpq   11cd2d0 <_ZN10rayon_core5sleep5Sleep11tickle_cold17ha65d63499113cf3aE.llvm.14863800856586354016>
 11cd2cd:       c3                      retq

The fault address (RDI) is zero.

Looking at rayon code it looks like there might be a data race that could cause this crash.
Registry::in_worker_cross does
        current_thread.wait_until(&job.latch);

and then returns. Meanwhile TickleLatch::set does
        self.inner.set();
        self.sleep.tickle(usize::MAX);

Is it possible that TickleLatch::set sets the inner latch, which releases the in_worker_cross thread's wait_until, so that thread returns, destroying the StackJob, after which the TickleLatch::set continues and dereferences garbage to get self.sleep? I can't see anything that would prevent such a race...

Needless to say this crash is highly intermittent and we have no way to reliably reproduce it.

Assuming this analysis is correct, I guess one way to fix it would be to change TickleLatch::sleep to TickleLatch::registry with type &Arc<Registry> and clone it in TickleLatch::set before setting the inner latch.

Please see #740, and test that if you're able.
That was forgotten because we hoped to land Niko's scheduler rewrite, which had a similar problem and fix already. But we should probably just do a patch release with #740 while we finish the rewrite...

I think it probably would be a good idea to ship a fix given that the UAF has been known for over two months now.

If there are other known unfixed UAF/safety bugs then it would be helpful if you document them somewhere for future reference.

I guess we could use an "unsound" label or something, but really we just shouldn't have let this fix linger at all. I'll merge that PR and prepare a point release.
I'm not aware of any other such bugs. There are deadlock hazards like #592, but that's a different category of problem.

I've released the fix in rayon-core 1.7.1, and rayon 1.3.1 also bumped its requirement.

Assuming this analysis is correct, I guess one way to fix it would be to change TickleLatch::sleep to TickleLatch::registry with type &Arc<Registry> and clone it in TickleLatch::set before setting the inner latch.

Kudos for coming to the same analysis and fix as I had -- that's nice validation!


Portis login

https://docs.portis.io/#/quick-start another username + pass login solution, could be cool to integrate. Looks pretty easy b/c it uses web3

@joeykrug -- think we should investigate this during a UI sprint (eg target it in V2) or should I put this into the backlog + Post V2?

It's simple and like 1 day 1 dev time of work and should improve usability by more than that, let's do it in V2 imo!

Would consider looking @ fortmatic login too (phone # + pin)

Should definitely do this as a product thing.
Just adding methods doesn't necessarily make things better, since there are more choices, etc.
I think anything related to connecting an account and bringing in funds should be designed out before we do any implementation
Paul
<EMAIL_ADDRESS>
From: Joseph Krug<EMAIL_ADDRESS>Sent: Tuesday, March 12, 2019 5:17:04 PM
To: AugurProject/augur
Cc: Paul Gebheim; Comment
Subject: Re: [AugurProject/augur] Portis login (#1092)
Would consider looking @ fortmatic login too (phone # + pin)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/AugurProject/augur/issues/1092#issuecomment-472230569, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAFP-uQcBO67fi6VHWeBhaWAK3AEu7Uiks5vWEOAgaJpZM4bGhmi.

Portis Docs:
http://ec2-52-26-194-35.us-west-2.compute.amazonaws.com/x/d?c=3829001&l=780dfe19-8973-4c4f-8168-c086d3c0b542&r=4ff75ee0-a02d-4a47-9678-b82fd5075796
When this goes into Development we'll want @cillianhunter0 @Chwy5 @matt-bullock to take a look and make sure they are fine with adding another login option + see if there's anything else we'd wanna update around it,

Can probably be added to the timeline for 1.14 or post 1.14 pre v2


Backport 2.16: Change Perl to Python in test builds documentation

Description
Backport of #2366 to mbedtls-2.16
Status
READY

Reviews approved, and CI passing so labelling as 'ready to merge'.


STORM-2416: Improve Release Packaging to Reduce File Size

This brings the binary distribution file size down from > 200MB to ~79MB.

+1

Looks great otherwise. I'd be +1 once build issue is fixed.

@ptgoetz
Could you make the change for master branch? Maybe 1.0.x branch too.

@HeartSaVioR Yes, that's my plan. I prioritized the 1.1.x branch since it's needed for the 1.1.0 release.
I'm not sure we want to apply it to the 1.0.x branch since we have a number of releases from that branch and this change would change the structure of the distribution. I could go either way, though, and am open to others' opinions.

@ptgoetz
OK either applying or not applying this to 1.0.x is fine for me.
Given that you're planning to apply this to master branch as well, I think this is mergeable now.
+1


Publish as a Github Action workflow step

To make your work more easily usable by others, I think you should create an action.yml file, and probably a few other template files:

https://notiz.dev/blog/build-and-publish-your-first-github-action

However, it looks like there is a recent addition to the Marketplace that already does this, so maybe you don't want to offer a competing option... https://github.com/marketplace/actions/cuda-toolkit

The mentioned action currently does not allow to install only specific subpackages on Ubuntu, and also installs samples.
So while full install works fine (except CUDA 9.2. on Ubuntu 16 - some error with log), it takes considerably longer time (4 minutes), compared to 'opt-in' solution without samples. In my case, installing nvcc, cufft, cudart and nvml takes just under a minute.

Perhaps then there is sufficient reason for this to exist as another published action in the Marketplace!

cuda-toolkit on the marketplace is broken and not well maintained. It emits node v12 deprecation warnings, and although someone has opened Jimver/cuda-toolkit#193 to fix that, the maintainer doesn't appear to be working on it. The project's last release was 7 months ago and all commits since then have just been Dependabot. Furthermore, unlike this repo, it does not support latest CUDA v12.
I would very much like to see this repo packaged onto the GitHub Marketplace for GitHub Actions.


epitope and specificity information

This is an issue that Bjoern brought up on the CRWG mailing list, whether part of MiAIRR or just support in formats is worth discussion.

The alternative is to describe the exact epitope recognized as part of the MIARR standard. That is absolutely possible, but will require a substantial update -  I thought the point of using the IEDB record was that this takes away that complexity. If the epitope should be described, minimally you will need

epitope type (linear peptide, discontinuous amino acids, non-peptidic)
epitope molecular structure (3 different fields, depending on type above)
epitope source protein
epitope source organism

There are a bunch of additional fields (e.g. is this the exact epitope / an epitope containing region / a partial epitope). But as a start you might want to err on the minimal info above.

The primary use case at that point was that people are using e.g. MHC tetramer staining to isolate epitope specific repertoires.

Here is what we've arrived to with the VDJdb database: https://github.com/antigenomics/vdjdb-db
It lacks link to IEDB ids, but its a planned feature and is quite easy to add given a linear peptide sequence. Things with non-peptidic epitopes are far more complex. Moreover, the specificity can be inferred by other methods (e.g. expanded T-cell clones from culture).

text copied from #524
IEDB has released a beta query API for their database.
Of interest is they have a curie_map endpoint which returns a structure similar to our CURIEMap. I've noticed however that the IRI goes to the human-readable html page instead of the API which returns machine-readable JSON. For example, IEDB_EPITOPE: 7355 resolves to:
https://www.iedb.org/epitope/7355

versus
https://query-api.iedb.org/epitope_search?structure_id=eq.7355

The nice thing is that now we should be to add a single field in Rearrangement if we want to link to an epitope.
Though that opens the question on whether we should put entries into our CURIEMap for resolving IEDB_EPITOPE or if we should use IEDB's?
We also can consider how we might link with receptors. The IEDB API has tcr_search and bcr_search endpoints:
https://query-api.iedb.org/tcr_search?limit=5
https://query-api.iedb.org/bcr_search?limit=5

The receptor ids have IEDB_RECEPTOR as their CURIE though oddly it's missing from the above curie_map. However, we might want to consider how we can link our Receptor to IEDB's.

So my question would be, which objects in the AIRR data model do we connect with epitope and specificity?
It sounds like linking to Receptor is the main one, but what about individual Rearrangements? If I have a Rearrangement that is a T-cell with a CDR3 of "AALIQGAQKLV" would you want in the AIRR TSV Rearrangement file the ability to record that that CDR3 has a known antigen or specificity (that is there is a TCR in IEDB with "chain1_cdr3_seq": "AALIQGAQKLV").
Does it make sense to add a field to the Rearrangement that is "receptor_id":"IEDB_RECEPTOR:57"
Or is that plain wrong...

So my question would be, which objects in the AIRR data model do we connect with epitope and specificity?
It sounds like linking to Receptor is the main one, but what about individual Rearrangements? If I have a Rearrangement that is a T-cell with a CDR3 of "AALIQGAQKLV" would you want in the AIRR TSV Rearrangement file the ability to record that that CDR3 has a known antigen or specificity (that is there is a TCR in IEDB with "chain1_cdr3_seq": "AALIQGAQKLV").
Does it make sense to add a field to the Rearrangement that is "receptor_id":"IEDB_RECEPTOR:57"

Definitely. Actually I think this was the original idea, like in the recent U24 proposal for links between AIRR rearrangements and IEDB. Now Receptor may be the more logical link, but I don't know if a rearrangement will always be able to be tied to a receptor or not.
For example, tcrmatch is an example tool that could suggest links between rearrangements and IEDB receptors.
At the very least, we (AIRR) should coordinate our plans for Receptor with IEDB as they also have the same conceptual object.

My 2 cents:

Biologically, the reactivity of a Receptor is determined by both chains (i.e., two Rearrangements) unless proven otherwise. There are examples for one of the chains dominating the reactivity, but these are exceptions and should not guide our data model. Therefore having a receptor_id in the Rearrangement object is a no-go. Note that this is different from our discussion whether Cell and Receptor objects MAY be merged into a combined tabular structure for on-disk representation.
I agree that we want interoperability with IEDB and in the short run this could be implemented via a direct reference to their database. But this should not stop us from developing a universal ID generation mechanism for Receptor objects.

@bussec just to be clear, I was suggesting a receptor_id as an ID that points to external information - essentially another annotation as per tcrmatch if I understand correctly.
I did not mean to imply that it was the ID of the Receptor object in the AIRR data model.
Not sure if that clarification helps or not, but my use of receptor_id was ambiguous...

Therefore having a receptor_id in the Rearrangement object is a no-go.

To make sure I understand, this is because we'd need to put arrays in the receptor_id fields which we are trying to avoid with the TSV format? Or it's because normally only one chain is sequenced, so identifying it with a specific receptor would be misleading? If you knew both chains, it seems reasonable to link them with receptor_id in the TSV.
TRA, receptor_id==abc
TRB, receptor_id==abc
I likely confused the matter by mixing a biologically-measured receptor pair, with inferences about similar receptors and similar specificities with tools like tcrmatch. Those inferred links will need to be put in their own fields versus using receptor_id which should be reserved for the biological receptor pair, but the ID nomenclature may be the same for both.

We are probably talking about separate questions here:
What is in a reference?
(independent of the level of experimental/inferential support that the referenced object has)
What does a reference (OWL: Object Property) connecting a rearrangement with an external Receptor description signify?

IS_COMPONENT_OF_ABSTRACT_RECEPTOR, i.e., this sequence would give rise to a component of the receptor described in the referenced record. This does neither imply that the actual receptor has been observered in the current experiment nor that there is any evidence that the annotated rearrangement has actually been paired with the other component REQUIRED for the receptor.
IS_COMPONENT_OF_OBSERVED_RECEPTOR, i.e., this sequences gives rise to a compontent of a receptor that has been observered (directly or indirectly) within the study. This implies that
a. all other REQUIRED components MUST also have been observed in the study and
b. that these components MAY be grouped solely based on receptor_id.
CONTAINS_RECEPTOR" as **merged reference** from Cell`, i.e., this sequence has been observered in a Cell together with all other REQUIRED component for the referenced receptor.
nothing, i.e., such a reference does not exist in the Schema and we decide not to allow for merged objects in our data representations

Ad 1: IMO there is nothing problematic in making this statement, but it will make datasets very noisy, as ANY match based of an single Receptor component MAY be annotated. Furthermore, it could be mistaken as a way to group rearrangements, which it is explicitly not. So in @schristley's example, the references would be a (weak) suggestion in what the rearrangement might be involved in. I am not convinced that this provides an added value to the user.
Ad 2: This introduces a separate grouping mechanism into the schema, an is at variance with what we discussed in #410, i.e. that there is no link in the Schema between rearrangement and receptor and Cell is the object responsible for observed and inferred chain-pairing.
Ad 3: This is closest to the current situation. It suffers from the fact that we have different representation and assumes that we do allow for merged objects.

What is in a receptor?
In terms of the Schema, it would not be a problem to use the proposed Receptor object also for inferred reactivities, if these reactivities are based on a complete receptor sequences. Therefore aggregared scores of a tetramere matrix (or something similar) could be attached as properties to an Receptor object that contains full-length amino acid sequence of both chains, but they cannot exist in isolation. Therefore we would need to discusses whether we would like to link to hypothetical receptors derived from such predictions or find another mechanism for linkage.
How do queries for reactivity work?
I see mainly two scenarios:

Alice wants to know whether there is any reactivity data associated with a rearrangement. Under option (4) this would mean:

query /rearrangemnt for some features of a rearrangement, retrieving the cell_ids
query /cell for the receptor_ids associated with the respective Cell
resolve the identifier and retrieve the Receptor description from an external repository

Bob wants to find rearrangements that are associated with a given reactivity. Under option (4) this would mean:

query the external repository for the reactivity, retrieve a receptor_id (which MAY be a CURIE)
query /cell for the Cells that contain the receptor_ids. Note that the endpoint will return Cell objects that contain direct references to Rearrangements
retrieve the respective rearrangement_ids from /rearrangement.

reference Option (3) would allow to skip the /cell query if merged objects are allowed for API queries.
"external repository" means external to the ADC, otherwise this would be a standardized query to the /receptor endpoint
under Reference Option (1), Alice will potentially get swamped with receptor annotations that are difficult to validate

For the record, this query to IEDEB looks for info for a TCR CDR3
curl https://query-api.iedb.org/tcr_search?chain2_cdr3_seq=eq.ASSIRSSYEQY

The main points originally raised by Bjoern are now included in #404.


Displaylink and evdi working after fresh installation, after reboot, fails to add providers to xrandr

Initially please make sure the problem hasn't been already addressed:

[ x ] Are you using the latest driver?
(https://www.displaylink.com/downloads/ubuntu)
[ n ] Are you using the latest EVDI version?
(https://github.com/DisplayLink/evdi/releases) evdi 1.7.0-2 [AUR]
[ x ] If you are using a DisplayLink device, have you checked 'troubleshooting'
on DisplayLink's website?
(https://support.displaylink.com/knowledgebase/topics/103927-troubleshooting-ubuntu)
[ ? ] Is this issue related to evdi/kernel?
(if it is rather connected to DisplayLinkManager please take a look at support
https://support.displaylink.com or forum https://www.displaylink.org/forum/)

Arch 5.9.14 kernel
Xorg 1.20.10-2
Displaylink <IP_ADDRESS>-4 [AUR]
Evdi 1.7.0-2
I can build and install displaylink on my Arch 5.9.14 kernel, and immediately after installing displaylink, I can see all of my xrandr providers, activate the displays and have no issues.
After a reboot, the providers are gone and cannot be added back. My only way to resolve this is to repeat the displaylink build and install process:

yay displaylink

NOTE: /sys/devices/evdi/version is showing 1.8.0
* displaylink.service - DisplayLink Manager Service      Loaded: loaded (/usr/lib/systemd/system/displaylink.service; disabled; vendor preset: disabled)      Active: active (running) since Mon 2020-12-14 14:16:01 EST; 26min ago     Process: 11860 ExecStartPre=/sbin/modprobe evdi (code=exited, status=0/SUCCESS)    Main PID: 11861 (DisplayLinkMana)       Tasks: 51 (limit: 9125)      Memory: 72.3M      CGroup: /system.slice/displaylink.service              -11861 /usr/lib/displaylink/DisplayLinkManager
Dec 14 14:16:01 bolverkr systemd[1]: Starting DisplayLink Manager Service...
Dec 14 14:16:01 bolverkr systemd[1]: Started DisplayLink Manager Service.
`
No errors from systemd for displaylink, evdi, udl and dkms all appear to be loaded and working after the reboot, but no display providers are found / listed.

I guess you shouldn't be using version 1.8.0 as it's incompatible with the last DisplayLink Manager release.
For kernel 5.9 and 5.10 there were created two releases: 1.7.1 and 1.7.2 (this one supports both 5.9 and 5.10).
Might be worth checking those :)

That is dupe of https://github.com/DisplayLink/evdi/issues/236

Created #245 to gather those, with a bit more [hopefully] useful title.
Closing this one


What's the recommended approach for protecting a staging site from public access.

I have a staging website, staging-xxx.azurewebsites.net and I want to protect it from public access.
That's it: no search indexer find it, and no one without access to see it. Also, the website have user management (identity) and I want a solution that doesn't mess with the user authentication, so I can  be able to send to specific person (and grant access) for review and use, including registering, log in, etc. stuff that normally QA or a client would do.
I read a lot about OAuth2 and OpenIdConnect stuff and seems very complex for this simple scenario. (maybe not, but there are too many flows and none looks simple for this simple case).
I'm currently using Basic-Auth which I know is a crappy solution, that works perfect for my needs but I would love to make it right.
I read about restrict it via IP Address, unfortunately, in Uruguay (where I live) IPs changes every 12 hour, so having this kind of solution will not work in my scenario.
As I should not be writing security software, I'm asking for:

What is the recommended way to implement a solution for this? how is people doing it? It seems to me like a pretty general scenario, so someone should be doing it right. I just need to know what to learn/read/understand. Any guidance is welcome.

If possible, can you provide a sample I can take a look at?

Thanks!

We don't have anything for this.
Azure Websites does support this sort of thing. In your site configure control panel there's a authentication / authorization section. That then shuts down access to anyone not inside your Azure AD.


Building the CLI interface for spotDL

spotDL's backend is ready. It still needs a CLi
** @phcreery working on this. **

I have been working on a personal implementation of this. Is this what you have in mind? It's definitely not done and has a couple of bugs.
https://github.com/phcreery/spotdl-cli

Is this for the new version or current one? This issue was created for the new version that's under dev.

I have been working on a personal implementation of this. Is this what you have in mind? It's definitely not done and has a couple of bugs.
https://github.com/phcreery/spotdl-cli

And No, not what I was thinking, i was thing much simpler. See the reader (it's a muck, but the usage was what I was thinking of) fork

This was for the old version. Okay.

Are you interested in building the cli for the new version?

Yea, I would like to help

Cool. So basically I had a crappy time trying to get the cli to work with multiple processes. And the total rework had some major problems (like, you simply couldn't package it for pypi) but at the same same time has ~100% accurate search.
At the moment I'm reworking the new version.
You could start up on the cli by figuring out how it's going to play out visually.
Like,
If searching it goes like this,

     $description or a mockup screenshot you cooked up with python or paint.

If downloading it goes like this

     $description or a mockup screenshot you cooked up with python or paint.

And get familiar with the dependencies you might want to use for the cli.
If your interested you can look up the progress on the rework at my repo. The docs are practically useless as of now, so just look up test.py over there.

I see that test.py is being used for testing purposed on the songObj use. Is this all the progress in the v3 rework?
I am a little confused as to what is going on in your fork. Looking through the commit history, I noticed there once was a directory containing the old code, has it all been replaced already? Basically what is the current status of the rewrite, and it it all being put in your fork?

(⊙_⊙;)
Well, I actually finished a fully working version. The only issue was that (a) the whole downloads handling was not super reliable and (b) when I tried to build a distribution, it simply would not work. The cause of both problems is that I'm using multiprocessing to speed up the work. Considering that I have to rewrite almost half of the codebase to fix it, I figured I might as well rework the whole thing. which is what you see now.
You can check out the workings of the completed (but un-distributable) version at this commit.
My current work could use some docs I guess. I put that up as soon as possible.

No sweat! Okay, I see now. I was looking at the number of files in v3 (11 .py files) compared to the ritiek master branch (~46 .py files) which is good in terms of cleanup! I just needed to get a feel of what the progress is to know what I have that needs done.
Thanks for the help!

cool. Just a note, don't focus on v3. spotdlv3 cant be packaged (cuz of the afore mentioned multiprocessing problems) look into the spotdl folder instead.

An ad-hock temporary (may be permanent) CLI is ready. For release purposes, it is ready.


[rojak-api] [bug] Embedding join leads to no result

Using join (translated to SQL INNER JOIN) leads to empty results on embedding stuff when the joined table is empty (kayak sekarang table mention dan news_sentiment masih kosong). Harus diganti jadi left_join supaya lebih fleksibel hasilnya.

fixed by #167


Do not rely on HLIR to introduce padding

For non byte-aligned headers.

Codecov Report

Merging #102 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #102   +/-   ##
=======================================
  Coverage   97.05%   97.05%
=======================================
  Files          10       10
  Lines        1698     1698
=======================================
  Hits         1648     1648
  Misses         50       50

Impacted Files
Coverage Δ

p4c_bm/gen_json.py
97.96% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91cdebe...6190242. Read the comment docs.


failing to build cartographer due to protobuf library conflicts.

I'm trying to build cartographer_ros under Ubuntu 16.04 / Ros Kinetic using catkin build.

(the lines above Cmake Error reading my_lib_protodsgdsag/usr/local/lib and
include_directory can be ignored, I have them printed out for debuging reasons)
See the attached picture of the error message I get. From what I found on so far, Ubuntu 16.04 comes with Protobuf version 2.6.1 but cartographer needs at least 3.0.0. I updated to 3.6.1 (successfully I believe) but now there seems to be an issue with the libraries.
How can i resolve this issue? I tried to create simlinks from the old library files at /usr/lib/x86_64-linux-gnu to the location where the updating of protobuf put the corresponding lib files for protobuf 3.6.1, which is at /usr/local/lib. But this couldn't resolve the issue either.

Did you try building cartographer_ros using the old CLI "catkin_make_isolated" ?
Reference from this GIST https://gist.github.com/kdaun/51b7d19bbcd5e0798c8415cd093078fb
and this topic #625
which make the cartographer_ros use the isolated protobuf 3.4.1 (which is not going to effect the as-is system protobuf 2.6 that you're currently on)


List of ALL differences with CPython (for beginners)

It would be nice to have a complete list of all differences with CPython. So that the there would be only defaults that differ from CPython. Such differences can be learned from documentation but to have such a very relevant concise list would be great for beginners.

Hello @kiwi0fruit,
that would be an absolutely useful thing. I'm - again - getting stuck with using Transcrypt because there is a lack of information about what is possible, what can be enabled by pragmas and what does not work with the underlying JavaScript semantics.
Here is a neat demonstration I just puzzled together from a given use-case:
#__pragma__("opov")

def test(s):
	def getWord(stack):
		print("getWord", stack)
		ret = ""

		if stack[-1] != "x": # <- stack[-1] works only when opov is ON
		# if stack[len(stack) - 1] != "x" # <- workaround for opov is OFF
			return ret

		while len(stack) and stack[0] not in " ":
			ret += stack.pop(0)

		return ret

	for word in s.split(" "):
		word = [ch for ch in word]
		word = getWord(word).lower() # <- lower() works only when opov is OFF, else getWord() is called twice??
		# word = word.lower() # <- workaround for opov is ON
		print(word)

if __name__ == '__main__':
	test("Hellox Worldx Test Blahx")

Checking a list's last item (here: stack[-1]) is not possible, except opov is enabled
Calling getWord(word).lower() is called twice when opov is enabled, but not when  opov is disabled
To loop over stack until it is empty, I normally would just say while stack but this doesn't work in any case, only  while len(stack) is a workaround on this?

Am I doing all wrong or is it really the case that I can't use Transcrypt for Python source code without taking heavy attention on all JavaScript-related semantic problems?

You're not doing it all wrong.
For the past half year the focus was on supporting JS modules.
Now that this is done, priority will be given to:

Getting out reporting bugs and making some "low hanging fruit" improvements.
An improved/enlarged set of tutorials
Concise docs about limitations / differences with CPython
A website with live examples of increasing complexity

As it is all done in evening hours and weekends, it won't be finished overnight.
All I can say is that right now the primary target is to lower the threshold / learning curve for Transcrypt.
Kind regards
Jacques

My faulty code from above runs well now with patch #561, maybe this is OK


Add a version to documentation

Add package versions to better understand where the documentation refers to

Done. The new documentation has multi-version supports.


How to customize a JupyterHub and BinderHub landing pages

For http://agu-binder.2i2c.cloud,  we want a customized landing page. The configuration for that hub includes the following:
binderhub:
  config:
    BinderHub:
      template_path: /etc/binderhub/custom/templates
      extra_static_path: /etc/binderhub/custom/static
      extra_static_url_prefix: /extra_static/
  initContainers:
    - name: git-clone-templates
      image: alpine/git
      args:
        - clone
        - --single-branch
        - --branch=master
        - --depth=1
        - --
        - https://github.com/2i2c-org/agu-binder-custom-templates
        - /etc/binderhub/custom
      securityContext:
        runAsUser: 0
      volumeMounts:
        - name: custom-templates
          mountPath: /etc/binderhub/custom
  extraVolumes:
    - name: custom-templates
      emptyDir: {}
  extraVolumeMounts:
    - name: custom-templates
      mountPath: /etc/binderhub/custom

My expectation is that by pushing changes to https://github.com/2i2c-org/agu-binder-custom-templates that should cause a change to the landing page visible at http://agu-binder.2i2c.cloud.
I assume I need to do something with deployer to trigger the appropriate helm action so cause that the template repo to be cloned again so that this hub picks up the changes and visible at http://agu-binder.2i2c.cloud.
Trying
deployer deploy 2i2c agu-binder

causes a helm upgrade
helm upgrade --install --create-namespace --wait --namespace=agu-binder agu-binder /home/jmunroe/2i2c-org/infrastructure/helm-charts/binderhub --values=/home/jmunroe/2i2c-org/infrastructure/config/clusters/2i2c/agu-binder.values.yaml --values=/tmp/tmpldq4skso

but apparently no change to http://agu-binder.2i2c.cloud. I assume this because there has been no change to the helm charts so nothing needs to be modified.
Question: how can can trigger helm to rebuild this pod so that the changes made in https://github.com/2i2c-org/agu-binder-custom-templates get picked up?

Great question, @jmunroe! I think for our hubs' home page customization, in addition to the initcontainer, we have another container that 'syncs' changes every 5-10 minutes. That's missing here, and I'll add that today.

Thanks! I've made an initial pass at applying 2i2c-branding to the landing page.
Some of the "skills" required in customizing a landing page for Binderhub/JupyterHub:

(community task) CSS/HTML experience
(community task) knowledge about Jinja2 templating
(community task) setting up binderhub/jupyterhub locally to be able to develop changes to the templates pages
(2i2c task) update relevant helm charts and/or k8s to be able to deploy a changes to the infrastructure

Web dev can be "easy" if all of the tooling is already set up but can be daunting for the newcomer.  Example: the binderhub template references a dist/styles.css file which isn't a single stylesheet from the binderhub code base but a generated artifact from webpack used in deploying binderhub.
Customization of landing pages (for either JupyterHub and BinderHub) could be something that our communities can be empowered to do.
Future action: develop sufficient documentation to make this possible/reasonable.

As a feature, this is tracked in https://2i2c.productboard.com/feature-board/7803674-product-ideas/features/26289139/detail


Modulation with the sum fusion method

Hello,
Thanks a lot for your work.
I had a question about the sum fusion. When you compute the unimodal output, you divide by 2 the bias of each linear layer. I understand why we do that in the concatenation fusion, but why also in the summation?
if args.fusion_method == 'sum':
            out_v = (torch.mm(v, torch.transpose(model.module.fusion_module.fc_y.weight, 0, 1)) +
                     model.module.fusion_module.fc_y.bias / 2)
            out_a = (torch.mm(a, torch.transpose(model.module.fusion_module.fc_x.weight, 0, 1)) +
                     model.module.fusion_module.fc_x.bias / 2)

Thank you

Hi, thanks for your attention.
Sorry for this minor bug in the code. We have fixed it and thanks a lot.

Great, thanks


Dataset download and camera_path format

Hi, the link of a dataset of Loader Truck scene gave me the error of the file does not exist, could you fix this? By the way, I wonder about the format of the camera_path_filename mentioned in the render uncertainty command
ns-render-u camera-path --load-config={PATH_TO_CONFIG} --unc_path {PATH_TO_UNCERTAINTY} --output-path {PATH_TO_VIDEO} --downscale-factor 2  --rendered_output_names rgb depth uncertainty  --filter-out True --filter-thresh 1. --white-bg True --black-bg False  --camera_path_filename {PATH_TO_CAMERA_PATH} ? Thanks.

Thanks for reporting this! The link should be fixed now.
The path should be the path to the .json file. A sample camera path exists in the linked zip file.


[New Feature]: Use the COM API for getting ProductCode from MSI installer(s)

Description of the new feature/enhancement
Here's a good test candidate: https://github.com/stevencohn/OneMore/releases/download/4.14/OneMore_4.14_Setupx64.msi
Some MSI installers don't expose ProductCode GUID in the Get-AppLockerFileInformation -Path .\OneMore_4.14_Setupx64.msi | Select-Object -Property Publisher command in PowerShell:

Here's an example with the COM API (i.e. https://www.powershellgallery.com/packages/Get-MsiProductCode/1.0/Content/Get-MsiProductCode.ps1) = .\Get-MsiProductCode.ps1 .\OneMore_4.14_Setupx64.msi:

This is the same ProductCode you get when you install the MSI installer on the machine:

This should make @vedantmgoyal2009's automation for fetching ProductCode a bit better.
Proposed technical implementation details (optional)
No response

This can also be used for the UpgradeCode and ProductName in the future.
The only thing that I'm having issues with is releasing the com object. I tell it to release, and I still get this error when attempting to delete the file -
Remove-Item : Cannot remove item C:\Users\Trenly\AppData\Local\Temp\Trenly.Test v1.0 - 2022.01.24-12.37.37.msi: The process cannot access the file 'C:\Users\Trenly\AppData\Local\Temp\Trenly.Test v1.0 - 2022.01.24-12.37.37.msi' because it is being used by another process.
At D:\Git\winget-pkgs\Tools\YamlCreate.ps1:749 char:75


Meross MSG200

I have an MSG200 (non HK) with 2 garage doors (1 & 2).  When I connect it and log in, it shows:
[12/30/2023, 10:44:08 AM] [Meross] Initializing Meross platform...
[12/30/2023, 10:44:08 AM] [Meross] Initializing child bridge 0E:85:85:ED:B3:AA
[12/30/2023, 10:44:08 AM] [Meross] Initialising plugin v10.2.8 | System darwin | Node v18.17.1 | HB v1.7.0 | HAPNodeJS v0.11.1...
[12/30/2023, 10:44:08 AM] [Meross] Plugin initialised. Setting up accessories....
[12/30/2023, 10:44:09 AM] [Meross] [Smart Garage Door Opener] has been added to Homebridge.
[12/30/2023, 10:44:09 AM] [Meross] [Smart Garage Door Opener] initialising with options {"connection":"hybrid","garageDoorOpeningTime":20}.
[12/30/2023, 10:44:09 AM] [Meross] [Large Garage Door] has been added to Homebridge.
[12/30/2023, 10:44:09 AM] [Meross] [Large Garage Door] initialising with options {"connection":"hybrid","garageDoorOpeningTime":20}.
[12/30/2023, 10:44:09 AM] [Meross] [Small Garage Door] has been added to Homebridge.
[12/30/2023, 10:44:09 AM] [Meross] [Small Garage Door] initialising with options {"connection":"hybrid","garageDoorOpeningTime":20}.
[12/30/2023, 10:44:09 AM] [Meross] [Smart Garage Door Opener] initialised with id [2209298533592961070348e1e9a9ae01}].

And then:
[12/30/2023, 10:44:14 AM] [Meross] [Smart Garage Door Opener] current target [closed].
[12/30/2023, 10:44:14 AM] [Meross] [Smart Garage Door Opener] current state [closed].
[12/30/2023, 10:44:14 AM] [Meross] [Large Garage Door] current target [closed].
[12/30/2023, 10:44:14 AM] [Meross] [Large Garage Door] current state [closed].
[12/30/2023, 10:44:14 AM] [Meross] [Smart Garage Door Opener] has been reported [online].
[12/30/2023, 10:44:14 AM] [Meross] [Smart Garage Door Opener] has been reported [online].
[12/30/2023, 10:44:14 AM] [Meross] [Large Garage Door] has been reported [online].
[12/30/2023, 10:44:14 AM] [Meross] [Small Garage Door] has been reported [online].

As far as I can tell.. all is good with config.
However, when I add the Meross as a Bridge to HK, it always says the Small Garage door is open and trying to open or close the Large Garage door controls the Small garage door.
I figure I have some configuration issue but I can't for the life of me figure out what I am doing wrong.  :/
Any ideas would be appreciated when time permits.
Thank you!

Check out this post, the port mappings are off by 1.
Meross app Port 1/Garage Door 1 ----> HomeKit Smart Garage Door Opener
Meross app Port 2/Garage Door 2 ----> HomeKit Garage Door 1
Meross app Port 3/Garage Door 3 ----> HomeKit Garage Door 2
https://github.com/bwp91/homebridge-meross/issues/546#issuecomment-1828692888


portforward.go:331] an error occurred forwarding

K8s 1.9.3 kubectl on MacOS (to 1.9.3 K8s Cluster on Container Linux)
Doing a simple port-forward (f.e. to tiller) on my mac bash results in:
E0216 15:52:41.029173   40160 portforward.go:331] an error occurred forwarding 44134 -> 44134: error forwarding port 44134 to pod 33eb364bdf5a87418a05acd16e35822ec4146eed7e1fca4d41ab2a8cc43b594b, uid : exit status 1: 2018/02/16 14:52:40 socat[6313] E getaddrinfo("localhost", "NULL", {1,2,1,6}, {}): Temporary failure in name resolution

This exception occurs only after trying to send data.
Any idea what can be the cause of this?
Complete log:
> kubectl -n kube-system port-forward tiller-deploy-5b48764ff7-pm6cd 44134:44134 --v=10
I0216 16:32:23.189909   44953 loader.go:357] Config loaded from file /.../kubeconfig
I0216 16:32:23.210702   44953 round_trippers.go:417] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.9.3 (darwin/amd64) kubernetes/d283541" https://<IP_ADDRESS>/api/v1/namespaces/kube-system/pods/tiller-deploy-5b48764ff7-pm6cd
I0216 16:32:25.047948   44953 round_trippers.go:436] GET https://<IP_ADDRESS>/api/v1/namespaces/kube-system/pods/tiller-deploy-5b48764ff7-pm6cd 200 OK in 1837 milliseconds
I0216 16:32:25.048006   44953 round_trippers.go:442] Response Headers:
I0216 16:32:25.048022   44953 round_trippers.go:445]     Content-Type: application/json
I0216 16:32:25.048036   44953 round_trippers.go:445]     Content-Length: 2985
I0216 16:32:25.048048   44953 round_trippers.go:445]     Date: Fri, 16 Feb 2018 15:32:24 GMT
I0216 16:32:25.086489   44953 request.go:873] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"tiller-deploy-5b48764ff7-pm6cd","generateName":"tiller-deploy-5b48764ff7-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/tiller-deploy-5b48764ff7-pm6cd","uid":"6472e10f-12f8-11e8-adc0-0671000007fc","resourceVersion":"2211","creationTimestamp":"2018-02-16T09:04:28Z","labels":{"app":"helm","name":"tiller","pod-template-hash":"1604320993"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"tiller-deploy-5b48764ff7","uid":"64712df6-12f8-11e8-adc0-0671000007fc","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"tiller-token-pst7g","secret":{"secretName":"tiller-token-pst7g","defaultMode":420}}],"containers":[{"name":"tiller","image":"gcr.io/kubernetes-helm/tiller:v2.8.1","ports":[{"name":"tiller","containerPort":44134,"protocol":"TCP"},{"name":"http","containerPort":44135,"protocol":"TCP"}],"env":[{"name":"TILLER_NAMESPACE","value":"kube-system"},{"name":"TILLER_HISTORY_MAX","value":"0"}],"resources":{},"volumeMounts":[{"name":"tiller-token-pst7g","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/liveness","port":44135,"scheme":"HTTP"},"initialDelaySeconds":1,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"httpGet":{"path":"/readiness","port":44135,"scheme":"HTTP"},"initialDelaySeconds":1,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"tiller","serviceAccount":"tiller","nodeName":"k8s-bc-worker-839b7c4","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-02-16T09:04:28Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-02-16T09:04:40Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-02-16T09:04:28Z"}],"hostIP":"<IP_ADDRESS>","podIP":"<IP_ADDRESS>","startTime":"2018-02-16T09:04:28Z","containerStatuses":[{"name":"tiller","state":{"running":{"startedAt":"2018-02-16T09:04:32Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/kubernetes-helm/tiller:v2.8.1","imageID":"docker-pullable://gcr.io/kubernetes-helm/tiller@sha256:394fb7d5f2fbaca54f6a0dec387cef926f6ae359786c89f7da67db173b97a322","containerID":"docker://9aa5cb229a52d53da9dc53d73588067c668657ee937d034b0713191992b3371d"}],"qosClass":"BestEffort"}}
I0216 16:32:25.130036   44953 round_trippers.go:417] curl -k -v -XPOST  -H "X-Stream-Protocol-Version: portforward.k8s.io" -H "User-Agent: kubectl/v1.9.3 (darwin/amd64) kubernetes/d283541" https://<IP_ADDRESS>/api/v1/namespaces/kube-system/pods/tiller-deploy-5b48764ff7-pm6cd/portforward
I0216 16:32:32.044524   44953 round_trippers.go:436] POST https://<IP_ADDRESS>/api/v1/namespaces/kube-system/pods/tiller-deploy-5b48764ff7-pm6cd/portforward 101 Switching Protocols in 6914 milliseconds
I0216 16:32:32.044583   44953 round_trippers.go:442] Response Headers:
I0216 16:32:32.044606   44953 round_trippers.go:445]     Connection: Upgrade
I0216 16:32:32.044625   44953 round_trippers.go:445]     Upgrade: SPDY/3.1
I0216 16:32:32.044647   44953 round_trippers.go:445]     X-Stream-Protocol-Version: portforward.k8s.io
I0216 16:32:32.044662   44953 round_trippers.go:445]     Date: Fri, 16 Feb 2018 15:32:29 GMT
Forwarding from <IP_ADDRESS>:44134 -> 44134
Handling connection for 44134
E0216 16:32:36.106965   44953 portforward.go:331] an error occurred forwarding 44134 -> 44134: error forwarding port 44134 to pod 33eb364bdf5a87418a05acd16e35822ec4146eed7e1fca4d41ab2a8cc43b594b, uid : exit status 1: 2018/02/16 15:32:35 socat[13536] E getaddrinfo("localhost", "NULL", {1,2,1,6}, {}): Temporary failure in name resolution

ok, the reason for this:

empty /etc/hosts file in kubelet container (rkt on Container Linux)
because of a missing "--hosts-entry host" in the kubelet-wrapper script


Catalog rule is not working according to selected attributes, if category is not selected.

Framework Version: 0.1.6

Hi, has this bug been fixed in version 0.1.7? I am still have this issue. The discount does not work select or not select the category. Try using attribute SKU = 'product id' or SKU = 'product name'. Not working for me.

Revamped the entire discount system.


Should maintainers have access to git configuration?

Currently, mods/git.py only shows the config box to superusers:
def prepare_project_hook(self, request, project):
    if not request.user.is_superuser:
        return
    project.extra_info.append({"title": "Git configuration",
                               "class": "info",
                               "content_html": self.build_config_html(request,
                                                                      project)})

Is this intended?

FWIW I added the docs tag because this is (somewhat) blocking the creation of documentation - you cannot really have per-project maintainers until this is fixed.

Good point. Overall I think maintainer code could use quite some renovation. Adding the docs tag back.

Fixed by commits f845f6359ae939d72c40db46710d676755944b0a and (sorry for rushing) f020fcdf1e95ab3f19a9f19e3ea9c3de1c7623fe.


Hollow Buttons

Yeah! Hollow Buttons! Just add the hollow class

I pushed up the fixed include.

I pushed up an addition to the docs and added a test. Interestingly - there some clients it didn't work on at all - https://litmus.com/checklist/emails/public/836f9f6
We'll need to get them working in Outlook.com to get merged in. Is that something you can look at?

Sure. I will see if I can get it done next week.

So... I have discovered that if your link does not have an actual URL set as the href, Outlook.com will ignore certain link styles. Crazy!
I could duplicate the hollow button issues in Outlook.com when I had href="#". However, once I set it to href="https://emailstacks.com" everything worked like gravy!
With that said, I have found a couple small bugs in the hollow styles. I will push up changes today.

I have fixed the hollow buttons so that they are now working properly in all clients, including Outlook. Here is a link to my test...
https://www.emailonacid.com/app/acidtest/display/summary/Z33aiBixtvJ2mHuAIP4NUma21RRVWWKpgCsFETzL9IdH0/shared
Some of my changes modify normal button styles in order to make sure that hollow buttons are sized the same as normal buttons. For example, tiny and small buttons had the padding added twice: once to the anchor and another to the td. This was causing issues with hollow buttons. Now everything is uniform.


Fails to add new user in 10.11.6

I have a user on El Capitan, and when I try to run fde-rekey, the .pkg. I get this cryptic error:
installer: Package name is
installer: Upgrading at base path /
installer: The upgrade failed (The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.)

When I manually download fdeadduser and then ran the postinstall script, I got this:
2017-11-08 08:45:30.308 sysadminctl[91150:11667284] Creating user record…
2017-11-08 08:45:32.125 sysadminctl[91150:11667284] Assigning UID: 2
2017-11-08 08:45:33.047 sysadminctl[91150:11667284] Creating home directory at /Users/fderekey
Found User "fderekey" in user list.
Traceback (most recent call last):
  File "fderekey.py", line 314, in <module>
    main()
  File "fderekey.py", line 290, in main
    if not add_fv_user(fdeadduser, fv_user, fv_password):
  File "fderekey.py", line 151, in add_fv_user
    subprocess.call(add_user_cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied

Any ideas? This isn't on AD, so no password requirements.

Okay. Ran it again, and got this weird bit in the beginning.
2017-11-08 09:02:15.112 sysadminctl[93077:11680865] Creating user record…
2017-11-08 09:02:15.132 sysadminctl[93077:11680865] ### Error:-14135 File:/Library/Caches/com.apple.xbs/Sources/Admin/Admin-633/DSRecord.m Line:501
2017-11-08 09:02:15.133 sysadminctl[93077:11680865] Assigning UID: 2
2017-11-08 09:02:16.143 sysadminctl[93077:11680865] Creating home directory at /Users/fderekey

That looks fine. Can you provide the full log from /var/log/install.log after running the pkg.

Okay, @weswhet. Finally got back to the machine. This is what's in the /var/log/install.log:
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Product archive /Library/Managed Installs/Cache/fde-rekey-1.0.2.pkg trustLevel=100
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: -[IFDInstallController(Private) _buildInstallPlanReturningError:]: location = file://localhost
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: -[IFDInstallController(Private) _buildInstallPlanReturningError:]: file://localhost/Library/Managed%20Installs/Cache/fde-rekey-1.0.2.pkg#fde-rekey.pkg
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Set authorization level to root for session
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Administrator authorization granted.
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Will use PK session
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Using authorization level of root for IFPKInstallElement
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Starting installation:
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Configuring volume "Macintosh HD"
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Preparing disk for local booted install.
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Free space on "Macintosh HD": 48.87 GB<PHONE_NUMBER>2 bytes).
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: Create temporary directory "/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T//Install.92957x6RhxK"
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: IFPKInstallElement (1 packages)
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: Adding client PKInstallDaemonClient pid=92957, uid=0 (/usr/sbin/installer)
Nov  8 08:54:15 NAMEOFMACHINE installer[92957]: PackageKit: Enqueuing install with framework-specified quality of service (utility)
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: ----- Begin install -----
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: request=PKInstallRequest <1 packages, destination=/>
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: packages=(
	    "PKLeopardPackage <file://localhost/Library/Managed%20Installs/Cache/fde-rekey-1.0.2.pkg#fde-rekey.pkg>"
	)
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: Will do receipt-based obsoleting for package identifier com.github.square.fderekey (prefix path=)
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: Extracting file://localhost/Library/Managed%20Installs/Cache/fde-rekey-1.0.2.pkg#fde-rekey.pkg (destination=/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/C/PKInstallSandboxManager/7F9A5C77-AE68-476B-8B3D-C67D8F0E61B6.activeSandbox/Root, uid=0)
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: prevent user idle system sleep
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: suspending backupd
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: Using trashcan path /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/PKInstallSandboxTrash/7F9A5C77-AE68-476B-8B3D-C67D8F0E61B6.sandboxTrash for sandbox /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/C/PKInstallSandboxManager/7F9A5C77-AE68-476B-8B3D-C67D8F0E61B6.activeSandbox
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: Shoving /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/C/PKInstallSandboxManager/7F9A5C77-AE68-476B-8B3D-C67D8F0E61B6.activeSandbox/Root (1 items) to /
Nov  8 08:54:15 NAMEOFMACHINE installd[79389]: PackageKit: Executing script "./postinstall" in /private/tmp/PKInstallSandbox.L4dZBp/Scripts/com.github.square.fderekey.XH5ErT
Nov  8 08:54:16 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:16.197 sysadminctl[92964:11675276] Creating user record…
Nov  8 08:54:16 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:16.217 sysadminctl[92964:11675276] ### Error:-14135 File:/Library/Caches/com.apple.xbs/Sources/Admin/Admin-633/DSRecord.m Line:501
Nov  8 08:54:16 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:16.218 sysadminctl[92964:11675276] Assigning UID: 3
Nov  8 08:54:17 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:17.277 sysadminctl[92964:11675276] Creating home directory at /Users/fderekey
Nov  8 08:54:17 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:17.277 sysadminctl[92964:11675276] Directory at path:/Users/fderekey already exists
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:18.799 sysadminctl[92981:11675513] Killing all processes for UID 3
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:18.801 sysadminctl[92981:11675513] Securely removing fderekey's home at /Users/fderekey
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:18.834 sysadminctl[92981:11675513] Deleting Public share point for fderekey
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: 2017-11-08 08:54:18.843 sysadminctl[92981:11675513] Deleting record for fderekey
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: Found User "fderekey" in user list.
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: Tried 5 times to add user to FileVault and Failed each time...
Nov  8 08:54:18 NAMEOFMACHINE installd[79389]: ./postinstall: ERROR! Failed to add user "fderekey" to FileVault...
Nov  8 08:54:19 NAMEOFMACHINE installd[79389]: PackageKit: releasing backupd
Nov  8 08:54:19 NAMEOFMACHINE installd[79389]: PackageKit: allow user idle system sleep
Nov  8 08:54:19 NAMEOFMACHINE installd[79389]: PackageKit: Install Failed: Error Domain=PKInstallErrorDomain Code=112 "An error occurred while running scripts from the package “fde-rekey-1.0.2.pkg”." UserInfo={NSFilePath=./postinstall, NSURL=file://localhost/Library/Managed%20Installs/Cache/fde-rekey-1.0.2.pkg#fde-rekey.pkg, PKInstallPackageIdentifier=com.github.square.fderekey, NSLocalizedDescription=An error occurred while running scripts from the package “fde-rekey-1.0.2.pkg”.} {
	    NSFilePath = "./postinstall";
	    NSLocalizedDescription = "An error occurred while running scripts from the package \U201cfde-rekey-1.0.2.pkg\U201d.";
	    NSURL = "file://localhost/Library/Managed%20Installs/Cache/fde-rekey-1.0.2.pkg#fde-rekey.pkg";
	    PKInstallPackageIdentifier = "com.github.square.fderekey";
	}
Nov  8 08:54:19 NAMEOFMACHINE installer[92957]: install:didFailWithError:Error Domain=PKInstallErrorDomain Code=112 "An error occurred while running scripts from the package “fde-rekey-1.0.2.pkg”." UserInfo={NSFilePath=./postinstall, NSURL=file://localhost/Library/Managed%20Installs/Cache/fde-rekey-1.0.2.pkg#fde-rekey.pkg, PKInstallPackageIdentifier=com.github.square.fderekey, NSLocalizedDescription=An error occurred while running scripts from the package “fde-rekey-1.0.2.pkg”.}
Nov  8 08:54:19 NAMEOFMACHINE installd[79389]: PackageKit: Running idle tasks
Nov  8 08:54:19 NAMEOFMACHINE installd[79389]: PackageKit: Removing client PKInstallDaemonClient pid=92957, uid=0 (/usr/sbin/installer)
Nov  8 08:54:19 NAMEOFMACHINE installd[79389]: PackageKit: Done with sandbox removals
Nov  8 08:54:20 NAMEOFMACHINE installer[92957]: Install failed: The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.


Recent files ui (Stock vscode Ctrl+R)

I couldn't find a way to get the recent files dialog to open, so I removed the Ctrl-R keyboard shortcuts this plugin adds and now some strange things are happening. How can I restore those ctrl-r shortcuts? And is there a way to open the recent files ui? Thanks!

Just open shortcuts.json and find removed shortcuts. It will be with the minus sign.

I think I found them in keybindings.json. I ended up removing the default cmd+r mapping and assigning it to openRecent like below:
    {
        "key": "cmd+r",
        "command": "-workbench.action.reloadWindow",
        "when": "isDevelopment"
    },
    {
        "key": "cmd+r",
        "command": "workbench.action.openRecent"
    },
    {
        "key": "ctrl+r",
        "command": "-workbench.action.openRecent"
    }


UI album year

Thanks for developing spotify-player. I enjoy using it a lot. One thing that was nagging me was that the interface did not give me a sense of time when browsing an artist's page. I therefore added a config option to display the album names prefixed with the year.
As there was no issue mentioning this, I don't know if this implementation fits your ideas for spotify-player, just let me know if it doesn't. Also, my Rust is a bit rusty, so let me know if you have some suggestions before merging this.

Thanks for fixing the formatting & merging this change.


TypeError: Network request failed

My app is crashing when the Google Analytics fetch is failing.
Should .hit return a Promise so I can catch errors after sending a hit?
"TypeError: Network request failed"

@dylancompanjen good idea.  I'm happy to accept a PR if you have some time to make that change.

@ryanvanderpol Sure, I finally had some time :).

@dylancompanjen thanks for the PRs.  I've merged them in and released 1.0.7 to npm.


Flashli 1.1.11 does not offer protectli_fw6_DF_v1.0.12.rom on FW6D, flashes DF_v1.0.6 instead

On FW6D running Coreboot with the 1.0.6:
Boot Ubuntu 20.04 live, connect WAN, open shell
cd /tmp
wget https://github.com/protectli-root/protectli-firmware-updater/releases/download/v1.1.11/flashli.tar.gz
tar -zxvf flashli.tar.gz
cd flashli/
./flashbios

Expected behavior:
It should offer to flash protectli_fw6_DF_v1.0.12.rom which is current in repo for this release.
Actual behavior: It offers to reflash 1.0.6

Edit: Issue seems to be a packaging issue with 1.1.11
flashli/build/images/protectli_fw6d_DF_1.0.6.rom is in the .tar.gz for the release but the repo itself has 1.0.12

I was considering buying a Protecli product until I saw the issues on this project.  This public demonstration of Protectli's very poor issue handling skills leaves far too much to be desired...

Apologies for the reply. The issue was fixed in 721bb6a
At this time we are closing the issue. Please feel free to reopen this issue if there are any other concerns.


Copy does not work on preview tab

VSCode Version: 1.47.2
OS Version: Windows_NT x64 10.0.14393

Steps to Reproduce:

Open a markdown file and move cursor to any line
Open the markdown preview
Copy some text from the preview
Editor lines are copied instead of the preview

Does this issue occur when all extensions are disabled?: Yes

while pasting the copied text is pasted 2 times.

I cannot reproduce this bug with the steps you provided. I am on VSCode 1.47.2 and macOS Mojave 10.14.6.

Does this reproduce in the latest VS Code insiders build with all extensions disabled?


getting NAN

I'm setting the ui-date directive on an input and getting NAN when selecting a date. If I set type to date, it also throws NAN. Any ideas?
<input
    type="date"
    ui-date
    class="form-control"
    name="date"
    ng-model="showModel.date">

sorry I can't tell enough from this example without a jsbin or plunkr that reproduces the problem

no response, if this issue persists please provide an jsbin/plunkr etc.


Use JavaScript style casing?

This project uses snake_casing - it would match both the stdlib and every other JS library if it was in camelCase. Obviously a breaking change, and just a suggestion!

You are absolutely right. I came from python land and made some bad choices while naming the functions. Will probably stick with the current snake case for now, but this is already something that's been noted :-)


Problem with data encoding

Hi,
I'm trying to adapt this sample to use the local home sdk to interact with an STB.
I'm using the http request flow but i'm having some problems sending the correct data on a post request.
I'm not sure wha is happening, but it seems somehow my data is being changed before being sent in the request.
I have the data i need to sen in a buffer, that is returned by an encryption function from a lib provided by the STB manufacturer
In node.js app i simple do
req.write(buff.toString('latin1'), 'latin1')
or
req.write(buff)
and it works well.
With this this lib i've tried both:
radioCommand.data = buff.toString('latin1');
and
radioCommand.data = buff.toString();
But the data sent is different from the original. Besides the request failing on the STB, i can see on the device that the content-length header value is different from the buffer length.
Here's the log of the request (on the device side) from working request made from a simple node.js ap:
Content-Length:  24
Content-Type:    text/plain
Host:            <IP_ADDRESS>
Connection:      close
Hex                                                                                [m:hex]
0000000000 71 5a 8c 73 b4 60 2b 89 6b 6a 3d 28 51 d9 34 ce   qZ.s.`+.kj=(Q.4.
0000000010 0c 59 b0 57 b3 1c 7f 3b                           .Y.W...;

And the log from a request, with the same data, made using the local home sdk with radioCommand.data = buff.toString('latin1');:
Host:            <IP_ADDRESS>
Connection:      keep-alive
Content-Length:  31
Content-Type:    text/plain
Hex                                                                                [m:hex]
0000000000 71 5a c2 8c 73 c2 b4 60 2b c2 89 6b 6a 3d 28 51   qZ..s..`+..kj=(Q
0000000010 c3 99 34 c3 8e 0c 59 c2 b0 57 c2 b3 1c 7f 3b      ..4...Y..W....;

And the log from a request, with the same data, made using the local home sdk with radioCommand.data = buff.toString();:
Host:            <IP_ADDRESS>
Connection:      keep-alive
Content-Length:  38
Content-Type:    text/plain
Hex                                                                                [m:hex]
0000000000 71 5a ef bf bd 73 ef bf bd 60 2b ef bf bd 6b 6a   qZ...s...`+...kj
0000000010 3d 28 51 ef bf bd 34 ef bf bd 0c 59 ef bf bd 57   =(Q...4....Y...W
0000000020 ef bf bd 1c 7f 3b                                 .....;

if i use radioCommand.data = buff.toString('ascii');, the length is preserved but the data is not the same:
Host:            <IP_ADDRESS>
Connection:      keep-alive
Content-Length:  24
Content-Type:    text/plain
Hex                                                                                [m:hex]
0000000000 71 5a 0c 73 34 60 2b 09 6b 6a 3d 28 51 59 34 4e   qZ.s4`+.kj=(QY4N
0000000010 0c 59 30 57 33 1c 7f 3b                           .Y0W3..;

Any idea on what might be the problem and on how to solve it?
Thanks in advance

It seems that i've declared victory too soon :(
When i said it was working, i was forwarding the requests form my google nest mini, through my laptop to the STB, using mitmproxy. And with this setup it was working fine with a tcprequest.
Now i've removed the mitmproxy and it does not working. The problems seems to be that the http server in the STB nees a persistent connection to work, which i guess that's not the case when white a tcprequest.
Can you confirm this? Anyway to make it persistent?
I'm saying this because i tested sending the same hex packet directly to the STB using the app "PacketSender", and it only work when i ticked the box for persistent connection.
Thanks in advance

This seems related to https://issuetracker.google.com/175943368, would you be OK to move the discussion there?

This seems related to https://issuetracker.google.com/175943368, would you be OK to move the discussion there?

Sure. no problem

Sure. no problem


Point tests to quay images by default

What this PR does / why we need it:
When KubeVirt's e2e test are executed in a standalone mode (against an
arbitrary non-kubevirtci cluster), they fetch all the VM and utility
images from Docker Hub. That causes issues due to rate limiting.
With this change, the default registry is Quay.io, which is aligned with
the rest of the artifacts released under KubeVirt.
This issue is especially problematic in conformance testing, where there
is no option to overwrite the default --container-prefix through
Sonobuoy.
Release note:

NONE

/approve
/retest

/retest

/cherry-pick release-0.41


fast_dev_run should set max_steps

🐛 Bug

Trainer(fast_dev_run=True) should set max_steps accordingly, but max_steps is None
Please reproduce using the BoringModel and post here

https://colab.research.google.com/drive/1PFoozULca2zFtw0Ljor5V8AAGsGqe1Y4#scrollTo=quj4LUDgmFvj
To Reproduce

Expected behavior
max_steps should be set to 1 if fast_dev_run is True else fast_dev_run
Environment
Note: Bugs with code are solved faster ! Colab Notebook should be made public !

IDE: Please, use our python bug_report_model.py template.

Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).

You can get the script and run it with:
wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

CUDA:

GPU:

Tesla T4

available:         True
version:           10.1

Packages:

numpy:             1.18.5
pyTorch_debug:     True
pyTorch_version:   1.7.0+cu101
pytorch-lightning: 1.1.0
tqdm:              4.41.1

System:

OS:                Linux
architecture:

64bit

processor:         x86_64
python:            3.6.9
version:           #1 SMP Thu Jul 23 08:00:38 PDT 2020

Additional context

max_steps should be set to 1 if fast_dev_run is True else fast_dev_run

Do you mean?
max_steps should be set to 1 if fast_dev_run is True else max_steps
also val_check_interval, check_val_every_n_epoch be set to 1 as well??
in the video here it says, fast_dev_run won't create any logs or save checkpoints but unfortunately it does both. Not sure what's the intended behavior.

fast_dev_run won't create any logs or save checkpoints but unfortunately it does both

I noticed it too, it's a bug (I tried to explain here #4629). it should really just be a test that the training loop runs. logging is still executed to some extend but it should not get sent to the actual logger object, no files should be saved. The motivation is to be able to debug the script without polluting the working dir with tons of files :)

yeah, I believe a simple fix would be to disable loggers, checkpoint callback, earlystopping, ... maybe more in the init itself.

Will that fix the original bug filed here - ie. max_steps not being set, which can break the optimizers/schedulers that refer to it?

@tchaton ^^

which can break the optimizers/schedulers that refer to it?

@indigoviolet mind explain this a bit more, what do you mean here?

I meant that the lightning module might need to refer to self.trainer.max_steps to configure its optimizers/schedulers, and this breaks in fast_dev_run.

yeah then max_steps should be set correctly too in such a case. But fast_dev_run is meant to debug, I don't think anyone should set it to a value > max_steps (even if someone does, it will stop at max_steps only) so scheduler might get a higher number of iterations > fast_dev_run, but it won't break in such a case too.


[FEATURE] Support for Ignition and/or Combustion

Is your feature request related to a problem? Please describe.

Some operating systems, for example SLE Micro, no longer ship with support for Cloud-Init.
Describe the solution you'd like

It would be nice if, in addition to Cloud-Init, configuration of guests could be performed using Ignition or Combustion.
Describe alternatives you've considered

If direct support for Ignition and/or Combustion is not feasible, it would already be nice if one could at least add a "generic" configuration drive to the system and allowing the user to specify the directory structure themselves.
For example, have a template system "Configuration Drive" in which the user can create a template and specify: Name of the template (i.e. Ignition), Directory structure (i.e. /ignition), File system (i.e. ext4), Volume label (i.e. ignition), Volume size (i.e. 10MB), File name (i.e. /ignition/config.ign), File content (i.e. "auto-fill specified file with VM specific cloud config script content").
Then, during creation of a VM, the user could, in addition to filling the "Cloud Config" section, select whether they prefer the default "Cloud-Init" template, or one of the user created custom templates (like the above Ignition example). And then the system would populate take the "Cloud Config" script content and populate a temporary configuration drive given the template specifications and attach it during the first boot.
Additional context

As of right now, one would need to manually create a volume and fill it with the configuration data, which is not as convenient as using the "Cloud Config" script box and have the system fill the configuration volume automatically.

I'm running into a similar issue with rancher/rancher where I want to deploy RKE2 clusters on vSphere with SLE Micro as the base OS from a VMware template. There's no way to set the most basic thing like the hostname of the machines derived from the VM name. I must be missing something rudimentary and desperate enough to look for answers in this thread.

Related https://github.com/coreos/fedora-coreos-tracker/issues/1126

I am working on ignition files with Kubevirt / Harvester. I found out that passing an ignition file to a cloudInitConfigDrive works (see https://github.com/kubevirt/kubevirt/pull/9212). Sadly Harvester uses cloudInitNoCloud by default and we need to change it in the VM yaml. However it allows to store the ignition config into Harvester's Cloud Init templates, which is way easier than configuring the drive manually. A simple modification of Haverster to allow the easy use of ignition files would be to allow to choose between cloudInitNoCloud and cloudInitConfigDrive when adding a cloud-init / ignition config.

Hi,
I run into same issue, when setup a RKE2 Cluster with Rancher over my Harvester Cluster with OS OpenSuSE MicroOS.
I tried the following OS Appliance Images:

openSUSE-MicroOS.x86_64-ContainerHost-kvm-and-xen.qcow2
openSUSE-MicroOS.x86_64-kvm-and-xen.qcow2

IT hangs at "JeOS Firstboot" for me.
There seems to be an issue with ignition/combustion, which seems to be included as default inside of the Default Images.
As I see, per Default only Cloudinit seems to be supported?
As Solution, I will try to build my own image and include the following package:
https://software.opensuse.org/package/cloud-init-config-MicroOS
Do anyone know, which Image I should use, so that the Cluster Provisioning over Harvester will work fine?
Kind regards
Torsten

Still looking for this, are there any plans to support Ignite / Combustion?

@IQNeoXen solution worked for me to get Fedora CoreOS running, but I've not been able to find a solution for running OpenSuse MicroOS in a similar way. Has anybody found a solution?

Hi MicroOS / Harvester Community,
I tried quite a lot, to get the provisioning setup over Open SuSE MicroOS and Ignition up and running.
The result to having the ignition user_data inside of the "resource "harvester_cloudinit_secret"
https://registry.terraform.io/providers/harvester/harvester/latest/docs/resources/cloudinit_secret seems not to be work fine.
I configured also the configDrive, but that didnt help. Everytime when MicroOS is booting, its just going into the failure with JeOS Setup.
So, for me - I skipped the idea to have all done with Ignition. The Setup over Cloudinit with the MicroOS Image
https://download.opensuse.org/tumbleweed/appliances/openSUSE-MicroOS.x86_64-ContainerHost-OpenStack-Cloud.qcow2 is just working fine for me.
Also I had the question #4680 for uploading image stuff to Harvester, because I had grub stuck booting problems with the images.
ITs also working fine with the OpenStack Image and harvester Image resource! https://registry.terraform.io/providers/harvester/harvester/latest/docs/resources/image
Just a hint if Image is stuck booting on dracut / virtual console: Please dont keep the VM Disk Image too small ~50GB will help.
As Result of this challenge, I will soon provide a Harvester Module which is using the Harvester Provider for setting up VMs on Harvester!
Have a lot of Fun,
Your Torsten from @torsten-online


Make scripts portable

Test if they run on Linux as well.

Tested on Ubuntu and changed the script to determine the file size using wc -l.