Range Use Plan versioning

Range use plans are living documents and knowing what state they were in at any point time is going to be important moving forward.  Locking down past versions from erroneous updates (like the FTA data push) will be critical as well.
It looks like versioning was at least thought about during dev and when you update a plan record its id changes (which is odd).  Could be that a new record is getting created and the old one purged.  A better solution will be to add a version column to both the plan object and its children.  Some data duplication is expected but given the small data sets we are dealing with it's a non issue.
This issue is to track the work on adding versioning to the api, there will be separate issues for range-web and range-ios.

@calebissharp @micheal-w-wells Is this part of the versioning work we are working this sprint? If so can one of you move it to sprint backlog?

@calebissharp Is there anything I should be doing to support this?

@LisaMoore1 Other than the usual frontend testing to make sure everything is still working, I don't think so. I've also noticed a few bugs in endpoints that I've gone in and fixed to resolve some of the frontend issues.

@LisaMoore1 and @micheal-w-wells I'm going to leave this open until we're sure the JSON snapshots are capturing everything we need, but I think this is probably resolved.

Closing as this was broken into a number of related issues that have been addressed.


FATAL: all hosts have already failed -- aborting

TASK: [cis | 2.1.12 Disable chargen-dgram (disable xinetd service)] ***********
failed: [localhost] => {"failed": true}
msg: cannot find 'service' binary or init script for service,  possible typo in service name?, aborting

Fixed in master. ;)


JIRA: CWFHEALTH-208
Signed-off-by: Andrei Paplauski<EMAIL_ADDRESS>

Based on #61, I believe this will not be necessary. Let's keep this open until that one is merged.

Sure

Closing this in favor of #61


Merge develop into master

Merges develop branch into master with view of deleting develop and release branches

Out of date


Safety, Ethics and openpilot

This post addresses a project that is under active development. This issue will lead to pull requests which make changes to the code in order to address some of the forthcoming concerns. That being said, the questions I am asking reach much further than changes to the source code.
I am very happy to see autonomous driving code in the public domain. This technology will radically change our future; the more people who can work with and understand these concepts, the better. Recently @geohot gave a very motivating speech at a Udacity event. However, it was shocking to hear George claim that he had not heard any critical opinions regarding the safety of his plan. "His plan" being that Comma.ai provide the source code of openpilot while individuals assemble their own Comma.ai NEOs, modify the source code to hack into their particular cars, and run openpilot while driving on public roads. In my opinion this plan is dangerous, and needlessly so. I hope that the community can come to a place where we maintain all the social goods of this open source research opportunity without taking any unnecessary risks. This is especially important because the cost of failure is so high.  We are talking about possible fatalities here; fatalities which are completely avoidable.
Some Safety Risks:

Constant collaboration may prevent anyone from fully understanding the system While the end user has access to the source code it is unreasonable to expect that they will understand the edge cases and nuanced behaviors that come from the interactions of its many parts. This is especially true if openpilot is developed by dozens of people simultaneously in a distributed manner.
Failure modes are poorly communicated and understood Keep in mind that control of the vehicle is achieved by hacking into the vehicle's subsystems such as the Cruise Control or Lane Keep Assist functionality. The documentation for the nominal behavior of these subsystems is not released by manufactures. The behavior of these systems in the presence of a fault or bug in openpilot is even less clear.
Driver assistance systems introduce risks which are not currently addressed @geohot 's stated primary goal in releasing this project is to see the community provide implementations for vehicles beyond those hacked into by Comma.ai. The work required to hack into a car can vary greatly. Some systems use proprietary commands, and even those that are standardized may interpret commands differently. Many systems have different hard-coded limitations. For example @geohot has commented that there are different minimum speed requirements to activate Cruise Control among the cars he has personally worked on. It stands to reason that crowdsourcing is one economical way for a small startup to be interoperable with a wide variety of vehicles. The danger is that some unknown aspect of these systems will only be activated in rare circumstances. Because there has been no thorough systems analysis and no collaboration with the OEMs there is no mechanism to anticipate these rare events. Due to the development process that is being proposed for openpilot, many of these rare events will happen for the first time on the road, perhaps with deadly consequences.
Unknown capabilities of the perception system prevent users from making informed decisions openpilot fundamentally relies on the "visiond" system to provide the path that the car tries to follow. "visiond" is a closed source executable which relies on machine learning to function. It is not clear what data was used to train this module. Therefore, the end user can not form a reasonable expectation of what the car should understand! Furthermore, @geohot has openly admitted there are certain road markings that the car can not navigate well (such as merging lanes) but no exhaustive list of such limitations has been communicated, and perhaps not even created by Comma.ai.
Huge liability for openpilot users While the strategy of not selling anything allows Comma.ai to avoid regulation it does so at the expense of all the people who might contribute to this project. I am not a laywer, but I think any reasonable person can see how dangerous it is to use the Comma.ai NEO on public roads. In the event of a collision the driver of the vehicle using openpilot would almost certainly face legal scrutiny that they could not stand up to. Collisions aside, hacking into your vehicle is very likely to void your warranty. People might be willing to accept these monetary/legal risks, but it should be made extremely explicit to the contributors.
 Most people who download this project will not have the interest or the skill to use it. But somewhere in the distribution of skill levels there are people who are dangerous. They are skilled enough to follow the directions and get everything running, but not skilled enough to really understand what they are doing. Those people are usually kept away from safety critical applications through hiring processes or by being supervised by more senior individuals. The current development strategy is subjecting the public to developers who will take wild risks perhaps intentionally, but certainly unintentionally.

Autonomous vehicles hold great promise to reduce fatalities and enable fundamentally new and better ways to live our lives. There may be some unavoidable risks on the path to that better future, but there are many more avoidable risks. The above points are just a few examples of avoidable risks. To make an exhaustive list is beyond me, but I think it is something we as a community must create. And when we have that list of avoidable risks in hand, it is our duty as ethical developers to attack that list until there is nothing left on it.
edit: changed capitalization of OpenPilot to openpilot

I appreciate you engaging about these topics. Would prefer you did so with a real world identity and without a throwaway account though.
This is not a product for end users. As stated in the README, this is open source software for research purposes only. See https://github.com/CPFL/Autoware and https://github.com/PolySync/OSCC for examples of similar software.
Re: failure modes of closed source cars, this is a concern we have. I would love to see pressure on auto manufacturers to open this information up.
Re: visiond being closed, I address this in the presentation. Consider it a QM, https://en.wikipedia.org/wiki/Automotive_Safety_Integrity_Level#QM part. I believe the Mobileye EyeQ3, the chip Tesla uses, is QM as well. All code actually controlling the car is open source, and that's where the safety checks lie.
Re: the legal status of openpilot, comma.ai does not warranty this software for any purpose. People downloading this code are responsible for using it in a safe and legal way in whatever jurisdiction they may lie. We are not selling or shipping a product, and you are responsible for using this software safely and responsibly.
Re: some individuals will cause a hazard. This can be said about many things from kitchen knives to poorly assembled furniture. As with the two previous examples, which to date have killed way more people than self driving car testing, I believe most people will use this responsibly.
Bottom line, cars are dangerous. Inaction is still an action.

I wasn't the original writer, but your response is a good indicator that you're distancing yourself and Comma.ai from responsibility.  I hear and read increasing attempts to disconnect from accountability and insinuate that anyone who gets hurt brings it upon themselves.    Rather then defend yourself, try actually responding to what people are asking for.  Build a subgroup.  Asking for pull requests is simply a way to delay, deflect focus, and demand that others clean up the Proprietary Pilot mess.  Put up a cash bounty if you want to play that way.
You trumpet this science experiment as if it's superior to everything on the market (except perhaps Tesla).  You use it illegally on public roads.  You avoid government safety legislation.  You keep referring to your device as "open source" when only a subset is (don't fool yourself into thinking QM applies here).  You build up enthusiasm and market yourself freely and widely in the press, badmouthing those who are working aggressively to use and develop safety features, redundancy, and functionality at the risk of visible progress.  That's a cowardly shot for you to take because the bulk of people aren't keeping up with what these companies are doing.  Meanwhile, you're claiming that a crappy cellphone and rehashed CAN code for a narrow range of pre-existing driver-assist hardware replaces all of this.  People are get excited and shouldn't be.
People kill themselves with kitchen knives and poorly assembled furniture.   It's a poor comparison.  Those aren't autonomous or rely on artificial decision making designed by someone else.
Bottom line, you will kill people with a product the government pleaded you not to distribute and that a growing number of people are seeing for what it is: a cobbled together hack of Instructables.

Huh? Proprietary Pilot? Cash bounty? Don't fool yourself into thinking QM applies here?  We use it illegally on public roads? (citation needed)  People are get excited and shouldn't be?
The government asked us to delay sales of a product. We did exactly that. As most people seem to understand, this is research code only. Happy to engage constructively, but If this conversation becomes toxic I will close this issue.

Citation very publicized.  I don't work for you.  It's out there amongst all the other publicity you've been embracing.  There's plenty of lockerroom talk about how great, perfect, and revolutionary this project is, but little to none about safety and its use only as research.
I read the NHTSA response.  Why apply for advancement of a product based on code considered "research"?  For attention?  For credibility?  To attract developers?
George, if your project means something, stand behind it in all aspects.    Insurers, laws, and automakers identify the autonomous car now as the "driver" and responsibly party.  That means Comma.ai has/will have legal ties to consequences.  You posted two other projects where safety is deliberately written into the design.  It's discussed regularly on the mailing lists and publicly.  Do the same.  Make it (and your users) a priority.
It's good that you're willing to discuss things constructively despite a rephrase of words and threat of closing this issue (fortunately, it's being read and is archived).  I and others have been promoting constructive ideas and I've spoken honestly and openly.  I haven't read anything constructive by you yet, particularly in these areas which seem most concerning to many of us.  Mostly I see an unwillingness to talk genuinely about the issues.
If closing this issue is an easy way to move forward and live in the fog of fanboys and avoid criticism, no worries.  Others will speak up.

This is not an autonomous car. This is a repository on GitHub with free software that implements a driver assistance system. We discuss safety @ https://github.com/commaai/openpilot/blob/master/SAFETY.md
I'm closing this issue, if you have specific actionable concerns please open issues for them.

To the original poster, both the constant updating and car failure modes issues are things we'd like to address. Suggestions? We already had planned to make it more clear when the code updates, and we'll be probably adding a release branch for stability. Re: car failure modes, we've done our best to reverse engineer error codes from the car, but I agree a lot more work needs to be done in this area.

@toxicgumbo unfortunetly this whole project seems to be very limited, with core components closed source, and the founder very unfriendly :-( just look how he closes most of the issues.
If you'll like to contribute to an open initiative not tied to any company profits, that is looking to tackle and solve all problems related with Self Driving Cars, check out this new organization:
http://github.com/ossdc
Join the discussion here:
http://ossdc.org/join

I fail to understand the criticism towards this project here. It's free code and you do whatever you like with it. One could either make good things from it or bad. It's for research and exploration and not a product.
By definition of open-source and MIT license, there is no liability of any sort on the creators of this project. Everyone in the industry can learn and take the good parts and ignore the bad parts.
Can we please work towards making cars safer together? And not try to shoot down new projects for lacking complete code release or lack enhanced safety checks. It's not a product, it's not mature.

The original poster and follow-up criticism in this thread are not constructive. They cite fear and uncertainty but no concrete suggestions on how to deal with it.
Safety:
Human drivers are unsafe and unreliable. Having an additional assist system by definition can only help improve this. Please understand the difference between safer and safe. Nothing is perfect - not even your Tesla.
Open-source:
Trying to dictate what parts of the stack someone should open is like asking for a free lunch. Yes, some parts of not open. But its the same strategy as an Android phone vs. the Android OSS or the Chrome browser vs. Chromium project.
Research:
Personally I have learnt a lot by going through this project - I do not intend to assemble one, but it has been educational to me.
Public Roads:
A discussion about drive-by-wire cars is different from driving these development/research cars on the road. If someone chooses to use someone else's code on the road and put themselves at risk - that is their own fault and very silly of them.
Let's keep the discussion of open-source and research of self driving vehicles on one side and use of said vehicles and corresponding safety on public roads separate. By choosing the merge them, the user is taking the risk in their own hands and should very well be aware of what they are doing.
This is the same as driving - by choosing to drive the car yourself rather than let someone more experienced do it - you are putting yourself and others at risk.

Re: constructive criticism. We are always looking for ways to make openpilot safer. We believe that self driving car researchers are safer than other groups of people legally allowed on public roads, namely teenagers, old people, and "buzzed" people.

Hi George, I saw your recent presentation at Udacity and thought you would
make a good speaker for Singularity university. It would be a great way to
promote Comma and also I believe they actually pay presenter (nice bonus).
Here is there note back regarding my inquiry...
Exponential Conferences Support
7:01 AM (56 minutes ago)
to me
Hello Robin,
Thank you for your inquiry!
If you would like to submit a speaker request for yourself or another, just
shoot me an email with the following information:
The name of the summit (Exponential Manufacturing, Exponential Finance, SU
Global Summit, Exponential Medicine) that you think the speaker would be a
good fit for.
Please provide a little background on the speaker: personal bio, past
speaking events, company they work for, CV, ect
We will also need name and contact information for the speaker you wish to
nominate. It is also helpful to have a topic and brief description of the
talk.
Once I receive that, I can send it to my Conference Director for review.
Best,
Dave Kutz
SU Conference Support
<EMAIL_ADDRESS>On Mon, Dec 12, 2016 at 2:45 AM, George Hotz<EMAIL_ADDRESS>wrote:

Re: constructive criticism. We are always looking for ways to make
openpilot safer. We believe that self driving car researchers are safer
than other groups of people legally allowed on public roads, namely
teenagers, old people, and "buzzed" people.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/commaai/openpilot/issues/26#issuecomment-266396975,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ASltNJjOy3WgCO5rB715wXL4oWMCTzlcks5rHSXRgaJpZM4LJWRG
.

The following two comments seem to have gone missing:
26#issuecomment-266348650

Marius Slavescu<EMAIL_ADDRESS>to Mention, commaai/openpi., me
@toxicgumbo unfortunetly this whole project seems to be very limited, with core components closed source, and the founder very unfriendly :-( just look how he closes most of the issues.
If you'll like to contribute to an open initiative not tied to any company profits, that is looking to tackle and solve all problems related with Self Driving Cars, check out this new organization:
http://github.com/ossdc
Join the discussion here:
http://ossdc.org/join

26#issuecomment-266363034

Param Aggarwal<EMAIL_ADDRESS>to commaai/openpi., me, Mention
I fail to understand the criticism towards this project here. It's free code and you do whatever you like with it. One could either make good things from it or bad. It's for research and exploration and not a product.
By definition of open-source and MIT license, there is no liability of any sort on the creators of this project. Everyone in the industry can learn and take the good parts and ignore the bad parts.
Can we please work towards making cars safer together? And not try to shoot down new projects for lacking complete code release or lack enhanced safety checks. It's not a product, it's not mature.

Two important words in this thread: constructive and research.

The original poster and follow-up criticism in this thread are not constructive. They cite fear and uncertainty but no concrete suggestions on how to deal with it.

Constructive: Constructive is what one makes of it.  If criticism leads to a dismissal of concerns and closed tickets, then there is nothing constructive going on. If my tone is somehow offensive to others here, my apologies. However, expressing concerns does not require expressing concrete suggestions.  There is genuine fear to be recognized here--people are writing code around closed systems anticipating the movement of 3-ton vehicles in the company of the public--and "how to deal with it" is exactly what should be an ongoing discussion (from day 1).  By putting this code in the public and claiming a reliability level on par or exceeding systems known to be some of the safest around, the public is inherently less safe.  Concrete suggestion: Active and ongoing public discussion of security and safety concerns ranging from foggy weather, to CAN injection, to external electrical interference, to tire blowouts, device hardlocks, optics failures, HMI intrusion (deliberate or otherwise), non-responsive driver, unexpected response from subsystems, etc.
Research: The use of this word, here, in this project, is debatable; at best it's research with a soft R.  CAN packet recording/replay in "a garage in San Francisco" and programming around that (while hoping not to trigger undocumented and unknown proprietary module code) isn't research, it's more akin to hackaday.io jailbreak project.
And there-in is the major issue, particularly with issues of safety.  Misleading claims such as "It's about on par with Tesla Autopilot at launch, and better than all other manufacturers" create a false sense of security with those who see the examples in action with real people.  Moreover, users, testers, and developers cannot possibly understand the entirety of the codebase and must, on some level, put trust in the environment and code maintainers/developers having an emphasis on safety.  This is impossible, however, since we're talking about a best-attempt of reverse engineering on several vehicle platforms using propriety devices and no relationships with those product manufacturers or access to complete module code being explored through CAN sniffing.
As a reminder to folks, this public repository exists because the project was deemed unsafe--in a very clear and seemingly restrained statement--by the NHTSA ("dedicated to achieving the highest standards of excellence in motor vehicle and highway safety").
And here's a choice quote from The Verge:

Last year I met with most of the CEOs of the big auto companies. A lot of them are just hopeless, all adamantly convinced they must own self-driving technology, in house.

There are hundreds of vehicle manufacturers the world over and Mr. Hotz met with most?  And they're hopeless because, in reality, they have numerous business, engineering, and IP relationships involving hundred of employees and didn't want to contract with or purchase the IP of Comma.ai?  Perhaps they know what they're looking at.
Either way, open sourcing components of this software means it moves ahead, gains traction from free press, picks up some free developers, and attempts to send a big middle-finger to the NHTSA and those who weren't interested.  Short-lived gains.
This is where there's a difference between research and Research and why when someone like Musk says he's unconcerned, I believe him.
research here is small group of siloed people who are attempting to stand on the shoulders of giants and claim they are now on par with major OEMs and businesses (groups that do millions of simulations, tests, safety checks, data analysis, and use a variety of redundant sensors).  Where are the relationships with the companies making the driver-assist equipment?  Where are the NDA whitepapers?  Where's the access to source code and full behavioral processes?  Where are the PhDs, engineers, refereed papers, conference proceedings and everything that goes with doing Research?  Can a few people make a major contribution to the scene?  Sure, but don't claim this is safe when you're working with a shoddy cell camera behind a windshield in optimal lighting using basic understanding of the equipment you're controlling.  All these bold claims in writing and in the press are for brand inflation and are misleading.
Research elsewhere in the field, particularly where Mr. Hotz has called accomplished companies and research groups who develop safe and proven technologies, "losers".  The two projects mentioned above, Autoware and Polysync, actually do have their hands in Research.  Autoware is largely tied to the University of Tokyo and has established connections with trained researchers, engineers, driver-assist manufacturers, and has partnerships with OEMs.  Polysync has been in use for some time in autonomous and driver-assist vehicle research and also has partnerships with the big loser brands and OEMs.  They have very different lifestyles, but are taken very seriously because they don't exist in a siloed development environment, the relationships are made, and are reviewed and tested by experts.  For a good example of an automotive project, look at Automotive Grade Linux.  It's a different beast, but an open group that is fully transparent, doesn't go around trashing everyone in the industry, considers safety a top issue (as CAN injection/interference and ECU control/confusion are a reality), and has a working relationship with OEMs and academics.

is is the same as driving - by choosing to drive the car yourself rather than let someone more experienced do it - you are putting yourself and others at risk.

It's a bit generalizing to suggest here and in other words that driverless cars are safer than humans, particularly at this stage of the game.  If openpilot is to gain traction, it will need to leave the jailbreak scene and not burn bridges with equipment manufacturers getting there.  As is, it's at a state where it puts everyone interested at risk.  Even the occasional code comments expressing uncertainty suggest that.  "Open source" won't be a cover or excuse, either.  All eyes will look for someone to blame and they'll turn to Comma.ai, the developers, and possibly the equipment manufacturers who had no involvement.  It will come down to "this downloadable software killed a family of 5" and there won't be the maneuvering or fine-and-forget-it freedom that Tesla had.
Unless you try to pin the blame on the original "OpenPilot" group:  :-)
https://en.wikipedia.org/wiki/OpenPilot
All this aside, I firmly believe this project will be a cash cow in one form or another, whether it should be or not.  That might be through support, brand identity and sale, contract hiring, licensing of more mature code (debatable unless not based on reverse engineering), talks, general awareness and insight, etc.  It's easy for me to replace "Comma.ai" with "Megasquirt"--a company that people still think in 2016 is open source and that has built itself on the back of free developers, breaking engines through features over safety, giving a talk that starts off with a slide that says the users are to blame for problems, closing issues, banning people from IRC and their forum for speaking up.  I've worked closely with demagogues, cult of personalities, sociopaths, and arrogance--all have their place is product design, leadership, and deployment, but even those personalities knew safety inherent in the code and for users is the first design priority.

We believe that self driving car researchers are much safer than certain other groups of people legally allowed on public roads, namely teenagers, old people, and "buzzed" people.

Concrete suggestion: Assume they're not, get to know OEMs, work together, then program and test, test, test, accordingly.

Despite my concerns, keep up the momentum and best wishes.  Looks like a lot of fun in the right setting and circumstances.

(Responding, as I feel that someone needs to be vocal against the "regulate-it" argument.)

I appreciate your response and am able to see your perspective. It would be amazing if everything was the way you describe it.
I'm sure if they had 50 more people, he would assign 24 to legal, 24 to safety and continue advancing the state of technology with the rest two. The point being that just like education and health, if transportation is also going to be an ultra-regulated field - then we are going to be hurting innovation and hurting garage startups from ever being given a chance.
The way I understand it, they are looking for after-market and OEM manufacturers outside the US when no one in the US took interest. Also, they personally deem the AI layer as proprietary and everything else is commodity - hence the open source move. Once they get a partner and are able to work towards an integrated project, they will be able to do the kind of intensive safety-focused tests and enhancements as you mention.

I'm not completely sure what we are discussing here. If one has any ideas on how to make the code more robust and safe, they can open a PR. Also, I think his comparison to other systems was in terms of functionality/performance.

Off topic posts will be deleted. If you have specific actionable concerns please open issues for them. This is not the correct place to promote your comma.ai competitor, invite me to come speak, discuss if certain groups have a monopoly on the word "research", or debate whether Megasquirt is the devil. Your suggestion, "Get to know OEMs" has nothing to do with openpilot, an open source driver assistance system. This is not the comma.ai forum, this is a piece of free software.
Topic locked.


Get product recs

Added BDD Testing For Get Recommendations
Added UI for Get Recommendations

Codecov Report

Merging #90 into master will decrease coverage by 0.16%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #90      +/-   ##
==========================================
- Coverage   94.71%   94.55%   -0.17%
==========================================
  Files           3        3
  Lines         265      257       -8
==========================================
- Hits          251      243       -8
  Misses         14       14

Impacted Files
Coverage Δ

server.py
93.27% <100%> (-0.43%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ebc48d8...92e044d. Read the comment docs.

merge conflict resolved


Please add this parser to the list of parsers on esprima.org's speed comparison page

Hello
please add the jsRube parser to the list of parsers on esprima.org's parser speed comparison page.
It:

always records the location data on every node,
always records the 'raw' value on tokens that can have values,
catches as many early errors as it can (actually is going to catch all), and
has correctly[a] parsed the whole "3rdparty" corpus (in esprima's "test" directory), and
passes the (non-failure) ".tree.json" and ".module.json" esprima tests, and fails on the fail cases (though without any error message yet),

so I guess it would be decent enough a parser to have a slot on that venerable list (at least with a 'beta' after it.)
(a) in direct comparison against esprima's parses

:+1:
While we're in there, we should add version numbers next to each parser and update them all to the latest version.

Thanks!

Thanks!
(apologie for first posting with my other, indecent nick.)

@nenionoda Once jsRube gets some more adoption, then feel free to create a PR to add it (the site is on the gh-pages branch). There are too many parsers out there and we can't include everything.
@michaelficarra Every parser is served from the hosted browser version (often, from CDN). The version number is obtained from the parser itself (if it offers one).

@ariya This parser is notable for its performance. In a performance comparison, it would be appropriate to include it.


Token Logos are not always round

we dont always make the token logo a circle, leading to ugly white borders

Also happening in the Strategies page:

And the portfolio page

@tiagofilipenunes pr is merged but issue still open and I see missing round logo here:

@pingustar got merged into #756


jsdoc object generator function

Hi,
I started to work on a jsdoc type generator function within simple schema, so it can be used to print out the type of a schema that you later want to use in a @type{....} annotation for example
see this example https://github.com/mxab/simple-schema-jsdoc-example

@mxab I am thinking whether this might be better as an add-on package.

@aldeed sure makes sense
here you go mxab:simple-schema-jsdoc
I also rewrote it

Thanks. I added a link in the readme.

cool!


Add block for ECR

Summary

Implementation of an ECR version of https://github.com/PrefectHQ/prefect/pull/6303
Requires https://github.com/PrefectHQ/prefect/pull/6328, and consequently prefect>=2.1.0.
Relevant Issue(s)

Closes https://github.com/PrefectHQ/prefect/issues/6230

Can someone explain to me how coverage works? It's telling me that functions aren't covered even though they definitely are.

Can someone explain to me how coverage works? It's telling me that functions aren't covered even though they definitely are.

Can you post a coverage report so that we can see what it's claiming isn't covered?

interrogate..............................................................Failed
- hook id: interrogate
- exit code: 1

= Coverage for /Users/mz/dev/prefect-aws/prefect_aws/ =
- Detailed Coverage -
| Name                                                       |  Status |
|------------------------------------------------------------|---------|
| ecr.py (module)                                            |  MISSED |
|   ElasticContainerRegistry (L13)                           | COVERED |
|     ElasticContainerRegistry.login (L28)                   |  MISSED |
|     ElasticContainerRegistry._login_sync (L31)             |  MISSED |
|     ElasticContainerRegistry._parse_token (L37)            | COVERED |
|     ElasticContainerRegistry._get_client (L45)             |  MISSED |
|     ElasticContainerRegistry._get_token_and_endpoint (L48) |  MISSED |
|------------------------------------------------------------|---------|

- Summary -
| Name   | Total | Miss | Cover | Cover% |
|--------|-------|------|-------|--------|
| ecr.py |     7 |    5 |     2 |    29% |
|--------|-------|------|-------|--------|
| TOTAL  |    36 |    5 |    31 |  86.1% |
|-------------------------------------------------|
| (5 of 6 files omitted due to complete coverage) |
- RESULT: FAILED (minimum: 95.0%, actual: 86.1%) -

❯ pytest --cov-report term --cov=prefect_aws.ecr tests/test_ecr.py
=========================================================================================================== test session starts ============================================================================================================
platform darwin -- Python 3.8.3, pytest-7.1.2, pluggy-0.13.1
rootdir: /Users/mz/dev/prefect-aws, configfile: setup.cfg
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, env-0.6.2, flaky-3.7.0, asyncio-0.19.0, timeout-2.1.0, respx-0.19.2, flakefinder-1.0.0, cov-3.0.0
asyncio: mode=auto
collected 4 items

tests/test_ecr.py ....                                                                                [100%]

---------- coverage: platform darwin, python 3.8.3-final-0 -----------
Name                 Stmts   Miss  Cover   Missing
--------------------------------------------------
prefect_aws/ecr.py      37      0   100%
--------------------------------------------------
TOTAL                   37      0   100%

Required test coverage of 80.0% reached. Total coverage: 100.00%```

Ah interrogate is failing due to lack of docstring coverage. It looks like there are few methods on ElasticContainerRegistry that need docstrings. Test coverage looks great!

Oh that's what it's asking for 🤦 we're enforcing docstrings on private methods?

Enforcing docstrings on private methods appears to be the default behavior for interrogate. You can add ignore-semiprivate = true to setup.cfg to override that behavior.

This looks awesome, love this token expiration handling magic, so cool! Currently QAing this, will update soon

This is blocked pending a proper implementation of https://github.com/PrefectHQ/prefect/pull/6478

@zanieb what happened to this code? doesn't seem to be on master, thank you!

@feliperazeek this was not merged due to changes in the upstream design and this branch was deleted so this code is no longer available.


KubernetesIntegration.md is not pinned to specific versions

https://github.com/projectcalico/calico-containers/blob/master/docs/cni/kubernetes/KubernetesIntegration.md tells me to use http://www.projectcalico.org/builds/calicoctl rather than a specific version, and tells me to use https://raw.githubusercontent.com/projectcalico/k8s-policy/master/examples/policy-controller.yaml, which calls for the latest policy controller image.

CC @caseydavenport

Thanks for spotting these.  I definitely want to pin down all the places we reference master builds and make sure they're stuck to a specific version.

@robbrockbank what is the position on pinning version in the docs?  I thought master docs were meant to use the latest versions, whereas releases are supposed to pin these.

This doc has moved to the main Calico repo / docs: http://docs.projectcalico.org/v1.5/getting-started/kubernetes/installation/
Versions are pinned by this guy: https://github.com/projectcalico/calico/pull/104


Publish an initial version of the package

I've never published a go module before, so I'm looking at how to do that "correctly".

https://go.dev/doc/modules/publishing

Numbering

https://go.dev/doc/modules/version-numbers

The initial version is going to be v0.0.1, and will stay in v0.0.x until I feel confident enough to write a blog post, record a screencast and make a first initial announcement.

Additional resources about module development:

https://go.dev/doc/modules/developing
https://go.dev/doc/modules/release-workflow

One thing to try is coding against an unpublished module: https://go.dev/doc/modules/release-workflow#unpublished
which recommends to use the replace command in a local command package that uses it, as referenced here: https://go.dev/doc/modules/managing-dependencies#local_directory
module example.com/mymodule

go 1.16

require example.com/theirmodule v0.0.0-unpublished

replace example.com/theirmodule v0.0.0-unpublished => ../theirmodule

go mod edit<EMAIL_ADDRESS>go get<EMAIL_ADDRESS>
Although it might also be possible to use a go workspace to do that?

https://go.dev/doc/tutorial/workspaces

I also want to set up some proper github automation to run unit tests and have some funky little labels: https://github.com/wesen/raza


Update API doc samples for null safety

We need to go through all our API docs and update the sample code to null safety.

I would like to help.

Duplicate of https://github.com/flutter/flutter/issues/69123.


Use end-to-end middleware tests

From #38:

One complication is that properly testing this change to hawkrest means overhauling the tests to actually test the middleware end to end, like eg:
https://github.com/encode/django-rest-framework/blob/master/tests/test_middleware.py
(And ideally also overhauling the auth parts too, a la https://github.com/encode/django-rest-framework/blob/master/tests/test_authentication.py)

Seems useful. Thanks for filing.


Uncaught TypeError: Cannot read property 'indexOf' of undefined

Installed atom-ember-cli-helper
click on c/t button

Atom Version: 1.0.19
System: Mac OS X 10.10.4
Thrown From: ember-cli-helper package, v0.9.0
Stack Trace
Uncaught TypeError: Cannot read property 'indexOf' of undefined
At /Users/anil/.atom/packages/ember-cli-helper/lib/ember-cli-helper-view.coffee:84

TypeError: Cannot read property 'indexOf' of undefined
  at EmberCliHelperView.module.exports.EmberCliHelperView.getPathComponents (/Users/anil/.atom/packages/ember-cli-helper/lib/ember-cli-helper-view.coffee:84:26)
  at EmberCliHelperView.module.exports.EmberCliHelperView.switchFile (/Users/anil/.atom/packages/ember-cli-helper/lib/ember-cli-helper-view.coffee:96:51)
  at HTMLButtonElement.<anonymous> (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/atom-space-pen-views/node_modules/space-pen/lib/space-pen.js:220:36)
  at HTMLButtonElement.jQuery.event.dispatch (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/jquery/dist/jquery.js:4435:9)
  at HTMLButtonElement.elemData.handle (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/jquery/dist/jquery.js:4121:28)

Commands
     -8:12.7.0 settings-view:open (ul.list-inline.status-container)
  2x -8:04.9.0 core:backspace (atom-text-editor.editor.mini.is-focused)
  3x -8:03.4.0 core:confirm (atom-text-editor.editor.mini.is-focused)

Config
{
  "core": {
    "openEmptyEditorOnStart": false
  }
}

Installed Packages
# User
atom-html-preview, v0.1.11
auto-indent, v0.5.0
ember-cli-helper, v0.9.0
git-plus, v5.4.7
remember-session, v0.5.1
save-session, v0.15.7
terminal-plus, v0.6.5

# Dev
No dev packages

The c/t button toggles between the component and the template. From your description it seems you had no file open when pressing it. If so, then I have fixed the exception in v0.9.5.


Why retval of exit clone syscall tracepoint is always returning the same value?

The exit clone syscall tracepoint retval is always returning the same value -1994375301 that appears to be a pointer. Is there any easy way to get this value or will be necessary to use the bcc?
Code:
tracepoint:syscalls:sys_exit_clone / @start_clone[tid] /
{
  $end_clone = nsecs;
  printf("[%d, %d, %d, %lld, %lld] clone\n", retval, pid, tid, @start_clone[tid], $end_clone);
  delete(@start_clone[tid]);
}

Output:
[20487, 20487,<PHONE_NUMBER>045,<PHONE_NUMBER>465] execve: /sbin/tc
[-1994375301, 17990, 17990,<PHONE_NUMBER>825,<PHONE_NUMBER>729] clone
[-1994375301, 20488, 20488,<PHONE_NUMBER>850,<PHONE_NUMBER>018] clone
[20489, 20489,<PHONE_NUMBER>255,<PHONE_NUMBER>340] execve: /sbin/tc
[-1994375301, 3600, 20485,<PHONE_NUMBER>380,<PHONE_NUMBER>575] clone
[-1994375301, 14895, 15197,<PHONE_NUMBER>259,<PHONE_NUMBER>402] clone
[-1994375301, 3408, 20134,<PHONE_NUMBER>335,<PHONE_NUMBER>650] clone
[-1994375301, 14871, 20047,<PHONE_NUMBER>370,<PHONE_NUMBER>988] clone

I think this "-1994375301" value is garbage...
If I am not mistaken, what you want here is args->ret instead of retval.
AFAIK retval is for kretprobes/uretprobes.

Hello @caringi, thank you for helping me! In fact, I am using BCC now. Should not this detail be in the reference guide or somewhere in the README?

@brendangregg does make sense to use the retval builtin with tracepoints or should this throw an error msg?

I think it should throw an error message that says something like "retval is invalid in tracepoints; some provide an args->ret instead"

I think this issue can be closed now, right?


coreos-postinst: install link file for virtio_net naming compatibility

since systemd v226, virtio_net nics will use predictable device naming.
this could break existing network configurations, so for upgrades from
old versions of coreos, we install a networkd .link file to keep old nic
names the same.

LGTM


Playground: Add chromium support

Resolves GH-45. This PR adds support for the playground in Chromium browsers. This is done by instantiating the WASM module that contains the compiler inside of a web worker instead of the main thread.

Nice! Good job on the fix! However, I think we should change to import the script using ECMAScript module syntax since it looks cleaner, doesn't pollute the scope with stuff we don't know about, and have cleaner syntax overall when calling the functions. Firefox will support it in version 111, which is released March 14th. This won't require changes to wasm-pack and so on, and is generally the preferred way to do it!

Also, this does not work...I think you failed to commit the --no-modules hack to get around ES module syntax. Doesn't run on FF110.0.1 on Windows 21H2

Yeah, noticed that too, forced push a change a few seconds ago.

Nice! Good job on the fix! However, I think we should change to import the script using ECMAScript module syntax since it looks cleaner, doesn't pollute the scope with stuff we don't know about, and have cleaner syntax overall when calling the functions. Firefox will support it in version 111, which is released March 14th. This won't require changes to wasm-pack and so on, and is generally the preferred way to do it!

Opened #152!

I would love it if you made it render on load again as it did before, as it is a nice welcoming screen when the example document gets loaded

Hmm, I thought I fixed that again. Will take a look!

I still get the same problem sometimes, I think we still have a race condition between the document fetch and the parser/core loading, possibly... It would be nice to get this resolved before pushing something that works sometimes. (FF110.0.1 on Windows 21H2)

We might be able to solve this by fetching/displaying the welcome document strictly after we have loaded the core wasm, by sending an init event to the main (sync) JS. Or let the worker fetch the document itself

It seems like your last commit solved the problem. I think we are ready to merge it, but we should PR #152 asap to merge it on tuesday when the Firefox update goes stable and we can verify that ES modules work there as well! (Approved)


Added missing bin/vendors installer for composer

Installer from http://knplabs.com/blog/symfony2-with-composer

There is no need for this file at all anymore. Vendors are managed by composer directly. Why should we add another file that calls composer (supporting 2 of the composer commands but none of their options, so making it less powerful) ? If you want to keep bin/vendors, simply rename your composer.phar file as bin/vendors :)

Was more of a helper to download composer.phar really.


Fix collecting events in cairo0

Which components does the task require to be changed? (think hard pls)
snforge
Description
In forking mode, we don't catch the events from cairo0 contracts - needs to be fixed

We should copy signal logic for Cairo0 ext


Resized Display P3 image includes Display P3 color profile but it is missing color information

I'm working on adding wide color support to my application.
I've got my pipeline from device -> server -> Amazon S3 working and I can successfully upload images with the Display P3 color profile from device to S3 without a loss of color information, retaining the color profile.
Once the images are stored in S3, I have Sharp running in a Lambda function for Cloudfront origin requests to the S3 bucket. I have verified that a vanilla Cloudfront distribution is able to serve the wide color images correctly, but when they pass through Sharp resizing they lose color information.
I'm on the latest version, v0.29, and I'm using the attached canary image to test (it is an image that shows up in Display P3 but disappears in sRGB).

Original Canary
Broken by Sharp

If you download and inspect the images, you can see that the resulting image has the Display P3 profile but it has lost the color information.
I'm using the pipelineColorspace("rgb16") operator, and specifying the output icc profile. Is there something else I'm doing wrong here? Thanks!
Relevant code:
  S3.getObject({ Bucket: bucket, Key: originalKey })
    .promise()
    .then((data) =>
      Sharp(data.Body)
        .pipelineColorspace("rgb16")
        .resize(width, height)
        .withMetadata({ icc: iccProfilePath })
        .toColorspace("srgb") // I have tried with and without this line to the same result.
        .toFormat(format)
        .toBuffer()
    )
    .then((buffer) =>
      S3.putObject({
        Body: buffer,
        Bucket: bucket,
        ContentType: contentType,
        Key: newKey,
      }).promise()
    )
    .then(function () {
      console.log("Redirecting to resized at: " + `/${newKey}`);
      const response = {
        status: "301",
        statusDescription: "Moved Permanently",
        headers: {
          location: [
            {
              key: "Location",
              value: `/${newKey}`,
            },
          ],
        },
      };
      callback(null, response);
    })
    .catch((err) => callback(err));

Thanks, and sorry you were correct, the P3 profile is present in the GitHub image (I was looking at a previously downloaded image from a different issue with a very similar filename, d'oh).
I can reproduce the problem locally, and will need to spend a little time to debug exactly what's happening.
In terms of the built-in "p3" profile in libvips, this is DisplayP3, which is hopefully the more useful/common flavour.

Sounds good, thanks for taking a look!
I know libvips is capable of applying these profiles correctly because, on upload, I use libvips from Go to apply the Display P3 and sRGB profiles to the uploaded images before storing them in S3.
I'm using the vips_icc_transform API to go from their embedded icc profiles to one of the profiles I supply. (I normalize wide-color images from Adobe RGB, ProPhoto etc to Display P3, and RGB images to sRGB IEC61966-2.1 to ensure maximum compatibility.)

Commit https://github.com/lovell/sharp/commit/104464c2e00cf362992fa6170d4525ebcd1dcabe fixes this and adds a test that would previously have failed. This will be in v0.29.1 - thanks for reporting!

Awesome! Thanks so much! I'll test out the fix when I have some time.

Hi @lovell - I finally had a chance to pull this change and try it out.
Unfortunately, now I'm getting a different, weird error for both sRGB and P3 images:
"errorType": "Error",
    "errorMessage": "A number was expected",
    "stack": [
        "Error: A number was expected",
        "    at /var/task/node_modules/sharp/lib/output.js:1008:15",
        "    at new Promise (<anonymous>)",
        "    at Sharp._pipeline (/var/task/node_modules/sharp/lib/output.js:1007:14)",
        "    at Sharp.toBuffer (/var/task/node_modules/sharp/lib/output.js:136:15)",
        "    at /var/task/index.js:238:10",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)"
    ]

It's possible this is due to how I installed it - I ran:
npm install https://github.com/lovell/sharp#e044788f63b6cd1147717b181369d596b606942d
because the v0.29.1 release has not been pushed to npm yet.
Is there another way I should be installing it? Or could you push like a pre-release version of it?
The relevant code from my lambda function is:
  // Get the original object, resize it, and save it back to s3:
  S3.getObject({ Bucket: bucket, Key: originalKey })
    .promise()
    .then((data) =>
      Sharp(data.Body)
        .pipelineColorspace("rgb16")
        .resize(width, height)
        .withMetadata({ icc: iccProfile })
        .toFormat(format)
        .toBuffer()
    )
    .then((buffer) =>
      S3.putObject({
        Body: buffer,
        Bucket: bucket,
        ContentType: contentType,
        Key: newKey,
      }).promise()
    )
    .then(function () {
      console.log("Redirecting to resized at: " + `/${newKey}`);
      const response = {
        status: "301",
        statusDescription: "Moved Permanently",
        headers: {
          location: [
            {
              key: "Location",
              value: `/${newKey}`,
            },
          ],
        },
      };
      callback(null, response);
    })
    .catch((err) => callback(err));

Where iccProfile is "p3".

Please can you try adding the --build-from-source flag:
npm install --build-from-source https://github.com/lovell/sharp#e044788f63b6cd1147717b181369d596b606942d

Awesome! Everything seems to be working now. Thanks for all your help!

Thanks for checking, I'll re-open this until v0.29.1 is released with the fix.

v0.29.1 is now available - thank you for reporting this.

Just curious, is it must to specify pipelineColorspace here or sharp automatically picks the correct from from original image?
If sharp is not doing it, is it okay to just read metadata and pass .pipelineColorspace(metadata.space) for all images to make sure correct one is used?


Communication Plan

Meet every evening at 8 PM to collaborate and work on the group tasks.

all agreed to the communication plan


Object is possibly 'undefined' when accessing arrays via at

🔎 Search Terms
"Object is possibly 'undefined' array at"
🕗 Version & Regression Information
Tested with v5.3.3 in the Playground, with the target set to ES2022.
⏯ Playground Link
https://www.typescriptlang.org/play?target=9#code/MYewdgzgLgBADiAlmKEYF4YG0BQMYDee+MAZiCAFwwAMANMQL4MC6A3DjqJCADYCmAOl4gA5gAoEyVFhotB5EAEoO3CHyEiJUlBCw7Uw-mFFQAFgFoAjPMUqu4dQOFjJSXYICGUcTSUKKFXwAemCYZFAAJ0j+YFgAd09IsGRRagB5ACMAK1jYRDQECAhETN4ATxgAcgBXMAATflJkfnqqhx5nLTdpCC8fa387NhCwiJBovJhE5NSMnKmC+BBi0orqusbmsFb2tQ0XbXcZKwAmAGYAFltAkZhQmEzK828YLURgOmmzSrAQWFE-Fg5n4MAgngAtqCZikTDAzPwYgB+HBAA
💻 Code
const points = [
  {
    foo: 'bar',
  },
];

console.log(points[0].foo);  // ok, no warning
console.log(points[points.length-1].foo);  // ok, no warning
console.log(points.at(0).foo);  // incorrect warning: Object is possibly 'undefined'
console.log(points.at(-1).foo);  // incorrect warning: Object is possibly 'undefined'
console.log(points[1234].foo);  // by that logic, why not get the same warning here?

🙁 Actual behavior
Incorrect warning: Object is possibly 'undefined' for array.at(...).property);
🙂 Expected behavior
No warning for accessing array elements via .at, for the same reason there's no warning when accessing via index notation.
Additional information about the issue
No response

All of those are errors under noUncheckedIndexedAccess.  .at() is just strict by default because there's no means to change the type of the function depending on compiler options.

Hello, I noticed that the type narrowing doesn't seem to work with comparisons that involve greater and smaller signs even when checking the type beforehand. Is it possible to do this without resorting to a custom type guard?
Listing:
const list = [1, 2, 3, 4, 5];

const islastItemPositive = typeof list.at(-1) === 'number' && list.at(-1) > 0;
const islastItemNegative = typeof list.at(-1) === 'number' && list.at(-1) < 0;
const islastItemZero = typeof list.at(-1) === 'number' && list.at(-1) === 0;
const islastItemNotZero = typeof list.at(-1) === 'number' && list.at(-1) !== 0;

The TS compiler complains that list.at(-1) > 0 and list.at(-1) < 0; are potentially undefined. it doesn't show that error message for the last two lines.

I recreated the example in TS playground: https://www.typescriptlang.org/play/?target=10&ts=5.5.4#code/MYewdgzgLgBANgS2jAvDA2gRgDQwEy4DMuALLgKwC6A3AFC2iSxJwCG0AklAKYC2ACiAgIoCAG7dUMKAE8ADtxAAzeEigA6VlAAUAWkwBKVCjQByMAFdeAI24AnUzABkT1dE079RgHwwADHSMyCzsUFx8AHLcAOZa4pJosgrKbhpaeobGZpY29o4uqR4ZRgA8-oHgwRBsnDy8AFr2IFJJiiqI7uleWTDmVrYOzq4daZ6ZJmgBDJXM1aHhvBEgUI12zYnybYVd4ya9OQP5w2pF3QCEewFAA
Cheers


Maximum recursion depth

New Feature Proposal
Description
Add possibility to restrict recursion depth to a maximum number.
Background
We use AngleSharp to parse E-Mails. Those can come from good people and from bad (Spam). In the latter case, the E-Mails often contain invalid tags and invalid HTML.
 The original email is about ten times longer.
This one leads to a StackOverflow in CollectionExtensions.GetAllElements<T>(this INode parent, Predicate<T> predicate) while accessing child nodes.
Of course I know that the maximum depth depends on several factors, e.g. from where the code is called and how much is on the stack.
Any other suggestions to solve this issue?
//Edit: maybe we are able to use a iterative solution instead of a recursive one. Are there any reasons for the current choice?
Specification

You can always transform a recursive algorithm into an iterative one, however, the readability usually suffers.
In the given case (not sure if I remember) there is no simple mapping, so you would need to keep track of the elements via a stack (what a call stack gives you out of the box). Naturally, you can go there, but this stack is also limited by memory (not as much as the call stack, but of course there are physical limits) and not as efficient / optimized. Long story short: I think you will exchange one problem with another (in this simple transformation).
TL;DR: If there is a good solution algorithm any contribution would be welcome. There are many spots - this is just one of them.

For anybody who is interested in fixing these, I have done this before in https://github.com/AngleSharp/AngleSharp/pull/524, https://github.com/AngleSharp/AngleSharp/pull/526 and https://github.com/AngleSharp/AngleSharp/pull/544
Here is the code @FlorianRappl was taking about: https://github.com/campersau/AngleSharp/blob/7ef760cc6c4eb4f19e306038cbde4c0b64636a43/src/AngleSharp/Dom/NodeExtensions.cs#L113-L128

Currently I do not know what @FlorianRappl means by "simple mapping" in this case.
I will have to investigate the source code and the commits from @campersau in the next few days to fully understand the problem here.
But if it holds that an iterative solution is creating another problem, maybe we can inject a predicate through configuration?

The best solution would be an algorithm like this https://github.com/AngleSharp/AngleSharp/pull/524#issuecomment-276221408 which doesn't use a stack at all but it is very slow for Anglesharp because of the current NextSibling implementation.

@campersau I am not sure If I am missing the point. But why is the NextSibling slow? If I have calculated correctly, this should be an O(n/2) algorithm which is not very bad.

I can remember that I had changed one of the performance projects to do a small perf tests on both of theses implementations and the implementation with NextSibling was a lot slower. Sadly I don't have the exact numbers anymore.
The problem here is that it is a loop inside of another loop which makes it so slow. E.g. if you have a table with a lot of rows than for each row you would need to find the next sibling which in turn loops through the rows of the table each time.

@campersau @FlorianRappl what do you think about this static update of NextSiblings?
https://github.com/gabbersepp/AngleSharp/commit/5dce81a24ce71afb3eabd072f5409806847e9aba
Tests are green but I have no deep knowledge about this project so maybe you easily can tell me whether or not I should make a pull request from this commit.

@gabbersepp interesting idea, but I think this will invite some bugs and oddities + it is now inconsistent with PreviousSibling and the respective Element counterparts. However, worst part is additional memory consumption, which is now +1 field, but with all the described support would be +4 (or even more) fields per node.
Hence I am not sure if we should pursue this path.

@FlorianRappl It is a pointer to a Node which should take 4 Bytes on a 64Bit machine.
If we use a index based solution we can reduce it to 2 bytes per field. One byte would allow too little siblings.
Previous-/NextSiblings should be able to be combined into one index field if every Node hold it's current index in the list of parent's siblings.
You're right, there may inconsistencys right now. I have not search through the code for similar usages. :-)

It is a pointer to a Node which should take 4 Bytes on a 64Bit machine.

Do you mean 8 bytes (4 bytes on a 32-bit machine)?
If we have larger trees (and I guess this is exactly your target) then we talk about 1k - 10k nodes minimum, which means 4 (Prev, Next for Node and Element) * 8 * 1000 = 32k memory added for a document with 1k nodes (plus potential leaks as the GC suffers from the direct references, which means one needs to be super careful not only for obvious bugs, but also for non-obvious ones).
Don't get me wrong here - all suggestions are viable. I just want to point out that any change on Node level directly influences memory (as Node is the bread and butter in AngleSharp). So even a simple change like adding 1 field can have a huge impact.
Experience in AngleSharp shows that freeing nodes with recursive dependencies is also handled worse by the GC than it should be.
Alternative could be to have the child nodes implemented as a linked list. It is a lower level construct and therefore consistent by design. However, while the linked list would improve the speed on the traversal there, it will have other drawbacks.

Ah yes you're right. 8 bytes per reference.
Ok. I understand the problem.
I have thought a bit about this problem without success.
One optimization that can be done is to replace the "name" field with an index reference to a global list of tag names. 16 bit should be enhough and thus save 3/4 of the current field's size.
Searching for the correct location within all tag names should not take more time than now I think as .NET will do something similar when doing string interning.
This can compensate one of the mentioned fields :-D

Wanted to note that until a good solution is found, I am using this code to check the depth:
    ```

public static IHtmlDocument SaveParse(this HtmlParser parser, string content)
{
// ReSharper disable once AngleSharpStackOverflowCheck
var parsed = parser.Parse(content);
        foreach (var e in parsed.All)
        {
            INode node = e;
            int counter = 0;

            while ((node = node.Parent) != null)
            {
                counter++;

                if (counter > 500)
                {
                    throw new Exception("tree max depth reached. Skip to avoid SO");
                }
            }
        }

        return parsed;
    }

Runtime should be round about O(nlog(n)) for balanced html documents and n^2 for those "newsletters of death". I will make a performance test with a bunch of existing emails in our DB and update this comment with the results.

Performance statistics:

email count
error
success
ms
ticks
length of all emails
size / email
time / email
error / day

800142
158
799984
549297
1454777529
6160514531
7699
0.68ms
0.114

I'll try to come up with a solution in unlucky number 13.

I am currently looking into the problem and I have problems seeing where the issue is ...
The GetAllNodes extension method (I think this is the one you refer to) seems not to be recursive. Instead, it uses NodeEnumerator which has a special MoveNext implementation. MoveNext is called when iterating over foreach loops.
Can you provide such an email of death with a small code that you user to trigger the SO exception? Thanks!

@FlorianRappl Hey I will send you an example next week.

@FlorianRappl please see this gist:
https://gist.github.com/gabbersepp/033b5a1fa93f055a8b1d816db5cdcaf3
I have removed all sensible information.

Great, thanks!

Hm I got a green test for it ...
[Test]
public void IteratingThroughDomShouldNeverFail_Issue763()
{
  var content = Assets.GetManifestResourceString("Html.BrokenMail.txt");
  var p = new HtmlParser().ParseDocument(content);
  Assert.IsNotNull(p);
  var img = p.Images.Select(m => m).ToList();
  Assert.AreEqual(1, img.Count);
}

Anything I'm missing @gabbersepp ?

@FlorianRappl
Please try to use the max stack size as shown in the gist. It is a bit weird. If I use this email in out ASP.NET WebAPI projekt, a stackoverflow is thrown. If I copy the same code into a console app, nothing happens. To force the SO, I have to reduce the stack size.

Apparently, the stack trace has not the "big" impact (even though a larger stack potentially solves it, too) - its the var html = p.ToHtml(); which makes it crash.

I flattened the ToHtml implementation. Landed in devel.


Because of numerical precision, knots produced for different cutoffs with same knot spacing are not equal

Consider the following example-

Depending on the cutoff value when rmin!=0, the knots produced for the three bodies are slightly different (3.6 vs 3.6000000000000005) even though the knot_spacing (controlled by the resolution) is the same between the two bspline_config.
The implication of the above issue is that the templates (see here) produced by UF3 is inconsistent across cutoffs with constant knot_spacing.
In the above example, the template of bspline_config_smaller_cutoff (bspline_config_smaller_cutoff.templates[('W','W','W')]) should be a subset of  bspline_config_larger_cutoff. As the knots are at slightly different positions (3.6 vs 3.6000000000000005), this is not the case.
This is because while setting the template the knot values are used (see here).
Having inconsistent templates prevents the implementation of the drop_columns feature described in this pull request #86 .

This issue should not cause any problems in fitting the models if the features for the two cutoffs are generated independently.


[MOB-247] Discount items are not showing up as a line item

MOB-106 Line items are not being applied to transactions
What is this?
Non-discount items are showing up as a line item but the discounts are not.
What did I do?
Added values for determining if an item is a discount to CatalogItem

Looks good boss! 🔥


feat: "Using an SVG instead of png icons"

What kind of change does this PR introduce?

[x] Feature

Does this PR introduce a breaking change?

[ ] No

Information
I have made changes that enable Tauri to use SVG icons instead of PNG icons. This enhancement allows for better scalability and resolution independence, making Tauri more versatile when it comes to icon handling. SVG icons can be scaled without loss of quality, which is especially valuable for applications running on various devices and display resolutions.

Looks like a breaking change in the CLI helper tool. So that needs a closer look.
I'd like a bit more motivation as well. The positive of a single SVG is also the negative of a single SVG. You give up control on how it scales and renders. Going in manually to fine tune how your icon looks at 32x32 is sometimes needed to get it to look crisp.
But maybe my views on that are outdated, if you have any good resource to say SVGs are now objectively better on the big 3 desktop platforms I think that'd help motivate using this as the default option.

@Beanow So the idea was to intially only have support in the tauri icon: https://github.com/tauri-apps/tauri/issues/6389 - it currently works by re-scaling a single png into multiple pngs so svg input would be a win here (i'd still love to implement support for multiple input icons). I think only flatpaks actually support svgs rn.
@madabhi It looks like you aimed to make tauri itself work with svgs at runtime? The main problem with that is missing native support for svgs on all platforms, so we'd either need to change tauri itself to convert svgs to something the os understand at runtime, or go with what i wrote above and keep using mostly pngs.
In that context, every single change in this PR is broken.

The tauri.conf.json changes should break at the bundling point unless we have a fallback i forgot about, but should cause runtime problems later either way.
The json.icns change simply breaks the icon.icns generator for the macos icon, or maybe it has 0 effect since the file names don't actually effect the file type (still png).
The change in icon.rs is not broken by itself but doesn't solve anything either.

Sooooo, where do we go from here? I think adding runtime support for svgs in tauri is a bit too ambitious and maybe even inefficient, so imo we should stick with the tauri icon approach which was also worked on here: https://github.com/tauri-apps/tauri/pull/6444. That PR was already pretty close to what i imagined but it's still missing some work on the feedback provided until it could be merged, maybe you'd like to pick that up instead seeing that the author doesn't work on it anymore?


Create my_github_link.txt

Description

[x] I have added a folder(Named it as my GitHub user id)
[x] I have added a Readme.md file inside the folder
[x] I understand the GitHub Forking Concept fully

I have already done the necessary tasks, why is the PR marked stale?


Add group id into metadata for java use

Java daily updates needs group id to download the required package. The group id is missing from azure-sdk-tool script Update-DocMsPackage.ps1. The PR is to add it back.
Test update in daily branch:
https://github.com/Azure/azure-docs-sdk-java/blob/daily/2021-10-14/package.json#L1762

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow


chore: update size benchmark

Reproduce with bun run size on Viem.

PR-Codex overview
This PR optimizes the size of the viem package in the benchmarks folder by reducing its gzipped size from 35 KB to 3.78 KB. The createPublicClient function was replaced with createClient.
Detailed summary

Reduced viem package size from 35 KB to 3.78 KB
Replaced createPublicClient with createClient in benchmarks README.md

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@jxom is this using size-limit? I believe the numbers we have came from pkg-size.dev (uses esbuild) and I believe size-limit by default uses webpack?
means I should re-check ourselves using size-limit too
Regardless thanks for the PR - merging!

Think esbuild might include some weird side-effects. Not sure if that might improve your benchmark?

@jxom sg, I've separately also updated the thirdweb size (which was also smaller using size-limit) - we want to be accurate with this so I think now the methodologies are the same at least :)


Investigate possible issue with connection string or missing additional steps in .NET Core new db tutorial with SQLite

I got a report saying that if you follow this instructions migrations commands will create a database on the root folder but then execution will try to look for the database file in bin\debug, which will make the application crash.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: adb04585-f54b-70c0-e847-b044b388a4d5
Version Independent ID: 5e37f568-0621-a082-c75a-d2adf94dce8d
Content: Getting Started on .NET Core - New database - EF Core
Content Source: entity-framework/core/get-started/netcore/new-db-sqlite.md
Service: unspecified
Product: entity-framework
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

We've tested it and apparently is related to how you execute the project:

Executing the project from a powershell console using dotnet run works great.
Executing the project from visual studio doesn't work. Throwing a Microsoft.Data.Sqlite.SqliteException with the error message  SQLite Error 1: 'no such table: Blogs'.
Similarly, entering the /bin/debug/netcoreapp2.1 and running dotnet TestCoreApp.dll gives the same error.

It seems to be an inconsistency in the way dotnet and visual studio handle the local working directory. So... Changing it from the Project Settings > Debug > Working Directory and setting the path to the root of the project, makes it work again.
Should the tutorial be updated?

@Starkie well, we could add a note to the walkthrough saying that if you choose to use VS, you can set the working directory manually to be the root of the project, but the truth is that the whole walkthrough currently only covers the instructions for building the app using the command-line tools, so not sure how much value there is in mentioning VS just for this.
Anyway, I agree there is an inconsistency in the working directory between what you get using dotnet command-line and VS, and I am trying to figure out the right place to report it (I doubt it is by-design).
@DamianEdwards does this ring any bells? Any suggestion on where to report it?

cc @BillHiebert as he changed this on https://github.com/dotnet/project-system/pull/3073.

@Pilchie  - I made the change, but I wasn't the driver for it. If I recall, it was to be consistent with the full framework console project.

Thanks @BillHiebert, @Pilchie, @davkean: I went ahead and created https://github.com/dotnet/project-system/issues/3619 on your side.
I will keep this documentation issue open to decide how/if to update the docs after the outcome of https://github.com/dotnet/project-system/issues/3619 is known.
Note to self: the article contains a note that talks about paths in the connection string being relative to the output path, which supposedly became obsolete in 2.0, but may have become relevant again.


entry numbers added to some normative reference entries

@opoudjis i am not sure if this is my fault or a software issue. i am not sure if i should list it here (template issue) or under the relaton repo. any advice on a workaround would be greatly appreciated. thank you.
i shifted two references that were previously in my bibliography (rendering well) to the normative references clause of my document. the references were "commented out" in the bibliography so as to not be duplicated.
the generate assigned entry numbers (both [1]) to each reference in the normative reference clause; the PDF render is shown below:

the current code for my normative reference clause is shown here:
[bibliography]
== Normative references

* [[[IEC_SRD_63188_2022,IEC SRD 63188]]]
* [[[IEEE_2413_2019,IEEE Std 2413]]]
* [[[ISO_19125-1_2004,ISO 19125-1]]]
* [[[ISO_19170-1_2021,ISO 19170-1]]]
* [[[ISO_IEC_20802_1_2016,ISO/IEC 20802-1]]]
* [[[ISO_IEC_20802_2_2016,ISO/IEC 20802-2]]]
* [[[ISO_IEC_21778_2017,ISO/IEC 21778]]]
* [[[quic_rfc,IETF RFC 9000]]]
* [[[http2_rfc,IETF RFC 9113]]]
* [[[mqtt_v5,OASIS mqtt-v5.0]]]
* [[[ogc_geopose,OGC 21-056r11]]], OGC GeoPose 1.0 Data Exchange Standard
* [[[ogc_sensorweb,OGC 07-165r1]]], OGC Sensor Web Enablement: Overview And High Level Architecture
* [[[OGC_Simple_feature_1,OGC 06-103r4]]], OGC Simple Feature Access – Part 1: Common Architecture
* [[[w3c_did,W3C did-core]]], Decentralized Identifiers (DIDs) v1.0
* [[[w3c_did_spec_registries,W3C did-spec-registries]]]
* [[[w3c_json_ld,W3C json-ld11]]], A JSON-based Serialization for Linked Data
* [[[w3c_wot,W3C wot-architecture]]], Web of Things (WoT) Architecture

* [[[h3geo,1]]]
span:organization[Uber Technologies, Inc.]
span:title[H3: Hexagonal hierarchical geospatial indexing system]
span:type[website]
span:date[2024]
Available at: span:uri[https://h3geo.org]

* [[[graphql,1]]],
span:organization[Joint Development Foundation Projects, LLC].
span:type[webresource]
span:title[The GraphQL Specification Project]
span:date[October 2021],
Available at: span:uri[https://spec.graphql.org]

the metanorma version is:
Run metanorma --version
Metanorma 1.7.7
Metanorma::Cli 1.9.8
Metanorma::Standoc 2.8.9/IsoDoc 2.10.2
Metanorma::ISO 2.7.8
Metanorma::Iec 2.4.7
Metanorma::IEEE 1.2.8
Metanorma::Ietf 3.3.6
Metanorma::Generic 2.6.3
Metanorma::BIPM 2.4.7
Metanorma::CC 2.4.7
Metanorma::Csa 2.4.7
Metanorma::IHO 0.9.7
Metanorma::JIS 0.2.8
Metanorma::UN 0.12.9
Metanorma::Ogc 2.5.8
Metanorma::ITU 2.4.7

Thanks for reporting @ReesePlews !

no problem @ronaldtse; just wondering if that is something i am doing wrong.  i had talked with Nick about values for specific spans and these were from the bib (which was in my feeling, rendering ok) even if those span values could be modified a bit.

SDOs, and therefore SDO stylesheets, and therefore Metanorma, really don't expect to see documents in Normative References that are not standards, and that do not have an explicit document identifier. Giving normative effect to a website is not something typically foreseen. IEEE is exceptional in this regard, and even there, I'm not blown away by the level of detail of guidance in the IEEE style guide; it is clearly treating non-standards normative references as an afterthought. It refers out to the Chicago Manual of Style, but does that mean footnote (as the web example suggests), or author-date?
We should deal with cases like this better than we are, but this will have been the first Metanorma flavour to which we introduce either author-date or footnote citation. We have some scaffolding supporting that in in relaton-render, but no flavour has actually required them hitherto; https://github.com/metanorma/metanorma-standoc/issues/168 has been unactioned for 5 years.
It's clear what's happening here: there is no docidentifier, the IEEE stylesheet expects a docidentifier (what IEEE calls a designator, because we currently only cater for standards in Normative References), it's not finding one, it's assigning [1] in both cases as the metanorma-ordinal default (which would kick in in a Bibliography), and the result is hilarity.
But if we follow the IEEE style guide, websites such as these do not get Normative References entries at all. They get footnotes. 2021 IEEE SA Standards Style Manual, §12.3.6.
This is a lot of work to realise non-standards normative references the way IEEE expects, and I have a lot of pressing work that isn't getting done (not least of which is Plateau), and an ongoing complete vacuum of task priority-setting. If you want this, @ReesePlews, you're going to have to give me a timeframe, and @ronaldtse you're going to have to prioritise tickets, giving the ongoing gaping absence of additional Metanorma resourcing.
And I'm going to need someone in IEEE  Editorial to confirm that websites really do get bibliographic entries in the normative references, and (presumably) citations generated as footnotes out of cross-references—as opposed to just on-the-stop footnotes, which you can do right now. At minimum, a published IEEE standard displaying that kind of bibliographic reference to a website in its Normative References, and how it is cited; the Style Guide's vague mention of CMS is insufficient for this. We cannot undertake days of work on something that IEEE Editorial could reject as non-conformant.

thank you @opoudjis ; thanks for the feedback.  you mention

And I'm going to need someone in IEEE Editorial to confirm that websites really do get bibliographic entries in the normative references, and (presumably) citations generated as footnotes out of cross-references—as opposed to just on-the-stop footnotes, which you can do right now. At minimum, a published IEEE standard displaying that kind of bibliographic reference to a website in its Normative References, and how it is cited; the Style Guide's vague mention of CMS is insufficient for this. We cannot undertake days of work on something that IEEE Editorial could reject as non-conformant.

it was IEEE-SA during the editorial check that IEEE-SA told use if we used a document normatively (in case of a requirement) then we needed to add them to clause 2. it also seemed strange to me. ISO tends to make authors remove normative refs unless they are truly required and indispensable documents.
maybe my interpretation of what IEEE-SA was expecting "normative refs" are is beyond what IEEE-SA is thinking.
as for the current document, we can edit the generated word file, (or even possibly the pdf) to remove them manually, not an issue.
before getting started on this, lets check with @ronaldtse he has been in touch with IEEE-SA for another issue, so based on your reply, i now think it would be good to hear what IEEE-SA say/advise before any work/modifications happen.
thanks again for the feedback. i will let you and Ron coordinate if this should remain open or become a new issue for future work.

Most expedient thing to do here is to simply delete the ordinal identifiers from Normative References, no matter if Metanorma has inserted them, to ensure that they are never used.

Cross-references to these are currently defaulting to the anchor in the absence of an identifier. The IEEE Style Manual quite simply does not define what the designation of a non-standard normative reference is; the vague referencing of the Chicago Manual of Style implies Author-Date, but the treatment of internet sources implies footnotes. I'm not going to deal with this until it comes up, because in this instance, the source should be left as a footnote anyway.
The entries are currently being processed with a (NO ID) identifier in the normative references if I suppress it; I need to suppress (NO ID) in this flavour.

Internationalise "(NO ID)", which will allow it to be configured; it should be possible to set it to nil.

We need to change deep_merge so that nil can replace a value.

@opoudjis thank you for checking and addressing this issue. the document is out of ballot and will be revised in the coming days. i will let you know if there are any issues. thank you for working on this.

No problem, @ReesePlews . I didn't want to work on it because of the somewhat opaque instructions around how to reference websites, but Ronald had convinced me that it still needed to be addressed.
We've just got confirmation from IEEE that they expect Author-Date referencing for non-standards/websites, which will be a new ticket.

thanks @opoudjis i will generate the document tomorrow, unless it will take longer for the docker instances to be updated.


Add 32-Bit support

Add 32-Bit Version of 7-Zip 19.00.
Microsoft Reviewers: Open in CodeFlow

I guess this will have to wait then until multi-arch manifests are supported.


fix: playwright-tests failing

@seveibar sorry for miscommunication as i am currently working on it, and it now gets failed again i will get back to you when it passes.

@Anshgrover23 are you using the "playwright dev" command to fix these?

@seveibar no i am using bun run playwright command to check for these tests?

Use this
https://playwright.dev/docs/test-ui-mode


[Transaction] Move MetaStoreHandlerNotExistsException

Master https://github.com/apache/pulsar/issues/13918
Motivation & Modification
Makes exceptions on the Transaction client side clearer.
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API: (yes / no)
The schema: (yes / no / don't know)
The default values of configurations: (yes / no)
The wire protocol: (yes / no)
The rest endpoints: (yes / no)
The admin cli options: (yes / no)
Anything that affects deployment: (yes / no / don't know)

Documentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?

[ ] doc-required
(If you need help on updating docs, create a doc issue)

[x] no-need-doc
(Please explain why)

[ ] doc
(If this PR contains doc changes)

Closed as stale and conflict. Please rebase and resubmit the patch if it's still relevant.


texworks: fix build

Motivation for this change
closes #82844
gui now opens fine
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

[1 built, 0.0 MiB DL]
https://github.com/NixOS/nixpkgs/pull/82873
1 package built:
texworks

fixes broken application

Backport to 20.03 for ZHF?

oh, that would probably be a good idea


Export only applications in the current recruitment cycle for UCAS

Context
During the October 2020 Data Matching DSA Review we spoke with UCAS about two ways handling data in different recruitment cycles.
The first option was to only send through candidates from the 2021 cycle going forward.
The second was adding a 'recruitment cycle year' field in the data we send over and receive, which would require an amendment to the DSA. As discussed, both parties favoured the former.
Changes proposed in this pull request
Change UCAS export to only send candidate data from
the current recruitment cycle.
Guidance to review
Makes sense?
Link to Trello card
https://trello.com/c/E1Cjp1iK/2965-export-only-applications-in-the-current-recruitment-cycle
Things to check

[ ] This code does not rely on migrations in the same Pull Request
[ ] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[ ] API release notes have been updated if necessary
[ ] New environment variables have been added to the Azure config

Hopefully not 🤞 The applications deadline (end of cycle) is usually a few days before the start of new recruitment cycle.
Also UCAS said take the approach that they only supply applicants in the current cycle, and they keep the data from the two cycles completely separate. That's why the first few datasets we got in the new cycle were empty because of that.


Fuction Excel.decodeBytes(bytes) crashes my main thread

Hi, i tried to use the function Exce.decodeBytes like the example but to decode excel file interrump the progress indicator in the main screen. I tried using await but was the same issue.  I used the version 2.0.0-null-safety-3.
Future decodifica( Uint8List bytes)async {
return await Excel.decodeBytes(bytes);
}
Future  excelRead(File file) async{
var bytes = await file.readAsBytesSync();
var excel = await decodifica(bytes);
...
}
Thank you for your helping.

Please post the error log ?

There arent error log, only i cant see the progress indicator while the dependence is working, i suposed the issue its cause my Excel file has 10500 rows and 6 colummns. What do you think?

I can't help you without looking at the logs, because the logs will tell that which part of code is causing issue, and more deeper insights

maybe I have the same issue, I'm trying to use the same version and I get a error when decoding, this is my trace:
I/flutter ( 5354): Null check operator used on a null value
I/flutter ( 5354): #0      Parser._parseTable (package:excel/src/parser/parse.dart:422:9)
I/flutter ( 5354): #1      Parser._parseContent.<anonymous closure> (package:excel/src/parser/parse.dart:163:9)
I/flutter ( 5354): #2      Iterable.forEach (dart:core/iterable.dart:257:30)
I/flutter ( 5354): #3      Parser._parseContent (package:excel/src/parser/parse.dart:161:39)
I/flutter ( 5354): #4      Parser._startParsing (package:excel/src/parser/parse.dart:18:5)
I/flutter ( 5354): #5      new Excel._ (package:excel/src/excel.dart:74:12)
I/flutter ( 5354): #6      _newExcel (package:excel/src/excel.dart:17:20)
I/flutter ( 5354): #7      new Excel.decodeBytes (package:excel/src/excel.dart:84:12)
I/flutter ( 5354): #8      decode (package:stock_man/decoder/cei.dart:9:26)
I/flutter ( 5354): #9      OrderService.decodeFile (package:stock_man/service/order_service.dart:17:25)
I/flutter ( 5354): #10     OrderImport.build.<anonymous closure>.<anonymous closure> (package:stock_man/views/order_import.dart:25:28)
I/flutter ( 5354): <asynchronous suspension>

I tried debugging the code to see if I can help. I found out that the line 420 is duplicating the xl/ expression. In debugger I could see the $target variable has /xl/worksheets/sheet1.xml as value, which is a existing file, but after the interpolation, findFile method is called with xl//xl/worksheets/sheet1.xml

I also face the same issue
Excel.decodeBytes(bytes) is blocking the ui thread so, The progress indicator is freezed until the method is finished it's work

This can be fixed by moving the Excel.decodeBytes(_bytes) call to another thread, flutter will handle this using isolates.
The easiest and quickest way to do this is to make a top level function (Outside of any classes) that calls this package:
Excel parseExcelFile(List<int> _bytes) {
  return Excel.decodeBytes(_bytes);
}

Then you can call this using the compute flutter method like this:
Excel excel = await compute(parseExcelFile, _bytes);

A full example:
Excel parseExcelFile(List<int> _bytes) {
  return Excel.decodeBytes(_bytes);
}

class DataImport extends StatefulWidget {
  @override
  _DataImportState createState() => _DataImportState();
}

class _DataImportState extends State<DataImport> {
  void importFile() async {
    File file;
    try {
      FilePickerResult result =
          await FilePicker.platform.pickFiles(type: FileType.any);
      if (result != null) file = File(result.files.single.path);
    } catch (e) {}

    if (file != null) {
      Excel excel = await compute(parseExcelFile, _bytes);
      Sheet _dataSheet = excel.tables['Entries'];
      print('Loaded xlsx file successfully with ${_dataSheet.maxRows} rows');
    }
  }

  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Excel file import text'),
        centerTitle: true,
      ),
      body: Center(
        child: RaisedButton(
          child: Text('Pick file'),
          onPressed: importFile,
        ),
      ),
    );
  }
}

@justkawal It may be worth including information about wrapping the plugin function calls in a compute for new users of the package.

@hcaz where is _bytes declared in example

@MalikSamiAwan sorry, I missed the file.readAsBytes() from my example. I have updated the full example above to include it.

@hcaz Thanks for the quick response and help
but i am still facing this issue, i have xlsx file of around 468KB with 13 sheets and alot of data in it
can you please help me `outimport 'dart:io';
import 'package:excel/excel.dart';
import 'package:file_picker/file_picker.dart';
import 'package:film_finder/custom_widgets/custom_scaffold.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import '../custom_widgets/custom_buttons.dart';
import '../helpers/common_helper.dart';
import '../helpers/styles.dart';
Excel parseExcelFile(List _bytes) {
return Excel.decodeBytes(_bytes);
}
class TestingCodes extends StatefulWidget {
const TestingCodes({Key? key}) : super(key: key);
@override
State createState() => _TestingCodesState();
}
class _TestingCodesState extends State {
@override
Widget build(BuildContext context) {
return CustomScaffold(
child: Container(
child: Column(
children: [
CButton(
color: greenButtonColor,
borderRadius: 30,
title: 'Upload Xlxs File',
onClick: () async {
String sheetToEdit = 'data';
            try {
              var f = await FilePicker.platform.pickFiles(
                  allowMultiple: false,
                  allowedExtensions: ['xlsx'],
                  withData: true,
                  withReadStream: true,
                  type: FileType.custom,
                  allowCompression: false);
              if (f != null && f.files.isNotEmpty) {
                File file = new File(f.files.single.path!);
                pprint(file.path, 1);
                if (file != null) {
                  List<int> _bytes = await file.readAsBytes();
                  Excel excel = await compute(parseExcelFile, _bytes);
                  // Sheet? _dataSheet = excel.tables['data'];
                  // print(
                  //     'Loaded xlsx file successfully with ${_dataSheet!.maxRows} rows');
                  showSimpleDialog(
                      context: context,
                      title: 'Successfully!\nOrginal File Parsed');
                }
              }
            } catch (e) {
              pprint(e.toString());
              showNotification(context, e.toString());
            }
          },
          loading: true,
        ),
      ],
    ),
  ),
  heading: Center(
    child: Text.rich(TextSpan(children: [
      TextSpan(
        text: 'Import Data',
        style: subTitleTextStyle,
      ),
    ])),
  ),
);

}
}
`

Excel excel = await compute(parseExcelFile, _bytes);
gives
Null check operator used on a null value
above error

@hcaz Thanks your solutions worked
Issue was that my xlsx file contains charts which were not getting computed so for i have delete that chart sheet
Let me know if there is any work around for that as well
Thanks again
Saved my day

Is there any way we can open / Excel.decodeBytes file that contains chart in it

I think it would be nice to post these questions on stackover flow as this is not related to an issue in excel.
If you're having another issue or the plugin throws error then feel free to re-open the issue again.

I also face the same issue Excel.decodeBytes(bytes) is blocking the ui thread so, The progress indicator is freezed until the method is finished it's work

How did this problem solved??

@Mahmoud3193 With isolates, you can move the number crunching to a different thread, keeping the UI thread smooth.
https://github.com/justkawal/excel/issues/96#issuecomment-903656480
https://api.flutter.dev/flutter/foundation/compute.html?ref=blog.flutter.wtf


Introduce IQueryPreProcessor support

As a developer I want to have the option to define a IQueryPreProcessor for my query.
This could be useful in the following sample:
Given is the following query:
public class GetSharedLinkPermissionsQuery : IQuery<GetSharedLinkPermissionsQueryResult>
  {
      public Guid? SharedLinkId { get; set; }

      public SharedLink? SharedLink { get; set; }

      public GetSharedLinkPermissionsQuery(Guid sharedLinkId)
      {
          SharedLinkId = sharedLinkId;
      }

      public GetSharedLinkPermissionsQuery(SharedLink sharedLink)
      {
          SharedLink = sharedLink;
      }
  }

For this I would like to add a PreProcessor, which requests the SharedLink entity from the database, if the constructor with the Guid parameter was used.
Using this I can ensure, that in the IQueryHandler implementation both properties are always set, so that I can focus on the Core logic.

@robinmanuelthiel pls give me feedback on this feature request.

We should come up with written down rules and best-practises on how we write and sctructure Queries (and Commands) across our projects!

Documented https://github.com/wemogy/libs-cqrs/pull/52


run problem

show error outof index :
line 41, in train
net = get_net()
line 21, in get_net
net = getattr(main, cf_net['call'])(**dict_drop(cf_net, 'call'))
line 928, in init
in_shape if i == 0 else 0, btneck_widths)

I don't know why you get that error, sorry :-/
I've just downloaded the repo afresh to test it, and it works smoothly for me.

This happens also for me when using torch 1.4
The problem is that the ModuleList() object self.blocks (https://github.com/mary-phuong/multiexit-distillation/blob/master/main.py#L932) cannot be assigned as a list but should probably have something like:
self.blocks.append(self.Block(...))
instead of:
self.blocks[i] = self.Block(...))
The setup mentioned by @mary-phuong in the REAMDE.md works fine (i.e., torch 0.4)

Can you say how did you install the older version? My system is not supporting it, and even in colab  I could not use it.


Add more unit tests

Codecov Report

Merging #11 (c340fc7) into master (bc98a48) will increase coverage by 32.28%.
The diff coverage is 91.37%.

@@              Coverage Diff              @@
##             master      #11       +/-   ##
=============================================
+ Coverage     35.71%   68.00%   +32.28%
- Complexity        2       10        +8
=============================================
  Files             1        2        +1
  Lines            42      100       +58
=============================================
+ Hits             15       68       +53
- Misses           27       32        +5

Impacted Files
Coverage Δ
Complexity Δ

tests/Unit/AuthTest.php
91.37% <91.37%> (ø)
8.00 <8.00> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc98a48...c340fc7. Read the comment docs.

Codecov Report

Merging #11 (c340fc7) into master (bc98a48) will increase coverage by 32.28%.
The diff coverage is 91.37%.

@@              Coverage Diff              @@
##             master      #11       +/-   ##
=============================================
+ Coverage     35.71%   68.00%   +32.28%
- Complexity        2       10        +8
=============================================
  Files             1        2        +1
  Lines            42      100       +58
=============================================
+ Hits             15       68       +53
- Misses           27       32        +5

Impacted Files
Coverage Δ
Complexity Δ

tests/Unit/AuthTest.php
91.37% <91.37%> (ø)
8.00 <8.00> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc98a48...c340fc7. Read the comment docs.


(Issue) Hoarder Bugs did not spawn with default config settings on Assurance

Currently testing different configurations of HoarderBud on LAN mode. I have landed on Assurance and March with the default configuration file, and the first time I landed on Assurance, there were zero hoarder bugs on the entire map, and I stayed in the facility for the entire day.
I have not had this issue since, so I have no idea what caused it. I did not see anything strange in the logs when starting up Lethal either. Just bringing it to attention in case something is found later down the road relating to this.
Here is my plugin folder for other mods I currently have installed, and I also have the required hookgenpatcher in the respective folder.

If you happen to reproduce it again, please attach a OutputLog.log from the BepInEx folder.
I heard of this happening to someone else as well but never managed to reproduce yet

Solving as stale


Resolve magazine schedules on Content interface.

 GraphQL guidance: how to filter a query to find content items with page number but no Digital Edition URL