Question/Proposal: Hide elements outside of viewport to increase performance

Hi,
I'm looking to implement something like https://github.com/bvaughn/react-virtualized or https://github.com/ayrton/react-cluster in combination with react-custom-scrollbars.
Are you planning on including similar functionality in this project?
Thanks!

I think this functionality is not in the scope of react-custom-scrollbars and react-virtualized already did a great job. But it would be nice if we could integrate the Scrollbars component into react-virtualized. I looked at the code and found that we could replace the scrollingContainer of the Grid component here: https://github.com/bvaughn/react-virtualized/blob/master/source/Grid/Grid.js#L475-L481
But this would only be possible with changes in both libraries, which I'm open for. Creating a new issue on react-virtualized would be a first step!

Interesting idea.
On the surface it seems like this might be as straight forward as updating Grid to support a custom scroll-container and just falling back to the current, plain <div> by default.
I don't think I want to add a dependency to react-virtualized and I'm wary of anything that might negatively impact performance (although I don't think this would). I'm also a little wary of custom scrolling although your demo page looks great in the browsers I've viewed it in and your source code looks clean.
I am supportive of giving this a shot. :)

I took a closer look at this last night and it seems like react-virtualized Grid and ScrollSync components overlap pretty heavily with react-custom-scrollbars Scrollbars component. I'm not sure there's a lot of benefit of integrating the two, although I can certain pass additional information to the react-virtualized onScroll callback (eg. clientWidth, clientHeight, scrollHeight, and scrollWidth) to enable similar functionality.
I'll leave the ball in @malte-wessel's court. If you think there's a meaningful integration, let me know! I'm happy to discuss. :)

See discussion here

@malte-wessel Would you mind elaborating a bit on what you think needs to be done to use react-virtualized with react-custom-scrollbars?  I am interested in trying to make the two work together also but having a hard time understanding where to start.  Thank you!

Quick update to this issue in case anyone else sees it and is curious. react-virtualized now supports a custom renderer prop- renderCellRanges. This function is responsible for rendering (and positioning) the visible range of cells. Its signature is:
function ({
  columnMetadata:Array<Object>,
  columnStartIndex: number,
  columnStopIndex: number,
  renderCell: Function,
  rowMetadata:Array<Object>,
  rowStartIndex: number,
  rowStopIndex: number
}): Array<PropTypes.node>

Grid provides a default implementation of this (obviously) but you can override it if you want to render customized UI along with your cells (eg. gantt chart style overlays, custom scrollbars, etc). Doing it this way is a little more work but it gives you much greater flexibility in what your grid looks like and react-virtualized still manages all of the data windowing for you.
If you're curious, you can see the default implementation here: https://github.com/bvaughn/react-virtualized/blob/master/source/Grid/Grid.js#L729

@bvaughn Thanks for the update.  Appreciate the ability to customize this!


REST Showcase fix for broken JSPs using S2 link tags

Update:

Fix REST Showcase JSP issue for s:link tags that replaced HTML link tags.
s:link tags cannot use JSP EL in them.  Modified JSPs to use scriptlet, JSP expression, and OGNL expression to work with s:link tag.

Hi Apache Struts Team.
I noticed a small bug with the REST Showcase when testing S2 6.0RC1, which prevented the JSPs from rendering.  A JIRA can be opened for this PR, if it is considered OK after review.

Coverage remained the same at 50.509% when pulling 583d399c243a5665cb23a4770d4526533c472664 on JCgH4164838Gh792C124B5:localS2_26_RestShowcaseFix1 into a4e7034eabd06cb5f920a54dd8a5cafa2f585416 on apache:master.


Rnintentclean falls reports IFW Enhance module not installed.

Hello,
Even though Riru and IFW Enhance modules are installed, this app reports an error "Incactive Module'. I played around with the different versions of Riru and IFW only to realise that Rnintentclean does not work when Riru and IFW are up-to-date.
Will you please look into this issue?

Use IFWEnhance pre-release v8 from GitHub. It works with latest Riru v26 in Magisk repo


chore(deps): upgrade runtime dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-runtime-dependencies-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2584

Questions ?
Please refer to the Backport tool documentation


registerCssFile position restricted to POS_HEAD?

I understand that for JS I can define the position, but for CSS it is explicit and limited to POS_HEAD?
There are cases where CSS can and is needed to be placed at POS_END (to satisfy for example Google's Page Speed Insight Rules) and not block the html rendering process of the browser...
At the moment I have to manually insert the code inside the view?
I think even the registerLinkTag is out of the question for such scenario.
(same applies to AssetBundles CSS I believe)
Please advise.

css file can be linked only in head according to html standard.
and style tag is not recomended

You are correct. I guess the only solution then is to append the tag in the head using javascript, to avoid the render blocking and to be in accordance to html standards...
Thanks!

@yurii-github @antokara
Hmm seems HTML5 Recomendations has a little ambiguity about  position: http://www.w3.org/TR/html5/document-metadata.html#the-link-element
You see "Contexts in which this element can be used:" is "Where metadata content is expected".
There is no exact definition of this context. But the whole page describes the content of this context e.g. head, title, base, link, meta, style.
And there is a paragraph:
"A link element must have a rel attribute.
If the rel attribute is used, the element is restricted to the head element."
Seems that it has redundant or mutually exclusive phrases.
So could anybody clarify the situation with  tag position?
Does Google right about https://developers.google.com/speed/docs/insights/OptimizeCSSDelivery#example
Or it violates W3C recomendation?

TriAnMan
in that example they add link to head via javascript what is ok. about style, it is ok in head only too as i know (it was ok in html4 though)
link is generic tag so it may have different places for different rel. for stylesheet it should be in head only.
regards


Error creating containers

TLDR: this test should pass.
Running the following requests works correctly using the default configuration:
curl 'http://localhost:3000/' -i \
    -H 'Slug: my-container' \
    -H 'Content-Type: text/turtle' \
    -H 'Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"' \
    --data-raw $'<> <http://www.w3.org/2000/01/rdf-schema#label> "My Container" .'

curl 'http://localhost:3000/my-container/' -i -H 'Accept: text/turtle'

However, using a file store I get the following error on the GET:
Error: Expected entity but got literal on line 1.
    at N3Parser._error (/home/noel/workspace/community-server/node_modules/n3/lib/N3Parser.js:924:17)
    at N3Parser._readEntity (/home/noel/workspace/community-server/node_modules/n3/lib/N3Parser.js:200:21)
    at N3Parser._readPredicate (/home/noel/workspace/community-server/node_modules/n3/lib/N3Parser.js:310:37)
    at /home/noel/workspace/community-server/node_modules/n3/lib/N3Parser.js:1073:127
    at N3Lexer._tokenizeToEnd (/home/noel/workspace/community-server/node_modules/n3/lib/N3Lexer.js:385:7)
    at /home/noel/workspace/community-server/node_modules/n3/lib/N3Lexer.js:515:22
    at N3StreamParser._transform (/home/noel/workspace/community-server/node_modules/n3/lib/N3StreamParser.js:46:7)
    at N3StreamParser.Transform._read (/home/noel/workspace/community-server/node_modules/readable-stream/lib/_stream_transform.js:177:10)
    at N3StreamParser.Transform._write (/home/noel/workspace/community-server/node_modules/readable-stream/lib/_stream_transform.js:164:83)
    at doWrite (/home/noel/workspace/community-server/node_modules/readable-stream/lib/_stream_writable.js:409:139)

I have combined the storage configuration from config-file.json and the authentication configuration from auth-allow-all.json for the configuration I've used to reproduce this error.
It seems like the problem is actually on the POST, because these are the contents of the my-container/.meta file (It's missing the subject):
 <http://www.w3.org/2000/01/rdf-schema#label> "My Container".

Update: I've realized memory storage is not working properly either, it doesn't throw that error but the turtle that is returned is not valid (it's missing the subject of the schema:label property as well).

Looks like the <> is never properly interpreted: https://github.com/solid/community-server/issues/288


src/tcmalloc.cc:278] Attempt to free invalid pointer 0x2183da8  Aborted (core dumped)

Did you solve it?


fix: update to eye v3.23.1

fix: update to eye v3.23.1

:tada: This PR is included in version 4.22.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release
v4.22.2

Your semantic-release bot :package::rocket:


Allow infinite dependent keys when using withOnyx

Details
I found that the dependent keys weren't being loaded because we were only passing props to the key functions. This won't contain the already loaded Onyx data because that data is only stored in the HOC's state.
The solution is to pass both the props and the state to the key functions.
There were a couple more fixes I had to do to ensure multiple levels of depencies are resolved as expected.
Related Issues
Fixes https://github.com/Expensify/App/issues/27906
Automated Tests
All covered by unit tests
Linked PRs
TBD

Unfortunately, when tested in App, this caused major performance problems that locks up the browser. So, while it is a great change, I guess there are some other considerations that need to be made for this to be viable.
CC @chrispader @kidroca @hannojg I'd love to brainstorm with you guys on this one.

Unfortunately, when tested in App, this caused major performance problems that locks up the browser. So, while it is a great change, I guess there are some other considerations that need to be made for this to be viable.
CC @chrispader @kidroca @hannojg I'd love to brainstorm with you guys on this one.

@tgolen sorry that i didn't respond yet. i'm gonna try to look into this issue this week, if nothing has changed 👍🏼


Should we benchmark languages other than Fortran, why, and how?

I see great value in implementing a variety of simple yet real-world algorithms in Fortran and benchmarking them along multiple axes:

Different problem sizes (e.g. array or matrix size)
DIfferent compilers
Different optimization flags
Different hardware

How about different languages? What would be the main purpose of that?
Are we interested in comparing the performance of Fortran and other language implementations, using idiomatic, naive code (i.e. the code that a novice would write), and thus comparing the compilers capability to optimize?
Or are we interested in writing code in different languages that produces the same (or as similar as possible) assembly, and then compare the source code?

I would consider this benchmark repository also as an example repository to learn about possible usage of the language to write performant code. Having different languages in the benchmarks could serve as kind of Rosetta stone for users proficient in optimizing code in another language trying to learn about optimization in Fortran.

Copying my comment from #9.
We discussed quite a few times, e.g. in #2, to include other languages.
Why: to actually have a comparison across languages. I expect that across Fortran/C/C++ perhaps even Julia one can fiddle with the code to eventually get similar speed. And we should have that final code. But also what would be interesting (for me) would be code that you would reasonably write as a domain expert, say a physicist. And we should have those codes too.
How: that has to be discussed, for now let's have it in some form, such as in #9. I expect we'll have benchmarks for smaller arrays, that have to be run many times. I have a runner for that somewhere, I'll see if I can contribute it. Then I expect to have longer running tests, such as #9, which are not as sensitive to how it is timed.

@certik I guess I'm asking for more clarity beyond what's been already discussed. I interepret this as, there will be at least two versions of each problem in each language:

A hand-optimized implementation which is the fastest one to produce the correct result (anything goes, as long as the result is correct). I think the purpose of this would be to compare and be able to say, what's the fastest possible you can get for a given problem, for a given language implementation (e.g. GFortran, LFortran, GCC, CLang, Julia, etc.).
A naive, idiomatic, simplest implementation. I think the purpose of this would be to compare how expressive different languages are (syntax, built-in functions, etc.). The comparison of the timing will also tell us something here, perhaps how good an implementation is at optimizing idiomatic code.

Is the above aligned with your view? Is there some other comparison that is missing, beside these two?

I would expect to have many versions of the same benchmark. Besides the two you mentioned, also one that allows assembly intrinsics and one that does not. One that perhaps uses more array operations, one that does not. If you look at the benchmarks in https://benchmarksgame-team.pages.debian.net/benchmarksgame/, they have many versions for the same languauge. I would expect every time someone contributes an improvement, we can have a new version. Also we each have a different "taste" what constitutes "nice code", so making sure we each have a version that we like there would drive the point home I think. See also:
https://github.com/fortran-lang/benchmarks/issues/2#issuecomment-656223298

I think having multiple languages is desirable, in as many languages as possible, so that the reader can compare both the verbosity and performance. For example, in #9 one surprising feature was that the C compiler does a relatively good job of optimizing (probably inlining?) function calls, whereas Fortran suffers from a bigger performance penalty in this case (probably it doesn't inline the function calls?) At any rate, such comparisons are interesting, at least to me. Perhaps we could define some set of languages as "minimal" and strive to have each benchmark have implementations in at least those languages. For example, C/C++, Fortran, Python and Julia could be candidates, since they are all popularly used for scientific computing.

@arunningcroc I agree 100%.

Modern languages have diverged to such a great extent in philosophy and design that direct comparisons are exceedingly difficult, especially for comparisons based on performance.
Parallelism
Fortran is an inherently parallel language. For a direct language-to-language comparison, you will have to handicap Fortran by comparing only serial code, leaving a ton of performance on the table even on modest platforms (e.g., an 8-core laptop), which defeats the purpose of a performance comparison. Or you can compare parallel codes, but then the comparison is between Fortran and Language X + Parallel Programming Model Y.
Vectorization, Multithreading, and GPU Offloading
The Fortran committee intended for do concurrent to support such technologies as vectorization, multithreading, and GPU offloading.  When the NVIDIA compiler offloads do concurrent to a GPU, the Cray compiler vectorizes or multithreads do concurrent, and gfortran generates SIMD instructions from do concurrent, the compilers aren't just generously giving us a bonus.   They are fulfilling the language designers' wishes.  I don't know that there are constructs in the other languages that were designed with similar intent. With the other languages, you'll have to insert OpenMP, OpenACC, CUDA or other compiler-specific statements to achieve what you can do with standard Fortran without bolting anything on the side.  Then the comparison is really between Fortran and Language X + Compiler Technology Y.
Other implicitly parallel Fortran features include array statements, elemental procedures, and pure procedures invoked inside do concurrent.  I'm not at all clear that there are comparable constructs in other languages and I would hope that the features I'm naming would be exploited to maximal effect in anything that would be labeled as idiomatic modern Fortran.
Libraries
Idiomatic C++ and Python typically rely upon external libraries even for such basic things as multidimensional array functionality.  A C++ programmer wanting performance portability across a range of heterogeneous hardware architectures, for example, is likely to hand off as much performance-critical computation as possible to a library like kokkos.  If you write self-contained C++ without exploiting such libraries, that alone might disqualify the code from being idiomatic C++.  If instead you incorporate libraries, then you're comparing Fortran to Language X + Library Y it seems unlikely that the proposed effort will match the performance of highly optimized libraries.
Generic Programming
I suspect that any modern C++ library is a by definition generic (using templates), but Fortran's generic programming features are still under development, making an apples-to-apples comparison impossible without a ton of clunky include statements, conditional compilation, etc.
Bottom Line
Let's all write code in the languages that make us feel most productive while providing acceptable performance.  Fortunately, there could be an ultimate convergence as the language developers learn from each other.  Fortran 202Y will support generic programming.  C++23 will support multidimensional arrays.  I'm not sure any of the languages named will ever support parallel programming in the way that Fortran does, however, because too small a sliver of the languages' programmers require scalable parallelism for the languages to add something that so fundamentally changes the language's execution model.  Julia is an exception to the latter statement because its designers are targeting high-performance computing.
Lastly, performance analysis and tuning is a challenging research topic in its own right.  It's probably not a great idea to wade into this area unless one is taking it on as a subject of research and planning to dive deeply into it.

@rouson, @milancurcic I see you are both a little bit reserved about the purpose of the benchmarks repository. It seems your arguments show the dangers of how we can fail. Yes, I agree that those dangers are real. But what do you suggest we do? Do you suggest we abandon the benchmark effort? And if not, what scope do you see we do?
I still think we should benchmark against other languages. We should benchmark in parallel and compare against C++ and Kokkos, among other things. As a user that is exactly what I would like to see.

@certik my "Bottom Line" section accurately summarizes where I stand.  I worry that raw performance comparisons between the languages distract from other important considerations such as programmer productivity and fundamental differences in the languages' design philosophy.  I also worry that the comparisons could devolve into debates about whether what is written is truly idiomatic in the given languages if the focus is performance.  Most importantly, I worry about what happens when the goals of writing high-performing code and idiomatic code diverge.  I cringe when I see deeply nested do loops doing what an array statement or elemental procedure call or do concurrent could do, but I also recognize that the nested-do form might be required in performance-critical code with specific compilers.  There's a great deal of variation in what the compilers can do to exploit some of the features that make Fortran shine in its ability to express mathematical formulas compactly and clearly.
In addition to all the other features that I've mentioned so far, I would hope that idiomatic Fortran would make extensive use of intrinsic functions, which also can replace multiple lines of custom logic, but the performance of intrinsic functions is another area in which I would expect considerable variation across compilers, compiler versions, and compiler flags.  Consider, for example, that some compilers can be directed to call some user-chosen optimized BLAS implementation top support matmul.  This brings me to another interesting language difference.  There's a proposal to bring BLAS-like functions Into the C++ standard to what I suspect is an even greater extent than Fortran currently supports via intrinsic procedures.  This too will greatly complicate comparisons and require that whomever is writing the C++ code is up for keeping up with the rapid pace of change in C++, which is already a really big language.

@rouson I think objectively you're quite right that comparisons between the performance of languages frequently make little sense. Nevertheless, people are interested in them, as can be seen from the continued popularity of websites like the benchmarks game, and the recent discussions in the discourse forum surrounding e.g. Julia. Julia, for instance, includes benchmarks quite prominently on its page. I know I certainly sometimes look for benchmarks when evaluating languages, with the full understanding that this can't give me a full picture.
I don't think a benchmark or any other marker of language performance has to be 100% fair and perfect to be useful or just plain fun to look at. Indeed, I think Julia's performance page says it best:

These micro-benchmarks, while not comprehensive, do test compiler performance on a range of common code patterns, such as function calls, string parsing, sorting, numerical loops, random number generation, recursion, and array operations.
It is important to note that the benchmark codes are not written for absolute maximal performance (the fastest code to compute recursion_fibonacci(20) is the constant literal 6765). Instead, the benchmarks are written to test the performance of identical algorithms and code patterns implemented in each language. For example, the Fibonacci benchmarks all use the same (inefficient) doubly-recursive algorithm, and the pi summation benchmarks use the same for-loop. The “algorithm” for matrix multiplication is to call the most obvious built-in/standard random-number and matmul routines (or to directly call BLAS if the language does not provide a high-level matmul), except where a matmul/BLAS call is not possible (such as in JavaScript).

@certik  If performance comparisons are the primary aim, then I recommend contributing to Jeff Hammond's Parallel Research Kernels (PRK).  Parallelism is an essential ingredient in any performance discussion in the multicore/manycore/GPU era.  PRK contains Fortran, C++11, Julia, Python, Ruby, UPC, and more.  I would love to contribute to refactoring some of the Fortran code to be more idiomatic along the lines of what I wrote above.  For example, PRK's [Fortran kernels] contain 11 different matrix-tranpose implementations.  In my view, idiomatic Fortran simply calls the transpose intrinsic function unless there is a strong motivation for doing otherwise and that motivation is likely to be compiler-, platform-, and problem-specific.  I see that at least one of the PRK transpose implementations calls the transpose intrinsic function.  I would argue that most other implementations are not idiomatic Fortran and I'd only use another implementation if I had solid, quantitative evidence that (1) the performance is significantly better for the more verbose transpose and (2) the code in question is the critical bottleneck in the application.

@arjenmarkus I have to admit that the whole time I've been writing responses, I've been thinking it would be fun to look at the proposed code. :)   From that perspective, it's definitely a valuable exercise.  I like the analogy someone made to the result being a Rosetta Stone.

@certik @arunningcroc @milancurcic having now looked at the first example, I feel even more strongly that this effort could do more harm than good.  First, I urge you to not call the repository "benchmarks." If you do that, you're diving into a field with a long and controversial history.  Consider the following language near the top of the README for PRK repository, a multi-language comparison effort very similar to yours except for the parallelism:
"These programs should not be used as benchmarks. They are operations to explore features of a hardware platform, but they do not define fixed problems that can be used to rank systems. Furthermore they have not been optimimzed for the features of any particular system."
You could adopt similar language, replacing "hardware platform" with languages.

Second, @certik, I strongly disagree with the idea of launching a new project to write "code that you would reasonably write as a domain expert, say a physicist" unless you're going to have comparison code to demonstrate much more modern practices.  The majority of domain experts are writing a narrow subset of a 31-year-old version of Fortran.  As mentioned above, I cringe every time I see nested do loops doing what an array statement could do as I suspect is the case twice in just the 55-line Poisson solver.  I call such code Cortran: it's the Fortran program that a C programmer would write because they think they have to loop over all the elements of an array just to initialize the array.
Moreover, the program uses fixed-size arrays despite allocatable arrays being one of the features that most makes Fortran shine -- not just because of dynamic memory allocation but also (especially) because of automatic deallocation.  Let's hope no domain expert is still using fixed-size arrays!
Even the "optimized.f90" Poisson solver appears at first glance to be standard-conforming Fortran 90.  In fact, with some reformatting, much of it would be standard-conforming Fortran 77.   If this effort moves forward, I hope that every code will have a "modern.f90" comparison that separates interfaces from implementations, uses array statements and intrinsic procedures wherever possible, and uses pure and elemental procedures wherever possible.  Every feature that I just cited was in Fortran 95 so this shouldn't even be considered a big step in 2021, but keep in mind that each of these features has potential performance implications and most of them should be positive performance implications with a sufficiently advanced compiler.  There are numerous optimizations that pure facilitates, for example.  A great service to the community would be to explore the performance implications of a much more modern approach and contrast it with more archaic approaches even just within Fortran.  I would find that a lot more useful than the comparisons to other languages.
Moving into the 21st century, I would hope that any effort to write a substantial amount of new code would decompose the problem into procedures and then separate the procedure interfaces (in modules) from procedure definitions (in submodules), and use Fortran's facilities for parallelism and concurrency.  With the exception of submodules, every feature I've named has potential runtime performance implications -- usually positive implications with a sufficiently advanced compiler -- so using these features fits perfectly with the goals of the repository.  And even submodules have potentially positive compile-time performance implications, which might matter less for small kernels, but having procedure interfaces sure makes for a nice introduction to the high-level goals of different parts of the program as expressed.

@rouson, thanks for the feedback. I think your concerns can be alleviated:

Idiomatic vs most performing: I would like to have both. I expect to have 10 different Fortran versions for a given problem. Some of them will get the top performance, showing that it can be done; but perhaps it would not be code that I would like to actually use. Not idiomatic. Also: I would like compilers to improve performance of the "idiomatic" code. So just the fact that today some version is the fastest does not mean much: as compilers improve, this will change.

Parallel vs serial: I want both, serial performance is important and easier to reason about. But parallel performance is ultimately what matters for a lot of HPC codes (depending on how it is parallelized).

Jeff Hammond's Parallel Research Kernels (PRK): yes, the PRK is a significant subset of what I would like to see, but I see the scope much bigger.

Whether we like it or not, people will keep doing such comparisons and posting online. Such comparisons influence people's choices. I know from personal interaction that people watch the "benchmarks game" site.
We don't have to call the repository "benchmarks". We can call it "Rosetta". Or we can call it "how to solve a given problem in Fortran and other languages", so that people can learn what the options are: how to write idiomatic code, how to write the "simplest" code. How to write high performing code. And what is (currently) the top performing code+compiler+platform combination.
I think the harm comes from drawing (wrong) conclusions. But having codes that solve a problem can't harm. In fact, we have different opinions what "idiomatic" means in Fortran. For example I do not like object oriented for numerics. I know others do. So we should have both. A third person comes an says "I don't like either of these!", so they can write a third version that they think is the best and we should include it too.
Then we should have automatic tooling that can compile all such codes with different compilers and compiler options and time it. We should present it in a nice way, so that you can find code version that you personally like the most, and you can see how it stacks using current compilers. Then we should have a conversation how to improve compilers (or is it even possible for a given version). If it is not possible, then maybe that should not be the "idiomatic" way to write such code. And so on.

Regarding your comment at https://github.com/fortran-lang/benchmarks/issues/10#issuecomment-869787773, what you described should absolutely be one version of a Fortran code that solves a given problem. Depending on the problem, I might agree or I might not, I would have to see the code. If I do not agree, then I can submit another version. I expect we will have 10 versions easily. Then we can see them side by side, see how they perform, see how easy they are to read, to maintain, etc.
A physicist can absolutely learn how to program in modern style. That is what I meant. But it needs to be simple to learn.

@certik I agree with everything you wrote and if I can do it quickly, I'll contribute one modern.f90 companion to the poisson2d subdirectory.  It might also be nice to have a poisson3d version to nudge things a bit closer to the kind of problem likely to appear in applications and the kind of problem for which performance matters more.
While we're at it, I wonder if there should be Matlab versions or some open-source Matlab equivalent such as GNU Octave.  A surprising amount of real science happens in Matlab and the performance differences can be even more significant relative to Fortran than with the other languages mentioned so far.

I opened #22 just for the name change. Thanks @rouson for all the feedback. I feel good about this. I also opened tons of new issues for some ideas for benchmarks that we can add.

@rouson Just for the record, many versions of the Fortran codes were proposed in the Discourse topic, and I only included the ones that ran fastest on my machine. That includes a vertorized version. As for allocatable arrays, I don't really understand why that would be more modern. I'm after all dealing with a fixed size calculation. Any pointers on that? Anyway, I hope you contribute a modern.f90 version as well, and if you do, I can also run that on my machine with the same settings, then we can get the timings on it.
I also welcome the name change, but I'm dubious that any code we post just to compare similar algorithms in different languages could really do a lot of harm. We could call the repository "the instructions for world domination", and it would not change the content one bit. I think some trust in the intelligence of the reader is warranted here.

@arunningcroc I personally like fixed size arrays as well, as they are also automatically deallocated (just like allocatable arrays) and for a simple example like you did I think they are a perfect fit. But as I mentioned in https://github.com/fortran-lang/benchmarks/issues/22#issuecomment-870639621, I don't want to argue about this now what the "modern idiomatic" style is. For now I just want to have all approaches in, and we should have that discussion later.

I followed the discussion here and related threads a bit and I'm somewhat disappointed about the overall tone. I know that coding style can be a somewhat loaded topic, but let's not judge each other by the way we write small code examples, please.
Let's keep this a place for respectful collaboration such that we all can enjoy working on this project together.

@awvgk apologies for any judgement.  I do worry that a large part of what turns so many people away from Fortran is the older Fortran that they've seen.  If we're still writing code that is effectively Fortran 77 plus a tiny subset of Fortran 90, it's going to be very hard to attract new people to the language.  Moreover, it's worth noting precisely what some of the older constructs communicate to the reader and to the compiler.  A do loop, for example, tells the reader and the compiler that a certain set of steps are to be one in a certain order.  An optimizing compiler will nonetheless do its best to determine whether that order really matters and will violate the order in the name of performance.  Likewise, a patient reader will step through the logic mentally and often figure out that the order doesn't matter, but that's extra work and extra screen real estate.  I would still argue that we can do better by the compiler and by the reader and that there is inherent value in not serializing operations unnecessarily. I hope this way of communicating it is more factual than judgmental.

I am not against benchmarking implementations in different languages. My reservation is the same as with any other Fortran-lang project (stdlib, fpm, website etc.): Let's build things mindfully and with intention rather than just throwing things in there and seeing what happens.
Now, I understand and recognize that nobody here argued that we should just throw things in and see what happens. But, I also haven't seen a clear goal on what exactly we want to compare between the languages. Take for example #9: There, I'd like to have seen documented (in the README perhaps):

What is being measured and compared, exactly? My interpretation of that PR is that the goal is to compare the ability of compilers and interpreters to generate performant code from two variants of source code.
If so, are we sure that the sources between different languages are semantically equivalent? Can we say the same about the run-time libraries used? If they're different, we should document that and say that we're comparing the performance of idiomatic code that is not supposed to be semantically equivalent.
How were different programs compiled? Fortran and C versions are instructed to be compiled with -Ofast, but was NumPy compiled with -Ofast for the posted Python results? If NumPy was compiled with -O2, it may be more meaningful to use the same for Fortran and C versions.

My point is, if you ask and answer the question what exactly you're trying to compare, you'll have a better chance to make a meaningful (fair) comparison.
Ultimately, what I want to avoid is a benchmarks repository where Fortran implementations are inadvertently fine-tuned to demonstrate or even imply language superiority. As this is a common criticism of other benchmarks like those on the Julia website and some recent blog posts, I expect that we wouldn't want to repeat the same, with tables turned. All of us here are responsible from preventing language wars from happening.

It's a chicken and egg problem: we can't design and show what we are trying to do without first having a few benchmarks in, but we can't get the few benchmarks in because we do not have criteria to judge them.
I think we all understand the dangers of mindlessly benchmarking. Also nobody wants fine tuned benchmarks here. I think we have all explained enough what we do not want. So let's now discuss what we want.
I proposed a vision in #22.
That vision answers your questions questions:

What is being measured and compared, exactly? My interpretation of that PR is that the goal is to compare the ability of compilers and interpreters to generate performant code from two variants of source code.

Nothing is being directly measured. This is an example ("idiom") how to solve a 2D Poisson equation with certain boundary conditions using a first order finite difference scheme and relaxation method. We already have two such examples contributed. I would like to see even more. Yes, we are interested in timings and benchmarks for this too, as one of the many other criteria, such as readability, and how hard it is to write.

If so, are we sure that the sources between different languages are semantically equivalent? Can we say the same about the run-time libraries used? If they're different, we should document that and say that we're comparing the performance of idiomatic code that is not supposed to be semantically equivalent.

As proposed in #11, we need tests to ensure any submitted example / idiom returns exactly the same answer. Regarding libraries to use, there will be 10 codes in C++ let's say, so some can use other libraries, some might only use "pure C++". We can look at timings and other pros and cons and compare and everybody can make their own opinion which one is better.

How were different programs compiled? Fortran and C versions are instructed to be compiled with -Ofast, but was NumPy compiled with -Ofast for the posted Python results? If NumPy was compiled with -O2, it may be more meaningful to use the same for Fortran and C versions.

With as many compilers / options that our infrastructure allows. NumPy can be installed using Conda (probably how a lot of people would install it), so a version number should be enough to identify. This has been discussed previously: https://github.com/fortran-lang/benchmarks/issues/2#issuecomment-656232163
@milancurcic and others let me know if you agree / disagree with the vision I presented. If it is too early to tell, then let's simply at least try, and if it is not going in a direction we want, we can always remove this repository from fortran-lang later. If you have a different vision for what this particular effort could become, then please share it.

I proposed a vision in #22.

The vision is great, I like it a lot. In this issue we're discussing last bullet point specifically.

It's a chicken and egg problem: we can't design and show what we are trying to do without first having a few benchmarks in, but we can't get the few benchmarks in because we do not have criteria to judge them.

Why can't we? I very much think we can. We just need to ask the question.
For the 2-D Poisson problem, for example, the question could be: How fast are the executables produced by Fortran and C compilers given idiomatic (no matter how we define this) and semantically equivalent Fortran and C code?
Is the above not a meaningful, interesting, and simple enough question to ask? I think it is, and it's something we can measure.
Then, if we like the question and agree to start there, contribute the Fortran and C implementations like those in #9. Meanwhile, ensure they're both correct and produce the same results (#11). Then, we can look and the timings. Now we have a minimal framework to expand upon, and add other languages to the mix. (I suggested Fortran and C to start because they have companion compilers, so it's likely easier to make a meaningful benchmark).

NumPy can be installed using Conda (probably how a lot of people would install it), so a version number should be enough to identify.

This is not what I meant. I meant: If NumPy is not compiled with -Ofast, don't compile Fortran and C code with -Ofast. If we established the minimal framework I suggested above, we wouldn't have this discrepancy.
I'm happy that the discourse in this thread is shifting from "we're doing benchmarks" to "we're doing idioms, and maybe we'll do some benchmarks", but it doesn't make for a fair discourse because I originally asked "Should we benchmark other languages", and not "Should we include other languages at all".

What I proposed above as the "minimal framework" or "the first question" is just my idea of where to start, I'm not sure that it's the best way to go and I don't have a lot of experience in that area.
Is there interest in discussing what would be the minimal framework to start with? In other words, what to compare and measure? I would like that.
Or would you prefer to just focus on idioms (source code) for now, and not worry about benchmarks until later?

I as well don't think of allocatable and static arrays as modern or archaic. They're just different. When I don't need an allocatable array, I don't use it.
However, we should plan ahead, to facilitate easier testing and timing, problem size, like M in #9, should be a CLI argument, and this will require the arrays to be run-time allocatable.

Thanks @milancurcic, good points. Yes, if we can constructively figure out criteria, then by all means let's do that. I like what you started with the question. My comment would be that I think I want to answer more questions than just Fortran and C.
All I was saying is that I don't want this effort to die just because we do not have well fleshed out criteria.
The same with benchmarks. I very much want to focus on benchmarking. However, the objection was that only focusing on benchmarking would be harmful, so I am willing to let benchmarking go for now, and focus on idioms and code, and worry about benchmarking later.
In terms of what I care about long term, I would like to have the various maintained codes (in different languages) that solve the given problem (and where each of us can find that one version that we personally really like), and that could be benchmarked, and the infrastructure that allows this. For example, I want to have a NumPy version (or versions) that work. So that once I get to benchmarking, I don't have to worry about writing or debugging the codes, I already have them, and can concentrate on actually getting meaningful benchmarks out of it (as you said, then things should perhaps be compiled with the same options, although not in all cases, such as if you just want to compare the "default experience").

Great! I'd actually like benchmarks to move forward together with idioms, just with all these cautions that we discussed in mind. We have a good start with the 2-D Poisson, and I'd like us to work on it and polish it--I think mainly document it, ensure it's correct, and that it clearly conveys a message that we want to send. I think we'll have a minimal framework then to re-use for other problems.

I absolutely do not intend to kill this effort :).

A source of cross-language benchmarks is the Benchmarks section of the Fortran code on GitHub list I am maintaining.

On Tue, Jun 29, 2021 at 6:49 PM Milan Curcic @.***>
wrote:

For the 2-D Poisson problem, for example, the question could be: How fast
are the executables produced by Fortran and C compilers given idiomatic (no
matter how we define this) and semantically equivalent Fortran and C code?

Based on my extremely limited experience with performance analysis, I
suspect this is a much, much deeper, thornier issue than one might
immediately think and it's one that's likely to unnecessarily step on toes
and lead to unproductive discussions.  I'm not just saying this
hypothetically.  Without going into specifics, I'm watching these problems
happen in real-time elsehwere right now.  In that case, there exist some
reasonably widely known codes that could easily be misinterpreted as
answering exactly the question you're posing and yet experts in one the
languages involved can immediately identify fundamental problems in the way
that language's code is written that severely disadvantage it relative to
the other languages.  That's harmful.  The harm wouldn't be a big deal if
it weren't for the facts that (1) the people involved do not have the
freedom to chase after every such occurrence and volunteer time to fix the
problem because they have jobs with high-stakes deliverables, (2) the codes
in question live for years with no one fixing the issues, and (3) anyone
who naively encounters and runs the codes will reach conclusions that
incorrectly shed a bad light on other people's life work in developing the
languages and compilers and such.
Even setting aside the need for deep expertise in the language(s) in
question, there are so many issues related to the interplay between problem
size, hardware architecture (cache size, memory bandwidth, etc.), compiler
choice, compiler version, compiler options, algorithm choice, etc.
Computing enough cases to broadly range across the options along any one of
the variables just named will take a considerable amount of time and still
leave a high-dimensional parameter space to judiciously explore along all
of the other axes.  I recommend staying away from drawing any performance
conclusions that apply beyond one person's particular choice for each of
the aforementioned factors, and I doubt that my list was exhaustive.
Moreover, most compilers these days are lowering any given language down to
some language-independent, intermediate representation, at which point I
assume that all languages are functionally equivalent anyway, which makes
the whole exercise pointless to some extent.
And as I have pointed out at length, there are fundamental differences in
the design of the languages that prevent such comparisons.  Most of the
languages we're discussing have no semantic equivalent to coarrays so, at
best, you'll have to leave out one of the most important factors in
performance (parallelization) to do any language-to-language performance
comparison.  That defeats the purpose.
I think the best purpose of this repository is to serve as a sort of
Rosetta Stone for translating across various languages and  across various
idioms within a language.
Damian

Thanks @rouson, this is exactly the kind of perspective that we need. I didn't have sufficient foresight to recognize these issues when posing my question. I agree with pursuing only idioms and not benchmarks until we convince ourselves otherwise.

I agree with pursuing only idioms and not benchmarks until we convince ourselves otherwise.

Thank you. Looks like we are in agreement now. I still want the timings though! Just not presented as benchmarks initially.


Nominatim API returns HTTP 404

Nominatim is supposed to come with an public API, but the API doesn’t seem to be accessible on our instance. For example, the following request returns an HTTP 404 error:
https://nominatim.openhistoricalmap.org/search?X-Requested-With=high-expectations&format=json&q=Northwest%20Territory

nominatim.openhistoricalmap.org itself is up and running, but the /search endpoint is missing for some reason.
This causes Overpass turbo to return an error when the query includes a searchArea: shortcut, as generated by the in: operator in the wizard syntax. For example, if you enter historic=monument and type:node in "Northwest Territory" into the wizard, it generates the following query:
/*
This has been generated by the overpass-turbo wizard.
The original search was:
“historic=monument and type:node in "Northwest Territory"”
*/
[out:json][timeout:25];
// fetch area “Northwest Territory” to search in
{{geocodeArea:Northwest Territory}}->.searchArea;
// gather results
node["historic"="monument"](area.searchArea);
// print results
out geom;

But if you run it, you get an error that “Northwest Territory” can’t be found, even though searching for the same string in the osm.org search box (which uses a different endpoint internal to ohm-website) returns this boundary relation.

The geocodeArea: shortcut and in: operator would be very useful for creating quick, one-off queries without having to find the boundary relation first. More broadly, many command line tools, applications, and libraries rely on the Nominatim API.

Seems here it is missing the .html . eg:
https://nominatim.openhistoricalmap.org/search.html?X-Requested-With=high-expectations&format=json&q=Northwest%20Territory
The static files are stored in S3, and we are serving the site through CloudFront.

The static files are stored in S3, and we are serving the site through CloudFront.

Overpass turbo is attempting to connect to an API endpoint that serves JSON responses, not the HTML GUI. If the Nominatim frontend is only being statically published, then I guess this would be a request for dynamic publishing.

I see , if that is the case, this is the right  endpoint: https://nominatim-api.openhistoricalmap.org/search?X-Requested-With=high-expectations&format=json&q=Northwest%20Territory I  am going update it in nominatim.

https://overpass-turbo.openhistoricalmap.org/

This is working  in production now, good to close this ticker here :)


Added the option for default network parameters

Added the option for default network parameters.
Example:
apiVersion: v1
kind: Pod
metadata:
name: pod-example
annotations:
 v1.multus-cni.io/default-network: "{"mac":"aa:bb:cc:dd:ee:ff"}"

This will merge the configuration with the master plugins and pass the mac data to it.
This PR also allow to configure 1 addition network
Example:
apiVersion: v1
kind: Pod
metadata:
name: pod-example
annotations:
 v1.multus-cni.io/networks: "{"mac":"aa:bb:cc:dd:ee:ff"}"

without the slice [].

Pull Request Test Coverage Report for Build 740

12 of 14 (85.71%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 51.762%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

k8sclient/k8sclient.go
12
14
85.71%

Totals

Change from base Build 737:
0.3%

Covered Lines:
661

Relevant Lines:
1277

💛  - Coveralls


Cannot compile sourceview version 0.6.0

With gtk 0.5.0 and sourceview 0.5.0, all goes fine, until I try to use
gtk::ResponseType::Other(0)
which is only available from gtk 0.6.0.
So I change to gtk 0.6.0, sourceview ^0, and now get
error[E0433]: failed to resolve: could not find 'FileLoader' in 'sourceview'
Changing to sourceview to 0.6.0 makes no difference.
The doco shows 'FileLoader' as features v3_14, so I try adding that to sourceview, which gives me
error: failed to select a version for 'sourceview'.     ... required by package 'foo v0.1.0 (/.../foo)' versions that meet the requirements '= 0.6.0' are: 0.6.0 the package 'foo' depends on 'sourceview', with features: 'v3_14' but 'sourceview' does not have these features.
This is a known bug, which I reported recently (issue #92, now closed).
To work round the v3_14 problem, I clone sourceview branch 0.6.0, fix the 'v3_14' bug, and point cargo at my local copy. Now the error is:
    Updating git repository 'https://github.com/gtk-rs/pango' error: failed to select a version for 'gdk-pixbuf-sys'.     ... required by package 'sourceview v0.6.0 (/.../sourceview)'     ... which is depended on by 'foo' v0.1.0 (/.../foo)' versions that meet the requirements '*' are: 0.8.0 the package 'gdk-pixbuf-sys' links to the native library 'gdk_pixbuf', but it conflicts with a previous package which links to 'gdk_pixbuf' as well: package 'gdk-pixbuf-sys v0.8.0'     ... which is depended on by 'gtk v0.6.0'     ... which is depended on by 'foo v0.1.0 (/.../foo)' failed to select a version for 'gdk-pixbuf-sys' which could resolve this conflict
I'm stumped by that, as I am still somewhat of a noob with Cargo/git versioning, and would be grateful for some guidance on a work-round that works.  :)

From http://gtk-rs.org/docs/sourceview/struct.FileLoader.html , it says it requires the v3_14 feature to be enabled. :)

Cargo.toml:
name = "foo"
version = "0.1.0"
authors = ["austin"]
edition = "2018"

[dependencies]
gtk = "0.6.0"
sourceview = { version = "0.6.0", features = ["v3_14"], path = "../sourceview" }

main.rs:
use gtk;
fn main()
{
	println!("This requires GTK 0.6.0: {:?}", gtk::ResponseType::Other(0));
	println!("This requires sourceview 0.6.0: {:?}", sourceview::FileLoader::new());
}

I've now given up trying to get 0.6.0 to compile. The github code lacks the v3_14 feature, so you have to clone it and fix locally, but even then it will not play nicely with all the other GTK crates (at least, not in any cobination of versions that I can find)  I have now switched to 0.7.0, which has a few breaking changes, but only needed minor fixes to my code, which now compiles and runs just fine.


It would be interesting to to see charts visualizing SCB's per-age distribution data

Similarly to your staked bar chart from county-data.ipynb that shows regional distribution:

It would be also very interesting to see similar chart for per-age distributions (provided in SCB's Excel in Tabell 2)
Making that exercise in Excel formulas it might look like:
( 2020-06-05-preliminar_statistik_over_doda_inkl_eng+(61-rolling).xlsx , .ods.zip )

It would be really interesting to have interactive multi-selection for legend, to see all that groups together, each of it separately, and in any other combinations.

By the way, at
https://www.socialstyrelsen.se/statistik-och-data/statistik/statistik-om-covid-19/statistik-relaterad-till-covid-19/
it could be found even longer weekly statistic (for year 2015 - 2020) with per age-gender distribution
https://www.socialstyrelsen.se/globalassets/1-globalt/covid-19-statistik/statistik-relaterad-till-covid-19/antal-doda--per-alderskategori-och-vecka-2015-v1-2020.xlsx
And it looks like it constantly updating on weekly bases.
However age diapasons in that  Socialstyrelsen-statistics are different to thous from  SCB.
So it looks like only part of 80+ diapasons could be matched across that sources (to make some "check-sums" which allows one to evaluate cross-sources validity)
Also it looks little bit interesting, that initially some parts of 70+ diapason is spanned into 65 ... 80 diapason. From my point of view it could be related to pension age diapasons. In flavor of raising retirement age up to near 70, it looks like no one would like to highlight that fact that exceeding (natural) mortality starts exactly from 70+ year age, so that right after (near) start of pension age ...

By the way, at
https://www.socialstyrelsen.se/statistik-och-data/statistik/statistik-om-covid-19/statistik-relaterad-till-covid-19/
it could be found even longer weekly statistic (for year 2015 - 2020) with per age-gender distribution
https://www.socialstyrelsen.se/globalassets/1-globalt/covid-19-statistik/statistik-relaterad-till-covid-19/antal-doda--per-alderskategori-och-vecka-2015-v1-2020.xlsx
And it looks like it constantly updating on weekly bases.
However age diapasons in that  Socialstyrelsen-statistics are different to thous from  SCB.
So it looks like only part of 80+ diapasons could be matched across that sources (to make some "check-sums" which allows one to evaluate cross-sources comparison validity)

Interesting that even longer weekly statistics (with age-gender breakdown) also available across wide list of EU countries and their regions at Eurostat table
Deaths by week, sex, 5-year age group and NUTS 3 region, in tree (has really nice csv export)
Reference on this table could be found at the bottom of SCB page
https://www.scb.se/om-scb/nyheter-och-pressmeddelanden/folj-preliminar-statistik-om-dodsfall/


[ci] Fail CI on broken FPGA runs

When we introduced the CI runs on FPGAs we downgraded failures to
warnings to ensure that infrastructure problems wouldn't disrupt the
development workflow. The infrastructure has shown to be stable now and
is actually catching real bugs, so we can make the warning an error
again to raise visibility.

Just so it's easy to find, #4155 is a good example of a PR that caused one of the FPGA tests to fail but this went unnoticed due to continueOnError: true.

yeah i also ran into an issue on #4163 where an error was hidden.
Still I'm a bit hesitant making this part of CI.. mostly because it could mean that big structural changes to the design will be hard to merge if they impact a lot of things (I am selfishly thinking of things like power and reset...).
Some time in the future we will also need to re-organize the top to have our logical and physical hierarchies match better... that might be another.

This change is mostly about ensuring that we make an informed and conscious decision when breaking parts of the design. If we decide to do so, we can always disable the test in the CI config with a simple line as shown in this PR. Currently, we often don't even notice such breakage, as #4555 showed.


Remove cron trigger from pipeline job

The job is now successfully running using the event based triggering mechanism in openshift.
This task will update the pipeline job, deactivating the cron based trigger

Now thinking of leaving this in place after the listener missed a few days.  Having the listener trigger should provide the data earlier, however if for some reason that trigger fails, the cron job will ensure the process runs in the morning before the data is required.


fingerprint command

Add command calculating fingerprint of schema.

done in https://github.com/rkluszczynski/avro-cli/releases/tag/avro-cli-0.2.4


Feature/voiceover

updated https://github.com/pkluz/PKHUD/pull/213 according review

It will break XCTest - will reopen after fix

In this PR feature voiceover works properly + XCTests will work fine + allow to use HUDContentType for any local PKHUD as well as it used for global PKHUD via HUD class


SimpleRouteLocator: Change fields from private to protected to support extension

Hi team, I want to implement my own public Route getMatchingRoute(final String path) method, and I would like to extend the functionality from SimpleRouteLocator, the problem is that the fields are privated and I should implement all the code again.
I think that changing private to protected will do the trick. I don't want to redo all the code since that implies more responsibility to maintain a healthily code.
Please let me know if it's suitable. Thanks.

Pull requests welcome, though I'd prefer protected getters rather than protected fields.


No runner matching the specified labels was found: macos-11.0.

Description
Big Sur doesn't seem to be available.
Area for Triage:
Containers, Apple
Question, Bug, or Feature?:
Bug
Virtual environments affected

[ ] Ubuntu 16.04
[ ] Ubuntu 18.04
[ ] Ubuntu 20.04
[ ] macOS 10.15
[X] macOS 11.0
[ ] Windows Server 2016 R2
[ ] Windows Server 2019

Expected behavior
The runner should run in MacOS 11.0
Actual behavior

X This check failed.
No runner matching the specified labels was found: macos-11.0.

Repro steps
https://github.com/scottgigante/s_gd2/actions/runs/499768168

Hello, @scottgigante
macOS 11.0 pools will be transited to private preview. #2486

Hello, @scottgigante
macOS 11.0 pools will be transited to private preview. #2486

Thank you @al-cheb . If you'll excuse my ignorance, this was very much not clear from the docs.

Thank you @al-cheb . If you'll excuse my ignorance, this was very much not clear from the docs.


Remove Python 2isms

I've used the pyupgrade tool to remove a number of Python 2isms from the code.
I've split the various changes up into multiple commits to make it easier to review.

All tests are now green :-)

Thanks!


Multi-model scatter by default

Default behavior changes from only plotting the first to plotting all models.

Plotting several models after each other produces different colorbars, which is confusing, but then I would question the usefulness of knowing the point density with such level of detail, it consumes a lot of the 'ink' on the plot without adding a comparable amount of value.

Revisit this after completing #214

I would suggest closing this and open again from main (easier and safer than merging)

I would suggest closing this and open again from main (easier and safer than merging)

Indeed.


Section 3.4 Discover Response Supported Trust Frameworks

The discover response mentions   "subject_trust_frameworks_supported": [
"ebsi"
],
which should be adopted to EWC trust list accordingly.

Let's take this up for Friday's call or later on how to support trust frameworks in EWC.
The above example is how EBSI has done it today.


Update Haskell platform to 7.10.3

Can be the Haskell platform cask updated to v 7.10.3 (current 7.10.2)?
Thanx

It's super easy to do yourself, check out CONTRIBUTING - we have a cask-repair script that automates away the pain :)
Feel free to ask if you need help or have questions!

@maroselo I’m doing it now (though you should take up @adidalal on that suggestion). The cask actually needed a small edit, since url changed slightly.

https://github.com/caskroom/homebrew-cask/pull/19469

Thanx, guys!


App crash upon signup

Submitting signup for Bullying Policy Makeover on Thor leads to app crash below. Campaign is active (https://thor.dosomething.org/api/v1/campaigns/345)
Signed up ok on staging for Comics to the Rescue.
2016-02-23 17:05:51.263 DoSomething[1484:94285] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSPlaceholderDictionary initWithObjects:forKeys:count:]: attempt to insert nil object from objects[0]'
*** First throw call stack:
(
0   CoreFoundation                      0x0000000112fb3e65 __exceptionPreprocess + 165
1   libobjc.A.dylib                     0x0000000112a2cdeb objc_exception_throw + 48
2   CoreFoundation                      0x0000000112eb68ce -[__NSPlaceholderDictionary initWithObjects:forKeys:count:] + 318
3   CoreFoundation                      0x0000000112ec8c3b +[NSDictionary dictionaryWithObjects:forKeys:count:] + 59
4   DoSomething                         0x000000010e56c579 -[DSOCampaign dictionary] + 372
5   DoSomething                         0x000000010e55511f -[DSOCampaignSignup dictionary] + 150
6   DoSomething                         0x000000010e569efa __71-[DSOUserManager signupUserForCampaign:completionHandler:errorHandler:]_block_invoke + 279
7   DoSomething                         0x000000010e562578 __63-[DSOAPI postSignupForCampaign:completionHandler:errorHandler:]_block_invoke + 154

Can't replicate this now, wondering if this could have been a result of a result of patching servers yesterday....
Considering not passing a signup.dictionary but instead just the response object we receive from the POST request to avoid crashing...

Hoping we should no longer see these with #917 and #918, closing for now.


openssl3 support iphoneos install

openssl3 supports iphoneos install

ci failed

I ran the CI again, it seems to be working on macOS ARM (GitHub updated macos-latest to macOS 14 running on M1)


[Tab Complete] failed, the bell character shows up

This is SBCL <IP_ADDRESS>-c9e14929a, ...
...
Linedit version 0.17.5, smart mode, ESC-h for help.

🔔

After [Tab Complete] failed, the bell character shows up.

Same issue here

Same. It's a mistake inside the beep method found in terminal.lisp.
The character written to the console should be #\Bel (code-char = 7) and not #\Bell (128276).


feat(AlgebraicGeometry): the big etale site

For any morphism property P, we define the pretopology on the category of schemes defined by P as jointly surjective families of morphisms satisfying P. As an application, this is used to define the étale site.
The small site associated to P and with it the small étale site comes in a follow-up PR.

For reviewers: I think the large-import is harmless here, as the file is a leaf file. The "big" import is the file Mathlib.AlgebraicGeometry.PullbackCarrier. One option to avoid this would be to ask for IsJointlySurjectivePreserving P instead. I think though that this class is intendend to have a very restricted scope to avoid pollution with unnecessary assumptions. If people still prefer this way, let me know.

maintainer merge

This is great!
bors merge


New layout proposal - single compact group

Hello David,
all the best in the new year 2024!
We are currently evaluating whether your brilliant library can be used for our purposes.
We have large structures, but they are relatively flat - with few levels.
The problem is: Even with the compact layout, the chart becomes very, very wide.
We saw the idea for a new layout: https://github.com/bumbeishvili/org-chart/issues/236
It's a nice but very complex idea. On the one hand, it is complex to implement, and on the other hand, this structure could overwhelm some users.
Something much much simpler would make us happy, hence the following suggestion:
New layout proposal - single compact group
Instead of being divided into two groups, child-nodes are bundled in a single group.
The goal is to save as much width as possible. Since one of two compact groups is eliminated, parent-nodes can be positioned closer together.
New parameters (just as an example, can be named and organized completely differently):

compactGroupsCount with value range 1 or 2, default value is 2; relevant only if parameter compact == true (compact mode)
compactGroupsLayout with value range "top_right" or "bottom_left", relevant only if parameter compactGroupsCount == 1; states whether the one single compact group is on the left/right side, in the "top" or "bottom" layout, or top/bottom in the "left" or "right" layout

Of course we could try to implement this new layout on our own.. This would be very time-consuming for us (lack of know-how). On the other hand, if you could offer this then probably many people around the world would benefit, not only we as in the case if we enhance at our site.
Please consider whether you would implement such a layout and what time frame would be conceivable for the implementation.
Thank you in advance!
Nikola
Some Screenshots
1. Original - "top" layout, compact mode:

2. Transition to the new single group mode - "top" layout, compact mode, imaginary parameter compactGroupsCount set to 1, compactGroupsLayout == "top_right":

3. Final result - "top" layout, compact mode, compactGroupsCount == 1, compactGroupsLayout == "top_right":

(EDIT Jan 4: Order of nodes corrected)

Hi Nikola, I understand what you mean.
Unfortunately, I won't be able to build a layout similar to yours, since currently I don't have much time for it.
Although your suggestion still would be useful, since it gave me an idea to make it possible to override & add new custom layouts easily.
What I'd suggest for your case it to manipulate the data in a way, so that at the last level you have  a single node, which will have additional information attached, then you can set different heights for this last level and have a custom HTML so that you simulate single column layout
https://stackblitz.com/edit/js-pf6ouo?file=index.html

Thank you for the prompt response!
We have also thought about something similar, but did not pursue it further, as several disadvantages would arise:

The block of leaf elements is not collapsible.
The number of child elements in the expand button of the parent element is misleading.
There is no connection line comming from the parent for the individual child elements in the block, only one line for the entire block, which leads to a difference in the design of the connections of the leaf nodes, which ist not intuitively explainable to users (user asking: why is that?).
A fourth disadvantage, which is not as obvious as the first three: The vertical line in the block of child elements is an HTML line, whereas the other connection lines are SVG lines. This would lead to different rendering of the lines (in different browsers on different platforms), especially if we generate a PNG image or if we generate a PDF file and print it from the PDF file. Basically there are two types of lines (HTML and SVG) and three types of representations - the first on the screen in the browser, the second on the screen in a PDF reader, and the third - on paper after printing. It will be cumbersome or even impossible to keep the appearance of the two types of lines consistent in all three types of representations (at different scale factors and different browser and PDF engines).

Each user of your library must decide in her individual case whether she wants to use this solution with its disadvantages in a productive application.
We fully understand that you don’t have time at the moment. Nevertheless, would it be theoretically conceivable for you to provide this feature? Not now, but at a later point in time - sometime in the future?

Some disadvantages can be addressed

The block of leaf elements is not collapsible.

Leaf nodes aren't collapsible in the normal tree too

The number of child elements in the expand button of the parent element is misleading.
You have the possibility to display whatever you want in the expand button, similar to node

There is no connection line comming from the parent for the individual child elements in the block, only one line for the entire block, which leads to a difference in the design of the connections of the leaf nodes, which ist not intuitively explainable to users (user asking: why is that?).

you don't need a multiple line, what you need is to connect line coming from a parent to the last node (again, configurable) and then having a single line from each custom node to the left which will have similar styling. But unfortunatelly, animation won't work  well here

A fourth disadvantage, which is not as obvious as the first three: The vertical line in the block of child elements is an HTML line, whereas the other connection lines are SVG lines. This would lead to different rendering of the lines (in different browsers on different platforms), especially if we generate a PNG image or if we generate a PDF file and print it from the PDF file. Basically there are two types of lines (HTML and SVG) and three types of representations - the first on the screen in the browser, the second on the screen in a PDF reader, and the third - on paper after printing. It will be cumbersome or even impossible to keep the appearance of the two types of lines consistent in all three types of representations (at different scale factors and different browser and PDF engines).

Since you control the HTML , you can add an SVG line too, but yeah, it becomes more and more cumbersome

We fully understand that you don’t have time at the moment. Nevertheless, would it be theoretically conceivable for you to provide this feature? Not now, but at a later point in time - sometime in the future?

Maybe part of the the #236, but not as a separate feature

Thanks David! We are grateful for any kind of help!
I'll try a few things out over the course of this week and post the results here at the end.

Hello David,
I apologize for writing so late..
It wasn't so bad that a long time passed because it gave me a lot of time to think and I realized that there are several types of compactness.
That's why I suggest another, third parameter that determines the type of compactness, I'll call it "compactDensityLeaves".
compactDensityLeaves == "low": Leaf nodes are only grouped if they have no siblings that themselves have child nodes. The grouping occurs multiple times. The sequential arrangement is maintained naturally. See screenshot "Compact 1 group density LOW".
compactDensityLeaves == "medium": Leaf nodes are always grouped, with multiple groupings occurring under the same parent node if there are other non-leaf nodes (that have child nodes) between the leaf nodes. This explicitly preserves the sequential order. See screenshot "Compact 1 group density MEDIUM".
compactDensityLeaves == "high": Leaf nodes are always grouped, in a single group per parent node. Essentially this is the "density" of the current compact mode of the OrgChart library. I noticed that the current compact mode of the OrgChart library sometimes groups "at the front" and sometimes "at the end". In my experiment, the leaf nodes are always grouped at the front. See screenshot "Compact 1 group density HIGH"
It is conceivable to have another parameter for controlling the compactness of parent nodes, as in #236 - for example called "compactDensityParents"..
The playground: https://stackblitz.com/edit/js-5mpjgf?file=index.html
Thanks for your tips! They are incorporated, also the number of child nodes in the expand/collapse buttons now takes into account the number of children under the artificial nodes. It is clearly visible that grouping in a single group can reduce the overall width compared to two groups, see screenshots "Comparison".
Summary
New parameters (just as an example, can be named and organized completely differently):

new parameter compactGroupsCount with value range 1 or 2, default value is 2; relevant only if parameter compact == true (compact mode)

new parameter compactGroupsLayout with value range "top_right" or "bottom_left", relevant only if parameter compactGroupsCount == 1; states whether the one single compact group is on the left/right side, in the "top" or "bottom" layout, or top/bottom in the "left" or "right" layout; Relevant if you or a contributor manages to draw the svg lines like in the screenshot "Final result".

new parameter compactDensityLeaves with value range "low", "medium" and "high"

This would be a non-breaking enhancement, since

if compact == false then the normal classic view is displayed
if compact == true, compactGroupsCount == 2 and compactDensityLeaves == "high", then this would be the current compact mode

Screenshots
Compact 1 group density LOW

Compact 1 group density MEDIUM

Compact 1 group density HIGH

Final result
(if someone manages to draw the svg lines)

If you - the READER - are also interested in this stuff, then please leave a short comment "I'm interested too" so that we know that I'm not the only one on the planet who wants more layouts.
If you - the reader - have further ideas, please write about them here.
If anyone is an expert in svg and would like to contribute, please feel free to fork https://stackblitz.com/edit/js-5mpjgf?file=index.html and try to draw the svg lines as shown in the screenshot "Final result" above.

If you - the READER - are also interested in this stuff, then please leave a short comment "I'm interested too" so that we know that I'm not the only one on the planet who wants more layouts.
If you - the reader - have further ideas, please write about them here.
If anyone is an expert in svg and would like to contribute, please feel free to fork https://stackblitz.com/edit/js-5mpjgf?file=index.html and try to draw the svg lines as shown in the screenshot "Final result" above.

I'm interested too

Thank you milhlhat for your comment!
Do you have any ideas for improvement or further suggestions?
The ultimate goal here would be to be "data driven", such as the "hybrid" and "compact" data properties in the dabeng-OrgChart:
https://github.com/dabeng/OrgChart
However, this would be too much at once. A first step would be a global change of the layout through something like the parameters suggested by me. And if in the distant future one were to also add "data driven" parameters, this would be a "non-breaking enhancement," as the global parameters would act as defaults, and if other parameters for specific nodes were provided through the data, the data parameters would override the global ones for the specific nodes (specific overrides general).

This would be an amazing feature for things like organizational charts in companies with more than 30 employees or overall flat hierachies!


kubernetes security fixes:

added DD_SECRET_KEY in all django containers

Django SECRET_KEY get it's value from environment variable DD_SECRET_KEY or defaults to weak and known "." string as per settings.dist.py
DD_SECRET_KEY  environment variable was injected in the initializer container but not the other ones. I believe it's required for all django containers : more importantly uwsgi, but celery beat and celery worker too
As per django documentation https://docs.djangoproject.com/en/2.2/ref/settings/#secret-key :
Running Django with a known SECRET_KEY defeats many of Django’s security protections, and can lead to privilege escalation and remote code execution vulnerabilities.

added DD_CREDENTIAL_AES_256_KEY in uwsgi

DB_KEY get it's value from the environment variable DD_CREDENTIAL_AES_256_KEY or defaults to weak and known "." string as per settings.dist.py
DD_CREDENTIAL_AES_256_KEY was missing in the uwsgi container. It was injected only in the initializer.
I believe it's required (only) for uwsgi as it's used in the templating engine (see utils.py dojo_crypto_encrypt function, see dojo\templates\dojo\view_cred.html)
I've left it in the initializer "just in case"; all the secrets from the defectdojo secret are injected into the initializer:
        envFrom:
(...)
        - secretRef:
            name: {{ $fullName }}

increased DD_SECRET_KEY size

As per django advisor, this key should be at least 50 char long:
$ python manage.py check --deploy
WARNINGS:
(...)
?: (security.W009) Your SECRET_KEY has less than 50 characters or less than 5 unique characters. Please generate a long and random SECRET_KEY, otherwise many of Django's security-critical features will be vulnerable to attack.

add secure flag to session cookies when TLS is activated

when TLS is activated (either at ingress level or end-to-end TLS until the nginx container), set the secure flag to our cookies using SESSION_COOKIE_SECURE=True and CSRF_COOKIE_SECURE=True django variables
Note: you can check the instantiated values of all settings.py parameters using:
python manage.py diffsettings --all

(see https://docs.djangoproject.com/en/2.2/ref/django-admin/#diffsettings)

good news is: helm upgrade seems to seamlessly add the missing variables into the containers, restarts the pods, and doesn't break anything so far

@mtesauro OK thanks for the tip. Increasing to 128 can't hurt I can do that for sure! However I think randAlphaNum is fine if you don't mind:

It produces something equivalent (uses 0-9a-zA-Z; see https://github.com/Masterminds/sprig/blob/master/docs/strings.md#randalphanum-randalpha-randnumeric-and-randascii )
it's faster to write and read, being helm native
on the crypto side I think it's equivalent or better as a PNRG depending on the host : cf https://golang.org/pkg/crypto/rand/ . It's using either /dev/urandom or some other first-choice method, depending on the OS.

On Linux and FreeBSD, Reader uses getrandom(2) if available, /dev/urandom otherwise. On OpenBSD, Reader uses getentropy(2). On other Unix-like systems, Reader reads from /dev/urandom. On Windows systems, Reader uses the CryptGenRandom API. On Wasm, Reader uses the Web Crypto API.

@alles-klar  did you have a chance to have a look at this? thanks!

@ptrovatelli Alphanumeric is fine - here's what we've done historically in setup.bash for comparison - 128 characters restricted to a-zA-z0-9:
https://github.com/DefectDojo/django-DefectDojo/blob/master/setup/scripts/common/prompt.sh#L32


Do we want explicit instances of owl:Thing

Release 12 has explicit instances of owl:Thing for base unit instances, e.g.
gist:_USDollar
	a
		owl:Thing ,
		gist:BaseUnit
		;
	rdfs:isDefinedBy <https://w3id.org/semanticarts/ontology/gistCore> ;
	skos:definition "The base unit for currency."^^xsd:string ;
	skos:prefLabel "US Dollar"^^xsd:string ;
	gist:conversionFactor "1e+00"^^xsd:double ;
	gist:hasBaseUnit gist:_USDollar ;
	gist:unitSymbol "USD"^^xsd:string ;
	.

See also #775 & #783 for previous comments.

It would be good to ask the EDM Council to add another option to the serializer to remove them.

I just did the following to see where they are coming from.

start with gistCore12.0.0.ttl that downloaded from SA website
manually removed the 12 triples making instances of owl:Thing
loaded it into protege
Immediately saved it out from protege
Ran it through the serializer

RESULT: no explicit instances of owl:Thing, so neither Protege nor the serializer seem to be the problem.
Maybe just remove them and it's done?

I did not think the serializer was inserting them, but we wanted them removed if they are there. If Protege isn't putting them in, then it's not clear where they are coming from. We should ask if anyone is putting them in and make it clear in the style guide that they should not be used.

Added to agenda for next week.

Given the current behavior of onto_tool, we will not get rdfs:isDefinedBy assertions without owl:Thing assertions. (See links above.) Until onto_tool is fixed we shouldn't remove these.


Pip install argo-workflows using git issues warning and deprecation messages

Summary
Under Macosx Catalina, running under Python 3.9 / MiniConda, whenever installing argo-workflows (master) through pip, will issue warning and deprecation messages
Diagnostics
 pip install git+https://github.com/argoproj/argo-workflows@master#subdirectory=sdks/python/client

Collecting git+https://github.com/argoproj/argo-workflows@master#subdirectory=sdks/python/client
  Cloning https://github.com/argoproj/argo-workflows (to revision master) to /private/var/folders/90/2zp9njmn29g3ztv7drth0mzr0000gp/T/pip-req-build-fi5ycr7z
  Running command git clone --filter=blob:none -q https://github.com/argoproj/argo-workflows /private/var/folders/90/2zp9njmn29g3ztv7drth0mzr0000gp/T/pip-req-build-fi5ycr7z
  Resolved https://github.com/argoproj/argo-workflows to commit 318927ed6356d10c73fe775790b7765ea17480d4
  Preparing metadata (setup.py) ... done
Requirement already satisfied: urllib3>=1.25.3 in ./Miniconda3/lib/python3.9/site-packages (from argo-workflows===0.0.0-latest) (1.26.7)
Requirement already satisfied: python-dateutil in ./Miniconda3/lib/python3.9/site-packages (from argo-workflows===0.0.0-latest) (2.8.2)
Requirement already satisfied: six>=1.5 in ./Miniconda3/lib/python3.9/site-packages (from python-dateutil->argo-workflows===0.0.0-latest) (1.15.0)
Building wheels for collected packages: argo-workflows
  Building wheel for argo-workflows (setup.py) ... done
  Created wheel for argo-workflows: filename=argo_workflows-0.0.0_latest-py3-none-any.whl size=1257572 sha256=3fc84e987432f406930baa64b0145a9eb3a1be3ec490d42d16cefc3f0897de2e
  Stored in directory: /private/var/folders/90/2zp9njmn29g3ztv7drth0mzr0000gp/T/pip-ephem-wheel-cache-z7etsidr/wheels/23/e1/4c/a0bde4c731c604276076340e583cd0020a69a8419380f835ee
  WARNING: Built wheel for argo-workflows is invalid: Metadata 1.2 mandates PEP 440 version, but '0.0.0-latest' is not
Failed to build argo-workflows
Installing collected packages: argo-workflows
    Running setup.py install for argo-workflows ... done
  DEPRECATION: argo-workflows was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. Discussion can be found at https://github.com/pypa/pip/issues/8368
Successfully installed argo-workflows-0.0.0-latest

@terrytangyuan can you take  a  look?

I think you can ignore these warnings. Since the package can still be successfully installed, you can use it. Once this is officially released to PyPI, you'll be able to pip install argo-workflows and the warnings will go away since the version will be valid.

In the meantime, I'll also update the version number to be PEP compatible.

@davelajoie This warning will be fixed by https://github.com/argoproj/argo-workflows/pull/7525. Feel free to try again once it's merged.

Tx
I was under impression that pip install argo-workflows would install the old client api and not new SDK. I will test this once merge

Tx for the fix!


Fix syntax error FIELD=$eq.VALUE to FIELD:$eq.VALUE

Found that using FIELD=$eq.VALUE is invalid

I don't know why it went wrong, just changed a little, please advise

Haha, sorry, I understand it wrong.

The curl command under linux should use \ in front of $.


Initialization of the global tracer

Hey @mikebryant
I was checking the initialization of the global tracer, and I'm not sure this is the right approach. Specifically this is the part I'm talking about:
        if hasattr(settings, 'OPENTRACING_TRACER'):
            # Backwards compatibility with the old way of defining the tracer
            opentracing.tracer = settings.OPENTRACING_TRACER._tracer

Usually, they way we have done things for other instrumentation in Java, is to do it the other way around: if the user doesn't specify explicitly a tracer (through OPENTRACING_TRACER or OPENTRACING_TRACER_CALLABLE in this case), then fallback to opentracing.tracer).
Setting the opentracing.tracer instance right from python-django kinda breaks the contract, and I'm afraid that the user won't be able to set the global tracer himself if we do this - and also, I'm afraid other instrumentation libraries may also want to set opentracing.tracer each, and we end up in an override cascade.
Is there any need from your part to have this code running? Let me know, and maybe we can find a solution in the middle ;)

Fixed by #29
@mikebryant Let me know if you have any opinion or feedback on this ;)

You need to be able to have that happen lazily, after forking has happened (see all the issues with Jaeger)

Not really. I went through https://github.com/jaegertracing/jaeger-client-python/issues and didn't manage to find a related one...

Have you got an example of being able to use Jaeger with Django under mod_wsgi using the approach you're proposing?

So I haven't set up Django + mod_wsgi myself, but some (production) cases I have found work by  putting all the parameters needed for initializing your tracer right in settings.py:
# settings.py

TRACER_KEY = "dasd7ad87a"
TRACER_SERVICE = "DjangoServer01"
TRACER = MyTracerImpl(TRACER_KEY, TRACER_SERVICE) # create specific tracer implementation
opentracing.tracer = TRACER

...

OPENTRACING_TRACER = DjangoTracer(opentracing.tracer)

One alternative I see nevertheless is providing, based on very Django's needs, a callback at DjangoTracer set-up time:
OPENTRACING_TRACER_INIT_CB = my_init_cb

# In some place:
def my_init_cb(django_tracer):
   opentracing.tracer = django_tracer._tracer

Would that work?

@mikebryant Guess you were talking about #14 ? ;)

I believe the issue was https://github.com/jaegertracing/jaeger-client-python/issues/60, leading to deadlocks when the tracer initialisation was done in settings.py
The problem as I recall is needing to start the event loop after forking, which can't be done in settings.py, as that's read pre-fork.
My requirements are fairly simple, I want to be able to have static Django settings that just work, and I don't really want to have to stick python code somewhere to set up the global tracer object.

I want to be able to use opentracing.tracer inside python libraries, so I can inject spans and so on. I don't want to have to plumb a tracer object throughout every single function call I ever make, so I need to use the global for that.
I want to be able to use python-django to set up my global tracer, so I can configure it in one place. I won't want to need to write my own lazy middlewhere to wait until the forking has happened, and then set up the tracer object.

I want to be able to use python-django to set up my global tracer, so I can configure it in one place.

Sadly this could be requested in any instrumentation library (gevent, asyncio, tornado, sqlalchemy, you name it).  This wouldn't scale. And in Java, C# and Javascript the duty of the developer is to take the extra steps to write code to set the global Tracer and consume it later on. And nobody had complained so far.
In any case, I think at least I'd like to have this as an optional OPENTRACING_SET_GLOBAL_TRACER value (False by default) - nobody I know has requested/asked this feature -and personally I'm not sure whether it's correct or not-, but if it is needed in some scenarios, we can definitely provide it.
I will cook re-work so initializing this one is at least optional.

I don't mind having it hidden behind a setting.
I can put this another way, we have many django projects deployed, and I don't want to write code repeatedly. The last time I tried setting this up, I found that I couldn't instantiate the Jaeger tracing object in settings.py without deadlocking the system. And writing my own lazy initialization middleware in every project isn't optimal, so I want to push it back upstream, so there's a simple way of making Jaeger work.
As long as it's simple and obvious to set up this library in a way that satisfies the following requirements, I'll be happy

The global tracer is set up correctly
The django internals are set up correctly

(Not saying the library must do both of those in all cases, but I don't want to need to write more than a few lines of code in every single project to achieve this

#30 should do it. @mikebryant take a look whenever you have time, please.

Closing as #30 handles this issue by making this bit optional.


Missing headers for version 0.77?

Comment:
@saraedum @isuruf It seems like some of the headers are missing when bliss 0.77 is installed by conda. For example, the header bliss_C.h is not installed in micromamba/envs/digraph/include/bliss/ but it exists in the release tarball of bliss.
This means we can no longer use the conda version of bliss in the Digraphs package for GAP, as per:
https://github.com/digraphs/Digraphs/actions/runs/7422806823/job/20198886373?pr=597
Maybe I'm doing something wrong, happy to be told about it if I am.

This was fixed in #7.


Overview Fixes a bug that when using uvicorn and nginx with an upstream server via a unix socket results in a TypeError.

Stack trace:
[2021-02-14 05:24:40 +0000] [21] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/venv/lib/python3.8/site-packages/newrelic/api/asgi_application.py", line 332, in nr_async_asgi
    with ASGIWebTransaction(
  File "/venv/lib/python3.8/site-packages/newrelic/api/asgi_application.py", line 229, in __init__
    host, port = scope["server"] = tuple(scope["server"])
TypeError: 'NoneType' object is not iterable
 - "GET / HTTP/1.0" 500
Exception ignored in: <function Transaction.__del__ at 0x7f020a19e1f0>
Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/newrelic/api/transaction.py", line 308, in __del__
    if self._state == self.STATE_RUNNING:
AttributeError: 'ASGIWebTransaction' object has no attribute '_state'

Uvicorn will set the asgi scope server property to None if using a unix socket.
See: https://github.com/encode/uvicorn/blob/bd1a0962d36c66e823e13012df14a0fe7e36cc9c/uvicorn/protocols/http/httptools_impl.py#L133
The newrelic agent assumes this value is a tuple, and tries to split it into two variables for host and port if it is defined.
The fix is to check that the value is not None.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.James McDonnell seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


FilterHandler: fix handle empty batch of filtered records

This fixes https://github.com/Seldaek/monolog/issues/1401

Thanks


bugfix: make nat_rule_group work with panorama and ngfw

Description
Based on documentation nat_rule_group should work either with Panorama and NGFW, right now it not work as intended.

quick fix for ike_crypto_profile_tests.

Motivation and Context
It fix nat_rule_group resource to behave what was said in documentation.
How Has This Been Tested?
I build provider, test it with Terraform code and do go tests.
Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist

[x] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes if appropriate.
[x] All new and existing tests passed.

Shocking that this PR has been open since 2022 and nobody has bothered to merge or whatever with it
One has to wonder if Palo is actually bothered about TF support, and if this repo is still alive?


[Q]:  Is it possible to receive alerts when the training process becomes stuck?

Ask your question
Is it possible to receive alerts like email, when the training process becomes stuck.
If yes, how. Thanks a lot.

Hi @ericxsun , yes, this is possible via wandb.alert, you can use the multiprocessing module to run a separate thread or process (watchdog in the example below) that monitors the main training loop. If the training loop fails to report progress within a specified time (timeout in the example below), this thread/process assumes the script has hung and triggers an alert (wandb.alert(...)).
Here's an example script to achieve the same:
import wandb
import time
import multiprocessing
from multiprocessing import Process, Value
import os

PROJECT = "your_project_name"
ENTITY = "your_entity"
NUM_EPOCHS = 10

def watchdog(timeout, last_update, run):
    try:
        while True:
            time.sleep(timeout / 2)
            with last_update.get_lock():
                time_since_update = time.time() - last_update.value
            if time_since_update > timeout:
                # Trigger wandb.alert
                run.alert(
                    title="Training Hang Detected",
                    text=f"No progress for {int(time_since_update)} seconds.",
                    level=wandb.AlertLevel.ERROR
                )
                # Terminate the process group
                os.kill(os.getppid(), 9)
                break
    finally:
        run.finish()

def cleanup(watchdog_process):
    if watchdog_process.is_alive():
        watchdog_process.terminate()
        watchdog_process.join(timeout=5)
        if watchdog_process.is_alive():
            watchdog_process.kill()

def main():
    run = wandb.init(entity=ENTITY, project=PROJECT)

    # Shared value to track progress
    last_update = Value('d', time.time())

    # Start the watchdog process
    timeout_seconds = 80  # Set the timeout duration
    watchdog_process = Process(target=watchdog, args=(timeout_seconds, last_update, run))
    watchdog_process.start()

    try:
        # Simulate training loop
        for epoch in range(NUM_EPOCHS):
            # Update the progress tracker
            with last_update.get_lock():
                last_update.value = time.time()

            train_loss = 0.1 * epoch
            wandb.log({"train_loss": train_loss, "epoch": epoch})
            # Simulate work by sleeping
            time.sleep(5)

            # Simulate a hang after epoch 2
            if epoch == 2:
                print("Simulating a hang...")
                while True:
                    time.sleep(1)

    except Exception as e:
        wandb.log({"error": str(e)})
        wandb.alert(
            title="Training Error",
            text=f"An error occurred: {e}",
            level=wandb.AlertLevel.ERROR
        )

    finally:
        run.finish()
        cleanup(watchdog_process)

if __name__ == '__main__':
    multiprocessing.freeze_support()
    main()

This script will lead to an email being sent to you with the content as follows:

Note: You do need to turn on the alert from your W&B user settings as detailed in our docs.
Please let me know if the above helps.

Thanks so much @anmolmann

Happy to help, @ericxsun !


$exposeAside should return list of open side menus instead of active property if any menu is open

Should be very handy of you have multiple side-menu toggle buttons with multiple side menus
(My situation being a left and right menu button toggle and left and right side menus)
Maybe an activeMenus property with id or delegate name?

Greetings, @josdewitte!
My sensors indicate that this issue has been inactive for a while.
Due to this, the issue will be closed in 7 days.
If you believe the issue has not been fixed, please add a comment.
This will notify us so we can assist you, and will prevent the issue from being automatically closed.
Feel free to include the following information about the issue, so that we may assist you:

Can you provide a detailed description of this issue?
How would a robot go about replicating this issue?
What platform does this happen on? (iOS, Android, all, etc.)
What platform versions does this happen on? (Android 4.2, iOS 7, all, etc.)
Is this an issue with the Ionic Framework, or Ionic CLI?

Thank you for allowing me to assist you.

This issue has not been fixed! Ionitron! It seems the code is using the inner witdth of the content area instead of outer element width that contains the side menus and content when using > 1 menu.

Greetings @josdewitte!
My sensors indicate that I require more information in order to assist you.

Can you provide a detailed description of this issue?
How would a robot go about replicating this issue?
On which platform does this happen occur? (iOS, Android, all, etc.)
On which platform versions does this occur? (Android 4.2, iOS 8, all, etc.)
Is this an issue with the Ionic Framework or Ionic CLI?

Additionally, providing a codepen that replicates the issue is extremely helpful.
If you wish to get help using the framework itself, I recommended visiting the forum, where many humans will be able to offer wisdom and advice.
Thank you for allowing me to assist you.


Validate performance on Android One / old Samsung devices

 Issue by abarth
Wednesday Oct 14, 2015 at 00:11 GMT
Originally opened as https://github.com/flutter/engine/issues/1607

We should make sure our performance scales down to low-end devices.

We are now keeping track of device/gpu vendor specific issues in their own tickets. Closing. If there is a particular low end device we want to validate, we can open another ticket with specifics.