stage1: use systemd from host /usr

On hosts like CoreOS where we already have a copy of systemd that is up to date we should use that instead of relying on the default stage1 ACI version. So, we should have some new stage1 ACI option that we build that bind mounts in the /usr from the host and uses that instead of providing a copy of systemd in /usr.
This will help people packaging rkt for other OSes too.

:+1:

Do you suggest to bind mount the directory /usr or bind mount specific files such as /bin/systemd and its dependencies like libselinux.so.1?
If /usr from the host is bind mounted in the container, that gives the container access to a lot of things on the host. The application is chrooted but chroot is not proper isolation and the application can easily escape the chroot to get files from stage1 (including the /usr bind mount).
There has been some discussions about creating the stage1 aci at run-time:
https://bugzilla.redhat.com/show_bug.cgi?id=1169966#c23
So that when systemd is updated on the host, Rocket would automatically pick up the new version.
Maybe a Rocket package could contain things like diagexec, enter, init in /usr/share/rocket/stage1/ but no actual stage1.aci.
And when the user runs rkt run ..., it will render the stage1 rootfs on the fly with a combination of Rocket files from /usr/share/rocket/stage1/... and host files like /bin/systemd, /bin/sleep etc.

/cc @mattthias.

@alban Bind mounting in /usr seems like the quickest path to me and having a specific bind mount list would be a nice optimization overtime but I don't know. I don't really care which way we go.
Can we use overlayfs to take the "partial" stage1 and mount the /usr volume below it and then just store that into /usr/share/rocket/stage1 or whatever?
/cc @marineam

@alban: the idea was to make use of the host system, and in CoreOS the whole system is under /usr so there isn't any need to worry about different things being installed in /lib vs /usr/lib. Systems where /usr doesn't include everything would complicate matters. To cover the case of getting updates from the host distro into the stage1 aci perhaps we are better served by making it easy to build and ship distro-specific stage1s.
@philips I discussed this  with @vcaputo and he was reluctant to tie stage1 to CoreOS in the long term. For example if stage1 would benefit from a new nspawn but we have to hold back the systemd version in CoreOS because say networkd was buggy and segfaulting we would be holding back the rocket project needlessly. I have to agree, rocket is still a super young project and we need to help it move faster, not tie it down.
Instead we could make better use of the "stage1 is an aci" model and split stage1 from the rocket repo to allow rocket to use different stage1 implementations. These may include:

The existing stage1 build based on CoreOS binary images.
A  new stage1 built natively in the CoreOS SDK as a special target (a.k.a "board").
Alternatives built by/for other distros who need to fit rocket into their own build system and packaging policies.
Somewhere between the last two: a generic stage1 that isn't based on a particular distro but builds systemd and some or all of its dependencies from source. This could be used by developers or other distros who simply need to be able to build stage1 from source but the CoreOS SDK would be crazy overkill.
Experimental builds that prototype new features or designs: capsule, network designs that don't yet work with nspawn, alternate environments like LXC, QEMU, and beyond!

Once we figure out how to swap out aci based stage1 builds safely then it may be possible to dissolve some of the host-stage1 separation to make use of the host system's systemd/nspawn/etc instead of a pre-built aci. For the short term we are better served by focusing on using acis than trying to save a few MB in the CoreOS image.
Closing this issue and will open a new one for splitting out the stage1 build process from rocket.

I like the idea that installations of Rocket could build a stage1.aci at install time or so based on the host operating system.


organizeImports deletes modules in use

Issue Type: Bug
Expected Result:
import babel from "./babelrc";
export = { babel };

Actual Result:
export = { babel };

Steps to reproduce

Add below setting to VSCode

"editor.codeActionsOnSave": {
     "source.organizeImports": true
}

Paste code below into editor

import babel from "./babelrc";
export = { babel };

Save file and see import dissapeared.

VS Code version: Code 1.23.0 (7c7da59c2333a1306c41e6e7b68d7f0caa7b3d45, 2018-05-03T15:23:14.634Z)
OS version: Darwin x64 17.4.0
Extensions (30)

Extension
Author (truncated)
Version

bracket-pair-colorizer
Coe
1.0.47

vscode-generate-getter-setter
DSK
0.5.0

EditorConfig
Edi
0.12.1

material-icon-theme
PKi
3.3.0

code-settings-sync
Sha
2.9.2

pegjs-language
Sir
0.1.0

quokka-vscode
Wal
1.0.121

vscode-node-readme
ben
3.0.2

handlebars-preview
cha
1.1.0

npm-intellisense
chr
1.3.0

path-intellisense
chr
1.4.2

vscode-eslint
dba
1.4.8

vscode-dash
dee
1.10.0

tslint
eg2
1.0.28

vscode-npm-script
eg2
0.3.4

vscode-great-icons
emm
2.1.31

prettier-vscode
esb
1.3.1

nearley
kar
1.0.2

dotenv
mik
1.0.1

theme-monokai-pro-vscode
mon
1.1.6

debugger-for-chrome
msj
4.3.0

chai-snippets
nwh
0.1.0

vscode-code-outline
pat
0.1.1

LiveServer
rit
3.2.1

vscode-icons
rob
7.23.0

markdown-preview-enhanced
shd
0.3.5

autoimport
ste
1.5.3

open-in-browser
tec
1.1.0

code-navigation
vik
0.2.0

better-align
wwm
1.1.6

(4 theme extensions excluded)

This is the same issue I am experiencing. The setting does not seem to be respecting the ES6 object literal property value shorthand.
If you need a temporary workaround you can define the object like so:
import babel from "./babelrc";
export = { babel: babel };

and then the organizeImports setting will function as expected.

Duplicate of https://github.com/Microsoft/TypeScript/issues/23308
Should be fixed in insiders

After today's update of VSCode, organizeImports worked on most of my .jsx files except for:
import faPlay from '@fortawesome/fontawesome-free-solid/faPlay'
import faPause from '@fortawesome/fontawesome-free-solid/faPause'
import faStop from '@fortawesome/fontawesome-free-solid/faStop'
import faStepForward from '@fortawesome/fontawesome-free-solid/faStepForward'
import faStepBackward from '@fortawesome/fontawesome-free-solid/faStepBackward'
import faEject from '@fortawesome/fontawesome-free-solid/faEject'
import faExpandArrowsAlt from '@fortawesome/fontawesome-free-solid/faExpandArrowsAlt'

/**
 * @see https://github.com/FortAwesome/react-fontawesome#build-a-library-to-reference-icons-throughout-your-app-more-conveniently
 */
export {
  faPlay as Play,
  faPause as Pause,
  faStop as Stop,
  faStepBackward as StepBackward,
  faStepForward as StepFoward,
  faEject as Eject,
  faExpandArrowsAlt as FullScreen,
}

VS Code version: Code 1.23.1 (d0182c3417d225529c6d5ad24b7572815d0de9ac, 2018-05-10T16:04:33.747Z)
OS version: Linux x64 4.15.0-20-generic

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz (4 x 2556)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: disabled_softwarevideo_decode: unavailable_softwarevideo_encode: enabledvpx_decode: unavailable_softwarewebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 1

Memory (System)
15.39GB (2.25GB free)

Process Argv
/usr/share/code/code --unity-launch

Screen Reader
no

VM
0%

Extensions (26)

Extension
Author (truncated)
Version

vscode-toggle-quotes
Bri
0.3.1

bracket-pair-colorizer
Coe
1.0.48

vscode-markdownlint
Dav
0.15.0

EditorConfig
Edi
0.12.1

code-settings-sync
Sha
2.9.2

project-manager
ale
0.25.2

vscode-custom-css
be5
2.7.0

npm-intellisense
chr
1.3.0

path-intellisense
chr
1.4.2

mustache
daw
1.1.1

jshint
dba
0.10.18

githistory
don
0.4.0

eslint-disable-snippets
drK
1.1.4

gitlens
eam
8.2.4

tslint
eg2
1.0.28

vscode-npm-script
eg2
0.3.4

prettier-vscode
esb
1.3.1

auto-close-tag
for
0.5.6

debugger-for-chrome
msj
4.4.3

vscode-code-outline
pat
0.2.0

vscode-icons
rob
7.23.0

vscode-fileutils
sle
2.8.1

es6-mocha-snippets
spo
0.1.3

addDocComments
ste
0.0.8

vscode-import-cost
wix
2.6.2

ReactSnippets
xab
2.1.0

(2 theme extensions excluded)


Improved and post-mortem monitoring

Improve the online monitoring to:

Show session list summary with more info
Show more statistics and graphs, e.g. aggregation by task type or group (proposed in #32)
Show a timeline by groups or with significant (all or e.g. named?) events.
Only display the dependency graph optionally and for reasonably small graphs

As a future improvement, we could display the dependency graph of groups, clusters, task types etc.)

See the full details of every task and object (on demand)
Examine history of the events in some way (a timeline may be enough)
Do all of this for finished and closed sessions via re-running the server with the sa

This depends on a streaming API (#11) and may require a complete UI framework.

I would also note that, maybe the protocol will be changed later, but type of events are more or less fixed and do not see any major flow in design. However, the user interface (JS side) is mostly proof-of-concept, so if  someone has feeling that it needs a major revamp then I (as the author of the current dashboard) have nothing against it. Let me also remind that setup for developing dashboard is described in docs.

I would like to help here.
I believe there will be a bunch of PR's to get the features listed here. I could start with the session summary. Any special information, which should be included there? I believe something like session duration, processed tasks, status, right?

@Trojan295 Great! You are right, basic summaries like task durations, number of submits, IP of client, number of tasks/object are welcome. The current dashboard now supports only the chart of unprocessed tasks in time.
It would be also nice to separate the session page into "summary" and "detail". The full task-object dependency graph for large graphs is slow and usually not useful, so it would be nice to show summary of session without showing this graph.

We have made some progress with Dashboard. @Kobzol migrate whole dashboard into TypeScript and we have added task inspector and basic task summary.


Added support to utf-8 encoded unicode

I fixed a little bug that caused an error when serializing a unicode object with utf-8 characters.
Consider the following code:
schema = '["string"]'
serializer = pyavroc.AvroSerializer(schema)
serializer.serialize(u"barà")

Before the correction it would have raised a TypeError: function takes exactly 5 arguments (1 given). It failed because PyString_AsStringAndSize returned -1.
Now unicode objects are converted in UTF-8 strings before getting serialized.

Merged. Thanks for the code.


Make  GitHub repositories public by go live date

@gs-bracej - I'd like to propose a reset of Git logs for each module, so that, before switching the repositories to public, each repository shows one commit (with the same git message), authored by the module leader.

For Studio, it would be An
For SDLC, if would be Kevin
For Shared, it would be you

Right after that, I'd like to enforce branch protection, which will also prevent force pushes and git history hacks.
What do you think?

Sounds perfect to me!

done on Oct 19th


Fix COG to be in degrees and add heading true to position report

what would happen here if headingTrue is not available? Would the function get called with a null value or never be called?

Output requires at least one update from each input. The implementation is also naive: it will always use the latest values, even if some of them have stopped updating.


Develop a R̶o̶a̶d̶m̶a̶p̶ Wishlist

Hello All,
I'd like to help dask develop a roadmap, and @mrocklin requested that I open an issue here to kickstart the discussion.
For those of you unfamiliar, a roadmap is a listing of near- and long-term goals that the project has, but which have yet to be implemented. These goals are generally larger than single pull request. "Add GPU Support" or "Port to Python 3" are examples of some roadmap items a project might have. However, goals can also include activities that seem small and mundane, but are critical to the project, such as "Improve Documentation" or "Achieve 100% Test Coverage."
The purpose of this issue is so that roadmap items can be listed and commented on in the discussion. Once we have a reasonable sense of what folks would like to see on the roadmap, the contents should make its way to the website/docs in some fashion. It is also a good idea if it ends up being listed in priority order, so that higher priority items are closer to the top.
Some good example of project roadmaps are:

SymPy Roadmap
Bokeh Roadmap

A decent example of a project roadmap is Spyder's Roadmap which looks more like a timeline with milestones.
Roadmaps serve a joint purpose:

They help inform and attract potentail funders, and some funders require roadmap documents
They help attract contributors, as developers are more likely to jump in an help an establish project than start a new project to scratch their itch.

TL;DR: Roadmaps communicate a project's intentions. It would be a good idea for dask to have one, and I am happy to help facilitate!
So where would you like to see dask go?

@mrocklin - yeah, I hear you about the mixed use here.  Option 1 is clearly a contractual roadmap, "We plan to do X. Time, materials, and hands speed up the process of achieving X."  Option2 is more of a wishlist roadmap, "Given time, materials, and hands, we'd love to do X and are currently accepting contributions."
I think terms like "wishlist," "prospectus", "stretch goals," and "desired milestones" all fit the bill for option 2, if you would like alternative verbiage.
However, I am not exclusively describing option 2. I believe, that for most projects, it is tempting for the two cases you have described to make their way into the same document. This is because folks approach the roadmap question as, "What would I like the project to look like if I had infinite resources?" So every crazy idea that has ever popped into the developers heads get listed initially. These infitine resource ideas tend to get weeded out (or at least pushed down) during prioritization.
Rather, I am suggesting whatever the you all think is approriate for a roadmap document. If you believe it should be more contractual, then option 1 alone seems likt the correct path. If you'd like some of the more whishful ideas to make it in, they can certainly be included either:
a. Intermixed with the contractual items, or
b. Clearly deliniated into their own section so as not to be confused.
These roadmaps are reflections of the projects and the authors, and so it is really up to the developers as to how structure such documents.
In the situation where there are both contractual and wishlist elements, I am personally less interested in seeing the wishlist side of things. But that doesn't mean that you should or should not add them. There aren't really wrong answers here!

To be clear, I don't think that this project has the ability to write a contractual roadmap.  This assumes that the project has dedicated long-term resources and autonomy, which it doesn't.
Currently I answer "what is your roadmap?" questions by saying what current funding sources and what current volunteer developer communities care about.

I am personally less interested in seeing the wishlist side of things

Interesting, I was actually assuming that you were asking for the wishlist side of things.
If I were to write such a document on my own (which I have no particular desire to do) I would probably phrase it as a set of projects that are well-scoped and that people could take on in finite time.  This would likely be somewhat similar to this document for summer internship projects though aimed towards larger endeavors.

Yeah, roadmap to me implies a schedule, a sequence of milestones, or list of priorities.  Interestingly, this is not what the SymPy and Bokeh roadmaps that you linked are like.  They are more like strategic goals.  They could be perceived as what the primary developers intend to work on.
For dask, it may be helpful to list or tag outstanding PRs and issues that have the most strategic importance, but are not actively being worked on.  For example multi-index for dask.dataframe.  This has the advantage of using existing procedures (github issues and tags), and may more-or-less satisfy the two purposes @scopatz listed.
Perhaps adding a "Vision" section to the docs would be more useful than adding a "Roadmap" section.  However, I'm not sure how different this would be from the "Why dask?" section.
Pieter Hintjens' take on roadmaps for ZeroMQ is pretty interesting:
http://zguide.zeromq.org/page:all#How-ZeroMQ-Lost-Its-Road-Map

@eriknw Thanks for the link! I think the issue with ZeroMQ is that they attached dates to the roadmap item. I don't think it is a good idea to promise both features (or other deliverables) and dates when they will be delivered, for exactly the reason Pieter Hintjen says. An all volunteer, open source workforce is simply not reliable enough.
A "Vision" section would be great.  I do think this is distinct from the "Why Dask?" section, which tries to answer, "Why dask now?" A vision section would instead answer, "What is the future of dask, and how can I help improve it (via code, funds, etc)?"

Pinging this! Are there still questions or concerns about the format or structure that I can help answer? FWIW, I think that what Dask-ML has done for their roadmap has hit the nail on the head.

I think that my concerns from https://github.com/dask/dask/issues/3589#issuecomment-396321758 still hold.  From the discussion above I think that there is still some confusion about what you're trying to accomplish.  My original guess was that your direct mandate was the following:

Find features that some other company or funding agency might see and request to be implemented, so that a company like QuanSight might perform that work.

I'm definitely happy to find things like this.  I'm -1 on calling that wishlist of features a roadmap, which I believe offers a promise that we're actually going towards those things.  Your statement that you don't want a wishlist of features leaves me confused about the objective here.
If you're looking for a list of features that Dask developers plan to implement then that list only has a horizon of a few months, and so probably isn't worth keeping up to date.

So I think there are a couple of points of confusion that hopefully I can clear up.

Your statement that you don't want a wishlist of features leaves me confused about the objective here.

First, I would like to see a wish list of features. But in the discussion above we had set up a dichotomy between more readily achivable features (which we'd called contractual) and more blue sky ones (wish list). My claim above was intended to say that I personally am much more interested in more achievable features than less achievable ones. Also, drawing out my desire to see more achievable features as a personal preference was intended to relate that it should have no bearing on what dask chooses to put in a wish list; go full blue sky if you want! This is your wishlist, not mine.

If you're looking for a list of features that Dask developers plan to implement then that list only has a horizon of a few months, and so probably isn't worth keeping up to date.

I think planning items such as this are also fair game, and generally of interest to your user community. These kinds of feature listings I usually consider living documents. To that end, part of the longer term plan is to help update this document every 6 - 12 months. However, if that is not how you want to view this exercise, that is fine too! Again, this is Dask's, and it should fit the needs of the Dask community.

My original guess was that your direct mandate was the following:

Find features that some other company or funding agency might see and request to be implemented, so that a company like QuanSight might perform that work.

So this isn't quite what my mandate is. At this step it really is just the altruistic, "help open source project develop roadmaps / wishlists / prospecti / etc."  This is one of the main reasons I joined Quansight. I believe that open source projects should have roadmaps in the same way that they have licenses, contributor guidlines, governance documents, etc. I think that serve an important and oft overlooked communication function. This should be viewed in the context of open source sustainability.
I also know that these are notoriously difficult to write unless someone external is proding a project to do so. I am definitely personally guilty of not having roadmaps or wishlists for some of the major projects I am involved with (e.g. xonsh, conda-forge). So one of my hats at Quansight is to increase the number of projects out there with roadmaps. I am happy that I now have the oppurtunity to help work on this issue of roadmaps/wishlists more broadly, which has been bothering me for a few years (since SymPy put out thier roadmap).
That said, there is a separate follow-on marketing effort to produce promotional material (free for anyone to use) based off of the roadmap / wishlist. My role in that effort is only to communicate with the developer community and ensure that they have a strong voice in the promotional material that is created. We wouldn't put out anything against the wishes of the project.
I hope this helps clarifies some of the blocking issues here. As always, I am happy to continue answering questions or concerns about the process, goals, or just where I am coming from on this topic!

That does help.

To that end, part of the longer term plan is to help update this document every 6 - 12 months.

To be clear, when I say "we have a horizon of a few months" I think that that's really more like 2-3.  I think that the right way to communicate what people are working on at this time frame is through things like blogposts and meeting minutes rather than roadmap documents.  I don't know what people within the Dask organization are likely to be working on in six months (though they themselves might) so I'm not sure how I personally can make progress here.
Again, I'm happy to talk about "things that people might work on that would maybe be beneficial and interesting" but most of these are going to be external anyway, like "use dask to parallelize geopandas" or "consider how (in)effective a dynamic system like Dask is for numerical linear algebra" but those really depend more on the interests of those involved than on a central authority of the project.
By the way, if any other Dask developers want to join in please do.  I'm hesitant here, but others may have more optimistic thoughts.

Hello again, I am hoping we can reframe this conversation into more of a "wishlist."  That is, a list of features people have requested, including you all as developers.
Even if you could just bullet point the ideas here in this issue, that would be enough for us to move forward.
Thanks! I'd really appreciate it!

dask distributed data access without client:

client doesnot have an access to the filesystem but  scheduler and workers.
this is the case when the data is located in the cluster environment, but client notebook is running on a laptop :)

cluster nodes got  some  local fast nvmes but login node and scheduler not
would be cool to have: worker1-100:file://nvme/data-0-100
I know there are some design problems that client needs to know metadata,number of files etc,but again  this would be a nice way to deal with huge datasets on the remote cluster.

In the context of fsspec, which I have been thinking about a lot lately, I think it would be possible to any filesystems seen by workers as yet another filesystem implementation. This would take a little work do make happen, but could be a possible mid-term solution.

an another idea on shared data between users:
different clients are sharing readonly workers DataFrames. so one can load  the data  into the memory and ask different questions over the different clients. something like distributed database?

FYI the original Bokeh link above is out of date/irrelevant. We now make this distinction explicit, listing Areas of Work that would be in scope, contingent on resources. https://bokeh.pydata.org/en/latest/docs/vision.html#areas-of-work

This wish list or "areas of work" or whatever you want to call it would indeed be very helpful to have. Most helpful are the things that are not already in the pipeline and are likely to be delivered within 2-3 months. Bokeh's doc is a good example, and so is SciPy's. Those are things that many people would really like to have, but are in many cases not moving forward (or not as fast as we'd like), because of lack of developer time.
Try to formulate an answer to this question: if I gave you a few talented engineers with as the only constraint they have to work on something more specific than "general maintenance and code review", what would you let them work on?
This is a useful doc to attempt to (a) get more effort going into Dask, and (b) get that work done on the right things. Concrete example: say that my company employs a Dask maintainer. And that person would like to spend more work time on Dask. That's awesome, but at some point I'll need to find funding for that. Obtaining funding for "general maintenance" or "whatever that maintainer is interested in" is very difficult. Funders want to know what they're funding. It's equally useful for things like building a shared understanding of project direction, or for attracting new contributors and giving them a more complete picture of the project.

Again, I'm happy to talk about "things that people might work on that would maybe be beneficial and interesting" but most of these are going to be external anyway, like "use dask to parallelize geopandas"

I don't understand this statement. Are you saying Dask itself is basically done?

I don't understand this statement. Are you saying Dask itself is basically done?

Relative to its use, yes.  I think that outreach or applying Dask to other domains is likely much more important than improving Dask itself.  If I had a free engineer, I would tell them to go explore a particular domain (indeed, this is my current situation).  So today my list would be something like:

Use Dask in imaging
Use Dask is radio astronomy
Use Dask to build parameter servers
Use Dask on GPUs
Use Dask in GIS
Use Dask in genomics
See if Numba and Dask together can solve PDEs, where does this break down?

We did this with the Pangeo group, and it uncovered a ton of work to do that was highly pragmatic both for the project, and for the scientific discipline.  We also found that it was very easy to fund this work from government grants.  I would strongly encourage this model into the future.
If you want a list of improvements in Dask, I recommend talking to user groups like Pangeo.  They'll be able to provide a wishlist for you that is specific to their domain, and likely relevant for others (cc @rabernat).

Use Dask to build parameter servers
Use Dask on GPUs

Both of those are not other domains, they're major items for your wish list.

uncovered a ton of work to do that was highly pragmatic both for the project

maybe you did it all, but uncovering a ton of work is exactly the point where you put some work items on your wish list.

See if Numba and Dask together can solve PDEs

What then goes on your wish list is "improve interaction between Dask and Numba".

If you want a list of improvements in Dask, I recommend talking to user groups like Pangeo.

I'm suggesting the Dask team does this, and writes down the outcomes.  Why let me reinvent the wheel?
Anyway, I already have a partial answer here. Start of a wish list:

Implement a parameter server for distributed training in dask-ml (see https://github.com/dask/dask-ml/issues/171)
Improve Dask usability and performance on GPUs
Improve interaction between Dask and Numba

All of those things are perfectly understandable and great possible conversation starters with funders. If you just write down a few more items like this and put them on wiki or in the docs, this issue can be closed:)

Implement a parameter server for distributed training in dask-ml (see dask/dask-ml#171)

Whoa, that is definitely not a core issue.  I wouldn't put it on a Dask wish list.  It is important in the field of ML and deep learning though, so there is a sub-community that would care about it.  It is one of perhaps a hundred similar issues.

If you just write down a few more items like this and put them on wiki or in the docs, this issue can be closed:)

I personally am unlikely to do this anytime in the next month (or longer?), though I encourage other people within the Dask community to collaborate and write down such a list that reflects common needs.

I'm suggesting the Dask team does this, and writes down the outcomes.  Why let me reinvent the wheel?

If it was easy to do then I would be more than happy to do it.  The requests that come in are very broad.  I think that there is serious work to do here in collecting and analyzing any list that is shorter than 100 items long.  There are no wheels to reinvent here, so your help would be welcome.
It's not that I'm withholding some information from you, it's that there is no good answer here.  If an institution like NumFocus wanted to help gather information here, then that would be welcome.  You may also be interested in https://github.com/dask/dask/issues/4748

Whoa, that is definitely not a core issue.

Okay. Since Machine Learning is pretty prominently in the doc navigation, that wasn't clear to me. The dask vs dask-ml distinction (and distributed etc) seems more like a project-internal thing with "Dask" being a project and a set of maintainers that work on the various repos. I'm interested in Dask the project more than dask the repo. Anyway, this can easily be addressed by a short statement on scope on any wishlist type doc.

I personally am unlikely to do this anytime in the next month (or longer?), though I encourage other people within the Dask community to collaborate and write down such a list that reflects common needs.

Cool, happy to hear this. From the discussion so far it felt more like "we'd rather not have this".

If it was easy to do then I would be more than happy to do it. The requests that come in are very broad. I think that there is serious work to do here in collecting and analyzing any list that is shorter than 100 items long. There are no wheels to reinvent here, so your help would be welcome.

My suggestion would be to be pragmatic. An incomplete list is better than no list. It's not necessary to hold a big user survey to start. Example: the NumPy and SciPy roadmaps were written by the core teams just by discussing on the mailing lists and in person. Those docs have been very helpful in different contexts.

It's not that I'm withholding some information from you, it's that there is no good answer here. If an institution like NumFocus wanted to help gather information here,

I'd be happy to help facilitate and do some prep work, and then perhaps have some higher bandwidth discussion at SciPy'19 or in a call before then?


java.lang.ClassNotFoundException: org.glassfish.jaxb.runtime.v2.ContextFactory

Trying to use the example in a existing spring project (spring 3.x) getting
* What went wrong:
Execution failed for task ':genJaxb'.
> java.lang.AssertionError: jakarta.xml.bind.JAXBException: Implementation of Jakarta XML Binding-API has not been found on module path or classpath.
   - with linked exception:
  [java.lang.ClassNotFoundException: org.glassfish.jaxb.runtime.v2.ContextFactory]

I was running into the same issue. In my build.gradle.kts, what fixed it was adding jakarta.xml.bind:jakarta.xml.bind-api andcom.sun.xml.bind:jaxb-impl to my jaxb configuration. That way they gets added to the classpath when running the xjc ant task. The result was:
val jaxb by configurations.creating

dependencies {
    jaxb("com.sun.xml.bind:jaxb-xjc:$jaxb-version")
    jaxb("jakarta.xml.bind:jakarta.xml.bind-api")
    jaxb("com.sun.xml.bind:jaxb-impl:$jaxb-version")
}

We plan to rewrite this guide and already have an issue to remind us about the JAXB issues, so I'll close this one.
See https://github.com/spring-guides/gs-consuming-web-service/issues/54


Chore: Typo fixed in multiple files

This PR fix the typo in multiple files
Thes > The
JavasSript > JavaScript
benefitial > beneficial
Alternativey > Alternatively

Please check the following:

[X] Do the tests still pass?[^1]
[X] Does the code comply with the style guide?

[X] Run make lint from the Wagtail root.

[X] For Python changes: Have you added tests to cover the new/fixed behaviour?
[X] For front-end changes: Did you test on all of Wagtail’s supported environments?[^2]

[X] Please list the exact browser and operating system versions you tested:
[X] Please list which assistive technologies [^3] you tested:

[X] For new features: Has the documentation been updated accordingly?

Please describe additional details for testing this change.
[^1]: Development Testing
[^2]: Browser and device support
[^3]: Accessibility Target

Thank you @SusheelThapa  - I have added you to the contributor's list as 'Susheel Thapa' let me know if you wanted a different name listed.


nixos/brightnessctl: add package to system packages

Motivation for this change
When enabling hardware.brightnessctl, the program should be available.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @

Obsoleted by 9566ec034b9515ba2487d3625664109b59ca46f1.


Issues when paginating for a search results component

Hi!  I heard the podcast on this and liked the ideas behind it, and at the same time had a component that I needed to add infinite scroll to, so I thought I'd give it a whirl.  I'm on my second day now and I still haven't got this basically working in my app.  Hate create an issue for it but was unsure of what other venue I had. If I stick with this library I'd be happy to PR any shortcomings.
So here it goes...

No clear way to reset the impaginator dataset.
Since I'm a search component, my dataset changes as the user changes the search query.  Whenever the user changes the search.  I couldn't find a way to reset things at first.  The dataset seems to only be recreated when the pagination parameters change, which I didn't want to do.  I finally settled on just creating the dataset myself and passing it as a component attribute, which meant I also had to pass the dataset state as an attribute (since the dataset has an observer that sets it).  Anyways, that seems to get me farther, but also seems hackish
My list is rendering before the promise resolves from fetch
my fetch function is called, in which I make a query call to the ember store and return the resulting promise.  I add a then to that promise which sets the stats.totalPages and sets the result data on my own component (so that I can fill in decorating information on the UI about results), and I return an array of results
Here's the order I see:

fetch is called (returning the ember-data query promise)
list is rendered (with a bunch of undefineds)
then function runs and returns results
nothing re-rendered

Any guidance around this would be appreciated.  I can post some code as well, but I wanted to make sure my scenario is one that makes sense.

Also, what scope should fetch be under?  I find myself having to bind it to my own component so I can access component properties (to set information about about the result)

fetch function:
    fetchChats(pageOffset, pageSize, stats) {
        console.log('fetch');
        return this.get('store').query('chat-search-result', {
            query: this.get('chatSearchResultSuggestion.searchValue'),
            expand: 'to,from',
            targetJids: [this.get('activeRoom.jid')],
            pageSize: pageSize,
            pageNumber: pageOffset,
            order: 'recent'
        }).then((searchResults) => {
            console.log('then');
            const results = searchResults.get('content').mapBy('record');
            this.set('chatSearchResultSuggestion', ResultSuggestion.create({
                results: results,
                metaData: { total: searchResults.get('meta.total') },
                searchValue: this.get('chatSearchValue')
            }));
            stats.totalPages = searchResults.get('meta.total');
            console.log(`results of then`);
            console.dir(results);
            return results;
        });
    },

Code that creates a dataset to refresh on new search terms
    _resetDataset() {
        this.set('dataset', new Dataset({
            pageSize: 25,
            loadHorizon: 2,
            unloadHorizon: Infinity,
            fetch: this.fetchChats.bind(this),
            observe: (datasetState) => {
                this.set('datasetState', datasetState);
            }
        }));
    },

template:
        {{#search-pane
           resultSuggestion=chatSearchResultSuggestion
           isDisplaying=isSearchSelected
           searchPlaceholder=(t 'chat.chatSearch.searchPlaceholder')
           searchHelp=(t 'chat.chatSearch.searchHelp')
           performSearch=(action "searchChats")
           loadItem=(action "loadHistoricChat")
           clearSearch=(action "clearChatSearch")
        }}

            {{#impagination-dataset
               fetch=fetchChats
               dataset=dataset
               datasetState=datasetState
               page-size=25
               as |records|
            }}
                {{log "rendering results"}}
                {{log records.length}}
                {{#ember-collection
                   items=records
                   cell-layout=(fixed-grid-layout 800 80)
                   as |record|
                }}
                    {{log "rendering result"}}
                    {{log record}}
                    {{chat-search-result
                      result=record
                      loadItem="loadItem"
                      selectedResult=selectedResult
                    }}
                {{/ember-collection}}
            {{/impagination-dataset}}

        {{/search-pane}}

I was able to get the dataset component to re-render by setting the dataset in the observe dataset function after render so that it would hit the didReceiveAttrs hook of the dataset component, but I'm still unable to get my ember-collection to do likewise.  I'll try switching to each to see if its an issue with the ember-collection component, but I was under the impression the each would have rendering consequences for large datasets.
    _resetDataset() {
        this.set('dataset', new Dataset({
            pageSize: 25,
            loadHorizon: 2,
            unloadHorizon: Infinity,
            fetch: this.fetchChats.bind(this),
            observe: (datasetState) => {
                Ember.run.scheduleOnce('afterRender', () => {
                    this.set('datasetState', datasetState);
                });
            }
        }));
    },

So with {{#each}} it will load the first page for me, but no additional pages.  I'm assuming that's because {{#each}} isn't set up for virtual scrolling, and I would have to add the scroll event handling code.  So there's something different about ember-collection where its not handling the changes in the record collection. Next I can try virtual-each, which I believe is in the impagination demo, and see if I can get that to render.

virtual-each does in fact render, but I'm getting in some kind of infinite fetch render loop 😢I feel like most of my problems are because I'm implementing the resetting of dataset which the addon's not currently set up for or tested with, so I'm going to throw in the towel on this.  I might pull down the source and try to add the feature internally and see how far that gets me.  If anything this has all been an education on component rendering :-P

In _resetDataset do you need to tear down the observer for the old dataset?

Just a quick update...  Got rendering/infinite scroll working with ember smoke-and-mirrors.  I am considering incorporating ember-impagination as well to handle the fetching/unfetching of data.  Will probably spend more time on in in the next few weeks.  Its up to you guys whether you want to keep this issue open around the reset functionality or close it and track it elsewhere.
Thanks so much for the ideas and help thus far!

@toddjordan We definitely want to keep this open so that we can modify ember-impagination to fit this use-case. Quick question. If the component yielded an action to reset itself, would that work for you? e.g.
{{#impagination-dataset as |dataset|}}
  <button on-click={{dataset.reset}}>Reset!</button>
  <button on-click={{dataset.reload}}>Reload!</button>
{{/impagination-dataset}}
```hbs

Possibly.  So in this case I'd have to wrap my search-pane component with impagination as follows...
{{#impagination-dataset fetch=myFetch as |dataset|}}
  {{#search-pane resetAction=(dataset.reset) searchAction=(action "performSearch")}}
      {{#ember-collection items=dataset as |record|}}
        {{chat-search-result result=record}}
      {{/ember-collection}}
  {{search-pane}}
{{/impagination-dataset}}

The odd thing is I need to re-fetch based on the query given in search pane, but at the same time, I need the search pane to invoke reset when when a new term is entered.  It seems like a circular dependency, but I guess the search-pane could send an action (performSearch) up with the search query text as an argument to the parent component or route of the dataset, which would also implement the fetch function.

In that case could you not have the performSearch change the value of the myFetch which should have the effect of reseting the dataset (if it doesn't then that's definitely a bug).

That is to say, that the myFetch property is a function which closes over a set of bound search parameters: new search params recomputes new fetch function, recomputes new dataset.

Maybe I'm not understanding.  I thought I would need to somehow provide the search query string collected by the search pane to the fetch function. in the parent component.  That's essentially what perform search was doing: passing the new query information up to where the fetch function could access it.

I guess it would be a possible use of the mut helper, so instead of passing up an action I can just have the parent component own the query data.

I don't think you need to use the mut helper if the data is coming down from the parent component. Basically, the performSearch action would set a new fetch function which would come down again to the impagination-dataset.
Another option would be to have your search pane yield a fetch function which is then passed to the dataset.

ok, I see what you are saying.  performSearch would actually pass the fetch function up to the parent, in theory causing fetch to recalculate.  I think if I remember looking at the impagination component source, that the dataset was only created on init.  My memory may not be right though...
Also, in my case its a bit trickier, as the search-pane is reusable component that can search various entities within my app (doesn't really care what its searching, just provides a way to provide search criteria and yields to iterate results.  Today the performSearch closure action on the parent already knows whether its a person  or a chat search for instance and knows how to fetch appropriately.  That's why I was saying all the performSearch function really just needs to pass is the query data.

ok, you're right, dataset resets when fetch changes.

I think that the search pane can still just pass query data to its action. The fetch function would be computed in your controller or route, but it would have the query data as a dependency.

I want to start working on a PR to introduce a number of functions that exist on the dataset
[ ] refresh() - refreshes the dataset at the current read-offset
[ ] reset() - resets the dataset to read-offset 0
[ ] setReadOffset(offset) - sets the read-offset to offset
{{#search-pane searchAction=(action "performSearch")}}
  {{#impagination-dataset fetch=myFetch on-dataset=(action "observeDataset") as |dataset|}}
      {{#ember-collection items=dataset as |record|}}
        {{chat-search-result result=record}}
      {{/ember-collection}}
  {{/impagination-dataset}}
{{/search-pane}}

   _refreshDataset() {
        this.get('dataset').refreshDataset()
    },
    observeDataset: function(dataset) {
        this.set('dataset', dataset)
    }

Thoughts?

@cowboyd agreed, you certainly could.  It doesn't seem super intuitive but I definitely see that working
@flexyford I like that.  It keeps true to how I originally thought how the template should be layed out.

@flexyford @cowboyd  If you don't want to give the parent the actual dataset object and stay more "data-down", you can can expose an action called reset with a reset function as an argument.  That way, the parent component can just call reset when it changes a query or some other context that would cause the dataset to be reset.
Not sure if I like it better or not, but another option...
{{#search-pane searchAction=(action "performSearch")}}
  {{#impagination-dataset fetch=myFetch provideRefresh=(action "provideDatasetRefresh") as |dataset|}}
      {{#ember-collection items=dataset as |record|}}
        {{chat-search-result result=record}}
      {{/ember-collection}}
  {{/impagination-dataset}}
{{/search-pane}}

_refreshDataset() {
  this.get('refreshData')();
},
actions: {
  provideDatasetRefresh(refreshFn) {
    this.set('refreshData', refreshFn);
  }
}

@toddjordan Passing the entire dataset up does not entirely adhere to the DDAU principle. However, I really like the fact that these actions are available in the controller/route, AS WELL AS in a child component utilizing a similar syntax.
{{#search-pane searchAction=(action "performSearch")}}
  {{#impagination-dataset on-dataset=(action "observeDataset") fetch=myFetch as |dataset|}}
      {{#ember-collection items=dataset as |record|}}
        {{chat-search-result result=record on-no-results=(action dataset.refresh}}
      {{/ember-collection}}
  {{/impagination-dataset}}
{{/search-pane}}

Thanks for bringing up this point. I'll give it some additional consideration.

I've run into the need to get the dataset to refresh as well. Here's how I got it working.
// controllers/foo.js

import Ember from 'ember';

export default Ember.Controller.extend({

    store: Ember.inject.service(),

    query: null,

    actions: {

        fetch(query, pageNumber, pageSize, stats) {

            let params = {
                query: query
            };

            return this.get('store').query('foo', params).then((result) => {
                let meta = result.get('meta');
                stats.totalPages = meta.totalPages;
                return result.toArray();
            });
        }
    }
});

{{!-- templates/foo.hbs --}}

{{input value=query}}
{{#impagination-dataset fetch=(action "fetch" query)}}

With this approach, each time the value of query changes the fetch action is called again, the value of query is passed into the function, and the data refreshes.
Thoughts?

@seanrucker This is a brilliant approach when we want to refresh the data based on queryParams. And if that is your use-case for refreshing, then I highly recommend it! I will add this to the README!
I'm still considering alternative to enable filtering records on the front-end. If we want ember-impagination to handle filtering for us, then the value of query will not change, so we need a different way to refresh data such that filtering is applied to the latest dataset.

Added way to reset, reload, and refilter dataset in 0.2.0

Thanks!


Enhanced Traffic Shaping

Objective
To enhance Martian’s trafficshape package so that it can simulate desired network conditions more precisely. These features are needed in order to allow developers to write tests that exercise application features that are designed to react to suboptimal network conditions.
Key features of this effort include:

Apply traffic shaping behaviors on a per URL basis (based on a regex)
Specify throughput (mbps) for byte ranges
Specify hangs (with duration) or exits at byte offset

Background
These features are useful to anyone who is interested in writing tests that simulate poor network conditions: especially any tests that are serving video. More specifically, the test authors will be able to specify the “shape” of the video traffic as a configuration in the test and write assertions based on the intended behavior of client in dealing with the traffic.
Currently, the trafficshape package only allows setting the bandwidth and latency for all the requests and responses; the user cannot pick a few resources for which to shape traffic. This is insufficient for the testing situation where one wants to throttle a particular resource while letting the rest of the test infrastructure use the network normally.
The trafficshape package also does not have a way to keep track of what byte offsets are currently being written back to the client for a specific resource. This can be useful when the user wants to perform different actions or have different bandwidths depending on what byte ranges are being written back to the client.
Requirements

Traffic shaping behavior is configured from within a test by issuing a POST request with a JSON configuration to a new configuration endpoint /configure/trafficshape
Traffic shaping behaviors are defined on a per resource basis

Match on URL or parts of URL (using martianurl.Matcher)
Match based on URL regex (needs a new matcher)

Add the ability to specify

Downstream bandwidth for a given byte range
Hang for a given time at a specific byte
Close connection at a specific byte
Number of times to repeat this behavior

Design
JSON Configuration API
{
    "trafficshape": {
        "default": {
            "bandwidth": {
                "up": 10000000,
                "down": 1000000
            },
            "latency": {
                "up": 1000,
                "down": 100000
            }
        },
        "shapes": [{
            "url_regex": "example.com/video/stream",
            "throttles": [{
                "throttle": {
                    "bytes": "100-150000",
                    "bandwidth": 100000
                }
            }],
            "halts": [{
                "halt": {
                    "byte": 200000,
                    "duration": 5000,
                    "count": 1
                }
            }],
            "close_connections":[{
                "close_connection": {
                    "byte": 250000,
                    "count": 1
                }
            }]
        }]
    }
}

Default Latency and Bandwidth
These are the values that will be applied for the byte ranges that do not have throttles specified for them.
Throttle
Every throttle action takes in a byte range and a bandwidth. When the proxy is writing back bytes that fall in the range of a throttle’s byte range for the current url, it will be subject to the corresponding bandwidth.
Halt
Every halt action takes in a byte, duration and count (Explained below). The byte specifies at which byte to halt writing data back to the client and duration specifies how long to remain in this state before starting to write data back to the client.
Close_connection
Every close_connection action takes in a byte and a count (Explained below). The byte specifies what byte the proxy should be about to write back to the client before it should close the current connection with the client.
Count
Note the “count” property in the “halt” and “close_connection” actions . It is possible that in the course of writing the video, the client asks for byte range multiple times (this might happen when the existing connection is interrupted, and the client didn’t read part of the data written back by the proxy already, so it asks again for a range of the data that we have already written). So it is possible that they would want the action to be performed only the first time a specific byte is written back, or possibly more times. That is what the count specifies. A count of -1 (default) will specify that the action will happen everytime the specific byte is written back.
So the example json says:

Make the default bandwidth 100,000mbs, and do not add any additional latency.
And for any response containing a substring matching the url regex example.com/video/stream:

Write the bytes 100 to 150,000 (150kb) (with 150kb being exclusive) to the client at the bandwidth of 50,000 (50kb) per second. For all the other bytes outside this range, write back at the default bandwidth.
Upon reaching the byte 200,000 (200kb), perform an interrupt for 5000ms. Do this only the first time you are writing the byte 200kb back to the client. Do not do anything if you are writing back byte 200kb again.
Upon writing the byte 250,000 (250kb), close the current connection with the client. Do this 1 time only the first time when writing the byte 250kb back. (After we do this, the client may start a new connection and request the rest of the video, and we don’t want to perform the “close” again. Note, we almost always overestimate the data we write back to the client, so if we perform the close at byte 250kb, the client would probably ask for bytes, say 200kb onwards in a following request, and we do not want to close the connection again when we reach byte 250kb.)

For the byte_ranges, we support ranges to be expresses as “-5000”, which means “up till but not including byte 5000”, or “5000-”, which would mean from “5000 onwards”.

Why closed?


Support for Converting ISO 8601 Durations to Human-Readable Format

Context:
I frequently use Obsidian Web Clipper to extract information from web pages, including video metadata like duration. Many websites, such as YouTube, provide video durations in the ISO 8601 format—for example, PT6702S represents a duration of 6702 seconds.
Currently, the available filters (slice, date, calc) don't allow for an easy conversion of this ISO 8601 duration into a human-readable format like HH:mm:ss. Attempts to manipulate the string and parse it using the date filter lead to issues with time zone adjustments, causing inaccurate results.
Feature Request:
I propose adding a new filter or enhancing existing ones to support the conversion of ISO 8601 duration strings directly into human-readable time formats without time zone adjustments.
Proposed Solution:

New Filter (e.g., duration):

Parses ISO 8601 duration strings and formats them according to a specified pattern.
Example usage: {{schema:@VideoObject:duration|duration:"HH:mm:ss"}} would output 01:51:42 for PT6702S.

Thank you for considering this feature request!

Will be in 0.10.5


Update tutorial

Coverage remained the same at 100.0% when pulling 811ee7b48f6dd2a081a2877fd10688885f3e542e on update_tutorial into da9661b2cf64f2612a9dec9e0f735c2520e9b895 on master.


Hamburger menu replace with text, images in Resource folder don't show

I'm using Xamarin.Forms.Nuke and GlideX.Forms. I have to say loading images are quite fast right now.
I have a MasterDetail Page. In Android is working fine. I'm facing an issue with iOS. If I enable Nuke, the hamburger menu disappears and the title is shown.

If I disable Nuke, the hamburger is there.

Also, I notice other images in the Resources folder are not display but the images from an url are very fast. This is how I use Image:
<Image
    Grid.Column="1"
    HeightRequest="150"
    HorizontalOptions="CenterAndExpand"
    VerticalOptions="CenterAndExpand"
    WidthRequest="150">
    <Image.Source>
        <FileImageSource File="gmdclogo.png" />
    </Image.Source>
</Image>

Do you have any idea? Thank you

I'm afraid this has nothing to do with XFNuke


[Autocomplete] Accept suggestions with tab as per VS Code?

Tab currently adjusts line indent even while autocomplete is active. VS Code instead only indents when in the whitespace/beginning of the line and inserts a tab or accepts autocomplete elsewhere.
One for discussion.

So far as I'm aware there is no demand for this. Closing. We can reconsider based on user feedback. Would require tweaking CM key bindings.


Update LTS description

Explicitly mentioned a possibility to take feature backports to support new Kubernetes releases.
See https://github.com/containerd/containerd/pull/8943#issuecomment-1689110782 for more details.

/assign @akhilerm @fuweid @mikebrow

@AkihiroSuda CI/Windows Integration (windows-2019, legacyCRI) test failed. I don't think it's caused by my PR. Can you suggest what to do?

@AkihiroSuda CI/Windows Integration (windows-2019, legacyCRI) test failed. I don't think it's caused by my PR. Can you suggest what to do?

You can just ignore it in this case.

We can still clean up the phrasing here. Right now it is kind of...we will accept it, actually no we won't, but maybe 😆 Perhaps this is the reality of what some contributors already feel, but we can try to improve that and fix the wording in a follow up.


7nodes: Update configs to use latest format/options

Depends on
https://github.com/jpmorganchase/quorum/pull/127 and
https://github.com/jpmorganchase/constellation/pull/35/

This vagrant image won't provision until Quorum 1.1.1 is released (since it tries to pull that tag.)

This vagrant image won't provision until Quorum 1.1.1/1.2.0 is released (since it tries to pull that tag.)

This vagrant image won't provision until Quorum 1.1.1 is released (since it tries to pull that tag.)

Merged in https://github.com/jpmorganchase/quorum-examples/pull/45


Support for making libraries

Hello,
I use your GenercMakefiles in every of my projects as I find them very convenient and easier to use than other Make alternative.
Is it possible to add the option to build .a libraries instead of executables?
Best regards.

Hi @marespiaut, glad you have found this to be useful!
I don't have a good setup to test this at the moment, but I believe it should be a fairly simple change to make the file build libraries.  Be sure you've set BIN_NAME to library-name.a, then edit line 209 to be $(CMD_PREFIX)ar rcs $@ $^.   There a few other things you may want to clean up, for example the symlink created here is pretty pointless with a library.


[Hosts] Host summary onhover of IP address

This would be a convenience/UI enhancement to display an onhover dialog box with a summary of the details for a host.
I would think this should have a expandable headings for each main tab (services, vulnerabilities, operating systems, notes, hostnames, credentials)
These would show a number after them (eg. [+] Services (13)), and would be collapsed by default.
Any information/details that doesn't need to be collapsed would be shown as a string above the expandable headings.
An alternative to showing all this information onhover of the IP address would be to show it onhover for each particular cell/column. (Eg. hovering over the hostname for a particular entry would show the weighted list of all hostnames (and allow you to change the weighting/add new/etc?)

We've decided not to pursue this as the performance impact would be too high, and we have views to view the useful data quicker. If someone submits a PR and it performs well when we have a ton of hosts we'll accept it.


Argument value with dash not accepted when using single-dash key

If I define the following:
[Option('t', "test")]
public string MyArg { get; set; }

then when running with arguments -t -1234, Parser.Default.ParseArgumentsStrict returns false.  If I use arguments --test -1234 it returns true.  Maybe the parser could be smart enough to treat the next argument as the value?

It should work exactly the way you expect. Sounds like a bug, but we are not developing version 1.x anymore. If you upgrade to the latest beta (2.1.1) and are still seeing issues, please submit them to this new repository as this original fork has closed.
If you are unable to upgrade, I believe you can work around the issue by using -t=-1234 or -t-1234, but I haven't had a chance to test.


PhantomJS has crashed

On MacOS I'm getting:
./bin/phantomjs
Killed: 9

in case you haven't noticed, there's lots of "PhantomJS has crashed" messages on this board.  Nobody's going to read yours unless your title is more specific.

This project should get a real bug tracker -- then you won't have to flame legit bug reports that lack info.  There is a binary fix already out there, from Feb. 2015, at https://github.com/eugene1g/phantomjs/releases.  I thought this fix would be in the mainline by now, so I just downloaded the official binary, and yet it still breaks on OS X Mavericks.  Shouldn't the official site say Mavericks is NOT supported?

Fixed in 2.1.


boards: stm32h573i_dk: Enable debug support with pyocd

Support of stm32h5 targets with pyocd is required to allow debugging. Provide runner configuration and update board documentation.

@henrikbrixandersen Do you think it could sneak into v3.6.0 ?


Improve sign in button contrast

Try a dark color for the text

Similar to this

Hey @amendezm I would like to do this one as it seems like its in my level of knowledge (I am a beginner), so could I do this. Also could you guide me with how to run the project, I have forked and cloned this repository but cant find a way to run this project on my device.

Hello there @Aspireve. Yo can go ahead with this issue. I will assign it to you.
Did you try:

npm install
npm start ?

You must also have a twilio account, create a .env file and fill the values. See .env.example file

@Aspireve if you like share me your email so I can invite you to our slack workspace and send my .env file for DM

<EMAIL_ADDRESS>is my email
could you tell me a bit more about a .env file
also on running npm start I get
><EMAIL_ADDRESS>start
 > concurrently twilio-run npm:build
concurrently' is not recognized as an internal or external command,
operable program or batch file.
I did already run an npm install

Try install concurrently globally

@Aspireve I just sent you an email with the slack invitation

I did not receive any mail on that mail id, could you please send it again.

Oh now it says I need an Account SID to run, i guess that is the .env file that you will be providing right


MVP guidelines for Multiline box

Was reviewed by UX and Danny
Proposed Changes
New guidelines
Potential Breaking Changes
None I hope
Acceptance Criteria

[ ] The proposed changes are covered by unit tests
[ ] The potential breaking changes are clearly identified
[ ] README.md is adjusted to reflect the proposed changes (if relevant)

@gdostie Ready to Merge


Add support for VPCOptions in ElasticSearch

hello 😄
some days ago AWS announced support for VPCs in ElasticSearch service:
https://aws.amazon.com/es/blogs/aws/amazon-elasticsearch-service-now-supports-vpc/
Here is an example from official documentation:
POST /2015-01-01/es/domain
{
    "DomainName": "<DOMAIN_NAME>",
    "ElasticsearchVersion": "<VERSION>",
    "ElasticsearchClusterConfig": {
        "InstanceType": "<INSTANCE_TYPE>",
        "InstanceCount": <INSTANCE_COUNT>,
        "DedicatedMasterEnabled": "<TRUE|FALSE>",
        "DedicatedMasterCount": <INSTANCE_COUNT>,
        "DedicatedMasterType": "<INSTANCE_TYPE>",
        "ZoneAwarenessEnabled": "<TRUE|FALSE>"
    },
    ...
    "VPCOptions": {
        "SubnetIds": [
            "<SUBNET_ID>"
        ],
        "SecurityGroupIds": [
            "<SECURITY_GROUP_ID>"
        ]
    },
    ...
}

Is there any plan for adding support for VPCOptions?
Meanwhile, thanks for the fantastic job you are doing!
thanks!

Unfortunately, AWS hasn't added this the CloudFormation API, see http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticsearch-domain.html, so it isn't accessible to CloudFormation and by extension Troposphere.
To encourage them, please submit a feature request (the more they get, hopefully the quicker they will implement it).

I was looking in the wrong place http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-configuration-api.html#es-configuration-api-datatypes-vpcoptions 😢
thanks @craigbruce

While this is not yet documented, I did get the impression that this should be already working (keep in mind I haven't tested).

Fyi, the documentation has been updated, and it works.
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticsearch-domain.html

PR updated :) thanks 4 the spot!

Fixed via 0a38d79bf38c26c9dd543565c296e5f85f849827


Two projects same code but build fails in one

How did you install the Amplify CLI?
like description
If applicable, what version of Node.js are you using?
16.8.0
Amplify CLI Version
12.4.0
What operating system are you using?
MAC M1
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No
Describe the bug
I develop with feature branches.  a feature/branch has a git hook which starts the build on aws.amazon.com.  Testing it and then I merge to master branch which has a git hook o another amplify project to build and run.
So I had a build with the test environment works fine. After merging same code other amplify project (production) -> fail. Same node versions same amplify version. Problem with graphql.schema (according to the log). The comparing master the feature branch same code. I am stuck in the process completely now. can You have a look on it ? The error is :
Schema is the same....
BuildMasterFailed.txt
Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: a5625ae1-f83b-481c-a880-4f280207caa2; Proxy: null)
UPDATE_FAILED      DeleteFullTagResolver                                                                         AWS::AppSync::Resolver              Sun Sep 10 2023 17:10:57 GMT+0000 (Coordinated Universal Time) Resource update cancelled
Expected behavior
The build should run in both environment, because it is the same code.  Or fail in both environments with the same error
Reproduction steps
1.Make two amplify projects.
2.connect it via hook to two branches
3.Merge this branches
4. change to transformer 2
5. Make changes to schema.graphql
6. Build in dev
7. merge to master
8. build there
You must test it more often it is not always the case. Mostly in cases of graphql changes but also not always
Project Identifier
No response
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

Seems to be a fluke. Can you try the steps here to repair the broken state? https://github.com/aws-amplify/amplify-cli/issues/682#issuecomment-471084364
Looks like the issue happened on tagsByName.

Thanks for the hint I connected ("vernüpfen") in german a unit Resolver (whatever this is) and deleted it again.
Now I habe this error :                                  CREATE_FAILED               QuerytagsByNameResolver
I can delete it in Query there this a piplinerelover with tagsByName. Can I simply delete it ? Is it created again? It is my production environment so I am careful....  See screenshot is this pipeline meant which I should delete ?

I've though of another option that is safer than manually messing with the console.
Remove queryField: "tagsByName" from your schema on the FullTag model and amplify push. Then after is complete, return queryField: "tagsByName" to your schema and amplify push.
There will be a period when any queries that rely on tagsByName will fail because it will be replaced by the default query field. Once you finish the second amplify push the app should work correctly.

Thanks a lot , your suggestions works. The issue can be closed now.
I am in france at the moment so :
Merci beaucoup !!


[Bug] Bank Transaction table missing

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue
One of our 3 Xero instances doesn't have the bank_transaction table and most models are skipped because of this.
Relevant error log or model output
No response
Expected behavior
If the table doesn't exist, users should be able to not include it in the models being ran through a variable, just like the credit_note
dbt Project configurations
We just use the standard configuration for this package.
models:
xero:
+tags: 'default_transformation_xero'
+schema: "dataset"
xero_source:
+tags: 'default_transformation_xero'
+schema: "dataset"
Package versions

package: fivetran/xero
version: 0.4.0

What database are you using dbt with?
bigquery
dbt Version
version: 1.0.1
Additional Context
No response
Are you willing to open a PR to help address this issue?

[X] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[ ] No.

Hey @fivetran-joemarkiewicz thank you for the quick reply.
I already coded the changes, my only question would be about repo synchronization, because this change requires both changes in the dbt_xero and the dbt_xero_source. How would it be better to submit both PRs? I was thinking on linking the dbt_xero_source changes on the main one, would this be ok?
Creating the PR soon!

@santi95 this is great thanks so much for opening this PR 💯!
The way you have done this is perfect with the two PRs and the reference in this issue. I will review these for you tomorrow! The only thing I will ask off the bat is:

[ ] Make sure to update the README to include this new variable for both the source/modeling packages.
[ ] Increase the index of the package version in the dbt_project.yml files
[ ] Add your contribution to the CHANGELOG.md files for both the source/modeling packages

Those are what I was able to see at a cursory glance. Everything else I will review in more detail tomorrow 😄

@fivetran-joemarkiewicz Just uploaded the requested changes!
Thanks for everything

Hey @santi95 thanks again so much for your contributions! 🙌
I just cut the release for the dbt_xero v0.4.1 and you should see the release be live on the dbt hub at the top of the hour. Thanks again for your collaboration and please feel free to open another issue or feature request in the future if you encounter any issues or would like more flexibility to the package. 😄


Update to 0.17

https://github.com/elm-community/elm-maybe-extra/pull/13 is a more thorough update. Use that instead.


[Enhancement] Add a configuration item to control data checksum for block cache

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Add a configuration item block_cache_checksum_enable to expose the checksum switch for block cache, which enable/disable data checksum when read/write.
Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

run starrocks_be_unittest

run starrocks_admit_test

starrocks_be_unittest succeeded.

run starrocks_admit_test


best practices for persisting data?

Is there any best solution?

I'm not aware of any "best" solution. In my own project, I have at most about a million pictures in a duplo store and I use the GobEncode and GobDecode methods to save to and read from a file. But you obviously cannot make changes during that time. And it may not scale well to your needs. It works for me, though.
I have been thinking about using a cache library like Ristretto for duplo. But this would require some sophisticated changes to the architecture. It's certainly nothing that works out of the box.

thank you very much for your work, I wish the storage would take up less running memory, do you have a baseline memory usage analysis of millions of photos?  I'm using it on an embedded device
sorry, my English is not very good

Just checked and found that some of the stores actually contain about 2 million images. Gob-encoded and gzipped, they require around 500MB of disk space.

cool! I'm already using him on Nas


reimport-controller: add verbosity to klog calls

This also adds some new logging to the reimport controller

/lgtm


out of sync video after render!

I found out that video and audio were out of sync after rendering!

Hello, can you please tell me if there is a time with a black screen or an image without sound at the end of the video?
Was the render completely finished or was it partial?
And maybe you can send an example of a file with a video in which this was found in telegram: @JargeZ?

render completely finished.

continuing this thread, a more informative way of saying this would be the frame rate rounds down to the nearest whole number when exporting. for example 29.97 fps rounds down to 29 fps once exported and 59.94 fps rounds down to 59. the video also ends based on when the audio finishes rather than when the video finishes.

Oh, @Mr-Emerald thank you very much for helpful a comment!
I will fix this by copying the exact frame rate when creating a video from the original video.
I forgot the existence of non-integer frames per second values, and used int()

Hi @Mr-Emerald and @zacharyhub
Can you check this issue with build from this pre-release https://github.com/JargeZ/vhs/releases/tag/v1.5.0

By my checks, this should be fixed.
Reopen, if does not. Thank you!


i have problem getting distance in z-direction from the depth map..

I tried to use retrieveMeasure(DEPTH) and retrieveMeasure(XYZ)....i dont know how to access the resulted matrix in order to get the distance in z-direction for a certain pixel..which one of these 2 functions are correct to do so and how to access the resulted matrix in order to get distance in z?

Hi,
if you only want the depth on the Z axis, then you can use the retrieveMeasure(DEPTH) function.
To extract the depth value ( z)  for pixel (i,j) then you can do the following :
sl::zed::Mat mdepth = zed->retrieveMeasure(DEPTH);
float* zBuffer = (float*) mdepth.data;
z =zBuffer[i+j*mdepth.step/sizeof(float)];

retrieveMeasure(XYZ) returns the (X,Y,Z) world coordinates in a 4channels float buffer.
Best,
/OB/

well thanks alot Oliver...i just want to make sure i got this right
1- the value returned to z is the distance in mm right?
2- what's the difference if I used retrieveMeasure(XYZ) ? would return the xyz and color of each pixel? and if yes then do you know how to extract x,y,z values of each pixel?
I know i asked alot but I really appreciate your help :D

Hi,

Z value will be the distance in the unit you select during Init. In the InitParam structure, you can choose the Unit (structure entry)  between the available enum (mm, meter, inches....). By default, it's in mm.

retrieveMeasure(XYZ) and retrieveMeasure(XYZRGBA) will return a 4 channel float Mat. XYZRGBA will contain the RGBA color for the last float value (4 uchar (4*8b) <--> float (32b)) .
Extraction will need to take into account the 4 channel :

sl::zed::Mat mXYZmat= zed->retrieveMeasure(XYZ);
float* xyzBuffer = (float*) mXYZmat.data;
X =xyzBuffer [4*i+j*mdepth.step/sizeof(float)];
Y =xyzBuffer [4*i+j*mdepth.step/sizeof(float)+1];
Z =xyzBuffer [4*i+j*mdepth.step/sizeof(float)+2];

// for the color, in case you use XYZRGBA :
float rgba_val =xyzBuffer [4*i+j*mdepth.step/sizeof(float)+3];
unsigned char* rgbaPixel = (unsigned char*) & rgba_val;

uchar r = rgbaPixel[0];
uchar g = rgbaPixel[1];
uchar b= rgbaPixel[2];
uchar a = rgbaPixel[3];

//

Best,

Hi Oliver,
I am wondering why the white balance in XYZRGBA mode is so weird. The document says that I can change the value of White balance using setCameraSettingsValue but there is no WHITE_BALANCE constant. When I try to set Hue value, I get an exception in sl_zed64.dll
My SDK is version 1.1
When I use retrieveImage, it show normal color.
Best,
Vu

Hi,
Can you show picture for XYZRGBA?
I think it is a mistake in the documentation : you can modify the exposure, not the White Balance (which is automatic for now).
Changing the Hue will impact as well the color balance. You should use those settings only in Live mode. (ZED SDK 1.2.0 fixes a bug in setCameraSettingsValue() when svo mode was activated).
I would recommend to update your ZED SDK version as well.
Best,
/OB/

Hi Oliver,
I wrote some codes to analyse the video from SVO file. I need to retrieve both color image and position XYZ of some color markers. The following are my code to extract the color and XYZ, and the corresponding picture.
My code:
sl::zed::Mat img = zed->retrieveMeasure(XYZRGBA);
int* temColor = (int*)img.data;
float* temPos = (float*)img.data;
for (int i = 0; i < frameWidth * frameHeight; i++) {
color[i] = temColor[i * 4 + 3];
position[i * 3] = temPos[i * 4];
position[i * 3 + 1] = temPos[i * 4 + 1];
position[i * 3 + 2] = temPos[i * 4 + 2];
}
retrieve LEFT image

extract from XYZRGBA

Thank you.
Vu

Hi Oliver,
I have many questions,
Does sl::zed::Mat mXYZmat= zed->retrieveMeasure(XYZ); retrieve the x,y,z coordinates for a pixel in the current captured frame ? and if yes to which reference (where)?
I tried the above code of your comment
//code 1
X =xyzBuffer [4*i+j*mdepth.step/sizeof(float)];
Y =xyzBuffer [4*i+j*mdepth.step/sizeof(float)+1];
Z =xyzBuffer [4*i+j*mdepth.step/sizeof(float)+2];

and i got sometimes negative values,why ? and the z value is not the same if i use this code
//code 2
sl::zed::Mat mdepth = zed->retrieveMeasure(DEPTH);
float* zBuffer = (float*) mdepth.data;
z =zBuffer[i+j*mdepth.step/sizeof(float)];

Despite the fact the "code 2" is giving more accurate result but sometimes cause a segmentation fault .(i am sure of the validity  of the i,j which is being indexed).
my other question , how you do this indexing in both cases (code 1 and code 2) ? where i can learn how you made it ?
finally, what's the best and more accurate way to retrieve pixel (X,Y,Z)? with which ZED params ?
Thanks !

@sl-braun
another question,
which camera is the depth is calculated right or left ? as reference

Hi,
@eagletran : Use zed->retrieveMeasure(XYZBGRA) instead of zed->retrieveMeasure(XYZRGBA); Red and Blue channel seems flipped.
@yahiafarghaly

depth/Point cloud is mapped on the left image/sensor (Left is the reference)
for XYZ extraction, I would do like this :

0<i<width
0<j<height
          sl::zed::Mat cPointCloud= zed->retrieveMeasure(XYZRGBA);
          float x = ((float*)((uint8_t*)cPointCloud.data+j*cPointCloud.step))[4*i];
          float y = ((float*)((uint8_t*)cPointCloud.data+j*cPointCloud.step))[4*i+1];
          float z = ((float*)((uint8_t*)cPointCloud.data+j*cPointCloud.step))[4*i+2];
          float rgba =((float*)((uint8_t*)cPointCloud.data+j*cPointCloud.step))[4*i+3];

Or , For depth :
        sl::zed::Mat data= zed->retrieveMeasure(DEPTH);
        float* ptr_line_data = (float*)((int8_t*)data->data +  j * data->step);
        float dist = ptr_line_data[i];

Basically, it's the "j" line extraction (using the step)  and then take the "i" column of the Mat.
Best,
/OB/

Hi Oliver,
As you recommended, I upgrade my SDK to 1.2, but there may be some problems with the new SDK:

The VGA resolution never get to ~100Hz, it is very slow instead (~55Hz - ~75Hz). It works at ~60Hz correctly. (these numbers are in live-view mode, so no storage bottleneck effects). The SDK 1.1 can reach ~100Hz.
When I record the video with RAW mode, and play back (720 x 60Hz), the framerate is NA.

Can you tell me if these are bugs or intentional implementation? Do you have any suggestion to get the camera work at VGA 100Hz?
Thank you.
Best,
Vu

Hi,

With DepthViewer or Explorer? (I mean with or without depth processing)
the RAW mode does not contains a recorded framerate (using timestamp). Only the lossless and lossy mode.

Best,
OB

Hi Oliver,
I mean the Explorer only, without Depth processing.
Best,
Vu

Hi,
It comes from the OpenGL rendering. In the new SDK1.2, the default gl format is sync to the display refresh rate (probably 60Hz).
To "remove" that clamp, go into your NVIDIA control panel (right click on the desktop) >> manage 3D settings. In the list, you will find an entry called "Vertical sync" : Set it to off to override the application parameters. You should reach 100Hz then.
(Assuming you are still on Windows, but same settings are available under Linux)
Best,
/OB/

Hi Oliver,
Thank you for your support. It works at 100Hz now.
Best,
Vu

Closing the issue, since it started to drift... ;)


RNNParamsSavable breaks when there is more than one RNN

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):  Ubuntu 16.04
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.3.0
Python version: 3.5
Bazel version (if compiling from source): 5.2
CUDA/cuDNN version: 6.0
GPU model and memory:  Tesla K80, 12gb
Exact command to reproduce:

    sess = tf.Session()
    cell1 = cudnn_rnn_ops.CudnnGRU(1, 10, 10, input_mode="linear_input")
    sz = cell1.params_size()
    params1 = tf.get_variable("rnn1/rnn", sess.run(sz))
    save1 = cudnn_rnn_ops.RNNParamsSaveable(
        cell1, cell1.params_to_canonical, cell1.canonical_to_params,
        [params1], name="rnn1/rnn")
    tf.add_to_collection(tf.GraphKeys.SAVEABLE_OBJECTS, save1)

    params2 = tf.get_variable("rnn2/rnn", sess.run(sz))
    save2 = cudnn_rnn_ops.RNNParamsSaveable(
        cell1, cell1.params_to_canonical, cell1.canonical_to_params,
        [params2], name="rnn2/rnn")
    tf.add_to_collection(tf.GraphKeys.SAVEABLE_OBJECTS, save2)
    sess.run(tf.global_variables_initializer())

    saver = tf.train.Saver()
    saver.save(sess, "save")

Describe the problem
An exception is thrown:
"ValueError: At least two variables have the same name: rnn1/rnn"
This works if there is only one RNN. If I remove the name parameters the model will save, however only one of the two RNNs will have their weights/biases saved to the checkpoint as individual tensors, so it will break if I try to restore two CudnnCompatibleGRUCell RNNs from this checkpoint.
As far as I can tell there is some kind of name clobbering happening within tf.Saver, maybe because RNNParamsSaveable set op to none for the super class.

@protoget, could you take a look at this issue?

RNNParamsSaveable is deprecated. Please look out for the new cudnn_rnn layer APIs which takes care of cudnn opaque variable saving.


Improve the UI of footer of Campus Leaders page

Describe the bug
It looks a little unappealing. I would like to work on this improvement.
Expected behaviour
It should be a little dark blue and alignment of its components should be proper.
Screenshots

Additional context
I'm a GSSOC'23 contributor. Please assign me this issue.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@SahityaRoy  Please assign me this issue


Per workspace terminal history

Users often have multiple workspaces on the go, with very different projects. When switching between workspaces after restarting VS Code, the terminal history is shared, which can be frustrating as when switching to a project that hasn't been worked on for a while, the user then has to plough through a lot of history that is irrelevant to that project.
Proposal:
Per-workspace terminal history.
Each workspace saves it command line history individually, so only commands entered for that project are accessible.
I have done several searches and haven't found a similar feature request. Apologies if I missed it.

We can't support this sort of thing generically as we just launch the process and configuring the shell too much would be risky. You may be able to configure this yourself though, eg https://unix.stackexchange.com/a/204928

Entirely fair enough, and thank you for the handy tip!


Prepare release with Gherkin 8

Summary
This are minor changes to ensure Cucumber-ruby still works with gherkin 8 for the next release of the gem.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] I've added tests for my code
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

and the CI fails as it's using the gem and not the master version for cucumber/cucumber :)

@vincent-psarga the Gemfile will pick up gherkin from GHERKIN_RUBY if it's defined.
Could you set that in .travis.yml and point it to the read-only mirror at https://github.com/cucumber/gherkin-ruby ?


heart chamber task hangs when saving segmentations

I am currently running experiments using the heart chambers task on a set of NIfTI files (using python API). It works fine for most cases, but for files larger than ~ 200 MB, it hangs while saving, sometimes for at least 20 minutes (or sometimes just crashes).I tried compressing the files to .gz but im still getting the problem with the exact same files. Are there any solutions to this issue? or are just hardware constraints ? im running it on ubuntu 22.04.

This typically happens when you run into RAM limits. You need more RAM or a smaller image.


Merge feature/add yaml config to develop

Coverage remained the same at 100.0% when pulling 918b1634fdaa28a1e3d3900464f18cbb0faa2047 on feature/add-yaml-config into 8eb5173e739fa8f931f7bd8440fc968918517160 on master.

Coverage remained the same at 100.0% when pulling f3e0bbfc4fb0473d45c052858e09f20c86136075 on feature/add-yaml-config into 8eb5173e739fa8f931f7bd8440fc968918517160 on master.

Coverage remained the same at 100.0% when pulling 242089b45b336764c94ce6ad1dfc0fd3ce0d7751 on feature/add-yaml-config into 8eb5173e739fa8f931f7bd8440fc968918517160 on master.


Replace Multimap in InputMap with a HashMap<Action, SmallSet>

Should be much faster, easier to work with, and will enforce uniqueness.

Also makes insertion idempotent!


Feature/add database objects metrics

Scope & Purpose
This PR adds the following internal metrics for tracking the number of in-memory database objects:

arangodb_objects_databases: number of database objects in memory

arangodb_objects_collections: number of collection objects in memory

arangodb_objects_views: number of view objects in memory

[ ] :hankey: Bugfix

[x] :pizza: New feature

[ ] :hammer: Refactoring

[x] :book: CHANGELOG entry made

[x] :muscle: The behavior in this PR was manually tested

[ ] :computer: The behavior change can be verified via automatic tests

Backports:

[x] No backports required

Testing & Verification

[x] This change is a trivial rework / code cleanup without any test coverage.

http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/12465/

Obsoleted


calico not getting installed

I get the following error after enabling calico
calico_version = "3.9"
install_calico = true
==========Error=============
module.oke.null_resource.install_calico[0] (remote-exec): unable to recognize "calico-policy-only.yaml": Get http://localhost:8080/api?timeout=32s: dial tcp <IP_ADDRESS>:8080: connect: connection refused
==========Error=============
I do not see calico; Here is the get pods output.
NAMESPACE     NAME                                    READY   STATUS    RESTARTS   AGE
kube-system   kube-dns-autoscaler-7c97bb6dfb-w9tpd    1/1     Running   0          24m
kube-system   kube-dns-bdb5cb4bd-9rfdk                3/3     Running   0          17m
kube-system   kube-dns-bdb5cb4bd-gjg75                3/3     Running   0          17m
kube-system   kube-dns-bdb5cb4bd-t9nml                3/3     Running   0          24m
kube-system   kube-flannel-ds-2hzcr                   1/1     Running   1          17m
kube-system   kube-flannel-ds-bscwq                   1/1     Running   1          17m
kube-system   kube-flannel-ds-tntk4                   1/1     Running   1          17m
kube-system   kube-proxy-58s86                        1/1     Running   0          17m
kube-system   kube-proxy-cx86z                        1/1     Running   0          17m
kube-system   kube-proxy-dblh4                        1/1     Running   0          17m
kube-system   kubernetes-dashboard-768d48d6c9-xfjjr   1/1     Running   0          24m
kube-system   proxymux-client-<IP_ADDRESS>               1/1     Running   0          17m
kube-system   proxymux-client-<IP_ADDRESS>               1/1     Running   0          17m
kube-system   proxymux-client-<IP_ADDRESS>               1/1     Running   0          17m

Found out the root cause of this issue.
Admin node did not have copy of kubeconfig in $HOME/.kube that was causing this problem.

That's odd, let me check the dependencies. It is possible that because of the switch to run this on the admin node, the explicit dependencies got messed up.

This problem was caused by #110, hence closing this issue.


Ldd ldc 3

ignore LdcEqualsLdd and let all kernel support ldc != ldd.
This change let Tensile inherently support ldc != ldd.
So caller, ex. rocblas, doesn't have to handle ldc!=ldd.
We have compare performance before and after change.

compare performance between before and after this change.
ldd.xlsx

open later


Model files are missing

The following files on README.md are missing. Could you please re-publish the model files? Thank you in advance.
linear_Bayes_params_1.RDa: https://drive.google.com/open?id=11C5EqFloTV_S910ZnzY35ei55vlAzBD0
linear_Bayes_params_2.RDa: https://drive.google.com/open?id=1yBBEy16q-dY9kka42QCoy4KFC_yjqF8D

@sshojiro Sorry for the delayed reply. Due to the policy change of PoLyInfo, we are no longer allowed to publicly share model trained with data from PoLyInfo. I am instructed to take the model down until further instruction. We are working hard to clear the permission issue. However, there is no clear timeline on the re-upload plan for the models. If you are interested in our work, I recommend you to use our new platform called XenonPy, where openly available pre-trained models, including Tg using Polymer Genome data, are available. Sorry for the inconvenience causes.

@sshojiro Sorry for the delayed reply. Due to the policy change of PoLyInfo, we are no longer allowed to publicly share model trained with data from PoLyInfo. I am instructed to take the model down until further instruction. We are working hard to clear the permission issue. However, there is no clear timeline on the re-upload plan for the models. If you are interested in our work, I recommend you to use our new platform called XenonPy, where openly available pre-trained models, including Tg using Polymer Genome data, are available. Sorry for the inconvenience causes.


Add documentation for the AggregateProcessor, change window_duration to group_duration

Signed-off-by: graytaylor0<EMAIL_ADDRESS>Description

Add README documentation for AggregateProcessor
Rename window_duration configuration option to group_duration, as the AggregateProcessor does not have windows, but groups.

Issues Resolved
[List any issues this PR will resolve]
Check List

[x] New functionality includes testing.
[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1026 (3bbb608) into main (32c83db) will decrease coverage by 0.15%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1026      +/-   ##
============================================
- Coverage     91.36%   91.21%   -0.16%
  Complexity      683      683
============================================
  Files            84       84
  Lines          1993     1993
  Branches        168      168
============================================
- Hits           1821     1818       -3
- Misses          132      135       +3
  Partials         40       40

Impacted Files
Coverage Δ

...java/com/amazon/dataprepper/pipeline/Pipeline.java
88.57% <0.00%> (-4.29%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32c83db...3bbb608. Read the comment docs.


Python wrapper doc SDK2.0

Hi! Is there a full doc of the python wrapper available?
I saw the multiple exemples but it could be great to have a detailed list of functions and their parameters.

There is not. However, the built-in help() command works, and the wrapper
is a direct port of the C++ API

Ok. Can you tell me where you call this help function please, I tried from the python wrapper and C++ but it didn't works.
Thanks!

Oh ok! Thanks!


TypeError: login() takes exactly 3 arguments (4 given)

Reinstalled from current HEAD via python setup.py install:
% playmusiccl
Logging in to Google Play Music...
Traceback (most recent call last):
  File "/usr/local/bin/playmusiccl", line 9, in <module>
    load_entry_point('playmusiccl==0.6.1', 'console_scripts', 'playmusiccl')()
  File "/usr/local/lib/python2.7/dist-packages/playmusiccl-0.6.1-py2.7.egg/playmusiccl/__init__.py", line 7, in run
    main()
  File "/usr/local/lib/python2.7/dist-packages/playmusiccl-0.6.1-py2.7.egg/playmusiccl/playmusiccl.py", line 858, in main
    config.get("google_deviceid"))
  File "/usr/local/lib/python2.7/dist-packages/playmusiccl-0.6.1-py2.7.egg/playmusiccl/playmusiccl.py", line 61, in __init__
    self.logged_in = self.__api.login(email, password, device_id)
TypeError: login() takes exactly 3 arguments (4 given)

Config is the same as when I previously installed via pip, which got past this point.

Can you try running python setup --update gmusicapi.

It didn't seem to understand any kind of update command or option. But I removed gmusicapi* from /usr/local/lib/python, ran setup.py install again, and now I get further. Thanks.

My mistake, the flag is actually --upgrade.

And unfortunately it's not mentioned in the setup.py help for the install command, which I did check, or in the top level help.

Looks as if gstreamer0.10-plugins-ugly is the magic package that fixes it for me, though I also installed the lame, alsa and pulseaudio packages.


Docs: Remove duplicate strip section.

The profile.<name>.strip section was listed twice in the config docs. This removes that unnecessary duplicate.

@bors r+

:pushpin: Commit e6d1db06f7a84797662baa9d0b50c521b123949f has been approved by Muscraft
It is now in the queue for this repository.

:hourglass: Testing commit e6d1db06f7a84797662baa9d0b50c521b123949f with merge a879a1ca12e3997d9fdd71b70f34f1f3c866e1da...

:sunny: Test successful - checks-actions
Approved by: Muscraft
Pushing a879a1ca12e3997d9fdd71b70f34f1f3c866e1da to master...


CanvasOnChain: Show the canvas with all the pixel placements #29

Description

Changes on the PixelCanvas contract to fix the intialization script and iimplements a new method to retrieve all the pixels at once.
Presents the pixels retrieved from the contract.
Changes the scaffold config to use hardhat, otherwise the next:type-check would fail because the contract is not deployed yet.
Fixes the deploy script to keep the BatchRegistry deployment
⚠Disclaimer: This PR is based on Abidoyesimze's PR, which is still open, so you will see his changes too.

Additional Information

[x] I have read the contributing docs (if this is your first contribution)
[x] This is not a duplicate of any existing pull request

Related Issues
Closes #29
Your ENS/address:
0x2Bc096A12C5b37F035180aDeF70EB2B88351e5B8

@phipsae done, I've removed scaffold.config form the commit. But because of this I had to commit with --no-verify flag.
In the future it might be interesting to always rely on hardhat locally, instead of optimism, and use .env variables to control scaffold.config's targetNetworks on different environments.

@phipsae done, I've removed scaffold.config form the commit. But because of this I had to commit with --no-verify flag.
In the future it might be interesting to always rely on hardhat locally, instead of optimism, and use .env variables to control scaffold.config's targetNetworks on different environments.

What I meant is, that in this PR there shouldn't be any files from the PR of @Abidoyesimze.
What to you mean? Can't you change easily the network inside scaffold.config .

Thanks! And merged


SPIKE: Serve the static site and app from different storage accounts

Currently the static site and react app are served out of simplereport${ENV}app
Issues:

Deploying to the static site and app touch the same resource and purging more content from the CDN then needed
Locally the app is served from / and in the cloud the app is served from /api. This means that if a direct link is added in development the bug of it not working in our cloud env is not caught until deploying or a keen review

TODO:

[ ] create a new storage account for the react app
[ ] add routing to the new storage account https://app-${ENV}.simplereport.gov/ (We should leave off the env for prod https://app.simplereport.gov/)
[ ] update redirects, yarn build, and deploys to use the new storage account

Expected output:

Spec with a better way to serve out static files.

Time box: no more than 3 days for spec and another day to write tickets

#1772 and related issues would be simpler to solve if we had this.

test event publishing queue is also part of this storage account now. we probably want to keep that in this storage account and move everything else out

long term vision: react app, static site, and queue uploader to RS are in different containers. This will enable us to do blue/green deployment, automated rollback, single-button rollback, etc

We haven't had a product need for this in some time, so closing this issue. Will reopen if it becomes necessary again.