Pipeline Parallelism GPT2 Example

Hey I was trying to run the Megatron-LM code with your 3D parallelism. For pipeline parallelism, I unrolled the GPT-2 into an nn.Sequential like model and followed the AlexNet example to tweak the training script accordingly. But it keeps going out of memory for me even for a "train_micro_batch_size_per_gpu" of 1 in the 1.5bn model. I've disabled data & model parallelism and was running it on 8 GPUs with the partition method as "type:TransformerLayer".
I also see references to a "GPT2ModelPipe" in the code but I couldn't find it in the examples. Could you release this example for Megatron?

Hi @nitikasaran68, thank you for your interest in DeepSpeed! We're working on releasing a full example and tutorial. There have just been some initial tasks to address first, such as improved examples and some interface simplifications.

Hey, can you help me reproduce your GPT-2 3D parallelism results here (figure 5)
I'm unable to fit a batch size of even 1 with a single gradient accumulation step with the 4pp x 4dp setting on 16 V100s with 16GB memory. If I try 8pp and 2dp, a micro batch size of 1 fits for not more than 4 micro batches.
Here's how I'm wrapping the gpt2 model:
model = deepspeed.pipe.PipelineModule(layers=model.to_layers(),
num_stages=args.pipeline_depth,
loss_fn=loss_fn,
partition_method="type:GPT2ParallelTransformerLayer",
activation_checkpoint_interval=0)
I've gone through a lot of your code and I don't think I'm passing anything incorrectly but perhaps you could help me out?
I'm using this config:
{
"train_batch_size": 8,
"train_micro_batch_size_per_gpu" : 1,
"steps_per_print": 1,
"zero_optimization": {
"stage": 1,
"reduce_bucket_size": 5000000,
"allgather_bucket_size": 5000000
},
"zero_allow_untested_optimizer": true,
"gradient_clipping": 1.0,
"fp16": {
"enabled": true,
"loss_scale": 0,
"loss_scale_window": 1000,
"hysteresis": 2,
"min_loss_scale": 1,
"initial_scale_power": 13
},
"wall_clock_breakdown": false
}
gpt_options="
--model-parallel-size 1
--num-layers 48
--hidden-size 1600
--num-attention-heads 16
--batch-size 1
--seq-length 1024
--max-position-embeddings 1024
--train-iters 100000
--resume-dataloader
--train-data webtext
--lazy-loader
--tokenizer-type GPT2BPETokenizer
--split 949,50,1
--distributed-backend nccl
--lr 0.00001
--no-load-optim
--lr-decay-style cosine
--weight-decay 1e-2
--log-interval 1
--clip-grad 1.0
--warmup .01
--fp16

Hi @nitikasaran68, try setting activation_checkpoint_interval=1 when you wrap the module. Activation checkpointing is a big memory saver for these kinds of models.

software issue has been solved.


Does not seem to work with VS Version 17.12.0 Preview 5.0

When I click Blitz Search This from the Tool bar, nothing happens. There is no window opening or similar. If there is any way for me to get some logs for this extension, please let me know :)
Thanks for you work! Search looks great from the screenshot! :)

Try and download the installer for executable from
https://natestah.com/download

Any luck with that?

@Natestah Sorry for the late reply, only came around to checking it now. I downloaded the .exe which indeed opens. However through the latest version of the VS Extension nothing happens.

Thank you for trying that, I'll give it a go on 17.20.0 Preview today and see if I can reproduce this.

@TwentyFourMinutes I upgraded to that, and It worked for me.  However I did Identify one case were the command would happily do nothing when there is no active document,  this issue is fixed in 1.94 of the Blitz Search Extension. Please update and see if this resolves for you

@Natestah You nailed it, that was the issue. Thank you so much for your time and the extension :)


Don't validate unexpected files

Validators now have a "can_continue" function/attribute which can be used to skip the rest of the validators for the given file. This means that order of validators defined in the tasks is now relevant.
The way this is used now is to use the smoking gun validator's expected records check to bypass the rest of the validators if we get a file we don't expect. This will happen for c1, c3 cat 1, and c4 cat 1. (C1 and C4 cat 1 already had smoking gun validator so just the order changed. C3 cat 1 now has the smoking gun validator but it only performs the expected files check, not the data validation)

@rdingwell can you review this?


Update wiremock to 2.27.2

Updates com.github.tomakehurst:wiremock from 1.58 to 2.27.2.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.github.tomakehurst", artifactId = "wiremock" } ]

labels: test-library-update

Closing. This will be addressed in a separate PR.


Remove google analytics key in config file

Our conf.py file for Sphinx specifies:
html_theme = 'sphinx_rtd_theme'
html_theme_options = {
  'analytics_id': 'UA-xxxxxxxx-x'
}

The sphinx_rtd_theme documentation claims that this parameter enables google analytics tracking. However https://github.com/rtfd/sphinx_rtd_theme/issues/477 indicates that this parameter has been broken on and off since 2014. It may be fixed in the future by pull request https://github.com/rtfd/sphinx_rtd_theme/pull/411
I propose that we remove the analytics_id in html_theme_options. Let's continue to rely on the analytics code included by RTD hosting itself. This latter code is configurable via the admin panel for our project, and on each deployment RTD injects  into /_static/readthedocs-dynamic-include.js.

More details about the RTD injected tracking code
When un-minified, this is precisely what their code tracks with google:

(function(i, s, o, g, r, a, m) {
    i['GoogleAnalyticsObject'] = r;
    i[r] = i[r] || function() {
        (i[r].q = i[r].q || []).push(arguments)
    }
    ,
    i[r].l = 1 * new Date();
    a = s.createElement(o),
    m = s.getElementsByTagName(o)[0];
    a.async = 1;
    a.src = g;
    m.parentNode.insertBefore(a, m)
}
)(window, document, 'script', 'https://www.google-analytics.com/analytics.js', 'ga');
ga('create', 'UA-17997319-1', 'auto', 'rtfd');
ga('rtfd.set', 'dimension1', READTHEDOCS_DATA.project);
ga('rtfd.set', 'dimension2', READTHEDOCS_DATA.version);
ga('rtfd.set', 'dimension3', READTHEDOCS_DATA.language);
ga('rtfd.set', 'dimension4', READTHEDOCS_DATA.theme);
ga('rtfd.set', 'dimension5', READTHEDOCS_DATA.programming_language);
ga('rtfd.set', 'dimension6', READTHEDOCS_DATA.builder);
ga('rtfd.send', 'pageview');
ga('create', 'UA-xxxxxxxx-x', 'auto', 'user');
ga('user.send', 'pageview');

This shows that they track our page views on both their own analytics account and on the "user" one we specify.

/cc @sumedhpathak @clairegiordano

I don't think the setting in conf.py is even having an effect, so no reason to mess with it. There is a comment near it explaining.


Minor Issues with 1.0.0

I've noticed a few small things in working with the 1.0.0 release. These are all on a Windows 10 machine. In no particular order:

The project file has minor formatting issues. A couple of lines use tabs instead of spaces to indent. A couple of others are just indented too far. I tried running lein zprint to fix the formatting, which leads me to...
lein zprint doesn't seem to like Windows line endings. I think it just threw some sort of exception. I then used vim to create a copy of the file with linux line endings. Then lein zprint <file> appeared to work, but didn't alter the original file. Instead, there was a temporary file left in the project directory with the formatted content.
Many elements use the :none key from the color map. Its default value is :black which means those elements are invisible on console windows with a black background. Since most users will want to customize their color scheme, it's not a big deal. But it's a less than ideal out-of-the-box experience.
As noted in #140, the tests that try to run multiple zprints in the same JVM don't appear to work on Windows
Previously, you could just run lein test from the command line. With the new project file, the leiningen test and repl tasks both require with-profile +expectations in order to find the expectations dependency. Adding back a :dev profile with just the expectations dependency fixes that.
Running czprint-fn on source code that has ##Inf, ##-Inf, or ##NaN will throw an exception. This is due to the rewrite-clj parser, not zprint, but ideally zprint should be able to format any legitimate Clojure source file. There's an old issue, but I'm not sure how active the project is.

Thanks for this excellent library. Pretty printing in color makes the REPL experience so much more productive and enjoyable.

Thanks for all of these issues. I’ll get back to you tomorrow about many of them. I appreciate the time you spent noticing these things. It is easy for me to miss things, and another set of eyes always helps!
More later.

Lots to deal with here...

I forgot to format the project.clj file. I should add that to my notes.

Two problems. The windows line endings I expect (hope) I can diagnose myself. The file handling is going to be a challenge. I use a library to do the file handling in lein zprint. I presume you didn’t get a file with a .old extension?  If so, that’s not a bug. But if not, as I expect, then that’s a problem. I don’t have access to a windows system these days. So that’s going to be a challenge.

Yes, I agree about the black background. I have been around long enough that I grew up with terminals with green letters — orange was a great advance —  so I’m not a dark background person.  I think a style which is nice for a dark background would be a great idea. Probably needs to be based on the color work you did.  I could also see a default that was dark but not black if you color work has something like that available.

I think the issue is where there are two zprints running simultaneously. That is interesting and concerning, since it means that probably does not work on windows either. But I don’t n is what to do about that other than document it.

I was having trouble tearing shadow-cljs so I was looking at my Pom.xml file and found all of my :dev dependencies in there, under “scope test”. Which annoyed me, since as I read the Leiningen docs they aren’t supposed to be there at all.   Probably not a problem, but I wanted to get them out of there.

That said, I just do lein repl and it works fine. Now, I don’t use expectations from the repl, so maybe that’s why.
All of that said, I can see having expectations in a :dev dependency, as that will not annoy me if it shows up in the pom.
 I should write a contributing section of the docs, where I can talk about testing and, I suppose, the entire release process which is somewhat involved.

Yes, rewrite-clj needs some work, and I haven’t been excited about doing it myself.

I agree that zprint should work on anything that is legal Clojure. I’ll have to look at these things and see if there is anything I can do to work around this limitation. If you have some real code that doesn’t work, I would be glad to work with that in addition to my own tiny tests to work on this.

I think adding a contributing section in the docs is a great idea, perhaps with some guidance on submitting Issues and submitting Pull Requests. For example, here I just aggregated everything into a single issue, leaving it to you to split things out if necessary. This was partly because I'm not sure what your preferences are for creating issues.
BTW, for my use case (playing around at the REPL), the items above are at worst a minor inconvenience. But since I noticed, I thought I would pass it along.

I did not get a .old file, but there was a randomly named file with the formatted content in it.

I remember those monitors as well.

I think documenting it is fine. The only reason I even noticed is that I was running the tests to see if I had broken anything. If it's an OS-specific issue, it could also make sense to bypass those tests on Windows.

I usually just do lein repl as well. The only reason it was an issue for me was because I like to use integrant to reload code at the repl. When I ran (reset) it tried to load the testing namespaces and couldn't.

I don't have any real code that uses symbolic values. For exercising czprint-fn, something as simple as (def positive-infinity ##Inf) in a source file reveals the issue. If you call rewrite-clj's parse-string directly, even just (parse-string "##Inf") will do the trick. One possible work-around would be to pre-process source strings before calling parse-string. You already do this for tab expansion. Anyway, if you could transform symbolic values to tagged literals, you could transform them back when printing. For example, if you call (parse-string "#zprint/symbolic-value \"##Inf\"") you'll get a ReaderMacroNode. Its children will contain a TokenNode with zprint/symbolic-value and a StringNode with ##Inf. The transformation function would need to preserve ##Inf in strings and comments, so it's definitely not trivial.

I haven't dealt with all of this, but here are a few things:

For the ##Inf Clojurescript stuff I'm currently planning to wait until rewrite-cljc gets finished, and use that to solve this problem.

The next release of lein-zprint (tomorrow) will have completely different file handling, so let me know if the it works on Windows, if you would.  I think it has a chance of working.

Thanks, I'll keep working on the rest of this.

Several things (numbers bear no relation to previous scheme):

I can't seem to reproduce the problem with Windows line endings.  If I put in a file with \r\n line endings, it seems to format fine (and comes out with \n line endings, which is not great).  I can't get lein zprint to fail, either 1.0.0 or 1.0.1.  So this is going to be a challenge to debug.  Could you perhaps try this with 1.0.1 and maybe send me the file that doesn't work?

lein zprint rename not working -- as I said above, please try this with 1.0.1 and see how the new code works.

I still forgot to format the project.clj file.  I have actually now added that information to my release instructions.  Sigh.

Now you can't run lein test at all, since I converted the tests to expectations/cljc-test which doesn't yet exist in a downloadable release.  I ported expectations/clojure-test to Clojurescript, and the changes got in, but it hasn't yet released.
You would have to build it yourself in order to run the tests.  Moreover, having expectations/cljc-test in the pom.xml file did cause a problem with cljdoc, so now it is broken.  So I have to rework the project.clj file to get it out of the pom.xml.

I haven't dealt with multiple JVM's in tests (though of course the cljs versions of the tests don't do that).  I haven't figured out what to do about that for Window's testing.

I have more info on lein zprint in a Windows environment. With one exception, it works for files with dos, unix, and mac line endings (including renaming the file if :old? is truthy).  The exception is any file that has a comment with a windows-style line ending.  As you mentioned, the resulting file always has unix-style line endings.
After debugging this for a bit, the issue is in the zprint-file-str function.  Here's the simplest input that causes it to fail:
user=> (zp/zprint-file-str ";\r\n" "comment with crlf")
AssertionError Assert failed: (string-of? s r/linebreak?)  rewrite-clj.node.whitespace/newline-node (whitespace.clj:89)

The patch below fixes the issue. First, it splits lines using any of the three eol sequences.  Second, it replaces unix line separators with the host system's line separator in the final output.
diff --git a/src/zprint/core.cljc b/src/zprint/core.cljc
index d4b6f36..90062a2 100644
--- a/src/zprint/core.cljc
+++ b/src/zprint/core.cljc
@@ -1138,7 +1138,7 @@
        ; Make sure to get trailing newlines by using -1
        (let [; If the filestring starts with #!, remove it and save it
              [shebang file-str] (remove-shebang file-str)
-             lines (clojure.string/split file-str #"\n" -1)
+             lines (clojure.string/split file-str #"\r\n|\r|\n" -1)
              lines (if (:expand? (:tab (get-options)))
                      (map (partial expand-tabs (:size (:tab (get-options))))
                        lines)
@@ -1205,10 +1205,13 @@
              out-str (if range
                        (reassemble-range before-lines out-str after-lines)
                        out-str)
-             out-str (if shebang (str shebang "\n" out-str) out-str)]
-         (if (and ends-with-nl? (not (clojure.string/ends-with? out-str "\n")))
-           (str out-str "\n")
-           out-str))
+             out-str (if shebang (str shebang "\n" out-str) out-str)
+             out-str (if (and ends-with-nl?
+                              (not (clojure.string/ends-with? out-str "\n")))
+                       (str out-str "\n")
+                       out-str)]
+         (clojure.string/replace
+           out-str "\n" (System/getProperty "line.separator")))
        (finally (reset-options! original-options original-doc-map)))))
   ([file-str zprint-specifier new-options]
    (zprint-file-str file-str

The second behavior is reasonable, but I think a better choice is to let the user specify the desired line endings in the formatted output. The setting could be named something like :output-eol with a value from the set #{:cr :lf :crlf :host-env :file}.
I could also imagine values from the set #{:unix :linux :mac :macintosh :win :windows :dos}. The first three values choose a target line ending independent of the current OS or selected file. The :host-env value would use the line.separator for the host environment as in the patch above. The :file value would use the same line endings as in the original file.
That last option requires determing the eol style of the input file. That's not too bad if all the endings are the same. Where it gets interesting is if there's a mixture of styles. Here, you could just choose the one used most often or report an error and quit.

I was able to run lein test by cloning kkinnear/clojure-test and adding the following alias to its deps.edn so I could install it locally:
:install
{:extra-deps {deps-deploy/deps-deploy {:mvn/version "0.0.9"}}
 :main-opts ["-m" "deps-deploy.deps-deploy" "install" "expectations.jar"]}

With WSL 2 running Ubuntu 20.04, lein test succeeds. Tests also pass when splitting file strings on any line ending. On Windows, two of the tests that use with-out-str still fail, but I'm submitting a pull request with a fix.

Wow, go you for getting the testing framework working!  That's very impressive!
Thank you for all of your work to figure things out. I'm pleased that the file renaming (which isn't really renaming anymore) now works on Windows.
You have certainly pinpointed the major problem with the incoming line endings, but I'm wondering what zprint should do with strings with line-endings in them when :parse-string-all? is used instead of zprint-file-str.  I'm going to have to look a :parse-string-all and its relationship to zprint-file-str.  I may need to deprecate :parse-string-all?.  Anyway, thanks for finding this comment issue with rewrite-clj and such a nice fix for it.
Good thinking about how the tests would all fail if the strings didn't have \n line endings.
I'm not sure that doing line.separator is going to be a viable option in cljs, though I will check and it is a pleasing thought.  Without having put too much thought into it, I'm thinking that having zprint-file-str do on output what it received on input would be a reasonable thing to do.  Then all of the existing tests would pass, and any new tests could test that new behavior.
There are already so many configuration options, do you really think letting people specify the line endings seems like a worthwhile thing to do?  It probably wouldn't be that hard as long as we are already having to (maybe) change them just to make them like what came in.  Would you actually use that capability?
As you point out, determining what was input isn't trivial, but I would lean toward looking at the first 10 line endings and taking the majority approach there.  Either that or just using majority of the second, third and fourth line endings.  Or something like that.  No way I'm looking at the whole file to figure out the line endings.
Anyway, thank you for continuing to work through these issues, I really appreciate it.

Thank you for the kind words.  It's really no problem. I'm just glad I can make a small contribution to the quality library you've worked so hard to produce.
I think your suggestion of using the same EOL style as the input is actually the most natural behavior and presents the least surprises. We expect the layout to change, but not necessarily the EOL style. No option needed. In the rare case the user wants different line endings, the readily available dos2unix and unix2dos tools make it trivial.
I also think just sampling a handful of line endings works just fine. Most of the time they will all match. The few times I've seen mixed line endings, there was always a clear winner.

I'm working to get the line ending fixes to work across the whole product, and I'm wondering why you added `"\r" to this regex:
+             lines (clojure.string/split file-str #"\r\n|\r|\n" -1)

Do you know of an actual instance of people having simply "\r" as a line ending?  I'm trying to minimize the performance hit in the additional processing around line endings.
Thanks!

I looked into it a little bit. Best I can tell, the classic Mac OS (through version 9) used \r as the line ending. Mac OS X is a Unix and uses \n like other Unix systems. So \r is probably not that relevant for modern systems. That said, it is a legit ending. I think most programming editors (and utilities like Notepad++) can handle all three styles.

The line endings are now preserved in 1.0.2.  Thanks for your help in dealing with these.  Also your help with the colors and dark themes, also in 1.0.2.
You now need to run lein with-profile expectations test again, since without that, the SNAPSHOT version of expectations/clojure-test shows up in pom.xml and then cljdoc doesn't work.  Sigh.

As far as I know, this issue has been resolved.  Thanks again for your help.


Errors in cannon.demo.js for the constraint visualization

Hi,
I stumbled across some errors while playing with the demos.
To reproduce this Errors, go to the constraints demo and activate the option Rendering -> constraints.
The scene will immediately freeze with errors like Uncaught TypeError: Cannot read property 'bi' of undefined.
Here my (quick) solutions for the file cannon.demo.js (v0.6.2):

Line
Code
Issue
Solution

315
var nc = c.equations.normal;
CANNON.DistanceConstraint.equations is an Arrayan got no property .normals
var nc = c.equations[0];

341
var n = c.equations.normal;
CANNON.PointToPointConstraint.equations is an Arrayan got no property .normals
var n = c.equations[0];

347
diffLine.scale.set( -n.penetrationVec.x, -n.penetrationVec.y, -n.penetrationVec.z );
ContactEquation got no property .penetrationVec
Delete line

I hope this helps.
By the way, thanks for this awesome library! :thumbsup:

Rendering the penetration vector can be quite useful, but then you first have to enable it again in the ContactEquation:
- penetrationVec = ContactEquation_computeB_temp3,
+ penetrationVec = this.penetrationVec,

And add it back in the constructor:
- this.ni = new Vec3();
+ this.ni = new Vec3();
+ this.penetrationVec = new Vec3();

Then you can rewrite the rendering code like this:
            // Lines for distance constraints
            for (let ci = 0; ci < world.constraints.length; ci++) {
                const c = world.constraints[ci];
                if (!(c instanceof CANNON.PointToPointConstraint)) {
                    continue;
                }
                for (const equation of c.equations) {
                    const {bi, bj, ri, rj} = equation;
                    const relLine1 = p2pConstraintMeshCache.request();
                    const relLine2 = p2pConstraintMeshCache.request();
                    const diffLine = p2pConstraintMeshCache.request();
                    if (equation instanceof CANNON.ContactEquation) {
                        const {penetrationVec} = equation;
                        relLine1.scale.copy(ri);
                        relLine2.scale.copy(rj);
                        const s = 1 / penetrationVec.length();
                        diffLine.scale.set(-penetrationVec.x * s, -penetrationVec.y * s, -penetrationVec.z * s);
                        makeSureNotZero(relLine1.scale);
                        makeSureNotZero(relLine2.scale);
                        makeSureNotZero(diffLine.scale);
                        relLine1.position.copy(bi.position);
                        relLine2.position.copy(bj.position);
                        bj.position.vadd(rj, diffLine.position);
                    }
                }
            }

I figured that the length of the penetration vector is usually very low (e.g. 1/50), that's why I scale it to 1 for rendering it.


Prepare for azure-security-keyvault-jca 2.8.0 release

Description
Prepare for azure-security-keyvault-jca 2.8.0 release
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.


restic support alibaba cloud OSS

restic should add to support alibaba cloud OSS (Object Storage Service)

Restic supports Alibaba cloud OSS via rclone. Please have a look at the documentation on restic & rclone on how to use them.
IMHO there's no need to add yet another backend to restic.

@dimejo thanks, i'll have a try

Closing this as the support is already available through rclone.


Link broken

Hi, the link in the section "What identity should I install the CoE Starter Kit with?" that has link text "Conditional access and multi-factor authentication in Power Automate" and goes to https://learn.microsoft.com/en-us/power-platform/power-automate/conditional-access-and-multi-factor-authentication-in-flow is broken - it takes you to a 404 page.
I believe the correct link is https://learn.microsoft.com/en-us/troubleshoot/power-platform/power-automate/conditional-access-and-multi-factor-authentication-in-flow

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 3f619bbe-9303-8583-2cb9-a287811cec13
Version Independent ID: 8e256e0c-38fa-f2ca-58db-550bb6e9802a
Content: Set up the CoE Starter Kit - Power Platform
Content Source: power-platform/guidance/coe/setup.md
Service: power-platform
Sub-service: guidance
GitHub Login: @manuelap-msft
Microsoft Alias: mapichle

Thanks for your comment, @dbaars. This issue has been assigned to the doc author for review.

Thank you, looks like the link on the Power Automate site has changed since we authored this page! Made the change locally for our next release targeted for this week.

Fixed yesterday with our latest docs update!


add: dayBeforeMonth ability

Hi,
This PR adds a boolean if anyone wanted the day dropdown to be before the month dropdown.

I added the Added dateformatorder: OrderFormat.YDM param to specify the dateformat


[BEAM-12582] Adding jar packages only to Java environments

Adding jar packages only to Java environments
Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run XVR_Dataflow PostCommit

Run XVR_Spark PostCommit

Run XVR_Direct PostCommit

Run XVR_Flink PostCommit

R: @chamikaramj

Looks like the internal change should suffice.


Add hints for radial chart

it would be great to have hints on radial chart.

The functionality is now available on the 2.0.0beta release, this ticket can be closed.


[cypress] Add integration test for area, pie, world map

Running: explore/visualizations/area.js...
  Area
    ✓ works without groupby (2888ms)
    ✓ works with group by (2259ms)
    ✓ works with groupby and filter (1429ms)
  3 passing (7s)

Running: explore/visualizations/pie.js...
  Pie
    ✓ works with ad-hoc metric (2153ms)
    ✓ works with simple filter (1523ms)
  2 passing (4s)

Running: explore/visualizations/world_map.js...
  World Map
    ✓ works with ad-hoc metric (3701ms)
    ✓ works with simple filter (2159ms)
    ✓ can hide bubbles (1429ms)
  3 passing (8s)

@michellethomas @williaster @conglei

Codecov Report

Merging #5924 into master will increase coverage by 0.1%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #5924     +/-   ##
=========================================
+ Coverage   63.61%   63.72%   +0.1%
=========================================
  Files         386      386
  Lines       23532    23532
  Branches     2621     2621
=========================================
+ Hits        14971    14996     +25
+ Misses       8548     8523     -25
  Partials       13       13

Impacted Files
Coverage Δ

superset/views/core.py
74.22% <0%> (+0.29%)
:arrow_up:

superset/models/core.py
85.09% <0%> (+0.49%)
:arrow_up:

superset/connectors/sqla/models.py
81.13% <0%> (+0.75%)
:arrow_up:

superset/db_engine_specs.py
55.45% <0%> (+2.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42ab175...2a9cff1. Read the comment docs.

Codecov Report

Merging #5924 into master will increase coverage by 0.1%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #5924     +/-   ##
=========================================
+ Coverage   63.61%   63.72%   +0.1%
=========================================
  Files         386      386
  Lines       23532    23532
  Branches     2621     2621
=========================================
+ Hits        14971    14996     +25
+ Misses       8548     8523     -25
  Partials       13       13

Impacted Files
Coverage Δ

superset/views/core.py
74.22% <0%> (+0.29%)
:arrow_up:

superset/models/core.py
85.09% <0%> (+0.49%)
:arrow_up:

superset/connectors/sqla/models.py
81.13% <0%> (+0.75%)
:arrow_up:

superset/db_engine_specs.py
55.45% <0%> (+2.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42ab175...2a9cff1. Read the comment docs.

Codecov Report

Merging #5924 into master will increase coverage by 0.1%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #5924     +/-   ##
=========================================
+ Coverage   63.61%   63.72%   +0.1%
=========================================
  Files         386      386
  Lines       23532    23532
  Branches     2621     2621
=========================================
+ Hits        14971    14996     +25
+ Misses       8548     8523     -25
  Partials       13       13

Impacted Files
Coverage Δ

superset/views/core.py
74.22% <0%> (+0.29%)
:arrow_up:

superset/models/core.py
85.09% <0%> (+0.49%)
:arrow_up:

superset/connectors/sqla/models.py
81.13% <0%> (+0.75%)
:arrow_up:

superset/db_engine_specs.py
55.45% <0%> (+2.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8cb734d...4513053. Read the comment docs.

Codecov Report

Merging #5924 into master will increase coverage by 0.1%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #5924     +/-   ##
=========================================
+ Coverage   63.61%   63.72%   +0.1%
=========================================
  Files         386      386
  Lines       23532    23532
  Branches     2621     2621
=========================================
+ Hits        14971    14996     +25
+ Misses       8548     8523     -25
  Partials       13       13

Impacted Files
Coverage Δ

superset/views/core.py
74.22% <0%> (+0.29%)
:arrow_up:

superset/models/core.py
85.09% <0%> (+0.49%)
:arrow_up:

superset/connectors/sqla/models.py
81.13% <0%> (+0.75%)
:arrow_up:

superset/db_engine_specs.py
55.45% <0%> (+2.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8cb734d...4513053. Read the comment docs.

  (Run Finished)
      Spec                                    Tests  Pass…  Fail…  Pend…  Skip…
  ┌────────────────────────────────────────────────────────────────────────────┐
  │ ✔ dashboard/dashboard_tests.js     18ms      1      -      -      1      - │
  ├────────────────────────────────────────────────────────────────────────────┤
  │ ✔ explore/control_tests.js        00:13      2      2      -      -      - │
  ├────────────────────────────────────────────────────────────────────────────┤
  │ ✔ explore/visualizations/all.js   02:35     58     58      -      -      - │
  └────────────────────────────────────────────────────────────────────────────┘
    All specs passed!                 02:48     61     60      -      1      -

Much better time than before (13mins vs 18mins)


[add-resource] New Plugin/Theme

I would like to add my new plugin/theme to the Grav Repository._Here are the project details: ricardo118/grav-aviary-plugin

Added to GPM, should show up in next build.


Line numbers plugin does not work well with line-wrapping

Not sure whether this is inside the scope of Prism, but I figured it probably doesn’t hurt to ask…
Prism’s themes do not wrap horizontally overflowed lines by default, and the line-numbers plugin sort of takes advantage of this, and creates a span element for each newline in the code. But in some situations (e.g. print materials) it is still more desirable, or even necessary to override this rule (word-wrap: break-word;). Unfortunately this would break the line numbers’ layout:

I’m not even sure how this could be resolved. Maybe it’d be necessary to wrap each line in code in a div, and somehow match the size of the line number elements to their lines. Or even re-format the whole thing as a table. But hopefully Prism can (and is willing to) fix this.

@mortalis13 gave it a try some time ago: #560. It resulted in the creation on this repo: https://github.com/mortalis13/Prism-Wrap-Lines-Test
Note that he wanted each wrapped line to count as only 1. Is that also what you expect or do you expect your example code to display 8 line numbers?

I took some time to play around with his code again. There seems to be a way to achieve the expected result without modifying prism-core, but this can only work for raw text, since highlighted code might have HTML elements spanning multiple lines and my code would break them.
Original behaviour: http://jsfiddle.net/7wL6dtmw/
Modified plugin + css to make it work with wrapped lines: http://jsfiddle.net/7wL6dtmw/2/ (Tested in Firefox and Chrome)
This is far from being perfect. ><

@Golmote Yup, this is exactly what I need. I don’t currently need to highlight things, so I’ll probably go for your modified version. Thanks. :)

Needs compatibility also on <code> width change.

@Golmote :pensive:It does not seem to work properly. When you change the window size, the above problems will occur.

This was finally fixed with #584 merged.


Crypto: synchronous password hashing?

https://github.com/mikecao/umami/blob/5ecaf5587b6bb5968a4b0c53c110bf807dbe9fed/lib/crypto.js#L42-L48
Why would we use hashSync and compareSync in asynchronous functions? They could lead to reduced I/O efficiency.

It's not that big a deal performance wise, but yes, should be sync calls.


Using Accelerate train

Thank you for open this repo.
In example for DDP train:
accelerate config # configure the environment
accelerate launch src/finetune.py # arguments (same as above)

However. No accelerate library found in your repo.
+ from accelerate import Accelerator
+ accelerator = Accelerator()

+ model, optimizer, training_dataloader, scheduler = accelerator.prepare(
+     model, optimizer, training_dataloader, scheduler
+ )
...

These features are already integrated with the Trainer class. We do not need to specify them in the scripts.
https://huggingface.co/docs/transformers/main_classes/trainer#specific-gpus-selection

Thank you for your reponse.


Nifti viewer in browser for mean, std, and snr images

Will allow users to easily add the MNI or aligned MPRAGE brains to a quality control HTML page to see the effectiveness of registration on the brain, and also serve as the best way to visualize quality control while giving users incentive to use the service as this QC would be way more user friendly than that of other services

we have that already: http://viz.neurodata.io/KKI2009v03/
especially, once data are ingested into neurodata,
which should certainly be part of your pipeline.
On Fri, Jun 24, 2016 at 4:39 PM, ebridge2<EMAIL_ADDRESS>wrote:

Will allow users to easily add the MNI or aligned MPRAGE brains to a
quality control HTML page to see the effectiveness of registration on the
brain, and also serve as the best way to visualize quality control while
giving users incentive to use the service as this QC would be way more user
friendly than that of other services
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Upward-Spiral-Science/fngs/issues/21, or mute the
thread
https://github.com/notifications/unsubscribe/AACjcpsPbZR58civbxVkDoClRejMO-ZHks5qPECVgaJpZM4I-G3B
.

--
the glass is all full:  half water, half air.
neurodata.io, jovo calendar
https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York


There should be an easy way to create SSH keys for private repository access

For existing Magento 2 projects, we load many of our modules via composer from private bitbucket repositories. We use SSH authentication for these repositories, and the docker container running composer must be authenticated to pull down the repositories.
Now I realize if I run composer locally it will sync the vendor directory and all is well, but I am not always on a machine which has all of the platform requirements installed, which can cause issues.
I was able to get this working after messing around for a while, but basically I had to generate ssh keys in the phpfpm container, change ownership of the /var/www directory to app user, and store the keys in /var/www/.ssh because that is where composer attempts to find them and where it stores the bitbucket.org host in the known_hosts file.
Once all of the permissions and ssh keys were in place it ran nicely, but I feel this should be in the configuration of the docker containers. Preferably there would be an option to generate new SSH keys by default, or specify local SSH keys to be synced from the host to the container.
My experience with all of the above is fairly limited, so any other suggestions are welcome, this is just how I solved an issue when working with an existing project.
Cheers!

Seems specific to your use-case, but should be able to do this with volume mounting ssh keys. Nothing we can do here to aid with this, sorry.

you can sync your host private key to container .
under
services:
  app:
    volumes: &appvolumes

put thhe line below.
- ~/.ssh/id_rsa:/var/www/.ssh/id_rsa:delegated
@markshust it will be nice a nice feature.

This isn't a common ask, and seems like you already have what you need. I don't think there is anything we need to do here? 🤷♂

yes your are right  ,But  I face this issue all time on docker because am working with privates repo sometimes.
May be a comment line in dev.yml will help  ?.
By the way your docker is best for magento 2 dev . Thanks for your good work. ✌🏻

@vaseemishak sure thing, since this has been requested a few times I think that will work :) adding the comment to the next release

I just had this problem and found this solution. Really so good idea Vaseem!
Thank you so much! 👍

Also experienced this as an issue. We use a number of git repositories for modules and it is not exactly an uncommon occurrence. I think it might be worth documenting this workaround at the very least.


[snapshot] add additional ipad classifiers.

fixes: https://github.com/fastlane/fastlane/issues/6822
proof:

Looks great, thanks!!  🚀


Models load failed

Hi, error during build
ERROR character model load failed: /../ character.glb: not glTF magic
ERROR voxel materials load failed: /../materials/00001_dirt.png: Invalid PNG signature.
these files are in the folders

Looks like you forgot to do git lfs pull

Looks like you forgot to do git lfs pull

Yes, thank you. It would be great to add this mention in README.md

It is mentioned in build instructions on wiki, which is the first thing linked in README
https://github.com/Ralith/hypermine/wiki


acbuild should remove /etc/resolv.conf

I'm trying to use ubuntu-14.04-server-cloudimg-amd64-root as the base for my ACI, but I am unable to resolve host names when using acbuild run. It looks like systemd-nspawn is trying to populate /etc/resolv.conf, but is getting confused because that file is a symlink. I think it should be safe for acbuild to ensure that that file doesn't exist by the time that systemd-nspawn is run.

Related: https://github.com/appc/acbuild/issues/142

hrm, this means that the same ACI exhibits different behaviour when run under acbuild or under rkt (which doesn't execute the apps directly with systemd-nspawn), which is unfortunate

Think it'd be better to have a stance of just doing what rkt does? I wonder if crawfords image is also broken under rkt with respect to dns.

On Dec 18, 2015, at 12:35, Jonathan Boulle<EMAIL_ADDRESS>wrote:
hrm, this means that the same ACI exhibits different behaviour when run under acbuild or under rkt (which doesn't execute the apps directly with systemd-nspawn), which is unfortunate
―
Reply to this email directly or view it on GitHub.

I still haven't run it under rkt yet. Someone is supposed to help me with another issue after lunch...

https://github.com/coreos/coreos-baremetal/issues/67#issuecomment-188879911 is partly related.

Any clarity on correct behavior? I was tripped up earlier when an acbuild image (development) had an /etc/resolv.conf, while its counterpart docker image converted to an ACI (production) did not.
(Until I can have rktnetes use an ACI directly, I'm stuck paying attention to the what happens between image transforms)

Any update on this? I've run into issues where an /etc/resolv.conf was created & stored by acbuild and I had to add an acbuild step to remove it for the produced container image to function correctly.

I can put together a PR to remove this, and it would be like 3 lines, but it would always remove it regardless of whether or not the command run was intending to populate it.
In other words, it would be hard to tell the difference between the following commands:
acbuild run -- echo "hello, world!"
acbuild run -- /bin/sh -c 'echo "nameserver <IP_ADDRESS>" > /etc/resolv.conf'

Should I just always blindly remove it if it didn't exist before the run command, and rely on users having to use the acbuild copy to populate /etc/resolv.conf ?

How feasible is it to remove it before run and then if it exists after the command has finished, leave a marker so that it isn't removed in future invocations of run. Maybe that's to complex to easily reason about.

If I indiscriminately remove it before run it will always exist after run, because systemd-nspawn creates it if it doesn't exist, so I don't think that's a viable solution.

I think requiring acbuild copy is probably fine.

This also has the caveat that if you do an acbuild copy to populate /etc/resolv.conf, and then do another acbuild run command, your populated /etc/resolv.conf will be deleted for you.

when I run "acbuild --debug copy /etc/resolv.conf /etc/resolv.conf"
and then "acbuild --debug run -- cat /etc/resolv.conf" then I see the unmodified version.
any solution for this ?

@kfirufk An easy solution for you should be to swap the order of those, or add --engine=chroot to the run command.

ok.. i use docker://alpine:edge image. i noticed that when the image is first started, /etc/resolv.conf is configured properly to google's dns. after a minute or two /etc/resolv.conf is replaced and now contains 'nameserver <IP_ADDRESS>' which causing me problem cause there is no dns server in the lopback device as far as i understand.
running 'acbuild --engine=chroot run -- cat /etc/resolv.conf' shows the correct /etc/resolv.conf. does it mean that I need to run every commad with '--engine=chroot' ? and @dgonyeo, you wrote to swap the order of those... what do you mean exactly ?

Every time you do an acbuild run command without a --engine=chroot it will modify your /etc/resolv.conf. If you want to set what's in the /etc/resolv.conf file inside your container during the build, make sure you copy it in after you've done any acbuild run commands (unless you put the --engine=chroot flag on all run commands)

If you want to make the host's resolv.conf available for for reading, you could bind-mount read-only the hosts's /etc/resolv.conf in to the container.
This does not solve the write / copy problem.

so yeah.. adding --engine=chroot to all run commands resolved the issue.


Suggest: '() <$ x' -> 'void x'

Closes: https://github.com/ndmitchell/hlint/issues/1265

Due to Control.Monad.void being part of the Haskell 2010, I left void in Monad rules & as Control.Monad.void, not as Control.Functor.void.

Thanks!


ENI: calculateExcessIPs not consider minAllocate when usedIP+preAllocate less minAllocate and availableIPs greate than MinAllocate

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
I found the operator released more ips than expected, and it needs to allocate ips in the next sync cycle.
It causes unnecessary calls to cloud APIs and the Instability of the IP resource pool.
In this situation need to Release ip:
availableIPs:      61,
usedIPs:           29,
preAllocate:       20,
minAllocate:       60,
maxAboveWatermark: 0,

release ip count is availableIPs-usedIPs-preAllocate=61-29-20=12. But According to the design it only excess 1 IP availableIPs -minAllocate=61-60=1
Next resync cycle after 12 IP is released,  it is needed ip:
availableIPs:     49,
usedIPs:          29,
preAllocate:      20,
minAllocate:      60,
maxAllocate:      0,

need ip count is minAllocate-availableIPs=60-49=11
The test code for this situation is incorrect. But the comment 1 excess is right.
https://github.com/cilium/cilium/blob/3a23173e76cf5e379ff58d01cd74c5a2a57765ae/pkg/ipam/node_test.go#L42
Cilium Version
master--all release version
Kernel Version
no
Kubernetes Version
no
Sysdump
No response
Relevant log output
No response
Anything else?
https://github.com/cilium/cilium/blob/3a23173e76cf5e379ff58d01cd74c5a2a57765ae/pkg/ipam/node.go#L327-L352
Code of Conduct

[X] I agree to follow this project's Code of Conduct

/assign @wu0407

According to the documentation, minAllocate is only used in the beginning, and not during operations:
https://docs.cilium.io/en/stable/network/concepts/ipam/eni/#ipam-eni

spec.ipam.min-allocate
The minimum number of IPs that must be allocated when the node is first bootstrapped. [..] After reaching this watermark, the PreAllocate and MaxAboveWatermark logic takes over to continue allocating IPs.

So I think the behavior is correct. If you want to avoid excess API calls, you can instead set MaxAboveWatermark maybe?

Although the maxAboveWatermark setting can help reduce excess IPs, it does not solve the problem of excess API calls. Let's say you set the maxAboveWatermark to 5, and the result shows that there are 7 excess IPs (12-5=7).
Because maxAboveWatermark is a static setting, the stat of used IPs is dynamic, The aim is to prevent having an excessive number of IPs that exceed the actual need.

I'm not sure I understand - is the problem that maxAboveWatermark is an absolute value, not a percentage? Or is it that maxAboveWatermark is not respected when releasing excess IPs?

maxAboveWatermark is a related value. I expect not to release IP and allocate IP when usedIPs+preAllocate not over minAllocate + maxAboveWatermark and used IPs less than minAllocate + maxAboveWatermark. It is difficult to achieve this goal by setting maxAboveWatermark.
In this case, we can set maxAboveWatermark to 11 to avoid releasing IP and allocate IP.  But when usedIPs is 28, it cannot meet this goal. It will release 2 IPs and allocate 1 IP.
availableIPs:      61,
usedIPs:           28,
preAllocate:       20,
minAllocate:       60,
maxAboveWatermark: 11,

The release IP count is availableIPs-usedIPs-preAllocate-maxAboveWatermark=61-28-20-11=2
After the release 2 IPs
availableIPs:      59,
usedIPs:           28,
preAllocate:       20,
minAllocate:       60,
maxAboveWatermark: 11,

The needed IP count is minAllocate-availableIPs=60-59=1

I see, thanks. According to the documentation, minAllocate should not be used to compute the needed count after the initial set of allocations. But I do see that this seems no longer the case now that we have IP release. It seems the minAllocate logic is indeed broken at the moment.
Thanks for explaining this to me, I now agree that this is a bug. We should however also update the documentation to document the new formula.


In some cases, indented blocks are not displayed.

Hi @luoyansusi , could you create a public Notion page that reproduce this issue?
https://vue3-notion.vercel.app/page/{ public page showing indented blocks not display }

Close due to no minimal reproduction or pageId provided


eqMac 1.5.0 weird audio hang issues

While 1.5.0 did successfully fix any type of full-on hang (where eqMac has to be force quit and restarted), it seems to have changed them rather than fix.
As in, quite often now when I as much as stop a YouTube video (and thus I am no longer listening to any audio), and then resume the video, eqMac lags for a couple seconds and I hear no audio at all, until it eventually fixes itself and I can hear audio again.
If this only happened on longer pauses, sure, no worries.
But this happens sometimes even when I just press "next song" in Spotify. If there is as much as the smallest period of no audio, it can happen.
Very, very rarely eqMac fixes itself into turning my audio into chipmunk noises as well, or other distortion. Usually this fixes itself within a couple seconds. Once, however, the only way to fix it was to keep pausing and re-playing music until eqMac hung and fixed itself.

Thanks for the report, I have experienced this as well but very very rarely and it's hard to replicate, but I will keep trying.

Hi, Just checking in, is this still an issue for you?

I am closing this issue since I am not hearing back from you. If you are still having this issue, you are welcome to reopen and comment on it.


Provide better visibility creating ASP.NET apps in README

The current README doesn't do a good job of directing readers how to create ASP.NET apps using the dotnet-framework Docker image. We need to fix that.
At the very least, we should direct people to: https://github.com/Microsoft/dotnet-framework-docker-samples/tree/master/aspnetapp. I think we can do more than that.
/cc @kendrahavens, @glennc

This was addressed by the most recent samples refactoring - https://github.com/Microsoft/dotnet-framework-docker/issues/128


require CLI argument seems to be ignored

Operating System: macOS High Sierra 10.13.3
Node Version: 8.9.3
NPM Version: 5.5.1
webpack Version: 4.8.3
webpack-serve Version: 1.0.2

This issue is for a:

[x] bug
[ ] feature request
[ ] modification request

Code
CLI Command
$ webpack-serve --require babel-register --config config/webpack.babel.js

webpack.config.js
import path from 'path';
// snip...

Expected Behavior
Webpack config loads after babel-register loads.
Actual Behavior
require argument seems to be ignored & error is thrown:
✖ ｢config｣: An error ocurred while trying to load /Users/mhuggins/Development/nos/client/config/webpack.babel.js
              Did you forget to specify a --require?
/Users/mhuggins/Development/nos/client/node_modules/@webpack-contrib/config-loader/lib/load.js:84
    throw new LoadConfigError(e, configPath);
    ^

LoadConfigError: Unexpected token import
import path from 'path';
^^^^^^

SyntaxError: Unexpected token import
    at NativeCompileCache._moduleCompile (/Users/mhuggins/Development/nos/client/node_modules/v8-compile-cache/v8-compile-cache.js:226:18)
    at Module._compile (/Users/mhuggins/Development/nos/client/node_modules/v8-compile-cache/v8-compile-cache.js:172:36)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (/Users/mhuggins/Development/nos/client/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
    at loadJs (/Users/mhuggins/Development/nos/client/node_modules/@webpack-contrib/config-loader/node_modules/cosmiconfig/dist/loaders.js:8:18)
    at Explorer.loadFileContent (/Users/mhuggins/Development/nos/client/node_modules/@webpack-contrib/config-loader/node_modules/cosmiconfig/dist/createExplorer.js:226:12)
error An unexpected error occurred: "Command failed.
Exit code: 1
Command: sh
Arguments: -c webpack-serve --require babel-register --config config/webpack.babel.js
Directory: /Users/mhuggins/Development/nos/client

How Do We Reproduce?
Try to --require babel-register along with an ES6 webpack config file.
Additionally, try replacing "babel-register" with literally any string, and it doesn't seem to be attempting to require the file regardless of name.

Duplicate of #140

Is there any update on this? I am still getting this error?

@nisargrthakkar this goes for the entirety of Github and every project on it: If you don't see replies / tags / associated issues and/or pull requests listed in an issue - There have been no updates. So there's no need to ask.
https://github.com/webpack-contrib/webpack-serve/commits/master - a fix was released in v1.0.4 on June 7th.


LwM2M: Add support for DTLS + bugfixes

The following PR establishes the needed changes to support DTLS for the LwM2M library in Zephyr.
As well as fixes a few bugs related to these changes.

V2 changes per @rbtchc review: https://hastebin.com/ipubaconuz.cpp

Rebased on today's master [no code changes]

Perhaps I should split this up into 2 PRs:

coap config changes
DTLS support addition for LwM2M

Rebased on today's master and added "Open Source Foundries" copyright to changed files.  Diff: https://hastebin.com/fejefahine.diff

Split this into 3 PRs with these 2 being required prior to this working:
https://github.com/zephyrproject-rtos/zephyr/pull/5339
https://github.com/zephyrproject-rtos/zephyr/pull/5340

Added @dbkinder for doc changes.

Added issue for other samples not checking return value: https://github.com/zephyrproject-rtos/zephyr/issues/5387

Updated to return error value from mbedtls_ssl_conf_psk() in lwm2m_engine.c
Diff: https://hastebin.com/ipukuwiloj.cpp

Updated w/ doc change per @dbkinder
Diff: https://hastebin.com/wayocowemu.sql

Rebased on today's master.


fix(deps): Moving bch-token-sweep from external lib to package.json

:tada: This PR is included in version 1.7.2 :tada:
Your semantic-release bot :package::rocket:


[Question] Where can I get s390x and ppc64le build

I can see pipeline do support ppc64le nad s390x build based on makefile https://github.com/tektoncd/pipeline/blob/master/Makefile#L58-L64 , where can I get those build?

Thanks @vdemeester , this is really helpful, let me close this one and trace in #856


More CDK docs

Direct link to preview: https://medplum-www-git-cody-more-cdk-docs-medplum.vercel.app/docs/self-hosting/install-on-aws

Coverage: 93.96%. Remained the same when pulling bfa4a7518701404cd6f5fe2f91cb33bba622bde6 on cody-more-cdk-docs into b55dc480f58ffc7c9be40584115e3a2948920cfc on main.


Improve doc for adding functions to Stan

The documentation for adding functions to Stan has not been updated to reflect the fact that the math repository is split out into its own repository.
Fix

[ ] There should be one section for the stan-dev/math repository,  and one for the stan-dev/stan repository.
[ ] Math repository section:

[ ] Describe how functions are organized.
[ ] Describe how includes function.
[ ] Describe what basic unit tests are needed.

[ ] Stan repository section:

[ ] Point to function_signatures file.
[ ] Describe how signatures are structured a little (especially useful for permutations of types).
[ ] Describe what basic unit tests are needed.

Thanks for writing this out.
On Aug 28, 2015 3:42 PM, "Krzysztof Sakrejda"<EMAIL_ADDRESS>wrote:

The documentation
https://github.com/stan-dev/stan/wiki/Contributing-New-Functions-to-Stan
for adding functions to Stan has not been updated to reflect the fact that
the math repository is split out into its own repository.
Fix

There should be one section for the stan-dev/math repository, and
one for the stan-dev/stan repository.
Math repository section:

Describe how functions are organized.
Describe how includes function.
Describe what basic unit tests are needed.

Stan repository section:

Point to function_signatures file.
Describe how signatures are structured a little (especially
useful for permutations of types).
Describe what basic unit tests are needed.

—
Reply to this email directly or view it on GitHub
https://github.com/stan-dev/stan/issues/1595.

This is ancient and not relevant, closing.


Made Logger.Level.style and .name public

Closes https://github.com/vapor/console/pull/82

Hey @calebkleveter, you just merged a pull request, have a coin!
You now have 478 coins.


Broken link from search

I was searching for software testing topics but the link to the Introduction to Unit Testing was not working.
The link it sent me to was: https://train.oxrse.uk/material/HPCu/technology_and_tooling/unit_testing/introduction
There actually doesn't appear to be an introduction chapter anymore?

This is caused by
https://github.com/UNIVERSE-HPC/course-material/issues/29
But I'm not sure if there are other similar issues on the material repo.


feat: add hirosassa/tfcmt-gitlab

I added hirosassa/tfcmt-gitlab.

Thank you for your contribution!

Released. https://github.com/aquaproj/aqua-registry/releases/tag/v3.28.0


Start preparing day 1 materials

(This is a "Work in progress" pull request which is not ready to be merged yet; just up as a placeholder so you can see what I'm doing.)
Starts preparing for the workshop by adding:

[x] Updated README.md
[x] README for Day 1
[x] Preliminary prerequisites list for Day 1
[x] Add very basic "implicit solvent hydration free energy from compound name or SMILES" script for them to run the instant hydration session starts; will use reduced number of iterations
[x] Adjust default number of iterations to be smaller for implicit solvent (for efficient running in class)
[x] Add content discussing molecule preparation (Jupyter notebook)
[x] Add information on general running of implicit/explicit hydration calculations, what Yank is doing for these and why, a bit of explanation of YAML with links to docs
[x] Add instructions on/discussion of analysis of hydration calculations, instructions to analyze implicit solvent calculations

I'm moving the other items I'm adding to a separate PR so I can start merging some of the content.

@evohringer - feel free to take a look at the Session1.ipynb to see if it's along the lines of what you're thinking. This would be for the first afternoon working session.
This is basically a presentation I would show, but then it has interactive cells they can play around with, and then should be able to analyze hydration free eergy calculations (I still have a couple details on that, including some sample data, to fill in).

Hi David,
I think Session1.ipynb is perfect. It might be interesting to provide more concrete examples of molecules for the hydration free energy run. We could then divide the students in groups to see what they obtain. Something like a polar (phenol), an apolar (benzene) and a flexible polar molecule (Alcohol).
Thanks a lot for all your work!!


[glad] update to glad2

Library name
glad
New version number
glad2
Other information that may be useful (release notes, etc...)
Hi, this is not exactly a version update, but it doesn't look like the glad package in vcpkg uses the default branch glad2 of the glad repository. The functionality and include seem different between the master (glad) and glad2 branch. Can this somehow be updated please?

@LilyWangLL do you think it would be better to introduce a new package called glad2, similar to glfw3? Or is it possible to switch the branch from master to glad2 for this package, although that could break existing project?

I noticed that the upstream has been releasing version 2.0 since last year and is no longer updating version 0.1. Therefore, my point is to upgrade glad to version 2.0, and I am trying to update glad locally.

@jasperdrescher
I found that glad2 no longer provides complete CMake build files, the related links: https://github.com/Dav1dde/glad/issues/244. Glad2 only has one file CMakeLists.txt under the subdirectory cmake, which just provides functions. If we want to update glad to glad2, we need to write new CMakeLists.txt. I'm not very familiar with the use of glad, would you be interested in doing this?

This repo uses glad2 with CMake and it uses the glad_add_library function for CMakeLists: https://github.com/jasperdrescher/HelloOpenGL/blob/main/CMake/glad.cmake. If needed I can try to update the package.

@LilyWangLL I'll try to update the port to glad2 on a fork and see how far I get today 😄

@LilyWangLL trying to add a CMakeLists.txt to the glad port and testing that locally is more complicated/vague than I expected. It would be appreciated if you could help out, as I'm not as familiar with Vcpkg and the required steps when modifying a port.

You could submit a PR to do this, our team and other contributors could give the suggestions for this update.


Encode semantics of Session::keepalive_send in return type

Issue
The current implementation of Session::keepalive_send returns Result<u32, Error>. Users might suspect that if they receive an Ok(seconds) they should act on it and send another keep alive message after seconds has elapsed.
However, this is not the case if keepalive is disabled for the session since Session::keepalive_send will return Ok(0) indicating that keepalive is not set. Here's where that happens in libssh2:
https://github.com/libssh2/libssh2/blob/a88a727c2a1840f979b34f12bcce3d55dcd7ea6e/src/keepalive.c#L62-L66
    if(!session->keepalive_interval) {
        if(seconds_to_next)
            *seconds_to_next = 0;
        return 0;
    }

Possible Solutions
The most lightweight thing we can do is update the documentation for Session::keepalive_send so that it clearly indicates that users will get an Ok(0) back if keep alive is not set.
Alternatively, if we're alright with a breaking change for the next minor version release we could change the return type to Result<Option<NonZeroU32>, Error>, or Result<Option<Duration>, Error> and set the option to None if the C api returns zero. I think this is the best way to prevent accidental misuse of the API, but it does require a breaking change.

This is an excellent idea. I will make a point to include the breaking change in the next minor release.
Until then, I will update the documentation.

If this is still on the list to be implemented, i would suggest encoding the exact responses in the type system, instead of just using an Option. Something like this:
enum KeepaliveResponse {
  Seconds(u32), // Alternatively this could also be a `Duration`.
                // We should make sure this is the same type as
                // the `interval` passed to `set_keepalive` imo.
  NoKeepAliveSet,
}

fn keepalive_send(&self) -> Result<KeepaliveResponse, Error>


Docs for cranelift-codegen are not published in docs.rs

As discussed here, the cranelift-codegen crate only has docs for version 0.66, not 0.67 and 0.68 which are the more recently-published versions. The docs.rs logs are not available to determine why this failed.

Also, the Cranelift documentation pages themselves could use a review; some of the information is outdated (e.g., here)

docs.rs currently has docs published from Cranelift v0.83 (the most recent release); closing issue.


Model agnostic

quick refactor to allow for easily adding new model providers. This removes our usage of the openAIChat provider.
However trying with the new anthropic models is leading to errors due to bad api endpoints being used in llm-api. @matildepark we should sync on what to do about llm-api because it is becoming a headache

also addresses #21

Did you pull before starting the branch? The server folder should be included in this too I think.

good catch. thanks :)

thanks matidle :)


HTTP-request infinite loading (unclosed socket) due to spawn child processes using multiprocessing

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the FastAPI documentation, with the integrated search.
[X] I already searched in Google "How to X in FastAPI" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to FastAPI but to Pydantic.
[X] I already checked if it is not related to FastAPI but to Swagger UI.
[X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
import multiprocessing
import os
import time

import uvicorn
from fastapi import FastAPI

app = FastAPI()
processes = []

def keep_alive_process():
    while True:
        print(f"process {os.getpid()} is alive")
        time.sleep(1)

@app.post("/start")
async def start_processes():

    for i in range(4):
        process = multiprocessing.Process(target=keep_alive_process,
                                          args=())
        processes.append(process)
        process.start()

    return {'status': 'started'}

@app.post("/stop")
async def stop_processes():

    for process in processes:
        process.kill()
    processes.clear()

    return {'status': 'stopped'}

if __name__ == '__main__':
    uvicorn.run('main:app', timeout_keep_alive=10, log_level='trace')

Description
I don't really understand what's going on with http requests. When I started child processes and uvicorn timeout_keep_alive timed out, I tried hitting "stop" in the browser and got infinite loading and no HTTP connection log. But if I try to click other buttons or refresh the page, it works and I get two responses.
Netstat shows that socket bind on port 60862 is opened, but uvicorn log:
TRACE:    <IP_ADDRESS>:60862 - HTTP connection lost.
https://stackoverflow.com/questions/73940652/http-request-infinite-loading-unclosed-socket-due-to-spawn-child-processes-usi
Operating System
Linux
Operating System Details
No response
FastAPI Version
0.85
Python Version
3.10
Additional Context
No response

Swagger image


Prepare for GA

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.Core.Expressions.DataFactory


Add external downloaded jquery file to seed project

Hi I am using your seed for our project. I have implemented the Graphical interaction using Highchart in your seed. Its run fine till rc4 version of your seed then I have just migrated to stable version then also it is working on development mode when I have trying to deploy on the server then it showing that external downloaded files with following error: ``

and I have added that all five files into index.html
  <script src="app/qeasily/jqueries/jquery.min.js"></script>

  <script src="app/qeasily/jqueries/jquery.countdown360.js"></script>

  <script src="app/qeasily/jqueries/highcharts.js"></script>

  <script src="app/qeasily/jqueries/jquery.raty.js"></script>

  <script src="app/qeasily/jqueries/draggable-points.js"></script>

  <meta name="description" content="">

  <meta name="viewport" content="width=device-width, initial-scale=1">

and below screenshot of index.html file

For support requests you can use gitter or take a look at project's the wiki page in order to get more complete idea on how the it works.


warn on cookbook self-deps

cookbooks with self-deps:
name 'foo'
depends 'foo'
are useless and can waste cycles in the depsolver (particularly in
the non-solution case), and likely limit the possible choices of
depsolvers that we could pick at some point to replace gecode.
filtering out the self-dep here will prevent the chef server depsolver
from seeing the self-dep and needing to do the work to strip it.
also warn the user so that they can remove the self-dep.
in the future this will be a hard error.
foodcritic also has a [PR in for a] rule about removing these.

@danielsdeleo here you go first commit fixes up knife logging
@chef/client-core needs review

:+1:

:+1:


Ingress creation on pre-v1.18 clusters results in "path is undefined"

Creating an Ingress object on pre-v1.18 clusters (when pathType was expanded in the API) results in path is undefined. This behavior occurs in the following scenario:

Have a v2.5.2 Rancher instance installed
Have a v1.17 or earlier Kubernetes cluster being managed
Attempt to create an Ingress inside of the Cluster Explorer using the form method.
Input any path, request host, service, etc.

Observe that upon clicking of the Create button, an error of path is undefined is yielded.

This also appears to occur in v1.18 clusters. I upgraded the v1.17 cluster from the previous comment to v1.18. I observed the presence of the Prefix dropdown, but the result is the same:

The bug fix is verified with Rancher server rancher\rancher:master-head version e4ce376 and rancher\rancher:v2.5-head version 3463f5a
Steps:

Have a v1.17 or earlier Kubernetes cluster being managed

Attempt to create an Ingress inside of the Cluster Explorer using the form method.

Input any path, request host, service, etc.
Create ingress

Result:
Ingress is created without error.


style(frontend): modify string for missing date of date group

Motivation
Instead of showing Undefined for missing timestamps, we show No Date Available.

[OISY-130]


feat: enforce usage of nullish check utils

Motivation
An useful rule is to enforce the usage of utils isNullish and nonNullish from repo @dfinity/utils for a simple nullish comparison.
isNullish
Incorrect
param === undefined

param === null

param === undefined || param === null

Correct
isNullish(param);

nonNullish
Incorrect
param !== undefined

param !== null

param !== null && param !== undefined

Correct
nonNullish(param);

Future Improvements
The rule will be improved to recognize nullish check even with the following cases (and their negation counterparty):
const param: NonBooleanType | undefined | null;

if (param) {...}

param ? ... : ...

param && ...

param || ...

Tests
Create specific tests for the new rule.

Closed in favour of:

PR #32

because I changed my account


Support to use docker-compose profile

Issue
#

:beetle: Bug Fix
No

:trophy: Feature
Yes

:pencil: Refactor
No

:open_book: Documentation
No

:warning: Break Change
No

Description
Add --profile arg to kool start to Use Profiles With Compose

Notes

I don't know how we can create a test for this case, if someone can help it would be good.

@fbrissi for testing it...you only need to create a test case to check if the flags were added to the docker-compose internal command

@fbrissi kool is prepared to handle commands with the following structure:

[docker/docker-compose] [command] [arguments]

But this PR is trying to make the following structure to work:

[docker/docker-compose] [options] [command] [arguments]`

@fabriciojs do you have any thoughts? I was thinking about adding another array to command struct...other than Cmd and Args...to handle these options...

@fbrissi kool is prepared to handle commands with the following structure:

[docker/docker-compose] [command] [arguments]

But this PR is trying to make the following structure to work:

[docker/docker-compose] [options] [command] [arguments]. Ex.: docker-compose --profile profile-1 up ...

@fabriciojs do you have any thoughts? I was thinking about adding another array to command struct...other than Cmd and Args...to handle these options...

@danielsuguimoto I believe that the current Command interface is working well - right now we have several places that rely on the Args themselves to handle all sort of options. I don't think we have a need to differentiate them right now. I left a question for @fbrissi to understand why he chose to have that new interface/specific solution, instead of simply leveraging the Command.AppendArgs

@fabriciojs the profile option is not a docker-compose up argument...but it's a docker-compose one, it should be added before up command...I understood that @fbrissi wants to put the args before the command, instead of appending it after the command...


Thermal calibration updates

fixes problem introduced in https://github.com/PX4/Firmware/pull/6497/commits/24eb72e1ce199909c99c464758bcab178aa1451e
notifies system after param reset before starting temp calibration
more refactoring to reduce code duplication

Follow-up to #6497
@priseborough

I tested this with a Pixhawk and Pixracer board and was able to successfully calibrate the gyros, accelerometers and baro. Both boards were then re-powered and a commander legacy gyro and accel cal was performed. The boards were then re-powered and the corrected sensor values were checked and found to be OK.

Same for me: I also tested successfully the temp & legacy calib on a pixracer.
This is ready for review and/or merge.


fix: run all analyzer when no --filter given

Closes #412
📑 Description

refer to the #412
✅ Checks

[x] My pull request adheres to the code style of this project
[ ] My code requires changes to the documentation
[ ] I have updated the documentation as required
[x] All the tests have passed

Before fixing:
k8sgpt analyze --filter=VulnerabilityReport| wc -l
18
k8sgpt analyze | wc -l
18

Test Result with newly complied binary:
k8sgpt analyze --filter=VulnerabilityReport| wc -l
18
k8sgpt analyze | wc -l
102

ℹ Additional Information

Active filters are set in a file, by removing this check those settings will be ignored.
@matthisholleville thoughts?

Active filters are set in a file, by removing this check those settings will be ignored. @matthisholleville thoughts?

I think the check will do in https://github.com/k8sgpt-ai/k8sgpt/blob/main/pkg/analysis/analysis.go#L192.

Exactly, @AlexsJones. This is incorrect because we would only run all filters if "--filter" is not set and the "active_filters" in the configuration file are empty.

@matthisholleville  @AlexsJones .
Sorry that I didn't catch you:  per https://github.com/k8sgpt-ai/k8sgpt/issues/412,  when after k8sgpt  integration activate trivy ,  there's no way to run the "ALL" analyzers UNTIL you deactivate trivy (that will uninstall trivy.)
Is it a best user experience ?
my understanding of the UX design is  as below, but seems it's not. Do you consider the way about below ?

user  can integrate any plugin then activate them( install the plugin)
then user can get analysis for a plugin only ( ex:  --filter=VulnerabilityReport)
then user can get analysis for ALL analyzers ( ex: by default, no--filter given. )

k8sgpt  integration activate trivy , there's no way to run the "ALL" analyzers UNTIL you deactivate trivy (that will uninstall trivy.)
When you do this, you should be able to run k8sgpt analyze
And all analyzers work
The filters should also show something like this
filters list
Active:
> NetworkPolicy
> Node
> Pod
> CronJob
> PodDisruptionBudget
> VulnerabilityReport
> StatefulSet
> Deployment
> Service
> Ingress
> PersistentVolumeClaim
> ReplicaSet
> HorizontalPodAutoScaler
Unused:

If that is not working currently then there is a bug.
I am not really following what the issue is here.
It's completely possible for k8sgpt analyze to show only a few results if you've disabled analyzers with filters.

k8sgpt integration activate trivy , there's no way to run the "ALL" analyzers UNTIL you deactivate trivy (that will uninstall trivy.)
When you do this, you should be able to run k8sgpt analyze And all analyzers work
The filters should also show something like this
filters list
Active:
> NetworkPolicy
> Node
> Pod
> CronJob
> PodDisruptionBudget
> VulnerabilityReport
> StatefulSet
> Deployment
> Service
> Ingress
> PersistentVolumeClaim
> ReplicaSet
> HorizontalPodAutoScaler
Unused:

If that is not working currently then there is a bug.
I am not really following what the issue is here.
It's completely possible for k8sgpt analyze to show only a few results if you've disabled analyzers with filters.

Yes, it's a bug, Alex. @AlexsJones
Detail  reproduce steps with 0.3.2.
(1) before integration, everything is fine.
#  k8sgpt  filter list
Active:
> PersistentVolumeClaim
> Service
> StatefulSet
> CronJob
> Pod
> ReplicaSet
> Ingress
> Node
> Deployment
Unused:
> PodDisruptionBudget
> NetworkPolicy
> HorizontalPodAutoScaler

(2) activate trivy
#  k8sgpt  integration activate trivy
2023/05/17 08:48:54 creating 1 resource(s)
2023/05/17 08:48:54 CRD clustercompliancereports.aquasecurity.github.io is already present. Skipping.
2023/05/17 08:48:54 creating 1 resource(s)
.......
2023/05/17 08:48:56 release installed successfully: trivy-operator-k8sgpt/trivy-operator-0.13.2
Activated integration trivy

(3) ALL others are REMOVED from Active filter list, only Trivy left (I believe it's unexpected and the root cause )
I think the expected behavior  is adding VulnerabilityReport into the original Active List instead of exclude them all.
#  k8sgpt  filter list
Active:
> VulnerabilityReport (integration)
Unused:
> Pod
> ReplicaSet
> PersistentVolumeClaim
> Node
> Deployment
> Service
> Ingress
> StatefulSet
> CronJob
> HorizontalPodAutoScaler
> PodDisruptionBudget
> NetworkPolicy

(4) k8sgpt  analyze only shows Trivy reports.
#  k8sgpt  analyze
AI Provider: openai

0 mspider-system/mspider-gsc-controller-7f99674b5f(Deployment/mspider-gsc-controller)
- Error: critical Vulnerability found ID: CVE-2022-31045 (learn more at: https://avd.aquasec.com/nvd/cve-2022-31045)

1 mspider-system/mspider-work-api-795bb68cb9(Deployment/mspider-work-api)
- Error: critical Vulnerability found ID: CVE-2022-31045 (learn more at: https://avd.aquasec.com/nvd/cve-2022-31045)

2 skoala-system/hive-d5bc5bb58(Deployment/hive)
- Error: critical Vulnerability found ID: CVE-2022-31045 (learn more at: https://avd.aquasec.com/nvd/cve-2022-31045)

3 skoala-system/sesame-686ffc4bc9(Deployment/sesame)
- Error: critical Vulnerability found ID: CVE-2022-31045 (learn more at: https://avd.aquasec.com/nvd/cve-2022-31045)

I think the problem actually in pkg/integration/integration.go
when the first time invoking Integration.Activate()
        // HERE: the "active_filters" in viper(`~/.config/k8sgpt/k8sgpt.yaml`) is blank, because ONLY `coreAnalyzerMap` so far.
        activeFilters := viper.GetStringSlice("active_filters")
        // So the `activeFilters` will be BLANK.

        // then  the `mergedFilters` will only have `VulnerabilityReport ` after run `k8sgpt  integration activate trivy`
        mergedFilters := append(activeFilters, integrations[name].GetAnalyzerName())

close, @matthisholleville already fix it in https://github.com/k8sgpt-ai/k8sgpt/pull/432/files

close it , https://github.com/k8sgpt-ai/k8sgpt/pull/441 has fixed it


Bootup and shutdown module broken again

It no longer won't open (nothing happens when I try to open).
Not even empty editor like the last time it broke down.

It should not be happening. I can not reproduce it.
What is your Webmin version? What is the server OS? If you open the browser, do you have any console errors?

Strange now it works for me too.
Earlier I was struggling with a malfunctioning VM (running in KVM via cloudmin).
Since it was the fw, I had no internet connection at the time and that particular module just refused to open.
Now I tried it again, and the module opens just fine.

I will close this for now, as you can no longer reproduce it.
You are welcome to re-open it, in case issue re-appears and there'd be a way to reproduce it.


Test all functions for SD59x18

Use foundry to test all the functions for SD59x18 using the inputs from the hardhat tests.

Thanks a lot for this.

I know that technically it is cheaper to use uncheckedAdd, but we can just use add for the test values, to enhance readability. The gas reports are fudged by the test set arrays, anyway.
I know that it's more verbose, but I would prefer to write 0.000000000000000001 instead of 1.

Thanks a lot for this.

All tests that revert should be refactored to start with testCannot.
Great idea to name the functions in the test contracts like minPermitted and maxPermitted rather than using the minimum and maximum boundary values (e.g. -59.794705707972522261). But you should have also renamed the testing branches, for consistency purposes.
We should change the testing branches to match these names.
I know that technically it is cheaper to use uncheckedAdd, but we can just use add for the test values, to enhance readability. The gas reports are fudged by the test set arrays, anyway.
I know that it's more verbose, but I would prefer to write 0.000000000000000001 instead of 1.


Provide data/event flow examples and facilities

Currently ReactApplicationRoot and EcmascriptEngine support pushing events from native/C++ code into JS/React via the dispatchEvent mechanism.
dispatchEvent is a function registered/created against the global __BlueprintNative__ object by EventBridge.js.
It seems that a lot of apps and client code would benefit from a generic "event based" framework or pattern to communicate between C++ and JS.
For example, it would be nice to both send an event (consisting of an ID and payload/args) from C++ to Javascript and from Javascript to C++.
I might wish subscribe to a "parameterChanged" event in JS and also trigger/signal that same event back to C++.
Is this something that blueprint could/should offer via EcmascriptEngine/ReactApplicationRoot?
I could imagine a complimentary function to ReactApplicationRoot::dispatchEvent along the lines of ReactApplicationRoot::Listener::handleEvent.
i.e.
virtual void ReactApplicationRoot::Listener::handleEvent(const juce::String& eventType, const juce::NativeFunctionArgs& args) = 0;

(ReactApplicationRoot::Listener is purely there to illustrate a point, could be a lambda callback or whatever)
This might require some change or "fleshing out" of EventBridge  to support a publish and subscribe like interface.
The big initial question feels like "Should Blueprint provide this or should examples be given with data-flow patterns/implementations left to the users of Blueprint?"
@nick-thompson I'd love to hear your ideas around this area. I'm going to implement some sort of event based communication in my app over the coming days so will let you know if I come up with any interesting stuff.
For me it would be super useful just to have an inverse dispatchEvent function that I can call from JS and listen to in C++. We have plans to build a generic data model/controller system off some sort of simple pub/sub mechanism.
I don't think that dictates design patterns to users too strongly as there are plenty of other approach people could use but it does give a tool that various patterns can be built off of.
This could also work really nicely with a ValueTree where events are id'd to match a value tree element identifier. A component owning ReactApplicationRoot could then listen to events from JavaScript to update its value tree. Said component could also listen to the value tree itself and dispatch the same event to JavaScript, allowing straight forward 2 way data binding with an application's data model. This could allow super straight forward undo/redo behaviour in apps managed with ValueTreeState and avoid a tonne of if/else comparators on event identifiers. The events received could just map straight to ValueTree::setValue(event.id) etc. Should be interoperable with AudioProcessorValueTreeState to provide a solution for both plugins and larger standalone apps. We could probably provide some sort of helper class for this and offer it as the basic "data flow" tool. People could choose whether or not to use it in their applications.
Dave Rowland did a great talk on managing large scale apps with juce::ValueTree. Might make a nice reference.

@nick-thompson, in the middle of implementing two-way event dispatch at the moment as a test so I can open up a PR if you like the sound of this idea?

@nick-thompson, FYI I've got this working locally and so far seems pretty  good.

Hmmm a lot to think about here, thanks for starting the conversation @JoshMarler.

The big initial question feels like "Should Blueprint provide this or should examples be given with data-flow patterns/implementations left to the users of Blueprint?"

I think that's exactly the right place to start. I suppose my initial stance is that Blueprint itself shouldn't take a stance on data flow patterns, because every app is different and everbody has their own opinions on data flow patterns. In React you see a ton of Flux/Redux, but there are plenty of people who prefer an RxJS style approach, for example. However, I think Blueprint should provide some set of facilities for easily implementing your data flow pattern of choice.
EventBridge was an early idea and very much only a partially formed idea. It came about as I was building a Flux pattern for Remnant in Blueprint, where my one true data store was the AudioProcessorValueTreeState, all changes were broadcast as events into js land, and from js there were no direct ways to manipulate state, only to issue "actions," as in the Flux model, such as "parameter value update." That's why you only see the C++->JS direction. That's nice for Flux and for audio plugins, but I can imagine many apps even with a similar data flow pattern where you'd want the "one source of truth" living in js land and broadcasting change events to C++. So, thinking about that more now, I think your idea for building out EventBridge into a true pub/sub system that can go either direction is the right move.
I think that also provides a good opportunity to implement something I've done a few times but not quite yet integrated fully: a coherent way to throttle events going from C++->JS. Right now the GainPlugin calls into js on every parameter value change, and this event comes in all the time during a drag gesture or during host automation. I implemented what I called a "throttle map" (https://github.com/nick-thompson/blueprint/blob/master/blueprint/core/blueprint_ThrottleMap.h) that I think we could integrate into the EventBridge from C++->JS so that as you dispatch events you can choose if they should be throttled or not. Need to spend a minute on what that API might look like from the ReactAppRoot side of things.
Anyway, then carrying on to the automatic ValueTree 2-way sync– I think that's an awesome idea, but my initial reaction on it is that that falls into the "taking a stance on data flow patterns" category (at least, 2-way data binding is often quite at odds with the Flux model). Which makes me think that maybe it shouldn't be part of blueprint core (but absolutely should be a well documented example at least). I need to spend a little more time on it, and if you have some code ready I'd love to see it, that'd be quite helpful. Either way, I think you're totally right that many people would find that super helpful, and it would apply to many apps.
Hope that was a sensible ramble. I'll think on it some more, revisit tomorrow and look more closely at the API design side of this

Hey Nick,
Thanks for the discussion. Yep fair enough, I think I agree. I'd be really pleased just to see the two way pub sub mechanism there. I have that working at the moment but it's just a bare bones implementation for me to leverage. I'll think about the API some more today and maybe do the usual "open up a PR for Nick to fix :-)"
I've been down the redux route a couple of times in the past and found it "a bit wordy". I've more experience with Angular and at the time (when I was actually writing some JavaScript) it had a fairly opinionated rxjs approach which I quite liked.
I'm still very much in the exploratory phased with this stuff so I'll probably try a few different patterns. Flux is a thing I should probably catch back up on.
A value tree like system appeals as I'm working with a pretty big application with LOTS of events. Being able to bind events straight into some sort of "model" interface where each event ID maps to a model/ValueTree property key could save us a lot of repetitive boiler plate.
However, totally possible this approach is going to be too fixed in some cases and result in too much runtime type checking.
I suspect I'll also need multiple options so I agree, Blueprint probably shouldn't prescribe any ValueTree based approach, it should.just provide thing like the EventBridge to facilitate any pattern users might take. ValueTree binding will probably just make a nice example in the docs.

I'd be really pleased just to see the two way pub sub mechanism there. I have that working at the moment but it's just a bare bones implementation for me to leverage. I'll think about the API some more today and maybe do the usual "open up a PR for Nick to fix :-)"

Yea same, I think the two-way pub sub is definitely the right move. Please do share some code when you have it! I'll spend some time thinking on the API myself as well. I feel like it could really be quite simple

I've been down the redux route a couple of times in the past and found it "a bit wordy".

Definitely. I'm personally not a huge fan of redux, but so many people are. I often write an as-minimal-as-possible Flux pattern (as in the GainPlugin example) for my state management for smaller apps.

A value tree like system appeals as I'm working with a pretty big application with LOTS of events. Being able to bind events straight into some sort of "model" interface where each event ID maps to a model/ValueTree property key could save us a lot of repetitive boiler plate.

Right, yea, I can imagine many cases where this auto-enumerating event binding makes tons of sense. And this morning as I'm thinking through it more, while I do still think that Blueprint should probably only provide the EventBridge to facilitate this, this kind of value tree event dispatcher/synchronizer might make a great add to the "Extras" library in #53. Especially if our extras library will include bindings to juce-specific things like the juce::Slider, bindings to a juce::ValueTree seems to make a lot of sense there. I'm picturing something like:
class MainComponent   : public Component
{
public:
    //==============================================================================
    MainComponent()
        : treeSync(appRoot, state) // Automatically sets up listeners to call `appRoot.dispatchEvent` and receive events from js
    {
        addAndMakeVisible(appRoot);
        setSize(400, 300);
    }

    ~MainComponent();

    //==============================================================================
    void paint (Graphics&) override;

    void resized() override
    {
        appRoot.setBounds(getLocalBounds());
    }

private:
    //==============================================================================
    juce::ValueTree state;
    blueprint::ReactApplicationRoot appRoot;
    blueprint::extras::ValueTreeSynchronizer treeSync;

    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (MainComponent)
};

Curious if that's along the lines of what you've already written? I'm obviously hand-waving but the thought is that, since this lives on top of the EventBridge layer, it could very much be something that people take off the shelf if they want it, or leave it if they want some specific Redux/Flux/RXJS/Etc

@nick-thompson, Yes! Totally. The extras module sounds like a great place for a ValueTreeSynchronizer. I'm literally working on things like this right now.
I've got a very simple two way EventBridge in js which is working just fine but needs a littler more thought. I'm experimenting with the dispatchEvent interface at the moment so I'll have something to send over soon. I'm also building some JS components which can hook into this whole system so the "extras" module feels like something that could definitely start taking shape soon.
Two-way EventBridge:
import EventEmitter from 'events';

class EventBridge extends EventEmitter {
  constructor() {
    super();

    this.emit = this.emit.bind(this);

    this.emittingFromNative = false;

    // TODO: Make constructor arg/configurable?
    this.setMaxListeners(30);

    __BlueprintNative__.dispatchEventFromNative = (eventType, ...args) => {
      this.emittingFromNative = true;
      this.emit(eventType, ...args);
      this.emittingFromNative = false;
    }
  }

  // Override EventEmitter.emit()
  emit(eventType, ...args) {
    if (this.emittingFromNative) {
      super.emit(eventType, ...args);
    }
    else {
      __BlueprintNative__.dispatchEventToNative(eventType, ...args);
    }
  }
}

const _singletonInstance = new EventBridge();
export default _singletonInstance;

Can totally send over a PR soon with the pub/sub stuff. ValueTree utility class might take me a little longer :-) We aren't actually using ValueTree at the moment so I've bound events to our own "data-model" type of class but the pattern follows the exact same principles that binding to a ValueTree would. I'm experimenting with a couple of different data-model interfaces and ValueTree is the next on my list to tackle so if I get something nice together I shall send it over to you.
Cheers.


How to use WebGrab+Plus output file (guide.xml) as source of EPG in CumulusTV

Hi, I have created an EPG file using WebGrab+Plus (http://webgrabplus.com) which is in .xml format (attached), Which I am unable to use with CumulusTV. However, the same file is very much working with other IPTV players and I am able to see the EPG over there.
Did anyone tried WebGrab+Plus with CumulusTV before? Am I missing anything?
Pls change the file extension to .xml before using.
guide sample.txt

That's a format that is not supported.


dash app running on http://<IP_ADDRESS>:8080 is not working please help me

I just try explainx in python or django also
from explainx import *
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

X_data, Y_data = explainx.dataset_heloc()
X_train, X_test, Y_train, Y_test = train_test_split(X_data, Y_data, test_size=0.3, random_state=0)
# Train a RandomForest Model
model = RandomForestClassifier()
model.fit(X_train, Y_train)
explainx.ai(X_test, Y_test, model, model_name="randomforest")

it gives a running server like :- dash app running on http://<IP_ADDRESS>:8080
but when i click on this server, this server is not working
image for reference:-

please help me.

That both server is nor working
Running on http://<IP_ADDRESS>:8080/
Debugger PIN: 624-926-050
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Running on http://<IP_ADDRESS>:62731/
Debugger PIN: 330-309-028
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off

can you please help me

That both server is nor working
Running on http://<IP_ADDRESS>:8080/
Debugger PIN: 624-926-050
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Running on http://<IP_ADDRESS>:62731/
Debugger PIN: 330-309-028
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off

can you please help me

Our team Is working on the issue. This will be resolved soon.

Our team Is working on the issue. This will be resolved soon.

Thanks Sir . I will wait for Your response @tawabshakeel

Thanks Sir . I will wait for Your response @tawabshakeel

@tawabshakeel Sir, Is there any solution about that server i also try it into python or jupyter notebook its not working yet.
please help me

@epayush , Our team is working on the Issue . This will be resolved by the end of this week.