Watching is slow

Watching has slowed down a lot using the new npm scripts. Below are samples of the time watching takes. 29 seconds is a very long wait for livereloading. The gulp task took about 3 seconds. I'm hoping there's still a lot to improve here!
Are there some config settings we can override to improve speed, until the default situation improves?
Start watch: 46 seconds
[11:18:01]  ionic-app-scripts 0.0.23
[11:18:01]  watch started ...
[11:18:01]  build dev started ...
[11:18:01]  clean started ...
[11:18:01]  clean finished in 76 ms
[11:18:01]  copy started ...
[11:18:01]  typescript compiler started ...
[11:18:02]  lint started ...
[11:18:02]  copy finished in 459 ms
[11:18:19]  typescript compiler ready in 17.15 s
[11:18:19]  bundle dev started ...
[11:18:20]  lint finished in 18.64 s
[11:18:45]  bundle dev finished in 26.51 s
[11:18:45]  sass started ...
[11:18:48]  sass finished in 2.35 s
[11:18:48]  build dev finished in 46.11 s
[11:18:48]  watch ready in 46.33 s

Livereload: 29 seconds
[11:38:25]  typescript compilation started ...
[11:38:29]  typescript compilation finished in 4.58 s
[11:38:30]  bundle dev update started ...
JS changed:   www/build/main.js
JS changed:   www/build/main.js.map
[11:38:54]  bundle dev update finished in 23.23 s
[11:38:54]  sass started ...
[11:38:54]  sass finished in 211 ms

Please see https://github.com/driftyco/ionic-app-scripts/pull/44
This should ease up the watches memory usage and speed it up a bit.

I'm not sure about that as it seems to fix the issue where multiple instances are running concurrently, but my watch is this slow even when I save a single change (thus only creating a single instance).

Closed via https://github.com/driftyco/ionic-app-scripts/pull/44
This should be fixed in the next release of app scripts.

Even with the above fix and ionic-app-scripts 0.0.27, it takes ~4 seconds for TS to compile in watch and ~8 seconds to bundle it, in general it takes about ~15 seconds to reload. With the previous browserify + gulp build chain it took only 2 seconds. Is there any way to speed this up?

@mhartington Speed has definitely not changed with the new version. I'm still at 25-30 seconds for a watch reload and ~50 seconds for the initial build. Gulp did it in about 3 seconds for the same project.

@mhartington,  I agree with @Manduro  and @Kobzol  that this This issue is not related to #44 and should be re-opened. The slowness does not seem to be related to a bug, but rather the bundling, AoT compilation and tree shaking performed by the introduction of Rollup.  These steps slow down livereloading substantially.    You are adding 10-15 seconds to the build time to shave less than a second off browser load time, and it doesn't make sense to do that for a build that is going to be loaded only once before being overwritten by the next small change.
I think the ideal solution would be to provide a "fast build" config option that skips these optimizations (Aot Compilation, tree shaking) and is tuned to provide the shortest wait between the start of the build and completion of the browser reload, and ionic serve should use that configuration.  Hopefully that will bring us back to the super-fast reloads we were enjoying with previous beta releases.

Hi, unfortunately I must confirm that this is a huge problem. In beta11, with no AoT and rollup live reload was working great, just few seconds and changes were compiled to the script. Now, with the application that contains lots of .ts files and dependencies it takes almost 60 seconds to build .js output file. It seems, that not only rollup builds slows down everything, but typescript compiles all files as well, which is obviously invalid - only files, that were changed should be compiled, that is the way tsc --watch is working.
@mhartington, @manucorporat please reopen.

@mhartington Thanks for reopen. Below some logs on npm run watch:
-- initial build --
[15:31:10]  ionic-app-scripts 0.0.27
[15:31:10]  watch started ...
[15:31:10]  build dev started ...
[15:31:10]  clean started ...
[15:31:11]  clean finished in 473 ms
[15:31:11]  copy started ...
[15:31:11]  typescript compiler started ...
[15:31:11]  lint started ...
[15:31:12]  copy finished in 1.50 s
[15:31:14]  lint finished in 2.87 s
[15:31:33]  typescript compiler ready in 21.70 s
[15:31:33]  bundle dev started ...
[15:31:58]  bundle dev finished in 25.76 s
[15:31:58]  sass started ...
[15:32:02]  sass finished in 3.97 s
[15:32:02]  build dev finished in 51.91 s
[15:32:02]  watch ready in 52.11 s
-- after one letter change in .ts file --
[15:34:40]  typescript compilation started ...
[15:34:55]  typescript compilation finished in 14.31 s
[15:34:56]  bundle dev update started ...
[15:35:12]  bundle dev update finished in 16.06 s
[15:35:12]  sass started ...
[15:35:12]  sass finished in 28 ms
As you can see it takes about 30 seconds to build new main.js. It seems, that there is an issue with typescript 2.0, cause it shouldn't take 14 seconds to compile one changed file - I've reported that as well https://github.com/Microsoft/TypeScript/issues/11321. The other issue is rollup, which takes 16 seconds.

I have same issues as described early by other colleagues. For example:
[20:13:47]  typescript compilation started ...
[20:13:50]  typescript compilation finished in 3.05 s
[20:13:50]  bundle dev update started ...
JS changed:   www/build/main.js
[20:14:03]  bundle dev update finished in 12.72 s
[20:14:03]  sass started ...
[20:14:03]  sass finished in 53 ms
With beta.11 i had time watch timing between 1-2.5 s.

I would like to point out that tslint also slows down the build by a few seconds. TSLint is a handy tool and its a great idea to use it when doing a full build, but it is a  processing step that I would want to exclude when doing livereload/ionic serve.  It's not contributing as much to the slowness as the AoT compilation and tree-shaking, but if a speed-optimized build configuration is being worked on, tslint should be excluded as well.

@jaufgang Also, considering most of our editors will be linting as we code, it's mostly superfluous during each watch build.
How about a --no-lint flag?

i think a fully robust solution would have options to allow you to turn on/off individual build steps like tslint/AoT/tree-shaking via a project config file will different named build configurations, allow you to override those config file settings with command line switches, provide out of the box build configurations for production (slow build, fully optimized output)  and dev (fast build, no optimizations)
right now, I'd be happy with any means to speed up livereloading, even some advice on how to hack the app-scripts config files to speed things up while we are waiting for this issue to be addressed in a future release.

In version 0.0.29 some steps in 'ionic serve' was disabled, but still watch is 5 times slower then was previously with gulp. Example:
[09:18:02]  bundle dev update started ...
JS changed:   www/build/main.js
[09:18:12]  bundle dev update finished in 9.72 s
[09:18:12]  sass started ...
[09:18:12]  sass finished in 16 ms

I'm also having the same issue. It takes like 20 seconds for livereload to reload browser tab even if I simply change a variable value. Gulp was almost instant. It is extremely slow compared to Gulp.

I just want to add something here.
To the folks who are saying "Gulp was fast and instant", take note that gulp was also doing much less compared to what the build process is doing now.
It's not a fair comparison to make when you consider everything that is going on underneath.
https://github.com/driftyco/ionic-app-scripts#config-defaults
We're iterating quickly and have simplified the watch setup so we can do a bit less. 0.29 is out which has this in there. Also, changing up the npm scripts to match whats in the app base helps keep things up to date.
  "scripts": {
    "build": "ionic-app-scripts build",
    "watch": "ionic-app-scripts watch",
    "serve:before": "watch",
    "emulate:before": "build",
    "deploy:before": "build",
    "build:before": "build",
    "run:before": "build"
  },

It's clear that tree shaking, rollup bundling, AOT compilation etc. for production takes much more time than before (with gulp), but the problem is that the dev version is much slower than before. It seems that the TypeScript regression that slowed it down (https://github.com/Microsoft/TypeScript/issues/10018#issuecomment-251113388) will be fixed in the next TS release, what about rollup, is there something that can be done about it to speed it up in dev build?

...or even take rollup out of the build altogether for dev builds?  Really for dev builds, particularly when using watch/livereload, the only steps that are necessary are compile, concatenate and copy.  That's it.  Switching from gulp to npm scripts is fine, incorporating rollup and all of those other steps for production builds is great.  But for dev builds, anything that slows down the time between start of compilation and viewing the changes in the app or browser is just an annoyance.
@mhartington, you say its not a fair comparison because of everything going on underneath, but that misses the point that all of that extra processing doesn't add value for a dev build.
Why can't you simply create a dev build configuration using the new npm scripts that is functionally equivalent to the previous gulp implementation (compile, concatenate, copy) and keep all of these additional build steps (jslint, AoT compilation, rollup bundling, etc) for production builds?

Hello everyone! We are consistently working on getting our dev builds faster, and you will see improvement on this in the future. I would just like to clarify that the watch task, which runs when running ionic serve does not do AOT compilation, only our prod build (when running ionic run, ionic build or ionic emulate ) does AOT compilation. Thanks!

@ionic/app-scripts": "0.0.23" did work with   "allowJs": true but "0.0.29" resulting now in SyntaxError: Unexpected character '@'

@jaufgang  I'm asking myself the same questions. I would add to your point, another question. I saw that with webpack is possible to do hot module reloading (see https://github.com/AngularClass/angular2-webpack-starter). Using rollup we will miss this feature, correct? @adamdbradley

I totally agree with @jaufgang. I really appreciate work of whole Ionic team (you are guys awesome!!), but if dev builds will be much slower than it was before rc, then whole awesomeness of Ionic will be gone. @mharington IMHO if it is not possible to optimize rollup  (as it seems that is the issue) then going back to browserify is an option to think about. Still I so appreciate your amazing job!!! Thanks and no offense ;)

I'd just like to add another data point from my side (new to Ionic and busy going through various v2 tutorials): there definitely seems to be something strange happening with watch.  I'm using ionic serve, and after a couple of code changes, the rebuild was taking around 80 seconds (on a very decent PC).  Strangely, restarting ionic serve (and waiting for the app to come up) was faster than 80 seconds, around 20 in total.

Above is a screenshot of the console logs.  At first glance (from the timestamps and durations), it seems like multiple of these 'dev bundle update' are running in parallel, since the logs from 14:00:39 seem to start on 13:59:20.  Perhaps this type of thing is causing some locks / delays?

Just a quick follow up to my comment above, after another few days of general usage: the dev bundle / template update times seem to progressively get worse as time goes on.  Very roughly, every change seems to add ~2 seconds to the total rebuild time, which very quickly adds up.  E.g. on my system when ionic serve is first started, builds take about 12 seconds.
The important part is that this seems to be unrelated to what exactly is going on with the build (AOT optimizations, etc), it just gets slower and slower with use, even if you just add and delete a space for each rebuild.
On a side note, congrats to the team on Ionic, it's really an awesome framework so far.

when I development some time it take over 100 second for build
but it have good performance when it run on phone.
I hope we can fix this issue soon.

or even take rollup out of the build altogether for dev builds

I agree with this sentiment. I feel this is equivalent to Angular 2's process, during development you run npm start which simply runs the typescript watcher, compiling to the necessary javascript files, with a lightweight http server running in the background to serve them up. True, they currently advocate SystemJS but the same is possible for webpack.
I feel that trying to shoehorn the same process for development and production is fundamentally the wrong approach. They are very different things, with different requirements. What would be great is seeing fast compilation/reloading in the browser during development/watch mode, and fully optimised builds for production, which will necessitate longer compilation times.

Report on 0.0.33:
Initial start:
watch ready in 48.00 s
SASS file change:
sass update finished in 4.74 s
HTML file change:
template update finished in 131 ms (Amazing!!)
TS file change:
transpile update finished in 6.71 s
bundle update finished in 32.92 s
So HTML updates have definitely improved by now, but the TS livereload still takes as long as on the initial app-scripts release.

Same issue here
[11:36:33]  bundle dev update finished in 26.42 s [11:36:33]  sass started ... [11:36:33]  sass finished in 57 ms JS changed:   www/build/main.js [11:40:38]  templateUpdate started ... [11:40:38]  bundle dev update started ... JS changed:   www/build/main.js [11:40:55]  bundle dev update finished in 17.09 s [11:40:55]  templateUpdate finished in 17.10 s JS changed:   www/build/main.js [11:42:36]  templateUpdate started ... [11:42:36]  bundle dev update started ... JS changed:   www/build/main.js [11:42:51]  bundle dev update finished in 14.43 s [11:42:51]  templateUpdate finished in 14.43 s [11:43:35]  bundle dev update started ... [11:43:47]  templateUpdate started ... [11:43:47]  bundle dev update started ... JS changed:   www/build/main.js JS changed:   www/build/main.js [11:43:52]  bundle dev update finished in 17.68 s [11:43:52]  sass started ... [11:43:53]  sass finished in 87 ms JS changed:   www/build/main.js JS changed:   www/build/main.js [11:44:13]  bundle dev update finished in 25.75 s [11:44:13]  templateUpdate finished in 25.75 s [11:46:34]  templateUpdate started ... [11:46:34]  bundle dev update started ... JS changed:   www/build/main.js [11:46:46]  bundle dev update finished in 11.63 s [11:46:46]  templateUpdate finished in 11.63 s [14:38:33]  templateUpdate started ... [14:38:33]  bundle dev update started ... JS changed:   www/build/main.js [14:38:49]  bundle dev update finished in 16.43 s [14:38:49]  templateUpdate finished in 16.45 s
Any updates on this ? I get rollup does more than gulp, but can't we just use gulp for development and have rollup for builds?

HTML template rebuilds have drastically improved in the latest release. Additionally, we're adding webpack as the default bundler, with rollup as an option.
However, many of these numbers being reported is not apart of app-scripts control, such as the time it takes to compile TypeScript, and the time it takes for NGC to do Ahead of Time compiling. Also, AoT only happens on production builds, not dev buidls.
Also, in the more recently releases, tslint and sass runs in another core. We'll continue to improve, and using webpack's bundler may help. The other idea of letting system do the bundling, or all the above.
We'll continue to improve the build process and we'll keep this open to discuss various ideas. But one thing to point out is that there is not easy answer at the moment since Angular's change to using @NgModule. So it's not that there was a regression since beta11, but that there are new requirements which making building more difficult. On just the basic app, I saw startup times around 1300 milliseconds in production. Since our new bundling system startup times are entire second faster. So while the dev builds aren't the fastest right now, and we've got room for improvement, your users and your app experience is lightning fast because of these build tasks.
But I completely understand, and we'll keep improving build times with new ideas. Thanks

Got the same issue, i already updated script to version 0.0.36
[17:30:31]  transpile update started ...
[17:30:36]  transpile update finished in 4.60 s
[17:30:36]  bundle update started ...
JS changed:   www/build/main.js
[17:30:55]  bundle update finished in 19.05 s
[17:30:55]  watch ready

Timings are those. In fact mine are slower, in the future we might expect improvements in the performance. But but build process has become quite complex.

@adamdbradley Is webpack already the default bundler? Im on a v0.0.36 with rollup still as default.

@realappie With the latest release of app-scripts Rollup is still the default bundler. We are working hard on wrapping up some final issues with our webpack integration and hope to have it ready for testing asap. Thanks!

These are my bundle times right now

[15:01:40]  watch ready
[15:01:41]  transpile update started ...
[15:01:45]  transpile update finished in 3.33 s
[15:01:45]  bundle update started ...
JS changed:  www\build\main.js
[15:02:14]  bundle update finished in 28.93 s
[15:02:14]  watch ready
[15:02:14]  transpile update started ...
[15:02:21]  transpile update finished in 7.44 s
[15:02:21]  bundle update started ...
JS changed:   www\build\main.js
[15:03:02]  bundle update finished in 40.55 s
[15:03:02]  watch ready

When should we expect for an improvement? Right now it's impossible for me to work. And this is only with couple of pages.

@raducostea I understand what you talk about. As for templates, timings are acceptable, bundling is a little bit slower, for *.ts files is around 30 sec tops, maybe I shouldn't... but I got used to it, and spend that time thinking on good ideas ;) or reading the docs. When I need to make a lot of changes I stop the build process,  make edits and restart ionic serve. My rc1 app now has a couple of dozens of views and build process doesn't get noticeable slower than with a couple of views. Don't panic, just hang a bit, things will get better, I suffered a lot the transition from beta11 to rc0 and r0 to rc1 with app-scripts 0.0.30 and timings were about 70 secs, so I'm pretty happy about 20-30 they have achieved in dev mode with 0.0.36
I also spend the dead time trying to add issues here and and share things on the blog, seriously, ionic team has proven to be SUPER receptive, and they make gold improvements with your support and detailed feedback.

I created a simple ionic2-webpack2-starter project. Using the webpack-dev-server there, reloading the page after each typescript code change takes 800-900ms. In comparison ionic serve takes about 5 seconds at every change.

I had 5-7 second ionic serve with<EMAIL_ADDRESS>and !! 15-20 seconds with<EMAIL_ADDRESS>with rollup, webpack not working at all

@maxkunitsa,
That's weird. Have you tried version 0.0.39? I consistently see builds done in 2.5 or less seconds. Webpack should be a lot faster than Rollup. Can you add the --debug flag to the end of the watch script in the package.json and see if there is any strange output?
Thanks,
Dan

I agree, 0.0.39 works a lot better.

[18:27:26]  build update started ...
[18:27:26]  transpile update started ...
[18:27:27]  transpile update finished in 512 ms
[18:27:27]  webpack update started ...
[18:27:29]  webpack update finished in 2.02 s
[18:27:29]  build update finished in 2.53 s
[18:27:29]  watch ready

My build times vary a lot on 0.0.39, but on average it has definitely improved. I'm getting between ~5 and ~25 seconds. HTML updates are generally below ~15 seconds.

@danbucholtz tnx for reply. I updated to 0.0.39 and got such results:
ionic $ [19:24:56]  template update started ...
JS changed:   www/build/main.js
[19:24:56]  template update finished in 99 ms
[19:24:56]  watch ready
[19:25:01]  template update started ...
[19:25:01]  template update finished in 105 ms
[19:25:01]  watch ready
JS changed:   www/build/main.js
[19:25:07]  build update started ...
[19:25:07]  transpile update started ...
[19:25:08]  transpile update finished in 749 ms
[19:25:08]  rollup update started ...
JS changed:   www/build/main.js
[19:25:25]  rollup update finished in 17.02 s
[19:25:25]  build update finished in 17.77 s
[19:25:25]  watch ready
[19:25:37]  build update started ...
[19:25:37]  transpile update started ...
[19:25:37]  transpile update finished in 386 ms
[19:25:37]  rollup update started ...
JS changed:   www/build/main.js
[19:25:51]  rollup update finished in 14.15 s
[19:25:51]  build update finished in 14.54 s
[19:25:51]  watch ready
[19:25:55]  build update started ...
[19:25:55]  transpile update started ...
[19:25:55]  transpile update finished in 619 ms
[19:25:55]  rollup update started ...
JS changed:   www/build/main.js
[19:26:11]  rollup update finished in 15.70 s
[19:26:11]  build update finished in 16.32 s
[19:26:11]  watch ready

@maxkunitsa how many pages does your project have if you dont mind? Also what kind of file are you importing that is throwing that error? Is it a page? Or is it something like a json file?

@Manduro,
HTML updates and ionic serve updates both only modify the one file and recreate the bundle from that. It should take a handful of seconds at the most. Can you add --debug to the watch npm script and see if there is anything strange there?
My typical first build is ~10 seconds for a 10 page application, then subsequent builds are 2.5-3 seconds every time.
Thanks,
Dan

Yestarday I tried app-scripts v0.0.38 and webpack was blazing fast - update of my project took 5-8sec. Today, after installation of v0.0.39 I see big slow down - the same project, the same change in ts file and it takes 14-16sec. Any idea why the slow down?

@lleevvyy, yes, we added source maps back in. But it shouldn't have that big of an impact. Do you have an SSD? Hmm, thinking why that would be... I am consistently around 2.5 seconds.
Thanks,
Dan

@danbucholtz Yes, I have SSD. Below result of watch script with --debug.
0.0.39:
ionic $ [ DEBUG! ]  watch callback start, id: 0, isProd: false, event: change, path:
            /Volumes/Project/catalog/app/src/pages/main/main-page.ts  MEM: 780.4MB
[21:51:58]  build update started ... MEM: 780.4MB
[21:51:58]  transpile update started ... MEM: 780.4MB
[ DEBUG! ]  worker created, taskModule: lint, pid: 22055  MEM: 780.5MB
[ DEBUG! ]  worker created, taskModule: transpile, pid: 22058  MEM: 782.1MB
[21:51:59]  transpile update finished in 526 ms MEM: 782.1MB
[21:51:59]  webpack update started ... MEM: 782.1MB
[ DEBUG! ]  webpackUpdate: Typescript File Changed  MEM: 782.1MB
[ DEBUG! ]  webpackUpdate: Transforming paths  MEM: 782.1MB
[ DEBUG! ]  webpackUpdate: Writing Files to tmp  MEM: 782.1MB
[ DEBUG! ]  webpackUpdate: Starting Incremental Build  MEM: 782.1MB
JS changed:   www/build/main.js
[ DEBUG! ]  worker exited, taskModule: lint, pid: 22055  MEM: 936.6MB
[ DEBUG! ]  Webpack Bundle Updated  MEM: 936.6MB
[ DEBUG! ]  handleWebpackBuildSuccess: Resolving with Webpack data  MEM: 936.6MB
[ DEBUG! ]  webpackUpdate: Incremental Build Done, processing Data  MEM: 938.6MB
[ DEBUG! ]  Cached module paths: 1083,
            /var/folders/27/d46m2kjd233dbc152r3j96y80000gn/T/projectcatalogapp/modulepaths.json
             MEM: 938.6MB
[21:52:11]  webpack update finished in 12.67 s MEM: 938.6MB
[21:52:11]  build update finished in 13.21 s MEM: 938.6MB
[ DEBUG! ]  watch callback complete, id: 0, isProd: false, event: change, path:
            /Volumes/Project/catalog/app/src/pages/main/main-page.ts  MEM: 938.6MB

[21:52:11]  watch ready

[ DEBUG! ]  worker exited, taskModule: transpile, pid: 22058  MEM: 937.6MB

0.0.38:
ionic $ [ DEBUG! ]  watch callback start, id: 0, isProd: false, event: change, path:
            /Volumes/Project/catalog/app/src/pages/main/main-page.ts  MEM: 764.4MB
[21:54:32]  build update started ... MEM: 764.4MB
[21:54:32]  transpile update started ... MEM: 764.4MB
[ DEBUG! ]  worker created, taskModule: lint, pid: 22260  MEM: 764.4MB
[ DEBUG! ]  worker created, taskModule: transpile, pid: 22261  MEM: 747.2MB
[21:54:33]  transpile update finished in 501 ms MEM: 747.2MB
[21:54:33]  webpack update started ... MEM: 747.2MB
[ DEBUG! ]  webpackUpdate: Typescript File Changed  MEM: 747.2MB
[ DEBUG! ]  webpackUpdate: Transforming paths  MEM: 746.1MB
[ DEBUG! ]  webpackUpdate: Writing Files to tmp  MEM: 746.1MB
[ DEBUG! ]  webpackUpdate: Starting Incremental Build  MEM: 740.2MB
[ DEBUG! ]  Webpack Bundle Updated  MEM: 437.7MB
[ DEBUG! ]  handleWebpackBuildSuccess: Resolving with Webpack data  MEM: 437.9MB
JS changed:   www/build/main.js
[ DEBUG! ]  webpackUpdate: Incremental Build Done, processing Data  MEM: 438.5MB
[ DEBUG! ]  Cached module paths: 1083,
            /var/folders/27/d46m2kjd233dbc152r3j96y80000gn/T/projectcatalogapp/modulepaths.json
             MEM: 438.5MB
[21:54:40]  webpack update finished in 6.89 s MEM: 438.5MB
[21:54:40]  build update finished in 7.40 s MEM: 438.6MB
[ DEBUG! ]  watch callback complete, id: 0, isProd: false, event: change, path:
			/Volumes/Project/catalog/app/src/pages/main/main-page.ts  MEM: 438.6MB

[21:54:40]  watch ready

[ DEBUG! ]  worker exited, taskModule: lint, pid: 22260  MEM: 438.7MB
[ DEBUG! ]  worker exited, taskModule: transpile, pid: 22261  MEM: 41

@danbucholtz Look at memory usage after "Starting Incremental Build" - in 0.0.38 memory usage goes down in 0.0.39 goes up. Hope that helps to trace the problem.

@lleevvyy, @Manduro, how old is your computer? What sort of processor, ram, and SSD? Just trying to figure out what could be causing the slowness.
Thanks,
Dan

@danbucholtz MacBook Pro, Intel Core i5, 2,53 GHz, 6GB ram, SSD OCZ Vertex2... But, as I wrote -  the performance on 0.0.38 was very good and I've made the test again couple of minutes again. Have you seen my post above (about the difference in memory consumption)? Maybe that is the reason of the slow down?

@lleevvyy thanks for all the info! Out of curiosity, how many pages does your app have?

@lleevvyy, It makes sense to me that it is a little slower since we enabled a devtool of source-maps which is the slowest-but-best sourcemap webpack offers. See the configuration info here. One can provide their own webpack config and change this value if they'd like. See how to do that here.
I wouldn't expect it to be 2x slower, though. I am on a 16GB MBP 15", the most recent one, and no matter what I do when building out an app, I see 2.5 seconds. So I see basically zero difference on my machine between 0.0.38 and 0.0.39 in both a blank starter app being built out, and then our pretty large conference app example as seen here.
Is anyone else seeing really slow build times? @mirkonasato, any thoughts on why that might be?
Thanks,
Dan

@jgw96 8 pages, but quite complex, lots of other ts files, over 10k LOC (excluding scss and html) and few dependencies to js libraries.

@danbucholtz Yup, it is because of devtool=source-maps... When I disabled it, the times of webpack update are the same as in 0.0.38... Understand, that you cannot do too much with it, so I will override it in my webpack config. Thanks.
Ps. Well, I wanted to buy a brand new MBP, but not happy with the changes that Apple made few days ago.. Still, hardware refreshment is required in my case :-)

@lleevvyy good find! Yeah we are working on using the TypeScript source maps instead of the webpack ones which should fix this (: Thanks for all the debugging

@danbucholtz @jgw96 Thanks!

Just to confirm, 0.0.38 is faster on my machine as well. After a ts change 0.0.38 rebuilds in 1.1-1.2 seconds, with 0.0.39 it goes up to 2.8-2.9 seconds.
And yes I believe the devtool alone can explain the difference. With e.g. devtool: 'eval-source-map' it's pretty much as fast as 0.0.38.
(What's strange is that I don't see the original typescript code, not even with devtool: 'source-map', but that's a separate thing.)

@mirkonasato, in order to support roll-up too, we do the typescript stuff outside of the context of webpack. We have the actual Typescript source code and map available in memory so in version 0.0.40 we are mapping back to that. Seeing some weird results when used in conjunction with AoT right now, though.
I'm glad to here it's still very fast for you. I was worried there for a second. Thanks for testing! 💯
Thanks,
Dan

@danbucholtz @jgw96
My app: 10 pages, ~ 25 services, ~ 20 classes, ~15 common components used in pages, html/css for each page/component.
My MacBook Pro:  Intel Core i5, 2,6 GHz, 8GB DD3 ram, 256 GB SSD
Rollup:
JS: 5-8 seconds for build with 0.0.30 and 15-20 seconds with 0.0.38/0.0.39.
HTML: 5-8 seconds with 0.0.30 and 100-500 ms with 0.0.38/0.0.39.
Webpack
JS: 5-8 seconds for build with 0.0.39
HTML: 300ms - 800ms for build with 0.0.39
but i have error  'Error: Cannot find module "../../common/services/index/"(…)' in console (barrels doesn't work). This file is barrel, something like this:
export * from './utils/color.utils';
export * from './utils/localstorage.utils';
export * from './utils/math.utils';
export * from './utils/uuid.utils';

@danbucholtz Sure, below are my logs.
App info: ~25 pages, ~100 ts files, multiple dependencies.
Hardware (2010): i5-520M Arrandale, 8GB memory, SATA300 SSD.
Nothing crazy in the logs. Maybe it's just the sourcemap taking much longer on a larger app?
Debug logs:
Initial build ~50s (shows some weird double paths)
[09:18:38]  ionic-app-scripts 0.0.39
[09:18:38]  watch started ... MEM: 110.7MB
[09:18:38]  build dev started ... MEM: 110.7MB
[09:18:38]  clean started ... MEM: 110.7MB
[ DEBUG! ]  clean /Users/user/path/www/build  MEM: 110.7MB
[09:18:38]  clean finished in 3 ms MEM: 110.8MB
[09:18:38]  copy started ... MEM: 110.8MB
[ DEBUG! ]  worker created, taskModule: lint, pid: 1684  MEM: 111.0MB
[09:18:38]  transpile started ... MEM: 111.0MB
[09:18:39]  lint started ...
[09:18:51]  transpile finished in 12.86 s MEM: 269.7MB
[09:18:51]  webpack started ... MEM: 269.7MB
[ DEBUG! ]  Wrote .js files to disk  MEM: 270.8MB
[ DEBUG! ]  Starting Webpack watch  MEM: 270.9MB
[09:18:52]  copy finished in 13.94 s MEM: 275.9MB
[09:19:17]  lint finished in 37.84 s
[ DEBUG! ]  worker exited, taskModule: lint, pid: 1684  MEM: 714.3MB
[ DEBUG! ]  Webpack Bundle Updated  MEM: 714.3MB
[ DEBUG! ]  handleWebpackBuildSuccess: Resolving with Webpack data  MEM: 714.3MB
[ DEBUG! ]  Cached module paths: 1308,
            /var/folders/pr/ck8q921d21j8y6d5pmqy4z1c0000gn/T/usersjobdwvdfellofelloclient/modulepaths.json  MEM:
            715.0MB
[09:19:23]  webpack finished in 31.94 s MEM: 715.0MB
[09:19:23]  sass started ... MEM: 715.0MB
[ DEBUG! ]  sass outFile: /Users/user/path/www/build/main.css  MEM: 715.0MB
[ DEBUG! ]  sass includePaths:
            /Users/user/path/src,node_modules/ionic-angular/themes,node_modules/ionicons/dist/scss
             MEM: 715.0MB
[ DEBUG! ]  sass moduleDirectories: 244  MEM: 715.0MB
[ DEBUG! ]  sass userSassVariableFiles: 1  MEM: 714.4MB
[ DEBUG! ]  sass componentSassFiles: 157  MEM: 714.4MB
[ DEBUG! ]  sass, start postcss/autoprefixer  MEM: 766.0MB
[ DEBUG! ]  sass: postcss/autoprefixer completed  MEM: 741.9MB
[ DEBUG! ]  sass start write output: /Users/user/path/www/build/main.css  MEM: 741.9MB
[ DEBUG! ]  sass saved output: /Users/user/path/www/build/main.css  MEM: 743.1MB
[09:19:27]  sass finished in 4.18 s MEM: 743.1MB
[09:19:27]  build dev finished in 49.01 s MEM: 743.1MB
[ DEBUG! ]  watcher ready:
            /Users/user/path/Users/user/path/src/assets/,/Users/user/path/src/index.html,/Users/user/path/src/manifest.json,/Users/user/path/src/service-worker.js,node_modules/ionic-angular/polyfills/polyfills.js,node_modules/ionicons/dist/fonts/
             MEM: 743.8MB
[ DEBUG! ]  watcher ready:
            /Users/user/path/Users/user/path/src/assets/,/Users/user/path/src/index.html,/Users/user/path/src/manifest.json,/Users/user/path/src/service-worker.js,node_modules/ionic-angular/polyfills/polyfills.js,node_modules/ionicons/dist/fonts/
             MEM: 743.8MB
[ DEBUG! ]  watcher ready:
            /Users/user/path/Users/user/path/src/assets/,/Users/user/path/src/index.html,/Users/user/path/src/manifest.json,/Users/user/path/src/service-worker.js,node_modules/ionic-angular/polyfills/polyfills.js,node_modules/ionicons/dist/fonts/
             MEM: 743.8MB
[ DEBUG! ]  watcher ready: /Users/user/path/Users/user/path/src/**/*.html  MEM:
            743.8MB
[ DEBUG! ]  watcher ready: /Users/user/path/Users/user/path/src/**/*.ts  MEM:
            743.8MB
[ DEBUG! ]  watcher ready: /Users/user/path/Users/user/path/src/**/*.scss  MEM:
            743.8MB
[ DEBUG! ]  watcher ready: /Users/user/path/Users/user/path/src/**/*  MEM:
            743.8MB
[09:19:29]  watch ready in 51.27 s MEM: 743.8MB

TS file update: ~5-25 seconds, generally ~15.
[ DEBUG! ]  watch callback start, id: 0, isProd: false, event: change, path:
            /Users/user/path/src/pages/network/network.ts  MEM: 743.9MB
[09:23:58]  build update started ... MEM: 743.9MB
[09:23:58]  transpile update started ... MEM: 743.9MB
[ DEBUG! ]  worker created, taskModule: lint, pid: 1791  MEM: 743.6MB
[ DEBUG! ]  worker created, taskModule: transpile, pid: 1793  MEM: 743.7MB
[09:23:59]  transpile update finished in 309 ms MEM: 743.7MB
[09:23:59]  webpack update started ... MEM: 743.7MB
[ DEBUG! ]  webpackUpdate: Typescript File Changed  MEM: 743.7MB
[ DEBUG! ]  webpackUpdate: Transforming paths  MEM: 743.7MB
[ DEBUG! ]  webpackUpdate: Writing Files to tmp  MEM: 743.7MB
[ DEBUG! ]  webpackUpdate: Starting Incremental Build  MEM: 743.7MB
[ DEBUG! ]  worker exited, taskModule: lint, pid: 1791  MEM: 673.3MB
[ DEBUG! ]  worker exited, taskModule: transpile, pid: 1793  MEM: 673.3MB
JS changed:   www/build/main.js
[ DEBUG! ]  Webpack Bundle Updated  MEM: 673.3MB
[ DEBUG! ]  handleWebpackBuildSuccess: Resolving with Webpack data  MEM: 673.3MB
[ DEBUG! ]  webpackUpdate: Incremental Build Done, processing Data  MEM: 671.6MB
[ DEBUG! ]  Cached module paths: 1308,
            /var/folders/pr/ck8q921d21j8y6d5pmqy4z1c0000gn/T/usersuserpath/modulepaths.json  MEM:
            671.6MB
[09:24:13]  webpack update finished in 14.68 s MEM: 671.6MB
[09:24:13]  build update finished in 15.00 s MEM: 671.6MB
[ DEBUG! ]  watch callback complete, id: 0, isProd: false, event: change, path:
            /Users/user/path/src/pages/network/network.ts  MEM: 671.6MB

[09:24:13]  watch ready

HTML file update:
[ DEBUG! ]  watch callback start, id: 1, isProd: false, event: change, path:
            /Users/user/path/src/pages/network/network.html  MEM: 671.6MB
[09:25:35]  template update started ... MEM: 671.6MB
[ DEBUG! ]  templateUpdate, event: change, path: /Users/user/path/src/pages/network/network.html
            MEM: 671.6MB
[ DEBUG! ]  templateUpdate: updating webpack file  MEM: 671.6MB
[09:25:35]  build update started ... MEM: 671.6MB
[09:25:35]  transpile update started ... MEM: 671.6MB
[ DEBUG! ]  worker created, taskModule: lint, pid: 1834  MEM: 671.6MB
[ DEBUG! ]  worker created, taskModule: transpile, pid: 1835  MEM: 671.6MB
[09:25:35]  transpile update finished in 224 ms MEM: 671.6MB
[09:25:35]  webpack update started ... MEM: 671.6MB
[ DEBUG! ]  webpackUpdate: Typescript File Changed  MEM: 671.6MB
[ DEBUG! ]  webpackUpdate: Transforming paths  MEM: 671.6MB
[ DEBUG! ]  webpackUpdate: Writing Files to tmp  MEM: 671.6MB
[ DEBUG! ]  webpackUpdate: Starting Incremental Build  MEM: 671.6MB
[ DEBUG! ]  Webpack Bundle Updated  MEM: 589.5MB
[ DEBUG! ]  handleWebpackBuildSuccess: Resolving with Webpack data  MEM: 589.5MB
[ DEBUG! ]  webpackUpdate: Incremental Build Done, processing Data  MEM: 591.6MB
[ DEBUG! ]  Cached module paths: 1308,
            /var/folders/pr/ck8q921d21j8y6d5pmqy4z1c0000gn/T/usersuserpath/modulepaths.json  MEM:
            591.6MB
[09:25:42]  webpack update finished in 6.85 s MEM: 591.6MB
[09:25:42]  build update finished in 7.08 s MEM: 591.6MB
[09:25:42]  template update finished in 7.08 s MEM: 591.6MB
[ DEBUG! ]  watch callback complete, id: 1, isProd: false, event: change, path:
            /Users/user/path/src/pages/network/network.html  MEM: 591.6MB

[09:25:42]  watch ready

[ DEBUG! ]  worker exited, taskModule: lint, pid: 1834  MEM: 591.6MB
[ DEBUG! ]  worker exited, taskModule: transpile, pid: 1835  MEM: 591.6MB

SASS update:
[ DEBUG! ]  watch callback start, id: 2, isProd: false, event: change, path:
            /Users/user/path/src/theme/fonts.scss  MEM: 591.6MB
[09:27:08]  sass update started ... MEM: 591.6MB
[ DEBUG! ]  sass outFile: /Users/user/path/www/build/main.css  MEM: 591.6MB
[ DEBUG! ]  sass includePaths:
            /Users/user/path/src,/Users/user/path/src,node_modules/ionic-angular/themes,node_modules/ionicons/dist/scss
             MEM: 591.6MB
[ DEBUG! ]  sass moduleDirectories: 244  MEM: 591.6MB
[ DEBUG! ]  sass userSassVariableFiles: 1  MEM: 591.6MB
[ DEBUG! ]  sass componentSassFiles: 157  MEM: 591.6MB
[ DEBUG! ]  sass, start postcss/autoprefixer  MEM: 622.5MB
[ DEBUG! ]  sass: postcss/autoprefixer completed  MEM: 616.8MB
[ DEBUG! ]  sass start write output: /Users/user/path/www/build/main.css  MEM: 616.8MB
CSS changed:  www/build/main.css
[ DEBUG! ]  sass saved output: /Users/user/path/www/build/main.css  MEM: 611.7MB
[09:27:11]  sass update finished in 3.61 s MEM: 611.7MB
[ DEBUG! ]  watch callback complete, id: 2, isProd: false, event: change, path:
            /Users/user/path/src/theme/fonts.scss  MEM: 611.7MB

[09:27:11]  watch ready

@maxkunitsa, can you open a separate issue for that? We'll get it fixed.
@Manduro, if the build is taking too long, the best bet may be to override the webpack.config.js by following the docs here and providing a devtool value of eval. That will speed things up but the source maps will be lower quality and won't work as well.
Thanks,
Dan

Hi guys. I figured out that node version v5.11.1 is much faster than version v6.7.0 for mac users. what versions are you guys using?

@pascalwhoop tried node 7.0.0, 5.11.1 and 6.9.1 no any difference.

Tried everything, but nothing worked.
I've finally updated app-scripts to 0.0.39 (was 0.0.36) and there is a huge improvement for both windows and Ubuntu. From ~30/40sec for every modification on a source file to ~10sec.
"ionic run" is also faster.

The watch is significant faster now, so I am going to close this issue for now. I consistently see builds between 750-1500ms. We have a few more optimizations we'll make over the coming weeks as well.
Thanks,
Dan

@danbucholtz Thank you very, very much for your work!!!! I can confirm, that it is lightning fast with latest releases.

@lleevvyy, really happy to hear that! 😄 . We have a few more optimizations in mind that we'll give a try in the coming weeks.
Thanks,
Dan

@danbucholtz Thanks a lot for all the work you are putting in!! Things have certainly improved A LOT :)

Yes. but for me right now the fastest version remains 0.0.38 with about 1.2 s build time. The latest one takes about 4-5 seconds.

@danbucholtz Not fast, blazing fast! I'm having a 600% (Yep, 6 hundred) improvement from that buggy 0.38 to the latest. Out-stand-ing.

Is it just me or 0.0.47 seems about twice as slow compared to 0.0.46?

@mirkonasato,
We reverted back to source-map instead of eval for the webpack devtool. You can change it back by setting ionic_source_map_type in the package.json config. I personally prefer the speed to an original, high quality source map but the team decided to go with the better map.
Thanks,
Dan

@danbucholtz thanks, but there other options available that offer a better speed/quality tradeoff for development. Webpack 2 defaults to eval-cheap-module-source-map.
(And eval doesn't seem to be working properly anyway, but that's a separate issue.)

@mirkonasato,
Believe me, I exhaustively looked into the options and was very disappointed to see breakpoints not work correctly, get hit at wrong times, etc. eval and source-map worked best for us. I am very open to changing this if it works correctly.
What isn't working with eval? I have used the eval maps successfully in testing. Was able to set and hit breakpoints as expected, etc.
Thanks,
Dan

@danbucholtz eval actually works fine, sorry. Just have to close and re-open the app after changing source map type, otherwise Chrome somehow gets confused.

can u please tell me which package.json  to change the
ionic_source_map_type
to
eval
@mirkonasato  @danbucholtz
i cant find any of this settings
package.json in project folder or some where else

...
"config" : {
   "ionic_source_map_type" : "eval"
}
...

To be clear, config is a root node. It should result in a faster development build.
Thanks,
Dan

thx 2 or 3 second faster
[05:19:15]  build started ...
[05:19:15]  transpile update started ...
[05:19:15]  transpile update finished in 83 ms
[05:19:15]  webpack update started ...
[05:19:20]  webpack update finished in 5.25 s
[05:19:22]  build finished in 7.41 s

I'm currently using @ionic/app-scripts v1.3.7 and my webpack update is taking over 45 seconds to complete 😢

@sean-hill,
How do I recreate? That seems strange to me. Can you dive in and provide more info. Maybe open another issue with some details.
Thanks,
Dan


Scale Grid and 3x3 not doing anything on v11

Though some of the buttons work as intended, the two primary buttons on v11 no longer create a "draggable box" around the grid area like it did in previous versions. Is this an issue for others? I've tested module both with modules and solo, and neither measures the grid as intended.

Same issue here

It looks like canvas.app.plugins is no longer a thing (which housed the interaction object).  I have no idea when this changed or why.  I was able to get this working by using canvas.mousePosition instead but I'm not sure if it's the "most correct" way to do it.

Found a better fix and submitted a PR.

@atomdmac sounds like the developer is likely to abandon this project based on this comment from the other thread: https://github.com/jbhaywood/scaleGrid/issues/7#issuecomment-1762052619
perhaps you can publish your fork?

@jbhaywood I'm happy to take this over if you'd like.  Do you know what the procedure is for changing module ownership on the Foundry side?

@Perogy While we wait to hear back from the repo owner, feel free to git clone my branch into your modules folder :) It should "Just Work".

@jbhaywood I'm happy to take this over if you'd like. Do you know what the procedure is for changing module ownership on the Foundry side?
Edit: A helpful Discord user has relayed to me that the two of us need to reach out to the Foundry admins to swap ownership over to me. Apparently this is the preferred way for us to contact them about it. Let me know if you're interested.

Yeah, go for it. I took it over from someone else, so I'm happy to keep hand it over to someone else now. Go ahead and contact them with the form and let me know what the response is. I don't remember exactly what the process was when I did it, but I know it involved contacting the Foundry devs.

@jbhaywood I got a response from the admin over at FVTT.  They're going to reach out to you to confirm the transfer and then they'll make it so :)

Thanks @atomdmac! Let me know if you have any questions about the code. I'm happy to act as shepherd if needed.


feat: update getting-started.md

Purpose of this commit:
add a single line of bash script to easily setup the nightly toolchain(required in 'rust-toochain.toml' in main project)

This is my first tiny commit to greptime community,
looking forward to comments and  suggestions.

@xie-zheng Thanks for contributing! Will take a look at it soon.

Hi @xie-zheng , thank you for your suggestion. But if you are using rustup to manage cargo binary, it will resolve this toolchain.toml when you run any cargo commands within the project. There is no need to install the toolchain manually.
https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file

@sunng87 , Thanks for the reply. Guess the previous building failure is due to network problem, which failed the installation of nightly toolchain...


⚠ ClamAV has degraded performance

In 27543a1, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 236 ms

Resolved: ClamAV performance has improved in ea48eb2.


Add explain class and shap

PR Description
This PR adds an explain class and the SHAP package to pyMAISE.
Closes: #67
What changes were made?
Will ultimately include:

new explain class
updated CHF benchmark
unit tests

Reviewers: @{relevant-person}

[!IMPORTANT]
Please do not review SHAP package files! (I.e., explain/shap)

@npanczyk, just doing a quick look before you're ready for a review. One thing we need here is the following text in every SHAP-related file that has copied code.
 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Hi @npanczyk , thanks for a great work. I tested your branch on a clean venv and it worked like a charm on short (small search) and long (full search) environments. These are things to consider before merge:
1- After I installed pyMAISE on fresh venv, I had to install three more dependencies for shap. These were something we talked about to be added to pyMAISE default packages to install with these specific versions:
slicer-0.0.8
numba-0.60.0
cloudpickle-3.0.0
2- Find a way to pass verbose=0 to all tensorflow prediction lines associated with SHAP methods. These are coming from model.predict calls. There might be a way to silence those before or while passing them to DeepLIFT, KernelSHAP, and IG.
3- Let @myerspat  knows to add a clear acknowledge in the github page and docs page to SHAP package that we are using their implementation.
4- For the example CHF notebook, since we do not have a detailed documentation yet for that  capability yet, it is helpful to discuss something about the classes. For example, why you use nsamples=None (what that means and what would happen if you used nsamples=20 and so on), what is background samples for KernelSHAP, why that needs to be small from a computing cost perspective, and so on for what you feel is important to be described for the user when using such methods.


Force to fetch from networking always

I have the following code to fetch movies from networking and then cache it on memory and disk
let cache = MemoryCacheLevel<Movies, NSData>()
        .compose(DiskCacheLevel<Movies, NSData>())
        .compose(MoyaFetcher(provider: moviesProvider))
        .transformValues(JSONObjectTransfomer<Movie>())

but I need to always fetch from MoyaFetcher (is like NetworkFetcher).
There is a way to do this?

Hi @pietrocaselani, can you elaborate a bit more on your use case?
Why do you want to always fetch, if you have Disk and Memory cache in your setup?
If you want to always fetch and ignore disk and memory, you can just instantiate the MoyaFetcher without composing it with the other cache levels.

@vittoriom I want to save in memory and disk cache for offline support, for example, but if there is an active connection, I want to fetch from API.
So that way, I can display information right away (from memory or disk), and then fetch updated info on API.
I think I can do this using RxSwift, but if there is a way to do this with Carlos, that would be great.

Oh, I see.
I think there is an immediate answer, and a proper answer:
The immediate answer is that you can fetch from memory/disk when offline, and network when online, very easily: you can use a switched cache and check for internet reachability in the switch condition. The code would be something like (I didn't compile this code so take it with a grain of salt):
let offlineLane = MemoryCacheLevel<Movies, NSData>()
        .compose(DiskCacheLevel<Movies, NSData>())
let onlineLane = offlineLane
        .compose(MoyaFetcher(provider: moviesProvider))

let switched = switchLevels(offlineLane, onlineLane) { key in
  return offline ? .cacheA : .cacheB
}

let cache = switched
        .transformValues(JSONObjectTransfomer<Movie>())

The proper answer is that the previous answer means you will have to wait for the network request when you're online, even if you have something stored in the memory or disk cache.
This is because Carlos only deals with Futures and not reactive Signals. Thus, if a request is fulfilled with a value (say, something stored in the memory level), it can't be "refreshed" with a second one (which would be the one fetched from the network).
So if the first option is feasible for you, go for it.
If you want to immediately use the memory value but also fetch from the network and update the value with the latest one, you have to apply some reactive workaround.

@vittoriom I will handle this using RxSwift for now, but thanks for the quick answer!


RSS-Feed for blog

Please add RSS support (XML representation) to your blog such that everyone can follow this awesome project anonymously with their favorite feed reader.

There is an Atom feed at https://ocr-d.de/feed.xml that I forgot to link to in the templates.

Thanks a lot! Stay awesome!


Request Too Large for GPT-4o

Error Code: 429
Error Message:
Request too large for gpt-4o on tokens per min (TPM): Limit 30000, Requested 342389. The input or output tokens must be reduced in order to run successfully.

Description:
When attempting to process a request with the GPT-4 model, we encountered an error indicating that the request exceeded the allowed tokens per minute (TPM) limit. The current TPM limit is set at 30,000, but the request required 342,389 tokens.
Steps to Reproduce:

Post a question that can yield a large number of results(rows).
Upon generating the SQL and resulting DataFrame, the UI displays: "Error: The server returned an error. See the server logs for more details."
The error in the logs reads: <IP_ADDRESS> - - [13/Jun/2024 10:52:21] "GET /api/v0/generate_followup_questions?id=1733c148-e1f0-472e-ac6b-321c5a4eb2bd HTTP/1.1" 500 -
along with a message indicating that the request exceeds the TPM limit.

Expected Behavior:
The model should either process the request if it is within the allowed limits or provide clearer guidance on how to adjust the input to comply with the token restrictions.
Actual Behavior:
The model returns an error message without processing the request, indicating that the input or output tokens must be reduced.
Impact:
This issue prevents the processing of large requests and impacts our ability to utilize the GPT-4 model for tasks requiring extensive input or output.
Additional Information:

TPM Limit: 30,000
Tokens Requested: 342,389

Please let us know if additional information is required to address this issue.

Hi @zainhoda,
Is there any fix incorporated in the app, such as setting TPM or any other alternative?
Looking forward to your response!

Hi @zainhoda,
Any resolution to this issue would be greatly appreciated.
Thank you!

Any resolution to this issue?


Feature request: Multiple counts

It would be nice to be able to specify multiple counts in case a string needs to report two or more things that may or may not be singular.
Eg.

there is one apple and one pear
there are %(count1)s apples and one pear
there is one apple and %(count2)s pears
there are %(count1)s apples and %(count2)s pears

Hi @pafcu, although it'd be nice to have something like this in general, it's currently out of this library's scope.


Targets based on project inference not showing in tasks list

Bug description:
With recent versions of Nx it is possible to define a project target configurator to allow for project inference. This works perfectly when using the command line. The issue however is that these tasks are not displayed anymore within the Nx console plugin.
Steps to reproduce:

Have some project targets
Confirm they are visible within Nx console
Rewrite these targets by using project inference

Expected behavior:
Targets based on project inference should be visible too in the Nx console plugin.

Hello,
can you provide a sample project, I will try to open it with vscode and webstorm plugin and make sure it's working similary?
I started working on support monorepo without workspace.json/angular.json...

This is similar to what you need?

I have created an Nx project to illustrate the issue. It contains:

an Angular app
a plugin (called my-plugin), using project inference in the index file of the plugin, for the test target of that app

In the project.json of the app project, I removed the test target and validated that you can still run the tests by executing npx nx run angular-app:test
The issue now is that the test target is not visible anymore in the Nx console plugin (which was the case before using project inference):

BTW, we created the same issue for the vscode version of the plugin. It was fixed there to take extra information from the project graph.

Hello, thank you for the example.
Seems not working on the latest version of vscode

I will try to figure out how to support this.


Get-PodeAuthMiddleware  does not always return the entire auth object

This may be due to me not using the auth functions in the intended manner, but, it does work correctly the first time that it is called but then will only randomly work afterwards. In my setup, I have the following in my server script block: ( removed some portions for brevity)
param (
    [int]$Port = 443
)

Import-Module Pode
Import-Module ActiveDirectory

# create a server, and start listening on port 8085
Start-PodeServer -Threads 5 -Name "CustomApi" -ScriptBlock {
New-PodeAuthType -Basic | Add-PodeAuth -Name 'Login' -ScriptBlock {
    param($username, $password)
    $SecPassword = ConvertTo-SecureString -String $password -AsPlainText -Force
    $userCred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, $SecPassword

    $test = $(try { Get-ADUser -Properties memberof,SID,SamAccountName,UserPrincipalName -Identity $username -Credential $userCred -ErrorAction Stop }Catch { $false })

    if ($test -ne $false) {
        return @{ User = [ordered]@{
                'ID'       = $test.SID
                'Name'     = $test.SamAccountName
                'UPN'      = $test.UserPrincipalName
                'AdGroups' = $test.memberof
                'cred'     = $userCred
            }
        }
    }
    return $null
}

$o365Routes = Get-ChildItem -file  -Path "C:\CustomRestAPI\O365"
foreach ($o365Route in $o365Routes){
    Add-PodeRoute -Method Get -Path "/o365/$([io.path]::GetFileNameWithoutExtension( $o365Route))"  -Middleware (Get-PodeAuthMiddleware -Name 'Login' -Sessionless )  -FilePath "./O365/$o365Route" -PassThru
}
}

the portion of concern is that the "AdGroups' and "cred" don't always make it to the route so it will assume that the person is not a member of the appropriate AD Group, or when it reuses the credential for some of the other services, (we have a password sync tool, so the userName is predictable and the passwords are the same) it will fail due to not having a valid credential object.
As I mentioned above, the issue only occurs after the first rest call, then after that, it is only randomly successful...
This is on an internal server that is not accessible from outside my org domain and not available to anyone not in my org AD realm.
In the route, I check for group membership for specific groups (for the ones listed above, the user has to be an Exchange Admin) but when it is in the failing state, I notice that nothing is being returned by the Auth middleware for the "AdGroups" portion of the Auth object.

by the way, I have to use windows PowerShell since there is not currently any support for the V0.4578.0   ExchangeOnlineManagement cmdlets in PowerShell Core.

Hey @jhainau,
I believe somebody here once had a similar issue; if you add Import-PodeModule -Name ActiveDirectory at the top in your Start-PodeServer scriptblock, does that resolve the problem?
Start-PodeServer {
    Import-PodeModule -Name ActiveDirectory
}

The issue is that Get-ADUser does "work" without the AD module, but sometimes it doesn't retrieve everything. Using Import-PodeModule will load the AD module into all runspaces for Pode, and allows Get-ADUser to work normally.

My apologies, I did remember reading something about it a while ago, but couldn't find the info when I looked this time.
This does appear to fix my issue (once I placed the line in the correct location :-) )

Excellent! I'll close the issue, but if you still have troubles feel free to reopen! :)


add flags for registry auth (fixes #488)

Tested against GCR with _json_key authentication

This fixes #488


INVALID_IDX is not handled properly

Docs do not state which fields can contain it
No example handles it
We most likely should add some helpers to work with it (e.g. get all nodes of a way, get all nodes of a way only if all exist, etc)
Bonus: add support to flatdata for proper optional types (so that this kind of thing could not be forgotten)

This was handled a while ago once flatdata started supporting it.


Update Jupyter notebook examples

Add Google Colab links
Add torch.no_grad() decorator
Update VIAME notebook pip install

I've tested that all the notebooks run in Colab. The one issue is with the DRISE notebook, since I'm pip installing xaitk-saliency, but the DRISE implementation is not yet available in the latest pypi package.

I've tested that all the notebooks run in Colab. The one issue is with the DRISE notebook, since I'm pip installing xaitk-saliency, but the DRISE implementation is not yet available in the latest pypi package.

Maybe next sprint we can make a minor release?


Update to Chromium 108.0.5359.124/125

This PR updates to Chromium 108.0.5359.124/125.
There were no significant changes needed on our end.

Change log
Blog post

I'll merge it in an hour or two


bingo menu

bingo menu can be opened anywhere, including in the keyboard config screen and probably anywhere

bingo menu is supposed to be opened anywhere though I agree there should be some exceptions for the keyboard config screen and the text entry screen


Add assertion to split function ensuring valid windows

Issue: #2577
Description of changes:
This PR addresses issue #2577 and added assertions for OffsetSplitter and DateSplitter to ensure that the targets of the label have valid target lengths, i.e. match prediction_length.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Please tag this pr with at least one of these labels to make our release process faster: BREAKING, new feature, bug fix, other change, dev setup

I think we probably miss a few tests here @MarcelK1102 do you want to add them? This module is covered by https://github.com/awslabs/gluonts/blob/dev/test/dataset/test_split.py

@lostella Sure, I will do that!

Looks good now! Let's add the tests and then this can be merged!


only show ds form field tooltip when label is clicked

fixes #2196

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-trafficcontrol-PR/1653/
Test PASSed.


In navbar, provide the number associated with each "List" in parentheses

Please do work for this task in a branch called issue-105.
This would be a nice addition. Here's the current navbar:

It would be nice if, after an entry like "List Participants", there was the number of participants. For example, "List Participants (93)".


Add content of all plenary sessions

[ ] Add slideslive id for all plenary sessions

[ ] Add rocketchat ID for all QA sessions of plenary

[ ] Add zoom link for livestream

From previous issue:

[ ]  Rename the "Speakers" to "Plenary Sessions" in the page title
[ ]  Order the sessions by time (I didn't do the PST / GMT conversion)
[ ]  Gather the necessary information to be displayed. Can you make a list? I'll send an email to our program chairs and cc you to get those information.

Also, I'm thinking instead of opening a new page for each plenary session, can we make those as a collapse cards? (https://getbootstrap.com/docs/4.3/components/collapse/) In this way, the audience will stay on the same page for the whole plenary sessions on this day. (Just like a physical room experience).
But let's first get the basic version working and come back to this when we have time, unless you already know an easy way to do it.
Feel free to create another ticket if you feel you need some additional volunteers to work on this together.

hi @raosudha89.
do you have an excel with this information?

@georgepar Can you first put some dummy string for the SlidesLive ID, RocketChat channel name, and Zoom links?
We will need to add those once we gather all the information.

@georgepar  Since this is high-priority and needs to be done before the dry-run on Friday, I want to also work on this. Can you tell me what exactly are you working on and I can maybe work parallely on other things?

I didn't realize that @georgepar  is in a different timezone. Apologies for rushing you on this.
I have created a PR https://github.com/acl-org/acl-2020-virtual-conference/pull/209 for adding slideslive link and rocketchat channel to plenary sessions.
I tried to use collapse cards instead of opening the plenary information in the new page. But I realized that with the bio+abstract information for keynote, the collapse card becomes too large. So I reverted back to opening up a new page for each plenary session.
@hao-fang  let me know if that is okay.

I didn't realize that @georgepar is in a different timezone. Apologies for rushing you on this.
I have created a PR #209 for adding slideslive link and rocketchat channel to plenary sessions.
I tried to use collapse cards instead of opening the plenary information in the new page. But I realized that with the bio+abstract information for keynote, the collapse card becomes too large. So I reverted back to opening up a new page for each plenary session.
@hao-fang let me know if that is okay.

@raosudha89 That's good -- we just want to have something working for now. Everything else can be "Website Enhancements".

July 6

July 7

July 8

Here is the new instruction after today's meeting.

We will not expose Zoom links to the public now. Everyone will ask questions in RocketChat and the co-host of the Zoom meeting will pick up questions and ask the speaker (like an interview).

[ ] Remove zoom_link field fromPlenarySession dataclass in sitedata.py
[ ] Remove zoom_link field from plenary_sessions.yml

Opening Remarks: This is a pre-recorded talk. No QA associated with it. No RocketChat channel.

[ ] Add the presentation ID to plenary_sessions.yml. You can find it below from the SL_PRESENTATION_ID below.
[ ] Update the https://virtual.acl2020.org/plenary_session_opening_remarks.html page to make sure all information are shown.

Presidential Address

Submission ID
Title
Authors
All Author Emails
Type
SL_PRESENTATION_LINK
SL_PRESENTATION_ID

8000
Rewriting the Past: Assessing the Field through the Lens of Language Generation
Kathleen McKeown
<EMAIL_ADDRESS>Keynote
slideslive.com/38929460
38929460

8001
TBD
Josh Tenenbaum
<EMAIL_ADDRESS>Keynote
slideslive.com/38929461
38929461

8003
Test of Time Awards 2020
Barbara Di Eugenio
<EMAIL_ADDRESS>Test of Time Awards
slideslive.com/38929462
38929462

9000
Opening Remarks
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Opening Remarks
slideslive.com/38929463
38929463

9001
Closing Remarks
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Closing Remarks
slideslive.com/38929464
38929464

9002
Presidential Address
Hinirich Schütze
<EMAIL_ADDRESS>Presidential Address
slideslive.com/38929465
38929465

9003
NAACL 2021
Kristina Toutanova, Luke Zettlemoyer, Anna Rumshisky, Saif Mohammad
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Conference Announcements
slideslive.com/38929466
38929466

9004
ACL 2021
Chengqing Zong,  Thepchai Supnithi
<EMAIL_ADDRESS><EMAIL_ADDRESS>Conference Announcements
slideslive.com/38929467
38929467

9005
EMNLP 2020
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Conference Announcements
slideslive.com/38929468
38929468

9006
COLING 2020
Donia Scott, Nuria Bel, Chengqing Zong
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Conference Announcements
slideslive.com/38929469
38929469

9007
AACL 2020
Kam-Fai Wong, Kevin Knight, Hua Wu
<EMAIL_ADDRESS>Conference Announcements
slideslive.com/38929470
38929470

8002
Lifetime Achievement Award - Speech

<EMAIL_ADDRESS>Lifetime Achievement Award
slideslive.com/38929471
38929471

8004
Lifetime Achievement Award (LAA) - Introduction of the recipient
Hinirich Schütze
<EMAIL_ADDRESS>Lifetime Achievement Award
slideslive.com/38929472
38929472

8005
Distinguished Service Award (DSA) - Introduction of the recipient
Hinirich Schütze
<EMAIL_ADDRESS>Distinguished Service Award
slideslive.com/38929473
38929473

8006
Distinguished Service Award (DSA) - talk by the recipient

<EMAIL_ADDRESS>Distinguished Service Award
slideslive.com/38929474
38929474

7000
Announcement/Introduction BP Awards
Joyce Chai, Natalie Schluter, Joel Tetreault
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Best Paper Award
slideslive.com/38929475
38929475

9008
ACL 2023 Call for Bids
Tim Baldwin
<EMAIL_ADDRESS>Conference Announcements
slideslive.com/38929476
38929476


Actor model didn't update correctly when upgrade megatron to core-r0.6.0

I tested veRL using llama-2-7b(with a smaller train_batch_size), and the results seem to meet expectations,
(main_task pid=27483) step:3 - timing/gen:12.597 - timing/ref:0.522 - timing/values:0.457 - critic/kl:0.016 - critic/kl_coeff:0.001 - timing/adv:0.020 - timing/update_critic:1.744 - critic/vf_loss:0.317 - critic/vf_clipfrac:0.111 - critic/vpred_mean:-0.001 - timing/update_actor:1.537 - actor/entropy_loss:0.130 - actor/pg_loss:0.005 - actor/pg_clipfrac:0.002 - actor/ppo_kl:0.000 - timing/testing:11.108 - val/test_score/openai/gsm8k:0.188 - critic/score/mean:0.031 - critic/score/max:1.000 - critic/score/min:0.000 - critic/rewards/mean:0.028 - critic/rewards/max:0.997 - critic/rewards/min:-0.015 - critic/advantages/mean:-0.000 - critic/advantages/max:3.696 - critic/advantages/min:-6.231 - critic/returns/mean:0.023 - critic/returns/max:1.000 - critic/returns/min:-0.015 - critic/values/mean:0.441 - critic/values/max:4.250 - critic/values/min:-1.859 - response_length/mean:236.156 - response_length/max:443.000 - response_length/min:65.000 - prompt_length/mean:92.688 - prompt_length/max:146.000 - prompt_length/min:48.000

Then I want to migrate to Megatron 0.6, I made the following modifications.

Modification Megatron-LM

patch get_model_config like 0.4.0
Modify DistributedDataParallel class

Add self.data_parallel_group = data_parallel_group in __init__
Modify src=torch.distributed.get_process_group_ranks(self.data_parallel_group)[0] in line272, according to this PR.

Modification in veRL

import path

change from megatron.utls import print_rank_0, unwrap_model to from megatron.training.utils import print_rank_0, unwrap_model
change from megatron.optimizer/timer ...  to from megatron.core.optimizer/timer ...

modify megatron_actor/critic.py

modify the paramter to call zero_grad_buffer() & optimizer.step()
remove hidden_size parameter when call forward_backward_func() on line 223.

modify get_megatron_optimizer function like follow

def get_megatron_optimizer(
        model,
        config: OptimizerConfig,
        no_weight_decay_cond=None,
        scale_lr_cond=None,
        lr_mult=1.0,
        check_for_nan_in_loss_and_grad=False,
        overlap_param_gather=False  # add for verl
):
        ...
        import torch
        from megatron.core import mpu

        def init_state_fn(opt):
            for group in opt.param_groups:
                for p in group['params']:
                    if len(opt.state[p]) == 0:
                        opt.state[p]['exp_avg'] = torch.zeros_like(p.data)
                        opt.state[p]['exp_avg_sq'] = torch.zeros_like(p.data)

        per_model_buffers = {}
        for model_idx, model_chunk in enumerate(model_chunks):
            if hasattr(model_chunk, 'buffers'):
                per_model_buffers[model_idx] = model_chunk.buffers

        if config.use_distributed_optimizer:
            # return DistributedOptimizer(optimizer, config.clip_grad, config.log_num_zeros_in_grad,
            #                             check_for_nan_in_loss_and_grad, params_have_main_grad, config.fp16, config.bf16,
            #                             config.params_dtype, grad_scaler, model, overlap_param_gather)
            return DistributedOptimizer(
                optimizer=optimizer,
                config=config,
                grad_scaler=grad_scaler,
                init_state_fn=init_state_fn,
                per_model_buffers=per_model_buffers,
                data_parallel_group=mpu.get_data_parallel_group(with_context_parallel=True),
                data_parallel_group_gloo=mpu.get_data_parallel_group_gloo(with_context_parallel=True),
                data_parallel_group_idx=torch.distributed.get_rank(mpu.get_model_parallel_group())
            )

The code to reproduce is on this branch.
When I rerun the scripts, the metrics is weired, the critic/kl is always zero:
(main_task pid=32059) step:3 - timing/gen:13.604 - timing/ref:0.478 - timing/values:0.456 - critic/kl:0.000 - critic/kl_coeff:0.001 - timing/adv:0.020 - timing/update_critic:1.597 - critic/vf_loss:0.363 - critic/vf_clipfrac:0.147 - critic/vpred_mean:0.099 - timing/update_actor:1.583 - actor/entropy_loss:0.134 - actor/pg_loss:0.015 - actor/pg_clipfrac:0.001 - actor/ppo_kl:-0.000 - timing/testing:11.299 - val/test_score/openai/gsm8k:0.031 - critic/score/mean:0.094 - critic/score/max:1.000 - critic/score/min:0.000 - critic/rewards/mean:0.094 - critic/rewards/max:1.000 - critic/rewards/min:0.000 - critic/advantages/mean:0.000 - critic/advantages/max:4.966 - critic/advantages/min:-4.105 - critic/returns/mean:0.096 - critic/returns/max:1.000 - critic/returns/min:0.000 - critic/values/mean:0.578 - critic/values/max:3.250 - critic/values/min:-1.906 - response_length/mean:251.750 - response_length/max:479.000 - response_length/min:87.000 - prompt_length/mean:85.469 - prompt_length/max:154.000 - prompt_length/min:53.000

If I understand correctly, the critic/kl here is comparing the outputs of the actor policy and the ref policy. I printed the actor's parameter tensor, and upon comparison, the parameters indeed did not change before and after optimizer.step(), but the gradients are not zero (What’s even stranger is the critic_module seems to be updating normally).
Perhaps this is not an issue with veRL, but rather that I am not using veRL and Megatron correctly. However, this seems to be the most promising place to resolve my confusion. From this RFC, it seems that veRL is also trying to adapt to higher versions of Megatron. Do you have any suggestions?

Hi @Wodswos, thanks for your detailed feedback!
Your understanding of critic/kl is correct and I think the actor model weight is not updated when upgraded to v0.6.0.
I think this issue is only related to the actor model while the critic model can be updated correctly due to the mismatch between the newer Megatron-LM and veRL's HybridEngine implementation. Here are some analyses:

Since Megatron v0.6.0, the GradBuffer used in DistributedDataParallel in Megatron-LM was replaced by ParamAndGradBuffer in param_and_grad_buffer.py. In this commit, the Megatron-LM authors map all the model parameter to a continuous param_buffer.

In v0.4.0 (and also v0.5.0), we implemented an AllGatherPPModel in megatron_vllm.py to support our 3D-HybridEngine design. This approach establishes a contiguous buffer for each model parameter and directly maps it to the param.data in Megatron-LM. However, in v0.6.0 (and later), directly binding our model memory buffer to param.data ignores the new ParamAndGradBuffer implementation. As a result, the model parameters are not updated correctly.

Proposed Solutions
To address this issue in v0.6.0 and later versions, I recommend the following quick fixes:

Bind veRL's memory_buffer to ParamAndGradBuffer:
Instead of mapping veRL's buffer directly to param.data in the DistributedDataParallel module (as seen in memory_buffer.py), we should bind veRL's memory_buffer to the ParamAndGradBuffer for each model parameter. This ensures compatibility with Megatron-LM's updated parameter management logic.

Align Padding Logic:
Verify that the padding logic in veRL's MemoryBuffer (defined in memory_buffer.py) matches the padding logic in ParamAndGradBuffer (defined in param_and_grad_buffer.py). Consistent padding ensures that the buffers are aligned correctly for parameter updates.

By implementing these solutions, the actor model's weights should update correctly in v0.6.0 and beyond. Let me know if you need further clarification or assistance!


pysm.Instrument seems not working properly in some cases

Hello,
I have a problem using pysm.Instrument.
I use this standard sky configuration:
nside = 128
sky_config = {
'synchrotron': models('s1', nside),
'dust': models('d1', nside),
'freefree': models('f1', nside),
'cmb': models('c1', nside),
'ame': models('a1', nside)}

then I create a sky with:
sky = pysm.Sky(sky_config)

Now, I create the two following different instruments:

instrument_mono_noiseless = pysm.Instrument({
    'nside': nside,
    'frequencies' : np.array([143]),
    'use_smoothing' : True,
    'beams' : np.array([7.1]),
    'add_noise' : False,
    'noise_seed' : None,
    'sens_I': None,
    'sens_P': None,
    'use_bandpass' : False,
    'channel_names' : ['143_GHz'],
    'channels' : None,
    'output_units' : 'uK_RJ',
    'output_directory' : "./",
    'output_prefix' : "noiseless_mono",
    'pixel_indices' : None})

instrument_mono = pysm.Instrument({
    'nside': nside,
    'frequencies' : np.array([143]),
    'use_smoothing' : True,
    'beams' : np.array([7.1]),
    'add_noise' : True,
    'noise_seed' : 0,
    'sens_I': np.ones(1),
    'sens_P': np.ones(1),
    'use_bandpass' : False,
    'channel_names' : ['143_GHz'],
    'channels' : None,
    'output_units' : 'uK_RJ',
    'output_directory' : "./",
    'output_prefix' : "mono",
    'pixel_indices' : None})

I observe the sky through these two instruments with instrument_*.observe(sky) and then I read the corresponding maps that have been produced.

In the first case the created map does not have the polarization components but only the intensity one. In fact, you will get an error message if you try:
hp.read_map('noiseless_mono_nu0143p00GHz_total_nside0128.fits', field=(0, 1, 2))

while the right command would be:
hp.read_map('noiseless_mono_nu0143p00GHz_total_nside0128.fits', field=(0))

but this means that Q and U maps have not been generated.

In the second case, the three maps (I, Q, U) are generated but in a wrong way since the Q and U maps are in the same order of magnitude of the I map:
hp.read_map('mono_nu0143p00GHz_total_nside0128.fits', field=(0, 1, 2))

array([[-36.27698898,  14.9692564 ,  25.86756897, ...,  76.63456726,
         81.74451447,  67.1904068 ],
       [-36.27647781,  14.91915131,  25.87987328, ...,  76.58881378,
         81.76040649,  67.25320435],
       [-36.32805252,  14.94966221,  25.84389114, ...,  76.66394043,
         81.7594986 ,  67.2399292 ]])

Is there something I'm doing wrongly somewhere?

Hi, I think the problem is related to the number of frequencies passed to pysm.Instrument. When only one frequency channel is given the smooth function doesn't work properly, if more frequencies are given then everything appears to work well.


Update minimum supported Unity version in docs

Due to prefab changes in Unity 2018.3, that's our minimum supported Unity version.

/azp run mrtk_docs


GCP Secret Vault Error

Loading secrets from GCP via metadata service is failing in gcloud-sdk 0.25.0 (https://github.com/abdolence/gcloud-sdk-rs/pull/154).
Failed to refresh secrets: NetworkError(SecretVaultNetworkError { public: SecretVaultErrorPublicGenericDetails { code: "Unknown" }, message: "status: Unknown, message: \"gce metadata service error: Error retrieving data from metadata server: 404 Not Found(Not Found\\n). URL: http://metadata.google.internal//computeMetadata/v1/instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform\", details: [], metadata: MetadataMap { headers: {} }" })

A fix was added to gcloud-sdk 0.25.2

Hey, thanks for the PR.
While I'm fine with merging it strictly speaking it is not necessary since people would need to update their Cargo.lock file anyway (either for gcloud or this crate).

This is fixed in Google Cloud SDK crate v0.25.2. Anyone who bumps into it, please update your Cargo.lock file.

Hey, thanks for the PR. While I'm fine with merging it strictly speaking it is not necessary since people would need to update their Cargo.lock file anyway (either for gcloud or this crate).

This is why I opened the PR, because I had done a cargo update on our project after some code changes and wasn't sure why things stopped working. Spent hours looking at our code to realize it had nothing to do with our code.
Figured this would prevent someone else from spinning their wheels.

Hey, thanks for the PR. While I'm fine with merging it strictly speaking it is not necessary since people would need to update their Cargo.lock file anyway (either for gcloud or this crate).

This is why I opened the PR, because I had done a cargo update on our project after some code changes and wasn't sure why things stopped working. Spent hours looking at our code to realize it had nothing to do with our code.
Figured this would prevent someone else from spinning their wheels.

Thanks for the visibility bump. I also was hit by this error and spent hours looking for fix.

Sorry to hear this was disruptive for you. This whole thing is related to Hyper 1 (and related deps) upgrade, and hopefully now this kind of thing wouldn't happen for a while.


Edit monitor page doesn`t redirect after clicking save button

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
When I edited the monitor I considered that after clicking the save button not able to redirect to the monitor page.
👟 Reproduction steps

Go to the dashboard page and click the monitor form the monitor list.

And click the edit button

And click the save button.

👀 Expected behavior
Expected to redirect to the monitor page after clicking the save button on the edit page.
😓 Actual Behavior
The edit monitor page doesn`t redirect after clicking the save button.
🐻 Uptime-Kuma Version
most latest version
💻 Operating System and Arch
Ubuntu 22.04.1 LTS
🌐 Browser
Google chrome latest
🐋 Docker Version
No response
🟩 NodeJS Version
16.10.0
📝 Relevant log output
No response

Yes, it is.


Document DataGranule "umm" and "meta" members

Follow-up from #367.
Granules have granule["umm"] and granule["meta"] members that are not currently documented. What's the best way to document this? Expose them in a way that our autodoc gets it? Or write a how-to? Both?

The UMM and Metadata components are/should be documented elsewhere.  If they are documented elsewhere, I don't think earthaccess needs to document what these metadata components are.  Maybe just link to other online resources.  However, we should expose/demonstrate how to access these.
Would something as simple as granules["umm"].keys() do the job.  Along with dome common use cases such as the xml example here.

However, we should expose/demonstrate how to access these.

:100: Agreed we shouldn't document the structure of the data, just how to access the attributes from earthaccess!

However, we should expose/demonstrate how to access these.

💯 Agreed we shouldn't document the structure of the data, just how to access the attributes from earthaccess!

I also agree... better to maintain only one source. If you know already, could you say where these members are used in the earthaccess guide and a link to the source material?

I don't believe we have any documentation that references these attributes. Yet :)


React Transition Group

What version of React-Table are you using?
6.7.5
What bug are you experiencing, or what feature are you proposing?
I would like to animate enter and exit of data rows with React Transition Group but I'm getting errors:
Warning: Unknown event handler property `onExited`. It will be ignored. and Warning: Received `true` for a non-boolean attribute `in`. Is there any proper way to do this?
Use https://codesandbox.io/s/X6npLXPRW (by clicking the "Fork" button) to reproduce the issue.
https://codesandbox.io/s/p56wxv367
What are the steps to reproduce the issue?
Click on Remove one item button
Thanks in advance

However, I don't remember seeing anyone talking about using React Transition Group with ReactTable - it looks like you are on the right track but...
The issue that you will be hitting is that the props in ReactTable are all validated as of 6.7.5 and, as those props are not known by the existing components they are triggering an error (and I suspect are not being passed through due to the prop validation).  The problem, of course, there's no easy way to remove the propTypes without editing the code.
Normally I would suggest this be taken over to the #react-table Slack channel but I think this may need to be considered as a correction to the code (if we can find a way to do it and retain the existing props validation).
@frankolson & @tannerlinsley might want to provide an opinion in regard to this. Possibly a little too tight given how much can be overridden and what props a developer may need to pass ?

If anyone is interested, got this working with
<tbody {...getTableBodyProps()}>
 <TransitionGroup component={null}>
{rows.map((row, i) => {
            prepareRow(row);
            return (
              <CSSTransition key={yourUniqueDataKey} timeout={yourTimeout} classNames="yourClassName">
// etc...


Fix #462 flattening of iterators without len

Fixes https://github.com/python-streamz/streamz/issues/462

Thanks for putting this together


chore(deps): reduce chakra-ui dep size

chore: remove @chakra-ui/react and add only import @chakra-ui components
feat: minor feature for swapping chevron icon when menu is open/closed
feat: remove the TypeScript warning on chakra.div, in Option component. Wrapped in Box component to update the isDisabled prop

Thanks again for the PR!  It has been merged and released in v3.0.6


Plan for Stable Release?

Is there any plan for a stable release which we can reference in separate projects?
My team recently released some code which used Cartographer Turtlebot with a custom parameter file and instructed users to install the Cartographer code via the read the docs site. However, shortly after this we discovered that the default parameter file had been changed and since new users were pulling the most recent code, they got all sorts of errors when using the parameter file we provided.
This isn't the end of the world as it ended up being a pretty simple fix but I was just curious if there was any plans to publish a static/stable release that  others can reference so we don't run into this issue again? Given the current installation instructions, would the only real option be to tell our users to checkout a specific commit from each cartographer repository before proceeding with the installation?

Cartographer is pre-1.0 as of now. We plan to follow semantic versioning eventually, but as of now do not have a stable public API.
We plan to tag a 0.1.0 pre-releases soon, but we give no stability guarantees until 1.0.

No problem. I figured you guys were still in the early stages. I'll keep an eye out for the pre-releases and go from there. Thanks!


adding roosevelt university to list of universities so students from …

…there can use IntelliJ for free

@mauller77 Please provide us with an URL of a page or another proof (.pdf or a screenshot are OK) showing that the school recognizes the domain which you are submitting as an official email domain for the students. Thank you.

Something like this?
Regards,
Joshua Moller

From: Philip Torchinsky<EMAIL_ADDRESS>Sent: Friday, August 9, 2019 4:54 AM
To: JetBrains/swot<EMAIL_ADDRESS>Cc: Josh Moller<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [JetBrains/swot] adding roosevelt university to list of universities so students from … (#6570)
@mauller77https://github.com/mauller77 Please provide us with an URL of a page or another proof (.pdf or a screenshot are OK) showing that the school recognizes the domain which you are submitting as an official email domain for the students. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/JetBrains/swot/pull/6570?email_source=notifications&email_token=ADX2UQHDASP4V4EU3E3S643QDVEE3A5CNFSM4IHUZAAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD36KXYI#issuecomment-519875553, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADX2UQF6F2UJP6CTR75LYE3QDVEE3ANCNFSM4IHUZAAA.

@mauller77 I am sorry, GitHub strips out an attachment if you send it by email in reply to a notification from GitHub. Please attach the proof which you have sent already via GitHub web interface. Thank you!

@mauller77 There must be some mistake, I suppose. The .pdf which you added to your request says that the domain is mail.roosevelt.edu, not mail.roosevelt.com. The original file path in your request is lib/domains/com/roosevelt/mail.txt, and it represents mail.roosevelt.com email domain. Could you please:

Remove .pdf file from the request (it's not necessary there, you might wish to attach it to the comment (not including the file to the pull request itself), but it has been viewed already, thus we don't need it anymore).
Rename lib/domains/com/roosevelt/mail.txt to lib/domains/edu/roosevelt/mail.txt, if you wish to add mail.roosevelt.edu to the list of domains belonging to educational organizations known to JetBrains.

Thank you!

Sorry about that, the initial instructions told me to place the file into the com folder and I did not really look into the reason and purpose behind the com folder so that is my bad. I have moved it from the com folder to the edu folder. Sorry for the confusion.
Regards,
Joshua Moller

From: Philip Torchinsky<EMAIL_ADDRESS>Sent: Wednesday, August 14, 2019 1:22 AM
To: JetBrains/swot<EMAIL_ADDRESS>Cc: Josh Moller<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [JetBrains/swot] adding roosevelt university to list of universities so students from … (#6570)
@mauller77https://github.com/mauller77 There must be some mistake, I suppose. The .pdf which you added to your request says that the domain is mail.roosevelt.edu, not mail.roosevelt.com. The original file path in your request is lib/domains/com/roosevelt/mail.txt, and it represents mail.roosevelt.com email domain. Could you please:

Remove .pdf file from the request (it's not necessary there, you might wish to attach it to the comment (not including the file to the pull request itself), but it has been viewed already, thus we don't need it anymore).
Rename lib/domains/com/roosevelt/mail.txt to lib/domains/edu/roosevelt/mail.txt, if you wish to add mail.roosevelt.edu to the list of domains belonging to educational organizations known to JetBrains.

Thank you!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/JetBrains/swot/pull/6570?email_source=notifications&email_token=ADX2UQBNZY7KXPWAD4FRMILQEOXB5A5CNFSM4IHUZAAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4H5VYY#issuecomment-521132771, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADX2UQETKJATLHTUN2WD24DQEOXB5ANCNFSM4IHUZAAA.

@mauller77 Thank you for the update! Pull request merged.


Linux AMI harcoded

Please abstract the linux AMI information to a variable so customers that only have access to private market places, such as the F5 UDF environment, can still deploy the f5 provisioner.
Another option would be to search for the latest version of the AMI versus requiring a hardcoded version.
Related to issue #7

Will add provision for Linux as well as F5 AMI

Provisioner repo has been moved to devCentral: https://github.com/f5devcentral/FAS-provisioner. For any of the issue, we will fixit in devCentral Repo.
Please help duplicate the issue to devCentral repo.
Thanks @codygreen

migrated to https://github.com/f5devcentral/FAS-provisioner/issues/1


Not an Issue : Looking for a snippet

Hi yan,
Is there a reference to create  Section box like below Image.

@harimurra not sure what you are asking for? A sample how to include this image?

Hi yan,
not an  Image, am looking for How to Create a Rectangular Box to Contain Text.

Well you may want to look at
https://raw.githubusercontent.com/yan74/afplib/cf5cfe6589bf6409f1c6925f4c9821bb1f81462b/org.afplib.samples/src/main/java/org/afplib/samples/snippets/CreatePage.java
This puts "Hello World" on a page - in AFP text flows according to writing direction (aka inline direction) and font spacing. You can't define a box that aligns the text within. You would have to calculate dimensions of the text yourself and align accordingly by breaking the text into separate TRNs.

out.writeStructuredField(new AfpBuilder() 				.withMember(new AfpBuilder()								//  Color 						.with(AfplibPackage.SEC__COLSPCE, SECCOLSPCE.CONST_CMYK_VALUE) 						.with(AfplibPackage.SEC__COLSIZE1, 0) 						.with(AfplibPackage.SEC__COLSIZE2, 0) 						.with(AfplibPackage.SEC__COLSIZE3, 0) 						.with(AfplibPackage.SEC__COLSIZE4, 31) 						.with(AfplibPackage.SEC__COLVALUE, new byte[] { (byte) 0x00, (byte) 0x00,  0x00,0x1F} ) 						.create(SEC.class)) 				.withMember(new AfpBuilder() 						.with(AfplibPackage.AMI__DSPLCMNT, 1400)				 						.create(AMI.class)) 				.withMember(new AfpBuilder() 						.with(AfplibPackage.AMB__DSPLCMNT, 130)			 						.create(AMB.class)) 				.withMember(new AfpBuilder() 						.with(AfplibPackage.DBR__RLENGTH, 550)  // Section Box 						.with(AfplibPackage.DBR__RWIDTH, 850) 						.create(DBR.class)) 				.create(PTX.class));


Utilizing configure_gclk_divider_and_source and GENA enums its possible for multiple parties to attempt to take the same clock

Discovered in #113
Its possible for user and or library subsystems to both attempt to take the same clock causing panic.
We should look at how to have the clock tree hand out gclk 3-11 just like we do for glck0 and gclk1

This will no longer be possible with #450.


SOLR-17337: Show proper distributed stage id

https://issues.apache.org/jira/browse/SOLR-17337
Description
When showing track debug for custom stages in distributed requests, the stage id is not translated into a meaningful string. It is displayed as an empty string (see attachment). Furthermore, if you have multiple custom stages, the details for the most recent stage is displayed. The details for other stages is omitted.
Solution
If a given stage has no translation in the stage map, we format it's name in the format STAGE_<STAGE_ID>. By doing so, no stage details are omitted, even if you have multiple custom stages.
Checklist
Please review the following and check all that apply:

[x] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
[x] I have created a Jira issue and added the issue ID to my pull request title.
[ ] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
[x] I have developed this patch against the main branch.
[x] I have run ./gradlew check.
[ ] I have added tests for my changes.
[ ] I have added documentation for the Reference Guide

Triggered the GitHub jobs.  Is this too trivial to need a test?

There's only one valid answer to this, I'll add a test 👍

I think it really ends up being an integration test showing the behavior works?

I added a simple resolution test. Testing the distributed behaviour would mean adding a new test Solr configuration and adding a new test component that actually does a couple of distributed requests in custom stages :-/

I assigned it to me so I don't loose track of it!

I added a simple resolution test. Testing the distributed behaviour would mean adding a new test Solr configuration ...

The config APIs could be used instead of having another test Solr configuration -- e.g. https://github.com/apache/solr/blob/releases/solr/9.6.1/solr/core/src/test/org/apache/solr/handler/component/CustomHighlightComponentTest.java -- but yeah that would be quite a bit of code compared to a simple resolution test.

I think we need a gw tidy ;-)

closing in favour of #2594 variant


Not starting issue, errors on canceling open orders

Hello,
With a fresh install, here are the blackbot outputs on startup :
python BlackBot.py sample-bot.cfg
[Nov 05 2018 14:32:32 UTC] : Reading config file 'sample-bot.cfg'
[Nov 05 2018 14:32:32 UTC] : --------------------------------------------------------------------------------
[Nov 05 2018 14:32:32 UTC] :           Address : VALID ADDRESS
[Nov 05 2018 14:32:32 UTC] :   Amount Asset ID : WAVES
[Nov 05 2018 14:32:32 UTC] :    Price Asset ID : VALID ASSET CODE
[Nov 05 2018 14:32:32 UTC] : --------------------------------------------------------------------------------
[Nov 05 2018 14:32:32 UTC] :
[Nov 05 2018 14:32:32 UTC] : Cancelling open orders...
Traceback (most recent call last):
File "BlackBot.py", line 122, in
BLACKBOT.cancelOpenOrders(PAIR)
File "/home/cm/.local/lib/python2.7/site-packages/pywaves/address.py", line 858, in cancelOpenOrders
status = order['status']
TypeError: string indices must be integers
Can someone give me some help ?

What does your config file look like?

Had the same problem, you need to switch Asset ID and Price Asset ID in the config file, then it works.


Test that numpy literals in graph don't get inlined

OK, so it's important that numpy arrays stay where they are in the graph and don't get fused.  This is important for two reasons:

We can serialize values better if they are naked in the graph rather than enclosed within tasks
If a value becomes inlined into a fast-function task (like getitem) then it can be copied to many other tasks during fast_function_inlining.  This can cause blowup of serialization costs.

cc @eriknw any thoughts on how to address this cheaply?

Include the tasks you don't want fused in the keys= keyword argument to fuse.  All or most optimization functions accept keys=.

Can do.
General question about fuse though, do we want to fuse keys that point to
literals rather than tasks?
On Tue, Mar 14, 2017 at 1:01 PM, Erik Welch<EMAIL_ADDRESS>wrote:

Include the tasks you don't want fused in the keys= keyword argument to
fuse. All or most optimization functions accept keys=.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/2080#issuecomment-286489252, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszH80ZZU-n24jghy42-M8RhxkyEfhks5rlsf2gaJpZM4Mcyt1
.

OK, expanding on this a bit I actually also don't want to fuse the last task of a getitem chain.  When thinking about distributed computing we want to quickly split up large numpy arrays with getitem calls, and then move them to other workers as necessary before fusing with any other computation.  We want to avoid moving large intermediates.
So in regards to fusion this is a bit odd.  We want to fuse long getitem chains (this is very important for single-machine xarray workloads).  But we don't want to fuse a getitem chain to anything else that comes afterwards (like a +100 operation), this helps keep data movement easier in a distributed system.
My current approach uses the suggestion from @eriknw to specify a set of keys to specifically avoid fusing.  We start at the bottom of the graph from all data nodes (anything that isn't a task) and move up while there are getitem chains.  We claim the topmost element of that chain as a key that we don't want to lose/fuse away.  This costs two linear scans, one to find data and then one call to reverse_dict (possibly expensive).

I would like to merge this soon.  @shoyer I've run this against the xarray test suite and didn't run into any problems.  Still you might want to look at this if you get a chance.  This stops Dask from fusing getitem calls into other, non-getitem calls.

OK, I'm going ahead with this.