[ENHANCE] [DATA] Update NICD Hospitalization Data

Hi All,
Would it be possible to have an update regarding the following dataset?
covid19za/data/nicd_hospital_surveillance_data.csv

Edit by @vukosim
Resume updating NICD Hospitalisation CSV File
Option 1 - Scraper To Extract the Daily Table and then Update the CSV, see below for screenshot
Option 2 - Human Computation, A Volunteer to update the numbers every morning.

The challenge has been keeping up with the updates of the NICD Hospital Admissions reports. They are available here https://www.nicd.ac.za/diseases-a-z-index/disease-index-covid-19/surveillance-reports/daily-hospital-surveillance-datcov-report/
The second page of the daily reports has this table

Now if we can get someone to start doing backfill (start with 1 December 2021 for example, and work backwards it would be great @maximeLpt

Do you have a scrapper for that?

@maximeLpt no. It was initially filled in by a volunteer, day by day.

Is there no chance NICD can/will just share the table in Excel or CSV format?

A friend has made the request, we will see if she gets a response @anelda

Hey @anelda @vukosim let me know what the NICD says, and if not luck comes - then we can surely build a wrapper/scraper for the PDF's or find a way to parse the data from a different format?

Hi all,
I had a go at scraping the NICD datasets for data since the 27 October 2020 (when the daily reports table has 10 columns). There is still work to do to get some of the columns described in covid19za/data/nicd_hospital_surveillance_data.csv  (general, high care, isolation, total health care workers admitted) and in filtering for the many ways in which dates appear in the raw pdf files. Hope that this can be useful as a basis for scraping if there is no luck with NICD:
https://gist.github.com/sjbeckett/f1d3822db7d41d33dfddd01814a64481

I managed to write a fairly decent scraper for the pdfs. Perhaps the script can be enhanced by adding a filter for a particular day. eg. r Report_Date >= Sys.Date() to avoid re-downloading each file.  The script can run a cronjob.
Another script is the table scraper. The results were mixed. This could be due to changes to how tables are formatted , how the files are generated etc. Extracting each table from all files can be time-consuming. There probably exists a way to uniformly format the tables into a sound format in another language to technique.
directory containing the scripts

Hi all,
I had a go at scraping the NICD datasets for data since the 27 October 2020 (when the daily reports table has 10 columns). There is still work to do to get some of the columns described in covid19za/data/nicd_hospital_surveillance_data.csv (general, high care, isolation, total health care workers admitted) and in filtering for the many ways in which dates appear in the raw pdf files. Hope that this can be useful as a basis for scraping if there is no luck with NICD:
https://gist.github.com/sjbeckett/f1d3822db7d41d33dfddd01814a64481

Thank you. i think this is a good start and we can then try to start filling in where the scraper could not.

All
I did some work on the hospitalization data too.    Actually my son :-).
Not in the data/nicd_hospital_surveillance_data.csv file yet, but in another file with a different format.
See data/covid19za_provincial_raw_hospitalization.csv
The scraper lives in scripts/daily_nicd_datcov.R.
And the github workflow runs this script every night -- it appears as if the posting / publication of these files are manual, so I'm not 100% sure what the best time would be.
Remaining todo:

make sure the scraper runs stable
extract the relevant parts and update the old summary hospitalization file, which is partially done already.
g

Hi all,
I had a go at scraping the NICD datasets for data since the 27 October 2020 (when the daily reports table has 10 columns). There is still work to do to get some of the columns described in covid19za/data/nicd_hospital_surveillance_data.csv (general, high care, isolation, total health care workers admitted) and in filtering for the many ways in which dates appear in the raw pdf files. Hope that this can be useful as a basis for scraping if there is no luck with NICD:
https://gist.github.com/sjbeckett/f1d3822db7d41d33dfddd01814a64481

Just a note I updated the above gist to incorporate general and high care patient numbers.
Of note, isolation was no longer reported after 02-09-2020; and admitted healthcare workers were no longer reported after 08-10-2020.

Greetings everyone. I took a different approach to this issue and created an API which is free and publicly available. At the moment, the database has data from 01 July 2021 to 28 Dec 2021.
The API has 2 main endpoints:

/all:

Gets all data from the the database

/dates:

Allows for filtering data between 2 specific dates. Province is not required but you can specify it if you are looking for a specific province data..

Without Province

With Province

Getting the PDFs from NICD website and uploading the data to a cloud database is automated but annotating the table on the PDF is still manual because the PDF formatting varies inconsistently between the documents so that is the only manual process thus far.
Try the API and I would appreciate some pointers and thoughts:
https://covidza-data.deta.dev/docs

Happy new year everyone. There will be a bit more action to finalise this in the next 2 weeks. Thank you so much for the work and ideas.


fix 1692 set backup target is configured flag after setting

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Problem:

Without setting the flag, the "take backup" action is refused
Solution:

Set the configured flag when setting backup target
Then it is allowed to "take backup"
Related Issue:
#1692
Test plan:

Install Harvester

Set backup target

"take backup" on VM, and is allowed

Edit backup target to default value

"take backup" on VM, and is refused

@guangbochen @FrankYang0529
This is an partial update to PR 1656, follow the original process.
#1656
Not as our expectation, the flag still needs to be set explicitly.
Otherwise below function checkBackupTargetConfigured -> harvesterv1.SettingConfigured.IsTrue(targetSetting) will return error.
Alternative is to change it, but not this time.
pkg/api/vm/handler.go
 func (h *vmActionHandler) checkBackupTargetConfigured() error

 if err == nil && harvesterv1.SettingConfigured.IsTrue(targetSetting) {   ----> not meet, and return error

BTW:  With this fix, the https://github.com/harvester/harvester/issues/1673 is verified on Harvester ISO of today.


Unable to use spring-cloud-sleuth with weblogic jms

Weblogic jms ConnectionFactory impements both XAConnectionFactory and ConnectionFactory. In the latest release of sleuth this wont work properly, and while we want to use just ConnectionFactory, we will get LazyXAConnectionFactory autowired, because sources does not cover this combination.
You have already fix for this in master from march, commit: e099b6d9f40e7ac5afab58f77ccdf7773b1116a8
could this be cherry-picked into release branch? This blocks us from updating to newest spring/sleuth. If it's not possible, could you offer some workaround?

Please upgrade to 2.2.0.RC2. 2.2.0.RELEASE is right around the corner.

thanks.


Installation problem with css

I'm following the installation process on the Readme, but the CSS files are not created. When I launch this function : nodejs node_modules/gulp/bin/gulp.js less, nothing is created in the web/css folder.
I tried to give full rights to the files, so it is not a file rights problem.
In the end, I don't have any all.css file, and the design of the website is pretty broken. All the rest seems to be working perfectly otherwise.

Actually the code used is the following in gulpfile.js :
    return gulp.src('./src/**/Resources/public/css/*.less')
        .pipe(less())
        .pipe(concat('all.css'))
.pipe(gulp.dest('./web/css'));

But I checked all the bundles and there are no files corresponding. There are some .css files in src/GateBundle/Resources/public/css. It's the only files that could correspond and it's not .less files.

Hello @chougron !
You're right, the gulpfile needs some update, indeed there is not .less to compile anymore, I'll open an issue to clean this!
However, the .css files in the GateBundle should be moved in you web directory. This is done by the command php app/console assets:install (in the readme too).
Can I ask you to run it again to test?

Yes, the .css files were moved to the web directory.
But there is the file css/all.css being called in the admin pages, that doesn't exist.
I also notice that you have a different version of Semantic UI vendor/semantic-ui-css/*
The version I have by following the installation process is 2.2.4, but when I access the files on badger.akeneo.com, the version is 2.1.7.
Is is causing problems in the administration : The content of the page is behind the menu and the dropdown items appears one row under their parents (making them unable to select, since you hover out of the parent without hovering in the child)

Heyy you're right, we still use semantic ui for the admin part! I'd like to drop that, kind of "legacy" stuff.
I'll try a brand new install on my side to reproduce this problem, this part needs a clean up :)

I pulled the code today with the new changes, and the installation worked fine! Thanks!

Hell @chougron , yup, I put back the main.less file that was missing :)
I'm currently dropping the semantic ui theme on admin side to have the same everywhere ( #107 ) if you're interested.
Have a nice day!


Update ocm client and schemas

py-mtcli
Description
Short description of the change:

[Schema change]Renamed subscriptionConfig to config.
[Schema change]Moved the secrets attribute under the now renamed config attribute
Addon Loader updates to include methods for accessing attributes.
Update the ocm client to include secretPropagations attribute in the addon/addon-version payload.

LGTM. Just one thing, now that the secrets are present inside the config attribute, won't the existing Addons fail the schema validation?

Yes, but i will move them under config when i use this version of the cli in the mt-repo.

LGTM. Just one thing, now that the secrets are present inside the config attribute, won't the existing Addons fail the schema validation?

Yes, but i will move them under config when i use this version of the cli in the mt-repo.

OK then. Looks good to me!


Tests for key commands on ui menu

What issue are you having that you need harpoon to solve?
Problem: tests do not consider key presses and how the ui menu should react.
Solution: tests to cover that so i don't accidentally break peoples!

Hello,
I am willing to take this, it would be my first contribution to a neovim plug so might take a while. If you have any tips or pointers that'd be great :).

@PrathamLalwani The code is already commented out in the UI tests. It's just a matter of commenting it in, and ensuring the menu behaves according to keys pressed
You should be able to follow the tests that are already in there.

Great, Thanks!


Revert for 2103

Reverts the commits which appear likely to be causing #2103 to avoid unintentionally breaking code that uses Winston.

Note: the failed CI check was a decrease in test code coverage understandably associates with reverting a PR that increased it.


Add Deserialize to Tags

This was motivated by the web worker PR, I assume

I noticed it was missing earlier even, when refactoring osm-tags.
But now the webworker really needs this. (or the other refactor)


Product skeleton: Add the product details and add to cart button

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
When the user sees the product, they read a few details regarding it including Reviews, Name, Price, MRP and Image on the tile.
Use Case
This feature will increase the user experience.
Benefits
No response
Add ScreenShots

Priority
Medium
Record

[X] I have read the Contributing Guidelines
[ ] I'm a GSSOC'24 contributor
[X] I have starred the repository

@MAVRICK-1 hey are the contibution still going on?

@samr874 yup till 10 th August


Add Reactive Extensions & ReactiveUI Sample App

This PR adds a new project that shows how to use reactive extensions and ReactiveUI with gui.cs. After merging https://github.com/migueldeicaza/gui.cs/pull/903, we can now use the full power of Pharmacist and generate IObservable<TEventArgs> wrappers for every event in the Terminal.Gui assembly. This allows us to easily implement the MVVM (model view view-model) pattern and data bindings between views and view models. The new demo app looks like this:

Stumbled upon an issue related to reactive scheduling, will need some time to implement a custom scheduler.

Resolved the issues with scheduling by implementing a TerminalScheduler that can be used as the main thread scheduler in reactive Terminal.Gui apps, also added some getting started documentation. Regarding that unit tests — that reactive stuff won't break until we are marking our Actions with the event keyword. Assuming Pharmacist doesn't currently support <ProjectReference /> tags, we could wait until the new source generators-based Pharmacist version is out, and add the tests then. Thoughts?

that reactive stuff won't break until we are marking our Actions with the event keyword.

This is in now. I merged your PR on this.

we could wait until the new source generators-based Pharmacist version is out, and add the tests then. Thoughts?

I'm ok with that but am curious what the tests would test for.

@worldbeater it would be awesome if you tweeted about this!


Python3 error on repr(self)

Hello
Here's an error I get when running the library with Python 3.2.3
    return u'<TumblrAPI: %s>' % self.app_key
                            ^
SyntaxError: invalid syntax

I can solve it by removing the 'u' in front of '<TumblrAPI: %s>', but not sure if that's the way to go. Or why is it throwing an error in the first place.

This appears to be fixed in Python 3.5.2


Add statistics to PR preview

This PR updates the preview comment to also display layer size statistics.

Maybe add a comment in scripts/stats.js or in its output expressing in words that this is the length of the serialized style JSON.

Maybe add a comment in scripts/stats.js or in its output expressing in words that this is the length of the serialized style JSON.

Added in 14d4403

Sample output, from the github actions log:
Style size statistics

main
this PR
change

Layers
349
349
0

Size (b)
1005817
1005817
0

Layer count comparison

main
this PR
change

background
1
1
0

landuse
1
1
0

park
3
3
0

aeroway
7
7
0

landcover
2
2
0

boundary
9
9
0

water
3
3
0

waterway
3
3
0

transportation
299
299
0

building
1
1
0

transportation_name
3
3
0

poi
2
2
0

water_name
2
2
0

aerodrome_label
4
4
0

place
9
9
0

Layer size comparison

main
this PR
change

background
122
122
0

landuse
309
309
0

park
2365
2365
0

aeroway
2059
2059
0

landcover
361
361
0

boundary
13691
13691
0

water
912
912
0

waterway
3543
3543
0

transportation
920221
920221
0

building
292
292
0

transportation_name
8838
8838
0

poi
10094
10094
0

water_name
4647
4647
0

aerodrome_label
5951
5951
0

place
32062
32062
0

I manually copy/pasted the content from the output log of the github action, so that may or may not occur when the github action does the writing.

From some testing, it's happening at the browser/paste level so probably ignorable.

The random linking issue does not occur when writing directly to the Checks tab.

Never mind, I missed this in the Check tab.


Position of highlighted problem doesn't match linter output

I'm not entirely sure if this is a SublimeLinter problem, or the linter itself (pyflakes), but I'm seeing the wrong occurrence of a string highlighted as being the error. For example, this line of code:
print("sub2 force: %s" % force)

Looks like this:

but of course that makes no sense - there can't be a problem inside the string. The error is that the code should read:
print("sub2 force: %s" % self.force)

and if I chance the word inside the string, the second occurrence is correctly highlighted in that first version. And, the linter output in the status bar says Error: undefined name 'force', Line 41, Column 34 which is correct, even when the highlight is in the wrong place.
I've used Package Control to install everything, so I'm assuming everything is up to date - not sure how to get the version numbers out of the linter...

Are you using tabs or spaces? While I'm not confident this is the cause, this has been know to cause issues.

No - happens exactly the same with zero indentation. However, your comment did lead me to discover this pattern - certainly looks like something to do with word breaks, or valid variable name characters or something. I suspect it's a regex somewhere which needs a tweak when locating the offending string. However, the linter output explicitly gives the column position correctly, so not sure why a regex would be in use here...

I cannot reproduce this particular example with flake8, or the jshint issue but using eslint for that. flake8 issue 15 does also occur on my machine. @groteworld from the flake8 issue it seems you had a look into this. Did you find out if SL was at fault here or if the specific linters should be fixed?


Add REST API Status Checks

Summary
Add a basic REST API Status Checks.
Relevant Technical Choices
Added basic REST API status check.  For now it just returns true but in the future, better checks can be added to make sure that all communication are working correctly. Styling will follow in #4847
To-do
User-facing changes

Testing Instructions

Fixes #4790

I have hidden the feature and submit html to the endpoint. It still needs some more testing and fix some warnings. Is what we have now what you had in mind @swissspidy ?


Feature/issue 32 matrix exponential

Submisison Checklist

[ ] Run unit tests: ./runTests.py test/unit
-- ran the unit tests for the new feature I created (both high-level and internal functions)
[x] Run cpplint: make cpplint
[x] Declare copyright holder and open-source license: see below

Summary:
Add matrix_exp() functions: computes the matrix exponential of the input matrix. Uses method from Eigen/unsupported (Pade approximation + scaling and squaring), and a specialized algorithms for 2x2 matrices.
How to Verify:
Uses unit tests:
test/unit/math/prim/mat/fun/matrix_exp_test.cpp
test/unit/math/rev/mat/fun/matrix_exp_test.cpp
test/unit/math/fwd/mat/fun/matrix_exp_test.cpp
Internal functions that support matrix_exp() also have their own unit tests.
Side Effects:
None.
Documentation:
Doxygen.
Reviewer Suggestions:
Anybody familiar with matrix exponentials.
Copyright and Licensing
Metrum Research Group LLC
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

Note: I used and edited a file from Eigen/unsupported 3.3.0-beta2. The copyright information is written at the top of the file and is unchanged. Said file is stan/math/prim/mat/fun/MatrixExponential.h

@charlesm93, we're going to have trouble merging this pull request until we get Eigen 3.3 into the code base. If you can't tell, the diff is too big for GitHub to handle.
If you want eyes on this quicker, you can stage this as 2 separate pull requests: the first for updating the library to Eigen 3.3. That should have nothing but a change to the library in it. No other modifications. (We can't actually review that pull request per file.) The second pull request can branch from there and we can more easily see the diffs.
Something that Jenkins caught was a problem with this file:
stan/math/prim/scal/fun/ldexp.hpp
It should only include cmath and not stan/math/rev/core.hpp.
Some additional things for that file:

the first argument should be const T& a, the second argument is fine
the indentation levels are wrong for the function

I haven't looked at anything else. I just saw that Jenkins failed and I was hunting down why.

Yes, please don't roll Eigen library commits in with this branch.
Do you need Eigen 3.3 itself or just its matrix exponential function?  That is, if you take the matrix exponential function from Eigen 3.3, can you compile it with Eigen 3.2.9?   If so, you can just take the matrix exponential function file and modify it and use it to define a regular Stan function.  If you do that, include the Eigen license in the file for the modified function to make its provenance and licensing clear.

Hi Daniel, hi Bob,
I'll take a closer look tomorrow at the office, but a quick point:

No, I do NOT need Eigen 3.3. If I added it, it was by mistake. I
recreated the matrix exponential function in the stan::math namespace, and
it should be fully functional with eigen 3.2.9.

I'll edit the branch  first thing tomorrow.
~ Charles
On Sun, Sep 11, 2016 at 10:49 AM, Bob Carpenter<EMAIL_ADDRESS>wrote:

Yes, please don't roll Eigen library commits in with this branch.
Do you need Eigen 3.3 itself or just its matrix exponential function? That
is, if you take the matrix exponential function from Eigen 3.3, can you
compile it with Eigen 3.2.9? If so, you can just take the matrix
exponential function file and modify it and use it to define a regular Stan
function. If you do that, include the Eigen license in the file for the
modified function to make its provenance and licensing clear.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-246184128, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANUliiDZAFk-VPMFFz-OPEvGtykvPXUKks5qpBTogaJpZM4J5VLn
.

I updated the branch by:

removing eigen 3.3
correcting ldexp.hpp
(re-running the unit tests for the matrix_exp with double, rev, and fwd)

I hope this makes the pull request more manageable!
On Sun, Sep 11, 2016 at 5:08 PM, Charles Margossian<EMAIL_ADDRESS>wrote:

Hi Daniel, hi Bob,
I'll take a closer look tomorrow at the office, but a quick point:

No, I do NOT need Eigen 3.3. If I added it, it was by mistake. I
recreated the matrix exponential function in the stan::math namespace, and
it should be fully functional with eigen 3.2.9.

I'll edit the branch  first thing tomorrow.
~ Charles
On Sun, Sep 11, 2016 at 10:49 AM, Bob Carpenter<EMAIL_ADDRESS>wrote:

Yes, please don't roll Eigen library commits in with this branch.
Do you need Eigen 3.3 itself or just its matrix exponential function?
That is, if you take the matrix exponential function from Eigen 3.3, can
you compile it with Eigen 3.2.9? If so, you can just take the matrix
exponential function file and modify it and use it to define a regular Stan
function. If you do that, include the Eigen license in the file for the
modified function to make its provenance and licensing clear.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-246184128, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANUliiDZAFk-VPMFFz-OPEvGtykvPXUKks5qpBTogaJpZM4J5VLn
.

This code looks OK to me, but I'd like someone who knows matrix exponentials to have a look at test coverage.

Hi Bob,
Still working on your feedback. A few quick comments:

No edits to make/libraries and makefile are required, I removed all
modifications. Tbh, I'm not even sure why I changed them in the first
place.
I didn't include a 1x1 specialization, but this could easily be added
later
In frexp_test, the integer b stores the integral power of two computed by
frexp, and therefore only needs to be declared.
I will work out 3x3 and higher examples -- I just need examples for which
I can calculate the derivatives exactly.

On Mon, Sep 12, 2016 at 6:36 PM, Bob Carpenter<EMAIL_ADDRESS>wrote:

This code looks OK to me, but I'd like someone who knows matrix
exponentials to have a look at test coverage.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-246517487, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANUlislKNX_kryN8LrgLGlbZWlPxlKxrks5qpdQAgaJpZM4J5VLn
.

Makes sense on the integer return for frexp.
You can also use finite differences for testing.  They're accurate enough for that in most cases. There are built-in finite diff derivative calculators in the math library that can work beyond first-order derivatives.  @rtrangucci has the most experience with those.

I built a 3x3 test case using the same method I used for the 2x2 case (see
Moler and Van Loan, 2003, section 6).
For the reverse mode autodiff, I got an error testing gradients, when the
expected value was 0. Autodiff computed values such as -1.0913936e-11, and
EXPECT_FLOAT_EQ flagged this as a failure. I see this as a very minor
numerical error.
I switched to EXPECT_NEAR and used a "precision" of 1e-10. This took care
of the 0 gradients. Other gradients, where the value was, say 8.829017,
were however off by more than 1e-10 and failed, even though they had passed
the EXPECT_FLOAT_EQ test. All entries of the matrix passed the test when
the precision was 1e-9. Right now, I'm using EXPECT_NEAR(... ,  ..., 1e-9).
Should matrix_exp fail the test? If not, should the used "precision" for
the unit test be reported or documented somewhere?
On Mon, Sep 12, 2016 at 6:48 PM, Bob Carpenter<EMAIL_ADDRESS>wrote:

Makes sense on the integer return for frexp.
You can also use finite differences for testing. They're accurate enough
for that in most cases. There are built-in finite diff derivative
calculators in the math library that can work beyond first-order
derivatives. @rtrangucci https://github.com/rtrangucci has the most
experience with those.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-246520112, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANUliswcaBAw0ytb194FNMR8KAeATFpFks5qpda-gaJpZM4J5VLn
.

On Sep 14, 2016, at 11:54 AM, Charles Margossian<EMAIL_ADDRESS>wrote:

I built a 3x3 test case using the same method I used for the 2x2 case (see
Moler and Van Loan, 2003, section 6).
For the reverse mode autodiff, I got an error testing gradients, when the
expected value was 0. Autodiff computed values such as -1.0913936e-11, and
EXPECT_FLOAT_EQ flagged this as a failure. I see this as a very minor
numerical error.
I switched to EXPECT_NEAR and used a "precision" of 1e-10. This took care
of the 0 gradients.

That's fine.

Other gradients, where the value was, say 8.829017,
were however off by more than 1e-10 and failed, even though they had passed
the EXPECT_FLOAT_EQ test.

You usually want to test relative accuracy, not absolute accuracy.
I don't know if that'd make a difference.
EXPECT_FLOAT_EQ tests relative accuracy, not absolute accuracy.

All entries of the matrix passed the test when
the precision was 1e-9. Right now, I'm using EXPECT_NEAR(... , ..., 1e-9).
Should matrix_exp fail the test? If not, should the used "precision" for
the unit test be reported or documented somewhere?

You can doc it if you can characterize the accuracy,
but it's not really necessary.  And just testing a few inputs
won't give you a thorough look at precision.
Many of our tests use finite differences and they have very
limited precision.

EXPECT_FLOAT_EQ tests relative accuracy, not absolute accuracy.

That would make it difficult to test an expected value of 0, especially if
said value is obtained through a numerical approximation.
EXPECT_FLOAT_EQ(0, 1e-25) causes a failure.
I'll rewrite the test to use EXPECT_FLOAT_EQ for non-zero values and
EXPECT_NEAR(..., ..., e-10) for zero values.
This strikes me as a serious defect in google Test though, I might raise an
issue.
On Thu, Sep 15, 2016 at 4:42 PM, Bob Carpenter<EMAIL_ADDRESS>wrote:

On Sep 14, 2016, at 11:54 AM, Charles Margossian<EMAIL_ADDRESS>wrote:

I built a 3x3 test case using the same method I used for the 2x2 case
(see
Moler and Van Loan, 2003, section 6).
For the reverse mode autodiff, I got an error testing gradients, when the
expected value was 0. Autodiff computed values such as -1.0913936e-11,
and
EXPECT_FLOAT_EQ flagged this as a failure. I see this as a very minor
numerical error.
I switched to EXPECT_NEAR and used a "precision" of 1e-10. This took care
of the 0 gradients.

That's fine.

Other gradients, where the value was, say 8.829017,
were however off by more than 1e-10 and failed, even though they had
passed
the EXPECT_FLOAT_EQ test.

You usually want to test relative accuracy, not absolute accuracy.
I don't know if that'd make a difference.
EXPECT_FLOAT_EQ tests relative accuracy, not absolute accuracy.

All entries of the matrix passed the test when
the precision was 1e-9. Right now, I'm using EXPECT_NEAR(... , ...,
1e-9).
Should matrix_exp fail the test? If not, should the used "precision" for
the unit test be reported or documented somewhere?

You can doc it if you can characterize the accuracy,
but it's not really necessary. And just testing a few inputs
won't give you a thorough look at precision.
Many of our tests use finite differences and they have very
limited precision.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-247449702, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANUliur3DtvU2YeFOiIu8JNpMBZLCX6Mks5qqa3MgaJpZM4J5VLn
.

I think a newline got removed from the end of the files.

On Sep 19, 2016, at 10:29 AM, Charles Margossian<EMAIL_ADDRESS>wrote:
@charlesm93 commented on this pull request.
In makefile:

@@ -168,4 +168,4 @@ clean-deps:
clean-all: clean clean-doxygen clean-deps clean-libraries
@echo '  removing generated test files'
$(shell find test/prob -name 'generated_test.cpp' -type f -exec rm {} +)

$(RM) $(wildcard test/gtest.o test/libgtest* test/prob/generate_tests$(EXE))

$(RM) $(wildcard test/gtest.o test/libgtest* test/prob/generate_tests$(EXE))

Not sure what's happening here. It looks as though the exact same line code was deleted and added. I'm guessing GitHub is comparing revisions, as oppose to final code and current dev version.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

I don't know how to have lint not check a file.  You can
add
// NOLINT
to the end of lines to get it to not check those.

Bob

On Sep 19, 2016, at 10:34 AM, Charles Margossian<EMAIL_ADDRESS>wrote:
@charlesm93 commented on this pull request.
In stan/math/prim/mat.hpp:

@@ -107,6 +107,8 @@
#include <stan/math/prim/mat/fun/log_softmax.hpp>
#include <stan/math/prim/mat/fun/log_sum_exp.hpp>
#include <stan/math/prim/mat/fun/make_nu.hpp>
+#include <stan/math/prim/mat/fun/matrix_exp.hpp>
+#include <stan/math/prim/mat/fun/MatrixExponential.h>

Ok. Once I do this, cpplint complains about the format of the Eigen/unsupported code. I guess cpplint did not check the .h version, but it's certainly going through the .hpp file. Is there a way of telling cpplint to skip this file? What would be the pros and cons of doing so?
pro: I don't need to reformat 200 lines of code from Eigen/unsupported
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

I think we need some more tests for the case where the dimension is not small. You might copy some things (not involving complex numbers) from Eigen's tests:
https://bitbucket.org/eigen/eigen/raw/f6382682565c946d46612fe0e36e486bba1371ce/unsupported/test/matrix_exponential.cpp

Just make sure you attribute where they came from if you
use the code.  If you just use the numbers, it's no big
deal, but I'd still suggest listing the provenance of the test.

Bob

On Sep 23, 2016, at 11:36 AM, bgoodri<EMAIL_ADDRESS>wrote:
I think we need some more tests for the case where the dimension is not small. You might copy some things (not involving complex numbers) from Eigen's tests:
https://bitbucket.org/eigen/eigen/raw/f6382682565c946d46612fe0e36e486bba1371ce/unsupported/test/matrix_exponential.cpp
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

@bgoodri I looked through their test: I couldn't reproduce the results of the pascal test in C++ nor in R -- I think there may be an error in their code. The inverse test looks promising. I'll add it.
The test I went for randomly generates a matrix of the form A = S * B * Inverse(S), where B is a diagonal matrix. One can then compute exp(A) = S * exp(diag(B)) * Inverse(S). The code lets you pick the dimension of the matrix, currently 100 x 100.
Matrix_exp passes the test with a few notable exceptions:

when the expected value is 0, and the pade approximation wanders around e-10, 11, ...
when the expected value is <e-10, and the relative error becomes high because of numerical errors

I don't think these should be flagged as failures, so I tweaked the unit test to allow a higher relative error for these special cases. We should document these "exceptions" for users.
The decomposition technique should make it easy to implement tests for fwd and rev (and if not, I'll use finite diff). I wanted to share this approach, in case someone had a major objection (see test/unit/.../mat/.../matrix_exp_test.cpp for implementation).

On Sep 26, 2016, at 2:27 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
@bgoodri I looked through their test: I couldn't reproduce the results of the pascal test in C++ nor in R -- I think there may be an error in their code. The inverse test looks promising. I'll add it.
The test I went for randomly generates a matrix of the form A = S * B * Inverse(S), where B is a diagonal matrix. One can then compute exp(A) = S * exp(diag(B)) * Inverse(S).

Don't ever use inverse.  Use S * exp(diag(B)) / S.  You can
use Stan's mdivide_right function for the division.  Don't know
if it'll be more stable to do the multiplication or division first.

The code lets you pick the dimension of the matrix, currently 100 x 100.
Matrix_exp passes the test with a few notable exceptions:
• when the expected value is 0, and the pade approximation wanders around e-10, 11, ...

I think that's just a matter of setting the test up to provide
relative error measurements abs(a - b) / ((abs(a) + abs(b)) / 2) < epsilon
for some epsilon.

• when the expected value is <e-10, and the relative error becomes high because of numerical errors

Any idea why the relative error gets high when the scale is low?
All the relative error should scale with size.

I don't think these should be flagged as failures, so I tweaked the unit test to allow a higher relative error for these special cases. We should document these "exceptions" for users.
The decomposition technique should make it easy to implement tests for fwd and rev (and if not, I'll use finite diff). I wanted to share this approach, in case someone had a major objection (see test/unit/.../mat/.../matrix_exp_test.cpp for implementation).

Yes, at least finite diffs.

Bob

Don't ever use inverse.

What's wrong with inverse?
Any idea why the relative error gets high when the scale is low?
I don't think the scale is low, rather some entries are small relative to
other entries. If I look at exp(A), most entries have an order of magnitude
around 0 or 1, with some exceptions at e-12, e-13, etc.
Yes, at least finite diffs.
Do you mean you have an objection with using finite diffs, or that the
gradient tests should at least use finite diffs?
On Mon, Sep 26, 2016 at 2:34 PM, Bob Carpenter<EMAIL_ADDRESS>wrote:

On Sep 26, 2016, at 2:27 PM, Charles Margossian <
<EMAIL_ADDRESS>wrote:
@bgoodri I looked through their test: I couldn't reproduce the results
of the pascal test in C++ nor in R -- I think there may be an error in
their code. The inverse test looks promising. I'll add it.
The test I went for randomly generates a matrix of the form A = S * B *
Inverse(S), where B is a diagonal matrix. One can then compute exp(A) = S *
exp(diag(B)) * Inverse(S).

Don't ever use inverse. Use S * exp(diag(B)) / S. You can
use Stan's mdivide_right function for the division. Don't know
if it'll be more stable to do the multiplication or division first.

The code lets you pick the dimension of the matrix, currently 100 x 100.
Matrix_exp passes the test with a few notable exceptions:
• when the expected value is 0, and the pade approximation wanders
around e-10, 11, ...

I think that's just a matter of setting the test up to provide
relative error measurements abs(a - b) / ((abs(a) + abs(b)) / 2) < epsilon
for some epsilon.

• when the expected value is <e-10, and the relative error becomes high
because of numerical errors

Any idea why the relative error gets high when the scale is low?
All the relative error should scale with size.

I don't think these should be flagged as failures, so I tweaked the unit
test to allow a higher relative error for these special cases. We should
document these "exceptions" for users.
The decomposition technique should make it easy to implement tests for
fwd and rev (and if not, I'll use finite diff). I wanted to share this
approach, in case someone had a major objection (see
test/unit/.../mat/.../matrix_exp_test.cpp for implementation).

Yes, at least finite diffs.

Bob

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-249657118, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANUlig6yI_x0iYFDVDpp8zx28ROmjwi1ks5quBAzgaJpZM4J5VLn
.

I added unit tests for fwd and rev mode.
The code is non-trivial, but the matrix_exp and its derivatives are compared to an exact value. Should I add descriptive comments or a doc? Everything is based on using matrices of the form A = S * B * Inverse(S).
For fwd and rev test, I set the size of the input matrix to 10x10, to make the code run faster (particularly for rev). I could scale it up, if we want to test higher dimensions.

On Sep 26, 2016, at 2:56 PM, Charles Margossian<EMAIL_ADDRESS>wrote:

Don't ever use inverse.

What's wrong with inverse?

It's unstable numerically compared to matrix division.
And usually you are multiplying the inverse of a matrix by
a vector or matrix, so division is a viable alternative.

Any idea why the relative error gets high when the scale is low?

I don't know how they measure relative error or what the algorithm
is.  I'm also not sure what the right thing to do is.

I don't think the scale is low, rather some entries are small relative to
other entries. If I look at exp(A), most entries have an order of magnitude
around 0 or 1, with some exceptions at e-12, e-13, etc.
Yes, at least finite diffs.
Do you mean you have an objection with using finite diffs, or that the
gradient tests should at least use finite diffs?

No, I mean that you should use finite diffs to test if you
don't have an independent answer.

Bob

How slow is it to test larger matrices?
I also don't know if that's a general enough test.  You
mean that the matrices you test are generated as S * B / S
for some S and B?

Bob

On Sep 27, 2016, at 1:45 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
I added unit tests for fwd and rev mode.
The code is non-trivial, but the matrix_exp and its derivatives are compared to an exact value. Should I add descriptive comments or a doc? Everything is based on using matrices of the form A = S * B * Inverse(S).
For fwd and rev test, I set the size of the input matrix to 10x10, to make the code run faster (particularly for rev). I could scale it up, if we want to test higher dimensions.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

@bob-carpenter : I don't have a solid theory for what happens when the scale goes down, but I'm working on it. For now, here's a simple example where the error occurs, using 2x2 matrices:
A = (-1.97e-13, 9.9e-14, 5.94e-13, 2.98e-13)
Exact Solution =  (1.0, 9.88098e-14, -5.92415e-13, 1.0)
matrix_exp(A) = (1.0, 9.9e-14, -5.94e-13, 1.0)
The relative error for the small entries is very high. Pasted below is the C++ code:
    Matrix<double, Dynamic, Dynamic> m(2, 2), exp_m(2, 2), D(2, 2),
      D_inv(2, 2), A(2, 2), exp_A(2, 2);
    m << 1e-13, 0, 0, 1e-15;
    D << 1, 2, 3, 4;
    D_inv << -2, 1, 1.5, -0.5;
    exp_m << exp(1e-13), 0, 0, exp(1e-15);
    A << D * m * D_inv;

    std::cout << A << std::endl << std::endl;
    std::cout << D * exp_m * D_inv << std::endl << std::endl;
    std::cout << stan::math::matrix_exp(A) << std::endl << std::endl;

How slow is it to test larger matrices?

The test for larger matrixes is slow for the rev case:

1629 ms for 10x10
no response from terminal window after 10 minutes

For fwd:

1 ms for 10x10
72 ms for 100x100

I also don't know if that's a general enough test.  You
mean that the matrices you test are generated as S * B / S
for some S and B?

Yes. I randomly generate S, making sure that it is invertible. Then I randomly generate the elements of the diagonal matrix B.
This is based on Moler & Van Loan, 2003. They use this strategy to construct a 2x2 matrix and show how a Taylor series fails to approximate the matrix exponential. Since Padé is also a series method, I expect it to be a robust test. Padé, unlike decomposition methods, does not take advantage of the fact A = S * B / S, so the test should be general enough.

Even thought the inputs are are all on the order of 1e-14, the
outputs are order 1 and order 1e-14, which is probably where
the issue's arising.  So maybe relative error should be defined
relative to the entire set of matrix entries or maybe the diagonal
entries, in which case this is still low relative error.

On Sep 28, 2016, at 2:44 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
...
A = (-1.97e-13, 9.9e-14, 5.94e-13, 2.98e-13)
Exact Solution = (1.0, 9.88098e-14, -5.92415e-13, 1.0)
matrix_exp(A) = (1.0, 9.9e-14, -5.94e-13, 1.0)

On Sep 28, 2016, at 3:01 PM, Charles Margossian<EMAIL_ADDRESS>wrote:

...

The test for larger matrixes is slow for the rev case:
• 1629 ms for 10x10
For fwd:
• 1 ms for 10x10
• 72 ms for 100x100
...

I assume that 72ms for fwd is not testing all the derivatives the
way the rev case is.  But yes, 1.6s is slow enough that we don't
want to go a lot higher than that in our usual test runs.  You could
stretch it out to 30s, but try not to exceed that for a test for
a single function.

Bob

Here's how the relative error is now evaluated:

The maximum absolute coefficient of the matrices we are comparing times 1e-10
If we know the matrices have entries with roughly the same order of magnitude, use expect_matrix_eq (i.e. EXPECT_FLOAT_EQ).

The same principle applies to derivatives.

I assume that 72ms for fwd is not testing all the derivatives the
way the rev case is.

The input matrix A is a function of a set of parameters. In the fwd case, these parameters have d_ = 1, and essentially depend on an independent variable that is not explicitly declared. I'm testing for the derivative of the matrix_exponential with respect to that variable.
In the rev case, every parameter is an independent variable. In addition -- and maybe I got confused on how the grad() function works -- the matrices get constructed for every entry of the output matrix (see the code for the 2x2 case below). My understanding is that the expression graph needs to be reinitialized every time we call grad(). Confirmation or clarification welcomed.
But that explains why the rev case is computationally more intensive than the fwd case.
    for (size_t k = 0; k < 2; k++) {
    	for (size_t l = 0; l < 2; l++) {

    		matrix_v m1(2,2), m2(2,2), m1_exp;

    		AVAR a = -1.0, b = -17.0;

    		m1 << -2*a + 3*b, 1.5*a - 1.5*b, -4*a + 4*b, 3*a - 2*b;
   			m2 << -.735759, .551819, -1.471518, 1.103638;
    		m1_exp = stan::math::matrix_exp_pade(m1);
    		expect_matrix_eq(m2, m1_exp);

			matrix_v dm1_exp_da(2,2), dm1_exp_db(2,2);
    		dm1_exp_da << -2*exp(a), 1.5*exp(a), -4*exp(a), 3*exp(a);
    		dm1_exp_db << 3*exp(b), -1.5*exp(b), 4*exp(b), -2*exp(b);

    		AVEC x = createAVEC(a, b);
    		VEC g;
    		m1_exp(k, l).grad(x, g);
    		EXPECT_FLOAT_EQ(dm1_exp_da(k, l).val(), g[0]);
    		EXPECT_FLOAT_EQ(dm1_exp_db(k, l).val(), g[1]);
    	}
    }

On Sep 30, 2016, at 3:44 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
I assume that 72ms for fwd is not testing all the derivatives the
way the rev case is.
The input matrix A is a function of a set of parameters. In the fwd case, these parameters have d_ = 1, and essentially depend on an independent variable that is not explicitly declared. I'm testing for the derivative of the matrix_exponential with respect to that variable.
In the rev case, every parameter is an independent variable. In addition -- and maybe I got confused on how the grad() function works -- the matrices get constructed for every entry of the output matrix (see the code for the 2x2 case below). My understanding is that the expression graph needs to be reinitialized every time we call grad(). Confirmation or clarification welcomed.

var.grad() works on a precomputed expression graph, so it's not
rebuilt.
You can see the code for Jacobians in the math library to
see how this works:
stan/math/rev/mat/functor/jacobian.hpp
I'd suggest studying the arXiv paper to see how our autodiff
works and then asking us questions based on that (it'll help
sharpen the paper, too).

Bob

@bob-carpenter : Thanks for pointing me to jacobian.hpp and the autodiff paper. This allowed me to optimize the code, and test a 25x25 matrix in the same runtime.
In the previous version, instead of only resetting the adjoints, I redeclared all var (and matrix_v) variables, which was of course less efficient because this meant computing the values again.
I think the paper explains autodiff very clearly. I couldn't use the Jacobian function directly, since matrix_exp doesn't map to R^n (rather to R^n x R^n), but I followed its architecture closely.
I haven't said it in a while, but thank you for all the guidance.

When the formatting's done, let me know and I can take a final code review
pass over everything.  Great that this is going in and sorry it's been
such a struggle---you took on some complicated examples to kick off!

Bob

On Oct 13, 2016, at 9:39 AM, Charles Margossian<EMAIL_ADDRESS>wrote:
@charlesm93 commented on this pull request.
In stan/math/prim/mat/fun/matrix_exp_pade.hpp:

+// Copyright (C) 2009, 2010, 2013 Jitse Niesen<EMAIL_ADDRESS>+// Copyright (C) 2011, 2013 Chen-Pang He<EMAIL_ADDRESS>+//
+// This Source Code Form is subject to the terms of the Mozilla
+// Public License v. 2.0. If a copy of the MPL was not distributed
+// with this file, You can obtain one at http://mozilla.org/MPL/2.0/.
+//
+// This file was edited for to the Stan math library to create
+// the matrix exponential function (matrix_exp), 2016.

+#ifndef STAN_MATH_PRIM_MAT_FUN_MATRIX_EXP_PADE_HPP
+#define STAN_MATH_PRIM_MAT_FUN_MATRIX_EXP_PADE_HPP
+
+#include <stan/math/prim/mat/fun/Eigen.hpp>
+
+namespace stan {
It is from the beta version of Eigen 3.3 in unsupported (so not the currently implemented Eigen version in Stan), and has been modified to be auto-differentialble. Their functions were overloaded for float, double, and long double, but not for var and fvar. I created a template version.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Yes, Eigen::NumTraits is a class.   You can look these things
up in the Eigen doc.  It's pretty well indexed.

In that case, it should be ok to have using Eigen::NumTraits in the code, since it isn't a namespace level using statement (unless I am missing something, of course).

That is a using statement declared at namespace scope.  I answered the question without looking the the use the first time.  We don't want using statements at the namespace level, no matter what they are.

Ok - now I get what Daniel meant.

We'll be doing a release soon (probably in around 2 or 3 weeks time), so if you want this in 2.13.0, it should get wrapped up sooner rather than later with the accompanying stan-dev/stan issue to expose the function to the language.  I'm happy to help with doc.
No rush on our end if you'd rather wait.

The branch to expose matrix_exp() to Stan is Feature/issue-2043-matrix-exponential. I need to make sure all the tests are completed. The doc is for the manual?

Regarding documentation:
In function.tex, I am not sure where to put matrix_exp. The two options I see are \it{special matrix functions} (which seems ambiguous) and \it{linear algebra functions}.
I'm also debating adding something to programming.tex, under Solving Differential Equations. Using Matrix Exp is much more efficient than numerical solvers for linear ODE's.

@charlesm93, a few more comments:

This is cosmetic, but it matters: no tabs in code. The only place tabs are appropriate is in the makefiles.
This is substantial: please move the imported Eigen file into its own header file. Preferably called MatrixExponential.h like where it originally came from. I think it's ok to put all the functions in Eigen there. I'd actually prefer to leave it in the Eigen namespace. The matrix_exp_pade.hpp file should only include the last function: matrix_exp_pade.
What's with the C macro, LDBL_MANT_DIG?
Please remove the //  end namespace math comments.

I'm also going to suggest we don't reimplement frexp for all the instantiations. I don't see how this function is going to be practically useful anywhere. If you're modifying the code, you can modify the call to the frexp() function to use doubles only. I'll add an inline comment.

I was the one who suggested moving the Eigen function to an .hpp file and keeping it inline with our file naming conventions, but it won't bug me too much if it's the only .h file other than the signatures file.

@bob-carpenter : MatrixExponential.h is odd. On the one hand, it makes sense to include it in matrix_exp_pade.hpp, because MatrixExponential.h contains the code required to compute a pade approximation. On the other hand, we decided to minimally edit Eigen's original file. Keeping the file's original name and type is consistent with that. As a .h file, it is also not subject to cpplint's check.
I agree that one way or the other works fine. When we add Eigen 3 some of the functions in the structures of Unsupported/Eigen will become public, meaning I won't need to redefine these structures in MatrixFunction.h in order to construct a template version of computeUV(). I might move computeUV() to matrix_exp_pade.cpp then, and get rid of MatrixFunction.h, but that's hypothetical.

@syclik :

sorry about the tabs, I thought I had gotten rid of all them
The C macro, as far as I understand, is for floating points with higher than quadruple precision. It triggers matrix functions of Eigen/unsupported (matrix_exp among others) to call so-called StdStemFunctions. Setting LDBL_MANT_DIG > 2, I realized the statements in the macro of statement do not compile. They rely on features from Eigen 3.3-beta. So I need to adjust them. But before I do, I wanted to ask: is it ever the case that LDBL_MANT_DIG > 112 ? On my local machine LDBL_MANT_DIG = 64. This conditional only applies to double and long doubles, but I guess the precision would propagate to rev and fwd variables.

I'm also OK either way, so let's go with what Daniel wants, which
is to keep the original name.  I think if it's a .h it won't
get cpplinted anyway, so maybe we don't need to modify it at all?

On Oct 23, 2016, at 4:30 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
@bob-carpenter : MatrixExponential.h is odd. On the one hand, it makes sense to include it in matrix_exp_pade.hpp, because MatrixExponential.h contains the code required to compute a pade approximation. On the other hand, we decided to minimally edit Eigen's original file. Keeping the file's original name and type is consistent with that. As a .h file, it is also not subject to cpplint's check.
I agree that one way or the other works fine. When we add Eigen 3 some of the functions in the structures of Unsupported/Eigen will become public, meaning I won't need to redefine these structures in MatrixFunction.h in order to construct a template version of computeUV(). I might move computeUV() to matrix_exp_pade.cpp then, and get rid of MatrixFunction.h, but that's hypothetical.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Doc will be for the manual on the stan-dev/stan repo, not here.  This just needs the API doc for doxygen.

Sorry, I realize that I wasn't clear. I don't care that the file is renamed. What I want is the Eigen file to be pulled into the code base with minimal changes and have a different file for our code. I don't want one file with both things in it. I think the plan is to remove the file from our code base once we have Eigen 3.3. I would like that step to be easy and less error prone.

On Oct 25, 2016, at 2:11 PM, Bob Carpenter<EMAIL_ADDRESS>wrote:
I'm also OK either way, so let's go with what Daniel wants, which
is to keep the original name. I think if it's a .h it won't
get cpplinted anyway, so maybe we don't need to modify it at all?

On Oct 23, 2016, at 4:30 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
@bob-carpenter : MatrixExponential.h is odd. On the one hand, it makes sense to include it in matrix_exp_pade.hpp, because MatrixExponential.h contains the code required to compute a pade approximation. On the other hand, we decided to minimally edit Eigen's original file. Keeping the file's original name and type is consistent with that. As a .h file, it is also not subject to cpplint's check.
I agree that one way or the other works fine. When we add Eigen 3 some of the functions in the structures of Unsupported/Eigen will become public, meaning I won't need to redefine these structures in MatrixFunction.h in order to construct a template version of computeUV(). I might move computeUV() to matrix_exp_pade.cpp then, and get rid of MatrixFunction.h, but that's hypothetical.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Either way is fine.  This will be on the stan-dev/stan issue anyway.  Sorry for the confusion.

I would put it near the eigenvalues section but presumably in its own
section. And reference it from the differential equations section.
On Fri, Oct 21, 2016 at 10:01 AM, Charles Margossian <
<EMAIL_ADDRESS>wrote:

Regarding documentation:
In function.tex, I am not sure where to put matrix_exp. The two options I
see are \it{special matrix functions} (which seems ambiguous) and
\it{linear algebra functions}.
I'm also debating adding something to programming.tex, under Solving
Differential Equations. Using Matrix Exp is much more efficient than
numerical solvers for linear ODE's.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-255384935, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADOrqursx9W2LNb82WlpRiXiPo_REp7gks5q2MWtgaJpZM4J5VLn
.

Moving the conversation on the docs to https://github.com/stan-dev/stan/pull/2092.

I sent an e-mail to the Eigen developers about the macro. I've looked at result = arg.matrixFunction(stem_function_exp); and I don't see what stem_function_exp does to address the higher precision case (or how it computes a matrix exponential for that matter).
The function is defined in StemFunction.h:
template <typename Scalar>
Scalar stem_function_exp(Scalar x, int)
{
  using std::exp;
  return exp(x);
}

That code will use argument dependent lookup (ADL --- you
can find discussions online as to how it works) to find
an implementation for the Scalar type.  So if you
couple this with an implementation of exp, for
instance as in Stan's for autodiff, then this version
will be usable whereas a direct call to std::exp(x) wouldn't.

Bob

On Oct 31, 2016, at 10:14 AM, Charles Margossian<EMAIL_ADDRESS>wrote:
I sent an e-mail to the Eigen developers about the macro. I've looked at result = arg.matrixFunction(stem_function_exp); and I don't see what stem_function_exp does to address the higher precision case (or how it computes a matrix exponential for that matter).
The function is defined in StemFunction.h:
template
Scalar stem_function_exp(Scalar x, int)
{
using std::exp;
return exp(x);
}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

When LDBL_MANT_DIG > 112, the matrix function uses a different method to compute the matrix exponential. In Gael Guennebaud from Eigen's words:

In the general case it basically computes the eigenvalue/vector decomposition, apply the scalar-wise exp function (stem_function_exp) to eigenvalues, and then recompose the matrix. In practice, this is performed using a Schur decomposition instead of a full eigenvalue/vector decomposition.

When LDBL_MANT_DIG <= 112 on the other hand, the function uses a Pade approximation.
For this pull request, I propose to only use Pade's algorithm, and write a cautionary note in the docs for users running on LDBL_MANT_DIG > 112. I will also submit issues on (1) implementing Schur decomposition and (2) adding the Schur decomposition algorithm to the matrix_exp function.
Honestly, I don't think getting the LDBL_MANT_DIG > 112 case is a priority. Under the current algorithm, all unit tests pass with LDBL_MANT_DIG = 128.

If you know the Pade algorithm is going to go off the rails, you can throw an exception at run time based on input size or some other measurable piece of the algorithm.
Once you have derivatives implemented, then you may have to split those, too.

I have the real Schur decomposition on a branch.
On Wed, Nov 2, 2016 at 3:17 PM, Bob Carpenter<EMAIL_ADDRESS>wrote:

If you know the Pade algorithm is going to go off the rails, you can throw
an exception at run time based on input size or some other measurable piece
of the algorithm.
Once you have derivatives implemented, then you may have to split those,
too.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/math/pull/390#issuecomment-257970369, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADOrqtck3Usbuy6aMNmSKLPVV0CacmMtks5q6OGygaJpZM4J5VLn
.

@bob-carpenter : I'm not 100% sure if there is a method to predict the algorithm will go off the rail. Size alone won't do it. There is a theoretical upper-bound for the relative error (Moler & Von Loan, section 3, method 3), which depends on ||A||, the degree of the Pade approximation j and the squaring q (the latter two are automatically picked by the matrix_exp function).
Computing the upper bound is not hard per se, but it might be computationally expensive. One has:
rel_err <= eps * ||A|| * exp(eps * ||A||)
where eps = 8 * ( ||A|| / 2^j )^(2q) * ( (q!)^2 / ( (2q)! * (2q + 1)! )
I can imagine requiring the user to specify rel_err, kind of like we do for ODE integrators. This is overkill for pharmacometrics problem, but I can't speak for other fields. I'm putting this on the agenda for the next Stan meeting.

If you know the Pade algorithm is going to go off the rails, you can throw an exception at run time based on input size or some other measurable piece of the algorithm.

Scaling and squaring takes care of errors due to large ||A|| or widely spread eigenvalues [1]. Squaring however introduces a round-off error. Numerical experiments show that the relative roundoff error can be estimated using the upper-bound: un||A||_1, where u is the computer relative precision, n the order of the matrix, and ||A||1 the 1-norm of A [2].
For a C++ double, u ~ e-16. Thus for a 100 x 100 matrix, where each entry is of the order e+2, the upper bound would be: eps ~ e-16 * e+2 * e+2 * e+2 = e-10, which is acceptable. Computing this empirical upper bound shouldn't be too expensive, because Eigen's code already computes ||A||_1 when determining the degree of the pade approximation. n is trivial, and u is a constant.
I can add the exception. I am reluctant to do so because it seems overkill; on the other hand, the cost should not be too high.
Is there a standard relative error implemented in Stan? Is this something the user should specify (in my opinion, no)?
[1] Roger B. Sidje, Expokit: A Software Package for Computing Matrix Exponentials, 1998
[2] M. Arioli, B. Codenotti, & C. Fassino, The Pade Method for Computing the Matrix Exponential from Linear Algebra and Its Applications, 1996

I kicked off the failing Travis job again.  If that passes, I think we're good to merge.

Sorry---had some more comments.

@sycklik are you OK with this now?  It looks good to me and I can merge when the unit tests pass.

I don't understand the error Jenkins throws in the default test: use of undeclared identifier 'std' in matrixExponential.h. The changes in the last commit couldn't cause this error, right?

@charlesm93: no idea, but this isn't a fluke of an issue. You can verify by running:
make test-headers

Or just the test that failed
make stan/math/prim/mat/fun/matrix_exp_pade.hpp-test

@syclik Yes, this confirms that the last commit causes the error. Putting back the rev and fwd include statements prevents the error. Really odd. Time for some digging.

Just looked at it. Can you try putting in the include for the prim/mat version of the value_of_rec function?

On Nov 14, 2016, at 3:22 PM, Charles Margossian<EMAIL_ADDRESS>wrote:
@syclik Yes, this confirms that the last commit causes the error. Putting back the rev and fwd include statements prevents the error. Really odd. Time for some digging.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

ok, I'm on it

A solution that works for the matrix_exp function is to include <Eigen/Dense> in prim/.../value_of_rec. The other option would be to include <stan/math/prim/mat/fun/Eigen.hpp>, which includes two more files from Eigen. Not needed, but consistent with what rev/.../value_of_rec does.


Regression test for nominate command username regex

Usernames with non \w characters were not correctly parsed. E.g. @bob.johnson.

Fixed in 61c7b62debd602bbd8b1fcf2db9e65815a8ad21c.


New package: MarketCycles v1.2.0

Registering package: MarketCycles
Repository: https://github.com/flare9x/MarketCycles.jl
Created by: @flare9x
Version: v1.2.0
Commit: c41d86a57b6d6bc409c860bbdd7539544f557e75
Git reference: master

Ok i believe I have fixed these errors above. These changes have been reflected in my current master.

Now just register again, and it will update this PR.
[noblock]

ok commented on the commit for registry here:
https://github.com/flare9x/MarketCycles.jl/commit/097199f1d11bc041327aeea3686491a83734998f#comments

Closing in favor of https://github.com/JuliaRegistries/General/pull/7246


Parsing error for valid json. Breaks gitea 1.16

Hi all,
in our gitea 1.16 server (using this library via go chi) we are encountering a parsing error that does not happen with the standard library.
There seems to be a problem if a nil / empty object is encountered in the input string around the position 511. You can see the issue in the code example below.
package main

import (
	"encoding/json"
	"fmt"
	"reflect"
	"strings"

	jsonLibrary "github.com/goccy/go-json"
)

func main() {

	const jsonString = `{"OldCommitIDs":["4bd3240099b6ee57a119885831259e314e46f0a5"],"NewCommitIDs":["3a48205c4ed8429ce309bb3d17bf08ce14c2b3dc"],"RefFullNames":["refs/heads/develop"],"UserID":27,"UserName":"","GitObjectDirectory":"/data/gitea/repositories/xxxxxxxxxx/import-service.git/./objects/incoming-MODbLf","GitAlternativeObjectDirectories":"/data/gitea/repositories/xxxxxxxxxx/import-service.git/./objects","GitQuarantinePath":"/data/gitea/repositories/xxxxxxxxxx/import-service.git/./objects/incoming-MODbLf","GitPushOptions":{},"PullRequestID":504,"IsDeployKey":false,"IsWiki":false}`

	type GitPushOptions map[string]string

	type HookOptions struct {
		OldCommitIDs                    []string
		NewCommitIDs                    []string
		RefFullNames                    []string
		UserID                          int64
		UserName                        string
		GitObjectDirectory              string
		GitAlternativeObjectDirectories string
		GitQuarantinePath               string
		GitPushOptions                  GitPushOptions
		PullRequestID                   int64
		IsDeployKey                     bool
		IsWiki                          bool
	}

	fmt.Println("Build in Json")
	jsonStruct := reflect.New(reflect.TypeOf(HookOptions{})).Interface()
	err := json.NewDecoder(strings.NewReader(jsonString)).Decode(jsonStruct)
	fmt.Printf("Error %#v\n", err)
	fmt.Printf("Object %#v\n\n", jsonStruct)

	fmt.Println("Json Library Go-Json")
	jsonStruct2 := reflect.New(reflect.TypeOf(HookOptions{})).Interface()
	err2 := jsonLibrary.NewDecoder(strings.NewReader(jsonString)).Decode(jsonStruct2)
	fmt.Printf("Error %#v\n", err2)
	fmt.Printf("Object %#v\n", jsonStruct2)
}

Expected is that both variants result in the same objects and no error, but the second call produces the following error:
msg:"invalid character '}' looking for beginning of value", Offset:511
Kind regards

Thank you, that was a fast one.


chartExtents mis-calculation

Benoit - here are the details regarding the chartExtents issue we chatted about yesterday. I will try to put together a small codepen example when I have some time. The workaround of calculating the chartExtents ourselves is working fine for now.
Expected behavior and actual behavior:
Our use case:

plotting multiple series, each one represents a distinct timespan
each series plots the same pair of tags, but with different timestamps (the 3rd dimension)
we set chartExtents to 'dynamic' for all dimensions
we set the timeDomain to the px-vis-nav object to the aggregate time period (includes all series timespans), this is getting correctly set in the chart component.

Expected: the chart would properly calculate the chart extents to include all the data passed
Actual: chart extents are calculated incorrectly, showing only a subset of the full dataset.
Steps to reproduce the problem:

create a couple of series with chart data that includes a different range of timestamps for each series. I'll attach some sample data for our fail case.
set properties on the component as described in the above use case
chart extents are not calculated correctly, resulting in a subset of the data shown.

xy-extent-bug.json.zip
Environment (component version number, Browser (including version), operating system, hardware, etc):
component: v1.1.1
browser: chrome
os: mac OSX El Capitan (v10.11.6)
hardware: Macbook Pro

Fixed; will be released with the next vis minor release


Exclude a directory but include a sub file

Migrated issue report by @sickboy : https://github.com/TypeStrong/atom-typescript/issues/651#issuecomment-146892270: :rose:

@blakeembrey Excluding works again thanks.
BUT I can not override excludes anymore.
<EMAIL_ADDRESS><EMAIL_ADDRESS>
should still give me "rx.all.es6.d.ts" but it doesn't. The order doesn't seem to matter.
Would be great if it could do first match perhaps.
so then it should be configured as:
<EMAIL_ADDRESS><EMAIL_ADDRESS>
and then because it hits the first one, it should no longer look at the next

Thanks, working on it - I'm going to release 2.0 of the API and switch to globby. Almost done now, give me an hour :+1: It'll also fix the compilerOptions issue, which is the main reason to break backward compat.

Released with v2.0.0 :+1:

Way to go, man!

:rose: :heart_eyes:

Thanks, this fixed the glob issue.


Support importing with a binary store hint

MODE-2707

Build failed with Invalid maximum heap size: -Xmx=1524m
Should be -Xmx1524m

@hchiorean Could you look at this?

fixed upstream


fix: added a transfer button on the right of send button

Fixes #235
Changes:

Removed Floating Button completely
Added a Transfer Button on the Right of the send button
Made BottomBar Invisible for now

Screenshots for the change:

@mariobehling Please Review

@yashk2000 Please Review

How does it look in the saved configurations section?

@yashk2000 Saved Configuration is hidden for now

@yashk2000 it will be handled in the upcoming pr when the whole app will be redesigned

@adityastic ok. I'd also like to help with the upcoming pr if needed :)


[QUESTION] Doesn't work with Angular in production mode

Description:
Angular 8.1 project
The polyfill doesn't work in production mode.
It works in every other environment (local, integration, test).
No error is shown.
In angular.json :
          "optimization": true,
          "outputHashing": "all",
          "sourceMap": false,
          "extractCss": true,
          "namedChunks": false,
          "aot": true,
          "extractLicenses": true,
          "vendorChunk": false,
          "buildOptimizer": true

What could it be ?
Additional Information:

webvr-polyfill version:	0.10.10
Browser name/version/release channel: chrome android 76.0.3809.89
Operating System: Android 9

My error, the polyfill declaration must be done necessarely inside polyfills.ts, my error was I was doing that in the component code.


Add support for npmproj.

In preparation for merging the Razor VSCode extension I've added support for npmproj files. I tried to maintain the same functionality that AspNetCore offers for its npmproj support.
Added an extra --ignore-engines check to yarn install in preparation for VSCode's extension model. They expect there to be a vscode engine in the package.json that yarn doesn't understand.

aspnet/AspNetCore#13494

Only that we pass an additional

Any important differences from what's in AspNetCore?

Only that we pass an additional flag to yarn install (--ignore-engines). This is needed for VSCode extension development.

Merging this through to get everything in. Will address feedback in a future PR.


sentry_file should use Sentry publisher

Description
The file integration isn't published via the Sentry verified publisher. That should probably be changed.

Indeed, its published technically with the same token, not sure what happened, will take a look.

One can't directly upload packages to a verified publisher. They have to be transferred and then it works. But I don't know why it is like that.

https://pub.dev/packages/sentry_file done.
it was accidentally published with a personal ACC instead of the sentry.io ACC.


Extension issue

Issue Type: Bug
Extension Name: LiveServer
Extension Version: 5.7.9
OS Version: Windows_NT x64 10.0.10240
VS Code version: 1.71.2

:warning: We have written the needed data into your clipboard. Please paste! :warning:
Error: connect ECONNREFUSED <IP_ADDRESS>:80
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1157:16)

Error: connect ECONNREFUSED <IP_ADDRESS>:80
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1157:16)


Run make influx fails without error

Steps to reproduce:

Download latest tagged release source code (2.0.4 at this moment).
Uncompress it.
Go to uncompressed directory an run make influx.
Wait for error.

Expected behavior:
After finish compilation have a bin/linux/influx binary.
Actual behavior:
Simply an error without more info
% make influx
env GO111MODULE=on go build -tags 'assets' -ldflags "-s -w  -X main.commit=9b8546d" -o bin/linux/influx ./cmd/influx
# github.com/influxdata/influxdb/v2/chronograf/canned
chronograf/canned/bin.go:20:11: undefined: AssetNames
chronograf/canned/bin.go:23:18: undefined: Asset
# github.com/influxdata/influxdb/v2/chronograf/dist
chronograf/dist/dist.go:39:13: undefined: AssetInfo
chronograf/dist/dist.go:59:18: undefined: Asset
chronograf/dist/dist.go:74:11: undefined: Asset
chronograf/dist/dist.go:83:14: undefined: AssetDir
chronograf/dist/dist.go:84:14: undefined: AssetInfo
make: *** [Makefile:97: bin/linux/influx] Error 2

Environment info:

System info: Linux 5.11.5-arch1-1 x86_64
Other relevant environment details: go version go1.16.1 linux/amd64 and cargo/rust 1.50.0

@ogarcia as a work-around, running make generate before make influx should fix the issue. You could also run make build.

@danxmoran Ok, but make generate builds a lot of things (like ui). I'm looking for the minimal and faster mode to build influx binary.

@ogarcia unfortunately the current layout of the influxdb repository makes it difficult to build only the CLI code. For example, the error you originally hit was caused because UI-related files hadn't been generated. We're working on a plan to fix this problem by decoupling the CLI and server codebases.

@danxmoran ok, no problem, I can wait for it :wink:

Yes I believe it will pick up after Easter is finished linda c

Onwards and upwards

this is in progress. check out https://github.com/influxdata/influx-cli

We've finished moving the CLI code over to the linked repository, closing this.


Help Pages and Documentation

[x] Create a VitePress app hosted on GitHub Pages

Since the Docs are in their own repo, I'm closing this And putting all docs related issues in that repository


[Proposal] new checker for detecting unused variables with Dagger annotations

We have built a checker for detecting unused variables that have dagger annotations. More specifically, when a field with @Inject annotation is declared and unused, unnecessary code is generated. Similarly, when a method with @Provides annotation contains a parameter that is unused in its body, that also results in unnecessary code. The tests provide the necessary example scenarios where this checker can be useful.
This is a proposal PR and we are happy to update the implementation based on the feedback. Let us know.
Thanks!

@ronshapiro any thoughts on this?

I think I mentioned that checker to @hzsweers and was wondering what the additional value of this one was. I think that should subsume this one, no?

Ah ok, seems like we can shutter this then


numbered equations and figures

In a fair amount of academic writing, it is helpful to have numbered equations and figures so that they can be referenced in the text of the document. Is it possible to have "labeled" equations and figures so that they can be referenced and hyperlinked throughout Jupyterbook? On a related note, can captions be added to figures?

good question! I believe we could pretty-easily get the MathJax configuration to number equations:
http://docs.mathjax.org/en/latest/tex.html#automatic-equation-numbering
That'd require us making it possible for people to specify their own mathjax configuration in the config.yml file. I can see that taking two forms:

We let people somehow override the default mathjax configuration entirely.
We expose specific parameters that people can set themselves (e.g. number_equations: true)

what do you think?
re: your second point, that'd require us adding some new behavior to the markdown generation process, I believe!

I am wondering if we can do something similar to the markdown extensions used by bookdown: https://bookdown.org/yihui/bookdown/markdown-extensions-by-bookdown.html#equations. Then all of the internal referencing is hooked up.
It looks like they have similar support for Figures with captions: https://bookdown.org/yihui/bookdown/figures.html

Hmmm - I think bookdown uses RMarkdown for its markup...we could use pandoc and achieve something similar (see #94 for example), but I'm not sure that we'd have the development bandwidth to implement our own custom markdown syntax. What do you think?
re: captions I think the same applies here. You could use pandoc to include things like captions etc. That said, what's the way that a person would express the caption contents etc when using a jupyter notebook. Cell metadata maybe?

Yeah, I agree that we don't want to develop and maintain our own language. Restructured text + sphinx has this capability (e.g. Page on Maxwell's equations). Do you think that supporting rst content has significant overhead?

I haven't yet tried it, but I can create a minimal example and give it a go :)

I am interested in utilizing equation numbering as well. Please let us know your findings as they become available, @lheagy.

Am I facing a related problem -
I just uploaded a jupyter-book, and the math is not rendering properly. Additionally, note the weird table appearing at the bottom of the page
https://dhruvbalwada.github.io/SFM_notes/files/inertial_ranges_Kolmogrov.html
, which is not part of my md files, as far as I can tell.

@dhruvbalwada hmm - that actually looks different, I bet there is some uncommon notation in there (not uncommon in mathematics, but probably uncommon in people using markdown with mathjax).
Could you try updating to the master branch and trying it again, by any chance? On master we're now converting the notebooks directly into HTML instead of converting them to Jekyll Markdown and then to HTML, which means that we should (theoretically) get the exact same experience as one would get using nbconvert.

@choldgraf - I am not sure I understand your suggestion to update to the master branch. I only have one branch, which is the master branch. So all changes are made directly to it. Should I delete the _sites folder before uploading? I have been using the docker method to see the changes locally before updating, which creates the _sites folder.
The repo is here - https://github.com/dhruvbalwada/SFM_notes
P.S. - I am not a pro githubber.

I found part of my solution here - https://github.com/atom-community/markdown-preview-plus/issues/336

that's great to hear :-)  Markdown + mathjax are notoriously finnicky especially for equation-heavy content like yours (which looks cool!)

Would be nice to add links to some tutorials on using math with markdown,
to the documentation of jupyter-book.
On Fri, Aug 2, 2019 at 9:38 PM Chris Holdgraf<EMAIL_ADDRESS>wrote:

that's great to hear :-) Markdown + mathjax are notoriously finnicky
especially for equation-heavy content like yours (which looks cool!)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jupyter/jupyter-book/issues/95?email_source=notifications&email_token=AELEJQQDHXMO3M5OEVCN4ADQCRLYJA5CNFSM4GUOXWR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3OFXNA#issuecomment-517757876,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AELEJQR4QNIL2EPB6YLNI63QCRLYJANCNFSM4GUOXWRQ
.

--
Dhruv Balwada
Postdoc
Courant Institute of Mathematical Sciences
New York University

Being able to referenced numbered equations, sub-sections of a book chapter, other chapters, or figures is a pretty important selling point for tool for book authors.  It is very hard to write things when you have to keep going back to change the references (especially in the Jupyter cell based interface).  I noticed that the Jupyter contributed extension "Table of Contents" appears to created dynamically updates section numbers in the authoring environment. Similarly, I wonder if there could be a "jupyter-book authoring tools" extension to jupyter that would handle these references and resolving the correct numbers, etc... all directly within in the jupyter interface.  This would help authors a lot and would hybridize a wysiwg type authoring interface with one that is like latex.

I think this is an awesome idea - would love to see some tooling in Jupyter Lab to improve authoring

I am interested in using jupyter-book to release interactive versions of my scientific papers, and without equation numbering I don't see how I would be able to do this.

I'm thinking of trying to add the option figure captions as this would be useful for me. I'm trying to get a feel for where I might need to jump in to do this. Is the page_html() function used to convert markdown files as well as jupyter notebooks? If not, where are the .md files converted to html? Thank you!

Hey @NatalieThurlby I think that this is actually going to be easier to fix in a new backend for Jupyter Book that we are working on - you can check it out here: beta.jupyterbook.org and the repository where we're working on it is here: https://github.com/ExecutableBookProject/cli  I think that equation numbering should now work already! (https://beta.jupyterbook.org/use/math.html)

Hi @choldgraf Thanks, that looks interesting! I feel like I'm quite close to getting my solution to work, though. I am trying to use jekyll pandoc as the markdown converter for the jupyter book instead of kramdown.
I've tested jekyll-pandoc on a regular (non Jupyter Book) jekyll site and it's working as expected (it converts ![this bit](img.png) to something including`this bit).
However, when I build + serve the jupyter book (jupyter-book build ., make serve), I can see that it's instead <img ... alt="this bit">. I am trying to figure out what's happening by looking at build.py and page.py - is it that regular markdown files (such as features/markdown.md in the demo/minimal book), are exported to html by nbconvert rather than Jekyll?
As an experiment, if I run jekyll build in the demo book directory, I get <figcaption> in the _site directory, but if I run jupyter-book build, I get alt= in the _build directory.

yep - that does sound like the reason you're running into this issue. Jekyll is actually not doing any markdown parsing - that is all being done by nbconvert -> html, and jekyll just stitches the pages together (the reason we did this was to ensure that markdown -> html would behave the same as it does with jupyter notebooks).

also, just to clarify what I mentioned above, most likely the current version of jupyter book will become a "legacy" version that doesn't get a ton of support. New features, bugfixes, etc will come to the new beta version that uses Sphinx. Just wanna give you a heads up about that

Closing as this should be superceded by beta.jupyterbook.org


Don't pull rustc-serialize from num

This is a follow-up to the previous PR. I have overlooked that the num crate pulls rustc-serialize unconditionally (and it isn't used by the way), so I disabled it.

LGTM


Provided for consumption based data to be uploaded into PVOutput

Hi @claustepper, you seem to be the only one needing this additional data. So I'm not yet convinced I should merge this change. Also I changed a lot of other code recently, so your PR doesn't merge anymore.

Closed due to inactivity. Please do reach out if you want to reopen this case.


change icon for dashboard

This PR fixes #20043
the icon for shell has changed in recent vscode merge, which is no longer appropriate to represent dashboard, I am using the default icon for now and will request a new icon for it.
before:

after:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
4 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.01%) to 42.233%

Files with Coverage Reduction
New Missed Lines
%

extensions/notebook/src/jupyter/serverInstance.ts
4
75.5%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
28175

Relevant Lines:
62340

💛  - Coveralls


Releasing code plan

Thank you for your awesome work.
When will you release your code? It's will be a big help for the research community.
Thank you.

@TuanTNG  thanks for your interests. We will release the code in these two days, stay tuned.

@TuanTNG  code released, please check it.


upm: enable Elementz IR Proximity sensor library and example

Elementz IR Infrared Proximity /Obstacle Detector Sensor Module
This sensor module works on INFRARED, for obstacles with reflective surfaces
(white colored), the maximum range will be higher and for non-reflective
surfaces (black colored), the maximum range will be lower. This can in turn
be used for detecting white/black lines (in line follower ROBOTs) or
bright/dark objects (in object identification ROBOTs)
The library provided is libupm-elementzir.so
The example provided is elementzir.cxx where it will print "Obstacle detected"
whenever an obstacle is detected by the sensor. This is being tested on Raspberry
Pi 2 Model B V1.1.
The image of the sensor is at docs/images/elementzir.jpg
Signed-off-by: Saloni Jain<EMAIL_ADDRESS>Signed-off-by: Niti Rohilla<EMAIL_ADDRESS>

Resolved the Travis Issue and rebased and tested with lastest MRAA and UPM master.

Hi @nitirohilla and thanks for submitting your code! We currently have 3 very similar libraries for IR sensors in UPM: rfr359f, rpr220, and linefinder. This last one is probably the most complete too. I would rather not add another, unless there is functionality you feel like is missing. Instead, I'll be happy to extend the documentation for the linefinder library to list the Elementz IR sensor as supported if you can confirm it works as intended.

@nitirohilla would you be able to confirm if the linefinder driver works as expected with the Elementz IR sensor? Please let us know, so that we can take the necessary actions as mentioned above by @Propanu else we might have to go ahead and close the PR. Thanks!

No response from the original author. The linefinder driver in the UPM repository already provides a driver which would work with the Elementz IR sensor. Closing the PR.


HAWKULAR-1138 move to okhttp 3.x

Can one of the admins verify this patch?

If I run the itests on this PR with parent-pom built as snapshot (this PR),
it builds, but I got the runtime error during the tests:
[ERROR] Caused by: java.lang.NoClassDefFoundError: com/squareup/okhttp/Credentials
[ERROR] at org.hawkular.cmdgw.ws.test.ClientConfig.<clinit>(ClientConfig.java:32)

@Jiri-Kremser you need to also build hawkular-commons with the new stuff. It is not enough to just build parent-pom and get this to work.

you need to also build hawkular-commons with the new stuff. It is not enough to just build parent-pom and get this to work.

right, so far everything is -snapshot

note: those snapshot versions will not make it into master branches - I will eventually be removing from my PRs those snapshot versions once things start getting released in the proper order.

with hawkular-comons built with this PR + everything glued together via snapshots and local mvn repo, I got this during the tests:
http://pastebin.com/afT0f3Uy
.. a lot of retries to get some resource from inventory and it fails with
FAILED: agentDiscoverySuccess
java.lang.AssertionError: No success getting [org.hawkular.services.rest.test.TestClient$TestResponse$$Lambda$36/1714378342@dcfda20] after trying [500] times with delay [1000] ms

there were also following lines in the log:
WARNING: A connection to http://<IP_ADDRESS>:8080/ was leaked. Did you forget to close a response body?

I guess, it's better to wait for all the other PRs to be merged, because this one depends on them

I guess, it's (^) because there is also a PR to agent https://github.com/hawkular/hawkular-agent/pull/278

I'm getting this too
metricsUp(org.hawkular.services.rest.test.MetricsAlertingITest)  Time elapsed: 25.49 sec  <<< FAILURE!
java.lang.AssertionError: No success getting [org.hawkular.services.rest.test.TestClient$TestResponse$$Lambda$27/1860886124@4a891c97] after trying [50] times with delay [500] ms
	at org.hawkular.services.rest.test.TestClient$Retry.retry(TestClient.java:85)
	at org.hawkular.services.rest.test.TestClient$TestResponse.assertWithRetries(TestClient.java:224)
	at org.hawkular.services.rest.test.MetricsAlertingITest.metricsUp(MetricsAlertingITest.java:53)

This is going to have to be fixed.

I think its because the timeouts aren't long enough - the tests should wait longer. I'm seeing this as the cause:

Caused by: java.lang.AssertionError: expected [STARTED] but found [STARTING]

This is failing for some unknown, odd reason. This all passes locally for me:
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:50 min

Even though this is green, this is an error that is occurring repeatedly:
Caused by: javax.ejb.EJBException: java.lang.ClassCastException: org.hawkular.alerts.api.services.DefinitionsService$$$view16 cannot be cast to org.hawkular.alerts.api.services.DefinitionsService

LGTM. Note that the metrics version is still older, the release does not incorporate the updated commons.  And the alerts email plugin is tied to the version of alerts embedded in metrics.  If we see any issues between components and metrics we may need to revisit the version of metrics in use.