Connecting to a SQLite/PostgreSQL database in a script

Logica is a wonderful alternative to SQL, and I would like to connect to SQLite/PostgreSQL database in a l script for data query. However currently there seems to be limited support for this.
Connection to a SQLite database file seems to have been already implemented but has gone undocumented. We could use the @AttachDatabase annotation:
@Engine("sqlite");
@AttachDatabase("db", "databaseFile.db");
Table(..r) :- `(db.tableName)`(..r);

and then run this script in local command line. But this usage does not appear anywhere in the tutorial and examples.
We have an example of connecting to a PosgreSQL database in ipynb files with the help of sqlalchemy library:
# Connect to the database.
from logica import colab_logica
from sqlalchemy import create_engine
import pandas
engine =<EMAIL_ADDRESS>pool_recycle=3600);
connection = engine.connect();
colab_logica.SetDbConnection(connection)

which is great, but I'm not sure if this is possible in a script program.

Hi,
disregard if you are busy, but if you could give some pointers on how to go about doing this, that would be awesome. I'd like to use logica outside the notebook context in my regular python projects but wasn't sure about how to go about doing this.
if it's not cleanly cut out, you can even say go to this py file and cut this out and use it with this line of code :). I'll figure something out.

Hi @RAbraham !
I believe the functionality is there, we just need to test and document it better.
I've added an example of running a predicate on SQLite engine:
https://github.com/EvgSkv/logica/blob/main/examples/more/call_sqlite.py
Let me know if it covers your usecase.
If you do have time, adding a similar (tested 😄) example for Postgres would be great.
Thank you!

Thanks!

Is it possible to connect to an existing sqlite database? Is there an annotation I can use, or dynamically pass the connection string by environment variables or even pass the connection string or object? e.g. below.

db=sqla.create_engine('sqlite:////home/stephen/db1')

Same question for postgres? how do I connect my script to a postgres database with creds without using colab_logica.SetDbConnection(connection).

@AttachDatabase annotation lets us connect to a database. We do need better docs, meanwhile here is an example of script reading table Parent from heritage.db file and writing table Sibling there.
@Engine("sqlite");
@AttachDatabase("db", "heritage.db");
@Dataset("db");

@Ground(Sibling);
Sibling(x, y) distinct :- db.Parent(z, x), db.Parent(z, y);

PersonInfo(person:,
           children_count:,
           sibling_count:) :-
  db.Person(person),
  sibling_count += (1 :- Sibling(x)),
  children_count += (1 :- Parent(person, x));

To pass a custom database file name use flags. For instance the script above would look like:
@Engine("sqlite");

@DefineFlag("database_file");

@AttachDatabase("db", "${database_file}");
@Dataset("db");

@Ground(Sibling);
Sibling(x, y) distinct :- db.Parent(z, x), db.Parent(z, y);

PersonInfo(person:,
           children_count:,
           sibling_count:) :-
  db.Person(person),
  sibling_count += (1 :- Sibling(x)),
  children_count += (1 :- Parent(person, x));

And when calling use user_flags parameter:
logica_lib.RunPredicateToPandas(
    'sibling_script.l',
    'PersonInfo',
    user_flags={'database_file': 'my_custom_file.db')

At the moment it's not possible to pass a custom connection.
If you could extend the RunPredicateToPandas with that ability it would be great!
You can see that here we call a function that does accept connection argument. So if you would add a connection argument to RunPredicateToPandas  and pass it there it should work. For now testing it on your machine would be sufficient.
Let me know if you have further questions.

Great, I'll attempt it now and let you know. Thank you very much!

I'm trying to test this. How do I populate rows in the database from logica? According to the tutorial, I hacked this but I'm missing a concept or two

Does @Dataset mean a table or  a schema?
I'm unsure why the same variabledb is used both in AttachDataset and DataSet. Are they connected, as in, do they have to have the same value db?

@Engine("sqlite");

@DefineFlag("database_file");

@AttachDatabase("db", "${database_file}");
@Dataset("db");

db.Parent("A", "B");
db.Parent("B", "C");
db.Parent("C", "D");
db.Parent("B", "E");
db.Parent("A", "F");
db.Parent("A", "G");
db.Parent("G", "H");

@Ground(Sibling);
Sibling(x, y) distinct :- db.Parent(z, x), db.Parent(z, y);

PersonInfo(person:,
           children_count:,
           sibling_count:) :-
  db.Person(person),
  sibling_count += (1 :- Sibling(x)),
  children_count += (1 :- Parent(person, x));

Error:
Traceback (most recent call last):
  File "/home/rajiv/Documents/dev/python/mercylog/venv/lib/python3.8/site-packages/pandas/io/sql.py", line 1681, in execute
    cur.execute(*args, **kwargs)
sqlite3.OperationalError: no such table: db.Person

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/rajiv/Documents/dev/python/mercylog/logica_trial.py", line 28, in <module>
    result = logica_lib.RunPredicateToPandas(
  File "/home/rajiv/Documents/dev/python/mercylog/venv/lib/python3.8/site-packages/logica/common/logica_lib.py", line 149, in RunPredicateToPandas
    return RunQueryPandas(sql, engine)
  File "/home/rajiv/Documents/dev/python/mercylog/venv/lib/python3.8/site-packages/logica/common/logica_lib.py", line 137, in RunQueryPandas
    return pandas.read_sql(statements[-1], connection)
  File "/home/rajiv/Documents/dev/python/mercylog/venv/lib/python3.8/site-packages/pandas/io/sql.py", line 483, in read_sql
    return pandas_sql.read_query(
  File "/home/rajiv/Documents/dev/python/mercylog/venv/lib/python3.8/site-packages/pandas/io/sql.py", line 1727, in read_query
    cursor = self.execute(*args)
  File "/home/rajiv/Documents/dev/python/mercylog/venv/lib/python3.8/site-packages/pandas/io/sql.py", line 1693, in execute
    raise ex from exc
pandas.io.sql.DatabaseError: Execution failed on sql '

-- Interacting with table db.Sibling

SELECT
  db_Person.col0 AS person,
  (SELECT
  SUM(MagicalEntangle(1, x_9.value)) AS logica_value
FROM
  Parent, JSON_EACH(JSON_ARRAY(0)) as x_9
WHERE
  (Parent.col0 = db_Person.col0)) AS children_count,
  (SELECT
  SUM(MagicalEntangle(1, x_12.value)) AS logica_value
FROM
  db.Sibling AS Sibling, JSON_EACH(JSON_ARRAY(0)) as x_12) AS sibling_count
FROM
  db.Person AS db_Person': no such table: db.Person

I got this to work(removed the db dataset annotation for now and any reference to db). I also pre populated the database tables person and parent. But still curious about the above question. I'll try postgres next.
@Engine("sqlite");

@DefineFlag("database_file");

@AttachDatabase("db", "${database_file}");

@Ground(Sibling);
Sibling(x, y) distinct :- Parent(z, x), Parent(z, y);

PersonInfo(person:,
           children_count:,
           sibling_count:) :-
  Person(person),
  sibling_count += (1 :- Sibling(x)),
  children_count += (1 :- Parent(person, x));

Great questions!

Does @Dataset mean a table or a schema?
@Dataset specifies schema to be used as a default for @Ground, which connects predicates to concrete tables. "Dataset" is essentially BigQuery's term for schema. I'm sure there are nuanced differences which made them call it different.

If you write @Ground(P, "mydb.my_table") predicate P is to be written to "mydb.my_table".
And if you write @Ground(P) then P is to be written to <default dataset>.P. And if you don't specify dataset then it's assumed to be equal to logica_test.

I'm unsure why the same variabledb is used both in AttachDataset and DataSet. Are they connected, as in, do they have to have the same value db?

You can attach more then one database with @AttachDatabase command. And there can be only one default dataset. Here we pass the same value because we are attaching this database as db and specifying that this is where the tables are to be written by default.
If we didn't specify Dataset then the tables would be written to a temporary in-ram database attached to logica_test alias.
Your original program complains that db.Person is undefined, which is true.
Let me know if you have further questions!

re: postgres, I did what you said and it works :+1:
from logica.common import logica_lib
from sqlalchemy import create_engine

engine = create_engine('postgresql+psycopg2://postgres:somepwd@localhost', pool_recycle=3600)
connection = engine.connect()
result = logica_lib.RunPredicateToPandas(
    '/home/rajiv/Documents/dev/python/mercylog/logica/primes_postgresql.l',
    'Prime',
    connection=connection)

print(result)

It was a simple change of just adding connection to RunPredicateToPandas
def RunPredicateToPandas(filename, predicate,
                         user_flags=None, import_root=None, connection=None):
  p = GetProgramOrExit(filename, user_flags=user_flags,
                       import_root=import_root)
  sql = p.FormattedPredicateSql(predicate)
  engine = p.annotations.Engine()
  return RunQueryPandas(sql, engine, connection)

The one thing I found odd is that I had to install psycopg2 with pip in my virtual environment. I just skimmed through my notebook of logica on postgres and I don't see me installing it. Small thing but thought I'd share.

@Dataset specifies schema to be used as a default for @Ground, which connects predicates to concrete tables. ..

wow, Ground is elegant :).
I think I may have a different mental map of databases so just wanted to clarify that first.
In postgres for e.g., I think of referring to a table as db.schema.table where db is another database(
https://chartio.com/resources/tutorials/how-to-list-databases-and-tables-in-postgresql-using-psql/#listing-databases) within the same database server. I see a schema as something like a namespace.
In the e.g. above from chartio, the database is sales, the schema is public and the table is leads. If you don't specify a table, the default schema is public but I could have another schema with the CREATE SCHEMA command.
I think SQLite has no schema concept and maybe what you are referring to maps more to SQLite(and BigQuery?)

@RAbraham , oh I see. Then I think I didn't use the word "schema" correctly in my last reply.
Yes, it appears that in SQLite schema and database are synonyms while in PostgreSQL they are not. But I am not 100% sure :-)
Logica compiler builds SQL as a string. So the @Dataset just specifies the prefix that through dot gets appended to the table name corresponding to the predicate. So it could be called differently depending on the terminology of the backend engine used.
Hope this helps and at least it's clear how to use it :-) Let me know!

re: prefix append, nice. reminds me of the Law of Demeter but for network paths :)
PR: https://github.com/EvgSkv/logica/pull/204
lmk

Law of Demeter is an interesting analogy. :-)
Merged the PR. Thank you!


[ENH] Coalesce fix

Please describe the changes proposed in the pull request:

coalesce only needs bfill

**This PR improves coalesce **

PR Checklist

Please ensure that you have done the following:

[ ] PR in from a fork off your branch. Do not PR from <your_username>:dev, but rather from <your_username>:<feature-branch_name>.

[ ] If you're not on the contributors list, add yourself to AUTHORS.rst.

[ ] Add a line to CHANGELOG.md under the latest version header (i.e. the one that is "on deck") describing the contribution.

Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

Automatic checks
There will be automatic checks run on the PR. These include:

Building a preview of the docs on Netlify
Automatically linting the code
Making sure the code is documented
Making sure that all tests are passed
Making sure that code coverage doesn't go down.

Relevant Reviewers

Please tag maintainers to review.

@ericmjl

🚀 Deployed on https://deploy-preview-1042--pyjanitor.netlify.app

LGTM, thank you @samukweku!


TreeView and ListView: Per Node ColorTheme

Is your feature request related to a problem? Please describe.
I want to use custom Colors per TreeNode.
Describe the solution you'd like
https://github.com/migueldeicaza/gui.cs/blob/main/Terminal.Gui/Views/TreeView.cs#L456
I'd like to specify the ColorScheme per TreeNode.
Describe alternatives you've considered
Because all required members in Tree.Redraw are private and, Branch is internal and there's no way to specify a custom Branch type like a CreateBranchInstance, it's impossible to use a custom ColorTheme per Node.
Additional context
I have a Command-Runner, and need a (compact) way to differentiate between a Command-Node and a Folder-Node without wasting space for a prefix text or something else.
It would be great if this is possible for ListView as well.

For the ListView see the "List View With Selection" scenario.

Thanks for the hint regarding ListView. I fake a tree view now with ListView:

For the future, a NodeRenderer in TreeView like RowRenderer in ListView would be awesome.

For TreeView probably @tznind will tell something.

Thanks for this great suggestion.
I have created a pull request that lets you tailor the color of the tree view item.  We could look into providing full control via inheritance or row renderer but I think that would be quite a bigger  change.

I just pulled main is it looks amazing. That's exactly what I need. Thank you so much. For simplicity: Is it possible to create a new nuget version (maybe pre-release)?

I just pulled main is it looks amazing. That's exactly what I need. Thank you so much. For simplicity: Is it possible to create a new nuget version (maybe pre-release)?

@tig manages the nuget releases so could say more.  There are a few nice bugfix/feature PRs merged recently so hopefully a new release is planned.

I just pulled main is it looks amazing. That's exactly what I need. Thank you so much. For simplicity: Is it possible to create a new nuget version (maybe pre-release)?

@tig manages the nuget releases so could say more. There are a few nice bugfix/feature PRs merged recently so hopefully a new release is planned.

TBH, I do releases when someone asks for one ;-).
I'll do one asap!


Ruby documentation is defunct

Investigate uses omniref for Ruby documentation, which was shut down a few months ago.  I wanted to make a pull request but couldn't find an alternative with feature parity.  Right now, though, it's broken.

Thanks for filing this. I will look for a different site next time I'm on my computer. If you find one in the meantime please add it here or submit a PR with it!

I can find API docs for Ruby and for Rails but not Ruby and Rails like omniref.  I might not be googling the right thing; sorry I'm not more helpful.

It doesn't particularly have to include rails IMO. If users want rails docs to be searched, as well as ruby stdlib docs, they can configure that I guess.


DatePicker component does not work.

In my code I added
import {DatePicker} from 'material-ui';

Then i've got error
ReferenceError: document is not defined
at [object Object].componentWillMount (~/node_modules/material-ui/lib/overlay.js:64:34)
I also change my import to
const DatePicker = require('material-ui/lib/date-picker/date-picker');
Still no luck

Getting this error too with the latest release, hash that introduced this
https://github.com/callemall/material-ui/commit/d25d12287b1527bde12f15b6d5c2fae473826a4a#diff-26176e848c2cae459bdd7e0ce928752b

Any idea on how to fix it?

Hard to say without having the context of was the issue, but reverting to componentDidMount works on my end. @mmckelvy Can you validate?

@mimcabrera @mgibeau Odd that document would be undefined.  Are you by any chance rendering on the server?

@mmckelvy Can't we just go back to use componentDidMount?

@mmckelvy @mimcabrera @mgibeau Could you have a look at https://github.com/callemall/material-ui/pull/2021?

Still getting this error.... Please guys don't close the issue if your are not sure it's solved and merged. You just confuse people

@izziaraffaele Is this the same error?If now, could you open a new issue with the log so that we can have a look. Thank.

The issue is always the same... you use document in componentWillMount which brakes server side rendering since the DOM is not available on server side.
in overlay.jsx
componentWillMount() {
    this._originalBodyOverflow = document.getElementsByTagName('body')[0].style.overflow;
  },

https://github.com/callemall/material-ui/blob/master/src/overlay.jsx#L57

Exactly what I meant... I did npm installl from scratch, got the error "document not defined", found the problem on componentWillMount, I came here to report the issue but I saw you already solved the problem and close the issue.
Remember that what people get is this https://github.com/callemall/material-ui/blob/v0.13.1/src/overlay.jsx, so even if you fix the issue keep it open till you'll merge those changes in the npm package.

@izziaraffaele I see your point. Well, It's a matter of convention. Do we close an issue once it's merged to master or once it's released?

I did npm installl from scratch, got the error "document not defined"

Sorry for that. @shaurya947 Can we do a hotfix release?

How many people are using your master branch? how many popole are using your stable release instead? For how many people this is still an issue?
It doesn't matter if you solve it in the master branch, in you local machine or in the branch XYZ... Is it an issue for most people using your package? Then it should stay open...

@izziaraffaele I disagre. I'm gonna follow the facebook convention for react here. For instance https://github.com/facebook/react/issues/5337.

@oliviertassinari not sure if it's a "convention" or it's just what they did in that case ( BTW it's a feature not a bug ;) ). I would like to read some more info about it if you have others.
Anyways it's up to you.... I just think that the other approach lead to lot less questions, duplicated issues and headache for people using your package.

@izziaraffaele Alright, since it's not released and people keep opening issue for this, I'm reoponing the issue until we release.

@oliviertassinari Let's keep this issue open. We should release early next week and then we can close this.

Hi guys,
the fix for this issue is to move it on componentDidUpdate, and it works correctly with server side rendering also.
  componentWillMount: function componentWillMount() {
    //this._originalBodyOverflow = document.getElementsByTagName('body')[0].style.overflow;
  },

  componentDidUpdate: function componentDidUpdate() {
    this._originalBodyOverflow = document.getElementsByTagName('body')[0].style.overflow;
    if (this.props.autoLockScrolling) {
      if (this.props.show) {
        this._preventScrolling();
      } else {
        this._allowScrolling();
      }
    }
  }

HI guys, I came across the same issue while trying the framework. +1 to solve this one asap.

Releasing soon, should fix this..

The fix was release with v0.13.2.


Different height for Input and Select on Android

🐛 Bug Report
Height of Input and Select component is not equal on Android, but seems to be right on iOS
To Reproduce
Steps to reproduce the behavior:
Add Input and Select component on a flexDirection: "row" to clearly see the difference.

Expected behavior
Should have the same height (like on iOS)

UI Kitten and Eva version

Package
Version

@eva-design/eva
1.4.0

@ui-kitten/components
4.4.1

Environment information

System:
    OS: macOS 10.15.4
    CPU: (4) x64 Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
  Binaries:
    Node: 12.16.1 - /usr/local/bin/node
    npm: 6.13.4 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  SDKs:
    iOS SDK:
      Platforms: iOS 13.4, DriverKit 19.0, macOS 10.15, tvOS 13.4, watchOS 6.2
  IDEs:
    Android Studio: 3.6 AI-192.7<IP_ADDRESS>41897
    Xcode: 11.4/11E146 - /usr/bin/xcodebuild
  npmPackages:
    react: ~16.9.0 => 16.9.0
    react-native: https://github.com/expo/react-native/archive/sdk-37.0.0.tar.gz => 0.61.4

Can confirm the same issue for me.
It also doesn't really clear how you can change the height for the Select component.
<Select style={{height: }} .../> doesn't help.

Migrated to 5.0.0-alpha1, but Select appears to have the same problem as reported

The issue still exists in 5.0.0.
iOS works as intended, but Android has shorter Select.


Replaced Razor syntax with TagHelpers

Replaced all:

[x] @Html.Partial and @Html.PartialAsync
[x] @Html.RenderPartial and @Html.RenderPartialAsync

with <partial /> TagHelper

There are many indentation issue on the update and it's hard to review

IMHO, you can ignore the indentation issues. You just have to take the Razor syntax (those are replaced with TagHelpers) into account. The happened because of my VS settings. When I paste or save any .cshtml file, it automatically formats the whole document.

Please update your VS setting. For consistency, we need to fixed indentation before merge :)

@thiennn Should I close thus PR and reopen another one after fixing the indentation issues?

@rafsanulhasan either is fine. Pick the one that easier for you.

I've created a new PR #559


Add the future lighting toggle to mods

Blame git..................

ok fine... one attribute per line.....

Oops.

Seems like a pretty cool feature.


Cannot read property 'getUserMedia' of undefined

Chrome Version 81.0.4044.138 (Official Build) (64-bit)
OSX Catalina 10.15.2 (19C57)
pong.js:112 766 1440
planck-with-testbed.js:13564 On load.
planck-with-testbed.js:13193 Starting...
planck-with-testbed.js:13205 Loading apps...
planck-with-testbed.js:13156 Creating app...
planck-with-testbed.js:13592 Creating Canvas...
planck-with-testbed.js:13617 Creating stage...
planck-with-testbed.js:13159 Initing app...
planck-with-testbed.js:13165 Starting app...
planck-with-testbed.js:13662 Resize: 2880 x 1532 / 2
pong.js:205 ready!
5handtrack.min.js:25 Uncaught (in promise) TypeError: Cannot read property 'getUserMedia' of undefined
at handtrack.min.js:25
at new Promise ()
at Object.startVideo (handtrack.min.js:25)
at startVideo (pong.js:30)
at toggleVideo (pong.js:45)
at HTMLButtonElement. (pong.js:57)
planck-with-testbed.js:13662 Resize: 1362 x 1532 / 2
DevTools failed to load SourceMap: Could not load content for chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/sourcemaps/onloadwff.js.map: HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME

Hi,
Can you provide additional context? E.g. what code you were attempting to run?
The error suggests this might be chrome issue.
Explore to see if the discussion here is helpful https://superuser.com/questions/1523427/google-chrome-devtools-failed-to-parse-sourcemap-chrome-extension
-V


Jest-resolve broken: getMockModule throws an error

💥 Regression Report
A clear and concise description of what the regression is.
Last working version
Worked up to version: 26.6.1
Stopped working in version: 26.6.2
To Reproduce
Steps to reproduce the behavior:
jest.config.js
'moduleNameMapper': {
    '^ui(.*)': '<rootDir>/ui$1',
}

import xSelector from "selectors/ui.selector.js"

throw error
Configuration error:

Could not locate module ui.selector mapped as:
/Users/mamartynov/Projects/dc_offers_front/src/ui$1.

Please check your configuration for these entries:
{
  "moduleNameMapper": {
    "/^ui(.*)/": "/Users/mamartynov/Projects/dc_offers_front/src/ui$1"
  },
  "resolver": undefined
}

Expected behavior
not throwed
Link to repl or repo (highly encouraged)
Run npx envinfo --preset jest
  System:
    OS: macOS 10.15.2
    CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
  Binaries:
    Node: 13.6.0 - ~/.nvm/versions/node/v13.6.0/bin/node
    Yarn: 1.21.1 - ~/.yarn/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v13.6.0/bin/npm
  npmPackages:
    jest: 26.1.0 => 26.1.0

I can fix the regex, but the problem persists for others
This error here https://github.com/facebook/jest/blob/master/packages/jest-resolve-dependencies/src/index.ts#L78

From #10713 - @jeysal @brapifra ideas?

Should we just try-catch the getMockModule call?
@FrontendMMA do you have a reproduction we can pull down and test?

Sorry to see this regression! :crying_cat_face:
Should we just try-catch the getMockModule call?
I think this should work. A reproduction would be certainly helpful.
Happy to open a PR if needed!

Sorry, I can't provide the project
checked: try/catch works

Can you reduce it down? Just delete all code except the test file, the file it resolves to and the minimal jest config needed? A try/catch is not the correct solution I'd think - we shouldn't throw in the first place

https://github.com/FrontendMMA/jest-resolve--bug-example

change test file or tested file (App.js / App.test.js)
execute jest -o (repeats in a "onlyChanged" mode)

Thanks @FrontendMMA! I had to change some stuff to make it work and understand where the regression was, I opened a PR: https://github.com/FrontendMMA/jest-resolve--bug-example/pull/1
@brapifra would you be able to take a look at this? If you look at my branch it can be reproduced by removing the resolutions

Yes, I will look into this tomorrow @SimenB

Thanks!

Fix released in https://github.com/facebook/jest/releases/tag/v26.6.3


[ru] Localize function-as-a-service

Describe your changes
Localize function-as-a-service
Related issue number or link (ex: resolves #issue-number)
resolves #3060
Checklist before opening this PR (put x in the checkboxes)

[x] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[x] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

/label tide/merge-method-squash

Will we prefer "Функция как сервис" instead of the currently used "Функция-как-сервис"?


Support 128-bit integers & DocFragments

Add support for 128-bit integers.
Support Document Fragments in the Meta enum.
DocFragments are needed so that serde-yaml can always emit a Yaml object rather than something like a Vec.

(2) represents a slight mismatch with the way comment support is handled in alevinval's branch (please review and advise.).  IMHO, requiring serde-yaml to return Vec<Yaml> from it's functions incurs allocation overhead for every field serialized.  Meta::Fragments contain a Vec<Yaml>, so there is allocation for all fields with comments.  But, for all fields without comments, there is no additional allocation.

BTW, I just pushed changes to my fork of serde-yaml that use these changes to add comments, integer bases and block strings to serde-yaml: https://github.com/cfrantz/serde-yaml/tree/yamlformat-alevinval
You can take a look at tests/test_format.rs to see how things work.

Hi @cfrantz, I will need time to see what you've done to fully understand it. I can give you some reasons why we are differing in the implementation. The state machine of yaml-rust is such so that YAML nodes are always being inserted either into an array, or map.
The way I managed to add comment support was to keep comments in a stack, and dump then on:

After inserting a fresh new vec/map in the Yaml tree
Before finishing loading a map/sequence

For emitting, these things are taken into account, plus the special thing of peeking the next entry (in either the array or the map) to support inline comments.
What I mean is that if you have a comment to emit, it's only a matter of pushing it to the existing Vec/HashMap as a new entry. If you don't have to emit a comment, then simply don't push the Yaml entry.


FIX: TimelockController, operation is not ready

Hey, @PatrickAlphaC
I've run into the same issue as you do during live of "code a dao, python version".
So far the problem reproducing when trying to use ganache local chain, which in brownie of version 1.16.4 set as development network with this config,
Ganache-CLI
    id: development
    cmd: ganache-cli
    cmd_settings: {'accounts': 10, 'evm_version': 'istanbul', 'gas_limit': 12000000, 'mnemonic': 'brownie', 'port': 8545}
    host: http://<IP_ADDRESS>

Here's a patch which fixes timelockcontroller revert with simple delay between queue and execute trx
BTW,
thanks for your outstanding efforts at democratization of blockchain development,
best of luck!

Oh this is lovely!!! Thanks so much!!!


Set the maximum concurrency

How does the latest version set the maximum concurrency? Is the default maximum concurrent number four?

I would suggest not changing anything about the maximum number of connections made. If you do need to do so, you can set the httpMaximumConnectionsPerHost property on a URLSessionConfiguration and use it to initialize your SessionManager instance.

I know, but I can't change the number of concurrent downloads in the queue. Is the maximum number of concurrencies set in Alamofire?

No, there is no control over the level of concurrency in the Alamofire APIs. Alamofire 5 has rewritten how Alamofire handles queuing, but that won't give you control over a maximum either, just control over whether different parts of Alamofire run on serial or concurrent queues.

Oh, I know, thank you.


Update history import

Use direct module references
Add commented import against memory history for use with chrome extensions or electron.

Hi @tracker1 , are you available to address the comment? Thanks!


SWIFT remote not working without significant error message

What is your rclone version (eg output from rclone -V)

rclone v1.29

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Mac OS X Yosemite 10.10.5

Which cloud storage system are you using? (eg Google Drive)

RadosGW (Ceph) with SWIFT API
Version 0.94.6-1trusty (package from Ubuntu)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -v lsd lsswift:

A log from the command with the -v flag (eg output from rclone -v copy /tmp remote:tmp)

macsp:~ proto$ rclone -v lsd lsswift:
2016/04/28 17:12:02 Failed to create file system for "lsswift:": Bad Request
macsp:~ proto$
Here my remote config
[lsswift]
type = swift
user =<EMAIL_ADDRESS>key = secret
auth = https://keystone.cloud.switch.ch:5000/v2.0
storage_url = https://os.unil.cloud.switch.ch/swift/v1
I dont understand if the Bad Request is to the Keystone server for authentication or to the RadosGW to list the buckets

Try without the storage_url you shouldn't need that.
If that doesn't work then try the lsd with --dump-bodies and post the output (remove the Authorization lines).

I removed storage_url but still does not work. However with --dump-bodies looks like it is a problem talking to keystone
vpn-ho-d-130:~ proto$ rclone -v --dump-bodies lsd lsswift:/
2016/04/28 20:51:19 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2016/04/28 20:51:19 HTTP REQUEST
2016/04/28 20:51:19 POST /v2.0/tokens HTTP/1.1
Host: keystone.cloud.switch.ch:5000
User-Agent: rclone/v1.29
Content-Length: 91
Content-Type: application/json
Accept-Encoding: gzip

{"auth":{"passwordCredentials":{"username":"saverio.proto@switch.ch","password":"secret"}}}
2016/04/28 20:51:19 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2016/04/28 20:51:19 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2016/04/28 20:51:19 HTTP RESPONSE
2016/04/28 20:51:19 HTTP/1.1 401 Unauthorized
Content-Length: 114
Connection: keep-alive
Content-Type: application/json
Date: Thu, 28 Apr 2016 18:51:20 GMT
Server: nginx
Vary: X-Auth-Token
Www-Authenticate: Keystone uri="https://keystone.cloud.switch.ch:5000"
X-Distribution: Ubuntu
X-Openstack-Request-Id: req-aa3acd86-e927-42c0-b1be-0b2a51f80adf

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}
2016/04/28 20:51:19 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2016/04/28 20:51:19 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2016/04/28 20:51:19 HTTP REQUEST
2016/04/28 20:51:19 POST /v2.0/tokens HTTP/1.1
Host: keystone.cloud.switch.ch:5000
User-Agent: rclone/v1.29
Content-Length: 97
Content-Type: application/json
Accept-Encoding: gzip

{"auth":{"RAX-KSKEY:apiKeyCredentials":{"username":"saverio.proto@switch.ch","apiKey":"secret"}}}
2016/04/28 20:51:19 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2016/04/28 20:51:20 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2016/04/28 20:51:20 HTTP RESPONSE
2016/04/28 20:51:20 HTTP/1.1 400 Bad Request
Content-Length: 246
Connection: keep-alive
Content-Type: application/json
Date: Thu, 28 Apr 2016 18:51:20 GMT
Server: nginx
Vary: X-Auth-Token
X-Distribution: Ubuntu
X-Openstack-Request-Id: req-2a361d82-e212-4cbd-afde-c62cd293d5a9

{"error": {"message": "Expecting to find passwordCredentials in auth - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.", "code": 400, "title": "Bad Request"}}
2016/04/28 20:51:20 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2016/04/28 20:51:20 Failed to create file system for "lsswift:/": Bad Request

Is it possible you got your username or password wrong? That is what is looks like. The second attempt tries to log in the racks pace way which fails too.
Has your password got any funny characters in? (non alphabetic?)
Can you log in with a different tool (eg the openstack swift client?)

Hello,
the password has no special characters. However you were right ! I typed the wrong password :+1:
Now it still does not work but with a different error:
rclone -v lsd --dump-bodies lsswift:
2016/04/28 22:30:26 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2016/04/28 22:30:26 HTTP REQUEST
2016/04/28 22:30:26 POST /v2.0/tokens HTTP/1.1
Host: keystone.cloud.switch.ch:5000
User-Agent: rclone/v1.29
Content-Length: 95
Content-Type: application/json
Accept-Encoding: gzip

{"auth":{"passwordCredentials":{"username":"saverio.proto@switch.ch","password":"secret"}}}
2016/04/28 22:30:26 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2016/04/28 22:30:27 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2016/04/28 22:30:27 HTTP RESPONSE
2016/04/28 22:30:27 HTTP/1.1 200 OK
Content-Length: 406
Connection: keep-alive
Content-Type: application/json
Date: Thu, 28 Apr 2016 20:30:27 GMT
Server: nginx
Strict-Transport-Security: max-age=31536000; includeSubdomains
Vary: X-Auth-Token
X-Distribution: Ubuntu
X-Openstack-Request-Id: req-f8cf78ad-b2c4-471f-b182-2a9b30a18781

{"access": {"token": {"issued_at": "2016-04-28T20:30:27.191649", "expires": "2016-04-28T21:30:27Z", "id": "secrettoken", "audit_ids": ["secret"]}, "serviceCatalog": [], "user": {"username"<EMAIL_ADDRESS>"roles_links": [], "id": "870aab4e3af6441b8b5ebfb0dcaa8412", "roles": [], "name"<EMAIL_ADDRESS>"metadata": {"is_admin": 0, "roles": []}}}
2016/04/28 22:30:27 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2016/04/28 22:30:27 Failed to create file system for "lsswift:": Response didn't have storage url and auth token

I tried both with and without the following line in the conf file:
storage_url = https://os.unil.cloud.switch.ch/swift/v1
My keystone catalog has the following entry:
Service: object-store
+-------------+------------------------------------------+
|   Property  |                  Value                   |
+-------------+------------------------------------------+
|   adminURL  | https://os.unil.cloud.switch.ch/swift/v1 |
|      id     |     191be5f7b7f64277b6ab1e9a6564e0c3     |
| internalURL | https://os.unil.cloud.switch.ch/swift/v1 |
|  publicURL  | https://os.unil.cloud.switch.ch/swift/v1 |
|    region   |                    LS                    |
+-------------+------------------------------------------+

The swift openstack client works fine, so I assume the entries published in the keystone catalog to be correct.

Examining the service catalogue returned, there are no entries for swift in there.
In fact there is nothing in the serviceCatalog at all.
Has the user you are trying to use got permissions to use swift?
{
    "access": {
        "metadata": {
            "is_admin": 0,
            "roles": []
        },
        "serviceCatalog": [],
        "token": {
            "audit_ids": [
                "secret"
            ],
            "expires": "2016-04-28T21:30:27Z",
            "id": "secrettoken",
            "issued_at": "2016-04-28T20:30:27.191649"
        },
        "user": {
            "id": "870aab4e3af6441b8b5ebfb0dcaa8412",
            "name"<EMAIL_ADDRESS>            "roles": [],
            "roles_links": [],
            "username"<EMAIL_ADDRESS>        }
    }
}

I'd expect to see something more like this - note the serviceCatalog.endpoints.publicURL and token.id which is what the swift module is looking for.
{
    "access": {
        "serviceCatalog": [
            {
                "endpoints": [
                    {
                        "adminURL": "https://csproxy1.memset.com/...",
                        "internalURL": "https://csproxy1.memset.com/...",
                        "publicURL": "https://csproxy1.memset.com/...",
                        "region": "reading"
                    }
                ],
                "name": "memstore",
                "type": "object-store"
            }
        ],
        "token": {
            "expires": "2016-04-28T22:42:54.334167",
            "id": "...",
            "tenant": {
                "id": "...",
                "name": "..."
            }
        },
        "user": {
            "id": "admin",
            "name": "admin",
            "roles": [
                {
                    "id": "admin",
                    "name": "object-storage:admin",
                    "tenantId": "..."
                }
            ]
        }
    }
}

@ncw what version of Keystone are you running ? could be a problem with keystone on my side ?

I'm not sure what keystone version that is, but you should get the endpoints returned from a correctly configured keystone.  That is why I guess you've had to specify the storage_url to the openstack swift client - you shouldn't have to do that - it should read it from the service catalog.
So I would guess it is a problem with keystone config
One thing to try would be to put the region in the rclone config..

I fixed it ! It is a know issue in keystone:
https://answers.launchpad.net/keystone/+question/192094
If you dont specify your tenant you get a empty service catalog.
I had to add only my tenant name (in my case it is the same as the user name).
in rclone.conf I added the line
tenant =<EMAIL_ADDRESS>
and now it works. Thank you.

Great - well done!
Does that work without the storage url now?

Yes it works without storage url.
It is the tenant option that is really mandatory

I've written some more docs about these errors see commit below!


Can you please add an option to set the items programmatically?

Thanks in advance

I'll see what I can do. There's already an option to set a menu, which you can build it yourself, but I'll see if I can add an easier way

@rubensousa
Thanks. I forked your repo yesterday and did this already. I'm about to make a PR. I hope you merge it :)

Added in 1.5.0

@rubensousa
Thanks but I don't see a way to set the menu id programmatically and I think the library generates it randomly when building the menu in the toolbar, so I think there's no way to make the onClick listener work always, since getting the id would not work.
If I'm wrong please let me know how to do that, because the readme has nothing about this new feature. Thanks


SOAP support

Hello,
I was wondering if wiremock supports https SOAP web services. The examples all seem to me to indicate a path needs to be unique. In my case every request is a POST to the same end point like:
https://myserver.com:443/sdk

Where the payloads being sent would be unique, and the responses coming back would be unique. If this is possible would you mind showing me a simple example?
Thanks

How about SOAP support for the standalone version? Is this possible?

Nothing to stop you using it for SOAP right now. I did on a project a few months ago.

Problems I am facing are:

Wiremock in record mode records only the definitions. I need the actual responses recorded as well.
Using SOAP a given url may house quite a few different operations. Wiremock standalone is meant to cache only one response per given url.

Can you give me some guidelines on how to solve these two? Am I missing something or is this impossible to do with wiremock standalone?

You should see the response bodies being recorded under the __files directory. Is this not the case? Or is something else missing?
WireMock can match on body content and headers in addition to URL, so if your mappings specify a SOAPAction header and XML body you'll get different responses for different requests.

You should see the response bodies being recorded under the __files directory. Is this not the case? Or is something else missing?

It is there, but from 5 calls being made, only 1 is being recorded and it is not the actual call with the data and all that - it's just the WSDL definition.

WireMock can match on body content and headers in addition to URL, so if your mappings specify a SOAPAction header and XML body you'll get different responses for different requests.

In general, the SOAP protocol does not pass the action as a header - it's in the body of the request in XML format. Will have a look at using custom header. Feels like a workaround, but should do the job.

The header isn't necessary, you should be able to match on body and url alone.
The recorder is a bit simplistic in terms of how it figures out whether to treat a request as already arrived, so that's probably why you're seeing only one recording. Unfortunately this means you'll need to hand craft the additional variations (or code them).

@tomakehurst could you provide some examples of using Wiremock with Soap?

That's how I configured it to work with Soap calls.
/mappings/SayHello.json
{
  "request": {
    "method": "POST",
    "url": "/mocked-soap-service.wsdl",
    "headers" : {
      "SOAPAction" : {
        "contains" : "#SayHello"
      }
    }
  },
  "response": {
    "status": 200,
    "bodyFileName": "SayHello.xml",
    "headers": {
      "Content-Type": "application/xml"
    }
  }
}

/files/SayHello.xml
<?xml version="1.0"?>

<soap:Envelope
        xmlns:soap="http://www.w3.org/2003/05/soap-envelope/"
        soap:encodingStyle="http://www.w3.org/2003/05/soap-encoding">
    ...
    Message information goes here
    ...
</soap:Envelope>

Executing request in PHP
<?php
$params = array(
  'location' => 'http://localhost:8080/mocked-soap-service.wsdl',
  'uri' => 'http://localhost:8080/mocked-soap-service.wsdl',
);

$client = new SoapClient(null, $params);
var_dump($client->__soapCall("SayHello", array("SomeKindOfID" => 1234)));

@tomakehurst, I still think the recorder would benefit getting improved. It would spare the hassle of creating the content manually.

Can you be a bit more specific about what you're after?
SOAP support is on the roadmap (although admittedly not in my badly organised issues list), but there's quite a lot of different things that could entail.

Almost 8 months later but here I go.. just started using WireMock with SOAP web services and, due to the recorders "simplicity" to differentiate requests (as pointed by @tomakehurst), I just kept repeating:

Run WireMock with proxy-all and record-mappings enabled
Make a request to WireMock using Postman
Adjust bodyPatterns to use contains instead of matching the whole XML
Rename the mappings/.json file to .bkp
Repeat

After calling all my endpoints I just renamed .bkp files to .json, ran WireMock one more time and everything was working as expected. This isn't a superb solution but with a limited number os endpoints, helps (and I think it's better than handcrafting all requests).
Now, I think the recorder could be improved implementing different strategies to differentiate requests. I can see 3 different forms right now:

Body equality: that's how WireMock works today
SOAPAction header: when the header is used, it's certainly the way to go. The recorder could create the equality conditions (and differentiate future requests, of course) based on it
soap:Body's first element name: when there is no SOAPAction header, usually the first element of the body can be used to distinguish the request (ie: createUserRequest, cancelApplication). Same thing: the recorder could create equality conditions and differentiate requests based on it.

Does it make sense to you guys? I haven't take a look at the source code yet, but I believe it would be a nice addition to recording the never ending world of SOAP Web Services. :)

Have you tried using the new recorder? It's a lot more configurable than the original version and can construct scenarios from runs of identical requests.

Just read about it now (shame on me)! The "Repeats as scenarios" kills the need to rename/restart wiremock, indeed.
Anyway, I believe more generalist Request Body Matchers would be a great addition to SOAP services (the 3 different strategies I cited before). Matching the whole body makes sense in very little cases (at least in my experience!).
What u think? Worth opening another issue with this feature suggestion?

I have half an idea about this.
I agree that matching the whole body is often a bad idea - it can get very computationally expensive, and it ends up being brittle as the API changes.
I'd like to fix this by adding some additional record config with a collection of XPaths that should be used as body matchers.
For example, if I'm expecting to record two types request:
<user>
  <id>abd-efg</id>
</user>

and
<order>
  <orderNumber>567</orderNumber>
</order>

Then I can configure the recorder to find the ID elements:
"bodyXPaths": [
  "//id", "/order/orderNumber"
]

Which would produce mappings like:
{
  "request": {
    "bodyPatterns": [
      { "matchesXPath": "//id" }
    ]
  }
}

or
{
  "request": {
    "bodyPatterns": [
      { "matchesXPath": "/order/orderNumber" }
    ]
  }
}

Depending which request document it saw.
What I'm not sure how to handle is when more than one of your XPaths matches a given request.

Hmm.. yeah. That would solve the idea about matching based on body properties.
If more than one matcher is fired Wiremock should simply return the first (as it already does in scenarios cases, as far as I recall).
The last suggestion would be some to allow some kind of recorder config that would record specific headers and generate appropriate mappings to match that specific value (SOAPAction values, in this case).

Yeah, I guess in the simple case picking the first match would be fine.
It gets a little harder if there isn't a single field that's uniquely identifying e.g. you need to match on both first and last name. Perhaps the config model needs to account for that i.e. allowing you to specify (or imply) AND and OR.
Header specifications in the recorder are already implemented :-)

That's how I configured it to work with Soap calls.
/mappings/SayHello.json
{
  "request": {
    "method": "POST",
    "url": "/mocked-soap-service.wsdl",
    "headers" : {
      "SOAPAction" : {
        "contains" : "#SayHello"
      }
    }
  },
  "response": {
    "status": 200,
    "bodyFileName": "SayHello.xml",
    "headers": {
      "Content-Type": "application/xml"
    }
  }
}

/files/SayHello.xml
<?xml version="1.0"?>

<soap:Envelope
        xmlns:soap="http://www.w3.org/2003/05/soap-envelope/"
        soap:encodingStyle="http://www.w3.org/2003/05/soap-encoding">
    ...
    Message information goes here
    ...
</soap:Envelope>

Executing request in PHP
<?php
$params = array(
  'location' => 'http://localhost:8080/mocked-soap-service.wsdl',
  'uri' => 'http://localhost:8080/mocked-soap-service.wsdl',
);

$client = new SoapClient(null, $params);
var_dump($client->__soapCall("SayHello", array("SomeKindOfID" => 1234)));

But then do you mean we need to have one url for one kind of response. If i need to give 5 different kinds of response, do i need to add 5 urls and have five files served like you explained above?


Files being opened and unsaved, after replace all

search some content globally in the project or specify some folder,
enter replace content and click on replace all.
but in changed files like bitbucket updates, showing all the changes

it's showing all the files which are modified using replace, staying opened and unsaved.
Please resolve this soon.

Do you have "auto save" enabled?

no response. Closing.


Fixes for redesign thread ancestry feature

a follow-up to https://github.com/Expensify/App/pull/38722
cc @allroundexperts
Details

Fixed Issues

$ https://github.com/Expensify/App/issues/39246
$ https://github.com/Expensify/App/issues/39331
$ https://github.com/Expensify/App/issues/39301
PROPOSAL: N/A
Tests

Test 1:

Click on FAB > Start chat
Create a group chat with User B and C
Send a message in group chat
Right click on the message > Reply in thread
Mention User D and invite User D from Concierge whisper
As User D, navigate to the thread
As User D, verify that the blue "Thread" above the parent message is disabled, because they are not a member of main chat.
As User D, verify that the blue "Thread" above the parent message color is gray.
As User D, verify that the parent message is disabled
As User D, verify that the header does not contain a "From ..." link to the parent thread

Test 2:

Send a message
Right click on the message > Reply in thread
Send a reply
Delete the reply
Verify that when the reply is deleted, the gray "Replies" disappear together with the deleted reply

Test 3 (Only on Web and Desktop):

Send a message in chat.
Right click on the message > Reply in thread.
Right click on the parent message in thread > Edit comment.
Hover over the text in the edit composer.
Verify that the cursor is text cursor when hovering over edit composer in thread.

Test 4 (Only on Web and Desktop):

Send a message
Right click on the message > Reply in thread
Send a message in thread
Verify that you can't select and copy "Thread" and "Replies words in thread dividers.

[x] Verify that no errors appear in the JS console

Offline tests

Same behavior.
QA Steps

Same as tests.

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Uploading Recording 2024-04-02 042921.mp4…

Android: mWeb Chrome

https://github.com/Expensify/App/assets/77965000/5472a5e3-0d72-4923-ace5-1dc22c9bfe70

iOS: Native

https://github.com/Expensify/App/assets/77965000/72f1dba2-646d-4f01-a0df-726dca2354d6

iOS: mWeb Safari

https://github.com/Expensify/App/assets/77965000/93a43d61-1d40-4230-9ac4-513cf4392771

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/77965000/1b8edf07-4ebe-441e-84c0-491ebc078a20
https://github.com/Expensify/App/assets/77965000/961362eb-4a9a-4d3f-96a8-158d6080d5ac

MacOS: Desktop

Uploading Recording 2024-04-02 040344.mp4…

@rayane-djouah I see you linked https://github.com/Expensify/App/issues/39331 here, so does that mean the blocker will get fixed in this PR as well?

Ah nice, I will add myself for review as well then so I can take a look when it's ready

Seems @allroundexperts will review this one.

@rayane-djouah You are missing the screenshots/videos

@dannymcclain will hold the answer but given we used the link color to signify a link, I think we probably should. But I also don't mind it being the same color cause it does separate the Thread from Replies. But yeah, let's see what Danny says :)

Yeah, that makes sense to me but also curious for Danny's opinion.

Yeah I guess we probably should change it if it's not actually clickable. I think we have two options:

Use same color as "Replies" (text-supporting)
Keep the blue but make it 50% opacity (this would follow our disabled button state pattern)

But also, why is it disabled? How does it become disabled? Will it become un-disabled at some point? I think these questions are key to knowing which of the above choices is best.
@Expensify/design what do you think?

I prefer the idea of just "unlinking" the text and making it look gray. I think we try to avoid the disabled pattern if we can, especially since it looks a lot like offline patterns.

I prefer the idea of just "unlinking" the text and making it look gray. I think we try to avoid the disabled pattern if we can, especially since it looks a lot like offline patterns.

That works for me. 👍

One question though - how far "up the chain" do we expose parent comments to a user who only has access to a child thread? For example, if I have:

Thread Level 1

Thread Level 2

Thread Level 3

And I only have access to view Thread Level 3, should I even be able to see the original comment from Thread Level 1?

No I don't think so—I think in that situation you'd see the OG "thread message" from level 2 as the top-most ancestor in the thread, with no back links to the "higher up" messages.

Cool, that was my thinking too - thanks for confirming!

Did we get rid of the message highlight when navigating to an ancestor message?

When deleting, the Replies does go away as expected, but my messages just come back when I leave the thread and come back. Is that unrelated to this (or expected) or is something messed up?
https://github.com/Expensify/App/assets/7516624/12ff4536-b362-4679-8251-85f9210b3a97

Trying to check Test 1 (disabled ancestor threads because of private rooms) but I keep running into the forced update (on web and desktop) and can't seem to get around it.

Trying to check Test 1 (disabled ancestor threads because of private rooms) but I keep running into the forced update (on web and desktop) and can't seem to get around it.

According to this convo in Slack, it looks like you may just need to merge main again.

Ok this is generally super dope. I went through all the tests and everything seems to be working and looking as expected.
I had a couple little questions about the first test scenario:

Do we need to use the not allowed cursor on hover for the top-most message? I feel like since we've removed the link and everything, we could just use the regular cursor the same as the "replies" messages.
Is the hover action menu different for that post because it technically shows up in a chat that I don't have access to? Is that why I can't add emoji reactions and stuff?

cc'ing @Expensify/design to weigh in on my top bullet point.
https://github.com/Expensify/App/assets/7516624/e617d2ca-185d-4c29-a282-c841ec6b7446

Totally agree, I don't think we need the not-allowed cursor here.

Ok sweet. @rayane-djouah once we update that not-allowed cursor, I think from a design perspective we'll be set! 🎉

Yes, This is the expected behavior.

Excellent!

Do we need to use the not allowed cursor on hover for the top-most message? I feel like since we've removed the link and everything, we could just use the regular cursor the same as the "replies" messages.

Done!

Sorry, I should've been more clear in my initial comment. We shouldn't use the not-allowed cursor anywhere. We don't need it on the Thread text/icon because we already made it "regular" text instead of a link. So it should look/behave exactly like the Replies text/icon.

@hungvu193 Are you reviewing this one as C+?

@hungvu193 Are you reviewing this one as C+?

I saw @rayane-djouah tagged @allroundexperts for reviewing above.
I can review this one if needed.

Ahh I see, this was a regression of https://github.com/Expensify/App/pull/38722 so we need to assign the original assignees.

@allroundexperts Please take a look when you have some time!

@rayane-djouah https://github.com/Expensify/App/issues/39301 is still not fixed. Now we're completely hiding the parent message. I think we should still display the parent message but just not allow a click to it.
https://github.com/Expensify/App/assets/30054992/52796c46-583a-4e5d-81a0-8cd5f2b06663

Looks solid otherwise.

@rayane-djouah Why have we mentioned https://github.com/Expensify/App/issues/39246 in the issue description as a fixed issue then?

Okay, lets reset. Is #39331 fixed in this PR?

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/30054992/28c52345-3931-40fb-a109-191cb5ca5d87

Android: mWeb Chrome

https://github.com/Expensify/App/assets/30054992/a1907862-65f3-4bd9-8438-8e6f4345ece9

iOS: Native

https://github.com/Expensify/App/assets/30054992/839f8841-b46c-43e0-93ae-d4b012a47a48

iOS: mWeb Safari

https://github.com/Expensify/App/assets/30054992/d352e57d-46d7-494e-a363-dc5bb4f2ff16

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/30054992/9ce1a509-8ecf-4423-ad28-56173085b3bf

MacOS: Desktop

https://github.com/Expensify/App/assets/30054992/73dcd9fa-b4c1-4e67-8ce5-4f1bf5cbf80f

Verified the last bug. It seems to work well!
https://github.com/Expensify/App/assets/30054992/d49f607b-9f52-4031-b7b3-9d3de17f430b

@thienlnam All yours!

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to production by https://github.com/marcaaron in version: 1.4.71-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Improve GitHub Actions for pull requests

See https://github.com/Senzing/community-roadmap/issues/71

Pull requests:

https://github.com/Senzing/template-docker/pull/80


Grouping nodes by attribute

I am working on creating org structure which group users by department and team. Does this library support this scenario? Or has the possibility to extend to this? Happy to contribute if you see this is feasible.

Hi, I think yes, you can achieve this by treating groups as hierarchy,  and org chart nodes as groups. You have the possibility to define custom width and height for each node as well as generate custom content for each node, so in theory you have everything to be able to do t


"URI is not registered" for sap.tnt, sap.f, and sap.ui.layout when defining a namespace in the XML-template

OpenUI5 version: 1.98.0
Working with XML-templates, I've paid attention that not all libraries are recognized in XML-templates when defining a namespace. For instance, xmlns:tnt = "sap.tnt" and xmlns:f = "sap.f":
<mvc:View
	xmlns:mvc = "sap.ui.core.mvc"
	xmlns:tnt = "sap.tnt"
	xmlns:f = "sap.f"
	xmlns:core = "sap.ui.core"
	xmlns = "sap.m"
	controllerName = "com.MySandboxApp" />

raise an error «URI is not registered», the same happens to xmlns:l = "sap.ui.layout", while xmlns:mvc = "sap.ui.core.mvc" and xmlns:core = "sap.ui.core" are totally OK.
Is it an UI5 issue?

Searching for the message text, I found https://stackoverflow.com/questions/42569634/uri-is-not-registered-settings-languages-frameworks-schemas-and-dtds-in helps . Seems that some IDEs always try to fetch a schema from the given namespace URI. This does not work for UI5 namespaces as they are no real URIs, but only virtual ones. Nothing is hosted under that names. And, as you know, we  stopped providing XSD files as they always had issues (as discussed with you and others in e.g. #2751)
Means, if your IDE provides the options mentioned in https://stackoverflow.com/a/60734104 , then you should maybe chose "Ignore external resource".

Perfect, thanks!


Allow configuration of the number of informer workers

Currently the number of workers in the informer is hardcoded to 1. Being able to set this number higher will allow operator to process multiple CRD updates at once. This will help the operator be more responsive when it is managing a large number of CRD's.
Note: I think 1 is definitely the right default.

is there any reason why it has to be 1 at the moment? It seem that our operator is currently limited by this, because we manage want to mange many CRs at the same time. In our case we also create/delete them all the time, so there's really quite some work to do be done by the operator.

Number of Workers is now configurable!


feat(aide): impl OperationInput for axum-tungstenite

Use the  OperationInput impl of axum::extract::ws::WebSocketUpgrade for axum_tungstenite::WebSocketUpgrade as well.

Does your use case not allow NoApi to be used ?
You can extract any regular axum type with it if it has no api definition.
fn extract(NoApi(extracted): NoApi<TypeToExtract>)

In theory this would be a solution. But then there would be not API doc for a WebSocket endpoint in this particular case. So in practice I would love to see this change upstream ^^
What's your argument for not adding this?

We are having issues with conflicting versions, where some people need 0.5 and some 0.6 for sqlx for instance, so we had to pull the axum_sqlx_tx implementation out of the crate to allow for that. We are trying to keep dependencies to a minimum because of that, in favor of the new type pattern.
If you need the definition you can use the WithApi struct, although it might be better suited without requiring new type implementations.
Would something like UseApi(ws, ..): UseApi<tunstenite, axum::extract:ws> where the first parameter is extracted and the api taken from the second work for you ?

Or you can do a new type MyWs(pub tunstenite) and then impl deref, operation input, and FromRequest

Thanks for advice :)
I understand that you like to keep the dependencies minimal.
My brief opinion on the suggested solutions:

The new tpye approach: per project: an extra wrapper to maintain; all projects: wrapper needs to keep in sync with aide (in case of breaking changes)
The UseApi approach is a nice thought, but I encourage to not implement this. It's very easy that changes are made and the definition is not updated as someone forget a  UseApi.
IME WithApi usage causes the same "problems" as the  UseApi approach.

IMHO the impl of the  OperationInput within aide would be the cleanest solution. But if you don't like this, it is ok to me :)
Either I keep a fork or I switch to the newtype approach.

If you have a generic UseApi you could type alias to keep both types in sync.
Even if you do the impl in the crate it would not be more in sync than a type aliased UseApi. Usage wise it implements Deref and DerefMut so it is almost a drop in replacement.
Feel free to try the UseApi i pushed to master, with a type alias it should work well in your usecase.

Using the UseApi approach now. Thx :)


Crash ArangoDB v3.1 instance with export and cursor API

My environment running ArangoDB
I'm using the latest ArangoDB of the respective release series:

[ ] 2.8
[X] 3.0
[X] 3.1
[ ] self-compiled devel branch

On this operating system:

[X] Linux

[X] docker - official docker library

This is a http interface-related issue:
I'm using the http interface with this browser: curl    running on this OS: MacOS 10.10.5

[X] authentication is enabled? Yes and No. Issue persists with both.
[ ] using the cluster?
[X] _system database? Also works with different database.

» curl --version
curl 7.43.0 (x86_64-apple-darwin14.0) libcurl/7.43.0 SecureTransport zlib/1.2.5

These are the steps to reproduce:

start arangodb: docker run -p 8529:8529 --name arangodb -it -d -e ARANGO_NO_AUTH=1 arangodb:3.1 --log.level debug
Create collection: curl -X POST -d '{"name": "test"}' --dump - http://localhost:8529/_api/collection
Add some data:

curl -X POST -d '{"hello": "world", "counter": 1}' --dump - http://localhost:8529/_api/document/test
curl -X POST -d '{"hello": "world", "counter": 2}' --dump - http://localhost:8529/_api/document/test

Export the collection with a batchSize smaller than the actual collection size, i.e. a curser has to be created: curl -X POST -d '{"batchSize": 1}' --dump - http://localhost:8529/_api/export?collection=test
Query the next element: curl -X PUT --dump - http://localhost:8529/_api/cursor/<id> (using the id from the previous command)

The following problem occurs:
In 5) I get as a response: curl: (52) Empty reply from server and the arangodb instance is shutdown. Using ArangoDB 3.0 has no such issues. I stumbled over this, when using the collection.all function from python-arango driver and first filed an issue there. But this seems to be an ArangoDB issue.
Execution output (arango output at the end)
--- ~ » curl -X POST -d '{"name": "test"}' --dump - http://localhost:8529/_api/collection
HTTP/1.1 200 OK
Location: /_db/_system/_api/collection/test
Content-Type: application/json; charset=utf-8
Server: ArangoDB
Connection: Keep-Alive
Content-Length: 127

{"id":"130","name":"test","waitForSync":false,"isVolatile":false,"isSystem":false,"status":3,"type":2,"error":false,"code":200}%

--- ~ » curl -X POST -d '{"hello": "world", "counter": 1}' --dump - http://localhost:8529/_api/document/test
HTTP/1.1 202 Accepted
Location: /_db/_system/_api/document/test/154
Etag: "_ULfasHS---"
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 52

{"_id":"test/154","_key":"154","_rev":"_ULfasHS---"}%

--- ~ » curl -X POST -d '{"hello": "world", "counter": 2}' --dump - http://localhost:8529/_api/document/test
HTTP/1.1 202 Accepted
Location: /_db/_system/_api/document/test/176
Etag: "_ULfa0FK---"
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 52

{"_id":"test/176","_key":"176","_rev":"_ULfa0FK---"}%

--- ~ » curl -X POST -d '{"batchSize": 1}' --dump - http://localhost:8529/_api/export\?collection\=test
HTTP/1.1 201 Created
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 144

{"error":false,"code":201,"result":[{"_id":"test/154","_key":"154","_rev":"_ULfasHS---","counter":1,"hello":"world"}],"hasMore":true,"id":"189"}%

--- ~ » curl -X PUT --dump - http://localhost:8529/_api/cursor/189
curl: (52) Empty reply from server

The output from ArangoDB:
--- ~ » docker run -p 8529:8529 --name arangodb -it -d -e ARANGO_NO_AUTH=1 arangodb:3.1 --log.level debug
88be08a838f1a52ac01172437859e03e8a21daf938329829575f6bbbc70d2e0d
--- ~ » docker logs -f arangodb
Initializing database...Hang on...
Database initialized...Starting System...
{config} checking '///etc/arangodb3/arangod.conf'
{config} checking override '///etc/arangodb3/arangod.conf.local'
{config} loading '///etc/arangodb3/arangod.conf'
2016-12-07T08:17:56Z [1] INFO ArangoDB 3.1.3 [linux] 64bit, using VPack 0.1.30, ICU 54.1, V8 <IP_ADDRESS>, OpenSSL 1.0.1t  3 May 2016
2016-12-07T08:17:56Z [1] INFO using SSL options: SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2016-12-07T08:17:56Z [1] DEBUG file-descriptors (nofiles) hard limit is 1048576, soft limit is 1048576
2016-12-07T08:17:56Z [1] DEBUG permanently changing the gid to 999
2016-12-07T08:17:56Z [1] DEBUG permanently changing the uid to 999
2016-12-07T08:17:56Z [1] INFO file-descriptors (nofiles) hard limit is 1048576, soft limit is 1048576
2016-12-07T08:17:56Z [1] DEBUG using default language 'en_US'
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f0451000000 - 0x7f0453000000 (33554432 bytes), file-descriptor 6
2016-12-07T08:17:56Z [1] DEBUG found footer, reached end of datafile '/var/lib/arangodb3/journals/logfile-3.db', current size 10992
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f044f000000 - 0x7f0451000000 (33554432 bytes), file-descriptor 7
2016-12-07T08:17:56Z [1] DEBUG found footer, reached end of datafile '/var/lib/arangodb3/journals/logfile-4.db', current size 48
2016-12-07T08:17:56Z [1] DEBUG reading database parameters from file '/var/lib/arangodb3/databases/database-1/parameter.json'
2016-12-07T08:17:56Z [1] DEBUG database parameters: {"deleted":false,"id":"1","name":"_system"}
2016-12-07T08:17:56Z [1] DEBUG added document collection '_modules'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_frontend'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_aqlfunctions'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_users'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_jobs'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_graphs'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_statistics15'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_routing'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_statisticsRaw'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_apps'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_statistics'
2016-12-07T08:17:56Z [1] DEBUG added document collection '_queues'
2016-12-07T08:17:56Z [1] INFO JavaScript using startup '/usr/share/arangodb3/js', application '/var/lib/arangodb3-apps'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/collection'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/user'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/database/target-version'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/setup'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/teardown'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/install'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/uninstall'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/replace'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/upgrade'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/configure'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/configuration'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/set-dependencies'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/dependencies'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/development'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/tests'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/script'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics/short'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics/long'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/routing/reload'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/routing/routes'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/server/role'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/server/id'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/time'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/sleep'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_admin/echo'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_admin/long_echo'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics-description'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/test'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/execute'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/tasks'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/configuration/notifications/versions'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/explain'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/transaction'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/collection'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/user'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/database/target-version'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/setup'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/teardown'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/install'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/uninstall'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/replace'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/upgrade'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/configure'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/configuration'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/set-dependencies'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/dependencies'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/development'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/tests'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/foxx/script'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/traversal'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics/short'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics/long'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/routing/reload'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/routing/routes'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/server/role'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/server/id'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/time'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/sleep'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_admin/cluster-test'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_admin/echo'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/clusterStatistics'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_admin/long_echo'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/statistics-description'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/test'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/health'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/history'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/getSecondary'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/replaceSecondary'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/swapPrimaryAndSecondary'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/numberOfServers'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/cleanOutServer'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/moveShard'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/shardDistribution'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/rebalanceShards'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/supervisionState'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/database'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/endpoint'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/index'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  ''
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/auth/reload'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/aql/reload'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/execute'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/any'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/near'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/within'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/within-rectangle'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/fulltext'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/first-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/first'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/last'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/range'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/remove-by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/replace-by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/update-by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/tasks'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/aqlfunction'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/configuration/notifications/versions'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/explain'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/transaction'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/traversal'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_admin/cluster-test'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/clusterStatistics'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/health'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/history'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/getSecondary'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/replaceSecondary'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/swapPrimaryAndSecondary'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/numberOfServers'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/cleanOutServer'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/moveShard'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/shardDistribution'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/rebalanceShards'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/cluster/supervisionState'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/database'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/endpoint'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/index'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  ''
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/auth/reload'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT  '_admin/aql/reload'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/any'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/near'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/within'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/within-rectangle'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/fulltext'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/first-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/first'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/last'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/range'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/remove-by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/replace-by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/simple/update-by-example'
2016-12-07T08:17:56Z [1] DEBUG created JAVASCRIPT prefix  '_api/aqlfunction'
2016-12-07T08:17:56Z [1] DEBUG running database init/upgrade
2016-12-07T08:17:56Z [1] INFO In database '_system': Found 14 defined task(s), 1 task(s) to run
2016-12-07T08:17:56Z [1] INFO In database '_system': state standalone/existing, tasks updateUserModels
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f0446bf0000 - 0x7f0446ff0000 (4194304 bytes), file-descriptor 18
2016-12-07T08:17:56Z [1] DEBUG reached end of datafile '/var/lib/arangodb3/databases/database-1/collection-7-1373260109/journal-20.db' data, current size 328
2016-12-07T08:17:56Z [1] INFO In database '_system': existing cleanup successfully finished
2016-12-07T08:17:56Z [1] INFO using endpoint 'http+tcp://<IP_ADDRESS>:8529' for non-encrypted requests
2016-12-07T08:17:56Z [1] DEBUG bound to endpoint 'http+tcp://<IP_ADDRESS>:8529'
2016-12-07T08:17:56Z [1] DEBUG autoloading actions
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f0445fef000 - 0x7f04463ef000 (4194304 bytes), file-descriptor 23
2016-12-07T08:17:56Z [1] DEBUG reached end of datafile '/var/lib/arangodb3/databases/database-1/collection-60-1708716664/journal-71.db' data, current size 2952
2016-12-07T08:17:56Z [1] DEBUG autoloading actions finished
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f0443fef000 - 0x7f0445fef000 (33554432 bytes), file-descriptor 24
2016-12-07T08:17:56Z [1] DEBUG created datafile '/var/lib/arangodb3/journals/logfile-105.db' of size 33554432 and page-size 4096
2016-12-07T08:17:56Z [1] DEBUG autoloading actions
2016-12-07T08:17:56Z [1] DEBUG autoloading actions finished
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f045f802000 - 0x7f045f902000 (1048576 bytes), file-descriptor 26
2016-12-07T08:17:56Z [1] DEBUG reached end of datafile '/var/lib/arangodb3/databases/database-1/collection-56-2388228624/journal-88.db' data, current size 136
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f0441fef000 - 0x7f0443fef000 (33554432 bytes), file-descriptor 25
2016-12-07T08:17:56Z [1] DEBUG created datafile '/var/lib/arangodb3/journals/logfile-106.db' of size 33554432 and page-size 4096
2016-12-07T08:17:56Z [1] DEBUG connecting to ip endpoint 'http+ssl://www.arangodb.com:443'
2016-12-07T08:17:56Z [1] DEBUG {mmap} memory-mapped range 0x7f043ffef000 - 0x7f0441fef000 (33554432 bytes), file-descriptor 27
2016-12-07T08:17:56Z [1] DEBUG created datafile '/var/lib/arangodb3/journals/logfile-128.db' of size 33554432 and page-size 4096
2016-12-07T08:17:56Z [1] INFO ArangoDB (version 3.1.3 [linux]) is ready for business. Have fun!
2016-12-07T08:18:06Z [1] DEBUG Statistics: Installing regular tasks...
--- ~ »

I have reproduced the crash with 3.1. The problem is caused by the wrong API being called for the export cursor.
The first API call goes to POST /_api/export. This creates an export cursor.
The following API call goes to PUT /_api/cursor. This API is supposed to access a cursor created by /_api/cursor, not by /_api/export. This is also the reason for the crash.
Calling PUT /_api/export instead of PUT /_api/cursor for the second call will work properly and not cause a crash.
I have however added a check on the server-side so there will be no crashes anymore when the wrong API is called. That check will become available in 3.1.4.

👍