EDTF dates in BibLaTeX

New support for Extended Date/Time Format in Biblatex
Biblatex has recently added support for Extended Date/Time Format 1.0, conforming to levels 0 and 1. In brief, this includes, among other formats:

Added support for times. e.g. So one could identify a blog post as having (in Biblatex format) date={2016-07-18T20:26:06+10:00}, date=2016-07-18T20:26:06Z, or date={2016-07-18T20:26:06}
Negative date (occurring in the BCE/BC era). E.g. date={-0379}, which could output in biblatex, depending on the user chosen options, as "0380 BCE". Note the difference of 1 due to differences between astronomical V non-astronomical calendars.
Approximate dates. E.g. date={1567~}, which could output in biblatex, depending on the user chosen options, to "ca. 1567"
Uncertain dates. E.g. date={1567?}, which could output in biblatex, depending on the user chosen options, to "1567?"

For all the details, in one's latex install see:

The documentation biblatex.pdf p37, Database Guide > Usage Notes > Date and Time Specifications; and
latex/biblatex/examples/96-dates.tex ... for examples.

This is available in the production version of biblatex, 3.6. However, there are few bugs there so it might be easier to hold off until 3.7 is released before having a thorough look around.
I note also "PSA: Better BibTeX does not currently support Zotero 5.0 #555" which means you are in a state of flux ... and probably not wanting to write new code until Zotero 5.0 issues stablise.
Also perhaps it will be a rather large effort to verify zotero-better-bibtex properly handles exporting all the EDTF 1.0 date/time formats that biblatex now supports. It might be easier for you if I, and others, flag the relevant export problems one at a time.
That is, even if you want to delay attending to specific zotero-better-bibtex issues, in virtue of doing a major rewrite first, it might be better to feed you these issues one at a time. (Indeed you might prefer to attend to outstanding zotero-better-bibtex issues as you do your major rewrite).
Support times in the Zotero date field.
Consequently, the first problem I'm encountering is ...
Exporting a Zotero entry with a Zotero date in various extended formats (e.g. 2016-07-18T20:26:06+10:00, or 2016-07-18T20:26:06) produces unpredictable results. This is using "Better Biblatex" (not tested in "Better Bibtex").
2016-07-18T20:26:06+10:00
From ...
% Zotero
% Date: 2016-07-18T20:26:06+10:00

I get ...
% Biblatex entry via "Better Biblatex"
@article{barker_2016_turkey,
  author = {Barker, Anne},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  ...
  year = {2016-07-18T20:26:06+10:00},

I expect ...
@article{barker_2016_turkey,
  author = {Barker, Anne},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  ...
  date= {2016-07-18T20:26:06+10:00},

That is, the problem is exporting a date with time and timezone results in the date going to year rather than date.
2016-07-18T20:26:06
From ...
% Zotero
% Date: 2016-07-18T20:26:06

I get ...
@article{barker_2016_turkey,
  author = {Barker, Anne},
  date = {2016-07-18},

I expect ...
@article{barker_2016_turkey,
  author = {Barker, Anne},
  date = {2016-07-18T20:26:06},

That is,  the problem is exporting a date with time and no timezone results in the date going to date, as expected, but the datetime value has the time portion lost.
2016-07-18T20:26:06Z
From  ...
% Zotero
% Date: 2016-07-18T20:26:06Z

I get ...
@article{barker_2016_turkey,
  author = {Barker, Anne},
  date = {2016-07-18},

I expect
  author = {Barker, Anne},
  date = {2016-07-18T20:26:06Z},

That is,  the problem is  exporting a date with time and "Z" (for Zulu) timezone results in the time portion being lost.
I've sent the full entry to you as ReportID: IRFDJ8NT.

this one was fairly easy, for that reference I now get
@article{barker_2016_turkey,
  abstract = {Religious fundamentalism is a powerful force in Turkey where the military has a long history of intervening in politics to ensure the nation remains secular.},
  author = {Barker, Anne},
  date = {2016-07-18T20:26:06},
  journaltitle = {ABC News},
  langid = {australian},
  note = {Actual: 2016-07-18T20:26:06+10:00},
  rights = {http://www.abc.net.au/conditions.htm\#UseOfContent},
  timestamp = {2015-02-24 12:14:36 +0100},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  url = {http://www.abc.net.au/news/2016-07-18/turkey-coup-attempt-shows-division-over-wish-for-islamist-rule/7639292},
  urldate = {2016-07-24}
}

Thanks! I'm glad this one was easy.
However, I'm not seeing the fix at my end. I get the same faulty results for the same inputs.
I suspect either my install is not properly recognizing the updated zotero-better-bibtex plugin or I'm misunderstanding something about updating it.
To update I:

Downloaded  zotero-better-bibtex-1.6.78.xpi https://github.com/retorquere/zotero-better-bibtex/releases/latest
In Zotero > Tools > Addins > Cog icon > Install Add-on from File ... [Selected  zotero-better-bibtex-1.6.78.xp]
Verified that the "Last Updated" Value is "Tuesday, 25 October 2016".

Perhaps you could increment the version number at your end (e.g. to 1.6.79)?
Or perhaps I'm failing to properly recognize how GitHub works. Specifically perhaps I should read

retorquere released this 4 days ago · 2 commits to master since this release
( From https://github.com/retorquere/zotero-better-bibtex/releases/tag/1.6.78)

... as meaning ...

1.6.78 doesn't contain the subsequent commits

? In which case I have to wait for your next release?

You will either have to wait for the next release (which I hope will get done in a few days, I'm looking to include #585), or use https://github.com/retorquere/zotero-better-bibtex/releases/download/builds/zotero-better-bibtex-1.6.78-circle-3162.xpi in the interim.

zotero-better-bibtex-1.6.78-circle-3162.xpi fails to correct the errors. Given you have the hood open for #585 ... take all the time you need and I'll just wait for the next release. There's certainly no urgency from my end.

Not sure which errors you mean -- given the reference you submitted in IRFDJ8NT, I get
@article{barker_2016_turkey,
  abstract = {Religious fundamentalism is a powerful force in Turkey where the military has a long history of intervening in politics to ensure the nation remains secular.},
  author = {Barker, Anne},
  date = {2016-07-18T20:26:06},
  journaltitle = {ABC News},
  langid = {australian},
  note = {Actual: 2016-07-18T20:26:06+10:00},
  rights = {http://www.abc.net.au/conditions.htm\#UseOfContent},
  timestamp = {2015-02-24 12:14:36 +0100},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  url = {http://www.abc.net.au/news/2016-07-18/turkey-coup-attempt-shows-division-over-wish-for-islamist-rule/7639292},
  urldate = {2016-07-24}
}

If there are more errors, references that yield these errors would be appreciated.

I'd actually rather bundle multiple feature requests/bug fixes in a single release rather than releasing a new version for each separately.

Not sure which errors you mean

Each of the three errors I mentioned remain. That is, using the entry which you've quoted back to me and trying each of following values, in turn, in Zotero Date:

2016-07-18T20:26:06+10:00
2016-07-18T20:26:06
2016-07-18T20:26:06Z

So, sticking with example you've quoted back to me (Input date 2.), when I use Zotero Date: 2016-07-18T20:26:06 I get date = {2016-07-18}, as originally detailed. This is via Better Biblatex Quick Copy (Ctrl + Shift + C).
Given, however, you see the change at your end, and I don't at my end, I'm suspecting:

Some kind of install issue at my end. However,
In Zotero > Tools > Add-ons > Extensions ... I see "Zotero Better Bib(La)Tex 1.6.78-circle-3162" (when I click on "more" to expand the entry for BBT);
And I've tested this on a separate machine which has never before had Zotero Better Bib(La)Tex installed.

or

The changes you've made didn't compile into 1.6.78-circle-3162; or

The issues you are experiencing with #585 (I've only skimmed that thread) are repeating themselves here: some kind of platform funkiness. You wrote in #585 ...

but it'd be super nice if this wasn't in some way a platform dependency. I don't have a windows system at hand, and debugging on windows is a major PITA in any case.

... I'm on Windows 10.
or

Some other bizarre thing.

I'm using Zotero 4.0, not Zotero 5.0.
Perhaps a third party could be kind enough to tell us what they see at their end (and what OS they are using)?

I'd actually rather bundle multiple feature requests/bug fixes in a single release rather than releasing a new version for each separately.

Yes indeed: that'd be the way to do it. Maybe you'd rather close #585 first (among other issues), and that'll automatically help here. But if there is any test I can perform to help, let me know.

Ah, yeah, sorry, you need to go into about:config and set biblatexExtendedDateFormat to true to enable the new behavior. Since the support for this in BibLaTeX is so recent I can't make it default to enabled. I'll probably add a preference for this in the preferences pane, but I'm still contemplating whether I should merge it into a single preference with #566 (BBT already has too many preferences).
For those three dates (with the setting on) I get
@article{barker1_2016_turkey,
  abstract = {Religious fundamentalism is a powerful force in Turkey where the military has a long history of intervening in politics to ensure the nation remains secular.},
  author = {Barker1, Anne},
  date = {2016-07-18T20:26:06},
  journaltitle = {ABC News},
  langid = {australian},
  note = {Actual: 2016-07-18T20:26:06+10:00},
  rights = {http://www.abc.net.au/conditions.htm\#UseOfContent},
  timestamp = {2015-02-24 12:14:36 +0100},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  url = {http://www.abc.net.au/news/2016-07-18/turkey-coup-attempt-shows-division-over-wish-for-islamist-rule/7639292},
  urldate = {2016-07-24}
}

@article{barker2_2016_turkey,
  abstract = {Religious fundamentalism is a powerful force in Turkey where the military has a long history of intervening in politics to ensure the nation remains secular.},
  author = {Barker2, Anne},
  date = {2016-07-18T20:26:06+10:00},
  journaltitle = {ABC News},
  langid = {australian},
  note = {Actual: 2016-07-18T20:26:06+10:00},
  rights = {http://www.abc.net.au/conditions.htm\#UseOfContent},
  timestamp = {2015-02-24 12:14:36 +0100},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  url = {http://www.abc.net.au/news/2016-07-18/turkey-coup-attempt-shows-division-over-wish-for-islamist-rule/7639292},
  urldate = {2016-07-24}
}

@article{barker3_2016_turkey,
  abstract = {Religious fundamentalism is a powerful force in Turkey where the military has a long history of intervening in politics to ensure the nation remains secular.},
  author = {Barker3, Anne},
  date = {2016-07-18T20:26:06Z},
  journaltitle = {ABC News},
  langid = {australian},
  note = {Actual: 2016-07-18T20:26:06+10:00},
  rights = {http://www.abc.net.au/conditions.htm\#UseOfContent},
  timestamp = {2015-02-24 12:14:36 +0100},
  title = {Turkey Divided between Secular and {{Islamist}} Rule},
  url = {http://www.abc.net.au/news/2016-07-18/turkey-coup-attempt-shows-division-over-wish-for-islamist-rule/7639292},
  urldate = {2016-07-24}
}

#585 just cleared so I'm prepping for release, running tests once more on each of the 3 branches (#585, #587 and #590) slated for this release before I merge them. It'd be good to have confirmation that the output above is what you want, or whether there are still test cases you'd want to have added.

That's solved, glad it was you not me :)  (But I'm guilty of something else as I'll explain below).
Yes the three test cases now work at my end.
I ensured both of the following where true ...
extensions.zotero.translators.better-bibtex.biblatexExtendedDateFormat
services.sync.prefs.sync.extensions.zotero.translators.better-bibtex.biblatexExtendedDateFormat

If you are keen to have a crack at the other EDTF formats then: the EDTF 1.0 (level 0 and level 1) formats that biblatex supports include the following (I've essentially taken a representative sample from  biblatex.pdf p37, Database Guide > Usage Notes > Date and Time Specifications):

Negative date: -0876
Uncertain: 1723~
Approximate: 1723?
Uncertain and approximate: 1723?~ (Note the other way around "1723~?" happens to be illegal under EDTF 1.0).
Date range: 1988/1992
Season: 2004-22 (The "month" position should allow 21 to 24 for Spring, Summer, Autumn, Winter respectively).
Unspecifieds

199u
19uu
1999-uu
1999-01-uu
1999-uu-uu

Other notes:

The above is a mere representation. Those marks [?, ~, /, etc] can be combined with full dates and datetimes, e.g. For "1995-03-30/1995-04-05". And they can be combined with each other, E.g. "1934~/1936~"; or "-0379~".
A year must be four digits. E.g. "0343-02-03" is legal, "343-02-03" is not.

You'd just need to pass these through without processing.
For example, from ...
% Zotero
Date: -0379

.... to ...
 % Biblatex
 date={-0379}

I notice that, at the moment, you decrement the date to arrive at ...
 % Biblatex
 date={-0380}

... which is entirely my fault because of my misleading  prior remark ...

which could output in biblatex, depending on the user chosen options, as "0380 BCE". Note the difference of 1 due to differences between astronomical V non-astronomical calendars.

... these decrements, and other processing, will get applied by the biblatex system from raw *.bib entry fields, on their way to becoming a pdf. So we want ...
% Zotero
date={-379~}

% Biblatex *.bib file
@book{plato_2004_republic,
  author = {{Plato}},
  date = {-379~},
  title = {Republic},
  .....
}

% Biblatex (biber) processing results in pdf outputs
Plato (ca. 0380 BCE). Republic. Trans. by C. D. C. Reeve. 3rd edition. ...

For testing you could just use any entry (barker_2016_turkey would do) and just change the Zotero Date field every time.

  date = {-877},
  date = {1723~},
  date = {1723?},
  date = {1723~?},
  date = {1988/1992},
  date = {2004-22},
  year = {199u}
  year = {19uu}
  year = {1999-uu}
  year = {1999-01-uu}
  year = {1999-uu-uu}

"-877" is illegal, "-0877" is legal ... years must have four digits.
I suppose expecting BBT to begin to perform complex validation, which rejecting 3 digit years might become, is probably too much. That might be a bit like performing email address validation on websites.
But at a minimum you want to ensure "-0877" passes through BBT, and should therefore be your test case.

"1723~?" is exactly the wrong way around

Uncertain and approximate: 1723?~ (Note the other way around "1723~?" happens to be illegal under EDTF 1.0).

From Extended Date/Time Format (EDTF) 1.0 the relevant BNF is

UASymbol = ("?" | "~" | "?~")
... and I learnt of this only because Biblatex enforces it.

  date = {2016-07-18T20:26:06},
  date = {2016-07-18T20:26:06+10:00},
  date = {2016-07-18T20:26:06Z},
  date = {-0877},
  date = {1723~},
  date = {1723?},
  date = {1723?~},
  date = {1988/1992},
  date = {2004-22},
  year = {199u}
  year = {19uu}
  year = {1999-uu}
  year = {1999-01-uu}
  year = {1999-uu-uu}

The year decrement on negative dates was intentional. I'm trying to locate the reasoning behind it -- it's in my history somewhere. @nickbart1980, ring a bell?

Got it -- good spot, that negative-date hack was specific to CSL (#360) but it leaked into the BibTeX export. That date is now date = {-0876}

If date={-0379} and 0380 BCE are equivalent, I'm not adding an option for that. BBT has too many options as-is.

BTW this is the last issue to close before I cut a release.

(and email validation is actually not as hard as it's made to look - the rules on domain names are fairly rigid, and you basically should allow everything before the "@". Anything you forbid there is usually wrongly rejected)

In short, you can close the issue and cut your release.

If date={-0379} and 0380 BCE are equivalent, I'm not adding an option for that.

They are equivalent but you shouldn't allow, or at least not add an option for, "0380 BCE" in BBT for the reasons:

That it is illegal under EDTF 1.0;
And therefore it is illegal under Biblatex, as an input format.

... in addition to whatever other reasons you can point to.
The point of my example was for a related but separate purpose: to show that you shouldn't convert "-0379". So, yes, in addition to not converting "-0379" you don't need to allow for "0380 BCE".
All your test cases look good (and indeed it's a good idea to include the original three datetime formats).
If BBT passes those test cases I think that gets the lion's share of the EDTF formats that biblatex supports. That is, if you are agreeable I'm happy to consider the issue currently closed, and you can go ahead and cut your release, for the sake of "release early and often". That is, at a future point I probably will be able to identify some other test cases for a future release (but if you'd rather have me spend more time on the issue now, I can).
On email validation, to the limited extent I grasp the issue, I'm under the impression many webdevs attempt a rigorous validation of addresses, trying to implement the relevant RFC's, and frequently get their implementation wrong: wrongly rejecting valid addresses (as you allude to).
Even if I'm mistaken about that I think you get my points for our purpose here: the (strict) validation of EDTF date formats is complex; biblatex undertakes this validation; if you where to attempt that validation it would be a significant task; and, given, the significance of the task there's a liability to get a mismatch between what BBT accepts and what biblatex accepts.
The alternative, which I'm under the impression you've implemented, is to have a very permissive validation. So you, correct me if I'm wrong, essentially allow anything through that parses as a date once you allow for the exceptional characters I've brought to your attention (~,?,/,u etc), with some additional exceptions. That is, your validation is basic rather than strict, to avoiding wrongly rejecting something. If I've broadly understood your approach correctly, then I think that's the right thing for you to do.

If there's an EDTF pattern I don't yet pick up I'll gladly fix it, but my main worry is not about validating input but about recognizing it and transforming it to something that the Bib(La)TeX processor will understand.
I can't be in the validation business because I have no feedback channel. If I establish that your date is b0rked (if it were to say, e.g., that birthday when Jimmy got hammered and tried to kiss a chicken), I still have to do something with that (but in this case it's overridingly likely I'll just say "stuff it" ad put it in the year field like I would with forthcoming) because there's no way1 for me to tell you about it as the translators are sandboxed and can only write to the file you selected for export.
And EDTF dates are the easy case, because they're well-structured. I also have to deal with dates like September 20, 2006, 05/01/1992, 15 juin 2009, 8. juli 2011, Januar 1945, März 1, 2008, 四月 1, 1976... all of which are actual samples of dates that have found their way into my test cases.
Anyhoo, I've cut the release because it contained some urgent fixes, so we have time for this. If there's patterns I missed, do tell me about them.
What does the u mean? I currently treat the u dates you  gave me as invalid which means I'm stuffing in the year field rather than in the date field, as BibLaTeX seems to treat stuff in the year field more liberally and so allows for things like forthcoming to be in the Zotero date field (although that would be more cleanly achieved with status: forthcoming in the extra field, but if there's something in the date field I don't just want to ignore it).
1 there actually is a way but the UI would be clunky and everything about the Zotero internals for translators screams "you should most definitely not be doing this"

You are right that there are RFCs that cover email addresses, but a) they're more complex than most webdevs are willing to understand, and b) not every smtp server is strict about enforcing those rules and might accept more than the RFCs would permit -- in which case the user would have an out-of-spec but wholly functional email address. Either you pick a well-established library to do such validation (but many seem to think "simple regex, done"), or you need to be liberal about such matters.

I'm going to find a lib to parse EDTF as a special case. Too many edge cases.

Dates are gnarly things and EDTF does well to accommodate the gnarliness, yet be disciplined about it when doing so.
Yes finding a library for EDTF parsing could well ease your implementation burden. There's a few at https://www.loc.gov/standards/datetime/implementations.html , notably there's two javascript libraries. I haven't evaluated these.

If there's an EDTF pattern I don't yet pick up I'll gladly fix it,

Add these to the list ...
% 'unknown' and 'open' are string literals
unknown/2006
2004-06-01/unknown
2004-01-01/open

% Illegal under EDTF but biblatex allows
% it  as a special exception for reasons I
% can't recall (backward compatibility I
% think).
1997/

% Years exceeding four digits
% Not yet implemented by biblatex but it is part of EDTF 1.0 Level 1
% And I'm in the process of persuading the biblatex dev to implement it
% for the sake of completeness.
y170000002 % the year 170000002
y-170000002 % the year -170000002

Looking at the EDTF level 0 and level 1 table of examples and biblatex.pdf I think that exhausts all the types. After I get my hands on your implementation I'll, of course, attempt to break it with edge cases.

 [Out of order] ... I currently treat the u dates you gave me as invalid which means I'm stuffing in the year field rather than in the date field,

Yeah we want to just pass these through, unaltered, to date (I'm assuming you mean to flag what you are currently doing as an erroneous artifact of current code).

my main worry is not about validating input but about recognizing it and transforming it to something that the Bib(La)TeX processor will understand.

That helps me see two things.

That we agree that you shouldn't be too worried about (strict) validation of input and the main thing is just ensuring something gets through that biblatex understands; and
I'd failed to recall that Zotero date fields are very permissive and you therefore have to handle exotic date formats, including the colloquial date formats you mention 8. juli 2011, etc. That is, it's not just a matter of you passing through something basically biblatex valid ... but a matter of you sometimes performing transformations (e.g. From 8. juli 2011 to 2011-07-08).

On this second point is there are a Zotero 5.0 issue (I can't yet install it), or should there be one? That is, does Zotero 5.0 treat date fields differently from Zotero 4.0? Should Zotero 5.0 treat date fields differently by, for example, by allowing users to input any format but having Zotero convert the date into EDTF internally (if so this entail us lobbying the Zotero folks) - something which could ease the burden on the BBT side?

I can't be in the validation business because I have no feedback channel. If I establish that your date is b0rked (if it were to say, e.g., that birthday when Jimmy got hammered and tried to kiss a chicken), I still have to do something with that (but in this case it's overridingly likely I'll just say "stuff it" ad put it in the year field like I would with forthcoming) because there's no way1 for me to tell you about it as the translators are sandboxed and can only write to the file you selected for export.

Incidentally why wouldn't you put it in the date field rather than year field? And how about, whatever your target field, you write an error message to the field? ...
% Zotero
Date: that birthday when Jimmy got hammered and tried to kiss a chicken

% ... passing through whatever BBT translator
% (like Better Biblatex) to end up with something like
date={
      Mate, that's b0rked: I don't even know Jimmy.
      Source: 'that birthday when Jimmy got hammered and tried to kiss a chicken'
 }

... ?

why wouldn't you put it in the date field rather than year field?

biblatex accepts only EDTF strings in the date field, but you can put pretty much anything into the year field, something that’s useful, e.g., for free-form dates such as “13th cent.” – and this should certainly not trigger an error message.

I've been toying with those two libraries, but the results are not encouraging. edtfy doesn't recognize everything it should (false negatives) but also claims to recognize things like 2004-22 as the range 2004/0022 (sort of false positive). edtf.js is better but implements the working draft of the next version of EDTF but doesn't parse EDTF L0/1, and the workarounds I have to do to get it to work are stacking up.
BBT is far from 5.0-compatible, but I haven't seen anything in Zotero (or citeproc-js, which they use) that deals with dates better than BBT does. BBT uses the citeproc-js date parser as a fallback when it doesn't recognize the date, and already adds metadata to the dateparser that Zotero itself doesn't. The dateparser in BBT recognizes and normalizes a lot of things that Zotero/citeproc don't (yet). If Zotero would get better at date parsing I would not at all mind, but if I just use the Zotero date parser as it is a good chunk of my tests would fail.
I can't actually know that what's in the date field is or isn't meaningful; there's 13th cent. as @nickbart1980 says, but that's still in a language I could understand. 13 asr means the same thing if you happen to know the language, so I stuff that in the year field and BibLaTeX seems happy to generate good looking references from that.

Thanks @nickbart1980 for that. That draws my attention to the following in biblatex.pdf, p 37

year and month are not parsed for date meta-information markers or
times and are used as-is

Then from both (@nickbart1980 and @retorquere) of your explanations I think I see how BBT must work (and currently does work, mostly):

BBT fetches the value from Zotero Date field.
BBT then attempts to parse it as a date allowing for all sorts of coded exceptions (such as the EDTF extensions I've been pointing toward).
If BBT can't recognize it as a date, even after taking into account the exceptions, then it still could be that the source string is meaningful to the user. So ...

If BBT has recognized it as a date then, in Better Biblatex's case, it can (or, rather, this is what we want) write to the biblatex date filed as a EDTF date;
If BBT hasn't recognized it as a date then, in Better Biblatex's case, it must pass on the string unaltered because it could nevertheless be a meaningful date to the user. And it will pass it on to the biblatex year field due to this being a free form field from Biblatex's point of view. Under no circumstances should an error message be generated because that birthday when Jimmy got hammered and tried to kiss a chicken could well be meaningful to the user ... and they might want that string to end up in the bibliography as is. This illustrates why you (Emiliano)  "can't be in the validation business".

I've been toying with those two libraries, but the results are not encouraging. ...

Those EDTF libraries, then, sound unusable. Are you able, then, to carry on with your current approach? Which I presume is something like: Break ranged dates into two; Strip out EDTF characters "u, ?, ~, open, unknown, etc"; see if the date(s) parse as ISO8601 dates; if they do then write the original string to biblatex's date field (and let biblatex catch any errors that you can't).

By the way @retorquere, you wrote:

If date={-0379} and 0380 BCE are equivalent, I'm not adding an option for that.

... and you were just anticipating a possible way to do things rather than responding to a specific request from me for this.
I then mentioned you don't need to bother with that second date format.
Well, you have @nickbart1980 to thank for this. During early explorations of extending Biblatex to cope with a wider variety of date formats it was @nickbart1980 that both: suggested using EDTF as the input format; and only that input format. At one point I was indeed arguing for something like date={-0379} and date={0380 BCE}. Fortunately, he had the foresight to see that this would create difficulties for guys like you.

It's not so much that it's a problem for me, it's just that I've gotten a lot more conservative about adding options that exist only to generate "prettier" Bib(La)TeX rather than addressing functional cases. If there are Bib(La)TeX parsers out there that require 0380 BCE and other that require -0380, I'd consider it. But if it's only because 0380 BCE is prettier than -0380 (which it is), it would only add to the already baroque BBT preferences screen for little gain; after all, the aim is for BBT to generate Bib(La)TeX you don't need to look at because it just does The Right Thing (tm)(c)(r)(p)(aota).

If I pass a date to edtf.js this way:
date.replace(/unknown/g, '*').replace(/open/g, '').replace(/u/g, 'X').replace(/\?~/g, '%')

edtf parses everything I can throw at it. I still have a nagging feeling this might yield false positives, but I can't really think of any.

@JohnLukeBentley BBT does output comments in the Bib(La)TeX if you tell it to BTW; if you enable  qualityReport in about:config it'll get sassy about the quality of your BibTeX. But not in a way that will cause errors while compiling your document, and it's still very experimental while I figure out rules for what is and is not considered good for a reference. Plus you'd have to open the actual Bib(La)TeX file as there's no (proper) way for a translator to interact with the UI.
In the meantime, all tests run green using edtf.js, which means no regression in existing test cases. It does mean FF <41 is no longer supported, but I can live with that. Even Debian has ESR45 (which is what I use in my tests).

The current setup is then: if you have the hidden pref on, and you're exporting for BBLT, and a date (after mangling open, unknown, etc) is parsed by edtf.js without error, the unmangled date is simply passed into the date field. In any other case, the regular homebrew + Zotero date parsing applies.

Yes, that algorithm now makes sense to me, and sounds good.
From https://github.com/inukshuk/edtf.js I see that:

"X" is (strangely) used in place of "u" for unspecifieds. So your replace(/u/g, 'X') looks good.

However,

* makes more sense to represent an open date  ...
Also, when an interval is blank, it suggessts "incomplete" or "unknown".

So you appear to have  ...
date.replace(/unknown/g, '*').replace(/open/g, '')

... the wrong way around. It should be ...
date.replace(/unknown/g, '').replace(/open/g, '*')

edtf.js seems to be able to handle "?" and "~" without substitution. So perhaps you don't need replace(/\?~/g, '%'). Perhaps you need to specify the level (level: 1 in our case) when you perform the parse, to avoid errors you might be seeing ...

By passing level or types constraints to the parser, you can ensure EDTF.js will accept only dates supported by your application.

edtf.parse('2016?', { level: 0 }) #-> parse error
edtf.parse('2016?', { level: 1 }) #-> ok

"X" is (strangely) used in place of "u" for unspecifieds. So your replace(/u/g, 'X') looks good.

This is apparently a change in EDTF in the working draft of the new spec. edtf.js intends to parse the state of the working draft rather than the current spec ("EDTF.js fully implements EDTF levels 0, 1, and 2 as specified by WD 2016-02-16 of ISO 8601-2").

the wrong way around. It should be date.replace(/unknown/g, '').replace(/open/g, '*')

right, done.

edtf.js seems to be able to handle "?" and "~" without substitution. So perhaps you don't need replace(/\?~/g, '%').

It does handle ? and ~, but not ?~, which is replaced by % in the working draft.

Perhaps you need to specify the level (level: 1 in our case) when you perform the parse, to avoid errors you might be seeing ...

Nope, that just sets L0/1 of the working draft spec, not of the current spec. The author has said that significant changes to edtf.js would be required to support the current spec.

CircleCI is on the blink again, so my tests currently don't run, but the replace swap shouldn't affect them. When a date parses as EDTF, I simply pass whatever was in the date field to BibLaTeX. The EDTF parser reconstruction uses the working-draft notation so it'd get the X and % etc. I only use it as a validator.

Tests run green.

Thanks!
Good to know there are good reasons for the EDTF.js dev having those apparently strange characters. I suppose ISO 8601-2 might have more clout than EDTF (and EDTF is meant to track the ISO work anyway).
So it looks likes all your replaces are now solid. What's your ETA on the next BBT release (so I can test)?

Biblatex now supports years with more the four digits, according to the EDTF spec (at level 1).
... and, note, when a year more than four digits is permitted then only the year part is permitted, not other date components (month, day, etc) (this is under EDTF, I haven't found access to ISO 8601-2).
% Legal
y170000002
y-170000002

% Illegal
y170000002-03-22
y-170000002-03-22

This is based on my looking at the EDTF BNF and the Online EDTF Validator.

I reckon it'll be a few more days as I do some cleanup and add test cases, but there's a pre-release at https://github.com/retorquere/zotero-better-bibtex/releases/download/builds/zotero-better-bibtex-1.6.80-circle-3210.xpi which passes all my tests. The y170000002 isn't in there yet, I'm adding that to the cleanup branch (590b) I'm working on.

In Zotero I get a Message box error when exporting (Quick Copy) "An error occurred while trying to export the selected file".
In Zotero preferences on the Better Bibtex tab "Better Bibtex has been disabled"; "lib/edtf.js load failed: SyntaxError: class is a reserved identifier".

What version of Firefox are you on?

Wait, you're on standalone, correct? Damn, that means standalone is built on a rather old Firefox version.

What is a sensible interpretation of [2007] as a date? I've so far interpreted [<date>] date as origdate/date, but that doesn't really fly with [2007]. Is that origdate=2007, date=unknown or somesuch?

Yes that test was in the Standalone.
I also have the firefox extension. I'm on the latest firefox (49.0.2)
When I'm using the extension I get the following failures (I essentially run the same test cases you've previously listed)

2016-07-18T20:26:06+10:00 -> year = {2016-07-18T20:26:06+10:00} % Goes to year, not date.
2016-07-18T20:26:06 -> date = {2016-07-18}
-0877 -> date = {-878} % No leading zero
1723~ -> date = {1723}
1723? -> date = {1723}
1723?~ ->  date = {1723}
2004-06-01/unknown -> date = {2004-06-01/2004-06-01}
2004-01-01/open -> date = {2004-01-01/2004-01-01}

... and, as you warned,

y12345 -> year = {y12345}
y-12345 -> year = {y-12345}

Would it be better to port BBT to Zotero 5.0 first, understanding that this is going to take some time, then revisit this issue?

The two don't interfere, date parsing doesn't touch the parts that require rewriting for 5.0. AAMOF much of my current work exists of splitting off parts that are not affected by the async stuff -- one of which is the date parser.
I think I'm nearly there, there are some test cases that fail now that the dateparser has been cleaned up because I used to stuff things in the year field that now parse cleanly.

Oh that's just effing swell. Just got confirmation that zotero standalone is based in stone-age versions of Firefox and won't be getting updates for 4.0, so the date parsing is not going to work for standalone. And in a few weeks, standalone is going to be all there is.
Such fun.

Yeah, so my suggestion to wait for Zotero 5.0 was premised on the assumption that 5.0 would be based on a newer version of Firefox (or some other platform altogether?) that wouldn't necessitate whatever workarounds you were attempting.
Can I take it you now do intend to wait for Zotero 5.0? Whether you intend to wait or not, are you presently sure Zotero 5.0 won't necessitate the same workarounds (What platform is Zotero 5.0 based on?)?
Is the issue that edtf.js relies on a newer versions of JavaScript (6 or 7?) that is only supported in the newer versions of firefox?

I have no choice but to wait. 5.0 is going to be based on a FF version (45esr) that can run the es6 EDTF.js is written in unchanged.

Waiting seems, then, the thing to do. Good to have confirmed that FF in Zotero 5.0 will be around version 45.
Incidentally the current issue speaks in favour, as a general rule, of continuous integration and updating of one's dependent platforms and libraries. It might be worth suggesting to the Zotero team they architect it so that they can regularly update their FF base. The first one to make the suggestion wins ;)

They have CI, but don't have dependencies that require upgrading. The standalone versions have been a bit of a pain for them in the past, so I can sort of understand why they're not keen to upgrade when they found a version that works.

And in any case, Dan (from Zotero) was very clear that no upgrades will be happening for the 4.0 line, so suggesting they do automatic upgrades isn't going to accomplish anything that helps me. 5.0 will have a version that supports EDTF.

And in any case, Dan (from Zotero) was very clear that no upgrades will be happening for the 4.0 line, so suggesting they do automatic upgrades isn't going to accomplish anything that helps me.

I intended my suggestion with respect to 5.0, as something that might prevent similar problems in the future. I mean that version is so close to release that, indeed, it wouldn't make sense for them to make any significant change to their old 4.0 version.
But, with thanks, you also provide some explanation about why they might not be especially interested in keeping their FF base up to date even in Zotero 5.0.
In any case, as you mention, it looks like Zotero 5.0, based on FF 45, will largely supports EcmaScript 6, http://kangax.github.io/compat-table/es6/#firefox45 . With luck edtf.js doesn't depend on those EcmaScript 6 features not supported by FF 45 - I'd be surprised if it was a problem.

EDTF runs without issue in FF45. I'm on 45 ESR myself, and it was only when you said you had problems in standalone that I found out Standalone was built on such old versions.

When I use the Zotero Firefox extension (using FF 49.0.2) I get the errors I previously detailed. Perhaps with the FF extension and the standalone installed, the standalone Firefox base is used.
Indulge me being explicit: whatever is going on there I agree doing your port to Zotero 5.0 first, is the way to go.

BBT doesn't tolerate being active in both FF and ZSA at the same time especially well. I've tried to make progress there, but never really got it fully right, and Zotero for FF is going the way of the dodo anyway somewhere in Januari by last reports. That said, if you don't have both running, and you still see the errors on FF49, that will be a problem down the line.
Work on 5.0 is slow going as so much needs to be changed. If only to keep me motivated to work on BBT (because the 5.0 work is deeply depressing), I still work on 4.0 features if I don't expect they will add to the workload of porting to 5.0. Right now I'm still just working to pare down the things BBT uses that are affected by 5.0 (like direct DB access) so that I minimize work to be done in the port while I still have a working environment I can run through my test suite.

That said, if you don't have both running, and you still see the errors on FF49, that will be a problem down the line.

I get those errors with only the extension running.
Even if you think this will be a problem down the line, I'd suggest you wait for that line to be crossed before dealing with it. Not that you have much choice here.
I used to be reasonably good at EcmaScript version 3.0 (and still have robust references for this version). So after Zotero 5.0 release/BBT port you'll have me as a resource if, as a last resort, part of the solution might be to backport parts of EDTF.js.

Work on 5.0 is slow going as so much needs to be changed.  ....  the 5.0 work is deeply depressing

Yes, that's unsurprising on both counts. Hopefully, though, you might come across new aspects of their architecture that makes life easier for you.

Well the changes make full sense given where Javascript as a language is headed. I can't blame Zotero for not wanting to be stuck in the past, and for the Electron port which is certain to happen1 some of this would have happened anyway. Plus there are projected performance benefits in the new architecture2. It's mainly that

I deeply dislike the choices JS has made to deal with their single-thread architecture decision that led us to this place, and
all the work that needs to be done is only to accommodate this choice, there is no functional benefit, and there were alternative solutions 2, so from my pov the BBT port is just grunt work. A lot of it. I'm reasonably familiar with the insides of Zotero (although a little less so with 5.0), but I haven't come across anything in 5.0 that struck me as something that would make my life easier or more interesting (except perhaps in the apocryphal Chinese sense), so the only reward that lies on the other side of the port is that I don't have to do the port anymore. Well that and the continuance of BBT.

It's very subtle, but you may have glimpsed it's not a topic I enjoy dwelling on. I know this has to be done, so I'm making baby steps and many restarts while I figure out how to do this. But this, plus the disappearance of the FF extension soon (which means I may have to re-do much of my testing infrastructure) and the Electron port a year later (which is going to be much of this misery all over again) has sucked all the joy out of doing BBT, to be honest.
Given the state of things with reference managers, this port needs to be done. My hope is that once the port is done (and then the test suite in January, and then the Electron port...), I can get back to doing the fun stuff. Zotero is still the best game in town given that (IMO) no other extensible reference manager stacks up against it3, and the BibTeX generated by non-native-BibTeX managers is universally unusable crap, even (or perhaps especially) from the high-cost commercial ones. And JabRef4 has usability problems if you have loads of references for which you need a subset for each individual paper.

1 which is a good thing because Mozilla seems hell-bent on removing all of the  Firefox-unique features that made it such a good choice for Zotero.
2 although I would argue that even greater performance benefits are available in abandoning SQLite as a (primary) storage engine, which would coincidentally also do away with much or even all of the async changes introduced by 5.0. I have ditched SQLite in BBT a while ago and the performance difference is staggering. AAMOF one of the caches BBT keeps in its own database is to prevent Zotero from unnecessary re-serializing unchanged references from the SQLite DB before they are handed to the export translators -- according to my highly unscientific tests, about 80% of the time an export took was spent serializing objects, not generating BibTeX from them. This may actually have improved in 5.0, but that's because more of the references are kept in memory so serialization is done from memory to memory rather than from DB to memory -- going async had nothing to do with that improvement.
3 none of the open source reference managers come close in usability, and none of the closed source ones are extensible
4 AFAIC the gold standard where it comes to BibTeX reference management

I was unaware of the Zotero plan to port to Electron after 5.0. Good to know, at least in terms of how it is impacting you (two ports rather than one). Commiserations on that.
Yeah, in my survey of reference managers Zotero came out on top, with JabRef close behind. And your BBT sealed the deal. I too couldn't see anything else that touches it.
And the three core pieces of functionality that you've given us (and me in particular) through BBT - custom field handling; custom citation keys; and ordering the fields within an entry - bring me very close to the golden path between reference management and having those references end up in a paper.
It is a marvelous thing you've created, on top of Zotero. So I certainly am grateful for your intentions to keep BBT alive.
What do you use in place of SQLite? I'm part way through my Android Dev learning and SQLite is the promoted database, for local storage.

Oh SQLite is perfect to use, I didn't mean to imply there's anything wrong with it. There's many projects where I use SQLite. But there's two things that drag down performance in Zotero when using SQLite:

The ORM impedance mismatch between the SQL tables and the JS objects that Zotero deals with internally means back and forth translation, exacerbated by the fact that
SQLite is written in C, which means that everything that goes back and forth between SQLite and JS goes through a thunking layer, which has huge overhead. When I ported the indexer text-splitter from C in JS, which did a lot of thunking calls, it went from didn't-finish-in-15-minutes to 3.5 seconds for one of my test files.

As long as you don't do many little accesses but rather fewer bigger requests, SQLite can do just fine, even in JavaScript.
5.0 fixes the performance problems that were caused by these two issues in two ways: it caches more, and when it does access the DB it does so async, which doesn't make it any faster, but at least it won't freeze firefox while it's doing it.
Previously I would fetch the citation key from an SQLite database whenever I needed it (such as during export), and it was horrendously slow. Totally unusable.  BBT uses LokiJS these days. It's an in-memory, native-JavaScript DB that takes care of both problems above. It can store full JS objects and restore them from the database with all their methods intact (although restoring full active objects rather than just "dumb" unserialized JSON takes a little care setting up -- I haven't bothered). It also happens to be fully sync, but through smart indexing it's faster than the thunking you would otherwise suffer.
Performance was one reason that I went with LokiJS, but it turned out later that doing so saved me from another problem: the translators run only synchronous code, so they can't do any async stuff, so if BBT would still be doing SQLite calls it would be structurally hosed in 5.0. OK so technically the translators are not supposed to be able to do database access of any kind, but BBT needs it because I need to check among all references in your database for key duplicates as I generate them.
In-memory does have its downsides. Every save you do to SQLite is (pretty much) guaranteed to be stored safely if your call to the DB returns successfully. With in-mem, you need to make sure of this yourself. I have it set up so that it saves to disk when idle or every 5 seconds, whichever comes first, and when you close Zotero. I could really do a save on every change because that too is fast (sub-second even when keeping 4 backups for safety), but this is where it's at now. I don't do any caching of the my DB because there's hardly any instance where I can beat indexed searches of the DB; the two caches I do have in my DB are the cache for serialization (where I store the serialized copy of the Zotero reference that is handed to the translator) and for results (for unchanged references I don't re-generate the bibtex but use the previously generated version). This was needed for auto-export, which would otherwise fully freeze Zotero, sometimes for minutes. But LokiJS being JS-native, it's of no use to you.
For android, I'd seriously eye Firebase or CouchDB unless your app only intends to do local storage only. I don't honestly know about their performance, but I do know datasync seems deceptively simple but really isn't. For local-only, SQLite is great. It's robust, well-supported, and the contents of an SQLite DB is easy to inspect with a whole host of tools.

Thanks for all that detail.
"We've" recently made some adjustments to biblatex in terms of date range support where "unknonwn" or "open" is in one of the dates. Essentially we:

Plug a conceptual hole in EDTF  (support the case where open comes first, as in "open/2004-01-01"); and
Add support for alternative symbols from the Working Draft of 2016-02-16 of ISO 8601-2  (Which will probably supersede EDTF by incorporating it as a "profile"). "*" for uknown and nothing [adjoining a "/"] for open.

All you'd need to know is to use and support the following sort of test cases for "unknown" and "open":
% Unknown
date = {unknown/2006}, % EDTF
date = {2004-06-01/unknown}, % EDTF
date = {*/2006}, % WD ISO 8601-2
date = {2004-06-01/*},  % WD ISO 8601-2

% Open
date = {2004-01-01/open}, % EDTF
date = {open/2004-01-01}, % EDTF
date = {2004-01-01/}, % WD ISO 8601-2
date = {/2004-01-01}, % WD ISO 8601-2

If you are curious the working Draft of 2016-02-16 of ISO 8601-2 is available from http://listserv.loc.gov/cgi-bin/wa?A2=ind1602&L=datetime&T=0&X=1A10F96E1DF2376B79&Y=rden%40loc.gov&P=1351.
I haven't forgotten that EDTF support in BBT can only occur after your port.

I am also suffering from "Better BibTex has been disabled: lib/fold-to-ascii.js load failed: SyntaxError: class is a reserved identifier". Better BibTex worked fine as recently as a couple weeks ago, but without my updating anything, suddenly it stopped working. I'm running zotero standalone version <IP_ADDRESS>.
This is really bad since I completely rely on Better BibTex for citations and have a paper deadline in less than a week. I need a quick fix.
Is there a beta available for 5.0? Or is there an older version I could revert to that doesn't have this issue?
Thanks!!

@lrperlmu what version are you on?

Sorry, I think I must have launched the wrong version of zotero when I got that issue. I'm not sure what went wrong, but it's fine now. Redacting my comment.

@JohnLukeBentley now that I'm working on Z5 support (#555), EDTF dates have become possible -- in fact, preliminary support is already in the versions I'm testing  myself. Would you be able to put together a few test cases (both for import into Zotero and export to BibLaTeX)? I could add those to my test suite.

Yes, I'd be keen help with EDTF testing.
But my head has been out of the game (Biblatex, Zotero, zotero-better-bibtex) for a while and life is busy at the moment. However, if I started having a look at creating some test cases in about 7 days could that dovetail with your efforts?
If that sort of delay would potentially create a bottleneck for you (I imagine its going to save you time if we include EDTF support/testing as early as possible) just say the word and I'll see if I can begin earlier. I don't think there'd be much for me to do (I've already got generic test cases floating around on my hard drive) ... it's just a matter of me carving out the hour or so to remind myself how all these tools work and what sort of formats they require.

No rush. I'll be busy with Z5 support for weeks to come.

@JohnLukeBentley EDTF date stuff has been added to the Z5 beta, so test cases would be welcome.

By "Z5 beta" you mean the zotero-better-bibtex beta which supports Zotero 5, I take it. Thanks for the notice, I had observed that you had to rework the zotero-better-bibtex database and were asking others to hold off on testing.
So, in short, if I've understood you: there's no current obstacle to my doing some EDTF testing (and supplying you with those test cases).

Yep, that's what I meant. There's currently no obstacle to EDTF testing; enough of BBT works that it should be enough for EDTF testing even though BBT is far from complete and currently has some open issues even on what ought to work.

Got it. I should be free Soon™. But feel free to prompt me again if (a lack of) EDTF testing threatens holds things up.

Trust me, the Z5 port has enough work to keep me busy 24/7 for the next months at least.

Well that suits me :)

Emiliano, Just letting you know I'm slowly getting back into EDTF stuff.
For the moment I'm working on issues at Biblatex itself to do with EDTF. It'll be prudent for me to sort those issues first as that will inform the target test cases for zotereo better bib(la)tex.
Just as you've expressed with the Dilbert cartoon in #555: it remains difficult for me to predict time frames. I get the sense we are in a similar boat of uncertainty there. And, moreover, my delays aren't going to risk holding you up (in the worse case you'd just fix whatever EDTF issues I point to in a subsequent point release .. I assume).

No rush. There's still plenty of stuff in the backlog, and I've always employed the squeaky wheel system for BBT, as I really know fuck-all about bibtex. Getting test cases in early for EDTF was just an attempt to get ahead of that, but it would be a break from the norm.

(in other words the EDTF support being potentially wonky is not even close to being a show stopper)

(in other words the EDTF support being potentially wonky is not even close to being a show stopper)

Thanks. That's really what I was after being (re)confirmed.

@retorquere I'm back into it.
Before giving you a full list of test dates for EDTF I thought I'd highlight two bugs I've so far found.
I'm on Zotero 5.0.23, Zotero-Better-Bibtex (ZBB) 5.0.28. Using export Better Biblatex (Zotero > Menu > Edit > Preferences > | Export | > - Quick Copy - > Default Format: Better Biblatex).
Uncertain and approximate
Under EDTF we can express a single date that is both uncertain and approximate as 1273?~
(Note the other way around 1273~? happens to be illegal under EDTF 1.0. Maybe you want to create a Quality Report entry for that case: although, if I recall, you generally wanted to avoid the business of datetime validation).
In a Zotero Date field I have 1273?~. This gets wrongly exported by ZBB as date = {1273~}.
See Report ID: D5XSY7EY
Years exceeding four digits
In Zotero ...
Date: y-51234
Date: y52000
.... should be exported as ...
date = {y-51234},
date = {y52000},

... however, they are currently exported as ...
year = {y-51234},
year = {y52000},

See Report ID: 2R34Z97U

Unspecifieds
Unspecified date fields are not correctly Biblatex Exported.
For example in Zotero "1999-uu-uu" should be passed through as is but gets exported as "1999-01-01~".
For 5 cases (Title, Report ID, date):

Cool book 0110. TXIJE6RR. 199u.
Cool book 0120. E8VE2R3Q. 19uu.
Cool book 0130. ASKHF8J5. 1999-uu
Cool book 0140. JXLIDUU4. 1999-01-uu
Cool book 0150. MGWPBP5K. 1999-uu-uu

By the way, it's right that I can't Report Better Bibtex Errors on a multiple entries at once (I mean I've tried to select multiple entreis in Zotero and generate the report but the [References] Tab seems to contain only one entry)?

:robot: this is your friendly neighborhood build bot announcing test build 4542 ("a few more test cases").

:robot: this is your friendly neighborhood build bot announcing test build 4543 ("more test cases").

OK, those test cases starting from 1273?~ have been added.
You can submit more than one reference but if I show everything that is submitted and it happens to be more than a few K of text, Firefox freezes up, so I cut it off at an arbitrary point (5K I believe). I've added ellipses to indicate the cutoff.

:robot: this is your friendly neighborhood build bot announcing test build 4544 ("ellipsis after cutoff").

I'm not getting past the first hurdle. From Zotereo 1273?~ I'm still getting a Better Biblatex Export of 1273~.
I'm definitely on zotero-better-bibtex-<IP_ADDRESS>44.issue-590.xpi for this test (I've confirmed this Zotero > Tools > Add-ons ... I see the right version string in when I expand on the Zotero Better Bib(La)Tex entry). I have restarted my Zotereo.
A git merge issue at your end?

No, I mistyped a test case. I'll have a look tomorrow.

Is that EDTF btw? Or just a date format biblatex supports?

:robot: this is your friendly neighborhood build bot announcing test build 4552 ("edtf.js requires really ugly patches to support the old EDTF format that biblatex uses").

Alright, try 4552.

:robot: this is your friendly neighborhood build bot announcing test build 4561 ("Merge branch 'master' into 590").

I'm getting ....

The add-on "%S" count not be installed. In may be incompatible with this version of Zotero".

zotero-better-bibtex-<IP_ADDRESS>66.issue-751.xpi, by contrast, installs just fine. (And I assume I need to use the issue specific versions rather than the build that is numerically greater).

:robot: this is your friendly neighborhood build bot announcing test build 4585 ("adjust test cases after merge").

Yes, you need to install the issue builds -- the highest numbered that pertains to the issue at hand. The others won't have the changes that relate to this issue. I do this so I can safely do work in parallel and possibly abandon changes if they don't turn out well.
The message you got ("The add-on '%S' count not be installed...") is the way Zotero (or Firefox, really, which Zotero is still built on) tells you you've downloaded a file that's not a XPI at all... I usually keep a max of 6 builds because I want to discourage issue builds being used past their due. I just happened to put out so many builds in short succession that yours dropped away before you could get to it. I've bumped build retention to 11 -- I'll make changes later to just keep the latest build for open issues. But not now. I want to get a new pre-release out the door when #590 and #747 clear. 4585 is available for this issue now.
The full issue list for the port is now here BTW -- the Trello list is gone. Things are starting to speed up towards a full release. I'll be so glad to have this done.

Thanks for all that.
Uncertain and approximate, Years exceeding four digits, and Unspecifieds now work on my test cases. Getting the quality report fixed first helps with the current issue.

Is that EDTF btw? Or just a date format biblatex supports?

Biblatex.pdf, p 36 has it essentially right ...

Date fields ... adhere to EDTF (Extended Date/Time Format) specification levels 0 and 1. Also supported are the open-ended range specifications in section 4.5 of the current working draft of ISO8601-2

EDTF has levels 0, 1 and 2. So biblatex partially supports EDTF (and in doing so takes care of most of the cases of concern: what's left over are rare requirements).
... although the exact set of symbols, supported by biblatex, for "unknown" and "open" ranges is currently at issue. I'll try to get some clarity on this.
The current working draft of ISO8601-2 incorporates and supersedes EDTF (with minor changes as far as a quick look can determine). But given that ISO is in draft, and liable to change, the Biblatex folk are essentially sticking with the EDTF standard (as published at https://www.loc.gov/standards/datetime/pre-submission.html) for the time being.
As a separate issue: is there a way to have BBT sort the exported entries (e.g. by citekey or title)?
I'm finding having that ability would aid my testing (when correlating a list of Zotero entries with the resulting export).
After your answer to that question I'll provide a list of all the EDTF formats I have, for your regression testing purposes.

BBT also supports the draft because it uses edtf.js which only supports the draft -- I translate the dates to the draft format for EDTF testing/parsing. The changes between the formats are superficial but annoying, currently at a blind
date
      .replace(/u/g, 'X')
      .replace(/(\?~)|(~\?)/g, '%')
      .replace(/unknown/g, '*')
      .replace(/open/g, '')
      .replace(/y/g, 'Y')

Date parsing is easily one of the uglier parts of BBT.
BBT has no way to sort references currently and it'd be non-trivial to make it happen as it would affect the way the translators read the export cache.

IIRC, Zotero offers the references to the translators in the order they're on screen; if that's (still) true, sorting by title or citekey in Zotero should yield a sorted export.

Oh and this build includes the quality report.

Correction - it would not just affect the way the export cache is read, but for cached references, I don't have information except the citekey - I just get the full textual reference for the cache. So sort by key is non-trivial, but sort by title is effectively not possible.

I could create a ruby or python script which sorts the references by citekey.

On the small but subtle differences between EDTF and the new draft ISO I might start a new thread over at biblatex ... I'll let you know to keep you in the loop.
Here at BBT I've opened "Sort order of exported entries. By citekey or zotero order as on screen. #766":

The export order I get doesn't seem to preserve the Zotero screen order; and
"I could create a ruby or python script which sorts the references by citekey" sounds like you are seeing a potential trivial route to implementation.

... if either possibility could be trivially implemented that'd be great.
I mean following the zotero screen order would give us the possibility, among others, to sort by citekey. Because in Zotero we could sort by any column we like. But maybe your parser can't preserve the order in which entries are presented to you by Zotero??.
Alternatively, if you can't preserve the order in which entries are presented to you by Zotero, then maybe sorting by citekey turns out to be trivial??

The wonders of getting some sleep... not only can I not affect the order in which Zotero hands translators the references to be exported, I don't want this to be possible because it would mess up my test framework something bad.
The references are offered to translators either in order of creation or last-modified date (I forget which); you can sort by those in Zotero by adding those columns, and that would mean you'd get the exported references in the same order as they're present in the export.

This ruby script ought to do it:
#!/usr/bin/env ruby

refs = []

IO.readlines(ARGV[0]).each{|line|
  if line =~ /^@[a-z]+{(.+),$/
    puts $1
    refs << { citekey: $1, ref: '' }
  end

  refs[-1][:ref] += line
}

puts refs.sort_by{|ref| ref[:citekey] }.collect{|ref| ref[:ref] }.join('')

The state of the EDTF standard(s)
Let me catch you up on the state of the EDTF standard(s).
Biblatex has been supporting the EDTF standard (to "level 1") as hosted at the (US) library of congress. Biblatex supports a superset of that standard: a few extra symbols for backward and forward compatibility reasons; and for reasons of being sensibly more permissive. However, recently the EDTF standard at the library of congress has been substantially incorporated into a draft ISO Date Time revision. And so that EDTF standard is formally deprecated ....

This draft specification is superceded; EDTF functionality has been integrated into a draft revision of ISO 8601 to be published in 2017 or 2018. http://www.loc.gov/standards/datetime/pre-submission.html

For the sake of fixing these different standards in the mind, and to provide a vocabulary for talking about them, I'll stipulate the following (there's no need for you to follow the links or read any of this documentation ... let me be your EDTF consultant ;)):

"EdtfLoc", the draft EDTF specification hosted at the library of congress: http://www.loc.gov/standards/datetime/pre-submission.html. That is, the spec we've been going off.
"EdtfIso201x", the EDTF profile as found in the draft ISO documents:  ISO DIS 8601-201x Part 1; and ISO DIS 8601-201x Part 2 (as made available from http://www.loc.gov/standards/datetime/pre-submission.html). The current version of these are dated 2016-10-26. Essentially what matters for EDTF purposes is Part 2 (that is, you could get away with only reading Part 2). Within Part two "Annex C" stipulates the EDTF profile. However, that profile references eariler parts of the Part 2 document.
"Iso201x" The full draft datetime specification as found in the above ISO documents. That is, ISO DIS 8601-201x Part 1 and ISO DIS 8601-201x Part 2.
"Iso2004". The old ISO ... ISO 8601:2004 https://www.iso.org/standard/40874.html. This standard doesn't stipulate an EDTF profile.

EdtfIso201x is in its early days it: it warns that is subject to change. For the moment biblatex substantially remains with EdtfLoc. In any case, the differences between EdtfLoc and EdtfIso201x (to the respective "level 1"s) are, thankfully, minor.
What BBT ought do
Therefore for our purposes, the datetime formats BBT needs to support (for export to Biblatex), this can be made quite straightforward. BBT ought support:

EdtfLoc to EdtfLoc level 1 (as BBT has been doing - and largely what we've been working towards);
A few backwards compatibility markers (as BBT already does). Namely:

In ranges, open: [blank] (e.g. as in "/2004-01-01");
In ranges, unknown: "*" (e.g. as in "2004-06-01/*")

Forwards compatibility markers due to a biblatex flirtation with EdtfIso201x with respect to open ranges. Namely, ".." as in "../2004-01-01".
Support for a reduced precision time (to a minute) e.g. "2017-10-28T04:47" down from "2017-10-28T04:47:00". Because biblatex supports this even though EdtfLoc doesn't. Second precision, as in ""2017-10-28T04:47:00",  should remain as also supported.

Current EDTF BBT bugs/issues.
I've fired all my EDTF cannons (all the various relevant datetime formats) at BBT and the following are the only outstanding issues/bugs:

Add support for ".." to express an open range as in "../2004-01-01" or "2004-01-01/.." (while also preserving support for "open" and blank in open ranges).

ReportID: 89FGSMZP. citekey: simpson_cool_book_0396
ReportID: PI59DPQH. citekey: simpson_cool_book_0397

Add support for a reduced precision time (to a minute) e.g. "2017-10-28T04:47" down from "2017-10-28T04:47:00". Because biblatex supports this even though EdtfLoc doesn't. Second precision, as in ""2017-10-28T04:47:00",  should remain as also supported.
See ReportID: KC56EPHC (greenwald_2017_how). Currently coming through as year = {2017-10-28T04:44}, when it should come through as date = {2017-10-28T04:44},: the difference is between year and date.

Further discussion.
On the reduced precision issue. Edtfloc says:

A date/time string MUST be composed according to one of three representations as illustrated in the following three examples:

2001-02-03T09:30:01
2004-01-01T10:10:10Z
2004-01-01T10:10:10+05:00

Note: 'T' separating date and time must be upper case.

The date/time string MUST use 8601 extended form, i.e. date with hyphen, time with colon. Zone-offset may be omitted or included.  8601 extended format time zone designation consists of either a 'Z' to indicate UTC, or a '+' or '-' to indicate "ahead of UTC" or "behind UTC", followed by a 2-digit hour, followed optionally by a colon and the 2-digit minutes.

And the Edtfloc BNF stipulates

dateAndTime = date "T" time
     time = baseTime zoneOffset?
           baseTime = hour ":" minute ":" second | "24:00:00"

So, as written, a reduce precision datetime, like "2017-10-28T04:47" appears to be forbidden under Edtfloc.
I'm unclear if this is a good or bad thing. As far as my initial thinking goes: it is a bad thing on the basis that times with minute precision are common (and forcing such a time to go from "2017-10-28T04:47" to "2017-10-28T04:47:00" misrepresents the time).
It remains possible that the Edtfloc writers didn't intend to forbid reduced precision times (they could have, and should have, written in the spec "omitting seconds in a time is not permitted" if that was their intentions).
In any case, the biblatex folk (in effect @plk) have supported reduced precision datetimes (down to minute level, where a time is present), and that seems like the right thing to do even if not conforming to Edtfloc.
Conclusion
Let me know if the two issues/fixes are (or become) non-trivial. But on the assumption they are trivial to fix:

I'll get an update from you and retest;
That'll likely mean all my EDTF tests will pass: I'll then give you the full list of EDTF datetime formats.

I'll be out of action for about the next 8 hours (day time activities I need to attend to) ... but after that I'll try to make myself available to you as a priority (sleep cycles aside) ... to help you close off https://github.com/retorquere/zotero-better-bibtex/milestone/1

Alright, thanks for all that. To make sure I got it right: the strike-out text is there for posterity, but I don't need to act on it. Yes?
I don't currently have code in place to do content checks in the QR -- feel free to add an enhancement request for that, it's an interesting addition even if I don't currently know how involved it will be. But right now, the "do nothing" approach is what's possible. I could let the minute-level go through (by testing for EDTF-ness by adding :00 as one of the tests), but that would mean it would just show up in the date field with no mention of the potential problem in the QR.
The two test cases you added should be easy enough, but we can both use some shut-eye I think. Today has been incredibly taxing for me -- rest is the smart thing to do.

:robot: this is your friendly neighborhood build bot announcing test build 4628 ("fixes #590 for real (forgot to commit files)").

Done

rest is the smart thing to do.

Yes indeed. It sounds like we are in opposing hemispheres, timewise, (I'm at +11:00). If so, that neatly means we are tag teaming it.
Those strikethroughs were to make clear where I'd edited my post. Mostly for your benefit in case you read the original, wrong, post before seeing my edits. So, yes, you don't act on the strikethroughs.

Add support for ".." to express an open range as in "../2004-01-01" or "2004-01-01/.." (while also preserving support for "open" and blank in open ranges).

On build 4628 that works at my end!
To do

Another issue/bug.  For "unknown" date ranges:

unknown/2006 wrongly comes through as year = {unknown/2006}.  Should be date = {unknown/2006}. Report ID: DI8C2DXH. simpson_cool_book_0092.
2004-06-01/unknown wrongly comes through as  year = {2004-06-01/unknown}. Should be date = {2004-06-01/unknown}.  Report ID: HIEGAV27. simpson_cool_book_0093

I could let the minute-level go through (by testing for EDTF-ness by adding :00 as one of the tests), but that would mean it would just show up in the date field with no mention of the potential problem in the QR.

Yeah, let's do that.
At the moment, with minute-level precision coming through as year = {2017-10-28T04:44}, we'll get a biblatex compile warning like ...

WARN - year field '2017-10-28T04:44' in entry 'greenwald_2017_how' is not an integer - this will probably not sort properly.

... which is a bit cryptic.
By contrast, if you let it go through as date = {2017-10-28T04:44}, then we'll get a better biblatex compile warning ...

WARN - Entry 'greenwald_2017_how' (Biblatex-Tester-EdtfFull-FromZotero.bib): Invalid format '2017-10-28T04:44' of date field 'date' - ignoring.

:robot: this is your friendly neighborhood build bot announcing test build 4634 ("minute-level precision for #590").

:robot: this is your friendly neighborhood build bot announcing test build 4637 ("value-qr").

Check out the new QR, too.

Unknowns now work.

I see the QR. For  date = {2017-10-28T14:30}, I get ...

% 'date': not a valid ETDF date

All my EDTF dates now pass (against 4637)!
Maybe the thing to do is merge 590 into the master, and whatever else you think is outstanding for the "Rewrite for Zotero 5" (including #771?), and I'll retest my EDTF against the master. That is, as a test against a release candidate.
That would serve as a final check of my EDTF test dates (noting one's tests are themselves tested). At that point, I'll suggest, I'd post those datetime formats here.

:robot: this is your friendly neighborhood build bot announcing test build 4657 ("Merge branch 'master' into 590").

Here you go. This has everything bundled in.

(#771 should be done tomorrow somewhere)

Intro
As expected all my tests against zotero-better-bibtex-<IP_ADDRESS>57.issue-590.xpi pass. Therefore you can close this issue.
(Repeating earlier information here ...) Biblatex has been supporting the EDTF standard (to "level 1") as hosted at the (US) library of congress (which I'll reference as "EdtfLoc"). Biblatex supports a superset of that standard: a few extra symbols for backward and forward compatibility reasons.
Find below an exhaustive excercise of the datetime formats biblatex supports. At least, that is, as the main format variants go.
Three of the formats will throw up "WARNS" in biblatex:

date = {2017-10-28T14:30}, // Minute level precision. E.g.:

WARN - Entry 'musk_2017_picture' (Biblatex-Tester-EdtfFull-FromZotero.bib): Invalid format '2017-10-28T14:30' of date field 'date' - ignoring

date = {2004-01-01/..}, // EdtfIso201x. simpson_cool_book_0396. E.g.:

WARN - Entry 'simpson_cool_book_0396' (Biblatex-Tester-EdtfFull-FromZotero.bib): Invalid format '2004-01-01/..' of end date field 'date' - ignoring

date = {../2004-01-01}, // EdtfIso201x. simpson_cool_book_0397. E.g.:

WARN - Entry 'simpson_cool_book_0397' (Biblatex-Tester-EdtfFull-FromZotero.bib): Invalid format '../2004-01-01' of date field 'date' - ignoring

In biblatex the minute level precision warning is expected. The double dot ".." warning for open date ranges is a biblatex bug (biblatex.pdf mentions support for this symbol). For BBT purposes those three dateformats are correctly passed through to biblatex.
Let me know if you need further info (e.g. Report IDs against the following formats).
Legend

"EdtfLoc", the draft EDTF specification hosted at the library of congress: http://www.loc.gov/standards/datetime/pre-submission.html. That is, the spec we've been going off.
"EdtfIso201x", the EDTF profile as found in the draft ISO documents:  ISO DIS 8601-201x Part 1; and ISO DIS 8601-201x Part 2 (as made available from http://www.loc.gov/standards/datetime/pre-submission.html). The current version of these are dated 2016-10-26. Essentially what matters for EDTF purposes is Part 2 (that is, you could get away with only reading Part 2). Within Part two "Annex C" stipulates the EDTF profile. However, that profile references eariler parts of the Part 2 document.
For my own reference I've included my cite keys (in the format "author_basedate_titleword").

Common
Typical
date = {1995-03-01}, // chalmers_1995_facing

Approximate and Uncertain
date = {1590~}, // Approximate. galilei_1590_motion
date = {1256?},  // Uncertain. aquinas_1256_two
date = {1273?~}, // Approximate and Uncertain. aquinas_1273_compendium

Eras
origdate = {-0379~}, // Before the year zero ("BCE"). plato_2004_republic
origdate = {0125~}, // Not long after the year zero (“CE”). epictetus_1916_enchiridion

There is nothing special about origdate here. You could just as well use date.
Precision finer than a year
Times
date = {2016-07-18T20:26:06}, // Unpsecifed time zone. simpson_cool_book_0074
date = {2016-06-06T18:20:00+10:00}, // Local time zone. barker_2016_swiss
date = {2016-07-18T10:26:06Z}, // Zulu/UTC timezone. simpson_cool_book_0078

Minute level precision - Invalid under EDTF (triggers a ”WARN” under
biblatex)
// Minute level precision.
// EDTF and Biblatex Invalid. musk_2017_picture
date = {2017-10-28T14:30},

Dates
date = {2008-01}, // Month precision. kennett_2008_true
date = {2010-01-25}, // Day precision. mendelsohn_2010_enough
date = {2016-24}, // Season precision. sep_2016_stanford

Ranges
Common
date = {1487/1490}, // da_vinci_1487_codex

Open
date = {2004-01-01/open},  // Edtfloc. simpson_cool_book_0096
date = {open/2004-01-01}, // Edtfloc. simpson_cool_book_0097
date = {/2004-01-01}, // Biblatex backwards compatibility. simpson_cool_book_0099
date = {2004-01-01/}, // Biblatex backwards compatibility. simpson_cool_book_0098
date = {2004-01-01/..}, // EdtfIso201x. simpson_cool_book_0396
date = {../2004-01-01}, // EdtfIso201x. simpson_cool_book_0397

Unknown
date = {unknown/2006}, // Edtfloc. simpson_cool_book_0092
date = {2004-06-01/unknown}, // Edtfloc. simpson_cool_book_0093
// Biblatex backwards compatibility. Edtfloc invalid. simpson_cool_book_0094
date = {*/2006},
// Biblatex backwards compatibility. Edtfloc invalid. simpson_cool_book_0095
date = {2004-06-01/*},

Unspecified
date = {199u}, // Within a decade. simpson_cool_book_0110
date = {19uu}, // Within a century. simpson_cool_book_0120
date = {1999-uu}, // Within a Year (to month level). simpson_cool_book_0130
date = {1999-01-uu}, // Within in month. simpson_cool_book_0140
date = {1999-uu-uu}, // Wtihin a Year (to day level). simpson_cool_book_0150

Years exceeding four digits
date = {y-51234}, // A long, long time ago. rebels_y-51234_rebel
date = {y52000}, // A long, long time ahead. seldon_y52000_encyclopedia

Holy shitsnacks I saw that wall of text coming on on mobile and I just panicked there were 10 more edge cases to deal with :laughing:. Thanks man, merged, two more to go before release.

Thanks for all that EDTF coding. That really opens up the gates for where I want to go.
Incidentally, "wall of text" does not mean "lots of text" ...
https://www.urbandictionary.com/define.php?term=wall of text

A piece of writing that does not use proper grammar and generally looks like a giant essay with 20 to 400 sentences without using paragraphs or any bit of spacing at all.

... I look forward to the release and hoping that'll afford you a break, or at least allow you to spend far less time on BBT.

I stand corrected on the wall of text. In my defense, it looked a little packed on my mobile mail preview :worried:.
A break would be nice, but I'm in the process of putting together a PhD proposal, and that process, as I'll just blindly guess you know, allows for no breaks.
incidentally, @blip-bloop should now just ignore closed issues. Stay, @blip-bloop.


Improved locale_name's

There are two fixes here - the commit messages say it all. I'm not quite sure whether this will also work with RTL languages but upon reading the W3C article I have an impression that things will "just work" (that is, the language code will automagically travel to the other side of the string) thanks to the Unicode Bidi Algorithm. I'm really not a specialist in the area though.
As a quite a reasonable option - the language code might be entirely omitted from the UI (though it mandates reasonable distinction between locales in their names - but it's a good idea anyway).

I think it's important to include the language code in the UI; that provides the exact information for someone who wants to know. Moving that mechanism into the code makes sense, though.
I'll have to double-check with the translators on the names-for-languages in their language.

BTW, thanks for the pull request!

Please see my comment on https://github.com/coreinfrastructure/best-practices-badge/issues/844#issuecomment-309752142 as to why I don't think this pull request should be merged. I agree that the locale could be pulled out and displayed by the code, though I'm not sure it's worth the trouble.

I am going to have to disagree with you here Dan, It think having the language code automatically filled in rather than being part of the key is a good thing.  It helps prevent mistakes on the part of our translators.
@david-a-wheeler I do believe the change from Deutsche to Deutsch is consistent with what our German translator has provided in their translation.  @GeorgLink, care to chime in here.

I don't think we want to compute the locale strings than looking up their translation.
If a translator makes a mistake, the translator can fix it; that's very unlikely & not a big deal if the unlikely event happens.  There are advantages to the current approach: the translation shows exactly what users will see, and the translators get more control over it (e.g., if different symbols or order would be clearer in that locale).  There's also a minor performance issue: this occurs in a loop in a layout (which is shown on every page), so efficiency matters a little.  If the language code is automatically filled in, Ruby has to create new strings in a loop on every page, instead of just doing a lookup. Yes, that's a minor effect, I'd rather avoid string creation in a loop used on every single page.
Of course, if the names are wrong that needs fixing.  E.G., Deutsche to Deutsch.

"Deutsch" is correct for the language.
Thanks for catching this @jdossett

Credit where credit is due - @KitsuneRal spotted the misspelling of German. Thanks!
German has been corrected "Deutsch" in en.yml, and in the other translations.

Okay, we won't change how locales are displayed, but we did fix the names for Russian and German (in their respective languages) based on this report.
Thanks so much for the report & PR!


Support :matches()

body :matches(h1, h2) {
  color: red;
}

should yields into
body h1, body h2 {
  color: red;
}

poke @Eramdam

We might want to port this https://github.com/michelle/rework-matches

Done. I will release postcss-selector-matches and include it in cssnext tonight ir tomorrow :)

@MoOx good job (again?)

BOOM https://github.com/postcss/postcss-selector-matches

Closed by 0cf0c13bb20e1aa39c21aa5d733cabb1f7d9a11e

Released as 1.4.0


fix: show empty search view when search API return error

Explanation of Change

Fixed Issues

$ https://github.com/Expensify/App/issues/52907
PROPOSAL: https://github.com/Expensify/App/issues/52907#issuecomment-2491409822
Tests

Open search router icon
Enter the query: type:chat in:in category:car
Hit enter
After the screen navigates to search page, verify that: Empty search page is displayed

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

Open search router icon
Enter the query: type:chat in:in category:car
Hit enter
After the screen navigates to search page, verify that: Empty search page is displayed

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I used JaimeGPT to get English > Spanish translation. I then posted it in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
https://github.com/user-attachments/assets/9aff3bb0-e485-4d7d-8c0b-537991bf9c48

Android: mWeb Chrome
https://github.com/user-attachments/assets/e24cebdb-4792-47c7-8788-8bf7785ae0dd

iOS: Native
https://github.com/user-attachments/assets/eb552d97-7d15-41ae-b40d-765a38a70749

iOS: mWeb Safari
https://github.com/user-attachments/assets/02e9516d-e841-44cd-9604-793a00a7819d

MacOS: Chrome / Safari
https://github.com/user-attachments/assets/8a8c7f25-3ccf-4691-afa7-6778eb249a5b

MacOS: Desktop
https://github.com/user-attachments/assets/55675813-9069-4db6-acee-3c9d62a6e98c

Please update copies based on feedbacks provided in slack convo

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] For any bug fix or new feature in this PR, I verified that sufficient unit tests are included to prevent regressions in this flow.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/01e5b1ed-975a-4072-9ca4-6bd4977c0db1

Android: mWeb Chrome

iOS: Native

https://github.com/user-attachments/assets/50c31b4f-1b1b-46dc-93d9-bd4bee6d8547

iOS: mWeb Safari

https://github.com/user-attachments/assets/0f4acc50-51f1-402e-ba10-ff267eb9ded8

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/1ee1680b-593e-43d1-a686-991b0d940209

MacOS: Desktop

https://github.com/user-attachments/assets/24bdfce2-03f6-4279-96b7-a8207b5ee455

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.