Prefix Compression and faster startup time

Current status: I am going to start a self-review later/tomorrow
This is a lot of change for one PR but they are separated relatively well from each other.

Implemented the Prefix Compression Heuristic using a Trie-based
greedy algorithm

Integrated Prefix compression into the Vocabulary class
(templated for prefix compression / no prefix compression)

Converted SPO and SOP to Mmap based data

faster startup time for ServerMain by not iterating over the complete
Mmap Vector for statistics

Added nlohnmann/json as a submodule

IndexBuilder writes a configuration file (json) that contains
information needed for the prefix compression. Can be extended for
other settings and statistics

it is also possible to externalize entities from the vocabulary (e.g.
Wikidata statement ids)

it is possible to pass a settings-json file to the index builder.
Currently supported: Prefixes that shall be externalized (see above)

And don't worry about that test failure, I found the bug and will fix it tomorrow afternoon.
The test is not good there: it implicitly checks implementation details (assumes a certain size for the serialization of the MetaData instead of explicitly checking it or blackbox test it). The effect was a nondeterministic test failure (works on my machine) for code which is probably fine. You'll see what I mean tomorrow.

I think you check them out locally and commit that explicitly and then everyone else needs to do git submodule sync. I'm not 100% sure though.On Aug 22, 2018 18:02, joka921<EMAIL_ADDRESS>wrote:@joka921 commented on this pull request.
In .gitmodules:

@@ -4,3 +4,6 @@
[submodule "third_party/googletest"]
path = third_party/googletest
url = https://github.com/google/googletest.git
+[submodule "third_party/json"]

Yes indeed, how do I choose/fixate a certain commit of a certain branch of the submodule for my project?
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

Can you rebase on the freshly merged disambiguate optional work with the changes to operator[] as discussed in that PR. After that I think we can merge this.

So am I seeing this right, the last couple of e2e test failures were just flakyness?

@niklas88 The first one was a bug with getopt (I removed the -l flag which was still present in the e2e test).
The second failure I did not understand (worked fine locally, also with Release builds). Obviously the server failed to open, so I added a Server run in the foreground which of course let the test stall but yielded no results. After simply removing this again it worked fine so I assume that this one build I cannot explain (3ccb9ec) was some versioning/checkout problem with Travis.

I Implemented the changes we discussed on Monday so I think we are good for now. The things that could be improved are all stuff for separate PRs in my opinion. If you agree I will squash the commits and merge this, I just wanted to give you the chance to look at it again first.


Move bits/address_bits to AAL/PAL

FreeBSD RISC-V is the primary motivating example of a platform that really constrains the AS: it exposes a 38 bit AS for user programs (that is, it uses the sv39 page table format and cleaves the AS in half for kernel/user).  If we go with the traditionally hard-coded 48 bit AS estimate from ds/bits, we end up trying to allocate the whole user AS (48 bits in AS - 14 bits for a 16K chunk size + 4 bits for a MetaEntry = 38 bits of Pagemap).  As you might imagine, this doesn't go well for us (and it goes somehow even worse for us on CHERI when we ask for a 39-bit AS allocation...).  FreeBSD MIPS is similar and exposes a 39 bit AS and so also suffers (even if we only ask for half the AS for the Pagemap, it still doesn't really work out well for us), but I don't think that matters as much going forward.

Ah, software.  Let's see how this fares.  I think we don't need to be doing the SFINAE dance in PALPOSIX since those use inheritance (though, admittedly, curiously with PALLinux : PALPOSIX<PALLinux> rather than just PALLinux : PALPOSIX) and that suffices for ::address_bits.


Add try/catch/finally

http://www.w3schools.com/jsref/jsref_try_catch.asp

:-1: I don't know if there's like an official policy on this stuff, but I think the value of caniuse would be diminished by including every JS feature released in 1999.

Indeed, caniuse is intended for newer technologies, not to point out things that have been supported for ages like this. However I'll leave this issue open, as at some point I may make this searchable to point out that it's fully supported.

I'm sorry. I didn't check first that it was deprecated first, my bad. I agree that it would help if it redirected like wiki, with a redirection comment at the top, like wikipedia. Thanks!

Available at https://caniuse.com/#feat=mdn-javascript_statements_try_catch


How can I start json server as the below mentioned command not working?

C:\Users\ARIM SAHA>json-server --watch db.json
'json-server' is not recognized as an internal or external command,                                                     operable program or batch file.
this is happening in my windows 64 bit machine.

C:\Users\ARIM SAHA>json-server --watch db.json
'json-server' is not recognized as an internal or external command, operable program or batch file.
this is happening in my windows 64 bit machine.


Binary encoding issues

Encoding issues were found in adb_commands.py when using the str() function without specifying an encoder. And in filesync_protocol.py when trying to decode compiled or binary files. The file contents contain characters not found in 'utf-8' and 'ISO-8859-1' was used instead. These changes will allow the user to pull files such as apk applications off of the device.

Coverage decreased (-0.5%) to 47.667% when pulling e18ce8212dc64efa40af2debd077b4e9555403cc on BlackAp3rture:master into cb4ae46fc5d2b30254551ea71b3be4b64879c216 on google:master.

I signed it!

I signed it!

I signed it!

Coverage decreased (-0.8%) to 47.289% when pulling 1d8a179acc54168e2712fd6596f959b97353eabd on BlackAp3rture:master into cb4ae46fc5d2b30254551ea71b3be4b64879c216 on google:master.

Coverage increased (+0.008%) to 48.126% when pulling 1cc89c6a91b3910b91e1581176ff4d586e0ea307 on BlackAp3rture:master into cb4ae46fc5d2b30254551ea71b3be4b64879c216 on google:master.

Coverage increased (+0.2%) to 48.231% when pulling 483c89e45b1e2f8208bb64c368872929673fbdf6 on BlackAp3rture:master into c7fbfaefd8382751ae26af76ae707ad40c2279bd on google:master.


drivers: i2c: stm32: cleanup build issue if no I2C ports are configured

We can enable the driver, but not actually have any I2C ports enabled so
we will get build issues in this case.  Put ifdef protection around the
code so that we only build it if one of the ports is enabled.
Signed-off-by: Kumar Gala<EMAIL_ADDRESS>

Well the port enablement should eventually be driven by DT, so we really shouldn't have that issue.

@ydamigos made a good point. The change hides the problem and makes it more difficult to debug the issue. The compiler warning in case no ports are enabled is very much in place and helpful. We would better make it more not less explicit. E.g. by having something like the following check at the top of the file:
#if !(defined(CONFIG_I2C_1) || defined(CONFIG_I2C_2) || defined(CONFIG_I2C_3))
#error "No I2C port is enabled"
#endif

as en error or warning. If we don't want the warning it's better to leave the code the way it is now / abandon this PR.

The compile error breaks testsuite runs.  As I mentioned once we put the DT stuff in, this will not mater, as we'll use the DT configure a port being enabled or not.

But does it not mean that we configured the testcase incorrectly? So we should rather fix the testcase.

Yeah, so I looked into this and worked up a different solution.  We should fix the testcases, but will look at that in the future.


Create a person details page

Suggestion / Feature Request
The page should open when tapping on any cast of a movie or tv show and should properly display all the details of the person
Use this endpoint
https://developers.themoviedb.org/3/people/get-person-details
Create data classes accordingly, the design of the page is upto innovation but all details should be displayed

id like to work on this

assigning


Undo _JAVA_OPTIONS enrivonment variable

Ref sbt/sbt#3520
This undoes the environment variable addedin 2017-09-06 update of Travis CI:
https://docs.travis-ci.com/user/build-environment-updates/2017-09-06/

To validate this pull request against other sbt modules, please visit this link.

Holy Batman. All green on the first try (no restarting jobs). This was it.


TRAVIS-14 Replace sonarqube.com by sonarcloud.io

SonarQube.com has been renamed to SonarCloud.io and we need to update the addon accordingly.
I left the sonarqube addon (with a warning) for compatibility with yml files that are currently using it. We can remove it in a couple of months.
More information here: https://jira.sonarsource.com/browse/MMF-921

To be perfectly precise, "SonarQube.com" has been rebranded to "SonarCloud" - not "SonarCloud.io".
(cc @dbmeneses :-) )

Hello. This has been deployed to https://staging.travis-ci.org. Let us know how it goes.

I've validated this on staging.
So all good on my side @BanzaiMan !
@dbmeneses: I think you can submit the P/R for the documentation too. Thanks!

@BanzaiMan I created a P/R for the doc: https://github.com/travis-ci/docs-travis-ci-com/pull/1262. From our side we're ready to go to prod.


I don't think tuning the PIDs would fix this issue because the rover still needs to stop and check for obstacles, unless it can check for obstacles and turn at the same time. I tested some values with the PIDs and I was able to get a very smooth turn, but that's only when I changed the angle, but since the angle is static, it would have to keep turning if the obstacle was still within that turning angle.


Contract functions don't update on contract tab

This is a silly issue and super low priority
But check out the screen shot, and see that when I change the from using the ENS-Registry to using the ENS-Public Resolver the old function lingers.
Love this site, it is a life saver always.
Much Love,
Griff

you have to click "Access" @GriffGreen

@GriffGreen fixed also need you to check eos on contracts page 💖


provision multiple disks server

we need multiple disks server for some testing, like raid.
but looks tmt don't support it, so could we add this function in provision ?
maybe like --hardware "disk.size >= 500GB"  --hardware "disk.count >= 4"

Possible workaround:
disk:
  - size: ">= 0"
  - size: ">= 0"
  - size: ">= 0"
  - size: ">= 0"

This would turn into requesting 4 disks of any size.

As proposed in https://github.com/teemtee/tmt/issues/2721, we could have keywords in collections:
disk:
  count: ">= 4"

We have many testcases,which need more than two disks,the disk.count param would benefit us a lot:)

The suggested feature, with its specification proposed in #2789, and with changes proposed by me in https://github.com/teemtee/tmt/pull/2790#issuecomment-2020680709, did not seem to attract any real support from the rest of tmt community. Quite frankly, it was a bust.
@guazhang @skycastlelily would you mind commenting on how frequently you and your test face this kind of request in your tests and related work? Plus, more real-live use cases and demonstration of the added value, compared to the "copy & paste" workaround from https://github.com/teemtee/tmt/issues/2536#issuecomment-1838598488, are needed to get the necessary traction.
Cc @psss @lukaszachy @thrix

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk. That may limit the usability of such a solution - it's always harder to find the exact match, and when "at least" is acceptable to the test, this would artificially constrain the set of suitable machines.

Thanks for this note @happz. I didn't realize that use case and with it in mind I have to scratch all my complains .

Sorry, when discussing the proposal yesterday I did not know about this issue so I completely missed the motivation mentioned here.

We have many test cases, which need more than two disks, the disk.count param would benefit us a lot:)

@skycastlelily, @martinhoyer, what's the common number of disks you usually request in your test cases? I would just like to better understand what is a common scenario here.

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk.

Shouldn't the list of two disks be mapped to any guest which has at least two disks? I think it would be much more reasonable default. For example, when I just need a larger disk I would state:
disk:
  - size: >= 4 GB

In this case I'd say it does not make sense to limit search for single-disk guests only. Does it?

A note to the "copy and paste" workaround: it's equivalent to the == operator, but cannot handle >= or <= operators; it's not possible to express "at least N disks" by listing size: >= 0 entries under disk.

Shouldn't the list of two disks be mapped to any guest which has at least two disks? I think it would be much more reasonable default. For example, when I just need a larger disk I would state:
disk:
  - size: >= 4 GB

In this case I'd say it does not make sense to limit search for single-disk guests only. Does it?

That's hard to tell, given how disk works now. Suggests that the index might matter - we should fix that and state it doesn't, which would be much easier once the topology records what has been provisioned - and it's also a list. Yes, some might expect that the listing is "at least this and the rest doesn't matter", some might expect "exactly this".
I suppose we should not leave space for such guessing, and define what exactly disk means. Is it "at least N disks"? Ok, we state that. Can I request "at least N disks, with 1 of them at least this big"? yes, I can by repeating - size: >= 0 enough times. Or, is it "exactly this"? Ok, we state that instead. Can I request "more than N disks"? Yes, I can, by repeating - size: >= 0. Can I request "less than N disks"? Yes, by the same repetition, till the list has N-1 entries. Doable? Yes. User friendly? Nope.
That's why I like the idea of adding count, we could turn disk into "at least this, the rest does not matter" key explicitly while being able to control the number of disks, or, with possibly all keyword, apply requirements to all disks without repetition. It might look like a syntax sugar to some extent, removing the need for pretty pointless size: > 0 entries, but it would also remove some unambiguity from the specification, and the "number of disks" would not be driven by enumerating them.

@skycastlelily, @martinhoyer, what's the common number of disks you usually request in your test cases? I would just like to better understand what is a common scenario here.

Tough question. I think disk count is only valid if you need a specific type of physical storage, so that exclude all tests cases that need just n of any block devices, as there are ways the test can create block devices from existing storage space.
Therefore I imagine the use-cases would be among the lines of: "4 SATA drives", "2 rotational disks; 1 SSD", "2 tape devices", or something more complex like "5 self-encrypting nvme drives connected to specific raid/hba/sas controller".
@fsuba might have better insight on what is a usual requirement for Stratis/VDO/LVM testing.
From my (distant) point of view, if a tester has a complex hardware requirements, they probably know specific hosts that fits them, so it should just be a matter of specifying a list of hostnames to choose from. So I wouldn't go crazy with hard-to-implement filters.
I was also thinking if there is a way to tag specific hosts with custom descriptor at least in some of the provisioning methods, so users could use something like hardware: tag: 'something', but that's probably a bad idea.

Hi,
We don't have to think about this complicated scenario.
for example, how many more than 500G disks in the server? we don't  need consider it's SATA/SAS/SSD .
maybe we can add another parameter to filter disk model.

we don't need consider it's SATA/SAS/SSD

We do, but this can be already done with driver I think.

Hi, sorry for the late reply, I was on sick leave.

Plus, more real-live use cases and demonstration of the added value, compared to the "copy & paste" workaround from > https://github.com/teemtee/tmt/issues/2536#issuecomment-1838598488, are needed to get the necessary traction.

what's the common number of disks you usually request in your test cases?

Three. Without count, we need  specify   - size > 8G three times.
And we do have some testcases needing more than two  disks(disk.count >=  2)  , so, disk.count  would really benefit us:)


get km/h value for wind speed and wind gusts forecast data

solves https://github.com/FL550/dwd_weather/issues/32 (i guess)

Yep, you are right. I missed these for the forecast values. Thanks for your PR!


Place limit on fence begin and end row size.

The fence concept provided as part of the no-chop merges places start row and end row data associated with a metadata file reference to fence off sections of files.
There should be an maximum limit on the number of bytes that these values can hold.  On possibility may be to use the TABLE_MAX_END_ROW_SIZE property that was introduced in 1.7.0. Currently the default is 10K.
Allowing unbounded values would likely lead to performance issues reading, processing and storing the fence information as it grows larger.

Having some limit is probably good as long as it is configurable either with the existing property you mentioned or a new property.  The main concern here is that under normal use cases (at least for today, maybe new use cases will be used in the future) the ranges for fenced files are going to be generated as the merge and deletion operations will fence files using the existing rows in tablets. If the limit was hit then these operations could or would fail if the size was violated.
So if it's configurable though it would allow someone who really wanted to run those operations to do so if they were ok with the consequences of large range keys.
In practice, it's hard to say how much of a performance hit there would be until we test. The other thing is that most of the time (but not always of course) the ranges files will not be around for very long. Normal compaction cycles would eventually compact and get rid of them but of course there is no guarantee of that.

The actual fencing is going to follow the tablet split points, unless there's some other source for the fencing ranges that I don't know about... ? If they follow the split points, then there's no reason to believe any problems due to size are going to be more substantial than the normal tablet split point sizes, right?
One size optimization is to omit specifying the range when the range is the current tablet's range. If the range is empty, the current tablet is enough to specify its bounds. It only needs to be specified when there is a range that falls within the bounds (from a previous merge). This will make fencing ranges in metadata unnecessary for the vast majority of files. Only files that were previously involved in a merge would actually need to have a fencing range serialized in their metadata.


Allow owners/editors to test shop experience before publishing by using test APIs.

At some point owners and editors may want to see how the shop works without needing to actually use money. They can already see the shop of an unpublished event, but the payment would work according to whatever the global settings are.

Resolved with merge of pull request #304


AMBARI-24871 fix checkstyle violation (benyoka)

What changes were proposed in this pull request?
Fixed checkstyle violation introduced in 2bfbef2cab
How was this patch tested?
Checked that source code compiles.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4560/
Test PASSed.


EM124 Data Processing in Kluster

"Sonar model not understood" issue with the "em124" data

I believe this is as simple as adding an em124 entry to the sonar_translator at https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/xarray_conversion.py#L28, mirroring the existing em122 entry.
We'd probably need a sample file to ensure that is the case.
@barry-gallagher interested?

See also https://kluster.readthedocs.io/en/latest/for_developers/fordev_sonar.html

This is possible to add but we have a fair backlog on other projects so I won't promise when we'll get to it.  Probably this and #113 would get done at the same time.

Thank you for your response. Looking forward to the updates.

@tzutingchen I have this working locally, I'll need to combine with the other sonar support requests and include in a new version.  I'll report back here when that is done.

@tzutingchen This is fixed with v1.1.6.  You should be able to reinstall and get the update.


Show More Reports per Page

At the moment, only one report is shown per page:

It would be better to show more posts - 5 per page is probably sensible.
The number of posts to render per page is defined here.

just fixed it to load 5 each.

Nice one.


High memory usage of linker.evaluation.prediction_errors_from_labels_table

On our internal linking jobs, this has failed on a machine wiht 128Gb of memory, even when running:
compute_tf_tables(linker)
pipeline = CTEPipeline()
compute_df_concat_with_tf(linker, pipeline)
logger = logging.getLogger("data_linking")
logger.info("computed df concat with tf")
lab_as_sdf = linker.table_management.register_labels_table(labels_df)
prediction_errors = linker.evaluation.prediction_errors_from_labels_table(lab_as_sdf)

Particularly weird since the 'big' join condition
select blah
    from
    __splink__labels_prepared_for_joining as df_labels
    inner join __splink__df_concat_with_tf as l
    on l.person_id = df_labels.person_id_l
    inner join __splink__df_concat_with_tf as r
    on r.person_id = df_labels.person_id_r

Is essentially the same as that used in predict(), which works.
The difference is that the predict() one has
select blah
   from __splink__df_concat_with_tf as l
    inner join __splink__blocked_id_pairs as b
    on l."unique_id" = b.join_key_l
    inner join __splink__df_concat_with_tf as r
    on r."unique_id" = b.join_key_r

But in that case __splink__blocked_id_pairs is a much larger table and so should use more memory
This join is failing on a labels table with only 27,179 labels
This leads me to suspect that it's some other part of the SQL that's causing the problem
Error message is:
Error was: Out of Memory Error: Failed to allocate block of 32768 bytes (bad allocation)

Here's a test script that monitors memory usage:

import duckdb
import psutil
import time
import threading
import splink.comparison_library as cl
from splink import DuckDBAPI, Linker, SettingsCreator, block_on, splink_datasets
from splink.datasets import splink_dataset_labels

# Function to monitor memory usage in a separate thread
def monitor_memory_usage(interval=0.05):
    process = psutil.Process()
    while monitor_thread_active:
        memory_info = process.memory_info()
        print(f"Memory usage: {memory_info.rss / (1024 * 1024):.2f} MB")
        time.sleep(interval)

# Start the memory monitoring thread
monitor_thread_active = True
monitor_thread = threading.Thread(target=monitor_memory_usage, daemon=True)
monitor_thread.start()

# Your existing script
con = duckdb.connect()
con.execute("SET memory_limit = '90MB';")
db_api = DuckDBAPI(connection=con)

df = splink_datasets.fake_1000

settings = SettingsCreator(
    link_type="dedupe_only",
    comparisons=[
        cl.JaroWinklerAtThresholds("first_name", [0.9, 0.7]),
        cl.JaroAtThresholds("surname", [0.9, 0.7]),
        cl.DateOfBirthComparison(
            "dob",
            input_is_string=True,
            datetime_metrics=["year", "month"],
            datetime_thresholds=[1, 1],
        ),
        cl.ExactMatch("city").configure(term_frequency_adjustments=True),
        cl.EmailComparison("email"),
    ],
    blocking_rules_to_generate_predictions=[
        block_on("first_name"),
        block_on("surname"),
    ],
    retain_matching_columns=True,
    retain_intermediate_calculation_columns=True,
)

linker = Linker(df, settings, db_api)

# Execute your functions as usual
linker.training.estimate_probability_two_random_records_match(
    [block_on("first_name", "surname")],
    recall=0.7,
)

linker.training.estimate_u_using_random_sampling(max_pairs=1e6)

linker.training.estimate_parameters_using_expectation_maximisation(
    block_on("first_name", "surname")
)

linker.training.estimate_parameters_using_expectation_maximisation(block_on("dob"))

pairwise_predictions = linker.inference.predict(threshold_match_weight=-10)

df_labels = splink_dataset_labels.fake_1000_labels
labels_table = linker.table_management.register_labels_table(df_labels)

splink_df = linker.evaluation.prediction_errors_from_labels_table(
    labels_table, include_false_negatives=True, include_false_positives=False
)
splink_df.as_pandas_dataframe()

# Stop the memory monitoring thread
monitor_thread_active = False
monitor_thread.join()

print("Memory monitoring completed.")

This seems to be fixed by upgrading to pip install duckdb==1.0.1.dev4341 from 1.0.0


Is there a plan for rendering trees of mathmatics?

I just want to know can I render such a tree with Katex?

Sorry, no. The closest thing in KaTeX would be the CD environment.  We've had other requests for more powerful diagramming packages, but that sort of thing would require a very large programming effort. Of course, any contributions would be welcome, but I know of no one who plans to pursue this.


Test each minor version of the CodeQL CLI in PR checks

Opening this for discussion: this PR ensures that we test the latest patch version of each minor version of the CodeQL CLI we support.  This gives us better assurance that the versions of the CodeQL CLI we claim to support function correctly with the Action.  However, it does lead to an increase in CI jobs on a repo where the number of CI jobs is already becoming blocking due to concurrency limits.
Let's discuss whether we want this or not, and if we don't want it, let's come up with a principled way of deciding what versions of the CodeQL Bundle we test in PR checks.
Merge / deployment checklist

[x] Confirm this change is backwards compatible with existing workflows.
[x] Confirm the readme has been updated if necessary.
[x] Confirm the changelog has been updated if necessary.

Thanks for putting this up.
A point of discussion is that we now have 250 checks that run for each PR. This change added 60 new checks. Each check is fairly small and runs reasonably quickly. Every new PR check we add will add 24 new checks (previously, they would add 18).
I don't actually know if there are any real limits that we will hit if we keep on increasing the number of jobs at the rate.
I do know that an external contributor who ran the actions checks in a fork reached a limit and had quite a few failing jobs in their fork.

That's a really good point.
There is an org-wide limit of 500 actions jobs running concurrently I believe, so maybe we need to refactor/throttle how we do these checks before merging this.

OK...so we're not at the 500 checks yet, but we're inching towards it. I wonder if we can combine some of our PR checks into a single job. We'd still need to matrix them, but at least, in theory, we could run all the checks for actions version XYZ on windows in a single job.
We'd need to change the workflow generator to produce that.

Another option is reconsidering whether we have some checks that it'd be sufficient to run using a single bundle or a limited set of bundles, for instance the "Analyze: 'ref' and 'sha' from inputs" check.

I would like to get this in, but I think it's worth doing the other cleanups first so we don't make merging PRs any more painful.  I'm on vacation Friday and next week but will return to this when I get back.

I'm on vacation as well. I'll see if there's someone else who can take this over. Thanks for your input so far, @henrymercer.

Closing as stale, will pick up when there's capacity.


[Question] Editor integration, how to compile / minify JSHint source

Using the following contents in e.g. main-entry,js
import { JSHINT } from 'jshint';

JSHINT(`console.log("JSHint this!")`);
console.log(JSHINT.data());

followed by webpack -p main-entry.js results in a ~2.6MiB output file. Where the expected production bundle is around the ~1.2MiB. What am I missing?

We don't maintain any code that's specific to webpack, so the webpack maintainers might be better able to help. I'd start by inspecting the output bundle to get a sense for which files have been included.


ConfusedBy attribute

How to disable ConfusedBy attribute added to obfuscated assemblies
[module: ConfusedBy("Confuser.Core 1.2.0+4110faee9d")]

Hello, take a look at this post.

Please see: #70


No HTML generated from XDITA map and topics in 3.1.2

Expected Behavior
An xdita map and topics should generate html5 files the same way it is done with regular DITA 1.3.
Actual Behavior
The XDITA build runs with no errors or warnings in the log, and exit code 0. But no HTML files are generated except for index.html, which has no content.
(By the way I had to install the XDITA DTDs before the build would run at all.)
Possible Solution
I'm not sure how to fix it. If you give me a clue where to look, I will try for a fix.
Steps to Reproduce

Install XDITA DTDs to 3.1.2 OT.
Run XDITA map and topic from gist below with this 3.1.2 OT.
Inspect error-free log, and missing HTML output.
https://gist.github.com/markgif/a1e689ed50f786fd4c7909e8d00d76ea

Copy of the error message, log file or stack trace
https://gist.github.com/markgif/623ee1d848482a20fbec67a66604460e
Environment

DITA-OT version: 3.1.2
Operating system and version: Windows 7
How did you run DITA-OT? oXygen 20.1
Transformation type: html5

The problem is probably this one:
https://github.com/oasis-tcs/dita-lwdita/issues/16
All topicrefs in XDita are marked by default as resource only.

@raducoravu, I would like to see @markgif confirm this before it's closed.

@lief-erickson -- the issue @raducoravu identified would definitely have this result - he reported the defect in XDITA doctypes yesterday, and Carlos has already checked in a fix (not sure if he's created a new release yet). I think it's appropriate to close this item, as the defect is in the external OASIS product that doesn't yet ship with DITA-OT.
(I'm not sure if Radu verified the issue in the other tracker before Carlos closed it -- it doesn't look like any commit is attached to the issue.)

Encountered in some random internal tests same problem as @markgif, in a way this illuminates us about just how many people are using lw dita maps.

This DTD fix works, thanks @raducoravu !! And thanks @jelovirt for a basically sound lwdita plugin.

Will create a new release as soon as DITA TC approves/discusses my fix (created an entity for processing in the LwDITA map mod file)


Improve resiliance to ASGI protocol violations

There were a few cases in which TraceMiddleware could fail, all related to the ASGI application and/or server nastily violating the ASGI protocol.

Codecov Report

Merging #12 into master will increase coverage by 3.78%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #12      +/-   ##
==========================================
+ Coverage   89.13%   92.92%   +3.78%
==========================================
  Files           4        5       +1
  Lines          92      113      +21
==========================================
+ Hits           82      105      +23
+ Misses         10        8       -2

Impacted Files
Coverage Δ

tests/utils/asgi.py
100% <100%> (ø)

tests/test_integration.py
100% <100%> (ø)
:arrow_up:

ddtrace_asgi/middleware.py
85.96% <100%> (+4.83%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f23d2dc...55ae6a0. Read the comment docs.


Stop CSS JS

Should be able to stop CSS and JS with a user command
i.e. /?url=www.20q.net&stopcss=true
(20q.net page is always refreshing to the main page with js)
little tweaking with onpagerequest might do this

I added a little script with 'eight' that replaces 'location.replace' string of parsed page to 'location_dont_replace' to obfuscate the self-reloading javascript function. This fixes scraping 20q.net. This behavior will be active by default.
This issue will remain closed until a similar issue happens again.


Tweak install instructions

Trying to get to the bottom of the failing os tests.

Ok @szymon319 it looks as though this is a problem that's to do with Python 3.9 and MacOS.  It's a new version of Python and it's possible that there are some weird incompatibilities.  But it seems to install perfectly fine on 3.8, which is definitely good enough for now!
I'll open an issue with a suggestion for refactoring your testing a little bit, that will help with running the unit tests.

Ok @szymon319 it looks as though this is a problem that's to do with Python 3.9 and MacOS.  It's a new version of Python and it's possible that there are some weird incompatibilities.  But it seems to install perfectly fine on 3.8, which is definitely good enough for now!
I'll open an issue with a suggestion for refactoring your testing a little bit, that will help with running the unit tests.


AWSAccelerator-Pipeline errors due to quota limits

Describe the bug
The first time you deploy an LZA as per the implementation guide the "Deploy" stage fails.
This is because you have 3 steps executed in parallel - Network_Prepare, Security and Operations. Usually the "Security" step fails with Error calling startBuild: Cannot have more than 1 builds in queue for the account (Service: AWSCodeBuild; Status Code: 400; Error Code: AccountLimitExceededException; Request ID: <request-id>; Proxy: null).
I think in the prerequisites it should be mentioned that service quotas L-2DC20C30 & L-4DDC4A99 should be increased. Or it should be mentioned that it's possible for the "Deploy" stage to fail so it should be monitored and the failed steps in it should be retried.
So to TL;DR: This issue is to signal that by default failures occur on deployment so the service quota limits that needs to be handled should be explicitly mentioned in the guide either as prerequisite or as something that could potentially occur.
To Reproduce
Deploy LZA.
Expected behavior
No failures due to service quota limits if not mentioned in the guide.
Please complete the following information about the solution:

[x] Version: v1.5.2
[x] Region: eu-west-1
[x] Was the solution modified from the version published on this repository? No
[x] Have you checked your service quotas for the services this solution uses? Yes but after I hit the error
[x] Were there any errors in the CloudWatch Logs? Yes

Screenshots
n/a
Additional context
n/a

Hi @grumpper , this has been added to our Implementation Guide here so I'll be closing this issue. We appreciate you bringing this to our attention, thanks again for supporting the Landing Zone Accelerator!

Note: This can be altered via global-config.yaml @grumpper
limits:
  # Update Codebuild concurrency on the AWSAccellerator Pipleine
  # https://docs.aws.amazon.com/solutions/latest/landing-zone-accelerator-on-aws/prerequisites.html#update-codebuild-conncurrency-quota
  - serviceCode: codebuild
    # Concurrently running builds for Linux/Large environment
    quotaCode: L-4DDC4A99
    desiredValue: 3
    deploymentTargets:
      accounts:
        - Management

Noting that this issue is still occurring for me:
service quota 'Concurrently running builds for Linux/Large environment' L-4DDC4A99 limit: 15
LZA version: 1.8.1
BTW, Looks like in this version almost all concurrency limits related to codebuild have been bumped to 15.


make common validator

Coverage decreased (-0.06%) to 82.166% when pulling 8d5fcf579a76659446b8deec48243b278b70e721 on vmotso:accreditation_level into ca7276485a10e01cac48f5f0e226cb85f71cb594 on openprocurement:master.

Coverage decreased (-0.06%) to 82.166% when pulling 8d5fcf579a76659446b8deec48243b278b70e721 on vmotso:accreditation_level into ca7276485a10e01cac48f5f0e226cb85f71cb594 on openprocurement:master.


Major faults, how to solve them

</div>

Traceback (most recent call last):
File "/root/MasterHamsterKombatBot/cs.py", line 2321, in
main()
File "/root/MasterHamsterKombatBot/cs.py", line 2315, in main
asyncio.run(RunAccounts())
File "/root/MasterHamsterKombatBot/cs.py", line 2269, in RunAccounts
account.Start()
File "/root/MasterHamsterKombatBot/cs.py", line 1830, in Start
self.configData = AccountConfigVersionData.get("config", {})
AttributeError: 'NoneType' object has no attribute 'get'


Checklist: getting JULES.jl ready to merge with Oceananigans.jl

X-Ref: https://github.com/CliMA/Oceananigans.jl/issues/605
Opening this issue to keep track of what needs to be done before we think JULES.jl and its CompressibleModel can be merged into Oceananigans.jl.
We just released Oceananigans v0.38.0 yesterday whiles JULES.jl is relying on Oceananigans v0.24.1 so I'll do some updating first then see what needs to be done.
Probably want JULES to be running on the GPU before merging.

Hmmm, actually not sure whether merging the two packages is the best approach... But might as well work some more on JULES.jl and worry about whether to merge or not later.

Hmmm, maybe this issue can be a checklist of milestones before we have something interesting enough to consider submitting a JOSS paper?
Abstractions for base states (#50) and maybe a first microphysics scheme might be nice since it's developed for atmospheric applications, but we can think about that once the current list of milestones are exhausted.


Add battery types to the BOM

I guess from your video the main battery is some 6S lipo type.
Can you also add the electronics and remote battery types?

I'll add the batteries to the BOM. Yes the main battery is 6S but it's quite high drain, so I have 60C battery(s).
The smaller one powering the electronics is 7.4v - I'll find a link for it, but anything over 6v will do.


Create a Station Page 1.0

🖥 Frontend Issue
Description

Create a station display page.
Steps to Reproduce

Go to '...'
Click on '...'
See error '...'

Expected behavior

when click on station button on hero page, station selection page should be opened.
Actual behavior

button no working
Screenshots or Video

Are you participating in GSSoC?

[X] Yes
[ ] No

Assigned @Premkolte can you arrange this alphabetical??

Okk I'll do it in next pr. When I'll add more stations...


Upgrade dependencies

(Well, only one dependency so far. Will push individual commits here to get separate CI results.)
Testing Plan: Dev server

I discovered a SimpleBar-related regression, so don’t merge that yet.

These are great, merged, thanks @andersk!


chore(Calculus/ParametricIntegralInterval): small clean-ups

collect some very common variables
use refine and \mapsto instead of refine' and => (both are preferred now)

Easy to review. Best reviewed commit by commit (each says what it does). Preparatory cleaning for #10004.

bors merge


Add invitation to reuse code to the README

Changes in this PR

highlight code in DSI namespace
convert some links to the reference format

 Promote shareable code in README


There is no OutputHeader command

Convimg does not contain a command that will allow the user to output a particular string header at the start of an Appvar. This would allow programs to easily search for an Appvar using Detect(). This was a command convPNG included but convIMG does not.

I don't think that this is a useful enough command to implement, without a very particular reason. I'm open to the idea, but I still need a use case that shows why it would be useful.

1d934a0bd40c79019260a3d61db66e2b2df81cf6 introduced header-string (see here)

test

Yeah it should work now with the latest release


Add libsqlite3-dev to Ubuntu

Gem install is failing for me on Bitrise because libsqlite3-dev isn't installed.

Also Bitrise is shipping an out of date rubygems which is breaking bundle install.
Example issue:

https://github.com/sickill/rainbow/issues/44

I've updated the PR to install the latest rubygems.

Thanks for the PR @bootstraponline !
Just curious, what do you use sqlite dev for?

Just curious, what do you use sqlite dev for?

Instructure has a RubyGem that uses sqlite for integrating with Google's gerrit code review tool. I noticed our builds failed on bitrise due to missing headers / out of date RubyGems.

I've addressed all the comments. If there's anything else I should change, let me know.

Wow, the lib version print is pretty awesome @bootstraponline , thank you!
For reference, this is the current lib versions list, from the CI, printed by this modified system_report.sh:
=== Pre-installed lib versions ========
* zlib1g-dev: 1:1.2.8.dfsg-2ubuntu4
* libssl-dev: 1.0.2g-1ubuntu4.6
* libreadline6-dev: 6.3-8ubuntu2
* libyaml-dev: 0.1.6-3
* libsqlite3-dev: 3.11.0-1ubuntu1
=======================================

👍

Release schedule: will be deployed this weekend, as part of the weekly VM update for the Linux/Android (rolling release) stack.


This is just for practice puropse

this is to learn how the issue works and practice with it

This issue is for practice purpose only


Add an EmsFolder#vm_folder? method indicating if a folder can have vm children

Add a EmsFolder#vm_folder? method returning if a folder is able to
have child vms.
This checks if the folder is or has as an ancestor the hidden "vm"
folder.  All other folders are considered "yellow" and are unable to
have vms as children.
Fixes https://github.com/ManageIQ/manageiq/issues/19104

cc @Fryguy @skateman

So I didn't want to call this blue_folder? because that is very VMware specific, and if you haven't used the old J# vSphere Client you wouldn't even know what a blue/yellow folder is.
I settled on vm_folder? to indicate if the folder can be a parent to VMs but I'm open to other ideas.

cc @kbrock as you've has some experience here as well

LGTM, aside from my one comment.


[FEAT] Foldable Tabs / Sections in main.qvs

The script sections are all merged into one main section, and each part annotated with the tab name e.g. below.
It would be helpful to have a config setting allowing the choice of this behaviour or keeping the script structure intact with a separate file per section.
///$tab Contact Details
[Contact Details]:
LOAD ContactDetail,
ContactType,
Employee_ID;
//ID;
SQL SELECT ContactDetail,
ContactType,
Employee_ID,
ID
FROM Contact Details;

Thank, we will try to extend the Language file for region support. To have something like this:

Just as a sidenode. The intention is to have all script in one file, to make for example an full copy&paste of a script easy.

Thanks - now own view personally is that a choice would add value, as in a very complicated data model, having 1000's of lines of script code brought together sort of defeats the purpose of structuring the script anyway ;-)

because of no endtags for regions folding is only possible via:
https://code.visualstudio.com/api/language-extensions/programmatic-language-features
https://microsoft.github.io/language-server-protocol/specification#textDocument_foldingRange


Migrate Translation SDKs to Test Proxy

Description
Migrates azure-ai-documenttranslator and azure-ai-translation-text to Test Proxy.
azure-ai-documenttranslator was also updated to the V1.0 stable Swagger definition to match what was done in .NET. V1.0-preview wasn't able to run tests in the repository (unsure if that endpoint is no longer valid).
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-ai-documenttranslator


feat(dashboard): Refactor FiltersBadge

SUMMARY
Functional changes:

Filters badge won't be displayed if cross-filter isn't actually applied to given chart (before the filters badge would display all cross filters, even those incompatible)
Unset and incompatible native filters are not displayed in the filters badge anymore
The filters badge popover opens on hover instead of on click

Visual changes:

Restyled the filters badge popover

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before:

After:

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up FEATURE_DASHBOARD_CROSS_FILTERS=true

Alignment here looks weird. Should the lens icon be displayed in a non-hover color instead of being fully invisible? In that way the alignment would look better. CC @kasiazjc

I think the divider should be 1px. It's too thick. Also in the design the values are aligned to the right.

Rest looks good!

@eschutho Thanks for testing. Different styling of the tooltip is intentional - the original version would look ugly if chart's name was very long. It's going to be redesigned in the near future

Alignment here looks weird. Should the lens icon be displayed in a non-hover color instead of being fully invisible? In that way the alignment would look better. CC @kasiazjc

I think the divider should be 1px. It's too thick

Rest looks good!

Thanks Diego, that definitely makes sense! I'm going to test the non-hover state with Kamil.


Fresh Debian Install - dbus errors

Just did a new instal on Debian and dbus seems to be throwing errors when using the example start code:
(node:7903) UnhandledPromiseRejectionWarning: Error: No introspectable
    at /usr/share/nodejs/test/node_modules/dbus/lib/bus.js:129:15
(node:7903) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:7903) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

This happens during:
bluetooth.init().then(async ()=>{
  // listen on first bluetooth adapter
  const adapter = await bluetooth.getAdapter('hci0');
  await adapter.StartDiscovery();
  console.log("Discovering");
});

Is the user a member of the bluetooth group?

Thanks the user is root, there is no bluetooth group !

In case it helps, this happened to me too. I just took down the adapter and brought it back up again..
sudo hciconfig hci0 down
sudo hciconfig hci0 up

Worked after this running as root


Not showing any results for Mozambique.

I just entered to see the contributors and is not showing.

Now it's working.


Fix completion for scripts run via python

Current coverage is 88.10% (diff: 100%)

Merging #191 into master will not change coverage

@@             master       #191   diff @@
==========================================
  Files             4          4
  Lines           597        597
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            526        526
  Misses           71         71
  Partials          0          0

Powered by Codecov. Last update 8d87830...b0ed99a


Milights are working perfectly but RM bridge is set to scan mode but still wont work./

root@orangepizero:~/.homebridge# homebridge
*** WARNING *** The program 'node' uses the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see http://0pointer.de/avahi-compat?s=libdns_sd&e=node
*** WARNING *** The program 'node' called 'DNSServiceRegister()' which is not supported (or only supported partially) in the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!

Scan this code with your HomeKit App on your iOS device to pair with Homebridge:
┌────────────┐
│ 031-45-154 │
└────────────┘

[2017-8-30 01:36:40] Homebridge is running on port 51826.
Discovered Broadlink RM device at <IP_ADDRESS> (0d:43:b4:fc:1e:a2)
[2017-8-30 01:37:38] [Broadlink RM] Scan RF (scanning)
[2017-8-30 01:37:38] [Broadlink RM] [Hold down the button that sends the RF frequency]
[2017-8-30 01:37:40] [MiLight] [Bedroom] Setting power state to on
[2017-8-30 01:37:40] [MiLight] [Bedroom] Setting brightness to 100
Broadlink RM not able to scan anything. What am I missing ?

           {
                    "platform": "BroadlinkRM",
                    "name": "Broadlink RM",
                    "hideScanFrequencyButton": false,
                    "hideLearnButton": false,
                    "accessories": [{
                            "name": "RM Pro Learn",
                            "type": "learn-code",
                            "host": "<IP_ADDRESS>",
                            "disableAutomaticOff": true,
                            "scanFrequency": true
                    }]
            }

Entry in my config.json file which I have to manually add

I have the same problem, I have a RM 3 mini, my config is similar without the host....I try to scan the hex from my remotes and get nothing...did you found the solution for your problem?

@nunesdennis I also had the same problem. But my RM PRO, the one that supports IR+RF.

@nunesdennis does the RM 3 mini have RF support? I didn't think it did...
@Leafney have you tried explicitly creating a Learn RF button with the host set as @dhruvsahni showed above?

@lprhodes you are right, it does not =)


Publish a new release?

@aembler what about publishing a new release?
The latest one (1.2.0) still provides the old concrete5/doctrine-xml name and I guess that may cause problems if people use concretecms/doctrine-xml instead of concrete5/doctrine-xml (as suggested by https://packagist.org/packages/concrete5/doctrine-xml)

PS: the same also for https://github.com/concretecms/monolog-cascade (where GitHub issues are not enabled, that's why I'm writing here)

@aembler ConcreteCMS still have two concrete5/... composer packages dependencies:

monolog-cascade: it has already been renamed from concrete5/monolog-cascade to concretecms/monolog-cascade, but we don't have a release for it yet (so, we need to tag a new version)
oauth-user-data, which still doesn't have a concretecms/... counterpart yet. So, or we leave it as it is now, or we have to mark it as abandoned, rename its name in composer.json, and tag a new release

For concrete5 v8 we also have zend-queue that needs a new 0.9.* version to be tagged


[DO NOT MERGE] MIOpen immediate mode integration

This is the final (barring code review feedback updates) version of the code to integrate MIOpen Immediate Mode into the TF layer.
Notes

CI runs will continue to fail until we upgrade to ROCm 2.6.
The failure will be because the new Immediate Mode API is not present in the header files included in ROCm 2.5 and lower.
best to review this PR commit by commit
Immediate Mode replaces the existing "Find" mode, (i.e. there is no way to switch back to "Find" mode) Assumption is that once this PR gets merged, we will say bye-bye to Find mode (atleast as far as the TF Layer is concerned)

@whchung @jerryyin

@deven-amd @jerryyin , what's the plan to merge it?
We can use "Dropbox" approach to install a compatible MIOpen2.0.1 deb in Dockerfile.rocm to unblock this PR.

@sunway513 , we may need to take the "dropbox" approach.
We were waiting for ROCm 2.7 to merge this PR, but with ROCm 2.7 being DOA for TF, we need to decide whether to wait for ROCm 2.8 or take the "dropbox" approach.
Let me first update this PR to resolve all the merge conflicts that have arisen, and then update it to use the "dropbox" approach. In the meantime, we can figure out whether or not we should wait for ROCm 2.8.

Thanks @deven-amd , I don't think we would have to wait for ROCm2.8 - since the dependency is on MIOpen and it's online now. From my experiences, MIOpen 2.0.1 is compatible with ROCm2.6 user-bit environment.

@deven-amd , can we merge this PR now? The CI now runs on ROCm2.8.

@sunway513 working on this now.
Resolving all the merge conflicts is non-trivial (lot of changes in the routines changed by this PR, XLA upstreaming related changes, bfloat16 support changes, etc).

Closing out this PR as I have opened a new one for the same


How to remove a list item from fzf?

Category

[ ] fzf binary
[ ] fzf-tmux script
[x] Key bindings
[ ] Completion
[ ] Vim
[ ] Neovim
[ ] Etc.

OS

[ ] Linux
[x] Mac OS X
[ ] Windows
[ ] Windows Subsystem for Linux
[ ] Etc.

Shell

[ ] bash
[x] zsh
[ ] fish

0
I have the following script that pipes git stash list through fzf:
git stash list\
    --color=always\
    --pretty=format:'%Cred%gd%Creset - %s%C(yellow)%d%Creset %Cgreen(%cr) %C(bold blue)' |\
  fzf\
    --ansi\
    --tiebreak=index\
    --header "<enter> view stash | <alt-enter> apply stash | <alt-d> drop stash | <ctrl-p> toggle preview window | <esc> quit"\
    --preview "echo {} | grep -o 'stash@{[0-9]}' | head -1 | xargs -I % sh -c 'git show --color=always %' | diff-so-fancy"\
    --preview-window "hidden"\
    --bind "ctrl-p:toggle-preview"\
    --bind "alt-d:execute(echo {} | grep -o 'stash@{[0-9]}' | xargs -I % sh -c 'git stash drop %')+abort"\
    --bind "alt-enter:execute(echo {} | grep -o 'stash@{[0-9]}' | xargs -I % sh -c 'git stash apply %')+abort"\
    --bind "enter:execute: (grep -o 'stash@{[0-9]}' | xargs -I % sh -c 'git stash show -p --color=always %' | diff-so-fancy | less -R) << 'FZF-EOF'
  {} FZF-EOF"

I'd like to hit Alt+d and execute the following line but without abort:
`--bind "alt-d:execute(echo {} | grep -o 'stash@{[0-9]}' | xargs -I % sh -c 'git stash drop %')+abort"\
However, I would like to selected item to no longer display in the list.
Is this possible? Can you show me how?

No, it's not possible. In that case, I suggest that you use the traditional method of restarting fzf with the updated list in a loop in a shell script instead of using execute bindings. You can determine which key was pressed by setting --expect=alt-d.


key-bindings.bash: better support multiline commands

This makes it possible to see multiple lines of a multi-line command with the C-r keybinding when using FZF_CTRL_R_OPTS to provide a multi-line preview window, as done in https://github.com/junegunn/fzf/issues/577#issuecomment-473241837.

Thanks, but there are a few points against this approach.

We can't assume that the system has gawk installed. Mine didn't, and I had to install it using brew.`__fzf_history__`-bash: gawk: command not found

We could add a conditional branch, yes, but I'd really like to keep the implementation simple and minimal so that the users can take a look at it and quickly understand how it works.
For some reason, setting HISTTIMEFORMAT makes history much slower, that almost makes CTRL-R unusable for me. Tested on bash 3.2 and 5.0.> echo $BASH_VERSION
5.0.3(1)-release

> time HISTTIMEFORMAT="" history | wc -l
   96513

real    0m0.220s
user    0m0.129s
sys     0m0.201s

> time HISTTIMEFORMAT="foo" history | wc -l
   96514

real    0m16.960s
user    0m2.827s
sys     0m14.310s

This could be a bug of bash, but whatever the reason may be, we can't ship this version.

Hi, I've patched the code to address your concerns. PTAL.

Thanks for the update, but I'm still not sure if I want it because of the additional delay.
jg@mbp2018:/private/tmp> time HISTTIMEFORMAT="" history > /dev/null

real    0m0.128s
user    0m0.090s
sys     0m0.038s
jg@mbp2018:/private/tmp> time HISTTIMEFORMAT="" history | awk -f proc.awk > /dev/null
awk: proc.awk:16: (FILENAME=- FNR=88613) warning: Invalid multibyte data detected. There may be a mismatch between your data and your locale.

real    0m0.338s
user    0m0.417s
sys     0m0.050s

The difference is quite noticeable to me. Since I have very few multi-line commands in my command history, I do not like that I have to wait another 100ms every time I hit CTRL-R for no real benefit. So this comes down to personal preference. If you use multi-line commands frequently, I can imagine that you would want proper support for multi-line commands even if you have to wait a little longer, but that's not the case for me.
Also, I'm seeing some issues.
jg@mbp2018:~/.vim/plugged/fzf/shell (master)> echo "foo
> bar"
foo
bar
jg@mbp2018:~/.vim/plugged/fzf/shell (master)> echo "foo
> bar
> baz"
foo
bar
baz
jg@mbp2018:~/.vim/plugged/fzf/shell (master)> `__fzf_history__`awk: cmd. line:17: (FILENAME=- FNR=88613) warning: Invalid multibyte data detected. There may be a mismatch between your data and your locale.
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  97342  ### ###.###
  97343  #### ####
  97344  #### #######
  97345  ####
  97346  #### #######
  97347  #### #######
  97348  ### ######.###
  97349  ## #########/
  97350  ## ..
  97351  ### ######.###
  97352  ### ######.###
  97353  ## -## ######/
  97354  ## ###
  97355  ..
  97356  ####### ######/######.###
  97357  ## ######/
  97358  ##
  97359  #### .
  97360  #### __###_#######__
  97361  echo "foo bar"
>
  Press CTRL-Y to copy command into clipboard
  97362/97362 +S
>
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Having said that I'm leaning towards not merging it to the main repository and publishing the code as a plugin/extension in a separate repository (see my previous comment on that matter: https://github.com/junegunn/fzf/pull/1161#issuecomment-347002445).

Yes, that seems like a good idea to me. In any case, I just found out that macOS awk doesn't allow you to print out NUL at all, so I'd have to either use gawk anyway or implement some more complicated escaping scheme. Is there a particular format for the "plugins" that I should follow?

Fixed in #1837


Init

TODO:

Change contract addresses in Allbonds.ts (DAI -> flexUSD, ETH -> BCH, LPs)

Opened in wrong repo, sorry, mybad.


Youtube Playlists not returning latest videos

Describe the bug
Many youtube playlists add new videos to the end, not the beginning.
For example: https://www.youtube.com/playlist?list=PLaDrN74SfdT7Ueqtwn_bXo1MuSWT0ji2w
This shows the oldest 15 videos in the playlist, in reverse chronological order.
https://feed.eugenemolotov.ru/?action=display&bridge=Youtube&context=By+playlist+Id&p=PLaDrN74SfdT7Ueqtwn_bXo1MuSWT0ji2w&duration_min=&duration_max=&format=Html
To Reproduce

Put playlist ID into a youtube-bridge
see that it doesn't return the newest videos.

Expected behavior
I would like to be able to get the newest videos out of a playlist instead of the oldest.
Additional context
There was a similar bug report #647 that playlists aren't sorted by date, and that was fixed by #1162, but the example playlist in that puts new videos at the start of the playlist, not the end.
So playlists are now returned in reverse chronological order, but it's still the first X videos off the top of the playlist not the bottom.

Just made some investigation.
This is not working anymore and returns 0:
https://github.com/RSS-Bridge/rss-bridge/blob/1e75f9d3d558cb9dd91aab36a61cdc81f1c8dcb3/bridges/YoutubeBridge.php#L249
So it just fallbacks for fetching videos from youtube's videos.xml, which returns only first 15 videos from playlist.

Hello, @mtfurlan !
FYI, I have just pushed partial fix for this bug. It works correctly, if playlist has less than 100 videos.

Nice, thank you!
I do see it working, I just forgot to reply earlier.

This is not fixed yet. Only partial

There was a similar bug report #647 that playlists aren't sorted by date, and that was fixed by #1162, but the example playlist in that puts new videos at the start of the playlist, not the end.

I think adding a "Reverse order" toggle could be a workaround for this issue (Reversing the order and picking the entries off the top for any playlists where the playlist creator puts newer videos at the top).

I think adding a "Reverse order" toggle could be a workaround for this issue (Reversing the order and picking the entries off the top for any playlists where the playlist creator puts newer videos at the top).

Could you please provide, where did you find "reverse order" toggle?

Could you please provide, where did you find "reverse order" toggle?

I didn't find one. I suggested adding one to the YouTube bridge in order to work with playlists with the videos in reverse order.

i think a "Reverse ordering" checkbox in the bridge form can solve this nicely. if anyone has the courage to touch the youtube bridge code


fix(app_notifications): add toggle notification menu on second click

Description
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] Any dependent changes have been merged and published in downstream modules

Task linked: CU-86c0u82bh Toggle notification menu on second click

:tada: This PR is included in version 2.130.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


S3 gateway refuses connections

What happened?:
When attempting to interact with the S3 gateway, connections are being refused. It looks like pachd isn't actually listening on the s3gateway-port.
What you expected to happen?:
Ability to interact with S3 gateway using the documented requests.
How to reproduce it (as minimally and precisely as possible)?:
$ pachctl deploy microsoft <...> # Seems to happen in other targets as well
$ pachctl port-forward --namespace=<...>
Forwarding the pachd (Pachyderm daemon) port...
Forwarding the SAML ACS port...
Forwarding the dash (Pachyderm dashboard) UI port to http://localhost:30080...
Forwarding the dash (Pachyderm dashboard) websocket port...
Forwarding the PFS port...
Forwarding the s3gateway port...
$ curl localhost:30600
E0423 16:29:23.257477   41044 portforward.go:391] an error occurred forwarding 30600 -> 600: error forwarding port 600 to pod <POD_UID>, uid : exit status 1: 2019/04/23 20:29:23 socat[26205] E connect(6, AF=2 <IP_ADDRESS>:600, 16): Connection refused

Anything else we need to know?:
It looks like the port is forwarded through the service, but not exposed on the container. From pachctl deploy local --dry-run -o yaml:
Service:
spec:
  ports:
  - name: s3gateway-port
    nodePort: 30600
    port: 600
    targetPort: 0
  - ...

Pod (nothing on 600):
ports:
  - containerPort: 650
    name: api-grpc-port
    protocol: TCP
  - containerPort: 651
    name: trace-port
  - containerPort: 652
    name: api-http-port
    protocol: TCP
  - containerPort: 653
    name: peer-port
    protocol: TCP
  - containerPort: 999
    name: api-git-port
    protocol: TCP
  - containerPort: 654
    name: saml-port
    protocol: TCP

Environment?:

Kubernetes version (use kubectl version): 1.13.4
Pachyderm CLI version (use pachctl version): 1.9.0rc1 (pachctl and pachd)
Cloud provider (e.g. aws, azure, gke) or local deployment (e.g. minikube vs dockerized k8s): Azure

Thanks for the details, it's really helpful.
Did you first deploy pachyderm at v1.8.x, and then upgraded to 1.9rc1? If so, that would cause it.
Also, heads-up that s3gateway only works when enterprise mode is enabled. The docs are supposed to reflect this, but a few updates looks like they're still needed, and the docs haven't been rebuilt recently. I'll go ahead and fix that now.

@ysimonson Thanks for that info. As you guessed, I didn't know that enterprise mode had to be enabled. I'm going to do some more playing around (e.g. enabling enterprise mode, for starters) and I'll update this issue accordingly.

I tried a few things (in succession, not all at once):

Deleted all old Kubernetes resources, including etcd PV's and PVC's and redployed from scratch
Enabled enterprise mode – I tried this both via Dash and pachctl (get-state reported ACTIVE)
Edited the --dry-run output and kubectl apply -f'ed it with 600 exposed on the pachd container
Built master@HEAD and tried the same things in case the tagged release was missing something
kubectl port-forward directly to the pachd pod on port 600

In all cases, the connection was refused, though I have no issue interacting with the other exposed ports (i.e. I get 404 HTTP responses from some services, and some binary looking stuff out of GRPC).
If you can't reproduce any of this behaviour then I can just chalk it up to me doing something totally wrong. But any insight would be greatly appreciated.

Could you try pachctl port-forward, and then visiting http://localhost:30600/ on the existing deployment?

Right, that was actually the first approach I used for every configuration I tested. You can under the How to reproduce... section of this issue's description. curl assumes HTTP protocol, but I also tried using the exact URL you provided.
Each configuration gives similar "Connection refused" error output as described in the How to reproduce... section.

I don't know if this is related, but the documentation's recommended way of connecting to the S3 gateway doesn't seem to be implemented.

You can start the s3gateway locally by running pachctl s3gateway. Then point your browser or
favorite s3 tool at http://localhost:30600. The s3gateway also runs in the background on your
pachyderm cluster, which you can reach via http://:30600.
Note: A third way to access the s3gateway is using the explicit port forwarder (pachctl port-forward).
However, we don’t recommend it, as kubernetes’ port forwarder incurs overhead, and does not
recover well from broken connections.

I get the following:
$ pachctl version --client-only
1.9.0rc1
$ pachctl s3gateway
Error: unknown command "s3gateway" for "pachctl"
Run 'pachctl --help' for usage.
unknown command "s3gateway" for "pachctl"

Thanks for catching that, that's just yet another place where the docs are still catching up to the latest s3gateway changes. We did have that functionality, but removed it since it's always running in-cluster anyways. I've got an open PR to fix that docs references as well as the aforementioned ones.
I haven't managed to repro the issue you're seeing yet when kicking the tires locally, but am going to try an azure deployment next and see what happens.

Docs-related fixes are in. I deployed a new pach cluster (1.9.0rc1) on azure, and wasn't able to repro the issue when connecting via port forwarding. @brendangadd any other clues that you've come across in the meantime?

@ysimonson Awesome on the doc updates. At this point, I have to assume there's something weird lurking in the many layers of virtualization sitting under pachd which is particular to my cluster. Thanks so much for all the effort to reproduce the behaviour!
I'll see how I fare with a new vanilla cluster. I'll close this issue and re-open if I turn up something that is indeed Pachyderm-specific.
Thanks again!


First Pass - Pilot Variant Painting data analysis

Goal:

What proportion of variants and which ones show a signal relative to their WT?

Basic Analysis using mean profiles:

Using correlation coefficients (by Marzieh)

[x]   Replicate correlation + null distributions
[x]   list of correlation coefficient scores for each pair

Using MAP (by @yhan8)

[ ]   Replicate MAP + null distributions
[ ]   list of map scores for each pair

train a classifier to distinguish a particular variant from its WT? WTgene1_well1 vs VARgene1_well1 and so on for all combinations of replicate wells for each. (I guess at the single cell level?)

We will do this at the single-cell level but will build a single model, so I wasn't sure what you mean by "all combinations of replicate wells". Maybe you are referring to the way we do train-test splits? If so, yes, we'd want to factor in the experimental hierarchy in some way when splitting.

caveat: such classifiers may always seem effective (due to plate layout, slight changes in infection efficiency/expression, etc). So, we want to get a sense of what level of classification accuracy is significant.
to create such a null we can use replicates of all samples as the baseline. So the null pairings would be WTgene2_well1 vs WTgene2_well2 and so on for all the WT genes and similarly for all the variants?

This sounds sensible

(fancy extra detail) We could exclude the query gene itself in creating this null baseline - maybe that’s unnecessary if there are ~382 other samples on the plate. If there are lots of variants of one gene on a plate then we may want to do this step.

No need to exclude the query gene when creating the null in this manner, because our null hypothesis is that the wells are arbitrarily assigned WT and MUT labels. The fancy thing to do would be to have a separate null for each WT-MUT pair, where we only consider the wells of the WT-MUT pair and shuffle their labels. But that is an overkill

(ruled out alternative) for each gene’s null we could instead try to train a classifier to distinguish replicates of only the query gene itself: WTgene1_well1 vs WTgene1_well2 but this will likely always yield ‘successful’ classifiers due to plate layout effects.

Oh, maybe this is the same as my fancy idea right above. We can pay closer attention to the design of the splits when we actually do the experiment.

Furthermore, I don’t see why the LUAD case is any different than the WT/VAR case of Variant Painting experiments so I would also say Yes to your query there that the same approach is appropriate there.

I think you are right

I made a schematic. Sort of obvious now that I drew it out so I don't expect to spark major insight here, but adding a link to google slide  in case it helps anyone think through things.

This makes sense

To address your first comment, I agree it's sensible to build a single model that distinguishes WTgene1 (all wells) vs VARgene1 (all wells) but what I was proposing was different :D I propose actually training a bunch of small classifiers on each pair, like single cells in WTgene1_well1 vs VARgene1_well1 and so on with well2, well3, etc. I should've been more clear and said "all pairs of replicate wells" instead of "all combinations of replicate wells".
One reason to do this pairwise across individual wells is to put error bars on classification accuracy, I guess. But mainly to make it easier to calculate a realistic null because now we can use two replicates of a sample that we KNOW should look alike (WTgene2_well1 vs WTgene2_well2 and so on for all the WT genes and similarly for all the variants). I guess a downside of this approach, though, is that WTgene2_well1 vs WTgene2_well2 is likely to always be same-well-position whereas the query test WTgene1_well1 vs VARgene1_well1 is not :(
Still, if we instead make a single classifier for each WT-MUT, those values will almost certainly always seem to be accurate classifiers (due to technical variations) so to decide if they are significant, we need a suitable null. To make its null we need to get WTs with a similar  number of replicates  and similar number of single cells to be fair (?). And maybe choose those having similar plate positions? I dunno.

Some questions/notes:

My understanding is that we don't really care about overall phenotype impact score on the space of whole perturbations in the experiment. Instead, we care about the score on "if there is any consistent (across all cells) signature for WT versus mutant"? For example a 100% score means that there exist a phenotype that exists for all single cells of WT versus mutant (In contrast to the previous way of unbiased score given the full space, in which 100% (or 1) meant a signature that is in average the most distinct across all WTs and VARs of an experiment). Let me know if there is any flaw in my understanding.

If the above is correct and we indeed care about a phenotype that is consistent across single cells for WT versus VAR, we should be careful of the following:

The heterogeneity of the samples: we had a huge amount of heterogeneity across the samples of the Taipale Lab rare diseases datasets. I have not looked into subpopulation analysis data for the new VarChamp batches to have an idea of if this has changed in the new experiments. But wanted to give you a heads up given this prior knowledge.

Number of cells versus number of features for the classification problem, again this maybe is something that doesn't hold in the new batches of data as all the cells are now transfected but in the old batches we had small number of single cells for many wells and we should be careful about (n of features)>> (n of samples) which cause overfitting.

Position effect: in the first pilot batch of data that we analysed for VarChamp the plate layout was the same. If the position effect is strong, it is problematic for any method of scoring, but more problematic for a classifier which cares about all single cells in a well having a phenotype that other well dont have.

The overall amount of computational complexity we add to the problem (by using single cells) versus what we gain.

I make comments on the null after I fully understand your suggestions but for now wanted to give my two cents on this thread.

Yes, your understanding is correct!
Your bullet points are beautifully helpful. Yes, we do anticipate heterogeneity. Essentially every MUT will be distinguishable (classifiable) from its WT  just due to technical variations so we have to be careful how to set the null to know which ones are really distinguishable.
I agree with the rest of the points too.


D3 Zipline: Interactive Scatterplot

The nature of this D3 Zipline has yet to be determined. Please respond to this issue with ideas.

Claiming this data viz zipline.

@cdikibo Please describe what you intend to do with it.  We can't just have people calling "dibs" without some sort of description of what they intend to do.

@SaintPeter Sorry for my "dibs" behavior.
My original idea has been scrapped after the only way to get my dataset was to use XPath and Selenium AKA things that FCC is not about. My scrapped idea consisted of using chlorpeth map to visualize people killed by police. It was going to have user stories to highlight when searched by different cats eg gender. Plus it was going to have bottom that would pick random dataset, highlight it, give a name to that dataset and a link to an article.
Since there's already D3 Zipline using a map. I will come back with a new dataset(user stories with codepen link) that I will visualize in a non-map way.

@cdikibo OK - sounds good. Thanks for the further clarification. Please keep us posted on how this is coming along, as January 1 is fast approaching.

I have decided on an interactive scatterplot for this zipline.   I hope that choice adds to the diversity of the D3 ziplines as a whole.

@cdikibo Great! I agree - this will be a nice addition, since scatterplots are a staple form of data visualization :)

I also have a scatterplot I'm working on, I'll post the link here when it's reasonable to review and you guys can see what you think.

Here is my mostly finished scatterplot!   If it works for the curriculum great, if not I learned a lot while doing it and am a little better at D3 now than I was before.
http://codepen.io/AdventureBear/full/jWrbqB/

Like I had mentioned in the chat room, I think an idea of showing a camper one to three types of examples for each visualization (scatterplot) in this case could be helpful, but I'm not sure if that fits the current curriculum model. If cdikibo's zipline is used for the curriculum, mine could be used as an example during the instructive portion perhaps, or vice versa.   I didn't want to step on anyones toes, but I had already been working on this one since discovering the existence of the data and brainstorming about D3 projects for FCC.

I think it looks fab!  It just needs a legend or something so we can understand what it means to those of us who are not bike racing fans.

Thanks, I'll add that tomorrow.  select.append.circle...

Hi @AdventureBear,
Could you add the legend and link me to your dataset? We'll host it.
This looks amazing.
I'd like to start recording videos tomorrow :)

Yes I'll do all that, I'm just updating my JSON file so it's more accurate with the variation of the distances normalized.  It changes the order of the finishers a bit.

here's a gist of the updated data file
https://gist.github.com/AdventureBear/b616fd2a5c3e58e19999

otherwise it's pretty done, with exception to tinkering with the tooltip, but I think this is "good enough"

@AdventureBear excellent! I'll upload the dataset and update the import path on the official codepen fork of this! Thank you so much for your hard work on this.


Could you add DirectML version of S-D.cpp, please?

...

A new backend is more of a feature request for the ggml repo.


[master] latency fix for transparent mode

Reason for Change:

Issue Fixed:

Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests

Notes:

/azp run


Default placeholder is not populated

Which version and edition of Flyway are you using?
6.2.3, 6.2.4
Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)
Java API
Which database are you using (type & version)?
Postgres 11.6
Which operating system are you using?
Ubuntu
What did you do?
Little repo to reproduce my issue.
I try to get default placeholders in my migration scripts as it is described here.
Here is sample java main:
    public static void main(String[] args) {
        String schemaName = "test_schema";
        Flyway.configure()
            .dataSource(
                "jdbc:postgresql://localhost:5432/postgres",
                "postgres",
                "mysecretpassword")
            .locations("db/migrations")
            .defaultSchema(schemaName)
            .schemas(schemaName)
            .load()
            .migrate();
    }

And my scripts:
create table A(schema_name varchar);
insert into A values('${flyway:defaultSchema}');

What did you expect to see?
Placeholder was substituted by value test_schema.
test_schema string inserted into table A.
What did you see instead?
INFO: Current version of schema "test_schema": << Empty Schema >>
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to parse statement in db/migrations/V1__create_table.sql at line 1 col 37: Failed to populate value for default placeholder: ${flyway:defaultSchema}

I am seeing this same problem under Flyway 6.1.0.
It seems that the placeholder doesn't get filled in if the schema doesn't yet exist. I find that PostgreSQL (9.6.15) returns an empty string for SELECT current_schema if the schema does not yet exist, even if it is the sole member of search_path. (Flyway executes this select in PostgreSQLConnection::doGetCurrentSchema.)
My Java application fails to start the first time I run it, but before it fails, Flyway creates the schema and does some migrations, before hitting one needing flyway:defaultSchema. If I run the application again, it stays up, because the schema exists now.

What I'm seeing is that PostgreSQLConnection::doGetCurrentSchema() returns public rather than test_schema, so in the above example table A is populated but not with what I'd expect. I'm thinking that this placeholder value would be more correctly substituted by the defaultSchema config parameter rather than the current schema, which as we've seen may not exist or may be the wrong one. What do you think?

Fixed in c2297307f4bc056ea29a5e4d32f7eab8f39c81a4


Fill page "getting started"

The page "getting started" must be modified and refilled in order to explain the user path starting with Luos:

Big picture
Installation
First development
...

Fix with #14 Index page


[ru] Localize Policy as Code (PaC)

Description
This Issue is to localize Policy as Code (PaC) into Russian.
Checklist before opening your PR (put x in the checkboxes) — note that you have to add this checklist to your PR and check all the boxes for it to be approved.

[] This PR uses the dev-ru branch as a source/target.
[] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

ETA: TBD
Useful links:
Как внести свой вклад
Руководство по стилю
Slack channel
Telegram channel

I'd like to work on this issue


Add Neon Solitude sheet

Neon Solitude is a custom system, so it's not associated with any other system/sheet. Thanks! 😄

Thanks @pjbehr87 ❤


Incorrect symbol resolution

Right now when you use Ruby's master branch, it's always resolving yp_parse to the one found in Ruby, and not the one found in YARP. This means our CI is now starting to fail because when we push changes it uses the old one to test and fails. We need to change our linking to favor our own symbols over the ones found in Ruby.

Taking a look this morning, and I'm realizing (remembering) that -load_hidden is not available on linux (it's a mach-o/darwin feature). So I'm going to explore a couple of other ideas.

PR at #1069.


fix(out_of_lane): reduce "Same Points are given" messages

Description
Cherry pick https://github.com/autowarefoundation/autoware.universe/pull/5145
Tests performed

Psim
Effects on system behavior

Less debug messages from the behavior_velocity_planner .
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[ ] I've confirmed the contribution guidelines.
[ ] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@mkuri @TomohitoAndo
Autoware立ち上げると "Same points are given" がめっちゃ出る件のhotfix PRです、挙動の変更は一切なしなのでマージします


specify annotation-discrete-coices component

the goal is to merge this with #213 if possible
beyond what is described there, the component should:
Purpose
Let me annotate a generic categorical variable, i.e. one that no vocabulary has been standardized for or where I want to make up my own terms as I go along.
Use cases

I just want a data dictionary for this categorical variable, so that I can make a BIDS dataset. So all I want is to write a description for each level of the values. The UI element that displays the description lets me click it and I can now edit the description

input

value levels (object)

keys are column names
values are objects with unique column value as the key, and the current description as the value

output

mapping of unique value to description (columnName, value, description)

I am willing to map my unique value but I want to enter it myself as a URL (only URL pattern validation)

input

...

output

...

There are at least three possibilities that I can think of as to how this component can pick up its discrete choices – and for the annotation page layout to know to use this specialized annotation component:

The 'levels' values of the input data dictionary file determines that this component will be used on the annotation page
The user explicitly specifies the type of specialized component to be used to annotate the column in question.
The app config yaml file allows users to specify which specialized components get assigned to which column.

The latter possibility is one we have only briefly discussed, @surchs. There is the question of where this could occur. If not automatic, the user could select this in the column linking table of the categorization page, or just simply provide a dropdown in the annotation tab itself that swaps out specialized components.


If several parsers enabled how to select needed?

I have some files with .md and .rst formats.
Is it possible to bind parsers by extension?

I know this has been open for a while, but Markdown Preview does not support .rst format.  It is not meant nor planned to support .rst format.
Now to answer the question about whether a parser can be bound to a specific extension; this doesn't really make sense when the only type of document you parse is Markdown.  I personally have no interest in this feature, and sense there has been no comments from anyone else on this project, I suspect there is no interest from other contributors.
I will close this issue as there is no actionable item.


Add Suport for Firefox Sync

It would be nice, if the number of days are automatic synced with other Computers.

In case the addon someday becomes available in Firefox Android, please sync the settings there as well if possible.

Firefox Sync support was actually added a few years back (in 2020, actually, but it's not automatic. In the latest version, the sync buttons (along with import/export buttons) are under the Sync category in the popup.
The reason it's not automatic is that if you're syncing your browsing history, then History Cleaner only needs to be enabled on one device, and the deletions will sync automatically. So I want to support the use case of having History Cleaner enabled on one device, but disabled on another.


libp2p Requirements Doc - Roadmap to 1.0.0

Moved the libp2p items out of https://github.com/mgoelzer/ipfs/blob/master/ROADMAP-TO-1.0.0.md and into this file that will live in libp2p/libp2p
(Note:  after doing a bit of research, there really isn't any good way to do a table with checkboxes inside the cells in Github flavored markdown.  This is a known issue in Github.  The best workaround puts extra vertical whitespace in every cell.  So, instead of using the three column approach, I just triplicated each item for Go, JS and Rust.)
@jbenet @diasdavid
(Note:  once this PR is merged, https://github.com/ipfs/ipfs/pull/334 should be merged to remove the content from the ipfs/ipfs/ROADMAP-TO-1.0.0.md document.)

@mgoelzer you missed porting:

https://github.com/ipfs/ipfs/blob/master/ROADMAP-TO-1.0.0.md#ipfs-dht
https://github.com/ipfs/ipfs/blob/master/ROADMAP-TO-1.0.0.md#ipfs-pubsub

@mgoelzer any ETA to get this doc finished?

@diasdavid What do you think about merging in the current state so we can iterate?  I fixed up all the fruits and table structure.  The initial description section lists some known deficiencies in the current doc.
Alternative is to keep iterating in this PR.  Either works for me.
Also, these items are intentionally missing:

@mgoelzer @diasdavid a few thoughts:

By Node, do we mean Host in libp2p terminology?
Personally I consider the enumerated technologies/protocols listed as concrete implementations rather than interfaces.
Do we have criteria or a definition for done for each module? If not, I wonder if the term available would be more appropriate, as it conveys that an implementation exists, but does not set any expectations on feature completeness. To me, done also implies some kind of closure, which may be applicable in the context of specific feature sets we aim for with 1.0 for each module.

By Node, do we mean Host in libp2p terminology?

It is actually the other way. Host is a legacy term, the correct term is libp2p Node.

Personally I consider the enumerated technologies/protocols listed as concrete implementations rather than interfaces.

Agreed. Same that I was going with https://github.com/libp2p/libp2p/pull/39#issuecomment-414979622

[screenshot]

@diasdavid Yes, I had the same reaction as you. impl(emented) is already implied for every row in the table, unless otherwise stated, and the word interfaces doesn't make much sense in the section headings.  I (intentionally) left those alone because I wanted a verbatim copy of the original text in https://github.com/mgoelzer/ipfs/blob/master/ROADMAP-TO-1.0.0.md, thus making this PR a pure move-and-reformat with no change in semantics, which tend to be non-controversial.
That said, if the terms have no meaning to you, then they probably aren't of significance to anyone else.  I'm fine with taking them out.

I pushed a new commit fixing to remove "interface", "impl" and "implementation" per https://github.com/libp2p/libp2p/pull/39#issuecomment-414979622.
Did not change "Node" to "Host" per https://github.com/libp2p/libp2p/pull/39#issuecomment-415019885
Did not change "Done" to "Available" based on consistency-with-IPFS argument in https://github.com/libp2p/libp2p/pull/39#issuecomment-415110288 (I'm open to changing this later though)
I'd like to merge this, merge https://github.com/ipfs/ipfs/pull/334, and then we can iterate on this one and let IPFS iterate on its analogous list.  In subsequent PRs, I'd like to (1) verify the accuracy of the fruits, (2) fill in Performance Milestones TODO item with 10^6 and 10^9 node goals, and (2) add some docs requirements line items.

@diasdavid Forgot to respond to this:

@mgoelzer you missed porting:
https://github.com/ipfs/ipfs/blob/master/ROADMAP-TO-1.0.0.md#ipfs-dht
https://github.com/ipfs/ipfs/blob/master/ROADMAP-TO-1.0.0.md#ipfs-pubsub

I have another branch that adds on to this PR where I have included those.  My next PR will be based off of that, with some additional stuff mentioned above.


Public de::buffer API for buffering self-describing formats.

ℹ Note, this text has been updated based on the subsequent discussion.
This PR adds a public de::Buffer (formerly de::Content) API. It partially addresses issue #741.
Use Cases: The existing private Content structure is currently used for buffering when deserializing, e.g., tagged enums. However, it would also be useful outside of that context. I have two concrete use cases in mind: (1) Deserialization of structures where the exact type of certain parts is dynamically determined and (2) implementations of Deserialize which are not generated via the derive macros (e.g., by another code generator or manually written).
In fact, I am currently working on a tool which generates implementations of Deserialize from a schema. Unfortunately, replicating the functionality of Content in a third-party crate in an efficient way does not seem possible. Internally, Serde relies on __deserialize_content for an efficient implementation of Content.
Public API: Instead of exposing the internal details of Content, this PR introduces an API which mostly implements already public APIs (Deserializer, IntoDeserializer and Visitor) for opaque newtype wrappers around private structures. In theory, this should allow for Content to evolve while being committed to only a very thin public API most of which is already public anyway. At the same time, the provided API should be enough for both use cases.

Content has been renamed to Buffer capturing that the purpose of the type is buffering during deserialization.
Both Buffer and &Buffer can be turned into a respective Deserializer using the IntoDeserializer trait.
Buffer exposes a public as_str method which tries to extract a &str from the buffer. This is required by the code generated by serde_derive and for both use cases. Note that this method cannot be implemented externally.[^1]
There are various From implementations for constructing a Buffer. These implementations are generally syntactic sugar for BufferVisitor::visit_$type(value).unwrap(). However, they guarantee that a buffer can be constructed from the respective types while visit_$type could, in principle, return an error.

With these changes, the code generated by serde_derive does no longer depend on buffering internals but solely on the new public de:Buffer APIs. This means that external crates and code generators have the same interface available for custom Deserialize implementations which require buffering.
[^1]: One might think that BufferRefDeserializer<'a, 'de, E> could be used for this purpose. However, it calls visit_str for the String buffer, thereby erasing lifetime 'a from the string, and making it impossible to return a &'a str from the visitor.

I guess a similar approach should be possible for ser::Content as well.
I am looking forward to your thoughts and feedback regarding this proposal. I am willing to see this (or any alternative suggestion) through. In any case, I believe a public de::Content API would be valuable.

Thanks for the quick response! I am glad you are on board with the general approach.

From your description of your use case, it's not clear where all those From impls become needed.

They are not needed as it is always possible to construct the respective instances of Content via the visitor. In fact, I see them as mere syntactic sugar for the visitor, i.e., for ContentVisitor::new().visit_$type(value). As I see them as mere syntactic sugar, I feel that they do not add any additional commitments in terms of the API while adding a bit of  convenience for constructing Content (e.g., in something like TagOrContentVisitor). However, as you hinted at, my use case does not depend on that, so not having them is also perfectly fine with me.

Is it possible to deduplicate some of the trait impls related to Content between serde::de::content and serde::private::de?

Yes, I think, this will be possible.

I would want someone to brainstorm names other than Content.

How do you feel about Buffer (or alternatively DeBuffer for deserialization buffer)? I think Buffer would convey the purpose of the structure quite well – as you said, it is about buffering.
Another option maybe something with Any. However, it is not actually true that Content can be anything which could be deserialized if the concrete type were known because it is somewhat limited to self-describing formats. Yet another option maybe something with Value – however, de::value is already taken. Furthermore, both these options make it seem as if the structure is intended to be used for more than just buffering.

@dtolnay I finally got the time to address your points.

Content is now named Buffer and placed in de::buffer.
Everything Buffer specific from private::de::content has been moved to de::buffer preventing duplication.

Unfortunately, I am unable to figure out how to fix the issues with Rust version <= 1.25. I will leave a few comments indicating other issues which we need to address before this can be merged.

After giving the as_str method and From implementations a bit more thought, I think they are worthwhile. I updated the first post with an explanation as to why I think this is the case. From my side, this PR is ready to be merged now.
I am looking forward to your feedback.

I just tried to resolve the conflicts so that this PR is mergeable again. However, now the tests fail because trybuild appears to use another, now incompatible, version of serde_derive – which I do not know how to fix. 😕 I would like to avoid adapting the PR to new changes over and over again.
What is the best way for moving forward with this PR? Is there anything I can do?


Modify the variance of base distribution in RQSpline

Add argument to scale the covariance matrix of base distribution.

@alicjapolanska In the newest release and #123, the normalizing flow interface received a major rehaul. It is now possible to pass a custom distribution in, including a Gaussian distribution with a different variance.
As I am updating the documentation for the API, here is an example of how you can change the variance of the base distribution:
Consider this example gotten from test_normalizingFlow.py:
    key1, rng, init_rng = jax.random.split(jax.random.PRNGKey(0), 3)
    data = jax.random.normal(key1, (100,2))
    num_epochs = 5
    batch_size = 100
    learning_rate = 0.001
    momentum = 0.9
    model = MaskedCouplingRQSpline(2, 4, [32,32], 4 , rng, data_mean = jnp.mean(data, axis=0), data_cov = jnp.cov(data.T))
    optim = optax.adam(learning_rate, momentum)
    train_flow, train_epoch, train_step = make_training_loop(optim)
    rng, best_model, loss_values = train_flow(
        rng, model, data, num_epochs, batch_size, verbose = True
    )
    rng_key_nf = jax.random.PRNGKey(124098)
    model.sample(rng_key_nf, 10000)

we can replace line
model = MaskedCouplingRQSpline(2, 4, [32,32], 4 , rng, data_mean = jnp.mean(data, axis=0), data_cov = jnp.cov(data.T))

with
base_dist = Gaussian(jnp.zeros(n_features), jnp.eye(n_features)*0.5, learnable=False)
model = MaskedCouplingRQSpline(2, 4, [32,32], 4 , rng, data_mean = jnp.mean(data, axis=0), data_cov = jnp.cov(data.T), base_dist=base_dist)

Would you mind trying out the new API and let me know whether this works for your use case?