added binned class and tests

This pull request integrates a binning class Binned in the conversion module, that provides a representation of binned (clipped, unclipped) spike trains based on sparse matrices, which is an important feature for large sets of spike trains in combination with fine-grained binning. This binning class therefore acts more as a specialized data type within elephant. The advantage of this class is that it takes care of efficiently handling the sparse matrix management for the user (optionally keeping the “non-sparse” matrices in memory) and handles the clipping, but also keeps the time axis in order (similar to the AnalogSignal class of Neo). Please note: The existing binning function in elephant (conversion.binarize) has intentionally not been touched in this PR.

Coverage decreased (-0.42%) to 99.1% when pulling 0464916b3a34d5cbd31dfe5fefa05069294cef81 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

Coverage decreased (-0.42%) to 99.1% when pulling 0464916b3a34d5cbd31dfe5fefa05069294cef81 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

Some general comments on this PR.
(1) Alper, I'm sorry if some of the comments appear like nit-picking, but I think this is a very important class, which will be widely used within Elephant, so I think it's important to get it right.
(2) Second, a comment on method names. The Binned class (aka TimeHistogram if you like my latest name proposal!) seems very much like a scipy sparse matrix with some extra methods. I therefore think the method names should match the scipy names as closely as possible, e.g. toarray() instead of sparse_mat_unclip() and tosparse() instead of sparse_mat_unclip().
(3) The names "clipped" and "unclipped" are very unintuitive to me. Without carefully reading the code and documentation, I would assume that "clipped" applied to the time axis, i.e. you were clipping values off the start and/or end of the time histogram. I'm not yet sure what a better name would be. Also, given that obtaining the "clipped" version is as simple as abs(scipy.sign(self.mat_unclipped)) (actually, I'm not sure the abs() is desirable here), are the "clipped" methods really necessary? Finally, if we decide to keep the "clipped" methods, wouldn't it be more memory efficient to return a boolean array rather than an integer array containing only 1s and 0s?
(4) Giving the option to store the real array seems like a premature optimization. Do you already have use cases where profiling shows that converting the sparse array to a dense one is a major bottleneck?

@apdavison 1) I can understand that. Sometimes it is a little bit painful, but I hope it assures the quality of elephant. On the other hand I also learn a lot by discussion this things.

I can change the names, it is just a short refactorisation command. But I wonder if we are bound to use the numpy naming schemes, I can't remember that. Or is just convention because we use a lot of numpy functions?

Maybe I am biased, but in our lab if I say clipped or unclipped matrix everybody understands ^^  If there are better suggestion that makes it more readable or intuitive I am happy to change the name.
The abs function is needed not to get negative 1's.  See in the test_conversion the test_binned_neg_times_list method.
I would like to have a "clipped" version of the matrix, although it is just a simple command. At least it is more consistent that way.

3.5) Why should be a boolean array more efficient than an integer bit mask?

I do not have a performance profiling for the conversion. To build the dense matrix it should be in O(n^2), whereas a method call is one operation (assuming it was calculated before). Then storage is of course O(n^2).
The idea here was ,that it is possible to pass around the class (to other functions) and to have everything inside if needed. The other option would be, that the user stores locally the dense matrix in a variable and passes the variable and the class if needed.

But I wonder if we are bound to use the numpy naming schemes, I can't remember that. Or is just convention because we use a lot of numpy functions?

We are not bound to do it. I was suggesting it as a policy, as I think it will make code more consistent and hence easier to understand. @toddrjen do you have any comments on this?

The abs function is needed not to get negative 1's. See in the test_conversion the test_binned_neg_times_list method.

Ah, I see. This is for negative spike times.

Why should be a boolean array more efficient than an integer bit mask?

The mat_clipped property gives an integer array containing only 1 and 0:  e.g. [1 1 0 1 1 1 1 0 0 0] I'm not very familiar with NumPy internals, but I would expect this to take 32 or 64 bits per element, whereas [True True False True True True True False False False] should take only one bit per element, no?

I do not have a performance profiling for the conversion. To build the dense matrix it should be in O(n^2), whereas a method call is one operation (assuming it was calculated before). Then storage is of course O(n^2).

The idea here was, that it is possible to pass around the class (to other functions) and to have everything inside if needed. The other option would be, that the user stores locally the dense matrix in a variable and passes the variable and the class if needed.

I understand this, the question is whether n is ever large enough to make it worthwhile to store the dense matrix rather than regenerating it every time. The major cost of storing the dense matrix is increased code complexity; I was just wondering if you'd made any performance measurements to show at what values of n this increased complexity becomes justified.

Maybe I am biased, but in our lab if I say clipped or unclipped matrix everybody understands ^^ If there are better suggestion that makes it more readable or intuitive I am happy to change the name.

I'd like to have other comments on this. My concrete proposal would be:
  sparse_mat_unclip --> tosparse()
  sparse_mat_clip --> astype(bool).tosparse() or tosparse().astype(bool)
  mat_clipped --> toarray()
  mat_unclipped --> astype(bool).toarray() or toarray().astype(bool)

It seems numpy defaults to int32, at least on 32-bit systems.  It may be
int64 on 64-bit systems, but I am on my smartphone right now and it us
32-bit so that is all I can check.
However, due to how memory is addressed in computers, 1-bit values are not
technically possible. There are some hacky ways to make something like a
1-bit bool array, but numpy does not do that.  The standard bool, which
numpy uses, is 8-bit.  That is still a factor of 4 memory savings (or maybe
8 on 64-bit systems).

asfoo is the standard syntax in numpy.  I think it would be better to stick with that naming scheme.

numpy uses a mixture of asfoo (e.g. astype) and tobar (e.g. tolist, tostring).

Oops, you are right.

sparse_mat_unclip --> tosparse()
sparse_mat_clip --> astype(bool).tosparse() or tosparse().astype(bool)
mat_clipped --> toarray()
mat_unclipped --> astype(bool).toarray() or toarray().astype(bool)

I am not 100% happy with this naming.

sparse_mat_unclip --> tosparse()

This indicates a transformation (to a different object structure), which is not true. Since sparse_mat_clip is a getter and a property, something like get_sparse or just sparse_matrix would be more appropriate.

sparse_mat_clip --> astype(bool).tosparse() or tosparse().astype(bool)

Looks like method chaining. Same argument as before. get_sparse_asBool or sparse_matrix_asBool would be my proposal.

mat_clipped --> toarray()

Here it is not clear what is going to be an array. How about matrix_toBoolArray or matrix_asBoolArray? Likewise matrix_toArray or matrix_asArray?
I also would prefer to write mixed case, i.e., asBoolArrayinstead of asboolarray <- looks weird.

sparse_mat_unclip --> tosparse()

This indicates a transformation (to a different object structure), which is not true.

It is true. The object is a TimeHistogram, the method returns a scipy sparse matrix; this is a transformation.

Since sparse_mat_clip is a getter and a property, something like get_sparse or just sparse_matrix would be more appropriate.

I think it should be a method, not a property.

sparse_mat_clip --> astype(bool).tosparse() or tosparse().astype(bool)

Looks like method chaining. Same argument as before. get_sparse_asBool or sparse_matrix_asBool would be my proposal.

Yes, this is method chaining. I don't see a reason for having separate methods/properties to return integer ("unclipped") and boolean ("clipped") versions when the boolean version can be obtained so trivially.

mat_clipped --> toarray()

Here it is not clear what is going to be an array. How about matrix_toBoolArray or matrix_asBoolArray? Likewise matrix_toArray or matrix_asArray?

Sorry, that was a typo. It should be mat_unclipped --> toarray().

I also would prefer to write mixed case, i.e., asBoolArrayinstead of asboolarray <- looks weird.

Weird is in the eye of the beholder (-:  Mixed case is a major PEP8 violation. A more modern Python naming convention would be as_bool_array (the NumPy convention is a little outdated).

The object is a TimeHistogram, the method returns a scipy sparse matrix; this is a transformation.

I am still not convinced. You don't transform the class into an other structure. You request to give a sparse matrix, which is calculated inside/via the class.
Let's say we have the class bin and I call the method bin.tosparse() I would expect the returned object to be a sparse representation(?) of the class, which is not the case. What I want to say that the sparse method is not working on the class itself and changing its structure.

I think it should be a method, not a property.

Sounds ok.

Yes, this is method chaining. I don't see a reason for having separate methods/properties to return integer ("unclipped") and boolean ("clipped") versions when the boolean version can be obtained so trivially.

Ah, I thought you meant the name should look like this, that's why I said method chaining. Although the boolean version may be trivial calculated I think it should stay for convenience and it is not harming. We have such cases like the cv.

Weird is in the eye of the beholder (-: Mixed case is a major PEP8 violation. A more modern Python naming convention would be as_bool_array (the NumPy convention is a little outdated).

Agreed.

OK, compromise proposal:
  sparse_mat_unclip --> to_sparse_array()
  sparse_mat_clip --> to_sparse_bool_array()
  mat_unclipped --> to_array()
  mat_clipped --> to_bool_array()

Coverage decreased (-0.04%) to 99.49% when pulling 59ffeaf933b1d2aeeba1acead0e5ee16ed2efd47 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

Coverage increased (+0.04%) to 99.57% when pulling 58a1925dfa7aea5cf66d0a7708a6b4c4d48944c6 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

Coverage increased (+0.04%) to 99.57% when pulling 58a1925dfa7aea5cf66d0a7708a6b4c4d48944c6 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

Coverage increased (+0.04%) to 99.57% when pulling f62bb089dc92f7c728ec06d58b6066072f938f84 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

Coverage increased (+0.04%) to 99.57% when pulling f62bb089dc92f7c728ec06d58b6066072f938f84 on alperyeg:binning into 37946e29566210dce1d0b9dd972e23da631f7a21 on NeuralEnsemble:master.

@mdenker any further comment on this?
@apdavison  Michael correct me if I am wrong, I think Michael meant in later processing steps the boolean representation will be converted to int implicit internally or explicit by the user. So there could be overhead when converting from bool to int, because most of the time matrix operations will be calculated with integers.

in later processing steps the boolean representation will be converted to int implicit internally or explicit by the user.

This is what I disagree with, and why I asked for an example.

Here is a fragment of an implicit conversion example taken from the psth function, which will be PRed soon, after the binning class is available.
    if clip is True:
        bin_hist = bs.sparse_mat_clip.sum(axis=0)
    else:
        bin_hist = bs.sparse_mat_unclip.sum(axis=0)
    bin_hist = bin_hist.getA1()

It is still not adjusted to the actual changes, I just copied it. You can see the function here within the whole context.
On the other hand, is there any example which just uses the boolean representation? At some point there is always a conversion when doing (matrix) calculation.

@apdavison Yes, the question is whether it always makes sense for the class to convert int to bool, when the user then converts bool back to int.
A simple example that forms the basis of many analysis approaches, as Alper indicated above, is to calculate the spike complexity per bin assuming that individual spike trains in the object are different neurons:

a=np.array([[True,False],[True, True]])
np.sum(a,axis=0)
array([2, 1])
np.sum(a,axis=0).dtype
dtype('int64')

Of course, it works like this, but the question is whether it would not be beneficial to offer both return types separately (bool and int) for "clipped" versions.

My feeling is that the added complexity of offering two different return types is not worth it, since a bool array will give the same results as a {0,1} array. There may be a performance penalty from extra conversions, but until we have profiling data for a real-world use case, it feels like premature optimisation.
That said, I don't feel strongly about it - you should do what seems best.
On the other hand, we do really need to resolve the issue of the class name. Maybe we could have BinnedSpikeTrain as an alias or subclass of TimeHistogram, or we could drop TimeHistogram for now, and resurrect it later if need be. What will the return type of the psth() function be?

@apdavison I see your point about TimeHistogram being useful for things other than spike trains and that this is an argument to remove the semantic connotation from the name. If it would turn out that we wanted an object to represent specifically spike trains in the future (e.g., in order to retain annotations of the source Spike Train object), it could still be done as a subclass of the more general TimeHistogram object.
However, if we use that name, the elephant documentation must be very clear about how to represent a binned spike train with that class, because I fear many people will not automatically make the connection that a TimeHistogram object is designed (primarily) for that purpose

e.g., in order to retain annotations of the source Spike Train object

This is a good argument. OK, we seem to be converging on the "subclass" approach.

Well, at least that could be an idea for the future if it turns out to be useful... in general, I think a major challenge for the future will be to develop methods to work with these objects in a useful way (meaning in particular, retaining metadata info in the annotations of the Neo).

I see your point, I don't really not know what these bool->int conversion imply internally (where I am more concerned about memory duplication than execution speed in the case of large 0/1 arrays). I can live with keeping a to_bool_array() for now and adding a to_binary_array() (or similar) returning int at a later stage if it seems necessary or is wanted by users.

What will the return type of the psth() function be?
AnalogSignal. I have to adapt it to AnalogSignalArray, according to the latest discussions in the other PRs.
Should I change the name of the class to BinnedSpikeTrain and leave everything as it is?
We should definitely profile this class later on, to see if there are performance drops if there is a conversion from bool to int and to see if there are memory problems when just using int.

After various discussions I had about the naming of this class, I think we cannot make a final decision yet. Given the current input to the class are currently SpikeTrains objects, and given that the class currently cannot compute a histogram across the spiketrains, I propose to use the more specific term BinnedSpikeTrain for now, but simultaneously open a new issue regarding the naming of this function as "TimeHistogram" or other, possibly generalizing the class more (subclassing), just so we remember to be discuss again a bit later and further develop the API -- which also gives us the opportunity to collect user input regarding this name. This would allow us to restrict the terminology of "histogram" for the analysis functions that actually compute PSTH-like things for the time being.

ok with me.

Coverage increased (+0.05%) to 99.43% when pulling f3dd27572dd088e5a74ef7d1d91a536da639b5ba on alperyeg:binning into e2c5ce621575ee1b1f9aa172a080f0be12c63c61 on NeuralEnsemble:master.

Coverage increased (+0.05%) to 99.43% when pulling f3dd27572dd088e5a74ef7d1d91a536da639b5ba on alperyeg:binning into e2c5ce621575ee1b1f9aa172a080f0be12c63c61 on NeuralEnsemble:master.

Coverage increased (+0.05%) to 99.43% when pulling f3dd27572dd088e5a74ef7d1d91a536da639b5ba on alperyeg:binning into e2c5ce621575ee1b1f9aa172a080f0be12c63c61 on NeuralEnsemble:master.

@apdavison  or @mdenker can you merge this? Thanks!


Use the node-inspector on those and note here what they are.  I bet they are from old draconis or paleotest.
I use this custom mod to remap unknown nodes to reasonable equivalents, but I need to know the exact item_string for the bad nodes.  https://github.com/EdenLostMinetest/edenlost_custom/blob/main/init.lua


Update replace-an-expiring-client-secret-in-a-sharepoint-add-in.md

Q
A

content fix?
yes

New article?
no

Related issues?

What's in this Pull Request?
We had a very nasty Sev 1 support case where the customer tried to simply replace the expiring client secret with the new one and it didn't work.  We told them to follow this article exactly but apparently it had been working in the past and there was absolutely no tolerance for downtime.  Finally we got the PG to state categorically that only one client secret can be active per App Principal.  This is different from a regular AAD Application.  We feel it is very important we include this information, although feel free to change the format.  We can save a lot of trouble, time and money for the customer and support by clearly documenting the expected behavior.  Thanks you very much and don't hesitate to contact me with any questions.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/sp-add-ins/replace-an-expiring-client-secret-in-a-sharepoint-add-in.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: adambu sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/sp-add-ins/replace-an-expiring-client-secret-in-a-sharepoint-add-in.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

thx Adam, appreciated. Internal email coming up in few minutes.


Does it work on Stm32F4 ?

Hello AJ Alves.I want answer a question to you about your eFLL library.I have an stm microcontroller (STM32F429I DISCOVERY Board ) and i want do a project with fuzzy logic. Can i use your eFLL library using STMCube ide on this board for fuzzy logic projects ?

This code should work with any standard C++ compiler. I know it has been used on other STM32 controllers with IAR Workbench. Just add the code files to your project, and you can follow the examples for using the library.

Thanks @cdudzek


Restructuring 'Guides' section

This PR contains the following changes for the Guides section:

Moved the samples/ and tutorials/ folders to a new content/guides/ folder
Fixed img links
Updated sidebar to reflect new location
Wildcard redirect for these pages

@huguestennier FYI would love to get your 👀 on the redirects in particular.
@bigash42 or @jmacelroy please review and approve when able 🙏


CLI initial concept

Hey @NyxCode! This is a very rough first draft for the CLI tool we've been thinking about. It already handles all of our feature flags and the TS_RS_EXPORT_DIR environment variable
Todo

[ ] Explore a way of generating an index.ts file to reexport all the generated types

We could, after cargo test finishes, walk the output directory and create an index.ts containing export * from 'file'

[ ] Find a way to only run ts-rs tests through this CLI

Maybe through #253 and adding the -- --ignored flag to the command
Maybe we should gate the generation of tests?

We could add a feature flag to ts-rs that re-enables test generation (The entire derive could be turned into a no-op when the feature is off). This feature would always be enabled when using this CLI

[ ] When we consider it ready, it would be interesting to also make it available through cargo binstall
[ ] Discuss other features of the CLI

A configuration file so the user doesn't have to pass the flags every time?

How should the flags interact with the config file?

Should they be mutually exclusive?
Should they coexist with flags overriding the file?
Should there be a config flag to pass the path to the file?

An init subcommand to help the user create said config file?
Whatever else this CLI needs to do that isn't listed here

Feature wishlist

[ ] Customize representations (e.g number vs bigint) (@NyxCode)
[ ] Load configuration from configfile (@NyxCode)
[ ] (maybe) Combine multiple types in one file? (@NyxCode)
[ ] (maybe) generate index.ts file, re-exporting types (@NyxCode)

Checklist

[ ] I have followed the steps listed in the Contributing guide.
[ ] If necessary, I have added documentation related to the changes made.
[ ] I have added or updated the tests related to the changes made.

Awesome!
For ts-rs, we pretty often do technically breaking releases, but for most users, they are actually drop-in (unless e.g they implement TS manually). For the CLI, we probably want to get that right the first time, so I suggest we carefully plan what we want to support.
I've got a lot of open questions and ideas, but let me start with these two:

Should the library be still usable without the CLI? I think it'd be a good idea to keep cargo test working, or maybe require cargo test -- export_bindings, but to keep it usable without CLI.
Do we want to enable the *-impl features through the CLI? If we do that, then #[derive(TS)] must be a no-op normally, or code will fail to compile. Even then, if #[derive(TS)] only does something when the CLI is invoked, errors will only show up at that point. As far as I can tell, it'd make more sense for users to enable these features in the Cargo.toml.

[ ] Explore a way of generating an index.ts file to reexport all the generated types
- We could, after cargo test finishes, walk the output directory and create an index.ts containing export * from 'file'

That'd certainly work. Alternatively, the tests could just output the typescript (or/and metadata) to stdout, and the CLI could maybe combine them, or write the files (and an index.ts) itself.

Do we want to enable the *-impl features through the CLI? If we do that, then #[derive(TS)] must be a no-op normally, or code will fail to compile. Even then, if #[derive(TS)] only does something when the CLI is invoked, errors will only show up at that point. As far as I can tell, it'd make more sense for users to enable these features in the Cargo.toml.

You're right, the *-impl features should not be in the cli

Should the library be still usable without the CLI? I think it'd be a good idea to keep cargo test working, or maybe require cargo test -- export_bindings, but to keep it usable without CLI.

I think it should, for that we could either merge #253 or add a cargo feature that blocks test generarion, so users would do cargo test --features ts-rs/export or something like that

Alternatively, the tests could just output the typescript (or/and metadata) to stdout, and the CLI could maybe combine them, or write the files (and an index.ts) itself.

We'd have to find a way to separate what we want from cargo test's normal output (test result: ok. 1 passed; 0 ...)

I like that!

Awesome! I'll work on that later today, what do you think the feature should be called?

Awesome! I'll work on that later today, what do you think the feature should be called?

Oh, I'm terrible with names. But export seems reasonable :D

Do you think we should gate only the test generation behind the feature or should we make the whole #[derive(TS)] a no-op?
I considered both, but I think making #[derive(TS)] a no-op would be pretty intrusive, and potentially break lots of code (custom setups for exporting, custom TS impls with : TS trait bounds, etc.)

Makes sense! I added an extra check with the cfg!() macro to verify the feature before generating the test

[ ]  Customize representations (e.g number vs bigint)

Maybe we should revisit #94 to have a better idea of how to handle this

Getting #49359 would be pretty cool - we could parse the JSON output, and do interesting stuff with it.
$ cargo +nightly t --features ts-rs/export -- -Z unstable-options --format json 2>/dev/null
{ "type": "suite", "event": "started", "test_count": 10 }
{ "type": "test", "event": "started", "name": "export_bindings_complexenum" }
{ "type": "test", "event": "started", "name": "export_bindings_complexstruct" }
{ "type": "test", "event": "started", "name": "export_bindings_gender" }
{ "type": "test", "event": "started", "name": "export_bindings_inlinecomplexenum" }
{ "type": "test", "event": "started", "name": "export_bindings_point" }
{ "type": "test", "event": "started", "name": "export_bindings_role" }
{ "type": "test", "event": "started", "name": "export_bindings_series" }
{ "type": "test", "event": "started", "name": "export_bindings_simpleenum" }
{ "type": "test", "event": "started", "name": "export_bindings_user" }
{ "type": "test", "event": "started", "name": "export_bindings_vehicle" }
{ "type": "test", "name": "export_bindings_complexstruct", "event": "ok" }
{ "type": "test", "name": "export_bindings_gender", "event": "ok" }
{ "type": "test", "name": "export_bindings_point", "event": "ok" }
{ "type": "test", "name": "export_bindings_role", "event": "ok" }
{ "type": "test", "name": "export_bindings_simpleenum", "event": "ok" }
{ "type": "test", "name": "export_bindings_vehicle", "event": "ok" }
{ "type": "test", "name": "export_bindings_series", "event": "ok" }
{ "type": "test", "name": "export_bindings_user", "event": "ok" }
{ "type": "test", "name": "export_bindings_complexenum", "event": "ok" }
{ "type": "test", "name": "export_bindings_inlinecomplexenum", "event": "ok" }
{ "type": "suite", "event": "ok", "passed": 10, "failed": 0, "ignored": 0, "measured": 0, "filtered_out": 0, "exec_time": 0.031168 }
{ "type": "suite", "event": "started", "test_count": 0 }
{ "type": "suite", "event": "ok", "passed": 0, "failed": 0, "ignored": 0, "measured": 0, "filtered_out": 0, "exec_time": 0.0001069 }

#49359 seems kinda close. #50297, on the other hand, seems far off. That's unfortunate, since we could probably do really cool stuff with it, without being too hacky.

--show-output together with --format json seems especially cool. Any println! in the tests turns into
{ "type": "test", "name": "some_test", "event": "ok", "stdout": "hey\n" }

--show-output together with --format json seems especially cool. Any println! in the tests turns into
{ "type": "test", "name": "some_test", "event": "ok", "stdout": "hey\n" }

Now that would make it easier lol

With the "generate an index.ts" feature in mind, the CLI needs some way to get a list of all TS types and their respective .ts files. Solutions I can think of are

The tests communicate with the CLI by printing to stdout

That get's messy without --format json, I'm afraid

Maybe it'd be acceptable to require nightly for the CLI until that's stable, idk.

The CLI parses the .ts files to figure out which types are in them

Lot of duplicated work there
Alternative: We generate a comment in the first line of the .ts files, and just parse that.

The tests don't generate TS, but json instead, and the CLI does the conversion to TS
The tests generate an additional metadata file, which the CLI will parse

The tests generate an additional metadata file, which the CLI will parse

I think this might be the best way to go. Use the mutex file lock to append to a metadata file all the types' names anc paths, then parse the metadata file to generate the index.ts file

I think this might be the best way to go. Use the mutex file lock to append to a metadata file all the types' names anc paths, then parse the metadata file to generate the index.ts file

I suspect that it might be simpler to write one metadata file per test, but i'd be happy to be convinced otherwise.

I suspect that it might be simpler to write one metadata file per test, but i'd be happy to be convinced otherwise.

I agree that this would probably be simpler, but we'd have to walk the output searching for these metadata files, so we could just walk the diretory reading the ts files instead

This all kinda makes me curious - How would it look like if we tried to do that without a CLI? Gotta play with that during the weekend!

My guess is that it'd be pretty much impossible, at least the way we've implemented it so far, as it heavily relies on doing stuff after cargo test has finished

I suspect that's true. For something like example/, I think we could make it work with a static holding the current state, but as soon as there are multiple test executables, like in the ts-rs test suite, stuff get's very complicated.


Update CoC with additional event details

Project Jupyter's Code of Conduct—the base CoC for these events— mentions that events may have additional specifications to ensure that people are kept safe in real-time interactions. We'd like to add some specifics (like behavior in chat and what removal from an online event would be like) to make sure our event is safe and clear in expectations.
References:
PyCon US 2021 Code of Conduct

@trallard I think this is ready for review! Thanks in advance! 🌻

[ ] Get at least one other name on the moderators/CoC contact list
[ ] Would https://conducthotline.com/pages/about be helpful?

I have used CoC hotline in the past and it is great - we can definitely ask Thea to set this up for us

LGMT - just need a contact email


ali

ali
Originally posted by @alifkifi in https://github.com/microsoft/DataConnectors/issues/285#issuecomment-611519109

aloose


connect to external barcode scanner does not work

Hi there,
I'm using a android device which needs to connect an external barcode scanner, the scanner is an scanner box connect to the android PAD, just like the device in supermarket cashier machine computer.  tried a long time, but still not work, I wonder if there's special settings need to be done in the scanner side ?
Actually when I tried in emulator, and input characters & enter key in the computer keyboard rapidly, it seems work.
Thanks a lot !

So it works?

no, I mean it works with physical key press in the computer, but not work with scanner connected to android real device, the scanner has enter key setting in the barcode ending. How can I proceed to troubleshoot ?

Connect barcode scanner to android device
Open some text editor app
Try to scan barcode
If you dont see barcode appear as text in editor when you scan your barcode it means barcode scanner is not working properly

with text editor it's ok, I can see the barcode content, now the issue is I can't get the result with the example.

Put a breakoount on this line
https://github.com/shaxxx/flutter_barcode_listener/blob/486d3b48ac77cde40e4e7cf4926165af8215ada0/lib/flutter_barcode_listener.dart#L70
And see if it gets hit when you scan

Thanks a lot. I get the problem solved, it seems due to the external scanner feature.

One another question, from troubleshooting, the barcode callback always return lower case string, how can I keep the original value(case sensitive) ? and I put a special character "@" in the QRCode, it seems not recognized, how can I get to make special characters get captured ?
Thanks!!

Finally changed to use logicalkey instead of codepoint, it seems the codepoint way can't support special keys.
library flutter_barcode_listener;
import 'dart:async';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
typedef BarcodeScannedCallback = void Function(String barcode);
/// This widget will listen for raw PHYSICAL keyboard events
/// even when other controls have primary focus.
/// It will buffer all characters coming in specifed bufferDuration time frame
/// that end with line feed character and call callback function with result.
/// Keep in mind this widget will listen for events even when not visible.
class BarcodeKeyboardListener extends StatefulWidget {
final Widget child;
final BarcodeScannedCallback _onBarcodeScanned;
final Duration _bufferDuration;
/// This widget will listen for raw PHYSICAL keyboard events
/// even when other controls have primary focus.
/// It will buffer all characters coming in specifed bufferDuration time frame
/// that end with line feed character and call callback function with result.
/// Keep in mind this widget will listen for events even when not visible.
BarcodeKeyboardListener(
{Key? key,
  /// Child widget to be displayed
  required this.child,

  /// Callback to be called when barcode is scanned
  required Function(String) onBarcodeScanned,

  /// Maximum time between two key events.
  /// If time between two key events is longer than this value
  /// previous keys will be ignored.
  Duration bufferDuration = hundredMs})
  : _onBarcodeScanned = onBarcodeScanned,
    _bufferDuration = bufferDuration,
    super(key: key);

@override
_BarcodeKeyboardListenerState createState() =>
_BarcodeKeyboardListenerState(_onBarcodeScanned, _bufferDuration);
}
const Duration aSecond = Duration(seconds: 1);
const Duration hundredMs = Duration(milliseconds: 100);
const int lineFeed = 10;
class _BarcodeKeyboardListenerState extends State {
List _scannedCharCodes = [];
DateTime? _lastScannedCharCodeTime;
late StreamSubscription<int?> _keyboardSubscription;
final BarcodeScannedCallback _onBarcodeScannedCallback;
final Duration _bufferDuration;
var str = "";
final _controller = StreamController<int?>();
_BarcodeKeyboardListenerState(
this._onBarcodeScannedCallback, this._bufferDuration) {
RawKeyboard.instance.addListener(_keyBoardCallback);
_keyboardSubscription = _controller.stream
.where((charCode) => charCode != null)
.listen(onKeyEvent);
}
void onKeyEvent(int? charCode) {
//remove any pending characters older than bufferDuration value
checkPendingCharCodesToClear();
_lastScannedCharCodeTime = DateTime.now();
if (charCode == lineFeed) {
_onBarcodeScannedCallback.call(String.fromCharCodes(_scannedCharCodes));
  resetScannedCharCodes();
} else {
  //add character to list of scanned characters;
  _scannedCharCodes.add(charCode!);
}

}
void checkPendingCharCodesToClear() {
if (_lastScannedCharCodeTime != null) {
if (_lastScannedCharCodeTime!
.isBefore(DateTime.now().subtract(_bufferDuration))) {
resetScannedCharCodes();
}
}
}
void resetScannedCharCodes() {
_lastScannedCharCodeTime = null;
_scannedCharCodes = [];
}
void addScannedCharCode(int charCode) {
_scannedCharCodes.add(charCode);
}
void _keyBoardCallback(RawKeyEvent keyEvent) {
checkLetterStatus(keyEvent);
if (keyEvent is RawKeyDownEvent) {
str = str + keyValue(mCaps, keyEvent.data.logicalKey);
}
if (keyEvent.data is RawKeyEventDataAndroid) {
  if ((keyEvent.data as RawKeyEventDataAndroid).keyCode == 23) {
    _onBarcodeScannedCallback.call(str);
  }
}
// if (keyEvent.logicalKey.keyId > 255 &&
//     keyEvent.data.logicalKey != LogicalKeyboardKey.enter) return;
// if (keyEvent is RawKeyUpEvent) {
//   if (keyEvent.data is RawKeyEventDataAndroid) {
//     _controller.sink
//         .add(((keyEvent.data) as RawKeyEventDataAndroid).codePoint);
//   } else if (keyEvent.data is RawKeyEventDataFuchsia) {
//     _controller.sink
//         .add(((keyEvent.data) as RawKeyEventDataFuchsia).codePoint);
//   } else if (keyEvent.data.logicalKey == LogicalKeyboardKey.enter) {
//     _controller.sink.add(lineFeed);
//   }
// }

}
@override
Widget build(BuildContext context) {
return widget.child;
}
@override
void dispose() {
_keyboardSubscription.cancel();
_controller.close();
RawKeyboard.instance.removeListener(_keyBoardCallback);
super.dispose();
}
//********************************************************** */
//判断大小写
var mCaps = false;
void checkLetterStatus(RawKeyEvent keyEvent) {
if (keyEvent.data.logicalKey == LogicalKeyboardKey.shiftRight ||
keyEvent.data.logicalKey == LogicalKeyboardKey.shiftLeft) {
mCaps = keyEvent is RawKeyDownEvent;
}
}
String keyValue(bool caps, LogicalKeyboardKey logicalKey) {
if (logicalKey == LogicalKeyboardKey.digit0) {
return caps ? ')' : '0';
} else if (logicalKey == LogicalKeyboardKey.digit1) {
return caps ? '!' : '1';
} else if (logicalKey == LogicalKeyboardKey.digit2) {
return caps ? '@' : '2';
} else if (logicalKey == LogicalKeyboardKey.digit3) {
return caps ? '#' : '3';
} else if (logicalKey == LogicalKeyboardKey.digit4) {
return caps ? '$' : '4';
} else if (logicalKey == LogicalKeyboardKey.digit5) {
return caps ? '%' : '5';
} else if (logicalKey == LogicalKeyboardKey.digit6) {
return caps ? '^' : '6';
} else if (logicalKey == LogicalKeyboardKey.digit7) {
return caps ? '&' : '7';
} else if (logicalKey == LogicalKeyboardKey.digit8) {
return caps ? '*' : '8';
} else if (logicalKey == LogicalKeyboardKey.digit9) {
return caps ? '(' : '9';
} else if (logicalKey == LogicalKeyboardKey.numpadSubtract) {
return '-';
} else if (logicalKey == LogicalKeyboardKey.minus) {
return caps ? '_' : '-';
} else if (logicalKey == LogicalKeyboardKey.equal) {
return '=';
} else if (logicalKey == LogicalKeyboardKey.numpadAdd) {
return '+';
} else if (logicalKey == LogicalKeyboardKey.backslash) {
return caps ? '|' : '\';
} else if (logicalKey == LogicalKeyboardKey.bracketLeft) {
return caps ? '{' : '[';
} else if (logicalKey == LogicalKeyboardKey.bracketRight) {
return caps ? '}' : ']';
} else if (logicalKey == LogicalKeyboardKey.semicolon) {
return caps ? ':' : ';';
} else if (logicalKey == LogicalKeyboardKey.comma) {
return caps ? '<' : ',';
} else if (logicalKey == LogicalKeyboardKey.period) {
return caps ? '>' : '.';
} else if (logicalKey == LogicalKeyboardKey.slash) {
return caps ? '?' : '/';
} else if (logicalKey == LogicalKeyboardKey.keyA) {
return caps ? 'A' : 'a';
} else if (logicalKey == LogicalKeyboardKey.keyB) {
return caps ? 'B' : 'b';
} else if (logicalKey == LogicalKeyboardKey.keyC) {
return caps ? 'C' : 'c';
} else if (logicalKey == LogicalKeyboardKey.keyD) {
return caps ? 'D' : 'd';
} else if (logicalKey == LogicalKeyboardKey.keyE) {
return caps ? 'E' : 'e';
} else if (logicalKey == LogicalKeyboardKey.keyF) {
return caps ? 'F' : 'f';
} else if (logicalKey == LogicalKeyboardKey.keyG) {
return caps ? 'G' : 'g';
} else if (logicalKey == LogicalKeyboardKey.keyH) {
return caps ? 'H' : 'h';
} else if (logicalKey == LogicalKeyboardKey.keyI) {
return caps ? 'A' : 'a';
} else if (logicalKey == LogicalKeyboardKey.keyJ) {
return caps ? 'A' : 'a';
} else if (logicalKey == LogicalKeyboardKey.keyK) {
return caps ? 'K' : 'k';
} else if (logicalKey == LogicalKeyboardKey.keyL) {
return caps ? 'L' : 'l';
} else if (logicalKey == LogicalKeyboardKey.keyM) {
return caps ? 'M' : 'm';
} else if (logicalKey == LogicalKeyboardKey.keyN) {
return caps ? 'N' : 'n';
} else if (logicalKey == LogicalKeyboardKey.keyO) {
return caps ? 'O' : 'o';
} else if (logicalKey == LogicalKeyboardKey.keyP) {
return caps ? 'P' : 'p';
} else if (logicalKey == LogicalKeyboardKey.keyQ) {
return caps ? 'Q' : 'q';
} else if (logicalKey == LogicalKeyboardKey.keyR) {
return caps ? 'R' : 'r';
} else if (logicalKey == LogicalKeyboardKey.keyS) {
return caps ? 'S' : 's';
} else if (logicalKey == LogicalKeyboardKey.keyT) {
return caps ? 'T' : 't';
} else if (logicalKey == LogicalKeyboardKey.keyU) {
return caps ? 'U' : 'u';
} else if (logicalKey == LogicalKeyboardKey.keyV) {
return caps ? 'V' : 'v';
} else if (logicalKey == LogicalKeyboardKey.keyW) {
return caps ? 'W' : 'w';
} else if (logicalKey == LogicalKeyboardKey.keyX) {
return caps ? 'X' : 'x';
} else if (logicalKey == LogicalKeyboardKey.keyY) {
return caps ? 'Y' : 'y';
} else if (logicalKey == LogicalKeyboardKey.keyZ) {
return caps ? 'Z' : 'z';
} else {
return "";
}
// case KeyEvent.KEYCODE_GRAVE:
//     return caps ? '~' : '`';
// case KeyEvent.KEYCODE_APOSTROPHE:
//     return caps ? '"' : '\'';

}
}

Please use proper code formatting and code markdown tag, this is unreadable.


Make the activate setting more intuitive

Currently, a provider is activated from our config file using the activate parameter.  However, the presence of the config parameter is sufficient to trigger activation, leading to a counterintuitive situation in which setting "activate = 0" still activates the provider
Make activation more intuitive by requiring that activate be set to one of yes|true|1 to trigger activation.  Any other value, as well as omitting the parameter entirely, prevents activation (and also maintains backward compatibility.
It seems a bit heavyweight to create a test specifically to validate the plurality of these settings.  Instead, modify the exiting openssl config files in the test directory to use variants of these settings, and augment the default.cnf file to include a provider section that is explicitly disabled
Checklist

[x] documentation is added or updated
[x] tests are added or updated

OTC: What is the appropriate behavior for various bogus values of activate key in provider section?

I'd allow a reasonable set of yes/no pairs and error out on unknown values.

I don't think we can should error out on unknown values as that would be a behavioral change from the test if the 3.x stream.  I think any such change would have to wait until 4.0 I think

OTC: We should add explicit values for disabling activation and report an error on unknown values.

fixed activation setting to be in line with OTC response

This pull request is ready to merge

merged


Add Image Component

Instances presented in Design (#2):

Also includes Icons used

Hi @Israel-Laguan where do I find high quality icons?

Hi, I see two chat icons:
This:

and this:

@Israel-Laguan can you confirm me what is the final icon please?

I prefer the second but designer says the first.
Let's go with the first one, the one with a human figure

Hi @Israel-Laguan where do I find high quality icons?

Design link for Mobile have the icons, please check


Add pipe API in and out formatters for Features

Work in progress, opened pull request for review.
Fixes #1374

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1763/Build result: FAILURE[...truncated 57 lines...]	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)	at java.net.Socket.connect(Socket.java:579)	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618)	at sun.net.NetworkClient.doConnect(NetworkClient.java:175)	at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)	at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)	at sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:275)	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:371)	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)	at com.tikal.hudson.plugins.notification.Protocol$3.send(Protocol.java:99)	at com.tikal.hudson.plugins.notification.Phase.handle(Phase.java:45)	at com.tikal.hudson.plugins.notification.JobListener.onCompleted(JobListener.java:36)	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:201)	at hudson.model.Run.execute(Run.java:1783)	at hudson.matrix.MatrixBuild.run(MatrixBuild.java:306)	at hudson.model.ResourceController.execute(ResourceController.java:98)	at hudson.model.Executor.run(Executor.java:410)Failed to notify endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8' - java.net.SocketTimeoutException: connect timed out
Test FAILed.

I'm not sure why the counts don't add up for GTF and GFF3 when they are fine for BED and narrowPeak
- don't lose any features when piping as GTF format *** FAILED ***
  95 did not equal 114 (FeatureRDDSuite.scala:759)

- don't lose any features when piping as GFF3 format *** FAILED ***
  195 did not equal 199 (FeatureRDDSuite.scala:772)

Would we be duplicating features when partitioning?
scala> val gtf = sc.loadGtf("src/test/resources/Homo_sapiens.GRCh37.75.trun100.gtf")
gtf2: org.bdgenomics.adam.rdd.feature.FeatureRDD =
FeatureRDD(MapPartitionsRDD[28] at flatMap at ADAMContext.scala:1186,SequenceDictionary{
1->36081})

scala> gtf.rdd.count
res2: Long = 95

scala> val gff3 = sc.loadGff3("src/test/resources/dvl1.200.gff3")
gff3: org.bdgenomics.adam.rdd.feature.FeatureRDD =
FeatureRDD(MapPartitionsRDD[21] at flatMap at ADAMContext.scala:1166,SequenceDictionary{
1->1363541})

scala> gff3.rdd.count
res1: Long = 195

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1765/Build result: FAILURE[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1378/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains<PHONE_NUMBER>1603d0decd286abb1ceca7fc021d73 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1378/merge^{commit} # timeout=10Checking out Revision<PHONE_NUMBER>1603d0decd286abb1ceca7fc021d73 (origin/pr/1378/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f<PHONE_NUMBER>1603d0decd286abb1ceca7fc021d73First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1766/Build result: FAILURE[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1378/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 6b78a5814a64fc2414df00e87df31003ceca4b8c # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1378/merge^{commit} # timeout=10Checking out Revision 6b78a5814a64fc2414df00e87df31003ceca4b8c (origin/pr/1378/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 6b78a5814a64fc2414df00e87df31003ceca4b8cFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Rebased to pull in #1411, two unit tests still fail as discussed above.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1843/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1378/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 67905d5772d8457a83894e099926d7b5b45987b5 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1378/merge^{commit} # timeout=10Checking out Revision 67905d5772d8457a83894e099926d7b5b45987b5 (origin/pr/1378/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 67905d5772d8457a83894e099926d7b5b45987b5First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage increased (+0.09%) to 76.488% when pulling 193bab45a8dae9abb3f1814355015f33fec4ca62 on heuermh:feature-formatters into 07c1982fb6fafb959b96d4c3cb83fa7edbe25721 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1852/
Test PASSed.

@heuermh I just realized that I approved this but forgot to merge this. Is this good to go from your side? If yes, what I propose is:

Let's merge #1422
Let's update this PR to add docs to the Pipe API docs noting that these new formatters exist
Then, let's merge this PR.

How's that sound on your end?

Sounds good.  I'll push a doc commit and squash after #1422 is merged.

Coverage increased (+0.1%) to 76.537% when pulling 707567cb264f3f05cd3fded6e9e7e730b19b14c8 on heuermh:feature-formatters into 1cae7692e8fcb07ca744cebc6b6dfac263fe23f0 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1863/
Test PASSed.

Merged! Thanks @heuermh!


collections.abc should get imported explicitly

Currently some of the failures of the unittests are involving that abc is not part of the collections module since version 3.3.
https://docs.python.org/3/library/collections.abc.html#module-collections.abc
“New in version 3.3: Formerly, this module was part of the collections module.”
======================================================================
ERROR: test_protocol_custom_request_headers_list (websockets.test_client_server.SSLClientServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adam/github/notmy/websockets/websockets/test_client_server.py", line 97, in test_protocol_custom_request_headers_list
    self.start_client('raw_headers', extra_headers=[('X-Spam', 'Eggs')])
  File "/home/adam/github/notmy/websockets/websockets/test_client_server.py", line 279, in start_client
    self.client = self.loop.run_until_complete(client)
  File "/usr/lib/python3.4/asyncio/base_events.py", line 316, in run_until_complete
    return future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 275, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 238, in _step
    result = next(coro)
  File "/home/adam/github/notmy/websockets/websockets/client.py", line 143, in connect
    extra_headers=extra_headers)
  File "/home/adam/github/notmy/websockets/websockets/client.py", line 53, in handshake
    if isinstance(extra_headers, collections.abc.Mapping):
AttributeError: 'module' object has no attribute 'abc'

collections.abc should get explicitly imported in client.py and server.py.
…
import collections
import collections.abc
…

In fact, these modules should just import collections.abc instead of import collections.

Well, I missed that. :)


Make it clear that when you use Rust in Windows, that a Visual Studio license is required

The Build Tools, a prerequisite for Rust on Windows require a Visual Studio license. This is not mentioned anywhere in the docs, but should be made clear.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 200916ea-06cb-d017-a9e8-18d2216c5bac
Version Independent ID: f872813c-0ab2-0f5a-45e0-e81e8479470b
Content: Set up your dev environment on Windows for Rust
Content Source: hub/dev-environment/rust/setup.md
Product: dev-environment
GitHub Login: @stevewhims
Microsoft Alias: stwhi

Another option would be to split the MSVC toolchain via rustup talking to the Visual Studio license requirements while suggesting the stable-x86_64-pc-windows-gnu toolchain for those without a valid Visual Studio license.  Just helps users who are investigating from infringing on the commercial terms though they may still some open source process for the underlying mingw bits.

Thanks you, both, I'm looking into this.
-Steve

Hi, does this topic shed any light on the steps and requirements?
https://code.visualstudio.com/docs/cpp/config-msvc
Many thanks!
Steve

We understand the requirements - but what isn't clear is that the build tools require a license. These docs should reflect that. I'll follow up directly on Teams to share more details.

It wasn't the install instructions that were the problem, but more the fine print in the Note around the need for a commercial license even if you were not planning on using any VS product.  Some of our associates may not have an Ent/Pro license without going through procurement so since there is no gate like with other MSDN downloads infringing on the terms is quite easy.

UPDATE: I've added a note saying "Use of the Microsoft C++ Build Tools, or Visual Studio Build Tools, requires a valid Visual Studio license (either Community, Pro, or Enterprise)."


Update spark-env.sh with node-specific environment variables.

Noticed that the environment variables for spark defined in common.make_dse_env were not actually being used because resources/spark/conf/spark-env.sh was overriding them when it was sourced in.   Fixed this by updating this file with the proper environment variables from any environment variable in make_dse_env found in this file.
Without this, files get created in /var/lib/spark and /var/log/spark.

+1


Create PULL_REQUEST_test

https://github.com/yellowrook1/-img-0/pull/15#r891867173

[ ]

[ ] #15

[ ] #22

[ ] https://github.com/yellowrook1/-img-0/pull/15#discussion_r891867173


Initial Build: Landing

Add base HTML markup and CSS styling to landing page.
Acceptance Criteria (AC):

[x] Add basic HTML markup to index.html
[ ] Add base styling for the page to styles.css
[ ] Create a pull request for the above requirements

Added landing.css for styling.

Sent in a request for review.

I don't see any pull request to review for this


WIP: Add basic badge support.

This moves to hosting badges internally to Read the Docs.
This is a WIP, as the SVG code needs to be fixed. I don't know SVG super well, but we basically just need to fix the template so that it will properly render with variable width text (eg. varying project name length).

Looks great! :smile:

This is now fixed in another PR


converter not able to generate alpha channel

I exported a test-video from premiere - Quicktime with animation codec and 32bit depth.
I reimported the video to the timeline, and it has an alpha channel.
I also tried different export settings for the frame-rate and field-order, and not to include meta-data ...
But i always get
"An error occurred generating the alpha channel: ffmpeg exited with code 1".
ffprobe output's this info:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hand_v0_full_progressive_animation.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    creation_time   : 2015-07-11 13:44:59
  Duration: 00:00:08.71, start: 0.000000, bitrate: 176804 kb/s
    Stream #0:0(eng): Video: qtrle (rle  / 0x20656C72), bgra, 720x480, 175255 kb/s, SAR 109:120 DAR 109:80, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      creation_time   : 2015-07-11 13:44:59
      handler_name    : Apple Alias Data Handler
      encoder         : Animation
      timecode        : 00:00:00;00
    Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2015-07-11 13:44:59
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00;00
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2015-07-11 13:45:00
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00;00
Unsupported codec with id 0 for input stream 2

Do you have any ideas, as to what could be wrong here? Is it this strange stream2?
Thanks a lot!

As it states
Unsupported codec with id 0 for input stream 2

and this seems to be an empty audio track I'd try removing that track at first.
I'm perfectly able to convert progressive animation encoded video on my machine, but there's a lot of factors at play here. What OS are you on? How do you encode the video?

i'm on a mac yosemite. exporting with adobe premiere.
i seem to get this empty stream 2 with any quicktime settings supporting alpha.
even if i run ffmpeg with -map and map the 0:0 and 0:1 channel, then call ffprobe on the result, the 0:2 stream is still there :/ .
but i am not sure, that this is the problem, as i get the same error-message for other files like mp4 witch have no alpha channel.

To further debug this I'd try the following things:
First of all I'd try doing some really simple operation with ffmpeg on your video, sth like converting your video to mp4:
ffmpeg -i hand_v0_full_progressive_animation.mov -acodec libfaac -b:a 128k -vcodec mpeg4 -b:v 1200k -flags +aic+mv4 output.mp4

If this already fails ffmpeg has a problem with your video that I don't know about. If this succeeds I'd try running the command that seethru-convert generates to extract the alpha channel by hand:
ffmpeg -i hand_v0_full_progressive_animation.mov -vf [in] format=rgba,split [T1], fifo,lutrgb=r=maxval:g=maxval:b=maxval,[T2] overlay [out];[T1] fifo, lutrgb=r=minval:g=minval:b=minval [T2] alpha-output.mov

This should tell us if this is an issue with your video, ffmpeg or seeThru

I get an "Unknown encoder 'libfaac'", but with -acodec aac and -strict -2, converting to mp4 works.
But when extracting the alpha channel i get this error:
[NULL @ 0x7fa4dc80ee00] Unable to find a suitable output format for 'format=rgba,split'
format=rgba,split: Invalid argument

I'm using the latest ffmpeg 64bit binary (version 2.7.1).

Gotta love ffmpeg.
I'm not on a Mac so I have no clue what encoders are available on your machine. Could you try to find out what is available for you and try to convert your video without any further transforms. That way it'd be possible to find out if ffmpeg is able to process your video at all.

So i checked the ffmpeg filter doc https://www.ffmpeg.org/ffmpeg-filters.html and noticed that the filter is in "
Sooooo
ffmpeg -i hand_v0_full_progressive_animation.mov -vf "[in] format=rgba,split [T1], fifo,lutrgb=r=maxval:g=maxval:b=maxval,[T2] overlay [out];[T1] fifo, lutrgb=r=minval:g=minval:b=minval [T2]" alpha-output.mov

almost works ;)
But as there is no audio codex specified, ffmpeg wats to use AAC, which is experimental and without "-strict -2" the above call fails.
But i think for this step we should use a lossless codec.
I'm not firm with codecs, and therefore not sure, if i used the perfect one, but the following works perfectly fine on my machine:
ffmpeg -i hand_v0_full_progressive_animation.mov -vf "[in] format=rgba,split [T1], fifo,lutrgb=r=maxval:g=maxval:b=maxval,[T2] overlay [out];[T1] fifo, lutrgb=r=minval:g=minval:b=minval [T2]" -acodec pcm_s16le alpha-output.mov

:)

But as there is no audio codex specified, ffmpeg wats to use AAC, which is experimental and without "-strict -2" the above call fails.

Will this also produce the same error as in your initial post?

no. it produces
[aac @ 0x7f8d33015400] The encoder 'aac' is experimental but experimental codecs are not enabled, add '-strict -2' if you want to use it.

in any case:

the alpha-channel does not need audio - the audio should be in the rgb video
the audio should not be compressed anyway

and the problem is not the audio, but first of all the missing " for the filter spec.
at least that is my impression right now ..

the alpha-channel does not need audio - the audio should be in the rgb video

That's what I wanted it to do when writing it, but apparently ffmpeg works differently than i thought.

the audio should not be compressed anyway

This is what's supposed to happen here: https://github.com/m90/seeThru/blob/master/converter.js#L55, but apparently ffmpeg will just "copy" the audio when you don't specify anything like in https://github.com/m90/seeThru/blob/master/converter.js#L55
Two questions:

would you be able to produce a patch for that?
in case not, could you somehow send me a snippet of the video so I can do that?

And regarding:

and the problem is not the audio, but first of all the missing " for the filter spec.
at least that is my impression right now ..

That was just me unable to translate the fluent ffmpeg options to a shell command.

but apparently ffmpeg will just "copy" the audio when you don't specify anything
I think it doesn't copy it, but tries to use the default codec, which in my case strangely is the experimental AAC - and then fails

to remove the audio from the alpha output: add -an
ffmpeg -i hand_v0_full_progressive_animation.mov -vf "[in] format=rgba,split [T1], fifo,lutrgb=r=maxval:g=maxval:b=maxval,[T2] overlay [out];[T1] fifo, lutrgb=r=minval:g=minval:b=minval [T2]" -an alpha-output3.mov

works! :)
so probably adding '-an' here https://github.com/m90/seeThru/blob/master/converter.js#L32 could do the trick ... ?

would you be able to produce a patch for that?
Sorry - i tried, but i don't know how to develop with node ...

could you somehow send me a snippet of the video
Uploading to Dropbox ... i'll send you the link as soon as it's finished ...

Here is the link to the video ...
https://www.dropbox.com/s/3w525ddeedcd2lr/hand_v0_full_progressive_animation.mov.zip?dl=0

Thanks for the file. Weird thing is I can perfectly convert your video running a fresh npm install of 2.1.7 on my ubuntu machine running ffmpeg 2.7.1.
Adding the -an option makes ffmpeg crash though.
So I gotta look into this a little closer. Thanks for all the input in any case.

hm ... i'm not running an npm-install ... i donwloaded the binaries and just unzipped it to a folder + set the path-variable ...
does this work for you?
ffmpeg -i hand_v0_full_progressive_animation.mov -vf "[in] format=rgba,split [T1], fifo,lutrgb=r=maxval:g=maxval:b=maxval,[T2] overlay [out];[T1] fifo, lutrgb=r=minval:g=minval:b=minval [T2]" alpha-output.mov

because for me it produces
[aac @ 0x7f8d33015400] The encoder 'aac' is experimental but experimental codecs are not enabled, add '-strict -2' if you want to use it.

which could bet the cause of "crash" on my machine ...

Adding the -an option makes ffmpeg crash though.
I just found this https://www.npmjs.com/package/fluent-ffmpeg#audio-options - .noAudio() - maybe you could give it a try?

HURRAYYY!!
seethru-convert --in hand_v0_full_progressive_animation.mov --out test.mov
Processing hand_v0_full_progressive_animation.mov finished!
just add withNoAudio() :)
alpha.addOption(
                '-vf'
                , '[in] format=rgba,\
                split [T1], fifo, lutrgb=r=maxval:g=maxval:b=maxval,\
                [T2] overlay [out];\
                [T1] fifo, lutrgb=r=minval:g=minval:b=minval [T2]'
        ).withNoAudio().withVideoCodec(metadata.streams[0].codec_name);

Great! Would you mindestens opening a Pull Request with that fix?

sorry ... do what?
i can go to https://github.com/m90/seeThru/pulls
then "new pull request" takes me to https://github.com/m90/seeThru/compare
and then, no matter what i choose, it says there is nothing to compare ...

Ok, don't  worry :)
You'd have to fork the repository, commit the changes to your fork and then open a Pull Request against my repo so I could merge your changes and release the fix to npm.
If that's too much of a hassle for you I can apply your fix when I find the time to do so. No worries.
Thanks in any case.

ok - is pushed

2.1.8 including your fix is now available on npm: https://www.npmjs.com/package/seethru
Thanks!

great - tested and worked like a charm! thanks!


compose: Refactor drafts restoration code to reduce duplication

This PR is a fixed and updated version of #24680

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

Looks great, merged, thanks @N-Shar-ma and @SahilSingh177!


[FY24.4 Sprint 4] Add 3-6 additional flight maps to CASEI

This issue follows https://github.com/NASA-IMPACT/admg-casei/issues/634 and https://github.com/NASA-IMPACT/admg-casei/issues/635
Context
Back in FY24.1, we started visualizing flight paths for campaigns on CASEI. We started with 5 (see epic: https://github.com/NASA-IMPACT/admg-casei/issues/576) and have since expanded to ~8 (see YAML files https://drive.google.com/drive/u/0/folders/1R92_4YxYTo8oml0NHMCLFYJ-t-dHaWIR).

Example of a flight map for a campaign
As of FY24.3, we have published 27 campaign flight paths.
Known Challenges
We learned through past efforts in trying to automate the data collection and visualization process that currently it is NOT possible to fully automate this process for these reasons:

Inconsistent data formats as in NASA's case airborne data ownership is split across 12 DAACs that all do things a bit differently
Different ways to define measurement variable names. This might not seem like a big deal but it really makes it hard/slow to get buy-in when building out features quickly as terminology is for queries and UI filters gets challenged back and forth. There are initiatives on streamlining language, for example the official GCMD science keywords (which is a string of keywords) but each DAAC also has their own views and it’s an ongoing discussion.
Incomplete data. To get the full set of historical data to even visualize flights (let along instruments on each flight deployment), in many cases you’ll need to manually look for data that’s lock in PDFs still or old canned films for example that are not digitized. This is what the ADMG data curators have been doing and filling in for CASEI

Goal

We will continue to collect and visualize campaigns in batches (of around 3-6) because each campaign may need some manual data fixes.
We will incorporate fixes to common data issues along the way, and continue to learn about the extent to which we can still create a pipeline for visualizing nav data in the cloud without having to do it locally and save out GeoJSON files

Acceptance Criteria

3-6 additional nav data sets saved as YAML files https://drive.google.com/drive/u/0/folders/1R92_4YxYTo8oml0NHMCLFYJ-t-dHaWIR
Static locations for those campaigns https://docs.google.com/spreadsheets/d/17v-ZfeWoPZoCAVSs57Y3Q1gKUe6S49fZ8rC2KOO_myY/edit#gid=1924471089
3-6  flight tracks visualized and deployed to CASEI in staging
Deploy from staging to production

Future note for @LanesGood to move this issue out of backlog and label the correct PI Iteration and PI Sprint once the team has set up the project board for the next PI.

@als0076 I've processed AASE: http://admg-inventory-staging.s3-website-us-east-1.amazonaws.com/casei/campaign/AASE/
Let me know if it looks correct.

@willemarcel looks good!

@als0076 I've processed 2 new campaigns:
http://admg-inventory-staging.s3-website-us-east-1.amazonaws.com/casei/campaign/EPOCH/
http://admg-inventory-staging.s3-website-us-east-1.amazonaws.com/casei/campaign/ACCENT/

@willemarcel both campaigns look correct

quite sure this is done - closing ticket.


Option to select folder instead of file

I would like to have a file dialog to select a folder.
But this plugin only allows to select files (correct me if I am wrong)
Platform specs
Please provide the following info if this is a Unity 3D repository.

Unity version: 2021.3 LTS
Platform: Windows, Linux, macOS

You're correct. I have no knowledge if this is possible on iOS and on Android, picking folders won't return their raw filepaths since Storage Access Framework is used. An abstraction layer will be required which is present in the form of FileBrowserHelpers in SimpleFileBrowser plugin.

I would be happy with a folder selection dialog even if it is only for desktop platforms.
I expect this to be some kind of flag to the regular file selection dialog and thus pretty simple to implement.

This plugin doesn't support desktop platforms though.

I see. Guess this issue can be closed then as won't fix.


Question: Why not leader_only on the migrations?

https://github.com/oliveradria/symfony-elasticbeanstalk-optimization/blob/master/.ebextensions/symfony.config#L6
I see you run migrations, I'm curious if you happen to know:

Why are you not running it on leader_only
Is thereany problems in running a migration AND a schema:update (incase a bad bad developer forgot to run a migration differential?)

Hi! I haven't updated this in a while, when I have a few minutes, I will try to update the package (to Symfony 5).

Yes, can be leader only. In this case, it will probably just say "Nothing to migrate" on the non-leaders.
I would not advise running schema:update, but rather add the migration as soon as possible. What if Dev A forgot the diff, then Dev B gets the newest branch, does a diff because they updated some schema, then Dev B's diff would have the diff for both Dev A and Dev B changes, and it might fail in production, because Dev A already manually did a schema:update (e.g. something like "Table already exists" might pop up). Better perhaps to do something like doctrine:schema:validate and fail if there's an error.


[BUG] Twitter blue checks not shown on Feed

Twitter blue checks still dont appear properly, as you can see blue checks are present neither in Feed nor on Twitter user account names but the blue check appears only in tweets in user's Twitter page
Using 2.1.0 version
LineageOS 19.1
Android 12L
pics:
https://imgur.com/a/9ovM3rq
https://imgur.com/a/X1XfNU6
This bug is not that serious for me , if you can please prioritize this 👉 https://github.com/TheHCJ/Quacker/issues/7
EDIT:
Only when searching for Blue Twitter users the check appears correctly on their page
https://imgur.com/a/tHVzHEl

I am aware of this issue. The feed has used a different way of displaying tweet cards as the rest of the app. I will try to get this fixed in v2.1.1

Since I already have a fork of the Fritter project and that github do not allow to have another fork which is also a fork of the project Fritter, I can't fork Quacker.
So I cannot do a PR for this project.
Instead, I will provide here the patch for this bug.
Here i is:
fix_17.patch

Now merged your code changes in, should be in the next patch

I saw the merge that you did.
That last part of the fix:
    if (tweet.user != null) {
      tweet.user!.idStr = e['user_id_str'];
    }

was merged in the wrong function.
It should be in the TweetWithCard.fromData function, not the TweetWithCard.fromJson function.
It should be inserted at the line #872 of the lib/client.dart file.


fix(angular): add definite assignment assertion to props

Another option would be to turn off strict type checking on the project creation...
https://coveord.atlassian.net/browse/CDX-101

Another option would be to turn off strict type checking on the project creation...
https://coveord.atlassian.net/browse/CDX-101

https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-7.html#strict-class-initialization
I don't think we should, it's a good thing IMO to check this. It ensures proper typings in the end.


Inverse "react-native-to-css"

Is there a package that does the opposite?

Not that I know of. Feel free to give it a shot if you want though!

Darn. Thanks for the fast reply!


Problem related to nomic login

pip install nomic
nomic login # follow prompts to login
python -c "from nomic import atlas; print(atlas._get_datastream_credentials(name='contrastors'))"
bash report "bash: nomic: command not found"
system：ubuntu

hm this seems like nomic didn't login correctly. Can you run pip list | grep nomic and paste the output?

My guess here is that python is not referencing the same python installation that pip is install nomic into.
Try with python3 or inside a virtual environment @nioxinjiang3 .


when using udp NioDatagramChannel , the process will takes tens of thousand of file Descriptor

Expected behavior
when using udp NioDatagramChannel , the process will takes tens of thousand of file Descriptor
Actual behavior
in normal udp situation , every message will takes a fd , and close method should called on the socket to free the fd after msg was send . my process takes 20k fds , and they are ocopied by udp .
Steps to reproduce
Minimal yet complete reproducer code (or URL to code)
Netty version
netty-all-4.0.25
JVM version (e.g. java -version)
Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
OS version (e.g. uname -a)
x86_64 x86_64 GNU/Linux

Please upgrade to a recent version and report back. The version you are using is very old. Also please provide a reproducer

Am 13.05.2018 um 08:29 schrieb Adams.H<EMAIL_ADDRESS>Expected behavior
when using udp NioDatagramChannel , the process will takes tens of thousand of file Descriptor
Actual behavior
in normal udp situation , every message will takes a fd , and close method should called on the socket to free the fd after msg was send . my process takes 2w fds , and they are ocopied by udp .
Steps to reproduce
Minimal yet complete reproducer code (or URL to code)
Netty version
netty-all-4.0.25
JVM version (e.g. java -version)
Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
OS version (e.g. uname -a)
x86_64 x86_64 GNU/Linux
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

this fd leak is not caused by netty io ,  this issue could be closed , tks for your help ! :D


refactor: configs.lua

Introduce a Setup class and refactor into it, as opposed to the closure style class.
Improve type annotations

just curious what the starting point for the refactoring is. Mainly this might break some plugins and user configurations :)

The plan is to slowly move most of the functionality into core. However when reading the code I'm struggling to understand what's going on and a lot of the code hasn't been touched in years and it's evident to me areas have rotted and other parts have grown because there isn't really a well defined structure.
I don't think I've changed any functionality.

afaik some users use config.launch on after folder .   but just replace the dot with a colon after this pr. Yes, some code of this project has not been changed in a long time. because according to justin's idea, lspconfig only collects server configuration. More things need to be done in core.

Ok. launch looked like an internal thing only since it was only created after running setup. I'll add another alias without :.

Actually, what does calling launch() in after achieve if you need to call setup() anyway? Is it for enabling the LSP on other filetypes or something?

Is it for enabling the LSP on other filetypes

just instead autocmd FIletype .  like launch in after/ftplugin/lua.lua


Nsfw 2d not working

It say ercon connection refused in console

the commands work. don't spam tho otherwise ur ip gets banned from their api (nekobot)


Fixes #1581 Migrate LocationResourceTest to use common tests from Ent…

…ityResourceTest
Describe your changes :

See #1581 for details

Type of change :

[] Bug fix
[x] Improvement
[] New feature
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[] Documentation

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have performed a self-review of my own.
[x] I have tagged my reviewers below.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] My changes generate no new warnings.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

Reviewers

@amiorin

@amiorin, I am going to merge this in. Any comments can be addressed in a subsequent issue.


feat: Adding gitlab ci support to web_app_protection_example

What?

Added pipeline option using Gitlab CI [CIT-121]

Why?
In this PR, we implemented the option to use Gitlab CI.
Testing ?
Integration and Lint tests were performed.
Anything Else?
None.

/gcbrun

/gcbrun

/gcbrun

/gcbrun

/gcbrun


Increment version for ml releases

Increment package version after release of azure-ai-ml

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-ai-ml


Modification of Graph UNet, but it doesn't work..

I'm trying to modify GraphUNet to do regression for third column elements in pos[ : , 2 ] with my own datasets.
MyData has a lot of graphs, but each graph has same number of nodes and edges.
dataset[0] looks like Data(edge_index=[2, 74], pos=[74, 3], test_mask=[74, 1], train_mask=[74, 1]).
I would like to train my Net, but it does not update pred when I try to train with 10 epochs
When I print epoch# and loss values, I think the network does not learn anything..
epoch: 0 | total_loss : 0.08977822025347816
epoch: 1 | total_loss : 0.08977822025347816
epoch: 2 | total_loss : 0.08977822025347816
epoch: 3 | total_loss : 0.08977822025347816
epoch: 4 | total_loss : 0.08977822025347816
epoch: 5 | total_loss : 0.08977822025347816
epoch: 6 | total_loss : 0.08977822025347816
epoch: 7 | total_loss : 0.08977822025347816
epoch: 8 | total_loss : 0.08977822025347816
epoch: 9 | total_loss : 0.08977822025347816

Matt, can you help me out? Here is my code.
import os.path as osp

import torch
import torch.nn.functional as F
from MyDataset import MyData
from torch_geometric.nn import GraphUNet
from torch_geometric.utils import dropout_adj
import numpy as np
from torch_geometric.data import DataLoader

dataset = MyData(root='./')
data = dataset[0]

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()

        pool_ratios = [32/74, 8/32, 3/8]
        self.unet = GraphUNet(in_channels = 1, hidden_channels=16, out_channels=1,
                              depth=3, pool_ratios=pool_ratios)

    def forward(self, data):
        x = data.pos[:,2].view(-1,1)
        edge_index = data.edge_index
#         batch = data.batch
#         edge_index, _ = dropout_adj(data.edge_index, p=0.2,
#                                     force_undirected=True,
#                                     num_nodes=data.num_nodes,
#                                     training=self.training)
#         x = F.dropout(x, p=0.1, training=self.training)
# ***************************************
        x = x.float()
        x = self.unet(x, edge_index)
        x = x.float()
# ***************************************
        return x

device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = Net().to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=0.1, weight_decay=0.001)

def train(dataset):
    loader = DataLoader(dataset, batch_size=1, shuffle=True)

        # train
    for epoch in range(10):
        total_loss = 0
        model.train()
        for batch in loader:
            optimizer.zero_grad()
            pred = model(batch.to(device))

            label = batch.pos[:,2].view(-1,1)

            loss = torch.autograd.Variable(F.mse_loss(pred, label), requires_grad=True)
            loss.backward()

            optimizer.step()

            total_loss += loss.item()

        total_loss /= len(loader.dataset)
        print(f'epoch: {epoch} | total_loss : {total_loss}')

train(dataset[0])

Hi,
first of all, the line loss = torch.autograd.Variable(F.mse_loss(pred, label), requires_grad=True) looks really weird to me. In general, you do not need wrap the loss into its own Variable. It should require_grad by default.
Second, you should verify that model parameters receive a gradient, e.g., by looking at list(model.paramaters)[0].grad.
Otherwise, your code looks correct to me, although I'm a bit confused about the actual task. It seems like you simply want to reconstruct node features, which (a) does not look particularly useful to me, (b) might be difficult for GCN layers as they will smooth out node features.
In order to identify any issues with GraphUNet, you should first verify that a more simpler GNN, e.g., stacking a few GCNConv layers, already produces useful results.

Thanks Matt, I solved the problem by deleting loss = torch.autograd.Variable(F.mse_loss(pred, label), requires_grad=True) and added
            loss = F.mse_loss(pred.float(), label.float())
            loss.backward()

because it kept giving me errors when dtypes are not matched. Anyways, it is working now, and actually, I have done stacking pyg_nn.GCNConv layers with MyData, and it worked well with your kind help. :)
I need to reduce and then increase the dimension of the nodes (in terms of the number of nodes) so I can create like autoencoder architecture with graph data.
I have another question about the architecture of Graph-UNet. I can see that encoder uses TopK-Pooling and GCNConv layers, but how does the decoder recover the dimension of the node numbers because I only see GCNConv layers stacked in the decoder part. In my understanding, GCNConv layer changes the number of node features, not the number of nodes.
And also does the decoder actually have trainable parameters when it is unpooled?

The Graph-UNet is not really an auto-encoder, as it will use skip-connections between the different levels of coarsened graphs and maintains graph adjacency information from the encoder for the decoder part. There is no real bottleneck here. Unpooling is done by setting node features to zeros for the filtered out nodes and adding initial features as skip-connections.

Then what kind of GNN architecture would you use for node feature reconstruction by reducing the number of nodes and recovering them again?
What is the skip-connection in topkPooling? Does it just skip computation like using dropouts?
Is the Unpooling layer in GraphUNet commonly used unpooling layer to be used as decoder to recover the node features with the same number of input nodes?

This is an active field of research, which I am not that familar with. A naive approach is to aggregate node features into a global graph representation (GNN + global readout), and then convert your node features back via an MLP mapping [num_global_features] to [num_nodes, num_node_features]`. Note that you lose permutation-equivariance that way.
The skip-connection refers to skipping the "coarsening part", where final representations are computed based on initial feature representation + coarsened feature representation. You can read more about it in the Graph U-Nets paper.
That's indeed the most straightforward way to implement unpooling, which omits worrying about the right amount of nodes and reconstructing adjacency information.


WSLg environment variables not set in user distro

Windows build number:
22000.856
Your Distribution version:
Debian unstable
Your WSL versions:
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>
WSLg version: 1.0.42
MSRDC version: 1.2.3401
Direct3D version: 1.606.4
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22000.856
Steps to reproduce:
Issue is apparent immediately upon opening any distribution.

Open an installed Linux distro in Terminal or Command Prompt.
Run env.

WSL logs:
pulseaudio.log
stderr.log
weston.log
WSL dumps:
No response
Expected behavior:
Listing of environment variables should include those provided by WSLg, to wit:
WAYLAND_DISPLAY=wayland-0
DISPLAY=:0
PULSE_SERVER=/mnt/wslg/PulseServer

and GUI and sound-using apps should function normally,
Actual behavior:
Above-listed environment variables are not present, and GUI and sound-using apps therefore do not function. (Note: these are present in the system distribution; just not the user one.)
This is purely an issue with the environment variables, however. The WSLg filesystem is properly mounted, sockets are present, etc,, and manually setting the variables with:
WAYLAND_DISPLAY=wayland-0
DISPLAY=:0
PULSE_SERVER=/mnt/wslg/PulseServer

permits WSLg to function normally.

As far as I know, there are no such options disabling things:
.wslconfig:
[wsl2]
nestedVirtualization = true
kernel = C:\\ProgramData\\WSL-Kernel\\kernel
networkingMode = bridged
vmSwitch = Bridge
ipv6 = true
debugConsole = false

.wslgconfig:
[system-distro-env]
WESTON_RDP_DISABLE_FRACTIONAL_HI_DPI_SCALING=false

;start menu with distro name
WESTON_RDPRAIL_SHELL_DISABLE_APPEND_DISTRONAME_STARTMENU=true

;restart weston compositor and Xwayland by Left-Ctrl + Left-Atl + Backspace
WESTON_RDPRAIL_SHELL_ALLOW_ZAP=true

wsl.conf:
# Enable extra metadata options by default
[automount]
enabled = true
options = "metadata,umask=022,fmask=022"
crossDistro = true
[network]
generateResolvConf = true
generateHosts = true
[user]
default=avatar

And the issue persists both across distributions (I have an secondary Alpine distribution to use for debugging), and if I bypass all my usual startup scripts with wsl sh . Or even just wsl env:
HOSTTYPE=x86_64
LANG=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files/WindowsApps/Microsoft.PowerShell_<IP_ADDRESS>_x64__8wekyb3d8bbwe:/mnt/c/Program Files/AdoptOpenJDK/jre-<IP_ADDRESS>-hotspot/bin:/mnt/c/ProgramData/Boxstarter:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0/:/mnt/c/Windows/System32/OpenSSH/:/mnt/c/ProgramData/chocolatey/bin:/mnt/c/Program Files/dotnet/:/mnt/c/Program Files/Microsoft SQL Server/130/Tools/Binn/:/mnt/c/Program Files/Amazon/AWSCLIV2/:/mnt/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn/:/mnt/c/Program Files/Calibre2/:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Program Files/Azure Data Studio/bin:/mnt/c/Users/avatar/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0/LocalCache/local-packages/Python39/Scripts:/mnt/c/HashiCorp/Vagrant/bin:/mnt/c/Program Files/Microsoft SQL Server/140/Tools/Binn/:/mnt/c/Program Files/Microsoft SQL Server/150/Tools/Binn/:/mnt/c/Program Files (x86)/GnuPG/bin:/mnt/c/tools/gsudo/Current:/mnt/c/Program Files/GitHub CLI/:/mnt/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn/:/mnt/c/Program Files/Git/cmd:/mnt/c/Users/avatar/AppData/Local/Programs/Python/Python38/Scripts/:/mnt/c/Users/avatar/AppData/Local/Programs/Python/Python38/:/mnt/c/Users/avatar/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/avatar/AppData/Local/GitHubDesktop/bin:/mnt/c/Users/avatar/AppData/Local/Microsoft/WindowsApps:/mnt/c/Program Files/Oracle/VirtualBox:/mnt/c/Users/avatar/.dotnet/tools:/mnt/c/Users/avatar/.dotnet/tools:/mnt/c/Users/avatar/AppData/Local/Keybase/:/mnt/c/Users/avatar/AppData/Local/JetBrains/Toolbox/scripts:/mnt/c/Program Files/PowerShell/Modules/Pscx/4.0.0/Apps
TERM=xterm-256color
WSLENV=WT_SESSION::WT_PROFILE_ID
WT_SESSION=df416191-d76b-4074-b539-a9cf250b0d24
WT_PROFILE_ID={574e775e-4f2a-5b96-ac1e-a2962a402336}
WSL_INTEROP=/run/WSL/17498_interop
NAME=pallas
HOME=/home/avatar
USER=avatar
LOGNAME=avatar
SHELL=/usr/bin/zsh
WSL_DISTRO_NAME=Debian
SHLVL=0
PWD=/mnt/c/Users/avatar
OLDPWD=/mnt/c/Users/avatar
_=/usr/bin/env


Trouble update/install plugin "HTML AutoCloseTag"

Hi,
I have install spf13 to vim, but it has trouble update HTML AutoCloseTag. It showed a dialog box to login into github website.
Please fix the bug.
Thanks

#1014 check this issue.

The problem of amirh's HTML-AutoCloseTag still existed.
You'd better delete it manually, and use another auto-close plug-in.
：）

Can confirm that, the problem still exists.

Problem had been solved in this #1035

Can confirm that, the problem still exists.

Yep ... the problem still here

you can try   vi    ~/.vimrc.bundles   and enter code in vim ==> :%s/amirh/vim-scripts/

echo UnBundle 'amirh/HTML-AutoCloseTag' >> ~/.vimrc.bundles.local
Run the above line in terminal, and in vim :BundleClean.

the problem still here!!!

The problem is still here.

Plugin 'heracek/HTML-AutoCloseTag', {'name': 'HTML-AutoCloseTag2'}
UnBundle 'amirh/HTML-AutoCloseTag'
@farwish @shiquanwang add ~/.vimrc.bundles.local

It's still broken. Trying to install using the snippet at http://j.mp/spf13-vim3.

issue still exists....

I'm still getting it.

That's because NOBODY seemst to want to merge the commit that fixes it.

Still seeing this problem

Still seeing this problem

Forget it, this project is unmaintained. You're better off creating your own vimrc. It's not that hard.


Animal Need Water API:DoesAnimalHaveAccessToWater

I was trying out your API and I was getting false result from Animals that I know had water.  I noticed you store the indoor name in lowercase, but in the API call DoesAnimalHaveAccessToWater the 2 nameofIndoors checks are not transformed tolower.  I did a local compile with the tolowers added and the false negatives went away.
Just found out about this mod, nice addition to the game play.

Didn't notice that, thanks!


Increasing memory usage during training with DQN

Describe the bug:
Memory usage increases linearly during training when using DQN. When training with a large number of timesteps this can cause OOM. I originally noticed this when training on a custom env using DQN, but have since replicated using Cartpole environment.
Code:
import gym
from stable_baselines import DQN
from stable_baselines.deepq.policies import MlpPolicy

env = gym.make('CartPole-v1')
model = DQN(MlpPolicy, env, gamma=0.997, buffer_size=2500, batch_size=256, exploration_fraction=0.12952,
            target_network_update_freq=500, prioritized_replay=False, verbose=0)
model.learn(total_timesteps=7500)

Memory use plot (made using mprof run):

I would have expected the plot to plateau once the replay buffer was full (at 2500 steps), but the memory use continues to increase past this point.
System Info:
Describe the characteristic of your environment:
Linux, Ubuntu 18.04
Stable baselines version 2.9.0 without MPI (installed using conda) .
Python version 3.7.6
Tensorflow version 1.14
numpy version 1.18.1

Hmm, I have had multi-day DQN runs in the past with no leaking issues like this (and currently running one for a third day). I will try to replicate this once I have a machine available.

I did not observe the same behavior on my machine with the given code (plots below from mprof). Ubuntu 18.04, Python 3.6.9 (not conda), Tensorflow 1.14.0, Numpy 1.18.1, stable baselines 2.9.0. Smells like the reason is in one of the underlying libraries (or maybe even Python version).
With GPU:

Without GPU:

Thanks - I'll experiment with different libraries and try to isolate the problem.

I have now replicated the non-leaking behaviour (for python 3.6.9 and 3.7.6) by changing the setup process for my virtual environment.

I found that installing Tensorflow using Pip rather than Conda will fix the issue, although I haven't been able to find the exact package difference that caused the problem.
For reference (or anyone else having the same issue) the commands I used was:
conda create --name env python=3.7.6
conda activate env
python -m pip install --upgrade pip setuptools wheel
pip install stable-baselines[mpi]
pip install tensorflow==1.14

Closing this one as it does not seem to be related to stable baselines.

For the record:
I'm also observing this with the following environment (installed through pip):
Ubuntu 18.04
tensorflow 1.13.2
numpy 1.19.5
stable-baselines 2.10.0


Split Falco audit example into own page

This is a Feature Request

What would you like to be added
Make a new page that includes the Falco details from https://k8s.io/docs/tasks/debug-application-cluster/audit/#use-falco-to-collect-audit-events
Hyperlink from https://k8s.io/docs/tasks/debug-application-cluster/audit/ to the new page.
Why is this needed
https://k8s.io/docs/tasks/debug-application-cluster/audit/ is a long page, long enough that readers might get lost or give up.
Comments

Falco is a CNCF project.

/language en

/assign


Add an option to set GRUB keyboard layout

It would be very useful to be able to set GRUB's keyboard layout. For example to type a password when using an encrypted /boot.
Could an option be added to configure this?
Various sources to setup a keyboard layout:
https://wiki.archlinux.org/index.php/Talk:GRUB#Custom_keyboard_layout
https://askubuntu.com/questions/751259/how-to-change-grub-command-line-grub-shell-keyboard-layout

I'd still love this feature.
I'll try to come up with a solution now that my nix-fu is a bit better.

Here is an implementation: https://github.com/mdevlamynck/nixpkgs/tree/grub-use-xkb-config
It's not a good solution yet since it doesn't affect early grub (before it can read it's config in /boot) so it won't work when typing the password to access an encrypted /boot.
The issue is that grub-install used by nixos doesn't provide a way to give extra config to build the core.img embedded in the mbr / uefi. We could call directly the underlying command such as grub-mkimage and grub-bios-setup but that would mean reimplementing the whole detection and autoconfiguration that grub-install does for us.

Apparently only the terminal input at_keyboard module supports customized keyboard layouts. So it won't work for a lot of people. With my desktop and my usb keyboard I can't get it to work with my custom layout. This feature is probably too brittle in upstream grub to be properly integrated in NixOS.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/can-nixos-set-keyboard-for-grub-luks-password/29623/7


Authentication failure errors

sometimes fetch is performed with stale token and authentication error is logged

I already did an hour ago though? https://github.com/TheLastProject/lovelace-valetudo-map-card/releases/tag/v2021-08-12