Issue #295 - validate existing files using checksum in files.download

Added functionality for sha256sum and md5sum for files.download.

I'm not sure how to resolve the test failure.  I'm open to suggestions.

I was able to resolve the test locally by reverting this commit:
https://github.com/Fizzadar/pyinfra/pull/405/commits/f028fddf7363947ba1fa37f62acbfb9ce4a7a23b
However this will cause the flake8 failure.

This change allows both the flake8 and pytest to run.  However this seems like an added step that shouldn't be needed.
--- a/pyinfra/facts/files.py
+++ b/pyinfra/facts/files.py
@@ -157,8 +157,8 @@ class Sha256File(FactBase):
     def command(self, name):
         name = escape_unix_path(name)
         self.name = name
-        return 'sha256sum {0} 2> /dev/null || shasum -a 256 {0} 2> /dev/null \
-                || sha256 {0}'.format(name)
+        ret_str = 'sha256sum {0} 2> /dev/null || shasum -a 256 {0} 2> /dev/null || sha256 {0}'
+        return ret_str.format(name)

     def process(self, output):
         for regex in self._regexes:

Codecov Report

Merging #405 into master will decrease coverage by 0.19%.
The diff coverage is 81.25%.

@@            Coverage Diff             @@
##           master     #405      +/-   ##
==========================================
- Coverage   92.72%   92.53%   -0.20%
==========================================
  Files         105      105
  Lines        6019     5971      -48
==========================================
- Hits         5581     5525      -56
- Misses        438      446       +8

Impacted Files
Coverage Δ

pyinfra/operations/files.py
98.10% <0.00%> (-0.94%)
:arrow_down:

pyinfra/facts/files.py
99.11% <100.00%> (+0.21%)
:arrow_up:

pyinfra_cli/inventory.py
96.15% <0.00%> (-2.57%)
:arrow_down:

pyinfra/operations/server.py
99.12% <0.00%> (-0.88%)
:arrow_down:

pyinfra/api/connectors/util.py
78.81% <0.00%> (-0.85%)
:arrow_down:

pyinfra/api/connectors/sshuserclient/client.py
48.61% <0.00%> (-0.71%)
:arrow_down:

pyinfra/api/deploy.py
80.00% <0.00%> (-0.71%)
:arrow_down:

pyinfra/api/connectors/winrm.py
59.74% <0.00%> (-0.52%)
:arrow_down:

pyinfra/api/host.py
92.59% <0.00%> (-0.27%)
:arrow_down:

pyinfra/api/util.py
84.26% <0.00%> (-0.24%)
:arrow_down:

... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a30654...884f596. Read the comment docs.


NIFI-3343

Annotate client dto fields with JsonFormat annotation to allow default
Jackson ObjectMapper implementation to be able to deserialize JSON.
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

@jdye64 The client dto package was designed to be simple POJOs and with no dependencies to make it easy to consume for any clients of the REST API. Unfortunately, along the way we ended up adding a dependency on the documentation library (swagger).
This proposed change will add another dependency on a serialization library that the client may or may not be using. We have left all serialization logic to be specified via JAXB annotations to try to remain agnostic to whatever serialization or REST library the client is using.
In you happen to be using Jackson, things should (de)serialize correctly by just setting the (de)serializationConfig to use the JAXB annotation introspector. Here's an example from the NiFi codebase [1].
[1] https://github.com/apache/nifi/blob/master/nifi-commons/nifi-web-utils/src/main/java/org/apache/nifi/web/util/ObjectMapperResolver.java

@mcgilman that makes perfect sense. Thanks for pointing me to to code snippet. I'll close this PR and the JIRA


Add f to delimiter stripping for Python mode

So f-strings get terminated correctly.
This illustrates the problem I'm trying to fix.

Thanks. Merged as c45674b


Jason's first post

@wagerpascal pretty good1!  Pay attention to the commit titles- the automatically titled ones aren't very informative when we look to see what happened in what commit. :rose:


docs: Update documentation to include hints re async code

I wasn't sure how to refresh the widget with the data fetched from the API, so I opened an issue to get some help. It turns out it's pretty straightforward and should be done in the widget-task-handler.ts file.
I thought that having this documented with an example might be useful for other devs coming across this library.

It makes sense to add an example if it was not clear enough.
Thanks.


FATAL: 'external_hostname' is empty, please enter a valid String

The problem
Hello, im trying to configure the Addon on my Home Assistance instance and i get an error that the domain is not configured (but it it setup in the config).
What version of Cloudflared has the issue?
2.0.4
What was the last working version of Cloudflared?
No response
What type of installation are you running?
Home Assistant Supervised
Add-on YAML Configuration
No response
Anything in the logs that might be useful for us?
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/00-banner.sh
curl: (7) Couldn't connect to server
[11:01:27] ERROR: Something went wrong contacting the API
cont-init: info: /etc/cont-init.d/00-banner.sh exited 0
cont-init: info: running /etc/cont-init.d/01-log-level.sh
curl: (7) Couldn't connect to server
[11:01:27] ERROR: Something went wrong contacting the API
cont-init: info: /etc/cont-init.d/01-log-level.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-cloudflared-log: starting
s6-rc: info: service init-cloudflared-log successfully started
s6-rc: info: service init-cloudflared-config: starting
[11:01:28] INFO: Checking Add-on config...
[11:01:28] FATAL: 'external_hostname' is empty, please enter a valid String
s6-rc: warning: unable to start service init-cloudflared-config: command exited 1
/run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
/run/s6/basedir/scripts/rc.init: fatal: stopping the container.
s6-rc: info: service init-cloudflared-log: stopping
s6-rc: info: service init-cloudflared-log successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

Steps to reproduce the issue
I just did a normal installation, nothing different.
Additional information
Also im running HASS Supervised in Docker and i previously had DuckDNS setup but its uninstalled now.

Which host OS do you use? The error looks like you are using supervised with Ubuntu. Please note that only Debian ist supported by HA Supervised.
https://github.com/home-assistant/architecture/blob/master/adr/0014-home-assistant-supervised.md#supported-operating-system-system-dependencies-and-versions


[SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

What changes were proposed in this pull request?
Two new options, modifiiedBefore  and modifiedAfter, is provided expecting a value in 'YYYY-MM-DDTHH:mm:ss' format.  PartioningAwareFileIndex considers these options during the process of checking for files, just before considering applied PathFilters such as pathGlobFilter.  In order to filter file results, a new PathFilter class was derived for this purpose.  General house-keeping around classes extending PathFilter was performed for neatness.  It became apparent support was needed to handle multiple potential path filters.  Logic was introduced for this purpose and the associated tests written.
Why are the changes needed?
When loading files from a data source, there can often times be thousands of file within a respective file path.  In many cases I've seen, we want to start loading from a folder path and ideally be able to begin loading files having modification dates past a certain point.  This would mean out of thousands of potential files, only the ones with modification dates greater than the specified timestamp would be considered.  This saves a ton of time automatically and reduces significant complexity managing this in code.
Does this PR introduce any user-facing change?
This PR introduces an option that can be used with batch-based Spark file data sources.  A documentation update was made to reflect an example and usage of the new data source option.
Example Usages
Load all CSV files modified after date:
spark.read.format("csv").option("modifiedAfter","2020-06-15T05:00:00").load()
Load all CSV files modified before date:
spark.read.format("csv").option("modifiedBefore","2020-06-15T05:00:00").load()
Load all CSV files modified between two dates:
spark.read.format("csv").option("modifiedAfter","2019-01-15T05:00:00").option("modifiedBefore","2020-06-15T05:00:00").load()
How was this patch tested?
A handful of unit tests were added to support the positive, negative, and edge case code paths.
It's also live in a handful of our Databricks dev environments.  (quoted from @cchighman)

Test build #131281 has started for PR 30411 at commit 80170b4.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35885/

cc. @maropu @gengliangwang @cloud-fan @bart-samwel @zsxwing @dongjoon-hyun @HyukjinKwon
I've cc-ed all reviewers who left at least one review comment in original PR (#28841)
Just FYI to @cchighman as he's an author of original PR.

Test build #131284 has started for PR 30411 at commit 088b630.

First commit squashed the commits in origin PR. Remaining commits are my own to address my own review comments. So if you've done with the origin PR and it looked good to you, you may probably want to only look at remaining commits.

Test build #131284 has finished for PR 30411 at commit 088b630.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131284/
Test FAILed.

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35885/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35885/
Test PASSed.

Test build #131285 has started for PR 30411 at commit 71456f2.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35886/

Test build #131287 has started for PR 30411 at commit d216fcb.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35890/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35886/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35886/
Test PASSed.

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35890/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35890/
Test PASSed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35891/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35891/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35891/
Test PASSed.

Test build #131281 has finished for PR 30411 at commit 80170b4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131281/
Test PASSed.

Test build #131285 has finished for PR 30411 at commit 71456f2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131285/
Test PASSed.

Test build #131287 has finished for PR 30411 at commit d216fcb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131287/
Test PASSed.

Test build #131392 has started for PR 30411 at commit ce00b6d.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35996/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35996/

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35996/
Test FAILed.

Merged build finished. Test FAILed.

Thanks for the review, @maropu !
The origin PR has been open for months, and I only refactored a bit & fixed the doc. I'll merge this in early next week if there's no further comment.

The origin PR has been open for months, and I only refactored a bit & fixed the doc. I'll merge this in early next week if there's no further comment.

Yea, it looks fine to me. Thanks for the take-over, @HeartSaVioR and thanks a lot for the valuable contribution, @cchighman !

Test build #131392 has finished for PR 30411 at commit ce00b6d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131392/
Test FAILed.

Build failure is not related. I'll let it go and check just before the merge.

retest this, please

Test build #131506 has started for PR 30411 at commit ce00b6d.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36110/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36110/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36110/
Test PASSed.

Test build #131506 has finished for PR 30411 at commit ce00b6d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

OK. No further comments, and test passed. Merging to master.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131506/
Test PASSed.

Thanks all for reviewing and merging, and thanks again @cchighman for providing great stuff!

I am supportive of this change FWIW. I think it's good to have this feature.

Does it still need to be merged?

Does it still need to be merged?

No this is merged and will be available in new minor version, release 3.1.1.

No this is merged and will be available in new minor version, release 3.1.1.


Add DROP SHARD GROUP query

To evict old data manually, users should be able to drop shard groups. Query should be:
DROP SHARD GROUP <id>

Validation should be put in place to ensure that a shard group whose start and end times overlap with now can't be deleted. That is, you shouldn't be able to delete a shard group that is hot for writes.

Fixed via #6012!


Remove use of standard streams - introduce logging framework?

The existing code relies of System.err and e.printStackTrace() to output errors:

mostly in SQLiteJDBCLoader
there is some usage in OSInfo, but this can run as a command line so it should not be a problem
in some places in JDBC3DatabaseMetaData

In some places we also use DriverManager.println which seems to be the correct way to print JDBC connection related logs.
I would be enclined to remove the use of standard streams, and use a logging framework instead. Given we support Java 8, SLF4J is probably the best bet, though i noticed they finally released their 2.0 version, which kinda breaks existing apps.
Comments are welcome.

You have basically two options: SLF4J 1.7.x or JUL (the API, not the implementation). Have you checked with other popular JDBC drivers?

I think i have read somewhere that JUL before 1.9 was a no-no, but can't remember the details. Would need to investigate further.

Have you checked with other popular JDBC drivers?

pgjdbc uses JUL
H2 uses SLF4J
MySQL uses SLF4J

I think i have read somewhere that JUL before 1.9 was a no-no, but can't remember the details. Would need to investigate further.

Actually it's not about JUL, but JDK9 introduced System.Logger, which is akin to SLF4J. Some resources i found about it:

https://renato.athaydes.com/posts/java-system-logging.html
https://dzone.com/articles/system-logger
https://stackoverflow.com/questions/11359187/why-not-use-java-util-logging

In some places we also use DriverManager.println which seems to be the correct way to print JDBC connection related logs.

I checked on the web and could not find any good source of information about this. It's not used in pgjdbc nor H2 either.
If anyone has some information on that, i am quite interested.


Page Index Resets?

When I am handling server-side data and moving from page 1 to page 2, the page index resets to 1 every time. Any ideas why this could happen?

When using server-side data, there is only one page of data present in the table at a time. You must set the page prop to the page index you are displaying.

Is there a reason why page does not need to be displayed in the detailed Server-Side Data Example? There is a pages, but not a page.

In the example, there is no page prop, which means react-table will manage the state for the current page internally. When the Next button is clicked for example, the internal page index is incremented, then onFetchData is called with the new state. pages is something that cannot be managed internally by react-table, since it is information only the server knows about (since you are only displaying one page of data at a time). For this reason, you can optionally return a pages number to let react-table know how many pages are on the server for the given query.

I am still a bit confused. Initially, I set the page to 0. When I fetch the data, I set a store value for page to be the new index. However, it doesn't seem to change. Am I doing something wrong here?
<ReactTable      ...       pages={ store.getState().pages }       page={ store.getState().page }       manual       onFetchData={(state, instance) => {         DocumentActions.fetchDocs(state.pageSize, state.page, state.sorted, state.filtered);       }}

If you are using the page prop, then you would also need to use the onPageChange prop. Otherwise the page would always equal store.getState().page
...
page={store.getState().page}
onPageChange={newPage => DocumentActions.setPage(newPage)}
...

In following the example given above I'm running into the same issues, there seems to be a bit of a race condition when storing the page value in React setState.
... page={this.state.page} onPageChange={pageIndex => this.setState({page: pageIndex})} onFetchData={(state, instance) => {getData(state.pageSize, state.page)}} ...
When onFetchData gets call, setState has not finished updating the this.state.page value yet. Any ideas on what's going on?

I'm having the same issue, hopefully this issue will be re-opened

@kwallace50 I got the same problem. So the pageIndex got reset back to first value again and again. When I tried to set the value it will recursively change the state.
Any update on this?

I'm having the same issue, any help will be much appreciated.

Me as well. someone could bypass the problem?

@maxtotheguenther The widget itself is designed to do eager load. If you are going to do server pagination, you'd better hack a bit. Please let me know that you have a PR on this. For I myself I bypass by do eager load.

Any update on this issue?

To people who's using useTable hook, when using server-side data,You must set the pageIndex of initialState to the page index you are displaying.
Since pageIndex will be default as 0 if you don't set it. Every time you query data from server side it will rerender whole table page with new data and
initialState: {
        pageIndex: #pageIndex,
        pageSize: #pageSize,
      },

So if you don't set that pageIndex a dynamic value, it will rerender to its default value 0.
All overall, something like this:
= useTable(
    {
      columns,
      data,
      initialState: {
        pageIndex: controlledPageIndex,
        pageSize: 50,
      },
      manualPagination: true,
      pageCount: controlledPageCount,
    },
    usePagination
  )

controlledPageIndex is a prop from parent component.

Adding to the above comment, we should also set autoResetPage to false in order to prevent the table getting reset after the data has changed.
= useTable(
    {
      columns,
      data,
      initialState: {
        pageIndex: controlledPageIndex,
        pageSize: 50,
      },
      manualPagination: true,
      pageCount: controlledPageCount,
+     autoResetPage: false,
    },
    usePagination
  )

Adding to the above comment, we should also set autoResetPage to false in order to prevent the table getting reset after the data has changed.
= useTable(
    {
      columns,
      data,
      initialState: {
        pageIndex: controlledPageIndex,
        pageSize: 50,
      },
      manualPagination: true,
      pageCount: controlledPageCount,
+     autoResetPage: false,
    },
    usePagination
  )

this worked for me thank you

initialState: {
pageIndex: controlledPageIndex,
pageSize: 50,
},
manualPagination: true,

In version 8, it is autoResetPageIndex: false, instead of autoResetPage: false

Crazy how autoResetPageIndex is set to true by default, or am i missing something?

100% agree @oakis. I started looking into Github issues assuming this was a bug. For the majority of any true applicable use case, this should be set to false by default.

Crazy how autoResetPageIndex is set to true by default, or am i missing something?

It does indeed seem to be set to true whether you use server rendering or not. I also just spent hours trying to figure out why the pagination state was getting reset on the last render before trying to set autoResetPageIndex: false, which made the issue go away. I'm using 8.11.7.

you guys saved my day....
I was troubleshooting for hours trying to understand why it jumped back to the initial page after setting a page. Weird out-of-the-box behaviour

This should be false by default. Why would someone want to jump back to page 1 after they change/update some row data on page 5? weird. Thank you @GnaneshKunal  for saving me time.


Set up git hook to run flake8

I am working on this.

Reclaiming this one.

Nice job Silvia! Thank you for connecting it to RTD!


How to combine Metric3d and Droid-slam？

Hello, I ran Metric3d and Droid-slam separately yesterday, but I have a question. Can you provide some ideas on how to combine the two? The depth map is estimated through Metric3d and then input into Droid- Is auxiliary modeling performed in slam? Is this done to give the map a scale?

Hi, we initialize the droid's depth from metric3D, and using the rgbd mode in DROID-SLAM, i.e. constraining depth optimization not far from metric3D's prediction.


read replicated mount test

This test tries to replicate hyperdrive with existing mount with files into another hyperdrive and read it. At the moment the replication fails.

Closing this as per the comment in #258
Big thanks again for including a tape test with the issue. Makes debugging much easier on my end


Add an exception for no-unused-expression for quick null-checking

Bug Report

TSLint version: 3.15.1
TypeScript version: 2.0.2
Running TSLint via: CLI / WebStorm

TypeScript code being linted
  public ngOnDestroy(): void {
    this._indicatorsDataSubscription && this._indicatorsDataSubscription.unsubscribe();
  }

with tslint.json configuration:
    "no-unused-expression": true,

Actual behavior
TSLint reports left side of && operator as an unused expression.
Expected behavior
It would be nice to implement it as an exception. Otherwise user is forced to create if statements for those kind of disposals (especially annoying for disposing observables).

Duplicate of #1413
I've started to work on implementation, but postponed it because of vacation.
At this moment I have work in progress implementation that works on such simple example, but have false positives and false negatives in more complex cases.

Right, I had missed it. So my issue can be closed then, I will monitor #1413


collection_field_mapping support v.2.0

Until support for collection_field_mapping is deprecated, we need to support this configuration.
With the current changes, the importer will fail because the collection doesn't have a work identifier.
In the older version of bulkrax, there were default settings when creating a collection.
Bulkrax v2.0 removes the default settings. Therefore when the entry checks for missing required keys, an exception is raised.

Hi there --
Using either the to-be-deprecated configuration (with collection_field_mapping) or the parent-child configuration, Is it possible to import works into an existing collection that was created in the UI? For a collection created in the UI, I have the minted object ID (present in the URI), but I can't tell from the documentation whether Bulkrax can match on that ID, or if it needs a source_identifier for the collection, which I cannot provide in the UI. We are on Hyrax version 2.9.5.
Thanks for your help!

Hi @dolsysmith. It is possible to import works into an existing collection using the collection's ID with the parent-child configuration. You would simply put the ID in the same column that you would put the source_identifier.
For example, if you have configured your parent column to be called parents, your CSV would look something like this:

title
parents

My Imported Work
collection-id

There are more examples in the Wiki as well.
Hopefully that helps! If you have any other questions, let me know.

Hi @bkiahstroud, thank you for the response!
I have followed your example, and the following appears in my bulkrax.rb configuration:
config.field_mappings['Bulkrax::CsvEntry'] = {
      'parents' => { from: ['parents'], related_parents_field_mapping: true },
      'children' => { from: ['children'], related_children_field_mapping: true },
    }

However, when I attempt to import works to the collection whose ID I provide in the parents column, the works import without errors but are not added to the collection. When comparing the Raw Metadata and Parsed Metadata fields on the importer/entries page, I can see that the parents field is not being mapped to anything. (Not sure if that matters.)
I am trying to import into a public (not private) collection defined by my user in the UI.

I can see that the parents field is not being mapped to anything

@dolsysmith do you mean that "parents" does not show up at all in raw and/or parsed metadata? Or that it shows up, but the value is empty?
Also, would you mind showing me your full config.field_mappings? (I'm assuming parents and children are two of several mappings set up in the bulkrax.rb config file)

Sorry for not being clearer. parents shows up as a field in the raw metadata, but not the parsed metadata.
Here are the field mappings in bulkrax.rb:
config.field_mappings['Bulkrax::CsvParser'] = {
    "contributor" => { from: ["contributor"] },
    "creator" => { from: ["creator"], split: "; " },
    "date_created" => { from: ["date_created"] },
    "description" => { from: ["description"] },
    "identifier" => { from: ["identifier"] },
    "related_url" => { from: ["related_url"] },
    "rights_statement" => { from: ["rights_statement"] },
    "license" => { from: ["license"] },
     "source_identifier" => { from: ["source_identifier"] },
    "keyword" => { from: ["keyword"] },
    "title" => { from: ["title"] },
    "resource_type" => { from: ["resource_type"] },
    "gw_affiliation" => { from: ["gw_affiliation"] },
    "parents" => { from: ["parents"] }
  }

  config.field_mappings['Bulkrax::CsvEntry'] = {
      'parents' => { from: ['parents'], related_parents_field_mapping: true },
      'children' => { from: ['children'], related_children_field_mapping: true },
    }

After unsuccessfully trying to import to a collection, I added parents to the CsvParser hash above, but it seemed to make no difference.

Ah, I think I see what the problem is. The documentation was inaccurate -- the relationships field mappings should be assigned to the CsvParser, not the CsvEntry (I just fixed this in the docs).
So the field_mappings should look like this:
config.field_mappings['Bulkrax::CsvParser'] = {
    "contributor" => { from: ["contributor"] },
    "creator" => { from: ["creator"], split: "; " },
    "date_created" => { from: ["date_created"] },
    "description" => { from: ["description"] },
    "identifier" => { from: ["identifier"] },
    "related_url" => { from: ["related_url"] },
    "rights_statement" => { from: ["rights_statement"] },
    "license" => { from: ["license"] },
     "source_identifier" => { from: ["source_identifier"] },
    "keyword" => { from: ["keyword"] },
    "title" => { from: ["title"] },
    "resource_type" => { from: ["resource_type"] },
    "gw_affiliation" => { from: ["gw_affiliation"] },
    'parents' => { from: ['parents'], related_parents_field_mapping: true },
    'children' => { from: ['children'], related_children_field_mapping: true }
  }

Sorry for the confusion! Please let me know if that fixes the issue for you.

Thank you! Importing a CSV with a collection ID in the parents column now adds the items to the collection.

Awesome! Thank you for your comment, we refer users to our Wiki very often so it was really important that that got resolved. Please feel free to open new issues etc. if you have more questions in the future.


Image results grid improvement

Problem
The current image result view does not display the CC-license icons on sm breakpoint. The last mockup shared showed the icons below each image in a 2 columns grid, but there is an ongoing proposal for supporting 3d model content that needs to be taken into account.
Issues related

Problem detected: #1366
3d model support proposal: #1432
#1395

Proposal
Mobile
https://user-images.githubusercontent.com/895819/172160157-e260adfb-9c50-4d92-aee5-757fff81b26f.mp4
Desktop
https://user-images.githubusercontent.com/895819/172160850-0b90336b-a6d8-4502-9c29-d3814126c3cd.mp4
Description
On mobile, the layout displays images on a single column grid where each image has its cc-license icons below. The image component has three states: resting, hover, and focus. The video attached above shows the first and third ones, and the style changes are the focus ring and a color shift in the icons area from dark-charcoal-70 to dark-charcoal.
Since #1432 proposal suggests new hover styles, the image component inherits the cc-icons area change. On the other hand and time ago, I had the idea of reviewing the row height of the current grid as it looks good on big screens but tight on laptop sizes. See the image below of the current layout.

I was playing with different heights, and I like how 200px for the row height looks like. It will show three full rows, acknowledging its relativity due to device sizes, and considering that the WordPress header will be removed, the page will look clean and highlights more the content results.
See the image below and please share your thoughts.

Design file
Figma: Image results grid improvement

I think this looks quite nice. You don't have to mock it up, but I'm personally really curious what it would look like to show the icons on the desktop grid all of the time, below each image as on mobile.

It looks good to me too. Does this address the perceived header button 'misalignment' by keeping the button borders in the header when the page is scrolled?

@zackkrida, It will break the white space balance between rows since there will be more space for the license icons row. I am following a desktop consistency approach for all big screen breakpoints, the hover style in All results page is the same.
@sarayourfriend, This is the approach I am following:

The default header has the same left and right paddings as the content grid. The filters button wraps the header area through its borderline.
The scrolled header has fewer left and right padding values since the openverse, content switcher, and filters buttons outline the header differently. In this case, the bottom border of the header makes it feel outer and above the grid layout. This effect is intentional to highlight the content area and link the filter sidebar visually. It is a nav component placed above everything.

Default header
Scrolled header

Does this mean the header will shift "outwards" when you scroll? I worry that will be a lot of auxiliary movement in addition to the scrolling movement that someone is already trying to focus on.

I like that there is a border between the nav and the content below it but the horizontal expansion would be very strange considering the main vertical movement occurring during the scroll.

I tested different versions of the scroll behavior, with and without the spacing change, and the cleanest direction is no change at all of the paddings. Here is a prototype.
https://user-images.githubusercontent.com/895819/175322413-c1078f56-6556-4d91-86cf-6530fa40772f.mp4
The only change I am keeping is the border change for the content switcher and filter buttons. Nonetheless, in parallel, I am working on the new header, and this input is crucial for defining the new spacings.
Even when this issue is for the image grid, I am glad that we are touching the nav component so I tackle both design improvements.

@panchovm have we thought about the search result / header alignment issued discussed here as it relates to the new header in #1669 ?

Yes. Here is a quick update of the image results with the new header.


Provide implementation of futures::io::AsyncRead and futures::io::AsyncBufRead for Vec?

I found that only &[u8] is implemented of futures::io::AsyncRead and futures::io::AsyncBufRead, and Vec<u8> is implemented of futures::io::AsyncWrite.
But sometimes I need to push a param to a third-part function which accept a type need to implement both AsyncRead and AsyncWrite, but Vec<u8> can't, so can you provide it?

Directly implementing read for vec is probably not what you want, it has nowhere to store the currently read offset.
The question would be what sort of AsyncRead + AsyncWrite is expected? A network connection like one where the two halves are independent would be possible by combining a &[u8] backed reader with a Vec backed writer. A file like one where reading and writing affect each other would be possible to emulate using something like Cursor<Vec<u8>>

@Nemo157 Thanks, I think Cursor is a good choice.


Layouts can/may use old window reserved area value

Hyprland Version
 System/Version info
Hyprland, built from branch main at commit 2e763764bfdaee2039961c6c220b5d6af31712d4  (pointer-constraints: don't warp on entering a confined pointer).
Date: Sat Apr 27 16:15:33 2024
Tag: v0.39.1-91-g2e763764, commits: 4551

flags: (if any)

System Information:
System name: Linux
Node name: blkheart
Release: 6.8.7-zen1-2-zen
Version: #1 ZEN SMP PREEMPT_DYNAMIC Fri, 19 Apr 2024 09:51:33 +0000

GPU information:
0d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] [1002:73bf] (rev c0) (prog-if 00 [VGA controller])

os-release: NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

plugins:

Bug or Regression?
Bug
Description
If a window decorator dynamically changes the size of the decoration and calls repositionDeco() the layout won't be aware of the change until the next time something triggers a layout/window recalculation.
How to reproduce
I don't think there's an existing deco that triggers this: I discovered it while adding vertical tabs to the GroupBar deco.
When the active/visible window changes in the group, the sequence is:
layout receives replaceWindowDataWith, which calls
applyNodeDataToWindow, which asks the window for its reserved area.
It will get the OLD value of the reserved area for that window, which is whatever the reserved area was when the window was first added to the group.
The layout then sizes the window with the wrong reserved area.
The decos draw() call is eventually called and then it notices the difference in assigned geometry vs what it wants and calls repositionDeco().
This triggers the reserved area recalculation, but there's no trigger to tell the layout to readjust.
I can "fix" it by calling recalculateWindow in onPositioningReply, but this feels like a more general problem that decos shouldn't have to deal with individually.
Can the deco positioner just call recalculateWindow when it changes the reserved area, or is there a better/more efficient way?
Crash reports, logs, images, videos
No response

deco positioner should do it probably yea

fixed in d2899a6c272d4619295543b5b4c77b40fa1934bf


Using this ce_sdk in Win10_Unityx64

Hi mistafunk
I'm a user of city engine who want to generate building in Unity SDK. I can't export my model to fbx because my building model include too much area (I think this is the reason but not sure). I try to use your repo to import CityEngine SDK in my Unity but I have some problem.

My Visual studio is 2017 version (prt4unity required 2010), I re-target project and change platform tool-set  to v141 (VS 2017 support)
My CityEngine and Unity is x64, I'm not sure but I think your repo created for x84 user isn't it? When I run deploy.cmd I found two directories in top-level of repo. that is  "ProgramFiles(x86)\Unity\Editor" and "MyUnityProject\Asset\Plugins|Prt4Unity..." but all folder didn't have any file.

I'm not sure what is my mistake about this solution but I think I follow your Read me document, please recommended how can I fix this mistake.
Thank you in advance

hi PaPeaRL,
unfortunately, the prt4unity example is deprecated and we do not support it anymore. the only way forward is to upgrade the example code to a current unity version. additionally, i doubt that you can circumvent size issues encountered with FBX by generating the model inside unity. we are still hoping to resurrect support for the unity example in the future.
apologies for the mixed news & kind regards,
simon


Is there any docs about config sbt globally?

Not found valid settings about configure sbt globally, such as how to config "~/.sbt/" dir.
Version 0.13

Hey @acearth, here's the page in the docs we have for global settings: http://www.scala-sbt.org/0.13/docs/Global-Settings.html
Also, please have a look at this section about where to get help: https://github.com/sbt/sbt/blob/1.0.x/CONTRIBUTING.md#where-to-get-help-andor-file-a-bug-report
Finally, if you notice any issues in the documentation, we track and make changes to it at https://github.com/sbt/website.


Feature: Filter for Malware or specific vulnerability identifiers

Pre-submission checks

[X] I am not reporting a new vulnerability or requesting a new vulnerability identifier. These must be reported or managed via upstream dependency sources or services, not this repository.
[X] I agree to follow the PSF Code of Conduct.
[X] I have looked through the open issues for a duplicate request.

What's the problem this feature will solve?
It would be nice to have a command line flag to filter for packages containing malware. Within OSV.dev, malware receives the identifier prefix of "MAL-".
Describe the solution you'd like
The additional of command line arguments to help confirm vulnerabilities of interest.
Option 1) supply a vulnerability identifier search options:
--id, -i VULNID          Search for the vulnerability identifier specified by VULNID
Option 2) filter just malware
--malware, -m           Filter searches to only malware identifiers which are prefixed with "MAL-"
Additional context
Recently the fabrice typo squatting malware received an OSV.dev identifier:
https://osv.dev/vulnerability/MAL-2024-10573
Attempting to confirm positive identifications (for just malware) using pip-audit requires additional shell scripting.

Understood. It looks like Python advisory DB is missing a lot of details on malicious packages (e.g. typo squatting).
In lieu of a categorical filter, an option to look for specific vulnerability identifiers would be helpful.

Understood. It looks like Python advisory DB is missing a lot of details on malicious packages (e.g. typo squatting).

This would be good to raise with them! I don't know whether they consider malicious packages to be within their remit.

In lieu of a categorical filter, an option to look for specific vulnerability identifiers would be helpful.

This is what we intend the JSON and SBOM outputs for -- I've been hesitant to bake more filtering logic into pip-audit itself, since in the general case it starts to resemble predicates, etc. that are best handled by ingesting pip-audit into your database of choice and performing queries from there (or doing | jq <filter> on the output).
I understand not wanting to write glue code (I don't like writing glue either 🙂), but in a case like this I think it's important for pip-audit to define strong boundaries around what it will and won't do, to keep the tool's scope tractable.

Your note about using the jq command line was very helpful; I'd missed that detail in the usage docs. Here's the syntax i chose:
pip-audit -f json 2>/dev/null | jq '.dependencies[].vulns[].id | select(. == "MAL-2024-10573")'


Understand and reproduce the work

Hi,
Thanks for your great work!
I'm trying to reproduce the ViLD baseline in your repo but still having some trouble understanding it. Here are my questions:

In readme, the configs of ViLD and DetPro both point to the same file: detpro_ens_20e.py and you have confirmed in other issues that this is correct. In this case, how would you determine whether the model uses learnt prompt as in DetPro or manual prompt as in vanilla ViLD. Could you please share the running command for reproducing ViLD? Really appreciate it.

In those configs, they both use Shared4Conv1FCBBoxHead as the RoI head. However, this module seems not leverage text embeddings for training and inference. Should something like StandardRoIHeadTEXT be used instead? I'm not sure if my understanding is correct and please do correct me if my understanding is wrong.

Thanks for your time again!

Hi,
Thanks for your great work!
I'm trying to reproduce the ViLD baseline in your repo but still having some trouble understanding it. Here are my questions:

In readme, the configs of ViLD and DetPro both point to the same file: detpro_ens_20e.py and you have confirmed in other issues that this is correct. In this case, how would you determine whether the model uses learnt prompt as in DetPro or manual prompt as in vanilla ViLD. Could you please share the running command for reproducing ViLD? Really appreciate it.
In those configs, they both use Shared4Conv1FCBBoxHead as the RoI head. However, this module seems not leverage text embeddings for training and inference. Should something like StandardRoIHeadTEXT be used instead? I'm not sure if my understanding is correct and please do correct me if my understanding is wrong.

Thanks for your time again!

Hi，just replace the prompt path with the manual defined prompt which is lvis_clip_text_embedding.pt provided in the link or generated by yourself (I have written the generating process in the standard_roi_head.py).
You can try this
./tools/dist_train.sh  configs/lvis/detpro_ens_20e.py 8 --work-dir workdirs/vild --cfg-options model.roi_head.prompt_path=lvis_clip_text_embedding.pt  model.roi_head.load_feature=True


[wip] Fixed the issues and warnings from linters

This PR would fix all issues and warnings produced by golangci-lint static check.

Wow, I owe you a beer!

Thank you


file:rs_rgbd_launch error&  Object Analytics (OA) unavailable

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
D435i

Firmware Version
<IP_ADDRESS>

Operating System & Version
Linux (Ubuntu 18.04)

Kernel Version (Linux Only)
5.4.0-150

Platform
PC

SDK Version
2.54.1

Language
opencv

Segment
Robot

Issue Description
Hello:
We encountered several issues while trying to use a depth camera to locate an object:

We found that an error was reported when running the file:    rs_rgbd_launch :
[ERROR] [1693205097.460124267]: Failed to load nodelet [/camera/realsense_ros_camera] of type
[realsense_ros_camera/RealSenseNodeFactory] even after refreshing the cache: Could not find library corresponding to plugin
realsense_ros_camera/RealSenseNodeFactory. Make sure the plugin description XML file has the correct name of the library and that
the library actually exists.
[ERROR] [1693205097.460205152]: The error before refreshing the cache was: Could not find library corresponding to plugin
realsense_ros_camera/RealSenseNodeFactory. Make sure the plugin description XML file has the correct name of the library and that
the library actually exists.
[FATAL] [1693205097.463399012]: Failed to load nodelet '/camera/realsense_ros_cameraof type
realsense_ros_camera/RealSenseNodeFactoryto managerrealsense_ros_camera_manager'
[camera/realsense_ros_camera-3] process has died [pid 33838, exit code 255, cmd /opt/ros/melodic/lib/nodelet/nodelet load
realsense_ros_camera/RealSenseNodeFactory realsense_ros_camera_manager __name:=realsense_ros_camera
__log:=/home/yang/.ros/log/6685f41c-456e-11ee-b3dd-000c29862474/camera-realsense_ros_camera-3.log].
log file: /home/yang/.ros/log/6685f41c-456e-11ee-b3dd-000c29862474/camera-realsense_ros_camera-3*.log

2.We tried the Object Analytics provided by your official website but because of the software version that does not match we faild to run it. So we are wondering if there is a similar ROS wrapper that can be used on ubuntu 18.04 or 20.04.
3.As beginners,we would appreciate it if you can inspire us of how to use the camera to obtain the three-dimensional coordinates of a specific object
Thank you inadvance for your help!

Hi @NET-Y  I would not recommend using the ROS1 version of Object Analytics because of its significant age.  It may be possible to get it working using ROS1 Kinetic, Ubuntu 16.04 and the 2.3.2 version of the RealSense ROS1 wrapper that supports Kinetic, but it is likely not worth doing so.
The 2.3.2 ROS1 wrapper works with ROS1 Kinetic / Melodic / Noetic and Ubuntu 16.04 / 18.04 / 20.04.  It should be used with librealsense version 2.50.0 or 2.51.1.  Because development of the ROS1 wrapper has ceased and only the ROS2 wrapper is updated, the ROS1 wrapper does not have support for librealsense 2.54.1.
https://github.com/IntelRealSense/realsense-ros/releases/tag/2.3.2
If you need to perform object detection in ROS1 with Ubuntu 18.04 then it would be preferable to use the Melodic branch of darknet_ros
https://github.com/leggedrobotics/darknet_ros/tree/melodic
If you require 3D coordinates then you could use the Melodic or Noetic branch of darknet_ros_3d
MELODIC
https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d/tree/melodic
NOETIC
https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d/tree/noetic

Hi @NET-Y I would not recommend using the ROS1 version of Object Analytics because of its significant age. It may be possible to get it working using ROS1 Kinetic, Ubuntu 16.04 and the 2.3.2 version of the RealSense ROS1 wrapper that supports Kinetic, but it is likely not worth doing so.
The 2.3.2 ROS1 wrapper works with ROS1 Kinetic / Melodic / Noetic and Ubuntu 16.04 / 18.04 / 20.04. It should be used with librealsense version 2.50.0 or 2.51.1. Because development of the ROS1 wrapper has ceased and only the ROS2 wrapper is updated, the ROS1 wrapper does not have support for librealsense 2.54.1.
https://github.com/IntelRealSense/realsense-ros/releases/tag/2.3.2
If you need to perform object detection in ROS1 with Ubuntu 18.04 then it would be preferable to use the Melodic branch of darknet_ros
https://github.com/leggedrobotics/darknet_ros/tree/melodic
If you require 3D coordinates then you could use the Melodic or Noetic branch of darknet_ros_3d
MELODIC https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d/tree/melodic
NOETIC https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d/tree/noetic

Thank you for your reply! We are trying darknet_ros_3d to reach our goal.  We have completed the configuration of Darknet Ros. But When configuring darknet_ros_3d，it reminds us of the lack of gb_visual_detection_3d_msgs. We have no idea which folder to put this file in. And we fail to use CMake to compile the file. We would appreciate it if you could provide us with some suggestions.Thank you for your help!

I do not know for certain where to clone gb_visual_detection_3d_msgs to, but the logical location would be the src catkin workplace, where darknet_ros and darknet_ros_3d are also installed.

I do not know for certain where to clone gb_visual_detection_3d_msgs to, but the logical location would be the src catkin workplace, where darknet_ros and darknet_ros_3d are also installed.

Thank you for your reply.We have solved the problem .So I guess this issue can be closed?

It's great to hear that you succeeded!  As you have done so, I will close the case.  Thanks again!


How were the .dll and .so files for this project built?

Hello!
I want to know how you built the .dll and .so files for this project so I can try to build one on Mac one for personal use or even for use on this repo but I doubt you will trust me to that extent
Thank you for you time :)

By building https://github.com/effekseer/EffekseerForMultiLanguages

Thanks 🙏


ability to change repository path for the buffer-metrics-sidecar

We would like to be able to change the repository path for the buffer-metrics-sidecar image, so that we can pull it via our proxy cache. I found how to do this for the config-reloader image (logging.fluentd.configReloaderImage.repository), but I don't see a corresponding way to do it for the metrics sidecar

@ajchiarello it's logging.fluentd.bufferVolumeImage


Feature request: Parse escaped string in reverse

Summary
Would like to replace line breaks etc with escape strings for use on command line (eg printf "1.\n2.").
Example

to
1.\n2.\n
etc

Yes, good spot. Thanks!


phosh: 0.41.1 -> 0.43.0, {phosh-mobile-settings,phoc}: 0.41.0 -> 0.43.0, squeekboard: 1.41.0 -> 1.43.0

https://phosh.mobi/releases/rel-0.42.0/
https://phosh.mobi/releases/rel-0.43.0/
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 347307 run on x86_64-linux 1

4 packages built:

phoc
phosh
phosh-mobile-settings
squeekboard

Breaks phosh.tests.phosh
https://gist.github.com/Artturin/e37a5e00f9307fb841ab1b2c855d5909

@zhaofengli Any ideas since you're the maintainer of the tests?

@donovanglover, in the test logs it says there was a problem with phosh-mobile-settings.
It was looking for the text "Tweak advanced mobile settings" to appear on screen, after typing "mobile setting" at the home screen.
Do those steps work for you? Perhaps some strings need to be updated in the tests?

phosh is now completely broken since 9673dfc9770257cc8b48fed660fda689fd7c1bd2 (first noticed on my PinePhone). This can also be seen in the NixOS test. I should probably bisect seatd now.


issue #636  @censor_word_detection

Description
Please include a summary of the change and which issue is fixed. List any dependencies that are required for this change.
Fixes #636
Replace issue_no with the issue number which is fixed in this PR
Have you read the Contributing Guidelines on Pull Requests?

[ ] Yes
[ ] No

Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Documentation Update

Checklist:

[ ] My code follows the style guidelines(Clean Code) of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md
[ ] My documentation follows Template for README.md
[ ] My changes generate no new warnings
[ ] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works.

changes are done
On Sat, Mar 20, 2021 at 2:35 PM Kaustubh Gupta @.***>
wrote:

@.**** requested changes on this pull request.
In Censor word detection/README.md
https://github.com/avinashkranjan/Amazing-Python-Scripts/pull/680#discussion_r598085662
:

+Requirements
+

Run the command:

pip install better_profanity
+

+ Approach
+

Import the module profanity from better_profanity

Take user input

Check for the censored data using method profanity.censor()

Print the output on terminal.

It will print **** if profanity.censor(text) is used.

this never printed?
In Censor word detection/censor_word_detection.py
https://github.com/avinashkranjan/Amazing-Python-Scripts/pull/680#discussion_r598085715
:

@@ -0,0 +1,14 @@
+from better_profanity import profanity

+def detect(data):

try:

   '''profanity.censor(data) will print **** '''

this line is not need as it never printed ***
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/avinashkranjan/Amazing-Python-Scripts/pull/680#pullrequestreview-616896692,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AI6BY4SNSKZQ53746N7J62TTERQMPANCNFSM4ZQDHQ2Q
.

Done
On Sat, 20 Mar, 2021, 5:38 pm Kaustubh Gupta, @.***>
wrote:

@.**** commented on this pull request.
In Censor word detection/censor_word_detection.py
https://github.com/avinashkranjan/Amazing-Python-Scripts/pull/680#discussion_r598103667
:

@@ -0,0 +1,14 @@
+from better_profanity import profanity

+def detect(data):

try:

   '''profanity.censor(data) will print **** '''

@jeelnathani https://github.com/jeelnathani this is still left
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/avinashkranjan/Amazing-Python-Scripts/pull/680#discussion_r598103667,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AI6BY4XEOLHRJGJDEEJVBVDTESF27ANCNFSM4ZQDHQ2Q
.


Should ally.js support older browsers?

Browser support as of ally.js 1.0.1:

ally.js has been tested on IE10+, Android 4.4+ and iOS8+ alongside the latest desktop versions of Firefox, Chrome and Safari.

For me and my projects that's good enough. For future jQueryUI that is good enough, but for current jQuery to use ally.js we'd have to support IE8+.
This issue is here to collect your thoughts on the minimum browser version (specifically IE, but also Android WebView and iOS Safari) that ally.js should support. We're trying to assess where we should focus development time…
Simply comment with "IE8", "IE9", "IE10" (which means no change). If you see any other browsers (like UC Browser or Yandex Browser) in need of special attention, call for it.

Did you run your tests against the mentioned Browsers just to see where we're at? Maybe it's not "a whole lot" of extra work? Is it?

Did you run your tests against the mentioned Browsers just to see where we're at? Maybe it's not "a whole lot" of extra work? Is it?

not yet. not a bad idea, why did I not think of that?

:+1:

"IE8"
To clarify, I don't mind if features don't necessarily work, but the library should degrade gracefully so that I can use ally.js and not worry about it breaking other scripts on older browsers. No fatal errors should be thrown.
I guess it would also be good to highlight in the docs any exceptions to the general browser support for each feature/ component.

To clarify, I don't mind if features don't necessarily work, but the library should degrade gracefully so that I can use ally.js and not worry about it breaking other scripts on older browsers. No fatal errors should be thrown.

@braders: How would you make an ES5 library not fail in an ES3 browser (without including the ES5shim) though? Simply overwrite all ally.* functions with no-ops?

I guess it would also be good to highlight in the docs any exceptions to the general browser support for each feature/ component.

@braders: Are ally.is.focusRelevant notes and ally.maintain.disabled notes examples of that, or did you have something else in mind?

How would you make an ES5 library not fail in an ES3 browser (without including the ES5shim) though? Simply overwrite all ally.* functions with no-ops?

I'm thinking basic feature detection - If functions are used that simply don't exist in IE < 8 then break out of the relevant function. Something like
if( ! 'querySelector' in document) {
     return
}
//run the function in modern browsers

I guess it would also be good to highlight in the docs any exceptions to the general browser support for each feature/ component.

Exactly like that - Didn't notice that earlier. Great job.

I'm thinking basic feature detection - If functions are used that simply don't exist in IE < 8 then break out of the relevant function […]

I know the concept (and ally.maintain.disabled is doing that in case of MutationObserver on IE10). But since everything requires ES5, I would have to break out of everything, rendering the library completely useless on ES3 (… but not throwing any errors).

I've actually just noticed 'ES5-shim' is listed as an implict dependency on the readme. I've not seen it mentioned on the website though (might have missed it). Maybe it should be?
Sounds like it would be a big effort to add ie8 + 9 support without the polyfill though and I can't see people much caring as long as it works if the polyfill in place.

I've actually just noticed 'ES5-shim' is listed as an implict dependency on the readme. I've not seen it mentioned on the website though (might have missed it). Maybe it should be?

I guess Getting started is a good place to repeat the "system requirements"…

Sounds like it would be a big effort to add ie8 + 9 support without the polyfill though and I can't see people much caring as long as it works if the polyfill in place.

The ES5 shim is only necessary for IE8 (ES5 compat table).

:+1: Sounds good

I've started looking at adding support for IE9 in the branch issue/71-support-ie9. At this point the ally.js is fully functional in IE9 (as in all tests are green). The file size rose from 79K to 82K (gzipped 19K to 20K). ally.is.focusable shows 4 false positives for <object> and <keygen> (which still needs fixing).

As discussed in #112 we've made sure that ally.js can be loaded in IE8. executing any of the methods is a different story. Since Intern does not run on IE8, we have no way of verifying the libraries integrity on that browser.
Other than Adding full support for IE9 and loading support for IE8, we've switched UA sniffing from browser to rendering engine, in the hopes to catch more things on android browsers - which we are currently not testing automatically.

Since you mentioned jQuery - we've just rekeased jQuery 1.12 supporting IE6+ & 2.2 supporting IE9+. Those will be the last releases in the 1.x/2.x lines, we're now preparing for a jQuery 3.0 release that will only support IE9+. We're officially ending jQuery IE8- support.

Thanks for the heads up @mgol, much appreciated!


Rename to examples-go

We'll eventually want examples in lots of languages, so we should probably rename this repo to examples-go.

We'll rename at the end of day today (Wed, Sept 23).

Fixed.


bug: Star command is not giving a random entry on PGSQL

Describe the bug
https://github.com/kyranet/Skyra/blob/7f21f1c13924abb5277a37608704649c6bf113d3/src/lib/queries/postgres.ts#L182-L194
The SQL query does not sort the entries by random
To Reproduce
Steps to reproduce the behavior:
Write s!star
Write s!star
Both messages are the same
Expected behavior
Both messages are (ideally) random
Screenshots
On a server with +800 starboard entries:

Additional context
None.

Is the behavior also found with the JSON provider?

No, just PGSQL as the title describes.

Hmm, odd. A potential fix would possibly to be excluding the previous messageID from the query? But that would cause extra complications with how to pass it and how it's stored.

No, the correct way to do it is in here


remove bold letter and extra pranthesis

Signed-off-by: Kavitha Conjeevaram Mohan<EMAIL_ADDRESS>Description
Remove bold letter for since and remove unwanted paranthesis
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #559 (5db2a24) into main (6843640) will increase coverage by 19.04%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##               main     #559       +/-   ##
=============================================
+ Coverage     53.29%   72.33%   +19.04%
  Complexity      271      271
=============================================
  Files           236       38      -198
  Lines          7763     2136     -5627
  Branches       1601      236     -1365
=============================================
- Hits           4137     1545     -2592
+ Misses         3430      429     -3001
+ Partials        196      162       -34

Flag
Coverage Δ

dashboards-observability
?

opensearch-observability
72.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ponents/common/helpers/ppl_docs/commands/search.ts

...er/docTable/detailTable/table_row_btn_collapse.tsx

...lic/components/common/search/autocomplete_logic.ts

...ig_editor/config_controls/config_value_options.tsx

..._analytics/components/traces/span_detail_table.tsx

...lic/components/visualizations/assets/chart_bar.tsx

.../components/visualizations/charts/maps/heatmap.tsx

...onents/explorer/visualizations/lens_field_icon.tsx

...nts/visualizations/charts/shared/shared_configs.ts

...public/components/explorer/docTable/docViewRow.tsx

... and 188 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6843640...5db2a24. Read the comment docs.


Replace alpine with debian-slim

Updates the images we use when building Sonobuoy.
For the build image this also updates go from
1.12.0 to 1.12.1.
Due to the fact that golang doesnt offer a
stretch-slim image, the 1.12.1 stretch image
is used.
Stretch slim seems to come with what it needs
so the dockerfile CMD1 can just be removed. For
ease of future changes I left the CMD1 in place and,
just like on arm64, we just remove the line rather
than replace it.
Fixes #609
Signed-off-by: John Schnake<EMAIL_ADDRESS>

@emuetzel for building against the arm architecture is the Ubuntu image still appropriate/OK?

Right now trying to diagnose the failure; it does seem to be a legitimate hang right now.
When you try and retrieve the results you get the results as expected, but then it hangs as if it doesnt realize the stream is done. Presumably, something is different between the two versions of tar used on the different images.
In stretch-slim it is GNU tar 1.29 and on the older alpine image it is tar (busybox) 1.29
In both cases when I ruun the tar command I get some messages on stderr but we just ignore those at this time. On both OSs, the tar command finishes. Seems like the stream doesnt get closed or some terminal character isnt found now.

Swapped back and forth between new/old images with a local client. The alpine image worked fine even with the new client so the problem is on the container side. Adding debug messages showed that the problem has to do with stdout not being closed ever; it is just keeps the stream open for some reason. More investigation required.

I'm fairly boggled right now. Kubectl exec works just as well as it did before (or seems to) with the new image. However there is still the problem that the stdout stream doesnt ever get closed for some reason so we hang at copying stdout, effectively waiting for the command to finish.
Once the command would finish, the stdout reader should have an eof/break that causes the copy to end. Since updating to debian that doesnt happen. Will keep working on it.

I'm not 100% sure on the Ubuntu one (double checking with @timh). My other issue is, the arm e2e images don't work (or exist), so I'd actually recommend removing the arm builds until they are fixed.

@stevesloka Not sure we're on the same page on the arm one: it exists here https://hub.docker.com/r/arm64v8/ubuntu/ and I was able to do a docker pull arm64v8/ubuntu:16.04 without problem. We have talked in the past and agree that we need to either test it in a meaningful way if we're going to build it though. I can make a ticket for that (either test or remove and let the discussion play out there)

Solved
So the issue is in the way that the different tar implementations work in the different platforms.
The reason the command hung had nothing to do with the connection being left open, but instead was the fact that in one build the tar library seemed to realize that it was seeing an end of archive marker (lots of 0s to indicate the end of the archive) and stopped reading. The fact that it stopped reading (in one platform and not the other) caused the io.Pipe to stop being able to write to the other side (it had more 0s to write) and so the io.Copy hung.
This is all remedied by adding some draining function to ensure the pipe is cleared. If any non-zero data is read we can throw an error/warning otherwise the 0s are harmless. Going to look upstream and see if this is a problem too (we got this code from upstream originally)

@johnSchnake My point on arm is that if you do a sonobuoy run on an arm system, it will fail because upstream images don't exist. So we're essentially making a build which cannot run an E2E plugin. It would still work for other custom plugins, but not sure if anyone is actually doing that.

@stevesloka you're right, I misread your comment regarding the e2e images. Sorry about that. I'm with you otherwise about arm. I made #636 earlier today.

No worries, I wasn't all that clear in my other comment. =)

@johnSchnake - if you need GNU tar for this to work, you should install GNU tar everywhere - the busybox tar is no where near as complete.

It was actually the opposite problem; we went from alpine (which used the busybox tar) to the gnu tar. Something about the way the tar library is built and/or the end-of-archive markers meant that the tail of the archive (even though it was just 0s) was getting left in a reader which we expected to have been drained.
It didnt really have anything to do with tar itself, but I think much more likely something about the build of the stdlib tar reader leaving the 0s unconsumed on that platform.