Ability to use default/provided workflows, modify defaults, or create new workflows from scratch for funds/labs

As a manager, I want the ability to use default/provided workflows, modify them, or create new workflows from scratch for funds/labs
Is your feature request related to a problem? Please describe.
Currently, Hypha only support 4 'hard-coded' workflows for funds/labs. The flexibility in these is limited (but not impossible) however, building fully flexible configurable workflows in Wagtail/Hypha would allow for less reliance on developers to make changes to workflows adhoc and allow for more experimentation for funders when they want to try different funds/labs.
This could also enable adopters to 'share' workflows and rationale for workflows in an 'OSS content' way.
This is a very large 'feature' and is more at the Epic/META level in current description.
Is your feature request related to an existing functionality? Please describe.
Related to:

Existing workflows
Forms/fund creation in Wagtail
OSS content sharing
...plenty more!

Describe the solution you'd like
An idea of how to approach this could be a 'fund/lab workflow builder' in a sort of WYSIWYG way. This would need to be validated by user research that is more exploratory in nature.
Describe alternatives you've considered
Another option that is worth scoping is how existing workflows might be 'templates' that can be added to but not 'removed from'. An iterative way to explore this big feature is key.
Additional context

^^ Current dropdown of 'workflows' in test.hypha.app
Priority
Depending on the adopter:

Medium (slowing you down)

or:

Low priority (annoying, would be nice to not see)

This becomes critical when a existing funds/labs do not reflect future planning from a funder - this epic/META could rapidly move from low to high depending on their strategy and whether existing workflows fit those fund/lab strategies.
Affected roles

Staff (they would be using this fund/lab editor)
Developers (they would no longer need to hard code funds/labs)

Ideal deadline
Recommend review in Q2 rebasing for priority level post summit 2022

The workflows are hardcoded, you can pick on of the four that exist but not change them or add new ones.
This is mayor new development but it would make the system a lot more flexible for different organisations.

Users could currently select from at least four different workflows as noted in the original issue,  options include a single stage workflow or two stage workflow. Users could also change the names of each workflow status via translatable files.
While I could see the value making workflow selection more configurable, isn't Hypha already offering adopters a great deal of options and flexibility in terms of workflows?


Dictionary.gettext should not break on empty string

Description
Currently Dictionary.gettext will fail with NoMethodError if called with empty string.
A "gettext" method should handle empty strings without failing.
Examples
Bug:
pry(main)> Dictionary.gettext("")
=> NoMethodError: undefined method `start_with?' for #<Hash:0x0055e4a865d2a8>

Fix:
pry(main)> Dictionary.gettext("")
=> ""

@miq-bot add-labels bug
@zeari @cben @moolitayer @simon3z @mzazrivec please review

cc @ohadlevy ^^

maybe:
raise ArgumentError, "can't handle empty string"
is better then:
return ""
?

@miq-bot add-label bug
We saw this happen in /ems_kubernetes/protect screen after session timing out and redircting back from login.
The specific symptom was i18n_lookup("", "") performing I18n.t("dictionary..", ...) which weirdly returns the whole dictionary hash instead of single string, resulting in method errors down the line and an error page.
I like this fix returning "". :+1:
P.S. anyone familiar with how i18n works, is it right that i18n_lookup hardcodes  :locale => "en" ?!

@miq-bot add-label internationalization

I don't know our implementation of this , but return "" if text.blank? seems like a correct translation
:+1:

anyone familiar with how i18n works, is it right that i18n_lookup hardcodes :locale => "en" ?!

Yes, it is right.

@mzazrivec @Fryguy ping, back from holidays :-)
p.s.
This little fix can actually save time when looking for problems ...
For example: I am building an Ansible moduls for Policies, and when browsing to the policies edit UI in ManageIQ (to check the results of the Ansible run) I got the cryptic error " NoMethodError: undefined method `start_with?' ... " :thought_balloon: , I would be happier to get an error telling me I have no provider defined, witch was the real error :bug:

@Fryguy @mzazrivec ping

@Fryguy @mzazrivec ping


fix(deps): update dependencies and lowest provider version

Description
Update the lowest allowable provider version
Update the dependencies to their latest version
set location = null to avoid location bug in CBR service profile submodule
Release required?

[ ] No release
[ ] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

/run pipeline

/run pipeline

:tada: This PR is included in version 1.1.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


OB/OBC CRDs are not owned by OCS downstream (although it bundles them)

CC @nimrod-becker @jarrpa @davidvossel
Here is where it uses required: for these CRDs in the OCS CSV instead of using owned:
https://github.com/openshift/ocs-operator/blob/34106927774f51584ca6fa3b6ef4e9dc559db39e/deploy/olm-catalog/ocs-operator/0.0.2/ocs-operator.v0.0.2.clusterserviceversion.yaml#L263-L302
This is because OCS CSV is copying the same required that NooBaa CSV is using upstream.
However for downstream these CRDs should be owned by OCS operator.
The reason it works is that the operator bundle includes those CRDs, which seems like a fishy situation, as OLM is not aware that OCS operator is the one owning those CRDs.

I think we should have the OBC CRD separated, it would make sense from the community point of view.
Since OCS is already using "require", my understanding is that we will just need to remove the CRDs from the OLM package and publish the lib-bucket-provisioner CSV to the community operators (downstream).
@guymguym  can handle the changes.
@jarrpa @obnoxxx please review so we can progress.

Agreed, I think that's why we ended up using required instead of owned in the first place.


Request for a new release of MoveIt into Rolling

Description
Following up on https://github.com/moveit/moveit2/pull/2881 , it would be great to get a new release of MoveIt into Rolling.  Once that is done, we'll be able to remove the octomap package from Rolling itself, and thus be able to close https://github.com/ros/rosdistro/issues/41622
Your environment

ROS Distro: Rolling
OS Version: Ubuntu 24.04
Binary
New one, please :).
Which RMW (Fast DDS or Cyclone DDS)? N/A

Friendly ping here; this is the last remaining package before we can remove octomap from Rolling.

Just as an FYI, I've now opened up https://github.com/ros/rosdistro/pull/42561 .  Once that is approved and merged, Moveit will no longer compile in Rolling until a new release is made.

@sjahr, @henningkayser: Is there any reason (except time constraints) that you didn't cut a new release yet?
If time is the only constraint, I can offer to create a new release.

@rhaschke @clalancette Sorry I completely lost track of this and was out of office the past two weeks. Yes, at my end, time constraints are the only issue blocking this :see_no_evil: @rhaschke If you have the capacities to cut a new release, that would be awesome

Will do.

Release prepared: https://github.com/moveit/moveit2/pull/3002
But I'm lacking direct push permissions required by the release script.

Done: #3002


Backspace sends '(' instead of '_'

Backspace sends '(' instead of '_'. At least on my iCE40HX8K-EVB with iCE40-IO, see video https://youtu.be/2yIBnhapzEU . You can backspace by typing '_' using Shift+- on US keyboard.

I did this just now, but it still types ( instead of _ when I press BS. Also git pull showed Already up-to-date.

Sorry, my mistake. Pushed to my local GIT instead of the public one. Try again now :)

Yes, it works now.


Fix FacetGrid.map and allow specifying size/aspect with .plot shortcut

cc @clarkfitzg

Looks good. You might consider testing this in Common2dMixin for better coverage.


Case-insensitive sorintg

https://github.com/aldeed/meteor-tabular/issues/66
I want to know how can i implement this??

+1

+1

I believe the only way would be to add support for aggregate tables, and then you could use something like this SO solution: http://stackoverflow.com/a/22931481/1669674


Translations and Localized Strings

Hello,
Any recommendations for applying localisation strings to the strings used in PermissionScope?
I would prefer to use the cocoapod and override the methods but I was not able to get that working. We could of course copy the framework and edit ourselves but that doesn't seem the best approach.
For example the showDeniedAlert method uses hard coded strings and we want those to be localized.
Thanks

Hey, I see you use an extension called localized which looks for localized string with the relevant text as the key. We will go with that. Thanks


Add analytics controller tests

XN-1560
Adds unit tests for the data models and the AnalyticsController.
Those for the latter also double down as integration tests.
It also fixes a few bugs and upgrades to a newer version of Isar (still a git ref).
I also had to based it on the branch for the read and get methods, not master. I'll change it back to master once the other PR is merged.
Next up: Documentation.
(Heads up: in my case Github collapsed the file where the AnalyticsController file cause the diff is big, but yeh, that's where most of the changes are)

i'll close this and re-open it in a different branch off of master (since #747 has been merged) cause i don't want to mess up the history


In-line hashtags should render the same as frontmatter hashtags when previewing.

Please Select if your Request is Either something new or an Enhancement

[x ] Enhancement of an existing Feature.
[ ] Request of a new feature.

Please select the area your request applys to. (Multiple selections are Possible)

[ ] Onboard. Initial vault setup and import/export pods
[ ] Create. Note creation, lookup, snippets and templates
[ ] Retrieve. Backlinks, references, graph view
[ ] Structure. Refactoring, multi-vault and schemas
[ ] Publish. Sharing your repo with the world
[x ] Misc (Choose this if your not sure)

Is your feature request related to a problem? Please describe
I've tried using #tags in both the frontmatter area and inline hastags and I get different rendering from the V2 preview.  Tags from the frontmatter don't show up with the hashtag # prefix when previewed whereas inline hashtags do.
Describe the solution you'd like
Hashtags from the frontmatter should include the # hashtag prefix.
See example image here:

Describe alternatives you've considered
Since inline hashtags get color coded, you could auto-magically remove the hashtage # so that it looks the same as frontmatter hashtags.
Additional context
N/A.

Hi! In Dendron 0.61, we now have an option to add # symbols for the frontmatter tags. Please see this configuration: https://wiki.dendron.so/notes/f2ed8639-a604-4a9d-b76c-41e205fb8713.html#usehashesforfmtags
By adding useHashesForFMTags: true under site in your dendron.yml, you can add the symbol in Preview V2 and in published sites.


All subpages : Increase vertical space between sections

The current iteration leaves to little space above headlines. We need to get much closer to the designs that Zia provided. Please use them as a reference and increase the space accordingly.

design at web is different than illustration or design programs at we it related to grid and how it ll be responsive with all sizes
in bootstrap 4 it related with fixed padding and margin like 1-5 changing this value ll not response good with all size of screen

please more details in this issue or example ,we made equal space between each section

Please see the Illustrator files provided by Zia for the exact specification regarding the spacings. We need to  approximate those as much as possible.

now spaces increased to bootstrap max

@maximilianrang @Sacha96 what is status here did we still need space between sections ?

Works for me in the current state. What do you think @Sacha96 ?

all good now.


pixi: 0.13.0 -> 0.15.2

Description of changes
https://github.com/prefix-dev/pixi/releases/tag/v0.14.0
https://github.com/prefix-dev/pixi/releases/tag/v0.15.0
Closes #289845
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This should be pixi I think.
https://github.com/NixOS/nixpkgs/blob/68371c469f0666bfba0758e0a25e87814fbb2808/pkgs/by-name/pi/pixi/package.nix#L59


Use npm 'component-emitter' instead of 'events'

Because socket.io uses 'component-emitter'. 'events' has not been updated lately.

What is the difference? Should we consider this for other things as well?

Gozala/events has 245 stars, hasn't been updated to the latest Node specs and the author doesn't work on it much.
component/emitter has 358 stars and is used by socket.io. There is no comment on Node compatibility.
I don't know what the differences are but I like the idea of using only one repo rather than several. I also think socket.io would have put more effort into choosing a repo.
I do think its better if Feathers uses one emitter repo and, since Feathers is dependant on socket.io in practice, we could use the one they do.

I agree. I looked briefly into it and it looks like it provider the .off method that the Node event emitter doesn't. I will try today to use it instead of the events package for v3 core and see if it breaks anything (I don't think it would though).

I've switched to it in feathers-offline-*.
The only issue was I could not use it in class foo extends emitter. There a few es6 emitters but they have less than 20 stars.


Use query definition name as firstOperationName if any

Changes proposed in this pull request:

This change allows to use queries/mutations operation name as firstOperationName value.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for your first contribution! It will be in next release 🚀


Use readme.md file in nuget readme

It's better to use the readme file content in nuget description.
To achieve this we should create a HttpClientToCurlGenerator.nuspec file in the project like this:
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
    <metadata>
        <!-- Required elements-->
        <id>HttpClientToCurlGenerator</id>
        <version>2.0.3</version>
        <description>An extension for generating Curl script of HttpClient</description>
        <authors>Amin</authors>
        <summary>
          This extension will help you to see whatever is set in HttpClient in the form of a curl script.
        </summary>
        <releaseNotes>
          ...
        </releaseNotes>

        <projectUrl>https://github.com/amingolmahalle/HttpClientToCurlGenerator</projectUrl>
        <repository type="git" url="https://github.com/amingolmahalle/HttpClientToCurlGenerator.git" branch="main" />

        <license type="expression">MIT</license>
        <requireLicenseAcceptance>false</requireLicenseAcceptance>
        <tags>csharp HTTP httpclient curl</tags>
        <icon>icon.png</icon>
        <readme>README.md</readme>
    </metadata>
    <!-- Optional 'files' node -->

    <files>
        <file src="README.md" />
    </files>
</package>

Nice idea. I will do it.

@thanks dear @naeemaei

closed


Post-tutorial updates

pyviz.org is in pretty good shape after preparing and polishing a series of releases yesterday that updated all the materials and the website. There are a few to-do items now that we didn't have time for or are based on having given the tutorial live yesterday:
Quick:

[x] Fix PyViz logo in upper left of website; is being overwritten by "PyViz".  Maybe replace with wordmark instead of text?
[x] Fix formatting of images in https://pyviz.github.io/pyviz/tutorial/01_Workflow_Introduction.html
[x] Change theme colors to work with logo
[x] Make the paragraph symbol on intrapage links much lighter gray to make it nearly invisible
[x] Mention conda update conda ; conda update conda in the installation instructions (if an older conda was on their systems, installation was failing to find packages from the main channel, possibly the noarch ones)
[x] Add big copy of logo to front page
[x] Add project logos and some viz to the pyviz.org front page
[x] Add link to osm-1billion example (and mention how to download the data)
[x] Replace .opts with .options in examples
[x] Update holoviews, geoviews, param, parambokeh, and the corresponding HoloViews version number in 00_Setup.ipynb.
[ ] Merge in content from the PyData Cookbook
[x] Fix nbsite#59 (broken links for downloading notebooks, e.g. the one on http://pyviz.org/tutorial/11_Streaming_Data.html)
[x] Add links from each tutorial notebook on to the next
[x] Fix non-mpl options in 02 (again!)
[x] Fix data download path

Medium:

[x] Do bokeh.sampledata.download() in the doc build in Travis to avoid having 00 Welcome fill up with downloading messages
[x] Move paragraphs of text 00 Welcome into index.html, leaving just the mechanical bits as part of the first tutorial.
[ ] Simplify the 01 Workflow notebook; lots of good material, but needs to be quicker
[ ] Pin any dependencies that could conceivably vary? (#25)
[ ] Make pyviz tests cover installing as a package and other new functionality (#81)
[ ] Add reference data for notebooks
[ ] Add a colorcet logo
[ ] Add a section with links to other pyviz projects (nbsite, nbval, nbsmoke, autover, etc.)
[ ] Add twitter feed listening to multiple projects
[ ] Change 08_Custom_Interactivity.ipynb to use new RangeSelect tool
[ ] Reduce overlap between tutorial sections

Long-term:

[ ] Should fold in tutorial A1's material somewhere; it's odd to have it in an appendix.  It made sense when the tutorial was smaller, but now that there is always more material than can be covered, it's confusing to have it separate, so that the other material can't tell whether these concepts have been introduced.
[ ] Need automated scripts for making at least the first pass at the site logo and favicon bitmaps from a transparent SVG source; currently edited those by hand for pyviz.org and other project sites have non-transparent logos. (See https://github.com/pyviz/nbsite/issues/40)
[ ] Make sure everything is pip-installable as well as conda-installable (#81)
[ ] Move topics from datashader and holoviews sites into pyviz, as most combine multiple tools.

Here are some TODO items I've noted since discussing the notebooks with @jbednar:
Overview

[ ] Rename 'Using HoloViews "containers" for quick, easy data exploration.' to 'Working with tabular data'.

01_Workflow_Introduction

[ ] Add a note at the top of  to explain why this notebook is using slides unlike the others.
[ ] Try to make the difference between holoviews and hvplot more explicit in the text.
[ ] Re-enable the colorbar on the measles example.
[ ] Rename 'Dealing with large data and geo data ' heading to 'Branching out: large data, geo data, custom controls'

02_Annotating_Data

[ ] (Optional) Try to explicitly state in the text how holoviews tries to avoids mutating the data directly.
[ ] In bullet point describing element kwargs append qualification '(Very rarely needed)'.
[ ] Change 'Here x is a convenient...' to 'Here x is a short and convenient ...'
[ ] Remove space in  hv.Image(arr, ['x axis','y axis'], 'h').hist() dimension names.

03_Customizing_Visual_Appearance

[ ] Fix 'Onwards' section to link to next tutorial nicely (also check this across notebooks)

06_Network_Graphs.ipynb

[ ] Before datashading, point forwards to large data tutorial to help introduce datashader.

07_Geographic_Data

[ ] At start of this tutorial, add a sentence to explain that while HoloViews and associated tools are general and handle all sorts of data, one common special case that needs attention is when people often want to situate their data on maps.

08_Custom_Interactivity

[ ] Try to mention new Links feature that will allow custom interactivity without needing a live server.
[ ] Add some sentences to try and introduce DynamicMap properly (currently no introduction and very little introduction to HoloMap as it is).

Other tasks assigned to me

[ ] Go through the exercises notebooks and make sure sections we don't plan to skim work correctly.

Website

[ ] In 02_Annotating_Data remove $x$ / $y$ and use backticks (MathJax rendering issue on website)
[ ] Fix broken links to exercises on website (check they work in the notebooks!)

I expect to be able to address most of these quite quickly, as many of these are very minor tweaks.

Fix data download path

Can you expand on that? I fixed one issue (https://github.com/pyviz/pyviz/pull/76) after 0.9.9 was tagged.

One thing that should be fixed and needs a website build are the exercise links here e.g pointing to http://pyviz.org/exercises/ercise-1-making-data-visualizable.html

One thing that should be fixed ASAP and needs a website build are the exercise links

Will happen as part of #63 (the exercises aren't currently part of the website build at all).

So that problem is both fixed and not fixed

(just wanted to check that we were referring to the same problem)

Add links from each tutorial notebook on to the next

Apart from that, I think it would make it simpler for people if suggested diversions/more detail were separate from what to do next in the tutorial. However, I've neither given nor received this particular tutorial, so it's only a suggestion to consider for now.

[ ] - 09_Operations_and_Pipelines.ipynb starts using datashader without introducing it/why to use it?

Closing in favor of newer issues.


Hotfix: update Line descriptions

Line 2: change Downtown LA to USC
Line 51: change Wilshire Center to Westlake/MacArthur Park Station
Line 53: Change Avalon Station to Willowbrook/Rosa Parks Station
Line 230: Change Mission College to Sylmar Station
Line 256 change Commerce to CSULA
Line 260 Change Altadena to Pasadena

ref: https://github.com/LACMTA/metro-dot-net/issues/142


Add setInner method takes InnerSpec consumer

Summary
I want to use like this (in kotlin).
val actual = SUT.giveMeBuilder<Map<String, String>>()
    .setInner {
        it.keys("key1", "key2")
            .minSize(3)
    }
    .sample()

How Has This Been Tested?
InnerSpecTest#innerConsumer

Extension으로 변경했습니다.


[#642]feat(server): better default options for shuffle server

What changes were proposed in this pull request?

rss.server.buffer.capacity uses JVM heap size * ratio(0.8) by default
rss.server.read.buffer.capacity uses JVM heap size * ratio(0.4) by default
rss.server.disk.capacity uses disk space * ratio(0.9) by default

Why are the changes needed?
Fix: #642
Does this PR introduce any user-facing change?
Yes. Three new configurations are introduced, users can specify ratio values
for buffer, read buffer and disk capacity
How was this patch tested?
New UTs.

Codecov Report

Merging #662 (e19cb82) into master (c70d154) will increase coverage by 1.21%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master     #662      +/-   ##
============================================
+ Coverage     60.67%   61.88%   +1.21%
+ Complexity     1802     1712      -90
============================================
  Files           216      193      -23
  Lines         12458     9955    -2503
  Branches       1052     1012      -40
============================================
- Hits           7559     6161    -1398
+ Misses         4494     3461    -1033
+ Partials        405      333      -72

Impacted Files
Coverage Δ

...a/org/apache/uniffle/server/ShuffleServerConf.java
99.37% <100.00%> (+0.03%)
:arrow_up:

...he/uniffle/server/buffer/ShuffleBufferManager.java
83.50% <100.00%> (+0.75%)
:arrow_up:

...he/uniffle/server/storage/LocalStorageManager.java
86.59% <100.00%> (+0.15%)
:arrow_up:

...rg/apache/uniffle/storage/common/LocalStorage.java
55.41% <100.00%> (+4.09%)
:arrow_up:

.../java/org/apache/uniffle/server/ShuffleServer.java
62.06% <0.00%> (-1.98%)
:arrow_down:

.../apache/uniffle/coordinator/ClientConfManager.java
91.54% <0.00%> (-1.41%)
:arrow_down:

...org/apache/spark/shuffle/writer/AddBlockEvent.java

deploy/kubernetes/operator/pkg/utils/config.go

deploy/kubernetes/operator/pkg/utils/util.go

...org/apache/spark/shuffle/RssSparkShuffleUtils.java

... and 21 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Our memory is equal to rss.server.buffer.capacity + rss.server.read.buffer.capacity + extra memory used for program.

Our memory is equal to rss.server.buffer.capacity + rss.server.read.buffer.capacity + extra memory used for program.

From the point of view of production environment, the reserved system memory should be the half of xmx. If not, the full gc will occur when the server meets the high pressure.

Our memory is equal to rss.server.buffer.capacity + rss.server.read.buffer.capacity + extra memory used for program.

If I understand the shuffle server code correctly, read buffer and buffer capacity have some overlapping region, So it's possible rss.server.buffer.capacity + rss.server.read.buffer.capacity > 1.0.
The current default value may a bit larger consider the meta memory required for shuffle server.

They shouldn't have overlap.

IIUC, the buffer caches data in memory, when reading with memory shuffle, the data size is calculated both for buffer and read buffer, these two regions should be considered overlapped region?

No, they will copy memory. They shouldn't overlap.

@zuston do you have any suggestion ratio for buffer and read buffer then.

For us, read buffer size is the half of buffer capacity size. But from the dashboard, the read buffer size could be set to smaller.


Documentation updates and code improvements

added tooltips support for the Altair plot
fixed automatic sorting issue with the Altair
regenerate the documentation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

10 of 10 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 94.798%

Totals

Change from base Build<PHONE_NUMBER>7:
0.02%

Covered Lines:
1458

Relevant Lines:
1538

💛  - Coveralls


What is the specific date for EOL of azcopy

Hello,
Our account rep has told us that azcopy is going away and will be unsupported soon. Do you know the specific date we need to be migrated off azcopy and when support is being dropped?

This is not accurate, there are no plans to deprecate AzCopy.


Build error in read the docs

We are working on it.

Issue has been resolved. Check this pull request which was merged and solved it #4


fix typo in sidebar css filename

Coverage remained the same at 24.522% when pulling 98cdf070cd847ab3d6d2ce21a46ad7aca8189aef on fix-filename into 2540aca8e59b5da65f4c82867a8a799ad32cd9fd on master.

👍 I thought I already responded to this. Ship it.


pgml?

First, thanks for doing this. Great project.
Any chance you can add pgml: https://postgresml.org/docs/open-source/overview?

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."


fix several page_type errors

Also change the test-results Makefile target to behave more like a normal target

Alex can you sanity check these page_types.  They looked obviously wrong, which makes me worry that I'm missing something.


Consider deferring attached to emulate 1.0 callback timing

From @sorvell on August 29, 2016 21:27
Related issues: #71, #96.
In 1.0, elements were ensured to be readied before being attached. This allowed use of local dom and (at lease naive) measurement in attached.
This was achieved specifically by deferring attached until after ready if the system tried to call it too soon.
We could retain this guarantee without explicitly deferring attached by readying (flushing) an element before attaching its local dom. This would change the guarantees around ready. Specifically, it would be unsupported to look at shadowRoot in ready.
Perhaps we should simply add back the defer attached behavior from 1.0 to ensure as little breakage for users of the 'legacy' api as possible.
Copied from original issue: PolymerLabs/alacarte#102

This will likely not be addressed in Polymer.Element where we are not inclined to introduce new lifecycle (attached). Based on feedback we'll see if we need to add guidance around how to account for this.

Resolved via https://github.com/Polymer/polymer/pull/3947


Setup failed for grohe_sense: Integration not found.

Kudos for building this! I wanted to set this up and have tried following the readme, but I get this error when restarting home assistant:
Logger: homeassistant.setup
Source: setup.py:168
First occurred: 10:28:54 (1 occurrences)
Last logged: 10:28:54
Setup failed for grohe_sense: Integration not found.
I have added the token and copied the files to a grohe_sense folder under config dir. Any thoughts?

Same for me. Any ideas?

I had the same. I solved it by comment out the grohe_sense stuff in configuration.yaml, restart HA and then enabling the grohe_sense stuff again, followed by another restart of HA.

I have the same problem. Any ideas?


Automatically determine target stamp after port scanning and download

If the user selects the Download operation, after the progress display updates with the results of the port enumeration and port scanning processes (described in issue #138), the app should now determine the target Stamp to download to.
Decision Flow:

No modules of any kind are found to be connected?
* Leave progress display on-screen (for user review)
* Display error: "No BASIC Stamps found."
* NOTE: the intent is for the user to be able to read the error and view the list of ports scanned to help them determined what was done and what the results were
Only one instance of the source code's specified module type is found to be connected to the system?
* Automatically select that module type's port and perform the entire programming protocol on it.  This means reset, identify, program, verify.  Yes, the module will actually have been identified twice and programmed once through the processes of issues #138 and #139.
More than one instance of the source code's specified module type is found to be connected to the system?
* Prompt the user to select the module they wish to download to; "Please select which module to download to."
* If the user selects the Cancel button; close the display and cancel operation
* If the user selects a module, immediately perform the entire programming protocol on it.  This means reset, identify, program, verify.  Yes, the module will actually have been identified twice and programmed once through the processes of issues #138 and #139.

NOTE: It's not desired, but if there must be a Download button that they have to click after selecting the desired module, it's acceptable.  We'd like to avoid forcing this extra click, however.

No modules of the type specified in the source code are found to be connected, however, at least one module of another type were found to be connected?
* Prompt user with "BASIC Stamp X not found.  Select alternate module instead?" ;where 'X' is the type of BASIC Stamp that the code was written for.
* If the user selects the Cancel button; close the display and cancel operation
* If the user selects a module,

automatically modify the source code to change the $STAMP directive to reflect the new desired module type (you can use the tokenizer's returned information to find the specific source and location to change)
immediately re-tokenize the source code (for that new module)

If tokenization error, display error and cancel the rest of this download process

if no tokenization error, perform the entire programming protocol on the module that the user selected.  This means reset, identify, program, verify.  Yes, the module will actually have been identified three times and programmed once through the processes of issues #138 and #139.

I am labeling this as a UX issue because of the prompting in point 3.

@urbantumbleweed I think you are incorrect in labeling this as such

In response to @phated regarding this as a development issue, the UX/UI label has been removed.

@PropGit I think we should split this into multiple, related issues.  Do you agree?

You mean one issue for each of the four possibilities in the decision flow?
If it helps get the the work done, yes.  These are all part of one function really; they shouldn't be developed separately without thought of the other.  1 & 2 really need to be done at once.  3 & 4 can be developed later, but when 1 & 2 are made they need to set aside space for 3 & 4.

@PropGit I think 1, 2 and 3 are all able to be done at once, but 4 will be done at another time.

@phated Okay, yes, 4 is the most involved of any of them.

The issue has been split out into multiple issues.  For more information take a look at the individual issues #163, #164, #165, #166


Update :: Code Fixes and Implamentations

Updated the python module to use the version that uses requests for fetching proxies rather then ChromeDriver.

Code fixes and (ProxyChecker) partial implementation.


[Quant-noise] Why p of ActivationQuantizer is set to 0

❓ Questions and Help

What is your question?
I find that p of ActivationQuantizer is always set to 0, which means no noise will be added to activation, why are you doing this? Is it necessary?
Code
    a_q = ActivationQuantizer(quantized_module, p=0, bits=bits, method="histogram")

What's your environment?

fairseq Version (master):
PyTorch Version (1.5)

@huihuifan

Thanks for the interest. It is not specifically necessary, but we thought it was simpler as we did not robustly test activation quantization for all models.


Set up genome nexus documentDB

The current database only has one primary instance, there is no backup when it crushes.
Goal: Set up genome nexus documentDB to have replica settings.
Tasks:

[x] Create documentDB cluster
[x] Connect to documentDB
[x] Dump all data from current genome nexus database
[ ] Test (index, cache, etc...)
[x] Connect documentDB with app

Issue: After connecting to documentdb, the app compiling is always stuck at signal index building.
I did some tests and noticed the most time-consuming process is findByVariantIn
I switch genome nexus app to documentDB to test the performance of findByVariantIn with 50000 variants (signal has 110000 variants but it reaches the timeout limit so reduce to 50000 for testing purpose). Here is the results comparing with original MongoDB:

database
number of variants
average time

MongoDB
50000
20s

DocumentDB
50000
4min30sec

decided to look into k8s again, since the performance was not great for documentdb


Add Support for SSL Network Connections

We currently do not support SSL connections. This means that you always have to disable SSL from the commandline. We need to integrate support for openssl in the system. We should follow the same configuration setup as Postgres.
This was attempted before in #667 but the students working on it did not get very far. They hardcoded a bunch of paths for the Vagrant VM which is obviously incorrect:
https://github.com/cmu-db/peloton/blob/master/src/include/settings/settings.h#L46
We need coordinate with @ChTimTsubasa on this. I believe the issue is that the first packet that arrives from the client is the SSL handshake attempt, which I think causes problems in the current ProtocolHandler.

Concretely, this is because the first handshake to open (client send Version number and server response "YES") is done before ProtocolHandler is instantiated currently. Since sending a response back requires the participation of ProtocolHandler, this would cause SEG_FAULT.
In addition, such handshake looks specific to Postgres (MySQL uses a passive way to handle a new connection).

This issue has been adressed


Merge pull request #1 from capitalone/master

docs - update link for developer install in readme (#1129)

pull


Bokeh: full-screen resize in Colab

SO Q: How to change size of bokeh figure

It involve:
p = bokeh.plotting.figure(
                              plot_width=1350, plot_height=400,
                              min_border_right=14…)
p.sizing_mode = 'stretch_both'

setting plot_width, plot_height and sizing_mode (multiple diff values) seem to interact. Didn't work it out yet.


How to charge closing fees without an order match

It is an edge-case, but a trader could simply force-close the channel to avoid paying order-matching fees.
Even if there is no force-close, we do currently enforce frequent collaborative closures that I think circumvent the orderbook. This motivated me to leave the following TODO in https://github.com/get10101/10101/pull/863:
https://github.com/get10101/10101/blob/dadbc4bade0dc3777dafc1e8b9efff5ba7266fb0/mobile/native/src/ln_dlc/node.rs#L248

It is an edge-case, but a trader could simply force-close the channel to avoid paying order-matching fees.

Can you elaborate on this scenario? How would the user avoid paying the order-matching fees? Also, it looks like this part has nothing to do with the second part.

Can you elaborate on this scenario? How would the user avoid paying the order-matching fees?

An app user could:

Open a position, paying the corresponding order-matching fee invoice.
Force-close the LN-DLC channel (currently not supported, but we have to support it).
Not have to pay an order-matching fee for closing the position, since force-closing doesn't involve creating an order and getting a match.

With a small position this might not be profitable because the transaction fees might be more expensive than the order-matching fee, but this route might become more attractive with large positions.

Also, it looks like this part has nothing to do with the second part.

The point of the second part is to highlight that there is another way in which the app user can end up not paying the order-matching fee: by letting it be collaboratively closed on expiry. Such a scenario is related to the first one because in both cases there is no order-matching happening, which is strange in the context of an orderbook. I left it out of https://github.com/get10101/10101/pull/863, because:

I wasn't sure if we actually wanted to charge the order-matching fee in that case.
If we do want it, the implementation has to be different because we do not send a trade request from the app to the coordinator (which in the regular flow returns the order-matching fee invoice).

Thanks for the explanation, now the second part also makes sense to me why it is added to this ticket. 👍
Well in that case, how about only charging a fee for opening a position, but none when closing a position? We could as well charge twice for opening already covering the fees for closing the position upfront - as there is currently no way to not close the position after the expiry.
We might want to revise that when supporting perpetuals.

Let's try to remove the term: close position from our vocabulary and replace it with:

trade: is the act of adding or removing liquidity from the orderbook: i.e. a market order removes liquidity and a limit order (without immediate match) adds liquidity to the orderbook
close DLC channel: this is what is described in this ticket. A channel can either be force-closed or collaboratively closed. The latter one should always involve point 1.

When speaking about order matching fees/orderbook fees we talk about the first point.
If a trader has an open position and force closes his DLC channel, then the coordinator will need to find a replacement or otherwise he accepts unecessary risk. If he finds a replacement, i.e. matches a new trader, who will need to pay order matching fee.
That being said, in this case we would lose out one part of the order matching fee because the coordinator will remove liquidity from the orderbook.
I think there is nothing to do here for now.

Quoting myself :)

I think there is nothing to do here for now.

Closed as not planned for now


[Feature Request] Allow reserved users to remove themselves

As it currently stands, if you reserve a person onto an event they are unable to react to the - to remove themselves. We use this for clan events and I would like to auto-reserve raid leaders but allow them to remove themselves if needed.
Since it's tied to the users Discord account(User#1234), is there a way that it can realize that user is submitting a removal action?

If you're entering their discord names for them, it should allow them to use the reaction to remove themselves if their tag was spelled correctly.

So I went back and tested it this morning and I'm getting mixed results. I put a total of 6 users on the list and only two of us are able to remove ourselves.
I am using Discord ID's, and all roles have permissions to post/edit events.
This is how our roles were ordered beforehand:
Admin (Admin flag) [can remove]
Moderator (No admin flag) [can remove]
Bot (Admin flag)
Raid Leader (No admin flag) [cannot remove]
After your response I moved Raid Leader above the bot and tested again. They still cannot remove themselves. Is there any settings I should be double checking?
Thanks for your prompt responses. Love the bot! I've got another feature request I'll post this weekend for consideration.

Sorry for the late reply due to the holiday. If you want to PM me an invite to your server on Discord, I could take a closer look at the issue. My discord name is Sillvva#2532.


NullPointerException in Kafkaconnect

Environment

OS version: Centos 7.5
JDK version: AdoptOpenJDK 11.0.4
ClickHouse Server version: <IP_ADDRESS>
ClickHouse Native JDBC version: 2.5.4-shaded
(Optional) Spark version: N/A
(Optional) Other components' version: kafkaconnect version 6.1.1-2.12

Error logs
2021-05-27 14:54:32.813 +0800 [task-thread-sink-clickhouse-data-0] [ERROR] org.apache.kafka.connect.runtime.WorkerSinkTask - WorkerSinkTask{id=sink-clickhouse-data-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: null
java.lang.NullPointerException
2021-05-27 14:54:32.813 +0800 [task-thread-sink-clickhouse-data-0] [ERROR] org.apache.kafka.connect.runtime.WorkerSinkTask - WorkerSinkTask{id=sink-clickhouse-data-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: null
java.lang.NullPointerException
2021-05-27 14:54:32.813 +0800 [task-thread-sink-clickhouse-data-0] [ERROR] org.apache.kafka.connect.runtime.WorkerTask - WorkerSinkTask{id=sink-clickhouse-data-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:614)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:329)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:232)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:189)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:238)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException

Steps to reproduce
The message contains types (as per CH types): String, UInt64, Float64, DateTime. Some columns can be NULL, as is defined in CH schema, but looks like there is some NPE happens inside the driver when kafka message field is actually null.
Other descriptions
The same ingest works with official HTTP protocol driver, but would prefer to use this one as it works better in throughput and resource usage.

The stacktrace seems not related to Jdbc driver

Yes, it doesn't show anything about JDBC driver. However same exact setup with Postgres driver works fine, also same exact setup with official CH driver also works. Only difference between those is which JDBC driver is used.

I have not tried just released 2.5.5 yet, since I cannot restart kafkaconnect just yet. I will try the same setup with 2.5.5 a little later and will provide feedback here.

from my understand, the pipeline is pg => (dbz as kafka connect source) => kafka => (ck native jdbc as kafka connect sink) => clickhouse. if not, could you briefly illustrate the pipeline? which components are you using clickhouse native jdbc in?

we are changing the pipeline to use CH instead of PG. We have many topics to ingest into database (all different tables). Some work fine with this driver other have some problems.
Now: application => kafka => confluent-kafkaconnect + jdbc sink => timescaledb -- this one works fine.
New: application => kafka => confluent-kafkaconnect + jdbc sink => clickhouse -- this one works with official HTTP driver, but native driver has this error (only some sinks)

Upgraded to 2.5.5. Stacktrace is different now:
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:614)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:329)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:232)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:189)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:238)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
	at com.github.housepower.jdbc.data.type.DataTypeInt64.serializeBinary(DataTypeInt64.java:77)
	at com.github.housepower.jdbc.data.Column.write(Column.java:31)
	at com.github.housepower.jdbc.data.Block.appendRow(Block.java:93)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.addParameters(ClickHousePreparedInsertStatement.java:162)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.addBatch(ClickHousePreparedInsertStatement.java:95)
	at io.confluent.connect.jdbc.sink.PreparedStatementBinder.bindRecord(PreparedStatementBinder.java:115)
	at io.confluent.connect.jdbc.sink.BufferedRecords.flush(BufferedRecords.java:184)
	at io.confluent.connect.jdbc.sink.JdbcDbWriter.write(JdbcDbWriter.java:80)
	at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:84)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:586)
	... 10 more

that make sense, i see what happens in jdbc driver now.

integration-test passed, and will publish v2.5.6 soon

v2.5.6 is deployed.

Thanks, I will test it once it is available in central.

Update to 2.5.6, but still see same stacktrace:
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:614)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:329)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:232)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:189)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:238)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
	at com.github.housepower.jdbc.data.type.DataTypeInt64.serializeBinary(DataTypeInt64.java:77)
	at com.github.housepower.jdbc.data.Column.write(Column.java:31)
	at com.github.housepower.jdbc.data.Block.appendRow(Block.java:93)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.addParameters(ClickHousePreparedInsertStatement.java:162)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.addBatch(ClickHousePreparedInsertStatement.java:95)
	at io.confluent.connect.jdbc.sink.PreparedStatementBinder.bindRecord(PreparedStatementBinder.java:115)
	at io.confluent.connect.jdbc.sink.BufferedRecords.flush(BufferedRecords.java:184)
	at io.confluent.connect.jdbc.sink.BufferedRecords.add(BufferedRecords.java:169)
	at io.confluent.connect.jdbc.sink.JdbcDbWriter.write(JdbcDbWriter.java:74)
	at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:84)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:586)
	... 10 more

I think the stacktrace indicate the column type is Int64, not Nullable(Int64), can you provided the ClikckHouse table schema and the error record?

@pan3793
I needed to redact the names, but one of the tables where it fails looks like this:
production :) show create table data

CREATE TABLE data.data
(
    `col1` String,
    `col2` String,
    `col3` String,
    `col4` UInt64,
    `col5` UInt64,
    `col6` UInt64,
    `col7` UInt64,
    `col8` Float64,
    `col9` UInt64,
    `col10` String,
    `col11` String,
    `col12` String,
    `col13` String,
    `col14` UInt64,
    `col15` DateTime,
    `col16` UInt64,
    `col17` DateTime,
    `col18` UInt64,
    `col19` DateTime,
    `col20` UInt64
)
ENGINE = MergeTree
PARTITION BY toStartOfHour(col19)
ORDER BY col20
SETTINGS index_granularity = 8192

Hmm, I figured out that NOT NULL is the default in CH (which is the opposite of many other databases where NULL is the default).
I flipped some values around and it works now, the data is being written into table. However this is very different from how official driver behaves, it still can handle above schema and not fail the task for some reason :thinking:

Yes, ClickHouse use String to represent VARCHAR NOT NULL in most ANSI compatible Database, and Nullable(String) to represent VARCHAR NULL.
BTW, it's by design that throw exception when insert null to a not null column, I will close this issue, and feel free to reopen or open new issues if you encounter other questions.

Yes, while it is clearer now and I can ingest the data into clickhouse, one thing could be improved on this driver side is to have a better error handling for this case.
The error here is very generic and does not point in the right direction. The point of having error messages at all is to make it clear what happened and possibly point to correct solution.
I suggest to:

catch this exception
and check if the exception happened because of null being received for non-nullable column
if it is then throw another exception which provides much clearer message (i.e. received NULL value for NOT NULL column or something similar, maybe even include column name in the message if possible).

This will allow users to handle those issues in much cleaner way (and not bothering you with these issues regularly as it seems from issues history for this type of issue).


New-AzureRmOperationalInsightsWorkspace Wrong -SKU

Hi Team,
The documentation and the cmdlet have a problem with -SKU for Log Analytics. I can not create LAs with PowerShell. In old LA I have the "pergb2018" option, but it does not work when creating a new one.
Module:AzureRM.OperationalInsights
PS C:\dev> New-AzureRmOperationalInsightsWorkspace -Location $Location -Name $WorkspaceName -Sku premium -ResourceGroupName $ResourceGroup
New-AzureRmOperationalInsightsWorkspace : HTTP Status Code: BadRequest
Error Message: Pricing tier doesn't match the subscription's billing model. Read http://aka.ms/PricingTierWarning for more details.
PS C:\dev> New-AzureRmOperationalInsightsWorkspace -Location $Location -Name $WorkspaceName -Sku pergb2018 -ResourceGroupName $ResourceGroup
New-AzureRmOperationalInsightsWorkspace : Cannot validate argument on parameter 'Sku'. The argument "pergb2018" does not belong to the set "free,standard,premium,pernode,standalone" specified by the ValidateSet attribute.
Supply an argument that is in the set and then try the command again.

@PawelLiebich - This looks like an engineering issue or reference documentation problem. Those issues are handled by the engineering team. Please file a ticket at https://github.com/azure/azure-powershell/issues


Suggestion: Add an API function to the get all rows

I have a situation where I need to get all the rows to be used for a total calculation on a field. The reason why I believe an API function is required is because if the data is filtered by the API then I have no way of getting those filtered rows (I'd have to separately do that manually), so something like:
handler.getAllRows()
Would return all rows provided by the handler with the filtering applied (and maybe sorting? Whatever manipulation is being applied by the API)

That helper looks really helpful (saves me using lodash), so thanks for pointing it out - and thanks for adding the new function so quickly!


az interactive failed

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az interactive
Errors:
cannot import name 'ENV_ADDITIONAL_USER_AGENT' from 'azure.cli.core.commands.client_factory' (/usr/local/Cellar/azure-cli/2.7.0/libexec/lib/python3.8/site-packages/azure/cli/core/commands/client_factory.py)
Traceback (most recent call last):
python3.8/site-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 654, in execute
    raise ex
...
interactive/azext_interactive/azclishell/app.py, ln 31, in <module>
    from azure.cli.core.commands.client_factory import ENV_ADDITIONAL_USER_AGENT
ImportError: cannot import name 'ENV_ADDITIONAL_USER_AGENT' from 'azure.cli.core.commands.client_factory' (/usr/local/Cellar/azure-cli/2.7.0/libexec/lib/python3.8/site-packages/azure/cli/core/commands/client_factory.py)

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az interactive

Expected Behavior
Environment Summary
macOS-10.15.5-x86_64-i386-64bit
Python 3.8.3
Installer: HOMEBREW

azure-cli 2.7.0

Extensions:
aks-preview 0.4.43
interactive 0.4.3

Additional Context

interactive

@Andriy-dev please update the interactive extension with az extension update -n interactive

Thank you its working now

Duplicate of #12837


Update README.md

Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

[ ] I have read the CONTRIBUTING doc and adhered to the example notebook best practices
[x] I have updated any necessary documentation, including READMEs
[ ] I have tested my notebook(s) and ensured it runs end-to-end
[ ] I have linted my notebook(s) and code using black-nb -l 100 {path}/{notebook-name}.ipynb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

done


Modifies odo create for devfile components to not require a cluster access.

What type of PR is this?
/kind bug
What does does this PR do / why we need it:
It also modifies odo push to fill the project name in env.yaml if no project is not found in it.
Which issue(s) this PR fixes:
Fixes #3811
PR acceptance criteria:

[ ] Unit test

[X] Integration test

[ ] Documentation

[ ] I have read the test guidelines

How to test changes / Special notes to the reviewer:

odo create should work without a cluster.
odo create --project <name> should fill the project name in env.yaml during creation.
odo push should fill the project name in the env.yaml if none was provided during creation.

I'm seeing Validation showing up twice when I try to create --s2i component:
$ odo create --s2i nodejs
Validation   <---------
Validation   <---------
 ✓  Validating component [6ms]

Please use `odo push` command to create the component with source deployed

This one probably deserves a separate issue:
$ odo logout

$ odo create nodejs node --now
Devfile Object Validation
 ✓  Checking devfile existence [13129ns]
 ✓  Creating a devfile component from registry: DefaultDevfileRegistry [21051ns]
Validation
 ✓  Validating if devfile name is correct [33281ns]
 ✗  failed to push changes: Failed to start component with name node. Error: Failed to create the component: unable to determine if component node exists: deployments.apps "node" is forbidden: User "system:anonymous" cannot get resource "deployments" in API group "apps" in the namespace "myproject"

Couple of things related to the error message being printed upon failure:

Use quotes for name of the component. This could be easily done by using %q in code in place of %s:
failed to push changes: Failed to start component with name "node". Error: Failed to create the component: unable to determine if component "node" exists: deployments.apps "node" is forbidden: User "system:anonymous" cannot get resource "deployments" in API group "apps" in the namespace "myproject"

More importantly, the error message should be more clear to indicate that odo is not connected to any cluster.

I'm seeing Validation showing up twice when I try to create --s2i component:

There is a separate issue for that https://github.com/openshift/odo/issues/4422

This one probably deserves a separate issue:

Yes, it should be a different issue.

One last observation related to functionality. I created the component with --project flag. I know that the project doesn't exist on the cluster I'm going to login to. Should odo create a project upon odo push in such scenario? Definitely a separate issue, IMO.
$ odo create python --project randomproject
Devfile Object Validation
 ✓  Checking devfile existence [168192ns]
 ✓  Creating a devfile component from registry: DefaultDevfileRegistry [48038ns]
Validation
 ✓  Validating if devfile name is correct [25674ns]

Please use `odo push` command to create the component with source deployed

$ odo login https://api.crc.testing:6443 -u developer -p developer
Connecting to the OpenShift cluster

Login successful.

You have one project on this server: "myproject"

Using project "myproject".

$ odo push
 ✗  Failed to start component with name python. Error: Failed to create the component: unable to determine if component python exists: deployments.apps "python" is forbidden: User "developer" cannot get resource "deployments" in API group "apps" in the namespace "randomproject"

Sorry, I was lying about one last functionality observation in my previous comment. Here goes one more.
I'm connected to the cluster. The scenario is when user is by default working on default project on OpenShift which is the case when one has spun up a fresh CRC cluster.
$ odo project get
The current project is: default

$ odo create nodejs
Devfile Object Validation
 ✓  Checking devfile existence [16336ns]
 ✓  Creating a devfile component from registry: DefaultDevfileRegistry [26803ns]
Validation
 ✓  Validating if devfile name is correct [20604ns]

Please use `odo push` command to create the component with source deployed

$ cat .odo/env/env.yaml
ComponentSettings:
  Name: nodejs
  AppName: app

$ odo push
 ✗  Failed to start component with name nodejs. Error: Failed to create the component: unable to determine if component nodejs exists: deployments.apps "nodejs" is forbidden: User "developer" cannot get resource "deployments" in API group "apps" in the namespace "default"

# now the env.yaml has an entry for "Project"
$ cat .odo/env/env.yaml
ComponentSettings:
  Name: nodejs
  Project: default
  AppName: app

I don't know the answer to what exactly should be done in this case. But there should definitely be better error handling when odo push is done in such a scenario. Making sure that Project: default doesn't get populated in env.yaml might be a nice to have as well since we want to prevent users from trying to push stuff to default namespace on OpenShift.

One last observation related to functionality. I created the component with --project flag. I know that the project doesn't exist on the cluster I'm going to login to. Should odo create a project upon odo push in such scenario? Definitely a separate issue, IMO.

We can decide on project creation later. But the error that is currently thrown in this situation is quite misleading.
odo push should check if project/namespace exist. If not it should print nice error. Something like:
The project that you are trying to push to doesn't exist. Please create project first (odo project create myproject)

I'm connected to the cluster. The scenario is when user is by default working on default project on OpenShift which is the case when one has spun up a fresh CRC cluster.

In this case  odo push should error out. We had a check in odo create that prints an error if the user tries to create component in default project on OpeShift. We need to move this check to odo push

@kadel @dharmit Fixed

Confused by the integration tests. IIUC, we've added two tests - one for OpenShift, one for k8s. And we're skipping both if the environment is k8s.

We are not skipping both if the environment is k8s. One of the condition is
if os.Getenv("KUBERNETES") == "true"

and the other one is the opposite
if os.Getenv("KUBERNETES") != "true"

if os.Getenv("KUBERNETES") != "true"

:man_facepalming:
/lgtm


Thread starvation

I saw #78 and I triggered flashbacks to an issue I ran into when implementing my own io_uring runtime a year ago.
I had 1k frontend requests evenly split between threads and all blocked (parked) on 1 backend fetch. When the backend fetch finishes, the thread that issued it is woken and can process its frontend requests. However, the requests bound to the other threads will stay blocked on iouring until they get randomly woken by unrelated network activity. This can cause significant latency spikes depending on the traffic load.
Basically, once a thread calls submit_and_wait, there's no way to unpark it for non-IO work.
My solution was to was create an eventfd for each thread. Right before getting parked, the thread queues up a read on the eventfd and submits all of the tasks to uring. In order to unpark a thread, another thread writes to the eventfd using its local uring. There's some atomics sprinkled in to avoid unnecessary wakeups.
My repo is currently private at the moment because my Rust sucks but I can make it public if the reference would be useful.

We currently park with epoll, which wakes when uring has completions available. This means we don't actually have this issue, although it is something we need to keep in mind whenever we move tokio-uring into mainline tokio.

Ah apologies, I jumped to conclusions when I saw the comment in the PR. I agree that this shouldn't be an issue then.
Just to clarify, you're using epoll with the uring's FD instead of submit_and_wait. It does mean you have to issue two syscalls to park a thread (submit and then epoll) but you still get the benefit of batching with uring.
My runtime polls futures until they're all pending and then calls submit_and_wait to submit/park the thread. I'd love to see this implemented in tokio so I can delete my awful code!

@kixelated you are correct about our use of epoll. Also, something along the lines of what you are doing is the plan once we move tokio-uring into tokio (may be a while though).
 This is especially true if you use futures that aren't dependent on io_uring for waking (for example, async channels), as those can starve tasks you have which are dependent on the submit call.

Yeah, the plan was to call submit if the uring queue reached a certain size or some duration had elapsed. I also wanted to investigate IORING_SETUP_SQPOLL as an alternative.


why still need to use conf_mask but not y_true[...,4]

I m very confused with this, I still read know the loss_code but still don't know why,
what directly different with this,
 conf_mask = conf_mask + tf.to_float(best_ious < 0.6) * (1 - y_true[..., 4]) * self.no_object_scale    conf_mask = conf_mask + y_true[..., 4] * self.object_scale
here compute the pred_box and true_box iou in each picture, and get the best anchor for each per cell prod anchor_box,
1、iou_scores shape = [batch_size, grid_w, grid_h, anchor_num, max_box_per_image],
2、and then use，best_iou = tf.reduce_max(iou_scores, axis=4),  get each best iou for each cell,
shape =[batch_size, grid_w, grid_h, anchor_num]
3、if the iou <0.6,  it means the iou is not enough, so conf_mask[I]=0，
if iou >0.6, if means here probability have box, so conf_mask[i]=1
why still need to calculate the conf_mask but not use y_ture[...,4], just like the coord_mask, it's not easy to got is , every little help is very appreciate, thank in advance
@experiencor @hobson @mrshu @sirotenko @usatenko

why need to make a distinction between coord_mask, conf_mask, and class_mask , use the y_true[..., 4] directly is work well ,right?
help to receive you reply, thanks in advance

the code :
class_wt = tf.ones(num_classes)
class_mask = y_true[..., 4] * tf.gather(self.class_wt, true_box_class) * self.class_scale

is total equals to :
class_mask = y_true[..., 4]
so why still have to do like this, is there other detail I have oversight?  it's very appreciate to have any help , thanks in advance

why need to make a distinction between coord_mask, conf_mask, and class_mask , use the y_true[..., 4] directly is work well ,right?
hope to receive you reply, thanks in advance

no, it's not right to use y_true[..., 0] directly, if we use ,it more like to random choice some cell to compute the loss, not relative to the y_pred, so it's wrong

and one more thing:
maybe use  tf.exp(tf.sofrmax(pred[..., 2:4])) instead of tf.exp(pred[..., 2:4]) directly?

the code :
class_wt = tf.ones(num_classes)
class_mask = y_true[..., 4] * tf.gather(self.class_wt, true_box_class)

is total equals to :
class_mask = y_true[..., 4]
so why still have to do like this, is there other detail I have oversight? it's very appreciate to have any help , thanks in advance

I think class_wt sholud represent the importance of different classes. But here, it is set the same value for all classes, so it make no sense here.
And recent days, I am also working on the yolo2, and want to write this in tf2.0, so we can help each other.

I m very confused with this, I still read know the loss_code but still don't know why,
what directly different with this,

conf_mask = conf_mask + tf.to_float(best_ious < 0.6) * (1 - y_true[..., 4])
conf_mask = conf_mask + y_true[..., 4]   (forget to multiply best_ious >0.6 ? )

here compute the pred_box and true_box iou in each picture, and get the best anchor for each per cell prod anchor_box,
1、iou_scores shape = [batch_size, grid_w, grid_h, anchor_num, max_box_per_image],
2、and then use，best_iou = tf.reduce_max(iou_scores, axis=4), get each best iou for each cell,
shape =[batch_size, grid_w, grid_h, anchor_num]
3、if the iou <0.6, it means the iou is not enough, so conf_mask[I]=0，
if iou >0.6, if means here probability have box, so conf_mask[i]=1
why still need to calculate the conf_mask but not use y_ture[...,4], just like the coord_mask, it's not easy to got is , every little help is very appreciate, thank in advance
@experiencor @hobson @mrshu @sirotenko @usatenko

I have the same quesion, please tell me that if you solve the problem.

I m very confused with this, I still read know the loss_code but still don't know why,
what directly different with this,

conf_mask = conf_mask + tf.to_float(best_ious < 0.6) * (1 - y_true[..., 4])
conf_mask = conf_mask + y_true[..., 4]   (forget to multiply best_ious >0.6 ? )

here compute the pred_box and true_box iou in each picture, and get the best anchor for each per cell prod anchor_box,
1、iou_scores shape = [batch_size, grid_w, grid_h, anchor_num, max_box_per_image],
2、and then use，best_iou = tf.reduce_max(iou_scores, axis=4), get each best iou for each cell,
shape =[batch_size, grid_w, grid_h, anchor_num]
3、if the iou <0.6, it means the iou is not enough, so conf_mask[I]=0，
if iou >0.6, if means here probability have box, so conf_mask[i]=1
why still need to calculate the conf_mask but not use y_ture[...,4], just like the coord_mask, it's not easy to got is , every little help is very appreciate, thank in advance
@experiencor @hobson @mrshu @sirotenko @usatenko

I have the same quesion, please tell me that if you solve the problem.

I stil not got this


fix: upgrade to version 3.0.734

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign pmuir
You can assign the PR to them by writing /assign @pmuir in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jenkins-x-bot-test: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

vault
c73797b1da280c76b7eb1f6622e55382e671dff9
link
/test vault

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository. I understand the commands that are listed here.

@jenkins-x-bot-test: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

vault
c73797b1da280c76b7eb1f6622e55382e671dff9
link
/test vault

gsm
c73797b1da280c76b7eb1f6622e55382e671dff9
link
/test gsm

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository. I understand the commands that are listed here.


 One subpackage at a time.

Would be closed by something to the effect of feature/apply_codeformat_workflow, from #775 .
However, seems too impractical to implement this way. Better method might be to use directives from the pyproject.toml or a pre-commit,

Would be closed by something to the effect of feature/apply_codeformat_workflow, from #775 .
However, seems too impractical to implement this way. Better method might be to use directives from the pyproject.toml or a pre-commit,


Need help setting up Denoify

Hey @danielrearden,
Nice project,
I saw you attempted to use package.json -> denoify -> ports with a pika URL.
Unfortunately only deno.land/x and githubraw url are supported by this method.
Here what you need is to write a custom replacer that will tell Denoify how to replace the graphQL imports.
Do you need help with that?
Regards,

That would be amazing, @garronej 😍 Any help in getting this ported to deno would be greatly appreciated!

@danielrearden Ok it should be no problem, I will write the replacer for graphql... when I manage to free some time for opensource. :octocat:

@garronej I ended up writing the replacer myself. Closing this issue since we now officially have Deno support 🦕

Cool!

@danielrearden
Sorry about that:

Denoify should not forward this import statement to your replacer...
That said, I don't understand why you don't do that instead:


Changing download folder?

I'm currently working through the docs to try and find this as well, but have you seen any way to change the download location? Ideally I'd like to just download the video into a subdirectory relative to where the script is being run from, ie -
/puppetcam
    export.js
    ... etc
    /videos
        spinner.webm <- can we get it here?

Currently it gets dropped into the ~/Downloads/spinner.webm folder on my virtual machine, then I have to manually move to a linked folder to access (using Vagrant and Virtualbox with a *nix image)
I tried with a relative path as a filename, but no luck, still dropped it in ~/Downloads

You can only set relative paths from the download location as per the docs. To set a custom downloads folder you need to create a chrome profile with that appropriate location and use that with puppeteer. I thought it is easier to mv the file through a script than go through that hassle

Looks like that is indeed the best way to go, for any future readers, my solution using fs
fs.copyFile('/home/vagrant/Downloads/'+ exportname, __dirname +'/screencasts/'+ exportname, (err) => {if(err) console.log(err)});
fs.unlink('/home/vagrant/Downloads/'+ exportname, (err) => {if(err) console.log(err)});

I couldn't just do fs.rename since moving from a non-vagrant synced folder to a synced folder is across partitions


ImportError: cannot import name 'translate_llvmir_to_hsaco' from 'triton._C.libtriton.triton'

On a fresh Venv installing bitsandbytes with pip I get a weird error related to triton. I've confirmed triton is installed. I have tried uninstalling and reinstalling triton, it did not fix the issue.
Let me know if you would like me to run some other commands !
Report :
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/libbitsandbytes_cuda121_nocublaslt.so
/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/lib/jvm/java-17-openjdk/lib/*'), PosixPath('/usr/lib/jvm/java-17-openjfx/lib/*')}
  warn(msg)
/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/org/freedesktop/DisplayManager/Session0')}
  warn(msg)
/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//debuginfod.archlinux.org '), PosixPath('https')}
  warn(msg)
/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/org/freedesktop/DisplayManager/Seat0')}
  warn(msg)
/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('rofi/|usr|bin|pycharm/434339-0-archlinux_TIME46931076')}
  warn(msg)
CUDA SETUP: CUDA runtime path found: /opt/cuda/targets/x86_64-linux/lib/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 5.2
CUDA SETUP: Detected CUDA version 121
/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/libbitsandbytes_cuda121_nocublaslt.so...
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/__init__.py", line 16, in <module>
    from .nn import modules
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/nn/__init__.py", line 6, in <module>
    from .triton_based_modules import SwitchBackLinear, SwitchBackLinearGlobal, SwitchBackLinearVectorwise, StandardLinear
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/nn/triton_based_modules.py", line 8, in <module>
    from bitsandbytes.triton.dequantize_rowwise import dequantize_rowwise
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/bitsandbytes/triton/dequantize_rowwise.py", line 10, in <module>
    import triton
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/triton/__init__.py", line 20, in <module>
    from .runtime import (
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/triton/runtime/__init__.py", line 1, in <module>
    from .autotuner import Config, Heuristics, autotune, heuristics
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/triton/runtime/autotuner.py", line 7, in <module>
    from ..compiler import OutOfResources
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/triton/compiler/__init__.py", line 1, in <module>
    from .compiler import CompiledKernel, compile, instance_descriptor
  File "/home/tiemajor/projects/perso/python/ml/langchain/venv/lib/python3.11/site-packages/triton/compiler/compiler.py", line 14, in <module>
    from .._C.libtriton.triton import (add_external_libs, compile_ptx_to_cubin,
ImportError: cannot import name 'translate_llvmir_to_hsaco' from 'triton._C.libtriton.triton' (unknown location)

What triton version are you using? It seems to be a triton-related error rather than bitsandbytes since it fails at triton import. I cannot find the translate_llmvmir... function in the triton repo.

So I had the same issue with flash-attn and pytorch version 2.1.0, this was fixed by upgrading to 2.1.0.dev20230713+cu118, however flash-attn triton kernels would still hang, which required me to manually downgrade triton, as suggested here.
Hope this helps :)

@TimDettmers For reference I tried with triton 2.0.0.post1 as well as 2.0.0. I could not find a reference to the function in the triton repo as well so I assumed bitsandbytes might be doing something with triton. Closing.

Hi @seifane
I am currently facing the same error. What did you do to sort this out?
Thanks

@kushalj001 I could not manage to fix it so I switched to using llama.cpp

Hi @seifane
I faced the similar issue. Solution

I faced with the same error. Can anyone tell me what is the reason? "ImportError: cannot import name 'Config' from 'triton' (unknown location)"


Update dependencies

Tested each update separately, nothing is broken.
Changelogs
Autoprefix
https://github.com/postcss/autoprefixer/blob/master/CHANGELOG.md#61-bil-shaʿb-wa-lil-shaʿb
react-redux

https://github.com/rackt/react-redux/releases/tag/v4.0.0
https://github.com/rackt/react-redux/releases/tag/v3.1.0

eslint
https://github.com/eslint/eslint/blob/master/CHANGELOG.md
eslint-plugin-react
https://github.com/yannickcr/eslint-plugin-react/blob/master/CHANGELOG.md#380---2015-11-07
gulp-eslint
https://github.com/adametry/gulp-eslint/blob/master/CHANGELOG.md#110
chai:

https://github.com/chaijs/chai/releases/tag/3.2.0
https://github.com/chaijs/chai/releases/tag/3.3.0
https://github.com/chaijs/chai/releases/tag/3.4.0

chai-immutable:

https://github.com/astorije/chai-immutable/releases/tag/v1.2.0
https://github.com/astorije/chai-immutable/releases/tag/v1.3.0
https://github.com/astorije/chai-immutable/releases/tag/v1.4.0
https://github.com/astorije/chai-immutable/releases/tag/v1.5.0

Express

https://github.com/strongloop/express/releases/tag/4.13.0
https://github.com/strongloop/express/releases/tag/4.12.0

Awesome. Please let's wait with merge to friday, since I have trainging and during that time it's better not to update anything.

OK, understood.


Backend not running (exit with 2)

I got the following error in the log
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: --capture-output

I got the following error in the log
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: --capture-output

open the backend start_prod.sh and remove the  --capture-output flag.

open the backend start_prod.sh and remove the --capture-output flag.
I did that, but the error message now is mrs-backend | gunicorn: error: unrecognized arguments:. It's the same, but without --capture-output after it.

I'm assuming there was a breaking change in the gunicorn package at some point.
Workaround:
Recreate this file with the same content with Linux (WSL)

same problem under windows. actually docker should be the workaround for https://github.com/MycroftAI/mimic-recording-studio/issues/87 .... now we are already at the workaround of the workaround


Can not start timescale-prometheus with error

Hi there,
I am new to timescale db and I am trying to use it as a prometheus long term storage.
At first, My OS is  Centos 7.
I downloaded timescale-prometheus binary, after then execute binary like this.
$ wget https://github.com/timescale/timescale-prometheus/releases/download/0.1.0-beta.3/timescale-prometheus_0.1.0-beta.3_Linux_x86_64
$ chmod 700 timescale-prometheus_0.1.0-beta.3_Linux_x86_64
$ ln -sf timescale-prometheus_0.1.0-beta.3_Linux_x86_64  timescale-prometheus
$ ./timescale-prometheus -db-ssl-mode=disable -log-level=debug
It generates erros.
{"caller":"main.go:162","level":"info","msg":"Version:0.1.0-beta.3; Commit Hash: ","ts":"2020-09-03T08:48:38.005Z"}
{"caller":"main.go:163","config":"&{listenAddr::9201 telemetryPath:/metrics pgmodelCfg:{host:localhost port:5432 user:postgres password: **** sslMode:disable dbConnectRetries:0 AsyncAcks:false ReportInterval:0 LabelsCacheSize:10000 MetricsCacheSize:10000 SeriesCacheSize:0 WriteConnectionsPerProc:4 MaxConnections:-1} logLevel:debug haGroupLockID:0 restElection:false prometheusTimeout:-1 electionInterval:5000000000 migrate:true corsOrigin:0xc00015ea00}","level":"info","ts":"2020-09-03T08:48:38.005Z"}
{"caller":"main.go:174","level":"warn","msg":"No adapter leader election. Group lock id is not set. Possible duplicate write load if running adapter in high-availability mode","ts":"2020-09-03T08:48:38.005Z"}
{"caller":"main.go:186","level":"error","msg":"Aborting startup because of migration error: Error while trying to migrate DB: Error encountered during migration: error executing migration script: name idempotent/base.sql, err ERROR: function show_chunks(hypertable => text, older_than => timestamp with time zone) does not exist (SQLSTATE 42883)","ts":"2020-09-03T08:48:38.325Z"}
I don't know what there logs mean.
I'm newbee at timescale-prometheus. what's wrong with me? plz let me know. thank you for reading this.

Thank you for reporting this. The logs seem to speak errors related to running migrations with the DB. I am not sure if this is just in the releases or if it was fixed in some PR after the release. Anyways, we will get back to this soon. Thank you.

@tr2vil can you mention the version of your timescale-db?

@tr2vil can you mention the version of your timescale-db?

How can I check timescale-db's version ?

In additional, I want to build timescale-prometheus from source.
According to the post, (https://github.com/timescale/timescale-prometheus/blob/master/docs/binary.md#-building-from-source) ,
I type this but not working.
Fetch the source code of Timescale-Prometheus in any directory
$ git clone git@github.com:timescale/timescale-prometheus.git
Cloning into 'timescale-prometheus'...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Is this repository is still working?

How can I check timescale-db's version ?

SELECT default_version, installed_version FROM pg_available_extensions ORDER BY installed_version DESC LIMIT 1;

update_from_version = 1.7.3

Seems like your db version is correct.

I type this but not working.

Maybe, your ssh keys are not working. Anyways, try the HTTP way.
git clone https://github.com/timescale/timescale-prometheus.git

@Harkishen-Singh
timescale db version is described below.
tutorial=# SELECT default_version, installed_version FROM pg_available_extensions ORDER BY installed_version DESC LIMIT 1;
default_version | installed_version
-----------------+-------------------
2.0.0-dev       | 2.0.0-dev

Could you try that command again? I just updated it. I had copied from my system as it was specific to me.
@tr2vil

@Harkishen-Singh
I use timescale-prometheus for store prometheus exporter's metric into postgresql.
anyway, you talked to me
"Could you try that command again"
which command do you mean?
I compilied timescale-prometheus binary. and then try like this
ls -al
total 23440
drwxr-xr-x 2 root root     4096 Sep  4 18:10 .
drwxr-xr-x 3 root root     4096 Sep  4 18:05 ..
-rw-r--r-- 1 root root    16270 Sep  4 18:05 main.go
-rw-r--r-- 1 root root     6920 Sep  4 18:05 main_test.go
-rwxr-xr-x 1 root root 23966576 Sep  4 18:10 timescale-prometheus
./timescale-prometheus -db-ssl-mode=disable -log-level=debug
{"caller":"main.go:181","level":"info","msg":"Version:0.1.0-beta.3.dev.0; Commit Hash: ","ts":"2020-09-04T09:11:13.192Z"}
{"caller":"main.go:182","config":"&{listenAddr::9201 telemetryPath:/metrics pgmodelCfg:{host:localhost port:5432 user:postgres password: **** sslMode:disable dbConnectRetries:0 AsyncAcks:false ReportInterval:0 LabelsCacheSize:10000 MetricsCacheSize:10000 SeriesCacheSize:0 WriteConnectionsPerProc:4 MaxConnections:-1} logLevel:debug haGroupLockID:0 restElection:false prometheusTimeout:-1 electionInterval:5000000000 migrate:true stopAfterMigrate:false corsOrigin:0xc0002380a0}","level":"info","ts":"2020-09-04T09:11:13.192Z"}
{"caller":"main.go:193","level":"warn","msg":"No adapter leader election. Group lock id is not set. Possible duplicate write load if running adapter in high-availability mode","ts":"2020-09-04T09:11:13.193Z"}
{"caller":"main.go:206","level":"error","msg":"Aborting startup because of migration error: Error while trying to migrate DB: Error encountered during migration: error executing migration script: name idempotent/base.sql, err ERROR: function show_chunks(hypertable => text, older_than => timestamp with time zone) does not exist (SQLSTATE 42883)","ts":"2020-09-04T09:11:13.579Z"}
I'm sorry, but is it right way to start this binary to put the metrics of prometheus in postgresql?

which command you mean?

SELECT * FROM pg_available_extensions;
The main aim behind this is to know the default and intalled version of Timescaledb and the extension. This should give the necessary output.

@Harkishen-Singh
$ psql -U postgres -h localhost -d timescale
psql (12.4)
Type "help" for help.
timescale=# SELECT * FROM pg_available_extensions;
name     | default_version | installed_version |                              comment
-------------+-----------------+-------------------+-------------------------------------------------------------------
plpgsql     | 1.0             | 1.0               | PL/pgSQL procedural language
timescaledb | 2.0.0-dev       |                   | Enables scalable inserts and complex queries for time-series data

@Harkishen-Singh
In the timescale-prometheus doc, ( https://github.com/timescale/timescale-prometheus/blob/master/docs/binary.md#building-from-source) ,
It is said, prometheus remote write/read configuration is necessary to store prometheus metric
remote_write:

url: "http://:9201/write"
remote_read:
url: "http://:9201/read"

Is the 9201 port run by timescale-prometheus?

Yes, promethues should have that config and assuming that you are running locally, the connector address will be localhost.
Yes, 9201 is the port for timescale-prometheus. Prometheus will run and send live samples to port 9201 (on which connector is running).

@tr2vil could you confirm if everything is working for you after switching to 1.7.3 version for Timescaledb?

@Harkishen-Singh
Yes, I'll remind to you after reinstall timescale-prometheus version 1.7.3.

@Harkishen-Singh @JLockerman ,
I had reinstalled timescale version 1.7.3.
After then, I re-execute success like this. I'll do more test. Thank you for your helping~!
'''
$ ./timescale-prometheus -db-ssl-mode=disable -log-level=debug
{"caller":"main.go:181","level":"info","msg":"Version:0.1.0-beta.3.dev.0; Commit Hash: ","ts":"2020-09-07T01:26:27.472Z"}
{"caller":"main.go:182","config":"&{listenAddr::9201 telemetryPath:/metrics pgmodelCfg:{host:localhost port:5432 user:postgres password: **** sslMode:disable dbConnectRetries:0 AsyncAcks:false ReportInterval:0 LabelsCacheSize:10000 MetricsCacheSize:10000 SeriesCacheSize:0 WriteConnectionsPerProc:4 MaxConnections:-1} logLevel:debug haGroupLockID:0 restElection:false prometheusTimeout:-1 electionInterval:5000000000 migrate:true stopAfterMigrate:false corsOrigin:0xc000183b80}","level":"info","ts":"2020-09-07T01:26:27.472Z"}
{"caller":"main.go:193","level":"warn","msg":"No adapter leader election. Group lock id is not set. Possible duplicate write load if running adapter in high-availability mode","ts":"2020-09-07T01:26:27.472Z"}
{"caller":"extension.go:51","level":"warn","msg":"timescale_prometheus_extra is not available, proceeding without extension","ts":"2020-09-07T01:26:27.498Z"}
{"caller":"client.go:81","level":"info","msg":"host=localhost port=5432 user=postgres dbname=timescale password='' sslmode=disable connect_timeout=10","numCopiers":32,"pool_max_conns":50,"pool_min_conns":8,"ts":"2020-09-07T01:26:27.512Z"}
{"caller":"query_engine.go:13","level":"debug","msg":"Lookback delta is zero, setting to default value","ts":"2020-09-07T01:26:27.516Z","value":"5m0s"}
{"caller":"main.go:316","level":"info","msg":"Starting up...","ts":"2020-09-07T01:26:27.516Z"}
{"addr":":9201","caller":"main.go:317","level":"info","msg":"Listening","ts":"2020-09-07T01:26:27.516Z"}
'''


feat: add support for generic OpenID Connect authentication provider

Description
implementation for another authentication provider;
a generic OIDC IDP (ory, dex, authelia, keycloak, zitadel, gitlab, gitea, ...)
this feature was tested and developed against dex-idp but should be generic/configurable enough, to work against any arbitrary identity provider.
the requested scopes can be configured, while the read claims are still hardcoded.
Changelog

frontend: add login flow support for OpenID Connect
backend: add support for generic OIDC authentication

Checklist

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] The light- and dark-theme are both supported and tested
[ ] The design was implemented and is responsive for all devices and screen sizes
[ ] The application was tested in the most commonly used browsers (e.g. Chrome, Firefox, Safari)

(Optional) Visual Changes

the icons seem to originate from Figma and are exported/converted manually. as i have not access to the sources, i have edited an SVG i found on the internet. you might want to create an original one in Figma; same as for the Google and Microsoft icons

cool idea, skimming over it looks pretty good! i'll try to add a proper review in the coming days.

@mateo-ivc if you have time, could you take a look at this?


Decide if auto-include is correct

Right now we're auto-including Graphiti code in ActionController::Base source. This may be too aggressive.
First question is whether this has any potential downsides. If so, we should definitely let it be at least configurable. My inclination is to leave it for now and have us be less aggressive if people report issues.

See also https://github.com/wagenet/graphiti-rails/issues/5#issuecomment-488724054 and https://github.com/wagenet/graphiti-rails/issues/5#issuecomment-488733910

Mixin vs Railtie relevant to this one: https://github.com/wagenet/graphiti-rails/issues/10#issuecomment-490908327

I think with rescue_registry (#22) this is fine.


Add capability to gather the git commit log since the last release

Add capability to gather the git commit log since the last release and use that to start the release notes.
Warning: This is experimental.

There are some loose ends that I am working on in the code marked with TODOs to remove them.

Not all the bits are documented because I have not figured out the right API yet.

Add a git toolchain

It gets the path to the git binary and the path to your source tree
therefore it can only be used locally.
Fixes #257

Add a git_changelog() rule that will run git to get the change log.

Advances #228

Change print_relnotes to accept a changelog file, instead of running git locally.

print_relnotes now does not call git directly and more, so we could feed it a change log from any revision control system.

Next steps:

Add the capability to reduce the change log from everything to only
the commits that have RELNOTES sections.

Note to reviewers: This is under tested by normal standards. Doing so would require a large bit of scaffolding to test 5 lines of actual code that calls git and grabs the output. The issues are many

we can/should not run git in CI
we need to create a git repository of known content, but that may not test anything real
we can create a mock or fake git, but then we are not actually testing anything.

In contrast, the next phase, where we collapse a changelog to only have commits with RELNOTES, will require tests, but we can craft them from hand curated text files of git log output.

@fweikert you might want to take a look at where this is going. I don't think I would want this directly in Bazel, because of the git toolchain, but perhaps the next round, where we brew down the changelog by relnotes might be of use.  The overall goal is to get a well lit path so that any repository using rules_pkg for the distribution can create release notes with the identical techniques Bazel uses.

Thanks for looking at this. I may be heavily pre-occupied for the next week or two, so it may take a while for me to get back to this one.

I did a first round of nit fixes. I added some theory of operation to git.bzl. and more in places where I do funny things.
I'll address the trickier questions in a few days.

I did a first round of nit fixes. I added some theory of operation to git.bzl. and more in places where I do funny things.
I'll address the trickier questions in a few days.

Thanks for taking a look, but I'm not seeing the changes; were they pushed to github?

I don't know what happened with the push.  I got it going again.


No information provided

Could you please provide some discription on what does the provided code do?

Hi @sadransh !
Thank you for your interest for my project.
The code provided offer a structure to get the robot working in a complex indoor environment with autonomous navigation.  SLAM methodology is used and RTABmap is the main core.
The robot use UKF to filter visual odometer, wheel odometer and absolute IMU.

yeah! I saw the information on https://gbr1.github.io/erwhi_hedgehog.html
it's good to add it to the first page of the project.
have you tried adding t265 as SLAM processor ? Doesn't it make the robot move faster ? and make more accurate mapping?

yes!
There is a branch!
Unfortunately I'm preparing some stuff for Maker Faire Rome (Oct 18-20) and I have not completed the launch file and ukf modifications.


⚠ ETF has degraded performance

In 4cf2a27, ETF (https://markets.hankyung.com/equity-product/etf) experienced degraded performance:

HTTP code: 200
Response time: 6798 ms

Resolved: 비트코인 performance has improved in 06df349.