Research technologies to improve in-memory data model

Goal:  Increase the capacity of OpenRefine’s data processing by moving to an alternative, higher performance in-memory / on-disk data storage technology. See document Planned in Phase 2 enhancements
We can probably lower our memory overhead with Apache Arrow: https://arrow.apache.org/docs/memory_layout.html
Thad chatted with Arrow folks and they think Arrow is a great fit for OpenRefine especially considering our columnar operations.
Here's a bit more info on how Apache Arrow folks say we might leverage them...
https://apachearrow.slack.com/archives/C0S8Z7VBK/p1516934654000036

Thad Guidry [8:44 PM] Anyone know about us at OpenRefine ?  We're curious if/how Arrow might be useful for us to improve our ancient in-memory data model and processing so desktop and laptop users can work with more local data in OpenRefine ?
bhulette [10:28 AM] @Thad Guidry I hadn't heard of OpenRefine before, but it definitely looks like something that could benefit from the Arrow format. The biggest selling point would be the ability to easily interoperate with other tools that use Arrow (e.g. Spark, pandas, etc...) without any serialization costs.
[10:29 AM] I don't know what your current data model looks like, but there could be performance benefits from the columnar layout as well
Thad Guidry [10:53 AM] @bhulette That's described here https://github.com/OpenRefine/OpenRefine/wiki/Server-Side-Architecture
bhulette [11:05 AM] yeah that looks pretty amenable to the arrow format - a loose analogy could be that "column models" are specified by the arrow schema, and the "raw data" is stored in record batches/dictionary batches
Thad Guidry [11:06 AM] @bhulette gotcha
bhulette [11:07 AM] the column groups idea for storing a tree is pretty interesting
[11:08 AM] you would be able to specify blank cells in arrow using validity buffers
Thad Guidry [11:11 AM] @bhulette keep the ideas coming !  (fyi, we had also thought of Apache Ignite)
bhulette [11:12 AM] Im not sure if arrow could help with storing changes
[11:13 AM] but that could be a welcome addition to the project, if people like @wesmckinn think it's in scope :slightly_smiling_face:
[11:15 AM] is the OpenRefine server distributed?
Thad Guidry [11:16 AM] @bhulette no
[11:18 AM] @bhulette OpenRefine is used locally (desktop/laptop) to clean data.  We eventually want to separate the backend and frontend, so that we can do large transformations via streaming/batching against Apache BEAM, etc.  But our user base, once they get that big, typically use other tools.

We might also just leverage a wider surface than Apache Arrow itself and instead incorporate Apache Drill (Apache Arrow + Hadoop, NoSQL, and others support)
https://drill.apache.org/
I've taken the Docker Drill embedded version out for a spin and its really fast and performant, even when only given 4gigs Ram.  I was impressed with the ability to connect a few JSON files along with a large 500meg CSV file...all 3 being joined and compared with SQL operations at the same time.
I'd like to see more research by others to look into Apache Drill as a complete data backend for us.
I thought this was also interesting... someone connected Drill to Spark via Spark's dataframe

@wetneb @fpompermaier OMG, Apache Drill just keeps getting better and better.  They now support custom functions!
https://drill.apache.org/docs/develop-custom-functions-introduction/
Developers can code against the Java API provided and deliver a .jar file for their custom function(s).
After my 1 month of playing with Apache Drill on my Windows server, I am now thoroughly convinced for non-batch OpenRefine, we need to try to use it (along with its dfs storage plugin) to deliver exceptional value as a new storage backend that is highly configurable, supports a REST API, and a really nice Web UI for easy configuration for our more technical users.
Parquet files load and query amazingly fast... 4 Gig CSV file with 50 columns is query-able in seconds.
It also allows flattening JSON data (which is what we also do) and makes it easy to parse and query.
select id, type, name, ppu
from dfs.`/Users/brumsby/drill/donuts.json`;

A few details regarding Memory config & usage: https://drill.apache.org/docs/configuring-drill-memory/
@fpompermaier Do you think you could hack together a Proof of Concept for using Apache drill embedded instance with OpenRefine build on a new git branch ?  Linux only is fine for now, since I can just check out the branch and work on spinning it all up inside an Ubuntu Docker container on my Windows machine just to test it out once you have something marginally working.  Can you work on this over the next 2 months, please please please!?!?!

Hi @thadguidry , we've used Apache Drill and it's a very useful tool.
It's probably the best quick-and-dirty SQL tool for Big Data right now.
Refine + Drill could be a very good choice but, in the end, this is what Dremio is (https://swtrends.wordpress.com/2017/07/20/apache-drill-arrow-dremio/).
Maybe it could worth to improve Dremio at this point and move there many of the functionalities of OpenRefine..also because I won't have much time to work on this during this 2 months.
Have you have took a look at Dremio? It's based on the Apache Drill codebase with the idea of compatibility of other Big Data tools in mind. Moreover, using Apache Arrow as the in-memory model should bring huge benefits to the performance of the overall project.

@fpompermaier I took a look at Dremio, and found that Dremio is not essentially "Refine + Drill" completely.  My high level takeaway is that:

Its a different tool with a different purpose, providing easy data aggregation, merging, & summery.
It seems to support Undo/Redo with Historical Operations across Virtual Datasets (similar to OpenRefine)
It's SQL first, even though it lowers the barrier of learning by providing easy drop-down menus to perform common functions (similar to OpenRefine, but no alternative language choices like we have)
It's data wrangling capabilities are limited, since that is not the scope of Dremio, but its data aggregation & summery capabilities are very beneficial to our user base.

@thadguidry Rewriting OpenRefine's GUI, refactoring the codebase to make the backend pluggable, and then write OpenRefine-specific adapters for Drill is going to be a lot of work.  You could package up existing projects into a JavaFX app and reduce your maintenance burden to whatever UI enhancements you can't upstream.  🤷♂
P.S. Just ran across Kylo.

Removing the "to be reviewed" label: yes Arrow will be a natural choice when we revamp the data model in 2020.

Closing this now.  As @wetneb has stated before, we can take what we need from the discussion on this issue in later stages after we progress more on Apache Spark prototype.


🐛(frontend) prevent overwriting app state with the dataContext

Purpose
Avoid overwriting the refreshed access token.
Proposal
When appData updates occurred, the AppContentLoader component's rendering was triggered. This rendering action was causing a reset of the appData to its initial state, resulting in the unintended overwriting of crucial changes, including the recently refreshed access token.
Now, the AppContentLoader initializes the data provided by Django only during the initial rendering to avoid unintended overwrites.

Some optimizations may be required. Currently, requests are executed in parallel, which leads to each request triggering a refresh token request. I need to investigate whether there are any design issues in the current implementation.

[ ] Explore options to share the refresh token between parallel requests.
[ ] Consider executing comparison requests in a sequential order.


Use dirname(@FILE) instead of Pkg.dir, modernize Travis, and fix up REQUIRE

See commit messages

The test failure is a bit odd, can't find the dataset. Hoping it's a one-off fluke, so I hit it again.

Not sure why the test is failing. Have you tried to reproduce the failure on your end?

No I haven't, this was one of a big batch of many pull requests. Let's see if restarting gives a different result this time.

@tkelman it appears that the issue is that the location that the tests are being run from relative to the location of the downloaded files got moved somehow in your code change, so it can't find the enzo_tiny_cosmology file or anything else it downloaded to use as test datasets. Can you check the working directory that the tests get run from?

Oh, it gets run from the package directory. You should handle the downloading of the test data in your runtests.jl julia script so users could actually run Pkg.test("YT"), rather than only ever working on Travis.

@tkelman care to change runtests.jl so it downloads test data from within this PR? I'd be happy to see this merged in.

Guess I could take a look sometime later. Wish github had a "mark notification as unread" option

@tkelman I could try doing something about it--I haven't quite figured out how to use this new feature where you can add to someone's PR yet. I think it should go into this PR though.
Sorry about the delay on this--million balls in the air over here. I'll look into it tomorrow.
I'm very happy that someone's getting some use out of this code.

I just checked the box that allows you to edit the branch. I've also got a lot going on, no hurry.

I just submitted https://github.com/tkelman/YT.jl/pull/1, which should hopefully make the tests pass.

Would be more portable to use Base.download and Libz.jl, but better than nothing

Yeah, maybe I can try that after we merge this in. Tests are only going to pass for v4.x, but we shouldn't try to fix that in here.

Coverage increased (+1.7%) to 94.35% when pulling 46b01123cd4f1c5fa7340ea9c19b3bd84d44f204 on tkelman:nopkgdir into 77cbb09c1987e1d02cbb13ca0e09cae3a6946720 on jzuhone:master.

Tests passed on v0.4.x, failed on v0.5.x, which is what I expected, so I'm going to merge this in and try to make things compatible with both (and nightly) this week.
Thanks, @tkelman!


Example of Code Integrity

The page https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-defender-exploit-guard/enable-virtualization-based-protection-of-code-integrity#enable-hvci-using-intune refers to AppLocker CSP to Enable virtualization-based protection of code integrity but at not times i could find an example on how to do it. Can we get one posted??
Thanks

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6aa7dd86-8bcc-bb92-0a66-86e24701e5a4
Version Independent ID: 277e194c-0fa3-7da6-1814-f6951e440df6
Content: AppLocker CSP
Content Source: windows/client-management/mdm/applocker-csp.md
Product: w10
Technology: windows
GitHub Login: @lomayor
Microsoft Alias: lomayor

@officedocsbot assign @mypil

@schenardie  - Thank you for submitting feedback.
I will get this issue over to the Win10 ITPro writing team for investigation.
Thank you for reporting and making the docs better. Much appreciated.
I made a note to request the team to update this when the work is complete.

@schenardie Did you get a chance to see AppLocker CSP document at https://docs.microsoft.com/en-us/windows/client-management/mdm/applocker-csp which has all the details and example listed in the doc.
Thanks

Hi, yes i did. i do not see an example on how to deploy code integrity in it.
Can you point it out to me please if it is on the document?

Please see this image, see if this portion helps. Otherwise, you have to open a support case with Microsoft Tech team so that your request can be escalated to the respective team.
Thanks.
@mypil , Please guide share the process to open a support ticket with Microsoft tech support team.
Thanks.

So the documentation is unclear on how to enable Code Integrity and i need to open a support case??
Why do these features are documented then??
OK nvm close the issue then

@schenardie - Thank you for your response. Is there something within this article that you suggest to be added or edited? If yes, please share the appropriate text and where it should be placed, so we can have @joinimran  create a Pull Request to get it added.
Otherwise, if your issue is more of a feature request then for user feedback on product feature requests, you can submit via User Voice - https://support.microsoft.com/en-za/help/4021566/windows-10-send-feedback-to-microsoft-with-feedback-hub.
Alternatively, if you require technical support then you can open a support ticket. Please follow this link to contact support for Windows 10 products:
https://support.microsoft.com/en-us/hub/4338813/windows-help?os=windows-10
Please understand that this site is a product documentation site and not a support nor a feature request site. I hope this gets you pointed in the right direction. Thank you.

@officedocsbot close


PyPy tests fail after first run

Looks like this was introduced in #590.
Travis was configured to cache the .pyenv directory between builds and cloning the repo fails if the directory is not empty.
We have a few options on how to proceed:

Not cache the directory between builds (the previous behavior)
Only clone the pyev repo if the directory doesn't exist (similar to App Engine SDK)

Fixed by #595.

@jonparrott Looks like Travis creates the cached directories if they don't exists, so the check in #595 didn't work for me.  Created #630.


[MI-2256] Added an additional flag in the build command and bump plugin version

LGTM @manojmalik20


MyProfile - Validate input string for empty values

Issue Number
fixes #203

Describe the changes you've made

The handleInputChange callback is triggered on every keypress, also the user information is updated on every keypress and stored in the state variable user of the MyProfile component. However, when the save icon is pressed to update the information, no validation is done to check whether the field that was currently being updated is actually updated or not(Specifically, checking if any empty value has been passed or not). Skipping such validation results in unnecessary network calls. Hence, in the handleInputChange cb, I have set up a state updating function that updates the currently updating field on every keystroke. Thus, when the save icon is pressed, we can verify if the updated field holds any string or not(by checking its length). If it happens to be an empty string, an error toast is fired saying Input can not be empty. This ensures that the user actually enters a non-empty input.
Describe if there is any unusual behavior (Any Warning) of your code(Write NA if there isn't)
NA

Additional context (OPTIONAL)

Test plan (OPTIONAL)
A good test plan should give instructions that someone else can easily follow.

Navigate to My Profile (/myProfile).
Click on the pencil icon of any input fields.
Click on the save icon leaving the input field empty.
You can notice an error toast pop up.
Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] The title of my pull request is a short description of the requested changes.

Provide a Deployed link of route/page that needs to review
Preview: Deploy preview link here with the appropriate route
<!-- Please add deployed link to here text >

The problem I see with this implementation is that it really isn't validating anything. Values are stored regardless of the toast notification saying that the input cannot be empty.
First, we need to question ourselves what are the use cases:

a user profile may have a twitter handle that needs to be removed, therefore, empty inputs are a way to remove data. In this case we should detect this and tell the user "your data has been removed"
a user may have nothing in the twitter field, and then s/he clicks to edit, but adds nothing, and simply clicks on save. Nothing changed, so nothing was removed. In this case we can respond "nothing happened here"
a user has a twitter user and wants to fix it because s/he made a mistake in the name. In this case, we respond "your changes have been saved"
... maybe more scenarios, but I don't want to get away from the basic ones related to "empty VS non-empty"

Try to focus on that, and lets boost this PR

The problem I see with this implementation is that it really isn't validating anything. Values are stored regardless of the toast notification saying that the input cannot be empty.
First, we need to question ourselves what are the use cases:

a user profile may have a twitter handle that needs to be removed, therefore, empty inputs are a way to remove data. In this case we should detect this and tell the user "your data has been removed"
a user may have nothing in the twitter field, and then s/he clicks to edit, but adds nothing, and simply clicks on save. Nothing changed, so nothing was removed. In this case we can respond "nothing happened here"
a user has a twitter user and wants to fix it because s/he made a mistake in the name. In this case, we respond "your changes have been saved"
... maybe more scenarios, but I don't want to get away from the basic ones related to "empty VS non-empty"

Try to focus on that, and lets boost th

The problem I see with this implementation is that it really isn't validating anything. Values are stored regardless of the toast notification saying that the input cannot be empty.
First, we need to question ourselves what are the use cases:

a user profile may have a twitter handle that needs to be removed, therefore, empty inputs are a way to remove data. In this case we should detect this and tell the user "your data has been removed"
a user may have nothing in the twitter field, and then s/he clicks to edit, but adds nothing, and simply clicks on save. Nothing changed, so nothing was removed. In this case we can respond "nothing happened here"
a user has a twitter user and wants to fix it because s/he made a mistake in the name. In this case, we respond "your changes have been saved"
... maybe more scenarios, but I don't want to get away from the basic ones related to "empty VS non-empty"

Try to focus on that, and lets boost this PR

As a suggestion, if we see other web apps they don't respond to every action mostly they have saved,error,not valid only. I think the current implementation that we have in user story is fine from the user perspective. we can only change that for empty and invalid inputs API is not called.

There are two aspects that need to be considered here.

User experience.
Network calls.

The scenarios mentioned by @cyntss can be divided into either of the above-mentioned categories.
If there are any changes made(either updates or clearing the input field) only then the application should fire a network request with relevant toast messages. If no changes are made, an appropriate toast message can be displayed like, "nothing happened here" without making any network request because nothing has been changed. Sounds fair?
Consider the below as per the given scenarios:

Action
Toast message
Network call

Clear input field (E.g: remove Twitter handle)
Your data has been removed
✓

No changes made (Empty input field before & after update)
Nothing happened here
✗

Updated input field(Empty to non-empty)
Your changes have been saved
✓

No updates on the non-empty field (before update: ABC, after update: ABC) meaning the user had entered the edit mode but didn't really change anything
Nothing happened here
✗

Update input field(Non-empty to different non-empty value)
Your changes have been saved
✓

I think this covers most of the cases.

I've improved the PR that focuses on UX and displays relevant toast messages as per the actions depicted in the above table.
I've created a new function that drills down the component hierarchy to lift the state up for the input values. This is to track the initial value and the value after clicking the save icon, these two values are compared to determine if there have been any changes(updates, deletion) or not. Based on the comparison network calls and a suitable toast message is displayed. I've also considered corner cases where the user can open two or more inputs at a time, add value to one field, remove from the other, save another as it is. Under all the cases, appropriate messages are shown. I've also run lint tests and there are no errors.
@cyntss Kindly request you to review the changes and let me know what you think of it.


gif profile picture

I see it's possible to add a .gif as a profile picture.
However, when I do it, it's not moving, it's a still image.
Is it a bug ? If not, is there any chance to make it actually move like it's supposed to ?
Thanks <3

Thanks for your question, @Vincent-vst -- this feature has come up in the past. But in short, because avatars are all over the UI, animated avatars would be a massive distraction.

I guess at this point we could allow .gif avatars that animate on hover? @alexmv would that work?

zulipbot @claim

I don't think we have seen an articulated use case for this beyond the obvious one of memes, and it's quite a bit of work to do nicely (supporting our settings for on-hover animations only). So I'm closing as wontfix. Feel free to post in chat.zulip.org in #feedback an explanation for how this would help people enough to be worth significant developer effort if you'd like it reconsidered.


longpressable and self.parent_screen

Not sure why but this causes self.parent_screen to be nil:
class SelectTable < PM::TableScreen
  longpressable

  def model
    self.parent_screen.model
  end

end

while this works:
class SelectTable < PM::TableScreen

  def on_load
    self.make_longpressable()
  end

  def model
    self.parent_screen.model
  end

end

Long press events work the second way.  I don't know why longpressable causes clobbering of self.parent_screen.

Closing, feel free to update if it's still an issue.


[REQ] Add scrollTo method?

Most, if not all popular scrollbar libraries have a scrollTo method that supports pixel positions as well as top bottom left right strings. If you can suggest an alternative solution with your current code base, I'd be happy to hear :)
Thanks in advance.

+1

In case anyone has a similar problem, you can adjust scroll position using the element returned by the getViewElement method.
var scrollElem = yourScrollbarObject.getViewElement();
scrollElem.scrollTop = 42;
scrollElem.scrollLeft = 42;

Hi everyone, I made a small module just for this, check it out: pisces.
Even if it still as "beta" you can use most of its method without any problem (scrollToPosition, scrollToTop, scrollToRight, scrollToBottom, scrollToLeft....) or you use any other "scroll-to" library of your preference. At the very end you can find an example of how you can integrate it with gemini-scrollbar.
Regards.


git based dependencies doesn't work

Example deploy: https://code-flow-svuzhcsykp.now.sh/
The problem is that image-to-ascii contains a git based dependency, which now can't install, and throws an UnhandledPromiseRejectionWarning from a `404 (see logs in URL above).
This is the dep tree:
├─┬<EMAIL_ADDRESS>│ ├─┬<EMAIL_ADDRESS>│ │ └─┬<EMAIL_ADDRESS>│ │   ├─┬<EMAIL_ADDRESS>│ │   │ ├─┬<EMAIL_ADDRESS>│ │   │ │ ├──<EMAIL_ADDRESS>(git://github.com/chalk/ansi-styles.git#5dadc5c6333127f95dc0dc09ada4fbde8bf66e69)

I tried to make<EMAIL_ADDRESS>a direct dependency of my project to allow the code to still run, but the failed install process doesn't allow my project to start, so I can't work around the issue. Essentially the 404 is a total blocker for using this image-to-ascii project (pending a PR to it's repo!).
For reference, since the source doesn't work on the deployment (not sure why), here's the package.json dependencies:
"dependencies": {
  "ansi-styles": "^2.1.0",
  "busboy": "^0.2.13",
  "express": "^4.14.0",
  "image-to-ascii": "^3.0.3",
  "multer": "^1.2.0"
}

If this problem still occurs, please move it over to this repo. That's where we'll collect the bugs of the platform from now on (separate space, easier to work on)! 😊
@TooTallNate are they working now?

GitHub style URLs are working now. Generic git URLs not quite yet.


Updated calls so that they are correctly added in Tabix.

The tests demonstrate coverage and correct parsing of files. All pre-existing tests pass, and now, for the first time ever, the Tabix backend passes the compliance test suite!

This is great, thanks @kerrydc! It's a big step forward. The main issue I have with this is that we're doing a lot of VCF parsing, which in my opinion we should avoid at all costs. VCF is a very difficult format to parse, and we would be a lot better off if we could get upstream tools to do it for us. Can pysam do any of this for us? I guess we'll be moving over to @bioinformed's interface to the htslib code soon, which will hopefully remove the need to do any real parsing, so this might be a good interim solution.
Also, I prefer @mbaudrey's code in which convertVCFGenotype is a function in the variants module.

As an update to this, I've been playing with the pysam Tabix APIs. It turns out that there are two different interfaces, demoed here:
"""
Code to demo working with pysam VCF.
"""
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

import sys

import pysam

def main():
    vcfFile = sys.argv[1]
    # Option 1, use the VCF interface
    # This is in https://github.com/pysam-developers/pysam/blob/master/pysam/cvcf.pyx
    vcf = pysam.VCF()
    vcf.connect(vcfFile)
    print("samples = ", vcf.getsamples())
    print("header = ")
    for k, v in vcf.getinfo().items():
        print("\t", k, "->", v)
    for record in vcf.fetch(b"1"):
        print(
            record.contig, record.pos, record.id, record.ref, record.alt,
            record.qual, record.filter, sep="\t")
        # This causes an error...
        # info = record.info
        for sample in record.samples:
            print("\t", sample, "\t", record[sample])

    # Option 2, use the asVCF interface.
    # This is in https://github.com/pysam-developers/pysam/blob/master/pysam/TabProxies.pyx
    tabixFile = pysam.TabixFile(vcfFile, parser=pysam.asVCF())
    for record in tabixFile.fetch(b"1"):
        print(
            record.contig, record.pos, record.id, record.ref, record.alt,
            record.qual, record.filter, record.info, sep="\t")
        for index, sample in enumerate(vcf.getsamples()):
            print("\t", sample, "\t", record[index])

if __name__ == "__main__":
    main()

The first approach is a very complete validating VCF parser, which doesn't seem to be documented or fully maintained. The second is a simple position based approach, which will at least save us the trouble of breaking the records.
As a pragmatic interim approach while @bioinformed's htslib code comes on line, I'd suggest we use a mixture of these two interfaces to the get the information we need. We can get the header information from the first interface, and use the second to get the rows. We'll need to parse the INFO and sample strings by ourselves, that shouldn't be too bad for now.
@bioinformed, @AndreasHeger, it would be great to get your input here. How does this look to you? Is this a reasonable summary of where VCF support is in pysam?

Thanks, your summary is correct.
Option 2 (pysam.Tabixfile(fn).fetch(asVCF()) is the tabix interface. It
provides random access via genomic coordinates through tabix and lazy
parsing of columns within a row to avoid parsing samples 1-100 if only
sample 101 is required. It is however unaware of the structure of the
sample columns and ignores any information in the header of the VCF
file. Both are returned as uninterpreted text. This is a fairly thin
layer on top of tabix and fully supported.
Option 1 (pysam.VCF() is a full parser of the VCF format contributed
by Gerton. It uses Option 2 for random access and lazy parsing, but
is fully aware of the header and sample structure. It is quite
sophisticated in trying to work with VCF files of different versions
and those that are slightly malformatted. I believe it is quite feature
complete. Adding it to pysam was experimental and it has not received
as much attention as the samtools/tabix components.
Kevin is adding a layer to the bcf/vcf part of htslib which will be the
main supported API to interact with bcf/vcf files. He mentioned it
would soon be ready. We were discussing whether to wait for htslib 1.2
before releasing the next pysam version (0.8.2).
I am not sure how deep htslib goes into the interpretation of the header
or sample column data. Is this something that pysam will need to keep?
Best wishes,
Andreas
On 02/03/15 12:22, Jerome Kelleher wrote:

As an update to this, I've been playing with the pysam Tabix APIs. It
turns out that there are two different interfaces, demoed here:
"""
Code to demo working with pysam VCF.
"""
from  __future__import  division
from  __future__import  print_function
from  __future__import  unicode_literals
import  sys
import  pysam
def  main():
vcfFile=  sys.argv[1]
# Option 1, use the VCF interface
# This is in https://github.com/pysam-developers/pysam/blob/master/pysam/cvcf.pyx
vcf=  pysam.VCF()
vcf.connect(vcfFile)
print("samples =", vcf.getsamples())
print("header =")
for  k, vin  vcf.getinfo().items():
print("\t", k,"->", v)
for  recordin  vcf.fetch(b"1"):
print(
record.contig, record.pos, record.id, record.ref, record.alt,
record.qual, record.filter, sep="\t")
# This causes an error...
# info = record.info
for  samplein  record.samples:
print("\t", sample,"\t", record[sample])
 # Option 2, use the asVCF interface.
 # This is in https://github.com/pysam-developers/pysam/blob/master/pysam/TabProxies.pyx
 tabixFile=  pysam.TabixFile(vcfFile,parser=pysam.asVCF())
 for  recordin  tabixFile.fetch(b"1"):
     print(
         record.contig, record.pos, record.id, record.ref, record.alt,
         record.qual, record.filter, record.info, sep="\t")
     for  index, samplein  enumerate(vcf.getsamples()):
         print("\t", sample,"\t", record[index])

if  name  ==  "main":
main()
The first approach is a very complete validating VCF parser, which
doesn't seem to be documented or fully maintained. The second is a
simple position based approach, which will at least save us the trouble
of breaking the records.
As a pragmatic interim approach while @bioinformed
https://github.com/bioinformed's htslib code comes on line, I'd
suggest we use a mixture of these two interfaces to the get the
information we need. We can get the header information from the first
interface, and use the second to get the rows. We'll need to parse the
INFO and sample strings by ourselves, that shouldn't be too bad for now.
@bioinformed https://github.com/bioinformed, @AndreasHeger
https://github.com/AndreasHeger, it would be great to get your input
here. How does this look to you? Is this a reasonable summary of where
VCF support is in pysam?
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/server/pull/122#issuecomment-72641597.

The new htslib VCF/BCF API support for pysam is currently hosted in this
branch https://github.com/bioinformed/pysam/tree/bcf.  I've adapted the
above code to the new API, which allows access to headers, records, indexed
queries, and parses subsets of samples (if requested).
from  future import divisionfrom  future import
print_functionfrom  future import unicode_literals
import sysfrom pysam import BCFFile
def main():
# This is https://github.com/bioinformed/pysam/blob/bcf/pysam/cbcf.pyx
# Works on both VCF and BCF files
vars = BCFFile(sys.argv[1])
# Full header information is available
print('samples = ', list(vars.header.samples))

print("INFO headers =")
for k,v in vars.header.info.iteritems():
    print('\tINFO {} -> {}'.format(k, v))

# Return results only for the first sample
# Must be called prior to fetching records
vars.subset_samples([ vars.header.samples[0] ])

for record in vars.fetch(b"1"):
    print(record.contig, record.pos, record.id or '.', record.ref,

record.alts,
record.qual, ','.join(record.filter) or '.', sep="\t")
    for sample, geno in record.genos.iteritems():
        print('\t\tsample =', sample)
        for k, v in geno.iteritems():
            print('\t\t{} -> {}'.format(k, v))

if name == 'main':
main()
On Wed, Feb 4, 2015 at 4:48 AM, Andreas Heger<EMAIL_ADDRESS>wrote:

Thanks, your summary is correct.
Option 2 (pysam.Tabixfile(fn).fetch(asVCF()) is the tabix interface. It
provides random access via genomic coordinates through tabix and lazy
parsing of columns within a row to avoid parsing samples 1-100 if only
sample 101 is required. It is however unaware of the structure of the
sample columns and ignores any information in the header of the VCF file.
Both are returned as uninterpreted text. This is a fairly thin
layer on top of tabix and fully supported.
Option 1 (pysam.VCF() is a full parser of the VCF format contributed
by Gerton. It uses Option 2 for random access and lazy parsing, but
is fully aware of the header and sample structure. It is quite
sophisticated in trying to work with VCF files of different versions
and those that are slightly malformatted. I believe it is quite feature
complete. Adding it to pysam was experimental and it has not received
as much attention as the samtools/tabix components.
Kevin is adding a layer to the bcf/vcf part of htslib which will be the
main supported API to interact with bcf/vcf files. He mentioned it
would soon be ready. We were discussing whether to wait for htslib 1.2
before releasing the next pysam version (0.8.2).
I am not sure how deep htslib goes into the interpretation of the header
or sample column data. Is this something that pysam will need to keep?
Best wishes,
Andreas
On 02/03/15 12:22, Jerome Kelleher wrote:

As an update to this, I've been playing with the pysam Tabix APIs. It
turns out that there are two different interfaces, demoed here:
"""
Code to demo working with pysam VCF.
"""
from  __future__import  division
from  __future__import  print_function
from  __future__import  unicode_literals
import  sys
import  pysam
def  main():
vcfFile=  sys.argv[1]
# Option 1, use the VCF interface
# This is in https://github.com/pysam-developers/pysam/blob/master/
pysam/cvcf.pyx
vcf=  pysam.VCF()
vcf.connect(vcfFile)
print("samples =", vcf.getsamples())
print("header =")
for  k, vin  vcf.getinfo().items():
print("\t", k,"->", v)
for  recordin  vcf.fetch(b"1"):
print(
record.contig, record.pos, record.id, record.ref,
record.alt,
record.qual, record.filter, sep="\t")
# This causes an error...
# info = record.info
for  samplein  record.samples:
print("\t", sample,"\t", record[sample])
 # Option 2, use the asVCF interface.
 # This is in https://github.com/pysam-developers/pysam/blob/master/

pysam/TabProxies.pyx
tabixFile=  pysam.TabixFile(vcfFile,parser=pysam.asVCF())
for  recordin  tabixFile.fetch(b"1"):
print(
record.contig, record.pos, record.id, record.ref,
record.alt,
record.qual, record.filter, record.info, sep="\t")
for  index, samplein  enumerate(vcf.getsamples()):
print("\t", sample,"\t", record[index])
if  name  ==  "main":
main()
The first approach is a very complete validating VCF parser, which
doesn't seem to be documented or fully maintained. The second is a
simple position based approach, which will at least save us the trouble
of breaking the records.
As a pragmatic interim approach while @bioinformed
https://github.com/bioinformed's htslib code comes on line, I'd
suggest we use a mixture of these two interfaces to the get the
information we need. We can get the header information from the first
interface, and use the second to get the rows. We'll need to parse the
INFO and sample strings by ourselves, that shouldn't be too bad for now.
@bioinformed https://github.com/bioinformed, @AndreasHeger
https://github.com/AndreasHeger, it would be great to get your input
here. How does this look to you? Is this a reasonable summary of where
VCF support is in pysam?
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/server/pull/122#issuecomment-72641597.

This htslib API looks perfect - we get all of the information we need in an easy to use manner. When can we expect this in a released version of pysam? Write support isn't critical, but it would be nice to have. (I'm not sure if it was Kevin or Andreas sent this message - github says it's Andreas, but I got the email from Kevin)

I sent the last message with the adapted code sample.  I'm not sure why it
came out garbled.  Anyhow, I'm working on a minor API update and writing
some additional documentation in preparation for an alpha release.  Since
the code requires some fixes only present in htslib 1.2, Andreas and I are
discussing whether to merge the new API before or after the impending pysam
0.8.2 release.
-Kevin
On Wed, Feb 4, 2015 at 9:01 AM, Jerome Kelleher<EMAIL_ADDRESS>wrote:

This htslib API looks perfect - we get all of the information we need in
an easy to use manner. When can we expect this in a released version of
pysam? Write support isn't critical, but it would be nice to have. (I'm not
sure if it was Kevin or Andreas sent this message - github says it's
Andreas, but I got the email from Kevin)
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/server/pull/122#issuecomment-72858533.

I work with @kerrydc and know that he's gone on vacation. I'm going to close this PR for now so it doesn't block any other requests. I think it's reasonable to reopen this in the future when kerry gets back and build this on top of whatever is on master at that point.

Hi @jeromekelleher and all, we have just released pysam 0.8.2. This release includes @bioinformed 's bcf/vcf htslib API.
Please let us know of any comments and suggestions.
Best wishes,
Andreas

This is great news, thanks @AndreasHeger!


Improve the paraconf wrapper with support for default values

By adding support for default values, many try/catch could be avoided

closed


Update ReadME

Add FAQ Section
Update next version
Update link on current version

The new version number follows. On my Azure Machine is the setup process still running

Thank you :)

No Problem :)


Hong

Summary

Context

This is the third time you've submitted inappropriate PRs. Please refrain from repeating the same mistake in the future.
Also Cc @mmistakes (for increased attention).

@iBug I'd be curious to know what is the root cause for all these mistaken pull requests so it could potentially be noted in the documentation.
The repo has gone from the occassional mistaken pull request every couple of weeks or months, to one or more a day. This tells me there's some sort of tutorial, blog post, or other documentation out there inproperly instructing users how to fork this theme and then push up changes to GitHub.
I haven't been able to gleam any meaningful insights from referral traffic either to try and determine where said post or tutorial may be coming from.

I do know the theme is popular with academia using it as a research blog so my guess is there's a popular MM site out there or fork similar to academicpages that if identified could be updated with better PR documentation.

I've always assumed that GitHub's big green button after you push commits onto a forked repo is the culprit. It's so tempting to click on such an outstanding button that people just did so. Unfortunately this is some GitHub setting that we aren't up for to change it.
Possible remediations I've thought of include putting a noticeable "Getting Started" at the top of README, to instruct visitors to use the other "starter repo" instead of starting off forking this one. Hopefully this will reduce "personal forks" that aim at building their own websites instead of contributing to the theme.

And yes, we could add some more documentation to the theme and handling PRs. But judging by how little users read the other documentation it's likely wasted effort. By far the most popular method of using the theme is forking the repo instead of using the starter... and that mostly comes from blog posts and other users tell others that's how you install the theme... which isn't ideal.


nan loss while training

I have been trying to train a faster r-cnn network on another dataset and have run into issues with nan training loss. I have been trying to track down the nan loss and it seems to be coming from the bbox_tranform_batch function in bbox_transform.py.
Specifically, I have noticed, that these two are the first to become nan when the training begins
 targets_dw = torch.log(gt_widths / ex_widths)
 targets_dh = torch.log(gt_heights / ex_heights)

Can somebody think of a possible reason as to how this can fixed?

I faced the same issue while training on my own dataset which is similar to pascal voc. You need to modify these four lines while reading the ground truth according to your dataset. Check whether you need to remove or keep the +1.

Hi, I printed the values for gt_widths, gt_heights, ex_widths and ex_heights. None of them are zeros. Could it be a problem with torch.log described here
Edit: I tried experimenting with different images and I noticed that sometimes, all values in the tensor ex_widths are positive and all values in the tensor gt_widths are negative, which means that the log of negative values is being calculated. Any idea on why this could be happening?
What are the correct signs of gt_widths and ex_widths?

Did you solve the problem. I meet the same error. I need your help . thanks.

@zhengxinvip we did solve the problem.
Turns out, we were reading our annotations wrong in our code.
This code needs to have the coordinates of the top left and bottom right corners of the bounding boxes, so you might have to compensate for that if your annotations are not in that format.
Our annotations were in the form of  [l t w h]  and this is how we solved our problem.

check new comment from #594

I have been trying to train a faster r-cnn network on another dataset and have run into issues with nan training loss. I have been trying to track down the nan loss and it seems to be coming from the bbox_tranform_batch function in bbox_transform.py.
Specifically, I have noticed, that these two are the first to become nan when the training begins
 targets_dw = torch.log(gt_widths / ex_widths)
 targets_dh = torch.log(gt_heights / ex_heights)

Can somebody think of a possible reason as to how this can fixed?

how did you track down the nan loss??


showing singles regardless of showSingles param

singles are currently being drawn regardless of the showSingles value.

I should learn to read...
//     showSingles:    Boolean?
//         Optional. Whether or graphics should be displayed when a cluster graphic is clicked. Default is true.
this is a non-issue

if you could spare the time to submit a PR with any pearls of wisdom to clarify the points that initially confused you, i can make sure it lands.

This issue was totally on me. Not sure it needs any further clarification.


Display auto-import status in the status bar

There's a significant delay before auto-imports are ready in a TypeScript project after a file is opened for the first time, and probably every time the typescript server starts/restarts and a file is opened. I'm not sure if that's caused by the size of a project (about 30k lines of code), but that doesn't really matter.
What bothers me is that there's no way (or I haven't find a way) to tell when is VS Code ready to provide auto-import suggestions. So if I want to add a missing import, I have no other way to find out when the auto-import is ready other than repeatedly press ctrl + . and esc. See the attached GIF.
It would be really nice to have an indicator in the status bar which displays whether the use of auto-imports is ready or not yet.

We show:

When the js/ts service is starting or loading a new project

Yes, however, it doesn't display in the case I'm describing. If you modify a file outside of VS Code, then open that file in VS Code and try to add a missing import using ctrl + ., no status indicator is displayed and the import is not available until a few seconds later.

It also doesn't show the status in some other cases.
For a concrete example, I had a file open in the editor for quite some time, I added a ListItem from material-ui, I hit Ctrl+. to get the import suggestions, but there were't any. So I waited a few seconds, and then tried again - and they appeared, so I used them. Right after that, I added a ListItemText and the situation repeats. There is no status displayed anywhere for when these auto-imports are ready.
Please reopen @mjbvz


[Create New Issue]: Cyclically rotate an array in C++

Description
Description C/CPP
Domain
C/CPP
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign


Update jsDelivr links

jsDelivr switched to a fully automated system, that can serve files from npm and GitHub. This means all future releases will be available automatically, but will use a new link structure.
I updated the links now so you don't forget to do it when you release a new version.
You can find links for all files at https://www.jsdelivr.com/package/npm/natsort.
Feel free to ping me if you have any questions regarding this change.

Coverage remained the same at 88.571% when pulling 6e8dbfc6a5de9cb7a9b2bf9b8ccddb86f5496a86 on LukasDrgon:patch-1 into c248db6146b025b7d4526ec857b41527e0e7b8c8 on bubkoo:master.


Create per-stream setting for whether "resolve topic" is enabled

Some organizations may prefer not to use the "resolve topic" feature, or only use it in certain streams, and it can be hard to communicate these norms to users. We should therefore make it possible for organization to specify whether "resolve topic" is enabled in any given stream.
We can do this in the following way:

Under Organization settings > Other settings, add a dropdown setting: "Who can resolve topics (by default)?" (or some wording along those lines) We will need a question mark to explain what this means. Options:

Admins, moderators, members and guests
Admins, moderators and members [default]
Admins, moderators and full members
Admins and moderators
Admins only
Nobody

In the "Create stream" menu, add a dropdown below the message retention policy with the section labeled "Who can resolve topics?". Options: "Use organization default ()" [default],  plus the same options as above. We will need a question mark to explain what this means.
In the "General" tab of the stream management menu, "Who can resolve topics?" should be added to "Stream permissions", with the same options as in (2) above.

This setting should presumably use the new groups system defined in #19525.


Query timing changes data reported

The 5-min offset for data retrieval, combined with only hourly updates causes RPi-Monitor to interpolate the Status values, which makes decoding the flags very difficult.
Not sure how long the EBAM takes to generate an hourly report, but probably <5min.
To complicate data retrieval timing, the EBAM clock drifts independently of the Pi and needs to be accomodated.

Timing is hard here: to obtain "correct" timestamps, data must be obtained from EBAM, written to file, and queried by RPi-Monitor within the first minute of each hour.
To consider:

unknown synchronicity between EBAM & RPi clocks
unknown data record generation time of EBAM
query rate/offset of the RPi-Monitor service

Current plan:

establish new cron-based service to keep EBAM clock synced to Pi
change hourly DAQ script to query for new records, not just the last record

launch at top of each hour (hopefully to finish within first minute)
retry several times until new record obtained

(possibly) change RPi-Monitor query rate for EBAM data 10s → 1min?

may require 10s query rate to prevent 0:00-data from getting 0:01 timestamp
may require 60s query rate to prevent inter-minute averaging of values (which then become nonsense values)

Examples of the timing mismatch producing nonsense values: -32.195 in this case, where it should be either 4 or -99. (XREF https://github.com/XavierBerger/RPi-Monitor/issues/276)

unclear why status isn't updated until after first minute of hour (EBAM or python script?)

cron currently triggered at minute 1, not minute 0 - will fix!

interpolation at point 10:01:10 is not linear between adjacent ports (why?)
note RPi-Monitor query timing is aligned to zero-seconds (e.g. :00, :10, :20, :30, ...)

assuming EBAM & computer have same clock time, it's not possible to keep RPi-Monitor from including previous data in new minute's average (:00 will always hold old data)

unclear why status isn't updated until after first minute of hour (EBAM or python script?)

cron currently triggered at minute 1, not minute 0 - will fix!

Calling data script at minute 0 instead of 1, with 5 second query delay, does not produce latest data record because of EBAM clock offset (10:59:49 vs 11:00:05 = 16sec behind):
lar@ebam-plus:~/ebam-plus $ ls -l /tmp
total 16
-rw-r--r-- 1 root root  199 Jul 12 11:00 ebam.csv

tail -n 10 /home/lar/ebam-plus/src/getbam.log

Data Report
2019-07-12 10:59:49
Location, 3, W15096

Time,ConcRT(ug/m3),ConcHR(ug/m3),Flow(lpm),WS(m/s),WD(Deg),AT(C),RH(%),BP(mmHg),FT(C),FRH(%),BV(V),PM,Status
2019-07-12 10:00:00,+031663,+099999,+11.6,00.0,000,+024.5,041,694,+033.2,017,12.5,0,00512
4
gooddata = 1
New hour saved successfully.

Best (?) solution: set EBAM clock to intentionally run ~10sec ahead of RPi, so that data record is ready by start of new minute (xref #9)

prevent :00 values from holding-over old data
maintains sufficient time-synchronicity (i.e. constant 10s lead time is totally OK for hourly data sets)
does not pollute :40 or :50 values because data queried at most minutely

Still seeing strange averaging after start of each hour:

Try setting EBAM 30sec ahead of the Pi, and test RPi-Monitor at both 30s and 60s query rates.


Exception: Error writing to socket: Broken pipe (Errno) on Heroku

27 days ago without errors :tada: . Closed.


Split unit tests of tag functions

I agree the level of these tests is not ideal, I can work on this in another PR.


post url 404 after publishing

Issue Summary
After publishing a new post and clicking the card of published post, it ran into 404

Then I refresh the page, the post occurred.

But!!! when I refresh the page again, it run into 404 again too!

My site is self hosted with ghost, and to explore this issue, I did some test and found that the issue only comes out when using nginx, but I have no idea why yet.
Steps to Reproduce

install ghost image by docker
deploy ghost with k8s and use nginx ro do proxying
enter background and publish a new post
click the post's card and there is 404

Ghost Version
5.22
Node.js Version
16.13.0
How did you install Ghost?
from dockerhub
Database type
MySQL 8
Browser & OS version
chrome win10
Relevant log / error output
nginx logs when 404：
clicking post's card after publishing:
"GET /post-title-1/ HTTP/1.1" 404 2245 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36”
refresh the post's page:
"GET /post-title-1/ HTTP/1.1" 404 2245 "http://<IP_ADDRESS>/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"

Code of Conduct

[X] I agree to be friendly and polite to people in this repository

I found that 404 shows is because I’m using k8s development with several pods. When the pod used to fetch new post is the same with the one used to publish the new post, 200 returned; otherwise there will be 404.
I still don't known why the pods don't return the same result. Is there any reason with ghost's api or cache?

Sounds like you need to read this: https://ghost.org/docs/faq/clustering-sharding-multi-server/

Sounds like you need to read this: https://ghost.org/docs/faq/clustering-sharding-multi-server/

Soooo many thx !!! And could U please tell why multi-server is not supposed? Is this because the frontstage pages generated with handlebars are cached and will be update only after posts change (.e.g, publish a new post)?
I looked at the log of mysql, when I refresh the home page, there will be a sql to select posts. But in my thought if there is a cache, this sql log won't be there. I'm confused and eager the explaination :)


tf 1.15.0 and tensor2tensor on raspberry pi -- get_data_files problem

[Required] Your device (RaspberryPi3, LaptopPC, or other device name):  raspberry pi
[Required] Your device's CPU architecture (armv7l, x86_64, or other architecture name):  armv7l
[Required] Your OS (Raspbian, Ubuntu1604, or other os name):  raspbian
[Required] Details of the work you did before the problem occurred:  designed a t2t model
[Required] Error message:  see below
[Required] Overview of problems and questions:
I am seeing this error. I'm using your tf 1.15.0 for armhf and python3.7. Ultimately I will be using raspbian and buster. I also use tensor2tensor and this is where I see my error now.
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/bin/t2t_decoder.py", line 209, in main
decode(estimator, hp, decode_hp)
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/bin/t2t_decoder.py", line 110, in decode
checkpoint_path=FLAGS.checkpoint_path)
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/utils/decoding.py", line 227, in decode_from_dataset
checkpoint_path=checkpoint_path)
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/utils/decoding.py", line 316, in decode_once
for num_predictions, prediction in enumerate(predictions):
File "/usr/local/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 620, in predict
input_fn, ModeKeys.PREDICT)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 996, in _get_features_from_input_fn
result = self._call_input_fn(input_fn, mode)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1116, in _call_input_fn
return input_fn(**kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/data_generators/problem.py", line 803, in estimator_input_fn
dataset_kwargs=dataset_kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/data_generators/problem.py", line 888, in input_fn
self.dataset(*dataset_kwargs),
File "/usr/local/lib/python3.7/dist-packages/tensor2tensor/data_generators/problem.py", line 653, in dataset
contrib.slim().parallel_reader.get_data_files(data_filepattern))
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/contrib/slim/python/slim/data/parallel_reader.py", line 316, in get_data_files
raise ValueError('No data files found in %s' % (data_sources,))
ValueError: No data files found in /app/transformer/../data/t2t_data/chat_movie_30/chat_line_problem-dev
The error says that the 'dev' files don't exist but in fact they do. I don't know how to fix this. In the printout above the error shows up in the test phase. I can run the t2t code in the train phase and the code will save the data files but later when it comes time to read them it crashes again.

This is an issue that has nothing to do with the installer of this repository, just ask StackOverflow. Maybe the executing user doesn't have access to the file.

You may be right but I think if this were the underlying problem the error message would be 'Permission Denied', not 'ValueError: No data files found'. Do you have any other clues?

Close this if you want. I have not found a solution but I take your point. Thank you for your time.


Add modular setting to encoder

Add setting to the encoder to force modular or vardct mode, or use the
default (where in future implementations, encoder is allowed to e.g. use
heuristics on pixel content to choose this)
Also lets the SetDistance option set the quality_pair setting for
modular too. As dicussed with Jon, it's ok to translate distance to
a quality_pair value and expose only one setting in the API, rather than
expose the distance setting for vardct, and the quality pair setting for
modular, separately. The conversion formulas come from cjxl.cc and can
be tweaked if necessary.
Also alters the documentation of the SetLossless setting to specify what
this setting actually does as implemented, and do not be as strict in the
documentation which settings exactly this overrides since this can be
future implementation dependent.

Updated the comment, PTAL


Switch "Dev" to SLE Micro 6

All ALP repositories are removed and Dev and Staging are building against SUSE:ALP:Products:Marble:6.0


scale_openshift_wrapper should expose parameters to target machineset from different labels

Currently, trigger_scale.py hard-codes the machineset labels used for the scale task.
As a tester, I need to be able to target certain machinesets for certain workload tests, and would like to configure a single variable that influences this behavior.
Would require similar changes through benchmark-operator and perhaps higher up the stack.
Hard-coded labels:
https://github.com/cloud-bulldozer/benchmark-wrapper/blob/master/snafu/scale_openshift_wrapper/trigger_scale.py#L181-L184
https://github.com/cloud-bulldozer/benchmark-wrapper/blob/master/snafu/scale_openshift_wrapper/trigger_scale.py#L219-L222
https://github.com/cloud-bulldozer/benchmark-wrapper/blob/master/snafu/scale_openshift_wrapper/trigger_scale.py#L305
https://github.com/cloud-bulldozer/benchmark-wrapper/blob/master/snafu/scale_openshift_wrapper/trigger_scale.py#L319

Not relevant


✨ Change MotorGroup Output Velocity

Overview
Add the option to change the output velocity of the MotorGroup class
Motivation
The only way the output velocity of a motor group would change during runtime is because of a multi-speed transmission. While this mechanism is rare, it has the potential to make an appearance during the late season of the 2024-2025 VRC game High Stakes. The motor group class should automatically handle this.
Implementation
A setOutputVelocity method should be added to the MotorGroup class. This will change the value of the maximum output velocity member variable. Offsets may need to be applied to keep the reported angle the same as the angle that was reported before the output velocity was changed.

Offsets may need to be applied to keep the reported angle the same as the angle that was reported before the output velocity was changed.
as far as i can tell, this is not needed


gk-deploy get stuck on glusterfs pod creation

When deploying glusterfs using ./gk-deploy -g -n kube-system it gets stuck on waiting GlusterFS pods to start
Using Kubernetes CLI.
Using namespace "kube-system".
Checking for pre-existing resources...
  GlusterFS pods ... not found.
  deploy-heketi pod ... not found.
  heketi pod ... not found.
  gluster-s3 pod ... not found.
Creating initial resources ... serviceaccount "heketi-service-account" created
clusterrolebinding "heketi-sa-view" created
clusterrolebinding "heketi-sa-view" labeled
OK
node "k8s-node-1" labeled
node "k8s-node-2" labeled
node "k8s-node-3" labeled
node "k8s-node-4" labeled
node "k8s-node-5" labeled
daemonset "glusterfs" created
Waiting for GlusterFS pods to start ...

Analysing the pods, I see pods in Crash state
$ kubectl get pods -n kube-system | grep gluster
glusterfs-8vtkp                              0/1       CrashLoopBackOff   6          7m
glusterfs-chfld                              0/1       CrashLoopBackOff   6          7m
glusterfs-k2jm4                              0/1       CrashLoopBackOff   6          7m
glusterfs-p28w2                              0/1       CrashLoopBackOff   6          7m
glusterfs-wlvh4                              0/1       CrashLoopBackOff   6          7m

Inspecting one of the pods I see it fails on create OCI container Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "exec: \"/dev/init\": stat /dev/init: no such file or directory": unknown
Events:
  Type     Reason                 Age                From                 Message
  ----     ------                 ----               ----                 -------
  Normal   Pulling                46s                kubelet, k8s-node-3  pulling image "gluster/gluster-centos:latest"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-logs"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-lvm"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-config"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-dev"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-heketi"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-cgroup"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-ssl"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-etc"
  Normal   SuccessfulMountVolume  46s (x2 over 46s)  kubelet, k8s-node-3  (combined from similar events): MountVolume.SetUp succeeded for volume "default-token-bl9pp"
  Normal   SuccessfulMountVolume  46s                kubelet, k8s-node-3  MountVolume.SetUp succeeded for volume "glusterfs-misc"
  Normal   Pulled                 36s                kubelet, k8s-node-3  Successfully pulled image "gluster/gluster-centos:latest"
  Warning  Failed                 35s (x2 over 36s)  kubelet, k8s-node-3  Error: failed to start container "glusterfs": Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "exec: \"/dev/init\": stat /dev/init: no such file or directory": unknown
  Warning  BackOff                31s                kubelet, k8s-node-3  Back-off restarting failed container

Is there any work around for this?

I have this issue as well. I even tried just directly loading the glusterfs-daemonset.yaml and .json - backoff failures in both regards. It appears that it is using a coreos image even though I am running on ARM.

Is there an ARM image for Gluster i can pull?

@tgonzaga Sorry no one got back to you on this. If you're still interested, let us know if you still have the issue or if you found a workaround.
@azwildfire There is no ARM image for Gluster. All of our images are x86_64.

@jarrpa any roadmap or hopes of getting one? I am new to containers so this is part education and part experimentation. Could I build my own docker image and use it with the gluster yaml daemon?

@azwildfire No plans on our part to do so, no. But yes, you can certainly build your own container to do with as you like. :) You could use the CentOS or Fedora container definitions here as a base: https://github.com/gluster/gluster-containers


Typo

Just a little typo in the README : Frequently used extensOIns in RxSwift
BTW, nice project 👍

Thanks! I fixed it.
but I think there are many grammars and typos error.
If you find it, could you make a PR for me?


Subscribe callback is invoked twice per subscription after reconnection

Doing a subscribe() upon ready causes the subscribe callback to be invoked twice instead of once after a reconnection. However, doing subscribe() upon connect works as expected - the callback is invoked only once. What's the deal? Am I doing something wrong?
Code snippet:
var redisSubscriber = redis.createClient(config.port, config.host);
redisSubscriber.auth(config.pass);
redisSubscriber.on('error', function (err) {
	console.error("[redis-subscriber] " + err);
}).on('connect', function () {
	console.log("[redis-subscriber] Connected");
	redisSubscriber.subscribe('channel1');
}).on('close', function (why) {
	console.log("[redis-subscriber] " + why);
}).on('ready', function () {
	// redisSubscriber.subscribe('channel1');
}).on('subscribe', function (channel, count) {
	console.log("[redis-subscriber] Subscribed to %s, %d total subscriptions", channel, count);
});

Output:
[redis-client] Connected
[redis-subscriber] Connected
[redis-subscriber] Subscribed to channel1, 1 total subscriptions
[redis-client] Error: Redis connection to <IP_ADDRESS>:6379 failed - connect ECONNREFUSED
[redis-subscriber] Error: Redis connection to <IP_ADDRESS>:6379 failed - connect ECONNREFUSED
[redis-client] Error: Redis connection to <IP_ADDRESS>:6379 failed - connect ECONNREFUSED
[redis-subscriber] Error: Redis connection to <IP_ADDRESS>:6379 failed - connect ECONNREFUSED
...
[redis-client] Connected
[redis-subscriber] Connected
[redis-subscriber] Subscribed to channel1, 1 total subscriptions
[redis-subscriber] Subscribed to channel1, 1 total subscriptions

@jweyrich sorry for the closed issue without ever receiving a response.
The re-subscription is intended but you're able to disable that with the disable_resubscribing option in newer versions.


chore: allow newer python-dateutil versions

Thanks for the report @adrienbrault . We'll take a look and see if we can incorporate in the next minor version of the SDK.


Draft report

Please put a copy of images you want to put in the report under paper/images. Thanks!

@ay2456 For the method part, should I commit that part to you under the comment in this pull request or should I first merge this one and open a new pull request?


[NuGet.org Bug]: v3 API not available from Russia

Impact
I'm unable to use NuGet.org
Describe the bug
I've been unable to use dotnet restore, dotnet new install and other features. When enabling verbose output, it seems like my machine cannot access https://api.nuget.org/v3/index.json. I'm able to use it, albeit very slowly, through a VPN. Last time I've used .NET (a couple of years ago) this problem wasn't there.
Weirdly enough, https://www.nuget.org/api/v2/ seems functional and provides output in browser.
Repro Steps

Use a Russian IP (Beeline ISP)
Try to use dotnet restore on any project
Wait

Expected Behavior
Functional NuGet API
Screenshots
When using VPN:

Without VPN:

Additional Context and logs
No response

Magically resolved, no idea how and why


Implement Merge Sort (Python)

Implement Merge Sort in Python
We need a simple merge sort algorithm written in Python for our repository.
This issue is great for beginners looking to contribute in open source. If you wish to fix this issue please comment that you are working on it. Please read the readme.md and contributing.md files to learn how to contribute.
If you wish to contribute another algorithm or the same algorithm in a different language, please see if it hasn't been implemented and issue a new request if no issue has been made.
If you are going to work on the issue yourself, make sure you comment on the issue to let others know.

I would like solve this and want this task to be assigned to me.

@aayushsinha44 All right. I will leave it to you.

@prkhrv No problem man. Happens to the best of us.

Can I work on this?


Clarify permissions required for mc admin update

On the mc admin update reference page, this paragraph says the command may require sudo:
After running the command, a prompt displays to confirm the update.
Type ``y`` and ``[ENTER]`` to confirm and proceed with the update.
You may need to use ``sudo`` if your user does not have write permissions for the path where ``mc`` is installed.

mc admin update itself doesn't inherently require sudo. But the target location for the binary needs to be writable by the user executing the command. The user must be able to write to the path to stage a copy of the new binary before replacing the existing one.
Presumably humans aren't logging in as the user that runs the minio process or owns the install location. Maybe make it writable for an appropriate group? We should clarify with engineering what makes sense to recommend here.
includes/linux/steps-upgrade-minio-deployment.rst mentions the user running the minio process needs r/w on the binary, which isn't the same thing. Might be a good place to clarify, along with other text that references (or should reference) upgrading.
Related:
includes/minio-server-installation.rst mentions mc admin update (about source-based installations) but this include appears to not be used anywhere.

We don't want to tell people to be using sudo.


[doc] Fix spelling errors

This PR fixes spelling errors in the documentation. The errors were found by codespell.
I have a second commit locally, updating the documentation headers of

scripts/cmake/vcpkg_build_cmake.cmake
scripts/cmake/vcpkg_build_make.cmake
scripts/cmake/vcpkg_build_nmake.cmake
scripts/cmake/vcpkg_copy_tools.cmake

This needs to be merged, too, but I guess it will give a very long CI run. So I'm holding this back until explicitly invited, and this PR in draft state.

The CI errors reflects the missing changes to scripts/cmake/. As I said, I can push these changes when it is okay for CI load.

The CI errors reflects the missing changes to scripts/cmake/

Plus one missing change to ports/vcpkg-cmake/vcpkg_cmake_configure.md. Now included in the pending commit.

Pending commit: https://github.com/dg0yt/vcpkg/pull/2/files

Please apply this patch to your branch.

Please apply this patch to your branch.

Negative. The diff restores the spelling errors.
As I already wrote in my comments: what is needed is https://github.com/dg0yt/vcpkg/pull/2/files. I need to know if it is okay to let the pending commit hit CI because it will trigger a rebuild of many packages (without changes to the binary result).
(Before that, a native speaker might verify that the changes are valid, to avoid another expensive CI run.)

@dg0yt Some changes need you to modify the cmake file in scripts/cmake.

@dg0yt Some changes need you to modify the cmake file in scripts/cmake.

Yes, and I named them.
Well, I will just push that commit. Let's see how it rebuilds 1500 ports.

Oh no. Missing port version update.

Waiting for https://github.com/microsoft/vcpkg/pull/17628.

Please re-run the failed jobs. CI was stuck at some point.

/azp run

@dg0yt Because the modification involves several key functions or ports, and they have led to the rebuild of almost all ports.

"Only" documentation changes, but rebuilding the world and hitting the CI wall...
Isn't it possible to just re-run the failed jobs? Or shall I split the PR?

All the files in $/scripts/cmake are part of the base hash in the binary cache, so any change in those files is world-invalidating.

All the files in $/scripts/cmake are part of the base hash in the binary cache, so any change in those files is world-invalidating.

This is why I was asking for coordination already when opening the PR ;-)
And that's why I keep asking to re-run the failed jobs only. At least in standard GH/AZP integration, this is a just a push of a button.

Thanks @dg0yt !


BENCH-3686: Rename RStudio to R Analysis Environment

Planning to merge this PR next week once the UI change lands next week. https://github.com/verily-src/terra-ui/pull/1847

Update: Instead of deleting the existing rstudio dir, I will leave it up and remove it separately once the UI renaming lands in prod.


Show mini drawer on start

I want to open the mini drawer on start app, therefore I called result.openDrawer(). It works.
However, when I want to expand out the drawer to full view by swiping right, the drawer is stuck and won't expand.
May I know how to achieve this behaviour?
I tried to meddle with ICrossfader in order to try to solve above. I put some breakpoints in this sample code, it seems like the listener methods of ICrossfader is never called. I am not sure of how the ICrossfader() works but if I comment it out, there seems to be no difference in behaviour.
//define the crossfader to be used with the miniDrawer. This is required to be able to automatically toggle open / close
        miniResult.withCrossFader(new ICrossfader() {
            @Override
            public void crossfade() {
                boolean isFaded = isCrossfaded();
                crossfadeDrawerLayout.crossfade(400);

                //only close the drawer if we were already faded and want to close it now
                if (isFaded) {
                    result.getDrawerLayout().closeDrawer(GravityCompat.START);
                }
            }

            @Override
            public boolean isCrossfaded() {
                return crossfadeDrawerLayout.isCrossfaded();
            }
        });

fixed in v0.3.3


Forward ROCKSDB envvars through to build script legacy build

This PR should slightly reduce legacy bin build times in the build script when a system RocksDB lib is specified through the ROCKSDB_LIB_DIR environment variable.

Codecov Report

Merging #246 (a119067) into develop (c099fdf) will not change coverage.
The diff coverage is n/a.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@           Coverage Diff            @@
##           develop     #246   +/-   ##
========================================
  Coverage    33.56%   33.56%
========================================
  Files           24       24
  Lines         8213     8213
========================================
  Hits          2757     2757
  Misses        5456     5456

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Google Hangouts Chat - Clicking on links does not open them

Describe the bug

When I click on a link in a Google Hangouts chat, it does not open the link in my browser.
To Reproduce

Steps to reproduce the behavior:

Open Google Hangouts Chat service
Open a particular chat window
Click on any link shared in that chat.
See that it does not open the link in any browser

Expected behavior

On clicking a link, it should be opened in a browser installed in my machine.
Screenshots

In Ferdi

In Chrome

Desktop:

OS: Windows 10
Ferdi Version 5.5.0-gm.2

Server Used: [e.g. https://api.getferdi.com]

Debug information:
debugInformation.txt
Additional context

Sometimes even the right click does not work for the link as intended. The context menu appears and then disappears like a flicker. But I am not getting it always.

This issue is not stale, people are still experiencing it.

I seem to have run into this issue as well.  Test on Windows 10 and MacOS machine.  If you open the link via right-click -> open link, it works.


selectOnTab does not set focus to next field

Describe the bug
When using the option selectOnTab, the focus is not set to the next input. To jump to the next input, an additional 'tab' is necessary.
To Reproduce
The behavior could be seen on any TomSelect instance:

Create a simple TomSelect and set selectOnTab option to true
Click on a select input to open the dropdown
Select an option by using the up/down keys
Use the 'TAB' key to select the option
Result: the dropdown closes and the focus stays on the current input
One more 'TAB' sets the focus to the next input

Expected behavior
When I press TAB on an input, the focus should go to the next input because my selection is done. Except for a multi-select input, in this case, using TAB with selectOnTab enabled, the focus should stay on the input.
Selectize is also using this behavior, which is much more intuitive and user friendly. For single selects, the user already finalized his input with the TAB and will definitely go to the next step / the next input. So it's not necessary to require another TAB.
Additional context
The reason for this is the preventDefault(e) on line 739 in the onKeyDown method for KEY_TAB. I would suggest to check for a multi-select input and only call preventDefault(e) if we have one. Otherwise, do not call preventDefault(e) to jump to the next input.

I think that it's more an UX issue than a bug, but it sure is a bothersome one 😆
I'll try to check if the preventDefault is the only thing we need to change it


Fix revert panic

Codecov Report

Merging #1303 (264be42) into master (80a0018) will increase coverage by 0.02%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1303      +/-   ##
==========================================
+ Coverage   47.14%   47.17%   +0.02%
==========================================
  Files         203      203
  Lines       15867    15867
==========================================
+ Hits         7481     7485       +4
+ Misses       7399     7396       -3
+ Partials      987      986       -1

Impacted Files
Coverage Δ

catalog/rocks/entry_catalog.go
14.61% <0.00%> (ø)

catalog/mvcc/cataloger_create_repository.go
62.96% <0.00%> (ø)

pyramid/tier_fs.go
58.77% <0.00%> (+1.75%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80a0018...fd3dca9. Read the comment docs.

Codecov Report

Merging #1303 (264be42) into master (80a0018) will increase coverage by 0.02%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1303      +/-   ##
==========================================
+ Coverage   47.14%   47.17%   +0.02%
==========================================
  Files         203      203
  Lines       15867    15867
==========================================
+ Hits         7481     7485       +4
+ Misses       7399     7396       -3
+ Partials      987      986       -1

Impacted Files
Coverage Δ

catalog/rocks/entry_catalog.go
14.61% <0.00%> (ø)

catalog/mvcc/cataloger_create_repository.go
62.96% <0.00%> (ø)

pyramid/tier_fs.go
58.77% <0.00%> (+1.75%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80a0018...fd3dca9. Read the comment docs.


ModuleNotFoundError during Poetry Installation in Ubuntu 22.04

Hello there,
after forking and cloning of the repository, I was trying to install the dependencies for modification of the source code based on the README file. I was using a virtual machine with Ubuntu 22.04 LTS, but I got an error related to a missing module CacheControl. Probably it is related to Ubuntu 22.04 and the poetry installation regarding to this bug report. The fix would be to install CacheControl via sudo apt install python3-cachecontrol.
Maybe, this can be added as remark to the installation section in the README file?
Here is the traceback after executing of poetry install:
Traceback (most recent call last):
  File "/usr/bin/poetry", line 5, in <module>
    from poetry.console import main
  File "/usr/lib/python3/dist-packages/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/usr/lib/python3/dist-packages/poetry/console/application.py", line 7, in <module>
    from .commands.about import AboutCommand
  File "/usr/lib/python3/dist-packages/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/usr/lib/python3/dist-packages/poetry/console/commands/check.py", line 2, in <module>
    from poetry.factory import Factory
  File "/usr/lib/python3/dist-packages/poetry/factory.py", line 18, in <module>
    from .repositories.pypi_repository import PyPiRepository
  File "/usr/lib/python3/dist-packages/poetry/repositories/pypi_repository.py", line 11, in <module>
    from cachecontrol import CacheControl
ModuleNotFoundError: No module named 'cachecontrol'

Hopefully, I provided enough information, otherwise let me know. This is my first issue and participation in open source, so please feel free to give me any feedback.

Hi @FOehlschlaeger Friso, thanks for creating this issue, and welcome to open source 😄
I think you're right: it seems to be an issue with either Ubuntu 22.04 or Poetry. Thanks for the PR.


Fix bash completion for docker service update

Remove bash completion for docker service update --publish|-p
Before this PR, this option would be completed in both create and update.

nit: The name of _docker_service_update() is confusing because it is also related to non-update.

I agree. We also have this with _docker_container_run, which also serves create.
In both cases, a better solution would be to create _docker_a_and_b and delegate both docker_a and docker_b to it.
I'll create a PR for that.


[cppfs] Fix ssh dependendencies

Fixes #32273.
Fixes exported config.

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download
[x] The "supports" clause reflects platforms that may be fixed by this new version
[x] Any fixed CI baseline entries are removed from that file.
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

Tested usage successfully by cppfs:x64-windows:
cppfs provides CMake targets:

    # this is heuristically generated, and may not be correct
    find_package(cppfs CONFIG REQUIRED)
    target_link_libraries(main PRIVATE cppfs::cppfs)

Tested usage successfully by cppfs:x64-windows:
cppfs provides CMake targets:

    # this is heuristically generated, and may not be correct
    find_package(cppfs CONFIG REQUIRED)
    target_link_libraries(main PRIVATE cppfs::cppfs)

I hope you tested this for cppfs[ssh]. The exported config was incomplete with this PR.

Feature ssh tested successfully in the following triplet:

x86-windows
x64-windows
x64-windows-static

Tested usage successfully by cppfs[ssh]:x64-windows

Install:
PS F:\Feature-test\cppfs\vcpkg> ./vcpkg install cppfs[ssh]:x64-windows  --recurse
Computing installation plan...
The following packages will be rebuilt:
    cppfs[core,ssh]:x64-windows -> 1.3.0#3
The following packages will be built and installed:
  * libssh2[core,zlib]:x64-windows -> 1.11.0
  * openssl:x64-windows -> 3.1.1
  * zlib:x64-windows -> 1.2.13
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Restored 2 package(s) from C:\Users\v-zhli17\AppData\Local\vcpkg\archives in 756 ms. Use --debug to see more details.
Removing 1/5 cppfs:x64-windows
Elapsed time to handle cppfs:x64-windows: 47.8 ms
Installing 2/5 openssl:x64-windows...
Elapsed time to handle openssl:x64-windows: 73.7 ms
Installing 3/5 zlib:x64-windows...
Elapsed time to handle zlib:x64-windows: 26.5 ms
Installing 4/5 libssh2:x64-windows...
Building libssh2[core,zlib]:x64-windows...
-- Using cached libssh2-libssh2-libssh2-1.11.0.tar.gz.
-- Cleaning sources at F:/Feature-test/cppfs/vcpkg/buildtrees/libssh2/src/sh2-1.11.0-18ec9433d6.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source F:/Feature-test/cppfs/vcpkg/downloads/libssh2-libssh2-libssh2-1.11.0.tar.gz
-- Using source at F:/Feature-test/cppfs/vcpkg/buildtrees/libssh2/src/sh2-1.11.0-18ec9433d6.clean
-- Found external ninja('1.11.0').
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Fixing pkgconfig file: F:/Feature-test/cppfs/vcpkg/packages/libssh2_x64-windows/lib/pkgconfig/libssh2.pc
-- Using cached mingw-w64-i686-pkgconf-1~1.8.0-2-any.pkg.tar.zst.
-- Using cached msys2-msys2-runtime-3.4.6-1-x86_64.pkg.tar.zst.
-- Using msys root at F:/Feature-test/cppfs/vcpkg/downloads/tools/msys2/6f3fa1a12ef85a6f
-- Fixing pkgconfig file: F:/Feature-test/cppfs/vcpkg/packages/libssh2_x64-windows/debug/lib/pkgconfig/libssh2.pc
-- Installing: F:/Feature-test/cppfs/vcpkg/packages/libssh2_x64-windows/share/libssh2/usage
-- Installing: F:/Feature-test/cppfs/vcpkg/packages/libssh2_x64-windows/share/libssh2/copyright
-- Performing post-build validation
Stored binaries in 1 destinations in 268 ms.
Elapsed time to handle libssh2:x64-windows: 12 s
Installing 5/5 cppfs:x64-windows...
Building cppfs[core,ssh]:x64-windows...
-- Using cached cginternals-cppfs-156d72e2cf0a9b12bdce369fc5b5d98fb5dffe2d.tar.gz.
-- Cleaning sources at F:/Feature-test/cppfs/vcpkg/buildtrees/cppfs/src/8fb5dffe2d-bfd8827df6.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source F:/Feature-test/cppfs/vcpkg/downloads/cginternals-cppfs-156d72e2cf0a9b12bdce369fc5b5d98fb5dffe2d.tar.gz
-- Applying patch ssh-dependencies.diff
-- Using source at F:/Feature-test/cppfs/vcpkg/buildtrees/cppfs/src/8fb5dffe2d-bfd8827df6.clean
-- Found external ninja('1.11.0').
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: F:/Feature-test/cppfs/vcpkg/packages/cppfs_x64-windows/share/cppfs/copyright
-- Performing post-build validation
Stored binaries in 1 destinations in 601 ms.
Elapsed time to handle cppfs:x64-windows: 18 s
Total install time: 31 s
cppfs provides CMake targets:

    # this is heuristically generated, and may not be correct
    find_package(cppfs CONFIG REQUIRED)
    target_link_libraries(main PRIVATE cppfs::cppfs)

CMakeLists.txt:
cmake_minimum_required(VERSION 3.16)

project(test)

find_package(cppfs CONFIG REQUIRED)

add_executable(main usageTest.cpp)

target_link_libraries(main PRIVATE cppfs::cppfs)

UsageTest.cpp:
#include <stdio.h>
#include <cppfs/cppfs.h>

int main()
{
    printf("%s\n", "HELLO WORLD");
    return 0;
}

Config:
1> CMake generation started for configuration: 'x64-Debug'.
1> Command line: "C:\WINDOWS\system32\cmd.exe" /c "%SYSTEMROOT%\System32\chcp.com 65001 >NUL && "C:\PROGRAM FILES\MICROSOFT VISUAL STUDIO\2022\COMMUNITY\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\CMAKE\CMake\bin\cmake.exe"  -G "Visual Studio 17 2022" -A x64  -DCMAKE_CONFIGURATION_TYPES:STRING="Debug" -DCMAKE_INSTALL_PREFIX:PATH="C:\Users\test\source\repos\usageTest\out\install\x64-Debug" -DCMAKE_TOOLCHAIN_FILE=F:/Feature-test/cppfs/vcpkg/scripts/buildsystems/vcpkg.cmake "C:\Users\test\source\repos\usageTest" 2>&1"
1> Working directory: C:\Users\test\source\repos\usageTest\out\build\x64-Debug
1> [CMake] -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
1> [CMake] -- The C compiler identification is MSVC 19.36.32532.0
1> [CMake] -- The CXX compiler identification is MSVC 19.36.32532.0
1> [CMake] -- Detecting C compiler ABI info
1> [CMake] -- Detecting C compiler ABI info - done
1> [CMake] -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.36.32532/bin/Hostx64/x64/cl.exe - skipped
1> [CMake] -- Detecting C compile features
1> [CMake] -- Detecting C compile features - done
1> [CMake] -- Detecting CXX compiler ABI info
1> [CMake] -- Detecting CXX compiler ABI info - done
1> [CMake] -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.36.32532/bin/Hostx64/x64/cl.exe - skipped
1> [CMake] -- Detecting CXX compile features
1> [CMake] -- Detecting CXX compile features - done
1> [CMake] -- Found OpenSSL: optimized;F:/Feature-test/cppfs/vcpkg/installed/x64-windows/lib/libcrypto.lib;debug;F:/Feature-test/cppfs/vcpkg/installed/x64-windows/debug/lib/libcrypto.lib (found version "3.1.1")
1> [CMake] -- Found ZLIB: optimized;F:/Feature-test/cppfs/vcpkg/installed/x64-windows/lib/zlib.lib;debug;F:/Feature-test/cppfs/vcpkg/installed/x64-windows/debug/lib/zlibd.lib (found version "1.2.13")
1> [CMake] -- Configuring done (7.8s)
1> [CMake] -- Generating done (0.1s)
1> [CMake] -- Build files have been written to: C:/Users/test/source/repos/usageTest/out/build/x64-debug
1> Extracted CMake variables.
1> Extracted source files and headers.
1> Extracted code model.
1> Extracted toolchain configurations.
1> Extracted includes paths.
1> CMake generation finished.

Build:
>------ Build All started: Project: usageTest, Configuration: x64-Debug ------
  MSBuild version 17.6.3+07e294721 for .NET Framework

    Checking Build System
    Building Custom Rule C:/Users/test/source/repos/usageTest/usageTest/CMakeLists.txt
    usageTest.cpp
    main.vcxproj -> C:\Users\test\source\repos\usageTest\out\build\x64-debug\usageTest\Debug\main.exe
    Building Custom Rule C:/Users/test/source/repos/usageTest/CMakeLists.txt

Build All succeeded.


Roku command error

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (0.37.0 dev):
Python release (3.5):
Component/platform:
Roku
Description of problem:
Receiving this error when trying to play/pause, change track, etc
17-02-01 12:21:16 homeassistant.core: Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "C:\Python\lib\site-packages\homeassistant\core.py", line 1054, in _event_to_service_call
    yield from service_handler.func(service_call)
  File "C:\Python\lib\site-packages\homeassistant\components\media_player\__init__.py", line 364, in async_service_handler
    yield from getattr(player, method['method'])(**params)
  File "C:\Python\lib\concurrent\futures\thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Python\lib\site-packages\homeassistant\components\media_player\__init__.py", line 584, in media_pause
    raise NotImplementedError()
NotImplementedError

Expected:
Play/Pause to send commands to Roku
Problem-relevant configuration.yaml entries and steps to reproduce:
media_player:
  - platform: roku
    host: <IP_ADDRESS>

Traceback (if applicable):

Additional info:

Maybe somebody can help me out, I've been able to get it working but I don't fully understand why it works or why it wasn't working as is. All I did was change this function in roku.py from this:
def media_play_pause(self):
    """Send play/pause command."""
    if self.current_app is not None:
        self.roku.play()

To:
def media_pause(self):
    """Send play/pause command."""
    if self.current_app is not None:
        self.roku.play()

I guess I don't understand why the HA service play_pause is calling the service pause in roku.py.  Can somebody explain if this is a bug?

I did the same thing locally to get Roku to work.
I think the correct fix is to have both functions present - and that the root cause for the "bug" is that Roku reports it's status as "playing" even when it's paused.  As a result, the UI issues a media_pause event rather than a media_play_pause event when you click on the button it exposes and we need to have implementations for both as a result.


Https not supporting

I have used http://.....   and it was working but as soon as I changed my URL with secure (https) it's not connecting.
Please help me if is there any solution or scope available to use the secure connection.

@1983Alok Can you directly try using the http library and let know if that works?

Hi Rohit,
Thanks for the update but I have fixed the issue and now I am able to use
it with https.
On Wed, 3 Apr 2019, 23:28 Rohit Reddy Abbadi<EMAIL_ADDRESS>wrote:

@1983Alok https://github.com/1983Alok Can you directly try using the
http library and let know if that works?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/infitio/flutter-adhara/issues/3#issuecomment-479595148,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALUhoI5ycZ2evYplK2RCICsMOTcDHFJTks5vdOutgaJpZM4cU_ik
.

--
Disclaimer
------------------------------------------------------------------------------------------------------------------------------------------------------
Any views or opinions presented in this email are solely those of the
author and do not necessarily represent those of the company. Employees of
LMS Solutions (India) Pvt. Ltd. are expressly required not to make
defamatory statements and not to infringe or authorize any infringement of
copyright or any other legal right by email communications. Any such
communication is contrary to company policy and outside the scope of the
employment of the individual concerned. The company will not accept any
liability in respect of such communication, and the employee responsible
will be personally liable for any damages or other liability arising.

To handle CERTIFICATE_VERIFY_FAILED
https://github.com/dart-lang/http/issues/267


How to use hashsum?

It looks like the usage is hashsum --algo filename but when I run hashsum --sha1 filename it says
sum: error: Invalid options
Unrecognized option: `sha1`

Are you running hashsum or sum?

Hashsum, but it appears to be invoking sum.

How did you build the utils?  Also what operating system are you using?  I just built on Windows and can't seem to reproduce this issue.

Redox


changes in Readme.md

made changes to Readme.md

Hi thanks for your contribution. However we don't want people to just add themselves to the contribution list we want them to contribute with something first :) To make sure we're not confusing people we are not merging this. However feel free to make more pull requests with other changes :)


Fix build as per https://github.com/apple/swift/pull/69707

Please re-create this on compnerd/swift-build; but this seems reasonable.

Yeah I also sent identical https://github.com/compnerd/swift-build/pull/705

Merged the upstream branch back in. 🤞


Updated default rpc port to the number specified in the README.

In the readme under "How to test" the gnache server is set to port 8545. The library that grabs the rpc return 3000. Changing this to 8545 allows for interaction with the deployed contracts.
It looks like local users will also need to update the contract addresses in this file. Do you want to add instructions to the readme for that?

@TheHollidayInn Thank you for point this.


More Windows 11 matching UI elements

Description of the new feature / enhancement
For example, "PowerToys Run" feature need to use transparency effects.  on shortcut guide menu, the buttons should show with rounded corners instead of sharp keyboard buttons. Also, on Windows 10 I was able to see shortcut guide with just holding the Windows key but on Windows 11 I am not able to do it. It requires me to press Win+shift+/ Please add the old one back
Scenario when this would be used?
To match the overall Windows 11 UI. Consistency is requires while the app is evolving
Supporting information
No response

Thank you for the suggestion!
Related issues: https://github.com/microsoft/PowerToys/issues/12182, https://github.com/microsoft/PowerToys/issues/890, https://github.com/microsoft/PowerToys/issues/11963.

All known issues and stuff we're proactively working on. As this is tracked work, closing this issue out


Get rid of tokenizeViewportSoon in ViewModel and communicate modelVisibleRanges to the text model

Currently, the view model explicitly requests tokenizing some ranges in the text model every now and then (when scrolling, when the content changes, or when the tokenizationSupport changes).
Especially the handling of tokenizationSupport changes is weird - the text model tells the view port that the tokenizer changed, so that the viewport can re-request the text model to tokenize the viewport range (which does a heuristic tokenization initially, because background tokenization didn't catch up fully). This is because the text model itself doesn't know that it should focus on tokenizing the visible lines first (see here).
I suggest to implement this in a different way by inversing this mechanism: The tokenization implementation should know which ranges are currently visible (and when they change) and focus on those first.
The model already uses isAttachedToEditor to check if it is rendered by one or more text editors:
interface ITextModel {
  /** @internal */
  onBeforeAttached(): void;
  /** @internal */
  onBeforeDetached(): void;
}

Letting the model know which lines are visible just extends this idea. This could look like this:
interface ITextModel {
  /** @internal */
  onBeforeAttached(): IAttachedView;
  /** @internal */
  onBeforeDetached(view: IAttachedView): void;
}
interface IAttachedView {
  setVisibleRanges(ranges: LineRange[]): void;
}

It would be great to bring back that opening a file doesn't flicker (even the first frame has tokens)


thebe integration raises invalid hook call when rendering Output node

Describe the bug
#21 is introduced in version 0.1.37. When I'm rendering the ast contains Output node, which calls useNotebookCellExecution in Output component in @myst-theme/jupyter, the following hook raises:
export function Output(node: GenericNode) {
    const [executing, setExecuting] = useState(false);
    ...
}
-> "Invalid hook call. Hooks can only be called inside of the body of a function component"

Everything works fine for 0.1.36. Thanks for any response.
Reproduce the bug
Here is a sample ast:
{
  "type": "root",
  "children": [
    {
      "type": "block",
      "meta": "{}",
      "position": {
        "start": {
          "line": 0,
          "column": 0
        },
        "end": {
          "line": 1,
          "column": 0
        }
      },
      "children": [
        {
          "type": "mystDirective",
          "name": "code-cell",
          "args": "python",
          "value": "import numpy as np\nimport pandas as pd",
          "position": {
            "start": {
              "line": 2,
              "column": 0
            },
            "end": {
              "line": 6,
              "column": 0
            }
          },
          "children": [
            {
              "type": "code",
              "lang": "python",
              "executable": true,
              "value": "import numpy as np\nimport pandas as pd"
            }
          ]
        },
        {
          "type": "output",
          "id": "HnAzXQRvGWVyHgRtN9RK-",
          "data": [
            {
              "output_type": "execute_result",
              "execution_count": 4,
              "metadata": {},
              "data": {
                "text/html": {
                  "content": "
\n\n
\n
\n
\n      	\n      0	\n      1	\n      2	\n      3	\n
\n
\n
\n
\n      0	\n      0.748943	\n      0.745973	\n      0.582013	\n      0.450207	\n
\n
\n      1	\n      0.254574	\n      0.375159	\n      0.664602	\n      0.829163	\n
\n
\n      2	\n      0.208465	\n      0.168282	\n      0.172431	\n      0.069546	\n
\n
\n      3	\n      0.614599	\n      0.501844	\n      0.675450	\n      0.248261	\n
\n
\n
\n
",
                  "content_type": "text/html"
                },
                "text/plain": {
                  "content": "          0         1         2         3\n0  0.748943  0.745973  0.582013  0.450207\n1  0.254574  0.375159  0.664602  0.829163\n2  0.208465  0.168282  0.172431  0.069546\n3  0.614599  0.501844  0.675450  0.248261",
                  "content_type": "text/plain"
                }
              }
            }
          ]
        }
      ]
    }
  ]
}

List your environment
No response

Thanks @GYHHAHA for the bug report -- I put up a small PR to fix this. @stevejpurves (and me) this is a good reminder that we can't have hooks in the node-renderers directly because they aren't actually react nodes. I think that we should probably change this in the future -- I will open an issue for that!