Redis filter

What this PR does / why we need it:
envoy supports mongo/redis for telemetry. This PR enables the redis filter.
Just putting it out there..
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

Support for telemetry gathering from redis connections

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/pilot-presubmit.sh
bf832cd13ee2966a333ea0b18f84515580629aa1
link
/test pilot-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/pilot-presubmit.sh
57ca9dac5928463765de6ded5f1204a1b9c47a2c
link
/test pilot-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/pilot-presubmit.sh
6b4ca305d002d9252655c41c4052c68311375ddc
link
/test pilot-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/pilot-presubmit.sh
41090145bab58652bd018fbd9b64aca86b365ce1
link
/test pilot-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj any more comments?

@kyessenov ?


runDistributable skips backward slashes from args on Windows

Compose 1.2.0-beta03, Windows 11
fun main(vararg args: String) {
    println(args.toList())
}

plugins {
    kotlin("jvm")
    id("org.jetbrains.compose")
}

repositories {
    mavenCentral()
    maven("https://maven.pkg.jetbrains.space/public/p/compose/dev")
    google()
}

dependencies {
    implementation(compose.desktop.currentOs)
}

compose.desktop {
    application {
        mainClass = "MainKt"
        args += "C:\\Program Files"
    }
}

gradlew run output:
[C:\Program Files]

gradlew runDistributable output:
[C:Program Files]

Because of this we have failed tests in Gradle Plugin:
DesktopApplicationTest[Gradle '7.1.1'] > testAndroidxCompiler() FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at DesktopApplicationTest.kt:63

java.io.FileNotFoundException: C:\Users\Igor\AppData\Local\Temp\junit2362651168530675128\main-image.actual.png (The system cannot find the file specified)

DesktopApplicationTest[Gradle '7.1.1'] > proguard() FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at DesktopApplicationTest.kt:80

represents a file path

Probably it affects not only paths, but any data that contain backslashes. So, if the jpackage removes them, we should escape it (replacing \ by \).

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Prisma2 Preview23 + Nexus 0.11.x Query performance issue

This is an extension of #239, creating another issue per @janpio.
In order to address the N+1 field resolution issue in nexus-prisma via query batching in prisma2, we've upgraded to prisma2 preview23 and nexus-prisma 0.11.1, but did not see much improvement in performance.
The example query we're testing with is this:
await prisma.lessonTaking.findMany({
  where: whereArg,
  select: {
    id: true,
    lessonContent: {
      select: {
        id: true,
      },
    },
    courseTaking: {
      select: {
        id: true,
      },
    },
    assignmentTakings: {
      select: {
        id: true,
      },
    },
  },
})

Executing this query directly with prisma runs 5 queries in 53.26ms.  Running this query using a nexus resolver, e.g.:
t.field('lessonTakings', {
  type: 'LessonTaking',
  list: true,
  nullable: true,
  args: {
    where: 'LessonTakingWhereInput',
  },
  resolve: async (_, args, { prisma }) => {
   return prisma.lessonTaking.findMany({
     ...args,
    })
  },
})

Takes 2.01 sec, and runs 845 queries (mostly single select) based on the prisma logs.  We're not sure if the query batching isn't catching our queries for some reason.
The where statement used for this testing:
{
  student: { id: { equals: <id> } },
  assignedDate: { lt: <date> },
  assignmentTakings: { some: { submittedAt: { equals: null }, isSkipped: { not: true } } },
}

The example log is attached.
log_comparison.txt

Hey @Linktheoriginal 👋,
I've just made some tests and could not reproduce your issue on a simple datamodel.
Could you provide some minimal reproduction code for your issue? That would be super helpful 🙏

@Weakky I've added you to a repo with some reproduction code.

Hey @Linktheoriginal, thanks a lot for setting up some reproduction.
Here's the underlying problem: https://github.com/prisma/prisma-client-js/issues/562

Hello, any update on this? the issue @Weakky mentioned is closed but the query is still taking a long time to resolve. Many thanks for your awesome work.

@ALITHEALI in our project with @Linktheoriginal we go to another solution to generate our CRUD system and use tool to convert info: GraphQLResolveInfo to be available Prisma select object
with this way you have all control of your crud system and have best performance
here is our tools https://github.com/AhmedElywa/prisma-tools


Fixing broken demo

Hi there,
I noticed the demo page is entirely broken because of a 404. So I fixed it in this PR.
Kind regards,
Casper.

Hey @pnfcre, thanks for the PR.
The issue stems from us only fetching the library from unpkg when the url ends with github.io when the example page can also be served from github.com.
I made changes in 60eb1da06e6e342fffc1f514d6b7b595051f5bc6 that fixes the issue while also preserving the functionality of serving up the development version of the library when running locally.

Awesome, closing this!


img2pdf: 0.3.1 -> 0.3.2

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/img2pdf/versions.
meta.description for img2pdf is: '"Convert images to PDF via direct JPEG inclusion"'.

Checks done (click to expand)

built on NixOS
/nix/store/9x6ijrddck3pa2g8s3sadhvxbm9v2m6s-img2pdf-0.3.2/bin/.img2pdf-wrapped passed the binary check.
/nix/store/9x6ijrddck3pa2g8s3sadhvxbm9v2m6s-img2pdf-0.3.2/bin/img2pdf passed the binary check.
2 of 2 passed binary check by having a zero exit code.
2 of 2 passed binary check by having the new version present in output.
found 0.3.2 with grep in /nix/store/9x6ijrddck3pa2g8s3sadhvxbm9v2m6s-img2pdf-0.3.2
directory tree listing: https://gist.github.com/7adc25c72f7d64b258368a8917eb5a01
du listing: https://gist.github.com/61b3c8a2a7388bd406dc8ec34a7f67d0

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)
1 package rebuild(s)
1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
img2pdf

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/9x6ijrddck3pa2g8s3sadhvxbm9v2m6s-img2pdf-0.3.2 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A img2pdf https://github.com/r-ryantm/nixpkgs/archive/55a96df66a61c7cdc1c05fb6af4a233a93295308.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/9x6ijrddck3pa2g8s3sadhvxbm9v2m6s-img2pdf-0.3.2
ls -la /nix/store/9x6ijrddck3pa2g8s3sadhvxbm9v2m6s-img2pdf-0.3.2/bin

cc @veprbl for testing.

built and tested locally :+1:


3060 toolbar menu mui 5 styled

Fixes # .

Changes proposed in this Pull Request:

Update to use Mui-v5 styled components
Update component to extend sx prop
Update component docs for ToolbarMenu

**Also would like a suggestion on an example code snippet for this in the docs

Screenshots:
Temporary ToolbarMenu

To Test:

There is a showcase branch feature/3050-channel-value-mui-v5-styles-update that this can be tested in the SharedAppBar

                 <ToolbarMenu
                    label={'Toolbar-menu'}
                    menuGroups={menuGroups}
                    // style={{ backgroundColor: '#ffbb00' }} // specificity 1
                    classes={{ rotatedDropdownArrow: classes.green }} // specificity 2
                    // className={classes.blue} // specificity 3
                    sx={{ backgroundColor: '#bb00ff' }} // specificity 4
                ></ToolbarMenu>

Ensure that there are no breaking changes and that the old API properties still work the same and that you can still style inline styles using our classes prop.
Test sx prop extension.

@bkarambe, when you are going to be off for some time can you please add your comments and suggestions without blocking the PR with a change request?

@bkarambe, when you are going to be off for some time can you please add your comments and suggestions without blocking the PR with a change request?

If Suraj is out tomorrow & this PR is adjusted by end of day, might be easier to re-open the PR to move it along.


Express instrumentation in Lambda

Is your feature request related to a problem? Please describe

My setup includes deploying the express js in lambda and using this package https://github.com/dougmoscrop/serverless-http/, it is working fine,,, but some how the express instrumentation is not patching the express app if it is not using the "app.listen" method because it uses the package i mentioned. Is there a way to patch express while using the serverless-http.
Describe the solution you'd like to see

I didn't find any solution yet.
Describe alternatives you've considered

There are no alternatives as well.
Additional context

I did checked by keeping opentelemetry DiagLogLevel to ALL, i didn't find the log that indicates patched for express.

ping @dyladan

I'm not familiar enough with serverless-http to say if this is simple or not. Most likely you would have to create an instrumentation for that package, which may be simple or may be a lot of work.


FPGrowth should traverse from bottom to top.

This should have been addressed in e2efa185f827412297b5cfb4429d626b701272d2
It was fixed in FPGrowthStreaming but not FPGrowth...
https://github.com/stanford-futuredata/macrobase/commit/e2efa185f827412297b5cfb4429d626b701272d2#diff-17a5f28a45018f3282473cce06fb484fR267

This is kind of a bizarro bug. I basically just need to test the FPGrowth code more and eliminate redundancy between the two.
The problem is that frequentItemOrder is sorted from smallest to largest. So we want to traverse the tree when mining in order, and insert into the tree in reverse order. Super ugly, and it turns out the two were inconsistent, yet somehow the test case worked for this...
The right thing to do is to add a new test case. I really want to refactor all of this. Maybe we can discuss consolidation of the streaming and batch pipelines during tomorrow's call.

LGTM

427eb240b45dd35ae48595ff783528268068b195


docs: args parameter is not documented

The args parameter for each rule is not documented; when I first tried using rules_itest I didn't realise that passing args was possible! Could this parameter be documented where it is available?

args are supported on itest_service, itest_task, and service_test.
args are built-in attributes on executable rules and cannot be overriden, so I cannot add it to the rule's attributes, which means Stardoc won't pick it up when generating docs. If we want to add them to the docs, they need to be described in prose elsewhere, not in the attribute table. Any advice where you think it would make sense?

Ah, I see...
rules_js have a nice third paragraph in their js_binary section, that says:
"All common binary attributes are supported including args as the list of arguments passed Node.js."
https://docs.aspect.build/rulesets/aspect_rules_js/docs/js_binary/#js_binary
There could also be a simple inline example in the docs for each rule that includes some args.

A curious thing is that for the rules documented on the bazel site, whilst args doesn't appear in the detailed table, it does appear in the rule summary at the top, and includes a link to the common attribute documentation:
https://bazel.build/reference/be/c-cpp#cc_binary

Cool, I will add something like rules_js has. I'm not sure how the bazel rule documentation works, I suspect they have a more complex setup where they merge some custom-generated sections with stardoc output - probably more effort than is worth here.

Documented in https://github.com/dzbarsky/rules_itest/commit/5d204344f3df6b406d7a02672e7cd53eb45d95ca


Enable jinja "strict undefined behavior", fix templates, improve reporting

Most likely we need to contribute back to minijinja2 to get proper tracking information for "undefined errors" because right now all we see is this:

Current state of this PR: the corresponding minjinja fix has landed in their main branch, but not in a point release yet, which means our dep now looks like this:

So I don't love that for us and this PR can probably wait for a new minjinja to land (releases seem regular)

BUT! it looks awesome and is very useful: I pulled in color-backtrace for coloring and filtering and this helps me at least (I'm curious to see what @mistydemeo and @duckinator think about it specifically) narrow down what's wrong almost immediately:

color-backtrace is awesome (and the big dependency here is backtrace, which we already have) and has a RUST_LIB_BACKTRACE=full mode that even shows you the Rust source code:

Even though I'd like to wait for a minjinja point release to drop, I'm gonna stack my other PRs on top of this one because, damn, that's good tooling.

Note 1: I've forced backtrace capture, even if you haven't otherwise exported RUST_LIB_BACKTRACE=1 because I consider this equivalent to a panic, it might happen in CI, we're telling our users to report it and when they do, we'll want as much info as possible.

Note 2: this PR also fixes the few instances of us violating "strict undefined behavior", minjinja is going to get a little legalistic on us — the only thing y'all need to know is that if it complains something like:
      undefined value (in installer/npm/run.js:4)
   ╭─[4:1]
 3 │ const { run } = require("./binary");
 4 │ {%- if bin %}
   · ─────────────
 5 │ run({{ bin }});
   ╰────

Then you need if bin is defined, and the reason this happens is usually because the field is defined something like this:
struct Blah {
    #[serde(skip_serializing_if = "Option::is_none")]
    pub bin: Option<String>,
}

But in the actual case of bin? it's because it's rendered without any "context/globals" object once, as evidenced later in the code:
https://github.com/axodotdev/cargo-dist/blob/33316c58184a936ab293cc5567efc1fadba10453/cargo-dist/src/backend/installer/npm.rs#L89-L93
ie. while "rendering the directory template", it gets rendered once with zero context, and then it's removed and re-rendered with the correct context. That's a hack and I hate it, but with just if bin is defined instead of if bin, we can go ahead anyway:
{%- if bin is defined %}
run({{ bin }});
{%- endif %}

After discussing in Discord, we'll publish minjinja as axominijinja and switch to that for the next release, then switch back when it's done.

Taking this out of draft since it's ready for being merged imho!

I caught another template issue (a benign one) while working on https://github.com/axodotdev/cargo-dist/pull/1509 — which mostly tells me the homebrew stuff isn't all covered by unit tests

@mistydemeo I addressed whitespace issues — in fact, I made sure the template produces exactly the same input as before, just using those nice macros and the new "template input" structs.
I've also added a comment about the jinjaness of the macro (and a link to the surprising-to-find docs), and fixed the endif/end mishap.

The mac tests are failing because it's trying to install a linuxbrew-only package..


Fix typo (perfrom should be perform)

Task name: Use .NET V2
Description: Typo in the readme (perfrom should be perform)
Documentation changes required: (Y/N) N
Added unit tests: (Y/N) N
Attached related issue: (Y/N) N
Checklist:

[x] Task version was bumped - please check instruction how to do it
[x] Checked that applied changes work as expected

Please update task version as mentioned in PR template

@bishal-pdMSFT  all done :)

@bishal-pdMSFT  ready for merge


demo site error when creating a ticket

I tried to use the demo site, and it gave me error like:

Sorry, forgot to run migrations on the heroku instance when I upgraded it to latest django-helpdesk. Should work now!

Yep, works for me (thanks)


📃: Add Typing Speed Test Game

🔴 Title : Typing Speed Test
🔴 Tech stack : HTML    CSS   JavaScript
🔴 Objective :  test typing speed with fun
🔴 Summary : A game where users test their typing speed by typing a random passage. It can display the number of words per minute (WPM) and accuracy, with features like countdown timers and progress tracking.
📸 Screenshots
N/A

✅ Details to Include When Taking the Issue:
gssoc-ext,   hectoberfest

@iamrahulmahato please assign me this task

@iamrahulmahato please assign this task to me

the project is already present


XD-2794: Add MongoDB Source

The diff.

Thanks for this contribution, looks very good overall.
You'll need to change the gradle build to have your module project appear there (similar to the mongo sink)
Also, do you think you could be able to factor out the module options that relate to connecting/authenticating to a mongo instance out of FromMongoDbOptionMixin and IntoMongoDbOptionMixin, leaving out only the read (respectively write) details in those classes?

Applied the refactorings discussed above and merged as eb733edf34747e4c7bf35895d31f1211ba2f71f2


Lua scripts are not executed(0.5.2h)

Describe your bug here.
I packed the files in the example_mods into the assets directory and changed Paths.hx, but then some lua scripts didn't execute

BFSing.lua did not execute

Command Prompt/Terminal logs (if existing)
source/Discord.hx:68: Discord Client initialized
source/Discord.hx:25: Discord Client started.
source/FreeplayState.hx:368: dad-battle-hard
source/FreeplayState.hx:374: CURRENT WEEK: week1
source/LoadingState.hx:162: Setting asset folder to shared
source/Conductor.hx:75: new BPM map BUDDY []
source/FunkinLua.hx:76: Lua file loaded succesfully:assets/scripts/BFSing.lua
source/FunkinLua.hx:76: Lua file loaded succesfully:assets/scripts/ui.lua
source/Conductor.hx:75: new BPM map BUDDY []
source/FunkinLua.hx:76: Lua file loaded succesfully:assets/scripts/BFSing.lua
source/FunkinLua.hx:76: Lua file loaded succesfully:assets/scripts/ui.lua

Are you modding a build from source or with Lua?
Source
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
Yes,I packed the files in the example_mods into the assets directory and changed Paths.hx

0.5.2h and below are not supported

不支持 0.5.2h 及以下
thanks

不支持 0.5.2h 及以下
thanks

you're welcome?


[youtube] itag 272 has 4320 lines

Do you have an example URL? In #3128 it's reported as 2160p.

E.g.: https://www.youtube.com/watch?v=sLprVF6d7Ug - youtube_dl downloads f272, size: 7680x4320
272 format tested also here: https://github.com/zaps166/QMPlay2/commit/71c7a6c6264ceae7bebdd6a6cc2e3e216433c845

This is not always the case: RtoitU2A-3E
>youtube-dl RtoitU2A-3E -f 272
...
[download] Destination: 140611  (Trouble Maker) -    -RtoitU2A-3E.webm
[download] 100% of 551.40MiB in 02:55

>ffmpeg.exe -i "140611  (Trouble Maker) -    -RtoitU2A-3E.webm"
...
  Duration: 00:03:27.77, start: 0.000000, bitrate: 22262 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv), 3840x2160, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)

Moveover hardcoded metadata from _formats is only used when DASH manifest lacks this information that is not the case most of the time. So I don't see any problem here.

Ok!


Allowing CAT to use UniProt databases

I would like to use CAT with UniProts databases as they contain a more suitable set of proteins for my comparisons.
I am working on making CAT prepare work with this, but it requires a lot of manual parsing and fixing of TaxIDs to make UniProt compatible with the NCBI taxonomy.
Is this a feature that would be interesting to have or not? If not, just close this issue again.

Hi @openpaul,
For now I cannot really work on this myself, but if you have a working solution that you feel is stable and is useful to people please let us know! We would be happy to implement it. Feel free to fork our repository and PR.
Bastiaan

Thanks, goo to know. If I come up with something I will let you know. For now I close this issue again.


Windows client freezes when typing project/task/description

In a workspace with around 900 tasks and a few hundred projects, the Windows client consistently hangs a couple of seconds whenever I start typing in the "What are you doing" field or, when editing an entry and typing the project/task name. It freezes as soon as I've typed a single letter. As it unfreezes the list of suggestions is then displayed.
To mitigate this, I'd suggest doing one or more of the following

make the suggestion lookup asynchronous so that the client at least stays responsive
only do lookup when 3 or more characters have been entered
limit the number of results being returned to the client
only look up suggestions after a short delay (e.g. I type "web", then pause shortly, after which only suggestions for "web" are requested and shown to me, not for neither "w" nor "we")

As a follow-up to this I should add that the way we're using Toggl is that almost every entry is different, since people usually follow-up with a few notes about what they spent their time on this time around. This may be what's causing the delay in the app, since the suggestions include a distinct list of previous entries and their description (which in our case could be a very long list!)

Just found similar issue #1660 which claims this was solved in #1790. However, the issue persists.

This is indeed not fixed. I'm able to replicate this when the account has bigger amount of data (things to load into the autocomplete). Windows autocomplete is solved a bit different than the other platforms and I suspect that this solution is the reason of the slowdown.

@IndrekV is there any kind of timeline on this? Can we help testing somehow?

We're in mid-development. We've found the cause and worked out how to solve the issue. We are rewriting this part of the autocomplete. I hope we can share updated version in a week or two.

closed together with #2042 . Will be released to beta next week


On the screen "Seed phrase" there is no "Next" button in the zoomed mode.

"Next" button not appear in the "Seed phrase" screen in zoomed mode on iPhone 6s.
Steps:

Go to iPhone's Settings > Display and Brightness.
Set Display Zoom View = Zoomed.
Run Beam app for the first time.
Create a new wallet.
Generate the seed phrase.
Actual:
The button the "next" isn't displayed.
Expected:
The button is displayed. .

fixed. Maine build 2.3(3)

checked


Code rewrite and new statistical sensors

BREAKING CHANGES!
This will break existing installations!
Enhancement :

Statistics sensors for week, month and year.
Code refactor
More robust

@CM000n I may have found a way to change weeks stats from Sunday - Saturday to Monday - Sunday. What would you preferer? I believe the Tojota script uses the former.

Sounds promising. Yes, the Toyota script uses the Japanese week logic from Sunday to Saturday. Personally, of course, I prefer the ISO 8601 standard and thus a week beginning with Monday.
Apart from Japan, I think it is only the USA, China, Israel and some Arab countries that have a week beginning with Sunday.

Hello!
I am in favour also to the Mon-Sun cycle. ISO 8601.

@CM000n @Joonaskaru I will implement it in the backend in the next couple of days, and the change the code here afterwards.
I actually thought more countries used Sunday to Saturday. Nevertheless, all those countries are outside Europe anyways. And i also like to stick to standards, it makes everything easier. 😊
Stay tuned for an update!

Hey guys
I just pushed an update containing the changes we talked about, including some minor improvements on some other areas. 🎉
If you can test it and give me feedback it will be much appreciated 😃

I just had a strange behavior. After I just replaced the files in my Custom Component Folder with the current versions from the Test Branch, all Toyota sensors were created twice after the restart.
I was able to simply remove the old sensors and then delete the "_2" suffix from the duplicate sensors. Any idea what could be the cause of this?

I don't have that attribute with my car, so the error never showed up while i was testing, but it is a very simple fix. It just complains that you cannot add a float variable to a str variable. I have pushed a fix!
Thanks for the feedback!

And the problem with them being add again, is because i changed the unique_id attribute. This was a necessary change to make the code cleaner and better unfortunately. When releasing version 1.0.0 i will make a clean note for every updating from older versions.

Thank you, the problem seems to be solved now:

But aside from the fact that you seem to have forgotten another space 😉, I don't like that the unit for the average fuel consumption is now combined with the value.
This makes it much more difficult if you want to extract the value from the entity's attributes for other purposes, e.g. with the help of a template. Also, it breaks a bit with logic, since the other stats don't have a combination of value and unit either.
I think it is perfectly sufficient to refer somewhere in the readme to the units of the statistics collected.

I have never been able to use this attribute, so i never thought about, but you are right, it should only return the value and not anything else.


Fatal error when running read_csv

Running read_csv on https://jakevdp.github.io/blog/2015/08/14/out-of-core-dataframes-in-python/ failed hard, both in single threaded with some parsing exceptions and in multi-threaded with a hard kernel failure.  This occurred both in pandas master (which releases the GIL) and in pandas 0.17.0.
Conversation at https://gitter.im/blaze/dask?at=564f686d06a214f1080b06f7
Some things to consider:

Are we sending the right blocks of data to read_csv?  Is dask.utils.textblock running properly (cc @cowlicks).  E.g. does this run correctly before 3ce1de85 ?
Do recent changes to Pandas in releasing the GIL cause thread safety issues?

OK, I got data from here http://osm-x-tractor.org/LinkClick.aspx?link=https%3A%2F%2Fmega.co.nz%2F%23F!NkpnBT5J!JNiylh_dpx_QnvxQtOO_Yg&tabid=58&portalid=0&mid=473
Then, after extraction, ran this script
import dask.dataframe as dd
columns = ["name", "amenity", "Longitude", "Latitude"]
data = dd.read_csv('POIWorld.csv', usecols=columns)
with_name = data[data.name.notnull()]
with_amenity = data[data.amenity.notnull()]
is_starbucks = with_name.name.str.contains('[Ss]tarbucks')
is_dunkin = with_name.name.str.contains('[Dd]unkin')

starbucks = with_name[is_starbucks]
dunkin = with_name[is_dunkin]

dd.compute(starbucks.name.count(), dunkin.name.count())

and reproduced the error.
Running the same thing under gdb
$ gdb python

run play.py

I get this traceback
(gdb) run play.py
Starting program: /home/mrocklin/Software/anaconda/bin/python play.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
[New Thread 0x7fffecc0e700 (LWP 15841)]
[New Thread 0x7fffec40d700 (LWP 15842)]
[New Thread 0x7fffebc0c700 (LWP 15843)]
[New Thread 0x7fffeb40b700 (LWP 15844)]
[New Thread 0x7fffeac0a700 (LWP 15845)]
[New Thread 0x7fffea409700 (LWP 15846)]
[New Thread 0x7fffe9c08700 (LWP 15847)]
[New Thread 0x7fffe9407700 (LWP 15848)]
[New Thread 0x7fffe77ff700 (LWP 15849)]
[New Thread 0x7fffe6ffe700 (LWP 15850)]
[New Thread 0x7fffe67fd700 (LWP 15851)]
[New Thread 0x7fffe5ffc700 (LWP 15852)]
[New Thread 0x7fffe57fb700 (LWP 15853)]
[New Thread 0x7fffe4ffa700 (LWP 15854)]
[New Thread 0x7fffcffff700 (LWP 15855)]
[New Thread 0x7fffcf7fe700 (LWP 15856)]
[New Thread 0x7fffceffd700 (LWP 15857)]
[New Thread 0x7fffce7fc700 (LWP 15858)]
[New Thread 0x7fffcdffb700 (LWP 15859)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe4ffa700 (LWP 15854)]
_PyObject_Malloc (ctx=0x0, nbytes=28) at Objects/obmalloc.c:1159
1159	Objects/obmalloc.c: No such file or directory.
(gdb) where
#0  _PyObject_Malloc (ctx=0x0, nbytes=28) at Objects/obmalloc.c:1159
#1  0x00007ffff79c7d0d in _PyLong_New (size=1) at Objects/longobject.c:197
#2  0x00007ffff79c9aea in PyLong_FromLong (ival=<optimized out>)
    at Objects/longobject.c:254
#3  0x00007ffff7a7db0e in do_mkvalue (p_format=0x7fffe4ff8098, p_va=0x7fffe4ff8080,
    flags=0) at Python/modsupport.c:219
#4  0x00007ffff7a7e4d9 in do_mktuple (p_format=0x7fffe4ff8098, p_va=0x7fffe4ff8080,
    endchar=41, n=1, flags=0) at Python/modsupport.c:173
#5  0x00007ffff7a7dadf in do_mkvalue (p_format=0x7fffe4ff8098, p_va=0x7fffe4ff8080,
    flags=0) at Python/modsupport.c:204
#6  0x00007ffff7a7e67d in va_build_value (format=<optimized out>, va=<optimized out>,
    flags=<optimized out>) at Python/modsupport.c:464
#7  0x00007ffff7a7ea2f in Py_BuildValue (format=<optimized out>,
    format@entry=0x7fffe842ccf3 "(i)") at Python/modsupport.c:420
#8  0x00007fffe8429a18 in buffer_rd_bytes (source=0x7fffc8004170,
    nbytes=<optimized out>, bytes_read=0x7fffe4ff81f0, status=0x7fffe4ff81e0)
    at pandas/src/parser/io.c:123
#9  0x00007fffe8428881 in parser_buffer_bytes (nbytes=<optimized out>,
    self=0x7fffc8004060) at pandas/src/parser/tokenizer.c:610
#10 _tokenize_helper (self=0x7fffc8004060, nrows=nrows@entry=64688, all=all@entry=0)
    at pandas/src/parser/tokenizer.c:1872
#11 0x00007fffe8428c77 in tokenize_nrows (self=<optimized out>,
    nrows=nrows@entry=64688) at pandas/src/parser/tokenizer.c:1905
#12 0x00007fffe83edf74 in __pyx_f_6pandas_6parser_10TextReader__tokenize_rows (
    __pyx_v_self=0x7fffe46911a8, __pyx_v_nrows=64688) at pandas/parser.c:8186
#13 0x00007fffe83efb0f in __pyx_f_6pandas_6parser_10TextReader__read_rows (
    __pyx_v_self=0x7fffe46911a8, __pyx_v_rows=0x7fffe7c9af90, __pyx_v_trim=0)
    at pandas/parser.c:8399
#14 0x00007fffe83f173b in __pyx_f_6pandas_6parser_10TextReader__read_low_memory (
    __pyx_v_self=0x7fffe46911a8, __pyx_v_rows=0x7ffff7d7e190 <_Py_NoneStruct>)
    at pandas/parser.c:7749
#15 0x00007fffe83eede5 in __pyx_pf_6pandas_6parser_10TextReader_8read (
    __pyx_v_rows=0x7ffff7d7e190 <_Py_NoneStruct>, __pyx_v_self=0x7fffe46911a8)
    at pandas/parser.c:7509
#16 __pyx_pw_6pandas_6parser_10TextReader_9read (__pyx_v_self=0x7fffe46911a8,
    __pyx_args=<optimized out>, __pyx_kwds=<optimized out>) at pandas/parser.c:7469
#17 0x00007ffff7a61511 in call_function (oparg=<optimized out>,
    pp_stack=0x7fffe4ff8508) at Python/ceval.c:4237
#18 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#19 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=2, kws=0x7fffc8003078, kwcount=0, defs=0x7fffe864cc68, defcount=1,
    kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#20 0x00007ffff7a607fa in fast_function (nk=<optimized out>, na=<optimized out>,
    n=<optimized out>, pp_stack=0x7fffe4ff8738, func=0x7fffe8653268)
---Type <return> to continue, or q <return> to quit---
    at Python/ceval.c:4344
#21 call_function (oparg=<optimized out>, pp_stack=0x7fffe4ff8738)
    at Python/ceval.c:4262
#22 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#23 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=1, kws=0x7fffc8002730, kwcount=0, defs=0x7fffe8644d10, defcount=1,
    kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#24 0x00007ffff7a607fa in fast_function (nk=<optimized out>, na=<optimized out>,
    n=<optimized out>, pp_stack=0x7fffe4ff8968, func=0x7fffe864d7b8)
    at Python/ceval.c:4344
#25 call_function (oparg=<optimized out>, pp_stack=0x7fffe4ff8968)
    at Python/ceval.c:4262
#26 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#27 0x00007ffff7a609b6 in fast_function (nk=<optimized out>, na=<optimized out>,
    n=<optimized out>, pp_stack=0x7fffe4ff8ad8, func=0x7fffe86b5bf8)
    at Python/ceval.c:4334
#28 call_function (oparg=<optimized out>, pp_stack=0x7fffe4ff8ad8)
    at Python/ceval.c:4262
#29 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#30 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=1, kws=0x7fffe4704c30, kwcount=6, defs=0x7fffe87b0238, defcount=53,
    kwdefs=0x0, closure=0x7fffe864cb38) at Python/ceval.c:3588
#31 0x00007ffff79bff01 in function_call (func=0x7fffe864d268, arg=0x7fffe4708ba8,
    kw=0x7fffe4719b48) at Objects/funcobject.c:632
#32 0x00007ffff7990189 in PyObject_Call (func=0x7fffe864d268, arg=<optimized out>,
    kw=<optimized out>) at Objects/abstract.c:2040
#33 0x00007ffff7a5f236 in ext_do_call (nk=-462386264, na=1, flags=<optimized out>,
    pp_stack=0x7fffe4ff8de8, func=0x7fffe864d268) at Python/ceval.c:4561
#34 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2878
#35 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=6, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3588
#36 0x00007ffff79bfde2 in function_call (func=0x7fffe4771d08, arg=0x7fffe47124c8,
    kw=0x0) at Objects/funcobject.c:632
#37 0x00007ffff7990189 in PyObject_Call (func=0x7fffe4771d08, arg=<optimized out>,
    kw=<optimized out>) at Objects/abstract.c:2040
#38 0x00007ffff7a5f236 in ext_do_call (nk=-462347064, na=0, flags=<optimized out>,
    pp_stack=0x7fffe4ff90f8, func=0x7fffe4771d08) at Python/ceval.c:4561
#39 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
---Type <return> to continue, or q <return> to quit---
    at Python/ceval.c:2878
#40 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=2, kws=0x7fffc8001890, kwcount=0, defs=0x7ffff62e1920, defcount=1,
    kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#41 0x00007ffff7a607fa in fast_function (nk=<optimized out>, na=<optimized out>,
    n=<optimized out>, pp_stack=0x7fffe4ff9328, func=0x7ffff3ac8620)
    at Python/ceval.c:4344
#42 call_function (oparg=<optimized out>, pp_stack=0x7fffe4ff9328)
    at Python/ceval.c:4262
#43 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#44 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=6, kws=0x7ffff7f92060, kwcount=0, defs=0x7ffff6298840, defcount=1,
    kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#45 0x00007ffff79bff01 in function_call (func=0x7ffff3ac86a8, arg=0x7fffe4712108,
    kw=0x7fffe7c8a888) at Objects/funcobject.c:632
#46 0x00007ffff7990189 in PyObject_Call (func=0x7ffff3ac86a8, arg=<optimized out>,
    kw=<optimized out>) at Objects/abstract.c:2040
#47 0x00007ffff7a5f236 in ext_do_call (nk=-462348024, na=0, flags=<optimized out>,
    pp_stack=0x7fffe4ff9638, func=0x7ffff3ac86a8) at Python/ceval.c:4561
#48 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2878
#49 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=6, kws=0x7ffff7f92060, kwcount=0, defs=0x7fffe7c8f7e0, defcount=4,
    kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#50 0x00007ffff79bff01 in function_call (func=0x7fffe7c98378, arg=0x7fffe7c8bb28,
    kw=0x7fffe7c97f48) at Objects/funcobject.c:632
#51 0x00007ffff7990189 in PyObject_Call (func=0x7fffe7c98378, arg=<optimized out>,
    kw=<optimized out>) at Objects/abstract.c:2040
#52 0x00007ffff7a5f236 in ext_do_call (nk=-406275288, na=0, flags=<optimized out>,
    pp_stack=0x7fffe4ff9948, func=0x7fffe7c98378) at Python/ceval.c:4561
#53 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2878
#54 0x00007ffff7a609b6 in fast_function (nk=<optimized out>, na=<optimized out>,
    n=<optimized out>, pp_stack=0x7fffe4ff9ab8, func=0x7ffff6049b70)
    at Python/ceval.c:4334
#55 call_function (oparg=<optimized out>, pp_stack=0x7fffe4ff9ab8)
    at Python/ceval.c:4262
#56 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#57 0x00007ffff7a609b6 in fast_function (nk=<optimized out>, na=<optimized out>,
    n=<optimized out>, pp_stack=0x7fffe4ff9c28, func=0x7ffff6049d90)
    at Python/ceval.c:4334
---Type <return> to continue, or q <return> to quit---
#58 call_function (oparg=<optimized out>, pp_stack=0x7fffe4ff9c28)
    at Python/ceval.c:4262
#59 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:2838
#60 0x00007ffff7a61e41 in PyEval_EvalCodeEx (_co=<optimized out>,
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>,
    argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3588
#61 0x00007ffff79bfde2 in function_call (func=0x7ffff6049bf8, arg=0x7fffe7804630,
    kw=0x0) at Objects/funcobject.c:632
#62 0x00007ffff7990189 in PyObject_Call (func=0x7ffff6049bf8, arg=<optimized out>,
    kw=<optimized out>) at Objects/abstract.c:2040
#63 0x00007ffff79aa26c in method_call (func=0x7ffff6049bf8, arg=0x7fffe7804630,
    kw=0x0) at Objects/classobject.c:347
#64 0x00007ffff7990189 in PyObject_Call (func=0x7fffe7808508, arg=<optimized out>,
    kw=<optimized out>) at Objects/abstract.c:2040
#65 0x00007ffff7a57e13 in PyEval_CallObjectWithKeywords (func=0x7fffe7808508,
    arg=0x7ffff7f92048, kw=<optimized out>) at Python/ceval.c:4114
#66 0x00007ffff7aa2842 in t_bootstrap (boot_raw=0xd789c0)
    at ./Modules/_threadmodule.c:999
#67 0x00007ffff7719182 in start_thread (arg=0x7fffe4ffa700) at pthread_create.c:312
#68 0x00007ffff6d3947d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb)

cc @jreback

cc @fonnesbeck

I've tried this before the changes to textblock in 3ce1de8  and the problem persists, exonerating those changes (sorry for the needless ping @cowlicks, please ignore), but not necessarily the textblock functionality.

I think I'm seeing a related issue in the 02-Foundations.ipynb tutorial, which demonstrates how to use the multiprocessing module to compute a graph containing calls to pandas.read_csv. However, I get a PicklingError when I run the below snippet.
I know this is caused by pandas.read_csv because I can replace it with my own csv->dataframe function and it works just fine. I'm running dask 0.7.5 and pandas 0.17.1 in python 3.5.1.
Has there been any progress on this?
dsk = {'a': (pd.read_csv, filenames[0]),
       'b': (pd.read_csv, filenames[1]),
       'c': (pd.read_csv, filenames[2]),
       'na': (len, 'a'),
       'nb': (len, 'b'),
       'nc': (len, 'c'),
       'total': (sum, ['na', 'nb', 'nc'])}

from dask.multiprocessing import get
%time get(dsk, 'total')

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save_global(self, obj, name)
    906             module = sys.modules[module_name]
--> 907             obj2, parent = _getattribute(module, name)
    908         except (ImportError, KeyError, AttributeError):

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in _getattribute(obj, name)
    264             raise AttributeError("Can't get local attribute {!r} on {!r}"
--> 265                                  .format(name, obj))
    266         try:

AttributeError: Can't get local attribute '_make_parser_function.<locals>.parser_f' on <function _make_parser_function at 0x10982a950>

During handling of the above exception, another exception occurred:

PicklingError                             Traceback (most recent call last)
<ipython-input-13-e7b1fa25a8d9> in <module>()
      1 from dask.multiprocessing import get
----> 2 get_ipython().magic("time get(dsk, 'total')")

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/IPython/core/interactiveshell.py in magic(self, arg_s)
   2334         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2335         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2336         return self.run_line_magic(magic_name, magic_arg_s)
   2337
   2338     #-------------------------------------------------------------------------

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line)
   2255                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2256             with self.builtin_trap:
-> 2257                 result = fn(*args,**kwargs)
   2258             return result
   2259

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/IPython/core/magics/execution.py in time(self, line, cell, local_ns)

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194
    195         if callable(arg):

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/IPython/core/magics/execution.py in time(self, line, cell, local_ns)
   1161         if mode=='eval':
   1162             st = clock2()
-> 1163             out = eval(code, glob, local_ns)
   1164             end = clock2()
   1165         else:

<timed eval> in <module>()

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dask/multiprocessing.py in get(dsk, keys, optimizations, num_workers, func_loads, func_dumps, **kwargs)
     54         # Run
     55         result = get_async(apply_async, len(pool._pool), dsk3, keys,
---> 56                            queue=queue, get_id=_process_get_id, **kwargs)
     57     finally:
     58         if cleanup:

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dask/async.py in get_async(apply_async, num_workers, dsk, result, cache, queue, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, **kwargs)
    462     # Seed initial tasks into the thread pool
    463     while state['ready'] and len(state['running']) < num_workers:
--> 464         fire_task()
    465
    466     # Main loop, wait on tasks to finish, insert new ones

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dask/async.py in fire_task()
    458         # Submit
    459         apply_async(execute_task, args=[key, dsk[key], data, queue,
--> 460                                         get_id, raise_on_exception])
    461
    462     # Seed initial tasks into the thread pool

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/toolz/functoolz.py in __call__(self, *args, **kwargs)
    217     def __call__(self, *args, **kwargs):
    218         try:
--> 219             return self._partial(*args, **kwargs)
    220         except TypeError:
    221             # If there was a genuine TypeError

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dask/multiprocessing.py in dill_apply_async(apply_async, func, args, kwds, func_loads, func_dumps)
     73     dumps = func_dumps or _globals.get('func_dumps') or dill.dumps
     74     sfunc = dumps(func)
---> 75     sargs = dumps(args)
     76     skwds = dumps(kwds)
     77     return apply_async(curry(apply_func, loads=func_loads),

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dill/dill.py in dumps(obj, protocol, byref, fmode, recurse)
    190     """pickle an object to a string"""
    191     file = StringIO()
--> 192     dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
    193     return file.getvalue()
    194

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dill/dill.py in dump(obj, file, protocol, byref, fmode, recurse)
    184             return
    185     # end hack
--> 186     pik.dump(obj)
    187     return
    188

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in dump(self, obj)
    406         if self.proto >= 4:
    407             self.framer.start_framing()
--> 408         self.save(obj)
    409         self.write(STOP)
    410         self.framer.end_framing()

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save(self, obj, save_persistent_id)
    473         f = self.dispatch.get(t)
    474         if f is not None:
--> 475             f(self, obj) # Call unbound method with explicit self
    476             return
    477

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save_list(self, obj)
    768
    769         self.memoize(obj)
--> 770         self._batch_appends(obj)
    771
    772     dispatch[list] = save_list

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in _batch_appends(self, items)
    792                 write(MARK)
    793                 for x in tmp:
--> 794                     save(x)
    795                 write(APPENDS)
    796             elif n:

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save(self, obj, save_persistent_id)
    473         f = self.dispatch.get(t)
    474         if f is not None:
--> 475             f(self, obj) # Call unbound method with explicit self
    476             return
    477

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save_tuple(self, obj)
    723         if n <= 3 and self.proto >= 2:
    724             for element in obj:
--> 725                 save(element)
    726             # Subtle.  Same as in the big comment below.
    727             if id(obj) in memo:

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save(self, obj, save_persistent_id)
    473         f = self.dispatch.get(t)
    474         if f is not None:
--> 475             f(self, obj) # Call unbound method with explicit self
    476             return
    477

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save_tuple(self, obj)
    723         if n <= 3 and self.proto >= 2:
    724             for element in obj:
--> 725                 save(element)
    726             # Subtle.  Same as in the big comment below.
    727             if id(obj) in memo:

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save(self, obj, save_persistent_id)
    473         f = self.dispatch.get(t)
    474         if f is not None:
--> 475             f(self, obj) # Call unbound method with explicit self
    476             return
    477

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/site-packages/dill/dill.py in save_function(pickler, obj)
    717     else:
    718         log.info("F2: %s" % obj)
--> 719         StockPickler.save_global(pickler, obj) #NOTE: also takes name=...
    720     return
    721

/Users/z001j0s/anaconda/envs/boa/lib/python3.5/pickle.py in save_global(self, obj, name)
    909             raise PicklingError(
    910                 "Can't pickle %r: it's not found as %s.%s" %
--> 911                 (obj, module_name, name))
    912         else:
    913             if obj2 is not obj:

PicklingError: Can't pickle <function _make_parser_function.<locals>.parser_f at 0x10982a9d8>: it's not found as pandas.io.parsers._make_parser_function.<locals>.parser_f

The original issue here is being resolved within Pandas at https://github.com/pydata/pandas/pull/11790
The issue you've brought up @jesaerys is a different, although also valid one.
I can reproduce it as follows:
In [1]: import pandas as pd
In [2]: import dill
In [3]: pd.read_csv
Out[3]: <function pandas.io.parsers._make_parser_function.<locals>.parser_f>
In [4]: dill.loads(dill.dumps(pd.read_csv))
PicklingError: Can't pickle <function _make_parser_function.<locals>.parser_f at 0x7f1df3395158>: it's not found as pandas.io.parsers._make_parser_function.<locals>.parser_f

cc @jreback
Generally speaking pandas functions turn out to be tricky to serialize.
Fortunately @jesaerys you can resolve your problem by upgrading dask to 0.7.6 (currently only available by pip install) which switches from using dill to using cloudpickle
In [5]: import cloudpickle
In [6]: cloudpickle.loads(cloudpickle.dumps(pd.read_csv))
Out[6]: <function pandas.io.parsers._make_parser_function.<locals>.parser_f>

Interesting. Thanks for explaining, @mrocklin.

The pandas work has been merged.  Closing

I have installed pandas from source (git commit fe201a2663a89913ac9b024b3864a53836a49d36). I no longer get the segmentation fault however I a new error:
In [1]: from dask import dataframe as dd

In [2]: columns = ["name", "amenity", "Longitude", "Latitude"]

In [3]: data = dd.read_csv('POIWorld.csv', usecols=columns)

In [4]: data.compute()
---------------------------------------------------------------------------
CParserError                              Traceback (most recent call last)
<ipython-input-4-9b35b7662143> in <module>()
----> 1 data.compute()

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/base.pyc in compute(self, **kwargs)
     35
     36     def compute(self, **kwargs):
---> 37         return compute(self, **kwargs)[0]
     38
     39     @classmethod

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/base.pyc in compute(*args, **kwargs)
    108                 for opt, val in groups.items()])
    109     keys = [var._keys() for var in variables]
--> 110     results = get(dsk, keys, **kwargs)
    111
    112     results_iter = iter(results)

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/threaded.pyc in get(dsk, result, cache, num_workers, **kwargs)
     55     results = get_async(pool.apply_async, len(pool._pool), dsk, result,
     56                         cache=cache, queue=queue, get_id=_thread_get_id,
---> 57                         **kwargs)
     58
     59     return results

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/async.pyc in get_async(apply_async, num_workers, dsk, result, cache, queue, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, **kwargs)
    479                 _execute_task(task, data)  # Re-execute locally
    480             else:
--> 481                 raise(remote_exception(res, tb))
    482         state['cache'][key] = res
    483         finish_task(dsk, key, state, results, keyorder.get)

CParserError: Error tokenizing data. C error: EOF inside string starting at line 186116

Traceback
---------
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/async.py", line 264, in execute_task
    result = _execute_task(task, data)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/async.py", line 246, in _execute_task
    return func(*args2)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask/dataframe/io.py", line 52, in _read_csv
    return pd.read_csv(block, **kwargs)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 506, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 293, in _read
    return parser.read()
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 749, in read
    ret = self._engine.read(nrows)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 1199, in read
    data = self._reader.read(nrows)
  File "pandas/parser.pyx", line 766, in pandas.parser.TextReader.read (pandas/parser.c:7988)
  File "pandas/parser.pyx", line 788, in pandas.parser.TextReader._read_low_memory (pandas/parser.c:8244)
  File "pandas/parser.pyx", line 842, in pandas.parser.TextReader._read_rows (pandas/parser.c:8970)
  File "pandas/parser.pyx", line 829, in pandas.parser.TextReader._tokenize_rows (pandas/parser.c:8838)
  File "pandas/parser.pyx", line 1833, in pandas.parser.raise_parser_error (pandas/parser.c:22649)

On dask master and pandas master I get the following:
In [1]: import dask.dataframe as dd

In [2]: columns = ["name", "amenity", "Longitude", "Latitude"]

In [3]: data = dd.read_csv('POIWorld.csv', usecols=columns)

In [4]: data.head()
Out[4]:
  name   amenity  Longitude   Latitude
0  NaN  post_box  -0.206980  51.945875
1  NaN  post_box  -0.268633  51.938183
2  NaN  post_box  -0.274278  51.930210
3  NaN  post_box  -0.262108  51.851700
4  NaN  post_box  -0.112959  51.899246

In [5]: data.compute()
Out[5]:
                               name           amenity   Longitude   Latitude
0                               NaN          post_box   -0.206980  51.945875
1                               NaN          post_box   -0.268633  51.938183
...
202967                          NaN           parking   19.134865  47.491048
202968                          NaN           parking         NaN        NaN

[9431401 rows x 4 columns]

Does your problem stop if you turn off threading?
import dask
dask.set_options(get=dask.async.get_sync)

I was using version 0.7.6. Just tried installing from master instead. I now get a slightly different error message:
In [1]: import dask.dataframe as dd

In [2]: dd.read_csv('POIWorld.csv', usecols=["name", "Longitude", "Latitude"]).compute()
---------------------------------------------------------------------------
CParserError                              Traceback (most recent call last)
<ipython-input-2-a94425e48dd7> in <module>()
----> 1 dd.read_csv('POIWorld.csv', usecols=["name", "Longitude", "Latitude"]).compute()

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/base.pyc in compute(self, **kwargs)
     35
     36     def compute(self, **kwargs):
---> 37         return compute(self, **kwargs)[0]
     38
     39     @classmethod

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/base.pyc in compute(*args, **kwargs)
    108                 for opt, val in groups.items()])
    109     keys = [var._keys() for var in variables]
--> 110     results = get(dsk, keys, **kwargs)
    111
    112     results_iter = iter(results)

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/threaded.pyc in get(dsk, result, cache, num_workers, **kwargs)
     55     results = get_async(pool.apply_async, len(pool._pool), dsk, result,
     56                         cache=cache, queue=queue, get_id=_thread_get_id,
---> 57                         **kwargs)
     58
     59     return results

/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/async.pyc in get_async(apply_async, num_workers, dsk, result, cache, queue, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, **kwargs)
    479                 _execute_task(task, data)  # Re-execute locally
    480             else:
--> 481                 raise(remote_exception(res, tb))
    482         state['cache'][key] = res
    483         finish_task(dsk, key, state, results, keyorder.get)

CParserError: Error tokenizing data. C error: EOF inside string starting at line 295078

Traceback
---------
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/async.py", line 264, in execute_task
    result = _execute_task(task, data)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/async.py", line 246, in _execute_task
    return func(*args2)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/dask-0.7.6-py2.7.egg/dask/dataframe/io.py", line 53, in _read_csv
    return pd.read_csv(block, **kwargs)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 506, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 293, in _read
    return parser.read()
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 749, in read
    ret = self._engine.read(nrows)
  File "/home/toby/sandbox/dask/anaconda/lib/python2.7/site-packages/pandas-0.17.1+316.gfe201a2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 1199, in read
    data = self._reader.read(nrows)
  File "pandas/parser.pyx", line 766, in pandas.parser.TextReader.read (pandas/parser.c:7988)
  File "pandas/parser.pyx", line 788, in pandas.parser.TextReader._read_low_memory (pandas/parser.c:8244)
  File "pandas/parser.pyx", line 842, in pandas.parser.TextReader._read_rows (pandas/parser.c:8970)
  File "pandas/parser.pyx", line 829, in pandas.parser.TextReader._tokenize_rows (pandas/parser.c:8838)
  File "pandas/parser.pyx", line 1833, in pandas.parser.raise_parser_error (pandas/parser.c:22649)

I get the exact same behavior with threading turned off.
I'm using:
Python 2.7.11
Anaconda 2.5.0 (64-bit)
Ubuntu 14.04 LTS
I'm using the csv extracted from POIWorld_2<IP_ADDRESS>z

Yeah, ok, I can reproduce this (though I get a slightly different error).
@jreback this works for me fine at Pandas commit 567bc5ceb33d2147ca68b6eee9b180e6059ae247 (the thread safety fix) but fails on master.  Has something else happened around thread safety and read_csv recently in Pandas?

@mrocklin I downloaded the link at the very top and got these:
(python3)bash-3.2$ ls
POIWorld_2<IP_ADDRESS>z  POIWorld_2<IP_ADDRESS>z
(python3)bash-3.2$ ls -ltr
total 603216
-rw-rw-r--@ 1 jreback  staff  153624635 Sep 27 09:15 POIWorld_2<IP_ADDRESS>z
-rw-rw-r--@ 1 jreback  staff  155216424 Oct 15 15:50 POIWorld_2<IP_ADDRESS>z

what are these files? How do I get the same CSV you are working on

7z is a compression format.  I think I have a command line utility 7z on ubuntu.

In [4]: pd.__version__
Out[4]: '0.17.1+322.g81e3303'

In [6]: import dask

In [7]: dask.__version__
Out[7]: '0.7.6'

In [1]: import dask.dataframe as dd

In [2]: In [2]: dd.read_csv('POIWorld.csv', usecols=["name", "Longitude", "Latitude"]).compute()
Out[2]:
                               name   Longitude   Latitude
0                               NaN   -0.206980  51.945875
1                               NaN   -0.268633  51.938183
2                               NaN   -0.274278  51.930210
3                               NaN   -0.262108  51.851700
4                               NaN   -0.112959  51.899246
5                               NaN   10.834091  52.891618
6                     The Green Man   -0.143209  51.523681
7                         ÐÐµÑÑÑÐ¾ÑÐºÐ°   37.207465  55.995082
8                   Olav Tryggvason   10.395062  63.430501
9                               NaN   10.789311  59.967280
10                              NaN   10.789880  59.968475
11                              NaN   10.763690  59.958118
12                              NaN   10.854071  59.942319
13                      GrÃ¸nliveien   10.894301  59.934216
14                             Stig   10.817163  59.947828
15                         Hammeren   10.751901  59.993727
16                          Hallset   10.784337  60.024025
202963                          NaN    9.648820  47.280167
202964               Sandrose Hotel   28.852084  28.336509
202965                          NaN   19.136683  47.491223
202966                          NaN   19.137172  47.490743
202967                          NaN   19.134865  47.491048
202968                          NaN         NaN        NaN

[9431401 rows x 3 columns]

this is macosx

this worked for me on py27 as well.
note, that when I did conda update dask this downgraded pandas to 0.17.1, so I then had to subsequently install from master.
not exactly sure what that was.
so make sure that you are:
conda install dask
conda remove pandas
conda install pandas-from-conda-bld-dir

Anaconda sometimes pins package versions.  The dask conda recipes aren't restricting pandas to lower versions.

I am using miniconda here. but its still odd. I must have something else installed that forces that.

in any event going to post an RC on friday, so people can simply install from that and see if this repros.

Looking forward to the RC.  I think I've seen a number of small error pop up in dask.dataframe tests when running from master.

Still experiencing the same issue, using:

OSX Yosemite
Python 2.7
conda 3.19.1

and:

dask:   0.8.2-py27_0
pandas: 0.18.0-np111py27_0

Log:

---------------------------------------------------------------------------
CParserError                              Traceback (most recent call last)
<ipython-input-20-91435fb470e4> in <module>()
      1 from dask import dataframe as dd
      2 columns = ["name", "amenity", "Longitude", "Latitude"]
----> 3 data = dd.read_csv('POIWorld.csv', usecols=columns).compute()
      4 data

/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/base.py in compute(self, **kwargs)
     35
     36     def compute(self, **kwargs):
---> 37         return compute(self, **kwargs)[0]
     38
     39     @classmethod

/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/base.py in compute(*args, **kwargs)
    108                 for opt, val in groups.items()])
    109     keys = [var._keys() for var in variables]
--> 110     results = get(dsk, keys, **kwargs)
    111
    112     results_iter = iter(results)

/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/threaded.py in get(dsk, result, cache, num_workers, **kwargs)
     55     results = get_async(pool.apply_async, len(pool._pool), dsk, result,
     56                         cache=cache, queue=queue, get_id=_thread_get_id,
---> 57                         **kwargs)
     58
     59     return results

/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/async.py in get_async(apply_async, num_workers, dsk, result, cache, queue, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, **kwargs)
    479                 _execute_task(task, data)  # Re-execute locally
    480             else:
--> 481                 raise(remote_exception(res, tb))
    482         state['cache'][key] = res
    483         finish_task(dsk, key, state, results, keyorder.get)

CParserError: Error tokenizing data. C error: EOF inside string starting at line 295078

Traceback
---------
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/async.py", line 264, in execute_task
    result = _execute_task(task, data)
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/async.py", line 246, in _execute_task
    return func(*args2)
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/dask/dataframe/io.py", line 53, in _read_csv
    return pd.read_csv(block, **kwargs)
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/pandas/io/parsers.py", line 529, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/pandas/io/parsers.py", line 305, in _read
    return parser.read()
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/pandas/io/parsers.py", line 763, in read
    ret = self._engine.read(nrows)
  File "/Users/rparra/anaconda/envs/pydata/lib/python2.7/site-packages/pandas/io/parsers.py", line 1213, in read
    data = self._reader.read(nrows)
  File "pandas/parser.pyx", line 766, in pandas.parser.TextReader.read (pandas/parser.c:7988)
  File "pandas/parser.pyx", line 788, in pandas.parser.TextReader._read_low_memory (pandas/parser.c:8244)
  File "pandas/parser.pyx", line 842, in pandas.parser.TextReader._read_rows (pandas/parser.c:8970)
  File "pandas/parser.pyx", line 829, in pandas.parser.TextReader._tokenize_rows (pandas/parser.c:8838)
  File "pandas/parser.pyx", line 1833, in pandas.parser.raise_parser_error (pandas/parser.c:22649)

Hrm, unfortunately I'm having difficulty replicating on my machine
In [1]: import dask.dataframe as dd

In [2]: columns = ["name", "amenity", "Longitude", "Latitude"]

In [3]: df = dd.read_csv('POIWorld.csv', usecols=columns)

In [4]: len(df)
Out[4]: 9431401

In [5]: import dask; dask.__version__
Out[5]: '0.8.2'

In [6]: import pandas; pandas.__version__
Out[6]: u'0.18.0'

In [7]: import numpy; numpy.__version__
Out[7]: '1.11.0'

In [8]: import sys; sys.version_info
Out[8]: sys.version_info(major=2, minor=7, micro=11, releaselevel='final', serial=0)


Error out of the box

Ran
npx degit chromaui/intro-storybook-vue-template taskbox
Got
! could not fetch remote https://github.com/chromaui/intro-storybook-vue-template
! could not find commit hash for HEAD
Not finding many things to try w/ Google, anyone know what the issue is? Trying to do the Vue Storybook tutorial, this is step 1 :D

same i have that kinda error too

make sure you have git installed. If you use VS Code change the terminal to Git Bash

make sure you have git installed. If you use VS Code change the terminal to Git Bash

Will try that and report back. I tend to use GitHub Desktop instead of the command line. Thanks.

make sure you have git installed. If you use VS Code change the terminal to Git Bash

So it looks like I did have Git installed already in addition to GitHub Desktop. Also for some reason trying it today in an identical manner ( just copying and pasting the commands from the tutorial ) seemed to work. Maybe there were just some issues w/ my network at the time or something, though I did try it on multiple different days and it failed identically each time. Whatever the case it's working now and I'm excited to dig in. Thanks again for the assist.


[JS Renderes][Accessibility] [Focus Lost for ToggleVisibility]

JS Renderer
We are using JSRenderer to render Adaptive Cards into website as well as we are sending Adaptive Cards to Outlook. We are also going to send same adaptive cards to Teams as well
Version of JS Adaptive cards is 2.4.0
Details
We are using Toggle Visibility to achieve expand and collapse functionality. Now when we try to use tab key to reach to expand image and then press key to expand, focus is getting lost. However expected result is to set focus to toggled visible control i.e. collapsed image in this case.
It will be great help if we can get suggestion on how we can set focus of collapse image once we expand using key pressed event.

@dclaux possible for you to take a first pass at this and feasibility of a fix/workaround?

@shalinijoshi19 @niravk-microsoft There is no easy fix to this. We have done work to lose focus if the element being clicked (i.e. the element that triggers to ToggleVisibility) remains displayed after being clicked, but it is not the case in this scenario, where the "expand" button is made invisible and a "collapse" button is made visible.
The only potential solution I can think of would be to:

If the element being clicked on is still visible after ToggleVisibility has executed, focus stays on it
Otherwise, pick the first focusable visible element from the list of elements in the targetElements collection and give it focus
If there is no such element, pick the first focusable visible element in the card and give it focus
If there is no such element, focus is lost

If this is an acceptable algorithm, we will have to implement it across all renders. Note that it will not always produce the desired result.
The alternative to this solution is to introduce a new property in the schema to allow the card author to specify which element should have focus on completion of the ToggleVisibility action. But that would be a new feature, not a bug.

I think it will be great if we can try two phased approach i.e. start with the fix for potential solution and then later add feature as an alternative solution for the same. In that way, current accessibility bug gets fixed and in future as a part of feature, we are giving control to Host of card to decide where to set focus on toggle visibility.

@niravk-microsoft even the first solution is something that is new unplanned work and effectively will need to be discussed and then estimated before we can get you a timeline for this.. Can you help us understand the relative priority of this bug for you?

Toggle accessibility is inherently impossible to solve fully for accessibility. We should look into options here to provide a better experience in these scenarios.

I suggest simply adding a moveFocusTo optional property that can be set to a prioritized array of element ids. After the ToggleVisibility is executed, the focus is moved to the first focusable element in the array.
Example:
The below card switches from showing "input1" and "input2". With the specified moveFocusTo property, focus is moved to "input1" when "input2" is hidden and the other way around.
{
    "type": "AdaptiveCard",
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.3",
    "body": [
        {
            "type": "Input.Text",
            "placeholder": "Placeholder text",
            "id": "input1",
            "label": "Input 1"
        },
        {
            "type": "Input.Text",
            "placeholder": "Placeholder text",
            "id": "input2",
            "label": "Input 2",
            "isVisible": false
        }
    ],
    "actions": [
        {
            "type": "Action.ToggleVisibility",
            "title": "Tottle between input 1 and input 2",
            "targetElements": [ "input1", "input2" ],
            "moveFocusTo": [ "input1", "input2" ]
        }
    ]
}

We're getting conflicting info from different accessibility folks; closing until we can close on the right direction


Feature: Backup data read from stdin

This issue tracks implementing the feature that restic reads data from stdin and saves it to the repo (with dedup etc).
I can imagine several use cases, e.g.:

Saving SQL dumps directly: mysqldump db | restic backup -
Saving complete devices: dd if=/dev/sda | restic backup -

Before implementing this, we need to decide how to save this in the json data structures. I'd like restic to behave consistent with this, e.g. when the repo is mounted with the fuse backend, data read from stdin should appear as a file. I'd also like to be able to write this data to a file.
Any other thoughts or use cases that haven't been mentioned so far? Do you like the syntax (restic backup -), or should we rather use restic backup --stdin?

The shorthand for stdin - is good but what about making it optional with something like mysqldump db | restic backup. It's closer from pipe chains such as cat foo | grep bar | sort | less.

I'm a fan of - for explicitness. If a shells script uses a variable to specify the target, and that target turns out to be some reason (by mistake or due to filesystem errors or whatever), I'd like an error message and a negative return value to be cast rather than having it block for input and uploading that.

I just had a look at this issue and at the way chunking is implemented. Restic reads files twice, once for chunking and once for persisting. ATM, if we want to persist data from stdin without refactoring the chunker, the best solution would probably be to save stdin in a temporary file and to backup this file. WDYT?

                                                                                Sounds like a lot of writing to do, and requiring disk space. This versus just The normal reads and no disk space :)                                                                                Från: Bertil ChapuisSkickat: fredagen den 28:e augusti 2015 23:01Till: restic/resticSvara till: restic/resticÄmne: Re: [restic] Feature: Backup data read from stdin (#255)I just had a look at this issue and at the way chunking is implemented. Restic reads files twice, once for chunking and once for persisting. ATM, if we want to persist data from stdin without refactoring the chunker, the best solution would probably be to save stdin in a temporary file and to backup this file. WDYT?

—Reply to this email directly or view it on GitHub.

Yes, but I don't think it's possible to read from stdin twice without writing to a temporary location. If the typical use case is a mysql dump, then keeping everything in memory until the second read may not be feasible.

if we want to (...) without refactoring the chunker

Why do we not want to refactor the chunker? If that's the cleaner solution, lets do that.

Absolutely! If the chunker interface needs to be refactored, let's do that! It's our own component, not a holy cow ;)

What I would like to see is the ability to backup (and then restore!) special files in general, not just stdin. My immediate use case is backup of the VM image residing in a LVM logical volume. Currently I don't see a way of backing my image with restic, except for copying it to a temporary file.
Out of three similar backup programs - attic, borg and restic - only borg supports this much-needed and obvious feature with –read-special flag on backup and --stdout flag on restore.

Regarding reading files twice. I think this is a bad thing for at least two reasons:

Reading data twice slows things down. In many cases this won't be noticeable due to caching, or because read speed is not a limiting factor. On the other hand, I can easily imagine how it can become a limiting factor for a multi-core machine with not-so-fast disks doing backups of large files.
If a file is being changed during backup, the second read pass will possibly see different data which may potentially confuse the backup algorithms to the point of crash, invalid backup or repository corruption.

These are just speculative worst cases. But even if the algorithms are explicitly protected from changing data, this protection may be incomplete, or introduce some undesired side effects.
Of course, in a perfect world we would only backup frozen data, but in real world enforcing this requirement would be too cumbersome in many cases (personally I'm only prepared to go as far as creating a snapshot for my VM volume, but not snapshotting and remounting all of my filesystems), which would violate the first design goal.

I think going forward and refactoring restic to do single-pass is a good thing, including refactoring the chunker interface. This simplifies the design, gives us a better handle at resource (especially memory usage) by limiting the number of large buffers available and makes reading e.g. from stdin possible.
Is it still necessary to have something like borg's --read-special? Or is reading from stdin sufficient?

A separate flag is definitely worth it:

We may need stdin for other purposes (password entry, anyone?).
Specifying a special file on the command line will yield a sensible filename/target recorded in the snapshot.

Your second argument is convincing.

Hello,
At the moment does deduplication work with stdin snapshots?

Yes


Incorrect code collapse if the text has no indentation

Version: 1.43.0
Commit: 78a4c91400152c0f27ba4d363eb56d2835f9903a
Date: 2020-03-09T19:44:52.965Z
Electron: 7.1.11
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Linux x64 5.3.0-40-generic

Code to reproduce:
if (true) {
  const one = 1;
  const text = `first line
second line without indentation
`;
}

Result collapsing by curly braces

As you can see, the collapse is incorrect due to the lack of indentation in the second line of text.
Generally, the same behavior is observed when there is no indentation near any code within the curly braces. But no indentation in the text has the necessary use case.

Does this issue occur when all extensions are disabled?: Yes

What's the language mode of the editor?

@aeschli, typescript

@KostyaTretyak have you seen the info here? https://code.visualstudio.com/docs/editor/codebasics#_folding
In particular:

Folding regions are by default evaluated based on the indentation of lines. A folding region starts when a line has a smaller indent than one or more following lines, and ends when there is a line with the same or smaller indent.

Since the 1.22 release, folding regions can also be computed based on syntax tokens of the editor's configured language. The following languages already provide syntax aware folding: Markdown, HTML, CSS, LESS, SCSS, and JSON.

@gjsjohnmurray, thanks for pointing to this documentation, I have read it now. However, I continue to view this as unexpected behavior. It is logical to assume that when you try to collapse the code in curly brackets, it collapses in curly brackets, not through indentation.

Typescript has language aware folding and had an understanding of multi line strings.
Here's what I see when I try your example:

Please verify that your editor mode (lower right) is set to TypeScript.

Did you change the setting `"editor.foldingStrategy"?
It should be "editor.foldingStrategy": "auto"

I can't imagine when and for what I did it, but I had "editor.foldingStrategy": "indentation". I apologize for the time you have spent.


Bootstrap 5 installation instructions with webpack + incompatibility with PopperJS v2

The installation instructions in the readme do not cover clearly how you can use this with webpacker, so I thought to write the instructions for the steps I went through.

First, you need to install PopperJS version 1 which is not told in the installation instructions. You can get the multiselect working also with version 2 of PopperJS but you will see a bunch of errors in the console if you try to do that (more on that below).

npm i popper.js

Obviously, you also need to install this package

npm i @dashboardcode/bsmultiselect

You'll need to initiate the BsMultiSelect factory method in order to attach the input elements with it

import Popper from "popper.js";
import { ModuleFactory as bsMultiSelectFactory } from "@dashboardcode/bsmultiselect";

// in case the library is updated to be compatible with PopperJS version 2, you can replace this with:
// import { createPopper } from "@popperjs/core";
const createPopper = (ref, pop, opts) => new Popper(ref, pop, opts);
const multiSelectEnv = {window, createPopper};
const multiSelect = bsMultiSelectFactory(multiSelectEnv).BsMultiSelect;

Finally, you can attach the BsMultiSelect to the elements by using the just initiated multiSelect factory method

// With vanilla JS
window.addEventListener("DOMContentLoaded", () => {
  document.querySelectorAll("select[multiple]").forEach((select) => {
    multiSelect(select);
  });
});

// Or if you are using jQuery
$(() => {
  $("select[multiple]").each((_i, select) => {
    multiSelect(select);
  });
});

If you are trying to use PopperJS version 2, you can import the createPopper method directly from the package as described in the readme and the multiselect element will still work but you will see the following errors in the console:
validateModifiers.js:62 PopperJS: an invalid property has been provided to the "undefined" modifier, valid properties are "name", "enabled", "phase", "fn", "effect", "requires", "options"; but "preventOverflow" was provided.
validateModifiers.js:62 PopperJS: an invalid property has been provided to the "undefined" modifier, valid properties are "name", "enabled", "phase", "fn", "effect", "requires", "options"; but "hide" was provided.
validateModifiers.js:62 PopperJS: an invalid property has been provided to the "undefined" modifier, valid properties are "name", "enabled", "phase", "fn", "effect", "requires", "options"; but "flip" was provided.

This is because BsMultiSelect is not compatible with PopperJS version 2 at this point. This seems to be because of the way the modifiers option is passed to the createPopper method here:
https://github.com/DashboardCode/BsMultiSelect/blob/dda6c9246aa628ddeeb5e77460590e184d209901/js/src/plugins/CreatePopperPlugin.js#L20-L24
This is not according to the version 2 documentation:
https://popper.js.org/docs/v2/modifiers/
But if you take a look at the version 1 documentation, it seems to be in line with that:
https://popper.js.org/docs/v1/#Popper.Defaults

Thank you for this great report.  I will improve this soon, and give you more details on using BsM swith webpack (to include only those plugins you need).
But one note ahead:

const multiSelect = (new BsMultiSelectFactory(multiSelectEnv)).BsMultiSelect;

There should be not need for "new". BsMultiSelectFactory is just a method

There should be not need for "new". BsMultiSelectFactory is just a method

True that. I updated the instructions above.

@ahukkanen
can you test new version 1.1.16 ?
I was unable to reproduce warning messages, but have changed popper "modifiers" according newest popper documentation.

@rpokrovskij
Thanks a lot! I tested 1.1.16 and it works with Popper.JS version 2 without any console errors.
However, the usage instructions are a bit different than what you wrote above. Here is how I got it working:
import { createPopper } from "@popperjs/core";
import { ModuleFactory as bsMultiSelectFactory } from "@dashboardcode/bsmultiselect";

const multiSelectEnv = {window, Popper: createPopper};
const multiSelect = bsMultiSelectFactory(multiSelectEnv).BsMultiSelect;

There is no export named Popper in the @popperjs/core package, so I figured you meant the createPopper method.
Also, with Popper.js version 1, there is no createPopper export in the package, so you need to create the method yourself as explained in the original post.
As for coding conventions, I think it would be clearer if with version 2 we could provide the createPopper method for the environment and with version 1 the Popper constructor object would be provided. The current convention is quite confusing and it took me a while to figure it out.

@ahukkanen  Yes you are right I have wrote it wrong I'm sorry
view-source:https://dashboardcode.github.io/BsMultiSelect/snippetEsm.html
import {createPopper} from "https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>view-source:https://dashboardcode.github.io/BsMultiSelect/snippetEsmBs4.html
import Popper from<EMAIL_ADDRESS>I have intentionaly reserved createPopper and Popper property names as environment object's  for different Popper versions. This  inconsistency is "inherited" from popper. Different properties is also the "flag" to me : how to create popper's modifiers as for V1 or for V2.
"One moment" I will just stop support popper V1 and the inconsistency will be gone this way.

Just FYI, I tested 1.17.0 + I bumped Popper.js to 2.10.0, and now it works as advertised without the initialization inconsistency and without any errors in the console.
When I tested 1.17.0 + Popper.js 2.9.3, I still got the console errors when trying to provide the createPopper name to the environment variable.
I'm wondering if the problem was at Popper.js, haven't looked further into it. There were some changes in Popper.js 2.10.0 touching the options definitions but it didn't look like it would be related.

Thank you. Going to experiment with popper.js 2.9.3 .  Could you send me your snipped project with webpack.config

Here is my webpack.config.js:
/* eslint-disable no-ternary, no-process-env */

const path = require("path");
const TerserPlugin = require("terser-webpack-plugin");
const PostCSSPresetEnv = require("postcss-preset-env");
const CssMinimizerPlugin = require("css-minimizer-webpack-plugin");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const StylelintPlugin = require("stylelint-webpack-plugin");
const { WebpackManifestPlugin } = require("webpack-manifest-plugin");
const ESLintPlugin = require("eslint-webpack-plugin");

const isDev = process.env.NODE_ENV !== "production";

const esLintOptions = {
  extensions: ["js"],
  exclude: [
    "node_modules",
    "./config",
  ],
}

module.exports = {
  mode: isDev
    ? "development"
    : "production",
  stats: {
    colors: true,
    preset: "minimal",
  },
  ignoreWarnings: [
    {
      file: /bootstrap/,
    },
  ],
  performance: {
    hints: isDev
      ? false
      : "warning",
  },
  devtool: isDev
    ? "cheap-module-source-map"
    : "source-map",
  entry: [
    path.resolve(__dirname, "src/scripts/main.js"),
    path.resolve(__dirname, "src/styles/main.scss"),
  ],
  output: {
    publicPath: "/assets/",
    path: path.resolve(__dirname, "dist/assets"),
    filename: isDev
      ? "[name].js"
      : "[name].[contenthash].js",
  },
  plugins: [
    new ESLintPlugin(esLintOptions),
    new WebpackManifestPlugin(),
    new StylelintPlugin({
      files: "src/styles/**/*.scss",
    }),
    new MiniCssExtractPlugin({
      filename: isDev
        ? "[name].css"
        : "[name].[contenthash].css",
    }),
  ],
  ...(!isDev && {
    optimization: {
      minimizer: [
        (compiler) => {
          Reflect.apply(
            new TerserPlugin({
              terserOptions: {
                compress: {},
              },
            }),
            compiler
          );
        },
        new CssMinimizerPlugin(),
      ],
    },
  }),
  module: {
    rules: [
      {
        enforce: "pre",
        test: /\.scss/,
        loader: "import-glob-loader",
      },
      {
        test: /\.js$/,
        exclude: /node_modules/,
        loader: "babel-loader",
      },
      {
        test: /\.s?css/i,
        use: [
          MiniCssExtractPlugin.loader,
          "css-loader",
          {
            loader: "postcss-loader",
            options: { postcssOptions: { plugins: [PostCSSPresetEnv] } },
          },
          "sass-loader",
        ],
      },
      {
        test: /\.(png|jpe?g|gif|svg)$/i,
        type: "asset/resource",
        generator: {
          filename: `images/${isDev
            ? "[name][ext]"
            : "[contenthash][ext]"}`,
        },
      },
      {
        test: /\.(woff|woff2|eot|ttf|otf)$/i,
        type: "asset/resource",
        generator: {
          filename: `fonts/${isDev
            ? "[name][ext]"
            : "[contenthash][ext]"}`,
        },
      },
    ],
  },
  resolve: {
    alias: {
      // Helpful alias for importing assets
      assets: path.resolve(__dirname, "src/assets"),
    },
  },
};

and how you reference BsMultiSelect in your index.js?

I have the following dependencies at package.json (right now after the updates):
  "dependencies": {
    "@dashboardcode/bsmultiselect": "^1.1.17",
    "@fortawesome/fontawesome-free": "^5.15.3",
    "@popperjs/core": "^2.10.0",
    "bootstrap": "^5.0.1",
    "jquery": "^3.6.0"
  }

And of course a lot of dev dependencies, such as webpack, postcss, babel, linters, etc. But I don't think they are relevant regarding this.

Thank you.
Here view-source:https://dashboardcode.github.io/BsMultiSelect/snippetEsm.html popper  Popper.js 2.9.3 but there are no warningns. I hope with your help I will be able to reproduce them...

Got it. There is difference referencing min and not min Popper versions. "Not min" throw those warnings.

solved in 1.1.18


Native Create Issue

Closes #288
Closes #350

Created view controller to natively create an issue supporting a title and body text
Pass through the correct GithubClient to Settings VC (necessary for correct authentication)
Updated "Report a bug" tool to use issue creator (currently using a test repo)
On a successful creation, the user is segued to the issues view controller (replacing new issue, so they can't go back to the form!)

Example output: https://github.com/rnystromtest/Githawk-Playground/issues/2
Used API: https://developer.github.com/v3/issues/#create-an-issue

Added new issue button to all repositories that have issue enabled
Added markdown/action bar (includes a preview button)
Fixed issues on tablet so things appear in the correct panel

Still looks like 💩 - If anyone wants to give a hand making it look nice I would appreciate it 😅

I'd say for an initial implementation I'm happy with it now - Obviously we can add new functionality down the line for adding labels, assignees etc (assuming you have push permissions) but I think this is good?

Regarding design: a few years back I did something similar regarding native reporting, and think that looked pretty nice. Obviously a bit different in comparison with creating an issue, but design wise you get the idea.

@BasThomas You're more than welcome to branch off and try match that!
It looks a look more modular, and similar to Eureka just couldn't see a nice way of doing it without ballooning hence with the simpler scroll view

Will give this a go soon, def need to massage the design. I personally prefer the look of the stuff @BasThomas posted: grouped UITableView style, tho it might be overkill for title+body. I'm not sure yet the best way to incorporate the editing tools. Input accessory view is prob the best bet for this.

Design has never been a hotspot of mine, and editing tools are definitely up for discussion (I basically just copied what we had already to minimise on work!)
This does look quite nice though: http://1writerapp.com/
In essence a custom keyboard

Probably also want to add this option for repos you’re browsing via search. Will create a separate issue.

This is available via search and for report a bug - am I missing something @BasThomas
Sent with GitHawk

I don’t get it. 😬 What are you referring to?
I am expecting, if you look at the screenshot below, to also have an option to “Open an Issue”

@rnystrom Just updated with changes in master so the "Create Issue" is now in the overflow rather than it's own button
In terms of design, https://github.com/xmartlabs/Eureka would give us a basic form with all the structure done for us -- Would this be a solution you're okay with?

Is that project gonna be overkill tho? Will we use it for more stuff in the future? Nervous about taking another dependency for a simple design.
Sent with GitHawk

Further down the line I'm sure the form will get more complicated with files/images, labels, assignees, etc
Other forms include add/edit projects, maybe even editing repos
I'll give it a quick go now without Eureka and see how it ends up

@rnystrom

You mentioned before that we need to come up with a better way of doing markdown so I left the buttons out of this design. Is this what you had in mind?

@BasThomas ^^ also

Ya that looks pretty great, nice and simple. We should be able to add the markdown actions as the input accessory view, right? @Sherlouk can you confirm that's still possible here?

It should still be possible yea (it’s just a text view!) would we still want all the options though because it already has all the copy, paste, look up etc I feel like that Ui is soon going to get super busy
Sent with GitHawk

Added markdown controls ^^
@rnystrom @BasThomas

Yessss!

@rnystrom We waiting on anything to merge this now? 😄

Nice!

@Sherlouk don’t think so! I’ll give the code a quick look, but I also don’t mind refactoring anything on my own. Will merge today!
Sent with GitHawk

@rnystrom Hit me up when ready to merge and I can deal with the conflicts

@Sherlouk give it a go and DM me on Twitter so I can merge ASAP to avoid more conflicts!

@rnystrom Picking this up now, will be 5 minutes

All good!

May I ask why you have chosen a UITableView for this and not a UICollectionView or let's go wild and a UIStackView? I am still trying to find out when to use what. I really like UICollectionView myself as it's more powerful with it custom layout's support.

Originally it was a UIScrollView with a couple elements in, ultimately I would've been happy with a bit of auto layout magic but crucially we still needed to support all the screen sizes so scrolling was a must
In the end it came down to simplicity and "out of the box" design. A grouped UITableView gives us a consistent design not only across the app but also familiarity across iOS. Using static cells makes it super easy to implement and all values can be accessed without the need for a maze of delegates and hierarchy calls.
In this situation there was very little flexibility required, though when we do get the point of adding additional fields, such as labels, this may have to change.


[BUG] Unexpected state ordering in counts

Not sure if this should be treated as a bug or a documentation update.
Issue description

Expected behavior:

Consider the circuit
@qml.qnode(qml.device("lightning.qubit", wires=2, shots=1000))
def counts():
    qml.RX(np.pi/2, 0)
    qml.RX(np.pi/4, 1)
    return qml.counts()

In plain Pennylane, the 1st number in the string represents qubit 0, and the second qubit 1:
>>> counts()
{'00': 437, '01': 80, '10': 423, '11': 60}

I would expect the same from Catalyst.

Actual behavior:

Using the method suggested the catalyst docs (see first example here) to convert the Catalyst indices to states, qubit 0 and 1 are swapped when using Catalyst:
>>> basis_states, count_res = qjit(counts)()
>>> {format(int(state), '01b'): count for state, count in zip(basis_states, count_res)}
{'0': 393, '1': 463, '10': 62, '11': 82}

The missing padded zeros on the left hand side are inconvenient, but the swapping of 01 and 10 is the main issue.

Reproduces how often:
100%

System information:
pl dev and catalyst dev

Thanks for reporting this! Ideally we'll find a better way to return counts from Catalyst functions. With PyTree support, it might now be possible to just return the same dictionary that PennyLane returns :)
The other problem is the classic issue of endianness with quantum bit strings, always a headache 😞

Just to a minor comment on the second point raised in this issue (the endianness is fixed by the linked PR):
The get the padding right you have to set the number of qubits in the formatter string. The docs example only has one qubit, so format(int(state), '01b') gives the correct results. For your example with 2 qubits you would have to use format(int(state), '02b').


How to set number of toast on screen ?

how can i set to show only one toast at a time ?

@alielmajdaoui @retyui

why are you tagged me?

why are you tagged me?
You are not contributor ?

I am a contributor, but not a maintainer
On Sun, May 1, 2022, 11:08 Muhammad Mukthayar @.***>
wrote:

why are you tagged me?
You are not contributor ?
—
Reply to this email directly, view it on GitHub
https://github.com/alielmajdaoui/react-native-gtoast/issues/2#issuecomment-1114178667,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDSEGFWNPVDSBFGWDDGTILVHZCYBANCNFSM5UYYYFCA
.
You are receiving this because you were mentioned.Message ID:
@.***>

If you set an id for the toast, it will be shown once until it disappear.
https://github.com/alielmajdaoui/react-native-gtoast#unique-toasts


[BUG]

Is there an existing issue for this?

[X] I have searched the existing issues

Does this issue exist in the latest version?

[X] I'm using the latest release

Describe the bug?
I deploy branch1 to master, then immediately deploy branch2 to master.
Both deployments have GHA functionality where they check for file changes using this GHA changed-files functionality.
I have set fetch-depth: 0
Observations:

This runs fine when just 1 branch is deployed to master at a time.
When I rerun failed job manually - it fails again consistently.

this library is trying to get changed file names between

Pushed commit: commit that triggered this workflow on gha
2nd commit from head of the branch.

Since I immediately push another commit to dev branch before this first commits workflow to check file-changed starts - we see that both SHA are same for git diff

git diff commit_2 commit_2

Hence we get this error.
To Reproduce
Create branch1 from master.
Then create branch2 from master.
Add some unique changes to both branches.
Write some GHA code job as below:
jobs:
  seed_data_changed:
    name: Check if seed data files changed
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0 # fetch all history of all branches

      - name: Get changed files
        id: changed-files
        uses: tj-actions/changed-files@v31

What OS are you seeing the problem on?
ubuntu-latest or ubuntu-20.04
Expected behavior?
No error should be shown.
We should see file changes as usual.
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @aseem-heg Can you include the full logs output with debugging enabled to help resolve this issue faster?

Why are you closing the issue?

@aseem-heg This issue has been resolved in the latest release.