fix #70 first monadic law

by checking the arguments count, we can determine if a value was passed at all

Closing as this is fixed in https://github.com/qworks-io/monads/pull/72.


Colorblind Accommodation

If possible, can you change the green color to a darker green or add a dark green option? Although I am not extremely red-green colorblind, it's often hard to tell the difference between working and idle machines under the default settings. I know that there is the option to change the light green to other colors, but it would be nice.

The color settings are now freely customizable instead of being limited to a predefined selection.


Server trying to kick a client with a closed connection, when the connection shouldnt be closed

[x] I've searched existing issues and couldn't find a duplicate.

The server is trying to kick me while I'm still in the world and then it panics here https://github.com/AnonymousBit0111/Pumpkin/blob/master/pumpkin/src/entity/player.rs#L290-L300
note: on my fork I've changed it from asserting to just logging so that the server doesn't stop when this happens so I can investigate a bit better.
To Reproduce
simply join the server and fly around for a minute or so
Expected behavior
a client only gets kicked if there's a timeout of for some other valid reason
Environment (please complete the following information):

Operating System: MacOS ventura
Server Software Version/Commit: e1fa2b7c69198461cbf44728816f4e8d60047732

Fixed in https://github.com/Snowiiii/Pumpkin/pull/172


how to forgot a non answered request?

I have a serious problem with my bot.
when a request is not answered , bot sticked into the request and even user can not /start or...
how can I catch errors and say to telegram bot api to neglect that request and run next request?
there are many causes of errors in bot, user can stop and block it, my code maybe have bugs or even the webservice I am using can not respond in that time.
how can I catch them and avoid bot to stick in this error.

Its totally depend on your app structure, but you can specify expire time for any message.

thank you for your fast response,
I dont know how to specify expire time for messages?
but how can I try catch any kind of error and send response 200 to telegram?

I think the code I need is something like this:

try {
        $telegram->sendMessage([
            'chat_id'                  => '<PERSONS_ID>',
            'text'                     => 'Here is some text',
        ]);

    } catch (TelegramResponseException $e) {

        $errorData = $e->getResponseData();

        if ($errorData['ok'] === false) {
            $telegram->sendMessage([
                'chat_id' => '<ADMINISTRATOR ID>',
                'text'    => 'There was an error for a user. ' . $errorData['error_code'] . ' ' . $errorData['description'],
            ]);
        }
    }

or even more general.

As this is not a package problem, i'l closed it for now. Reopen it if you need any help.


Error codes should show unsigned

Missed this in #11123...

That should be an unsigned value. Not a signed one.

🤣

wait how the dickens do I get the negative return code?

I think you need a negative return from CreateProcess or CreatePseudoConsole (an early HRESULT-out, not a late one)
Set a commandline to ""cmd.exe"` or something?

yea that'll do


python310Packages.types-requests: 2.27.31 -> 2.28.0

Description of changes
Update to latest upstream release 2.28.0

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 178988 run on x86_64-linux 1

3 packages built:

gphotos-sync
python310Packages.types-requests
python39Packages.types-requests


Improving MTTR w/ Custom Tags - Add details around which type of token to create

Is your feature request related to a problem? Please describe.
Running the workshop Improving MTTR w/ Custom Tags ideally needs details around what type of Access token you need to create for the workshop.
eg. Ingest and Access token.
Describe the solution you'd like
Add detail into the Note which talked about where to get your access token around creating a new access token and making sure what type of token, eg access and ingest token.
Describe alternatives you've considered
There are a few steps in this lab which appear to be written because we are running a hosted workshop vs a self service workshop.
Additional context
https://splunk.github.io/observability-workshop/latest/en/other/dev-mttr-custom-tags/2-setup-app/index.html
this  is the link to the section I am referring too.

@billg-splunk This needs to be assigned to Derek but I can't find him

Hey gents I made a fork but I do not have a Splunk official GitHub account due to when I returned my was not recoverable and due to it using my Splunk email address I can't use one. Should I put changes in fork for someone to merge or what do u suggest ?:

@shabuhabs Yes please :-)

@shabuhabs Any update on this issue please?

Workshop to be retired and replaced with a new tagging workshop


When following USERMANUAL.md I get this error:TypeError: init() got an unexpected keyword argument 'unbound_message'

I am using Cloud Shell and when I am trying to run the first commend with CloudDQ I get this:
$ python3 clouddq_executable.zip --help
Traceback (most recent call last):
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/clouddq/clouddq/main.py", line 32, in <module>
    from clouddq.integration.bigquery.dq_target_table_utils import TargetTable
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/clouddq/clouddq/integration/bigquery/dq_target_table_utils.py", line 24, in <module>
    from clouddq.log import JsonEncoderDatetime
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/clouddq/clouddq/log.py", line 23, in <module>
    from google.cloud.logging.handlers import CloudLoggingHandler
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/py_deps/pypi__google_cloud_logging/google/cloud/logging/__init__.py", line 18, in <module>
    from google.cloud.logging_v2 import __version__
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/py_deps/pypi__google_cloud_logging/google/cloud/logging_v2/__init__.py", line 25, in <module>
    from google.cloud.logging_v2.client import Client
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/py_deps/pypi__google_cloud_logging/google/cloud/logging_v2/client.py", line 37, in <module>
    from google.cloud.logging_v2.handlers import CloudLoggingHandler
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/py_deps/pypi__google_cloud_logging/google/cloud/logging_v2/handlers/__init__.py", line 17, in <module>
    from google.cloud.logging_v2.handlers.app_engine import AppEngineHandler
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/py_deps/pypi__google_cloud_logging/google/cloud/logging_v2/handlers/app_engine.py", line 24, in <module>
    from google.cloud.logging_v2.handlers._helpers import get_request_data
  File "/tmp/Bazel.runfiles__m9_ww0t/runfiles/py_deps/pypi__google_cloud_logging/google/cloud/logging_v2/handlers/_helpers.py", line 22, in <module>
    import flask
  File "/usr/local/lib/python3.9/dist-packages/flask/__init__.py", line 4, in <module>
    from . import json as json
  File "/usr/local/lib/python3.9/dist-packages/flask/json/__init__.py", line 8, in <module>
    from ..globals import current_app
  File "/usr/local/lib/python3.9/dist-packages/flask/globals.py", line 56, in <module>
    app_ctx: "AppContext" = LocalProxy(  # type: ignore[assignment]
TypeError: __init__() got an unexpected keyword argument 'unbound_message'

I tried with version 1.0.0 and 1.0.7

I have the same error with CloudDQ 1.0.5
Interestingly, the _helpers.py line 22 does not have the Flask import anymore, so it looks like they removed it, but this is not the version used by clouddq.
I have opened #242 to try to fix this


Make collection.pluck expand id according to idAttribute

Using collection.pluck('id') with Mongo based _id as idAttribute i would expect Backbone to automatically map this for me so the backend doesn't shine through.

Nope -- the idAttribute is so that Backbone knows which attribute should be treated as the id. The actual value of the data is still _id, or whatever your id attribute happens to be. When manipulating or modifying it, the true name should be visible.


The hover color of the close caption button of does not match Windows 11

What feature or improvement do you think would benefit Files?
The hover color of the close caption button of does not match Windows 11's new color.
Requirements

This proposal will make Files fit in with Windows 11 better

Files Version
<IP_ADDRESS>
Windows Version
Windows 11 21H2 22000.1219
Comments
Expected behaviour found in version <IP_ADDRESS>:

Actual behaviour found in Preview Version <IP_ADDRESS>:

It'd be great if the caption buttons got some more love as a whole, but I realise Microsoft hasn't decided on what they should like. Gotta think about hover states, pressed states, active and inactive, light and dark mode, and fade effects.

Seems like an issue with WinAppSdk, can you open an issue on their repository?


FERC 714: transform of hourly demand table (dbf +xbrl)

Overview
Closes #3838. There is a tasklist in the issue!
What problem does this address?
most of the work here has been in cleaning the date formats 🙄

datetimes omigosh....

🟢 convert_dates_to_zero_offset_hours_xbrl hours being 01-24 or 01-00 (of next day.. probably  I put in some emails to ferc about this)
🟢 convert_dates_to_zero_seconds: some last record of the days being last second of the day (T23:59)
there are lots of overlapping timestamps rn

...

What did you change?
Testing
How did you make sure this worked? How can a reviewer verify this?
# To-do list
- [ ] If updating analyses or data processing functions: make sure to update or write data validation tests (e.g. `test_minmax_rows()`)
- [ ] Update the [release notes](https://catalystcoop-pudl.readthedocs.io/en/nightly/release_notes.html): reference the PR and related issues.
- [ ] Run `make pytest-coverage` locally to ensure that the merge queue will accept your PR.
- [ ] Review the PR yourself and call out any questions or issues you have
- [ ] For minor ETL changes or data additions, once `make pytest-coverage` passes, make sure you have a fresh full PUDL DB downloaded locally, materialize new/changed assets and all their downstream assets and [run relevant data validation tests](https://catalystcoop-pudl.readthedocs.io/en/nightly/dev/testing.html#data-validation) using `pytest` and `--live-dbs`.
- [ ] For bigger ETL or data changes run the full ETL locally and then [run the data validations](https://catalystcoop-pudl.readthedocs.io/en/nightly/dev/testing.html#data-validation) using `make pytest-validate`.
- [ ] Alternatively, run the `build-deploy-pudl` GitHub Action manually.

Just a lil lonely comment because it's not part of the code you edited therefore I can't comment directly on the line...
The doc strings for the OFFSET_CODES dictionary seems like it has a copy and paste error in it?

oh also, @cmgosnell reminder to add some color to the metadata/resources/ferc714.py module about the out_ferc714__hourly_planning_area_demand table.


html_helper: singleton tags ignored for indentation

#213 unit test passes
P.S. not 100% correct regex (does not take into account closing bracket) but should do for the case - incorrect tag is an internal dev mistake that is not expected in stable versions.

LGTM :+1:


New splash

Use Artwork by AOICARD as splash
https://www.reddit.com/r/Monero/comments/rou2q1/monero_artwork/

/cib


Support text/plain response as streaming request body

Changes
For APIs using the text/plain content type (today, only usage download), support these by simply streaming the response body back to the end user. This way, they can read the response and save it to a file or post-process it further.
Tests
Made a simple app that downloads billable usage to a temporary file:
package main

import (
	"context"
	"fmt"
	"io"
	"io/ioutil"
	"os"

	databricks "github.com/databricks/databricks-sdk-go"
	"github.com/databricks/databricks-sdk-go/logger"
	"github.com/databricks/databricks-sdk-go/service/billing"
)

func writeToTempFile(r io.ReadCloser) (*os.File, error) {
	// Ensure r is closed after reading
	defer r.Close()

	// Create a temporary file
	tmpFile, err := ioutil.TempFile("", "prefix-")
	if err != nil {
		return nil, err
	}

	// Copy data from io.ReadCloser to the temporary file
	_, err = io.Copy(tmpFile, r)
	if err != nil {
		tmpFile.Close() // Close and remove temporary file in case of errors
		os.Remove(tmpFile.Name())
		return nil, err
	}

	// Rewind the temporary file for further reading
	tmpFile.Seek(0, io.SeekStart)

	return tmpFile, nil
}

func main() {
	logger.DefaultLogger = &logger.SimpleLogger{Level: logger.LevelTrace}
	a, err := databricks.NewAccountClient(&databricks.Config{
		Profile:      "es-876214",
		DebugHeaders: true,
	})
	if err != nil {
		panic(err)
	}
	ctx := context.Background()
	res, err := a.BillableUsage.Download(ctx, billing.DownloadRequest{
		StartMonth:   "2023-08",
		EndMonth:     "2023-09",
		PersonalData: true,
	})
	if err != nil {
		panic(err)
	}
	tmp, err := writeToTempFile(res.Contents)
	if err != nil {
		panic(err)
	}
	fmt.Printf("tmp: %v\n", tmp.Name())
}

This works.

Codecov Report
Attention: 7 lines in your changes are missing coverage. Please review.

Files
Coverage Δ

openapi/model.go
5.19% <ø> (ø)

service/billing/api.go
0.00% <ø> (ø)

service/billing/model.go
0.00% <ø> (ø)

service/catalog/api.go
0.00% <ø> (ø)

service/iam/model.go
0.00% <ø> (ø)

service/settings/model.go
0.00% <ø> (ø)

service/settings/impl.go
0.00% <0.00%> (ø)

service/billing/impl.go
0.00% <0.00%> (ø)

:loudspeaker: Thoughts on this report? Let us know!.


Support intermediate artifacts

Hi all, thank you so much for developing LineaPy, looks great and I'm really excited about it!
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
When I develop a pipeline, I may want to integrate semantic steps to build my refined dataset table. As an illustration, master_data would be data loaded and assembled from a relational DB, whereas dataset would be the same table refined with some feature engineering.
Currently, if I try to do this I would save both master_data and dataset as artifacts, then create a pipeline like:
lineapy.to_pipeline(artifacts=[master_data.name, dataset.name],
                    dependencies={dataset.name: {master_data.name}},
                    framework='AIRFLOW', pipeline_name='my_great_airflow_pipeline', output_dir='airflow')

My issue is that Lineapy would then create steps to build master_data from scratch, and also to create dataset from scratch instead of loading master_data as a starting point. Like:
import pickle

def master_data():

    import pandas as pd
    from sklearn.datasets import load_iris

    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
        target=[iris.target_names[i] for i in iris.target]
    )
    iris_agg = df.set_index("target")
    artifact = pickle.dump(
        iris_agg, open("/home/oneai/.lineapy/linea_pickles/10dzyzx", "wb")
    )

def dataset():

    import pandas as pd
    from sklearn.datasets import load_iris

    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
        target=[iris.target_names[i] for i in iris.target]
    )
    iris_agg = df.set_index("target")
    iris_clean = iris_agg.dropna().assign(test="test")
    dataset = pickle.dump(
        iris_clean, open("/home/oneai/.lineapy/linea_pickles/5Tk63gO", "wb")
    )

Describe the solution you'd like
A clear and concise description of what you want to happen.
Ideally LineaPy would capture the dependency and build:
My issue is that Lineapy would then create steps to build master_data from scratch, and also to create dataset from scratch instead of loading master_data as a starting point. Something like:
import pickle

def master_data():

    import pandas as pd
    from sklearn.datasets import load_iris

    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
        target=[iris.target_names[i] for i in iris.target]
    )
    iris_agg = df.set_index("target")
    pickle.dump(
        iris_agg, open("/home/oneai/.lineapy/linea_pickles/10dzyzx", "wb")
    )

def dataset():

    import pandas as pd

    iris_agg = pickle.load(
      open("/home/oneai/.lineapy/linea_pickles/10dzyzx", "rb")
   )
    iris_clean = iris_agg.dropna().assign(test="test")
    dataset = pickle.dump(
        iris_clean, open("/home/oneai/.lineapy/linea_pickles/5Tk63gO", "wb")
    )

Is it planned to support this behavior?
Am I missing something?

Hi @PertuyF thanks so much for creating this issue. We are indeed working on this and really appreciate your input! We will get back with more detailed design proposals soon.

Hi @PertuyF thanks again for bring this issue up. Looks like you were running the following code to generate your two artifacts, master_data and dataset
import pandas as pd
from sklearn.datasets import load_iris

iris = load_iris()
df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
    target=[iris.target_names[i] for i in iris.target]
)
iris_agg = df.set_index("target")
master_data = lineapy.save(iris_agg, 'master_data')

iris_clean = iris_agg.dropna().assign(test="test")
dataset = lineapy.save(iris_clean, 'dataset')

and you were using the following code to generate your pipeline:
lineapy.to_pipeline(artifacts=[master_data.name, dataset.name],
                    dependencies={dataset.name: {master_data.name}},
                    framework='AIRFLOW', pipeline_name='my_great_airflow_pipeline', output_dir='airflow')

We totally agree the output you are currently getting is not ideal. The reason for this is LineaPy does not track the dependencies between artifacts generated within the same session and we are working on this.
There is probably more than one reasonable pipeline that can be generated from the artifact-creating code depending on what is your end goal.
There are at least three scenarios that I can think of:

You need to output both the master_data and the dataset at the same time.
You only need to output the dataset but you also need to recompute the master_data since it has been modified.
You only need to output the dataset and you don't need to recompute the master_data since it is already up to date (or never get updated).
..... probably more scenarios can be added

In order to generate pipelines for each scenario, LineaPy will first detect the dependencies between artifacts within the session in the background and the user will be able to generate pipelines for each scenario based on how they call the to_pipeline function.
The followings are some proposed solutions (not finalized yet and your input would be very welcome!):
For scenario 1 (outputting both the master_data and the dataset at the same time), we want to create a pipeline by running:
lineapy.to_pipeline(artifacts=['master_data', 'dataset'])

and we are expect a pipeline.py looks like
def get_master_data():
    import pandas as pd
    from sklearn.datasets import load_iris

    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
        target=[iris.target_names[i] for i in iris.target]
    )
    iris_agg = df.set_index("target")
    return iris_agg

def get_dataset(iris_agg):
    dataset = iris_agg.dropna().assign(test="test")
    return dataset

def pipeline():
    iris_agg = get_master_data()
    dataset = get_dataset(iris_agg)
    return iris_agg, dataset

if __name__=='__main__':
    pipeline()

For scenario 2 (outputting only the dataset and reusing the master_data), we want to create a pipeline by running
lineapy.to_pipeline(artifacts=['dataset'], recompute_dependencies=False)

and we are expect a pipeline.py looks like
def get_dataset(iris_agg):
    dataset = iris_agg.dropna().assign(test="test")
    return dataset

def pipeline():
    iris_agg = lineapy.get('master_data').get_value()
    dataset = get_dataset(iris_agg)
    return dataset

if __name__=='__main__':
    pipeline()

For scenario 3 (outputting only the dataset but recomputing the master_data), we want to create a pipeline by running
lineapy.to_pipeline(artifacts=['dataset'], recompute_dependencies=True)

and we are expect a pipeline.py looks like
def get_master_data():
    import pandas as pd
    from sklearn.datasets import load_iris

    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
        target=[iris.target_names[i] for i in iris.target]
    )
    iris_agg = df.set_index("target")
    return iris_agg

def get_dataset(iris_agg):
    dataset = iris_agg.dropna().assign(test="test")
    return dataset

def pipeline():
    iris_agg = get_master_data()
    dataset = get_dataset(iris_agg)
    return dataset

if __name__=='__main__':
    pipeline()

I hope all these proposed solutions make sense to you.
There are definitely other things that can be done here, for instance

Do we want to lift load_iris() out of the get_master_data() function and put it in a separate function?
Do we want to save the artifact again while running the pipeline? If yes, do you want to save them within the same LineaPy database or other place?

and we would love your input!

Thank you so much @mingjerli for sharing this reflection and giving me an opportunity to provide feedback!
These strategies you mention totally make sense to me, and probably scenarii 1 and 3 would fit my use cases the best. The reason is that for now I forsee LineaPy as an assistant toward productionizing prototyping. To me this would typically covers three main stages:
0. Prototype approach, probably the creative and messy way

transform a messy prototype into clean reusable code. Where LineaPy plays its role (artifacts versioning and pipeline creation)
integrate reusable code to production systems, if the prototype brings value. Where some steps of the pipeline could be reimplemented on dedicated framework

Typical example would be to explore an hypothesis, starting by creating master data from an existing relational DB, then engineering features into dataset, then training model. This is stage 0 and LineaPy would help ensuring reusability of the code (e.g. re-execute on updated data source, revisit after a while, handover to another data scientist,...)
If eventually models are worth it I will want to integrate the pipeline with my DataOps stack and my MLOps stack. This would be stage 2 and there comes my semantic:

master data could likely be reimplemented as SQL to create a view on existing data, thus handing over this step to data engineer to productionize, and the pipeline would evolve to feed on loading this view. Eventually we would have load_master_data() instead of the original code.
dataset would comprize non trivial feature engineering. As such it could live as is, but integrate its output as a new table in some central source of truth for versioning, thus leveraging DataOps tech stack
some model step (not discussed in original post, although the logical next step) would have to integrate with MLOps tech stack for versioning, monitoring, retraining, etc...

In my current vision, LineaPy would be involved to facilitate transition from prototype to production. Hence if we consider three artifacts master data, dataset and model from above:
I see scenario 1 useful to push dataset to DataOps and model to MLOps; we need both outputs.
And I see scenario 3 useful to reimplemente master data into the proper data framework; we need to be able to recalculate the artifact.
Scenario 2 would heavily rely on using LineaPy in a centralised way, and whether I do not exclude this case in the future, I believe I am no there yet. Although it also relates to the final stage where master data is simply loaded, with the difference that recalculation and versioning could be handled dynamically as needed (e.g. Upon change in data source).
Your additional points are very relevant.
Extracting load_iris()  as a standalone step could make sense as semantically data loading and data transformation could need to be separate.
Saving artifacts again upon running the pipeline would directly apply to the case of dataset above. And depending on the stage of the process it could be interesting to do so in LineaPy (stage 1, clean reusable code) or somewhere else (stage 2, integration with DataOps stack).
I hope this makes sense to you! I probably still have to give it some thoughts, although I prefer to throw ideas here so we can engage a discussion 🙂
Also, this is the use case from just one user. I would completely understand that others ways of working could require different behaviours.
Happy to continue this discussion as needed!
Best,
Fabien

Hi Fabien, thanks again for your detailed feedback!
We've been working on supporting scenario 1/3 for a couple weeks and will keep you posted once it's ready for use. In the meantime, would love to explore more how you are using lineapy and connect on slack!

Hi Fabien (@PertuyF), hope all is well with you. We are happy to share that we finally have the support for scenario 1! (We are working on the other two scenarios and will keep you posted as they get ready too.) This means that, with the latest version of lineapy (v0.1.5; released yesterday!), running to_pipeline() API would modularize artifact code into "non-overlapping" functions, where duplicate code blocks are factored out to reduce redundant computations (which might be expensive to run).
Hence, with the same example discussed earlier, i.e.,
import pandas as pd
from sklearn.datasets import load_iris

iris = load_iris()
df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
    target=[iris.target_names[i] for i in iris.target]
)
iris_agg = df.set_index("target")
master_data = lineapy.save(iris_agg, "master_data")

iris_clean = iris_agg.dropna().assign(test="test")
dataset = lineapy.save(iris_clean, "dataset")

running
lineapy.to_pipeline(
    artifacts=[master_data.name, dataset.name],
    dependencies={dataset.name: {master_data.name}},
    framework="AIRFLOW",
    pipeline_name="iris_pipeline",
    output_dir="./airflow",
)

would generate a module file that looks like:
### ./airflow/iris_pipeline_module.py

import pandas as pd
from sklearn.datasets import load_iris

def get_master_data():
    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names).assign(
        target=[iris.target_names[i] for i in iris.target]
    )
    iris_agg = df.set_index("target")
    return iris_agg

def get_dataset(iris_agg):
    iris_clean = iris_agg.dropna().assign(test="test")
    return iris_clean

def run_session_including_master_data():
    # Given multiple artifacts, we need to save each right after
    # its calculation to protect from any irrelevant downstream
    # mutations (e.g., inside other artifact calculations)
    import copy

    artifacts = dict()
    iris_agg = get_master_data()
    artifacts["master_data"] = copy.deepcopy(iris_agg)
    iris_clean = get_dataset(iris_agg)
    artifacts["dataset"] = copy.deepcopy(iris_clean)
    return artifacts

def run_all_sessions():
    artifacts = dict()
    artifacts.update(run_session_including_master_data())
    return artifacts

if __name__ == "__main__":
    run_all_sessions()

As shown, the modularized code now contains "non-overlapping" functions, e.g., get_dataset() no longer repeats the same computation as that in get_master_data() (the former instead takes the output from the latter to carry on its own processing).
Moreover, LineaPy can now smartly identify any "common" computation among different artifacts and factor it out into its own function — even if that common computation has not been stored as an artifact (check our recent GH discussion post for a concrete example). This of course further reduces any redundant computation in the pipeline.
Given your valuable inputs earlier, we would love to get your feedback on this new style of pipeline generation. Please give it a try and let us know what you think (or any questions)!


goog.date.UtcDateTime.fromIsoString adds 1900 to years with more than one leading zero

goog.date.UtcDateTime.fromIsoString('0000-10-15T23:50:58.165Z')
=> goog.date.UtcDateTime {date: Tue Oct 16 1900 02:04:26 GMT+0213 (Eastern European Summer Time)}

goog.date.UtcDateTime.fromIsoString('0009-10-15T23:50:58.165Z')
=> goog.date.UtcDateTime {date: Sat Oct 16 1909 02:04:26 GMT+0213 (Eastern European Summer Time)}

goog.date.UtcDateTime.fromIsoString('0099-10-15T23:50:58.165Z')
=> goog.date.UtcDateTime {date: Sat Oct 16 1999 02:50:58 GMT+0300 (Eastern European Summer Time)}

goog.date.UtcDateTime.fromIsoString('0999-10-15T23:50:58.165Z')
=> goog.date.UtcDateTime {date: Wed Oct 16 0999 02:04:26 GMT+0213 (Eastern European Summer Time)}

Related: #1142

I'm unable to reproduce this one using the latest NPM release: https://jsfiddle.net/L94g1tbh/

That said, we should probably add test cases for this.


EVP fetching: make operation_id part of the method identity

Because the operation identity wasn't integrated with the created
methods, the following code would give unexpected results:
EVP_MD *md = EVP_MD_fetch(NULL, "MD5", NULL);
EVP_CIPHER *cipher = EVP_CIPHER_fetch(NULL, "MD5", NULL);

if (md != NULL)
    printf("MD5 is a digest\n");
if (cipher != NULL)
    printf("MD5 is a cipher\n");

The message is that MD5 is both a digest and a cipher.
Partially fixes #9106

Documentation fixed

Not sure the appveyor errors are relevant to this PR...

CIs are happy now...

Maybe add a simple test similiar to the one shown in the commit body.

See #9106

Merged.
b4d3f203da6210f148b2a0c7bf5a802b55ba0e65 doc/internal/man3/ossl_method_construct.pod: follow common conventions
2ccb1b4ecab2c3ac1dc2ff81a48869a79afa7839 EVP fetching: make operation_id part of the method identity


[Bug]: Duplicate src in file path

What happened?
While running command php artisan module:filament:install ModuleName & selecting cluster option to Yes got below error
Illuminate\Contracts\Filesystem\FileNotFoundException
File does not exist at path /Users/ravinder/Sites/bigcommerce/apps/share-cart/vendor/coolsam/modules/src/src/Commands/stubs/filament-plugin.stub.
You will notice that there's duplicate src folder in the path
This comes from 'src' mentioned here https://github.com/savannabits/filament-modules/blob/71316d92a4c655ae03b3a7a69be951543199f1b9/src/Concerns/GeneratesModularFiles.php#L22
Then being added again from https://github.com/savannabits/filament-modules/blob/71316d92a4c655ae03b3a7a69be951543199f1b9/src/Modules.php#L52
How to reproduce the bug
After running command php artisan module:filament:install ModuleName to install filament in existing module
Package Version
v4.0.4
PHP Version
8.2.17
Laravel Version
11.0.4
Which operating systems does with happen with?
macOS
Notes
No response

Thank you for the elaborate description. This is confirmed as a bug, I am working on a fix and will merge shortly.


[BUG] gpg error when signing deb for python3.10

Describe the issue
The building process failed in the last step of signing the deb packages, with the following errors. However, the packages can still be installed without being signed.
W: python3.10 source: dependency-is-not-multi-archified python3.10-tk depends on python3.10 (multi-arch: allowed)
W: python3.10 source: dependency-is-not-multi-archified python3.10-gdbm-dbg depends on python3.10-dbg (multi-arch: allowed)
W: python3.10 source: dependency-is-not-multi-archified python3.10-tk-dbg depends on python3.10-dbg (multi-arch: allowed)
W: python3.10 source: dependency-is-not-multi-archified python3.10-gdbm depends on python3.10 (multi-arch: allowed)
W: python3.10 source: newer-standards-version 4.3.0 (current is 4.1.4)
W: python3.10 source: missing-runtime-test-file debian/tests/module-install-local paragraph starting at line 58
W: python3.10 source: missing-runtime-test-file debian/tests/module-install-user paragraph starting at line 66
W: python3.10 source: missing-runtime-test-file debian/tests/module-install-venv paragraph starting at line 74
W: python3.10 source: missing-runtime-test-file debian/tests/module-install-virtualenv paragraph starting at line 82
E: libpython3.10-dbg: no-shlibs-control-file usr/lib/aarch64-linux-gnu/libpython3.10d.so.1.0
E: libpython3.10-dbg: package-must-activate-ldconfig-trigger usr/lib/aarch64-linux-gnu/libpython3.10d.so.1.0
W: python3.10-lib2to3: description-starts-with-leading-spaces
W: python3.10-lib2to3: binary-without-manpage usr/bin/2to3-3.10
E: python3.10-lib2to3: python-script-but-no-python-dep usr/bin/2to3-3.10 #!/usr/bin/python3.10
E: python3.10-lib2to3: python-script-but-no-python-dep usr/lib/python3.10/lib2to3/pgen2/token.py #!/usr/bin/python3.10
E: python3.10: python-script-but-no-python-dep usr/bin/pygettext3.10 #!python3
W: python3.10-gdbm-dbg: priority-extra-is-replaced-by-priority-optional
W: python3.10-tk-dbg: priority-extra-is-replaced-by-priority-optional
W: python3.10-minimal: hardening-no-pie usr/bin/python3.10
W: python3.10-dbg: hardening-no-pie usr/bin/python3.10d
W: libpython3.10-stdlib: description-too-long
E: libpython3.10-stdlib: wrong-path-for-interpreter usr/lib/python3.10/cgi.py (#!/usr/local/bin/python != /usr/bin/python)
W: libpython3.10-testsuite: executable-not-elf-or-script usr/lib/python3.10/test/test_array.py
W: libpython3.10-testsuite: executable-not-elf-or-script usr/lib/python3.10/test/test_uuid.py
W: libpython3.10-testsuite: executable-not-elf-or-script usr/lib/python3.10/test/test_importlib/update-zips.py
E: libpython3.10: no-shlibs-control-file usr/lib/aarch64-linux-gnu/libpython3.10.so.1.0
E: libpython3.10: package-must-activate-ldconfig-trigger usr/lib/aarch64-linux-gnu/libpython3.10.so.1.0
N: 66 tags overridden (25 errors, 38 warnings, 3 info)
Finished running lintian.
Now signing changes and any dsc files...
signfile dsc python3.10_3.10.11-1+bionic1.dsc Anthony Sottile (deadsnakes<EMAIL_ADDRESS>gpg: keybox '/home/username/.gnupg/pubring.kbx' created
gpg: skipped "Anthony Sottile (deadsnakes<EMAIL_ADDRESS>No secret key
gpg: /tmp/debsign.Ti2hEhMg/python3.10_3.10.11-1+bionic1.dsc: clear-sign failed: No secret key
debsign: gpg error occurred!  Aborting....
debuild: fatal error at line 1081:
running debsign failed
gbp:error: 'debuild -i -I' failed: it exited with 29
NVIDIA Jetson Hardware
Jetson Nano
L4T / JetPack Version
JetPack 4.6.1
To Reproduce
For example, what command line did you run?
bash ./build_python3.sh --version 3.10
Additionally, libncursesw5-dev is missing and caused the build to fail in my first run. after installing it, the build continued (and ran into the above errors).

but this error seems doesn't break the following installation.

but this error seems doesn't break the following installation.

Nope, it does break. Making a repository fails too then.

this helped me:
pip3 install gbp

After running into the same issue trying to install python 3.10 on a jetson nano
I tried:
cd ~/Python-Builds/Python3.10-Dist/python3.10
gbp buildpackage -d --git-ignore-branch --git-ignore-new

It's building as we speak... 🚀

After running into the same issue trying to install python 3.10 on a jetson nano
I tried:
cd ~/Python-Builds/Python3.10-Dist/python3.10
gbp buildpackage -d --git-ignore-branch --git-ignore-new

It's building as we speak... 🚀

But now I get this:
Finished running lintian.
Now signing changes and any dsc files...
 signfile dsc python3.10_3.10.11-1+bionic1.dsc Anthony Sottile (deadsnakes<EMAIL_ADDRESS>gpg: keybox '/home/user/.gnupg/pubring.kbx' created
gpg: skipped "Anthony Sottile (deadsnakes<EMAIL_ADDRESS>No secret key
gpg: /tmp/debsign.gugaUrIW/python3.10_3.10.11-1+bionic1.dsc: clear-sign failed: No secret key
debsign: gpg error occurred!  Aborting....
debuild: fatal error at line 1081:
running debsign failed

Hello,
Im getting the same issue, while installing the  python3.9 from deadsnake repo.
@wzh1895 are you able to fix this?

signfile dsc python3.9_3.9.16-1+bionic1.dsc Anthony Sottile (deadsnakes<EMAIL_ADDRESS>gpg: WARNING: unsafe ownership on homedir '/home/blucursor/.gnupg'
gpg: WARNING: unsafe ownership on homedir '/home/blucursor/.gnupg'
gpg: keybox '/home/blucursor/.gnupg/pubring.kbx' created
gpg: skipped "Anthony Sottile (deadsnakes<EMAIL_ADDRESS>No secret key
gpg: /tmp/debsign.oHaTropE/python3.9_3.9.16-1+bionic1.dsc: clear-sign failed: No secret key
debsign: gpg error occurred!  Aborting....
debuild: fatal error at line 1081:
running debsign failed
gbp:error: 'debuild -i -I' failed: it exited with 29

For those having the gbp error, in the last line of the script put this
gbp buildpackage --git-ignore-branch --git-ignore-new

Hey guys, I am not sure this what you actually need, if you would like to only install python. I have received the same error regarding gbp and I have decided to insall python manually, avoiding deb package creation and installation.
I have installed python using this repos and everything looks good. (On the first glance)
https://github.com/JetsonHacksNano/python3.10/tree/ubuntu/bionic
./configure
make
make test
sudo make altinstall


Add sandbox option in vrep

This PR incorporates the latest changes from the vCluster API to update the sandbox option in vrep

Can I assume that without this change, replication from a source sandbox would not be possible? Even though the operator provides the source IP for a sandbox, vclusterOps couldn't utilize the source IP without recognizing that it originated from a sandbox.

I have no issues with this change, assuming my understanding is correct. You should coordinate with @jizhuoyu as you both are changing the same test (see #786)

Can I assume that without this change, replication from a source sandbox would not be possible? Even though the operator provides the source IP for a sandbox, vclusterOps couldn't utilize the source IP without recognizing that it originated from a sandbox.

Yes, your assumption is correct, vclusterOps needs to identify that the source IP belongs to a sandbox in order to function properly


FullyConnectedLayer: TensorShape set on OutputSlot[0] does not match the inferred shape. : [1,20,2] != [1,2]

when I use onnx model,the output dimension and elements are all right, like this:(std::cout<<"output size = "<<outputBinding.second.GetNumElements()<<std::endl;),but when I use armnn::IRuntime::CreationOptions options; there is an error :
terminate called after throwing an instance of 'armnn::LayerValidationException'
what():  FullyConnectedLayer: TensorShape set on OutputSlot[0] does not match the inferred shape. : [1,20,2] != [1,2]
the model is very simple by pytorch linear layer, input is [1,20,8],output is [1,20,2]

Hi @QHenry1990,
Can you provide the Onnx model to help us reproduce the issue?
Regards,
Francis.

Hi @QHenry1990,
Can you provide the Onnx model to help us reproduce the issue?
Regards, Francis.

OK,
model1.zip

Hi @QHenry1990,
I've reproduced the issue and am now looking into it.
Thanks,
Francis.

Hi @QHenry1990,
I have created a fix for this issue here: https://review.mlplatform.org/c/ml/armnn/+/9170. Can you please apply this patch and see if it works for you?
Thanks again for letting us know about it. All the best!
Kind regards,
Matthew

thanks a lot!

Hi @QHenry1990,
This fix has now been merged to the main branch so you can get it there. It unfortunately won't make it into the upcoming release (23.02) but it will be in the next one (23.05).
I will close this issue now because it has been fixed. Thanks again for letting us know about it. All the best!
Kind regards,
Matthew


Do not warn in packages

Do not have log output in packages that applications consume because the
output can mess with logs, stdout/stderr of applications and such and
there is nothing that the consumer can do about it other than change the
package that they are using.
Fixes https://github.com/docker/docker/issues/13373
Signed-off-by: Michael Crosby<EMAIL_ADDRESS>

LGTM

LGTM but it doesn't fix docker/docker#13373 because that is an entirely different problem :-). You may want to remove the Fixes comment from the PR description so that it doesn't accidentally close that issue .

how do you know it does not fix it? wanna make a bet? ;)

I am just relying on this comment:
https://github.com/docker/docker/issues/13373#issuecomment-108056772
Do you still think that issue is related to this warning message?

yep, I believe it is related

@crosbymichael Are you saying some code in docker captures the log messages and returns it as error ?

I don't understand how an api handler in /images/load POST method is affected by this warning message  when the portallocator is not at all involved in that code path. May be I am missing something.

@mrjana It makes reexec to same binary, you package write logs there.

@LK4D4 ah ok, to clarify the caller of reexec is expecting some output from the command but that is messed up by this log which makes the whole call fail?

@mrjana Yup.

nice catch! I believe the fix :) LGTM

LGTM still and about to be merged.

Btw, I think this needs to go into the docker1.7.0_integ branch too, so we can vendor only this code in docker.


Feature request: silent Zone around the code

Hi,
Normally the border around the QR code is one module width. You paint only a thin line around. Maybe the user could setup the wished width.
Edit: because I have the dark mode active. With light mode there is around all white and the silent zone is around. But with dark mode the QR Code is not readable from smartphone display.
But fine App, thank you.

I support this


Fix IDP POST response for IDP initialized logout request (for SLO)

Currently, when receiving a logout started by the IdP, this app send back a simple HTTP redirection to the IDP without any data.
When using Signed request/response, it results a 400 HTTP response from th IDP (Keycloak in my case).

This PR tests if the http_info prepared by SAML2Client.handle_logout_request is an HTML Post form. In this case, we just return this HTML which will be submited by the browser to the IDP to confirm the SLO request has been done for this SP.

Thanks.


fix: remove observable option

Remove observable option
Swap location of competitive and create group

Task linked: CU-3whgbza Remove Observable option from Create Scrim modal


3D images not supported?

Tried to use SmoothGradCAMpp, ScoreCAM and SSCAM on pytorch ResNet architecture.
SmoothGradCAMpp complains when taking the sum on the weights:
RuntimeError: The size of tensor a (5) must match the size of tensor b (3) at non-singleton dimension 1
Input: (1, 1, 79, 190, 158)
I printed the input and the received the following:
"Weight.View:" torch.Size([2048, 5, 1, 1]) "Hook Squeese:" torch.Size([2048, 3, 6, 5]) "Weights:" torch.Size([2048, 5]) "Hook:" torch.Size([1, 2048, 3, 6, 5])
Question: Is it reasonable that SSCAM allocates 18gb memory on the GPU?

Hello again @FredrikM97 :wave:
Thanks for helping to catch some bugs! Could you produce a minimal snippet to produce this error please?
I don't have a 3DResNet available so I'm not able to reproduce the error!
Also, where is the RuntimeError raised please?

No worries! Thank you for quick replies!
The error occurs at line 122 in cams.py:
batch_cams = (weights.view(*weights.shape, 1, 1) * self.hook_a.squeeze(0)).sum(0)  # type: ignore[union-attr]
It looks like
Here is a sample of a 3D nifti image and the resnet: 3D_network.zip
To open the file:
import nibabel as nib
img = nib.load(example_filename)

Here is a sample of the code I use:
model = agent.model.to(device)
model.eval()

cam = SmoothGradCAMpp(
    model,
    'layer4',
    batch_size = 1,
    num_samples = 1,
    std = 2.0,
    input_shape = (1,1, 79, 224, 224),
)
with torch.no_grad():
    scores = model(input_tensor)
    activation_map = cam(scores.squeeze(0).argmax().item(), scores).cpu()

不用担心！感谢您的快速回复！
错误发生在 cams.py 中的第 122 行：如下所示batch_cams = (weights.view(*weights.shape, 1, 1) * self.hook_a.squeeze(0)).sum(0) # type: ignore[union-attr]
以下是3D nifti图像和resnet的示例：3D_network.zip。请注意，resnet 是为灰度图像配置的。和三个输出类。
要打开文件：
import nibabel as nib
img = nib.load(example_filename).get_fdata()

以下是我使用的代码示例：
from resnet import resnet50
device = 'cuda'
model = resnet50().to(device)
model.eval()

input_tensor = torch.from_numpy(img).unsqueeze(0).unsqueeze(0).float()

cam = SmoothGradCAMpp(
    model,
    'layer4',
    batch_size = 1,
    num_samples = 1,
    std = 2.0,
    input_shape = (1,1, 79, 224, 224),
)
with torch.no_grad():
    scores = model(input_tensor)
    activation_map = cam(scores.squeeze(0).argmax().item(), scores).cpu()

**更新：**在 ResNet 中的平均池层之前，数据的形状是火炬。大小（[1， 2048， 3， 6， 5]）。所以钩子似乎正在起作用。但是，在之后：观察到以下形状：alpha = grad_2 / (2 * grad_2 + (grad_3 * init_fmap).sum(dim=(2, 3), keepdim=True))
alpha.shape: torch.Size([1, 2048, 3, 7, 7])
self.hook_g.squeeze(0).shape: torch.Size([2048, 3, 7, 7])
torch.relu(self.hook_g.squeeze(0)).shape:  torch.Size([2048, 3, 7, 7])
alpha.squeeze_(0).mul_(torch.relu(self.hook_g.squeeze(0))).shape:  torch.Size([2048, 3, 7, 7])

最后一步是执行，我认为这是新形状的原因.sum(dim=(1, 2)) ``torch.Size([2048, 5, 1, 1]).
Hello, have you finished your 3D visualization? Can you share it? Thank you very much.

Hi there :wave:
The issue raised by @FredrikM97 was solved last year. I'm not sure what you're referring to regarding the 3D visualization?
CAMs are working on 3D data if that's your question :)

Thank you for quick replies!

Hi there 👋
The issue raised by @FredrikM97 was solved last year. I'm not sure what you're referring to regarding the 3D visualization? CAMs are working on 3D data if that's your question :)
Thank you for quick replies!
My program has the following problems:cannot register a hook on a tensor that doesn't require gradient
This is my program:
file=r'E:\pythonProject\data\train\128\label404347_4 1.npy'
con_arr1 = np.load(file,allow_pickle=True)  # 读取npy文件
con_arr=con_arr1[0][0]
device = 'cuda'
model = resnet18(num_classes=1).to(device)
model.eval()

input_tensor = np.array(con_arr)
input_tensor=torch.from_numpy(input_tensor)
input_tensor =input_tensor.unsqueeze(0).unsqueeze(0).float().to(device)
print(input_tensor.shape)
cam = SmoothGradCAMpp( model,'layer1',num_samples = 1,std = 2.0,input_shape = (1,128,128,128))
with torch.no_grad():
scores = model(input_tensor)
activation_map = cam(scores.squeeze(0).argmax().item(), scores).cpu()


USM : TLS connection classification

What does this PR do?
Classify the connection if it contain TLS protocol and add it to the protocol stack and connection tags
system_probe_config:
    disable_tcp: false

network_config:
    enable_protocol_classification: true

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

@p-lambert please review as well

you have failed tests in the CI
If you have idea which test doesn't cleanup properly

   [4.14.262-200.489.amzn2.x86_64] Error:      	Received unexpected error:
   [4.14.262-200.489.amzn2.x86_64] error guessing offsets: could not start offset ebpf manager: probes activation validation failed: 1 error occurred:
   [4.14.262-200.489.amzn2.x86_64] * {UID:offset EBPFFuncName:tracepoint__net__net_dev_queue EBPFSection:}: couldn't set perf event bpf {UID:offset EBPFFuncName:tracepoint__net__net_dev_queue EBPFSection:}: error attaching bpf program to perf event: file exists


Make the Website Responsive

Hi I would like to contribute towards this can I give it a shot?

X

Hi I would like to contribute towards this can I give it a shot?

Yes

Hi I would like to contribute towards this can I give it a shot?

Yes

I would also like to work on it!

I would also like to work on it!

Ok

I would also like to work on this.

Thank you for showing you interest in the project.
@abhinav-huria  you have to wait for 3-5 days as 2 people are already assigned on this issue.
Kindly give some time to the assignees to resolve the issue then you can work on this issue till then look after some other issues.
We appreciate your patience.

sir, i would like to work on this project

sir, i would like to work on this project

Thank you for showing you interest in the project.
@himanshu007-creator  you have to wait for 2-3 days as 2 people are already assigned on this issue.
Kindly give some time to the assignees to resolve the issue then you can work on this issue till then look after some other issues.
We appreciate your patience.

Hello, can I work on this issue?

Hello, can I work on this issue?

Hello, can I work on this issue?

Thank you for showing you interest in the project.
@Sloth-Panda  you have to wait for 3-5 days as 2 people are already assigned on this issue.
Kindly give some time to the assignees to resolve the issue then you can work on this issue till then look after some other issues.
We appreciate your patience

Hello, can I work on this issue?

Thank you for showing you interest in the project.
@Sloth-Panda  you have to wait for 3-5 days as 2 people are already assigned on this issue.
Kindly give some time to the assignees to resolve the issue then you can work on this issue till then look after some other issues.
We appreciate your patience


resource_tracker: There appear to be 32 leaked semaphore objects to clean up at shutdown

I've got this message by resource tracker when I use spawn mode on my mac, the code is really simple:
import time
from mpire import WorkerPool

def global_runner(*args):
    print("I am Running", args)
    time.sleep(0.3)
    return

if __name__ == "__main__":
    params_group = [1, 2]
    with WorkerPool(n_jobs=2, start_method="spawn") as pool:
        pool.map(global_runner, params_group)

Can anyone help me solve this problem?

The version of python is 3.9.7

Hmm.. Unfortunately I don't have a mac to be able to debug this. When testing this on linux using python 3.9.7 your script works just fine and doesn't give the warnings. So it's probably mac specific.
However, it is only a warning at the end of the script, and a warning like this can be safely ignored. I.e., your code should still work fine.
If someone else wants to do some debugging, please do so. The only semaphores being used in mpire are within JoinableQueues in mpire/comms.py.

Don't ask me how, but I seem to be consistently getting this when using a WorkerPool with progress_bar=True. When I remove the progress bar (and remove start_method="spawn") the issue goes away.

There is definitely a bug here that leaks semaphores. It causes macOS to run out of file descriptors.

Bug has been found. Mac support will be added soon


docs: move away from 01.org doc links

Closes: #684

While checking the links, few things caught my eye on the SGX plugin page:

Updated, thanks!

Merging is still blocked. @hj-johannes-lee can you review?


Value lists are empty in a WIT form

Hi Avi,
I followed the guide to use this extension in an Azure Devops project. I created 3 string list type fields and I added in the User Story type, the control: Multi pickers.
I defined a control name then I defined the fields with their value: a field name 1 linked to my first list of values field and I did the same for the 2 others.
Once the control has been defined in the User Story, I suppose that the tree structure must be done in the Multi picker configuration via CSV file or in the generator. On the other hand, what should the name of the columns of my CSV correspond to ? Can it accepts accented characters?
For example one of my properties in the control is named "Story Complexity" and the value refers to a list of values field named "Team Story Complexity".
Currently in the form of a User Story the lists are present but empty.
Thank you in advance for your help
Chris

hi,
thank you for using my extension.
if the list is empty it is usually because of wrong spelling
I never try accented characters but this control ignores the first row and
the first column (these are for us), so I don't think that's the problem.
if you need more assistance please let me know, and please add your CSV file
and some pictures (of the configuration and of the user story).
On Thu, Jul 15, 2021 at 8:35 PM chrisngv @.***> wrote:

Hi Avi,
I followed the guide to use this extension in an Azure Devops project. I
created 3 string list type fields and I added in the User Story type, the
control: Multi pickers.
I defined a control name then I defined the fields with their value: a
field name 1 linked to my first list of values field and I did the same
for the 2 others.
Once the control has been defined in the User Story, I suppose that the
tree structure must be done in the Multi picker configuration via CSV file
or in the generator. On the other hand, what should the name of the columns
of my CSV correspond to ? Can it accepts accented characters?
For example one of my properties in the control is named "Story
Complexity" and the value refers to a list of values field named "Team
Story Complexity".
Currently in the form of a User Story the lists are present but empty.
Thank you in advance for your help
Chris
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUIZROXYETMIZ7O7HYVTTX4L4ZANCNFSM5AN766JA
.

--
בתודה,
אבי חדד
0542400091

Hi,
Here you can find the two list type fields to be related: for an environment I can have a set of delivery mode (DeliveryFlow).
"TribuDev Envrionment" field

"TribuDev DeliveryFlow

And the Multi-pickers control values:

The CSV file name is like control name : DeliveryWorkflow.csv. Data are separated by commas.

Then I uploaded this file in the configuration of the Multi Picker extension.

Finally I created a Use Story and there the lists of Multi-pickers controls are present but empty

Did I miss something ?
Thx

Hi Avi,
Have you an idea for my problem ?

Hi
Sorry for the delayed answer
I think you got that wrong
To populate the pick list, it takes the values from the csv file.
You dont need to use picker list but a single line field
בתאריך יום ד׳, 21 ביולי 2021, 10:57, מאת chrisngv @.***

:

Hi,
Here you can find the two list type fields to be related: for an
environment I can have a set of delivery mode (DeliveryFlow).
"TribuDev Envrionment" field
[image: image]
https://user-images.githubusercontent.com/12407532/126195328-58ce6198-2927-4fd0-8685-e7b82d251853.png
"TribuDev DeliveryFlow
[image: image]
https://user-images.githubusercontent.com/12407532/126196428-93be938d-4a0a-47f7-9978-142efa0b6aa5.png
And the Multi-pickers control values:
[image: image]
https://user-images.githubusercontent.com/12407532/126200670-a8ed29f7-cab2-4f08-bc98-bceca455190c.png
[image: image]
https://user-images.githubusercontent.com/12407532/126200711-6d55f8d1-e84a-4fd6-acdc-3e89096f73ea.png
[image: image]
https://user-images.githubusercontent.com/12407532/126200734-f06f3799-efda-434e-a880-d625c5a10060.png
[image: image]
https://user-images.githubusercontent.com/12407532/126200765-93adc268-da81-431c-b1cf-f1800d897035.png
The CSV file name is like control name : DeliveryWorkflow.csv. Data are
separated by commas.
[image: image]
https://user-images.githubusercontent.com/12407532/126451538-7f4f1887-160a-488c-b839-289623a76500.png
Then I uploaded this file in the configuration of the Multi Picker
extension.
[image: image]
https://user-images.githubusercontent.com/12407532/126451965-71acc707-3d18-4b5f-a593-3c17eddbd716.png
Finally I created a Use Story and there the lists of Multi-pickers
controls are present but empty
[image: image]
https://user-images.githubusercontent.com/12407532/126452503-707bdc5a-e579-404d-8248-4b38009f72ec.png
Did I miss something ?
Thx
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19#issuecomment-883975499,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUI27TF2AQZ4NADVDQL3TYZ4WPANCNFSM5AN766JA
.

I don't understand, can you make a sample with my cases ?

Can you send me the csv file
בתאריך יום ג׳, 27 ביולי 2021, 15:13, מאת chrisngv @.***

:

I don't understand, can you make a sample with my cases ?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19#issuecomment-887460420,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUI45VWBW44TBIBQHILLTZ2PFVANCNFSM5AN766JA
.

CSV File

Hi,
Have you receive CSV file ?

Hi Avi,
Are you here ?
Your extension seems to be very useful and I need it to my job, but I can't use it for now. Can you help me ?
Thx

Yes, I can.
Lets set a mitting for tommrow
בתאריך יום ג׳, 3 באוג׳ 2021, 10:56, מאת chrisngv @.***

:

Hi Avi,
Are you here ?
Your extension seems to be very useful and I need it to my job, but I
can't use it for now. Can you help me ?
Thx
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19#issuecomment-891624229,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUIZMWJDSOETXFWLHVPLT26OMHANCNFSM5AN766JA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

May be it will be difficult because I live in France and there is jet lag.
We can continue to chat on Teams but it will not be shared for other people with the same problem.

Ok lets schdule for tommorow
בתאריך יום ג׳, 3 באוג׳ 2021, 12:50, מאת chrisngv @.***

:

May be it will be difficult because I live in France and there is jet lag.
We can continue to chat on Teams but it will not be shared for other
people with the same problem.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19#issuecomment-891704071,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUIZLKS7NHSU2EIHIXADT263VXANCNFSM5AN766JA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

i seem to be having similar issues above
imported the csv, connected the fields.
and i cant even get the extension to show any of the options. on the WI form the extension loads or shows the spinner, then nothing appears

This happed when he cant find csv file with the name of the control name
בתאריך יום ו׳, 20 באוג׳ 2021, 01:58, מאת Nintex-Batman <
@.***>:

i seem to be having similar issues above
imported the csv, connected the fields.
and i cant even get the extension to show any of the options. on the WI
form the extension loads or shows the spinner, then nothing appears
[image: image]
https://user-images.githubusercontent.com/83791248/130154698-ba99dd92-38ae-4e95-a261-c937404affcf.png
[image: image]
https://user-images.githubusercontent.com/83791248/130154722-0c176da5-526a-4dfe-8f58-1901a58f364f.png
[image: image]
https://user-images.githubusercontent.com/83791248/130154733-2c75f9ad-7772-477f-b5d1-2c8e5b7ea825.png
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19#issuecomment-902305065,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUI3HDWUW3RE3R36ECPDT5WEBHANCNFSM5AN766JA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

This happed when he cant find csv file with the name of the control name בתאריך יום ו׳, 20 באוג׳ 2021, 01:58, מאת Nintex-Batman < @.***>:

ok, i didnt see anywhere that requires the file and field name needs to be same name
seems to have fixed my problem
thanks

@Nintex-Batman can you send me a sample of your CSV file in order to see it's format ?
Thx

I'm trying this config:

and it doesn't work; no values in dropdown in WIT

HiThank you for using my controlI can help you tomorrow.But right now i can tell you, that in the extenshion on the market there is examplesI think you csv is missing a column The first one is not used...it supose to be line count..... the real data is start from column 2 and row 2.The name of the control must be the same as the csv fileנשלח מה-Galaxy שלי
-------- הודעה מקורית --------מאת: chrisngv @.> תאריך: 6.9.2021  18:12  (GMT+02:00) אל: avih75/PickerFieldsControl @.> עותק: avih75 @.>, Comment @.> נושא: Re: [avih75/PickerFieldsControl] Value lists are empty in a WIT form (#19)
I'm trying this config:
and it doesn't work; no values in dropdown in WIT
—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

Hi,
I'm doing your process and same thing don't work.
I'm trying with CSV file with semicolon separators

==> KO
I'm trying directly in extension config:

==> KO

HiThank you for using my controlI can help you tomorrow.But right now i can tell you, that in the extenshion on the market there is examplesI think you csv is missing a column The first one is not used...it supose to be line count..... the real data is start from column 2 and row 2.The name of the control must be the same as the csv fileנשלח מה-Galaxy שלי -------- הודעה מקורית --------מאת: chrisngv @.> תאריך: 6.9.2021 18:12 (GMT+02:00) אל: avih75/PickerFieldsControl @.> עותק: avih75 @.>, Comment @.> נושא: Re: [avih75/PickerFieldsControl] Value lists are empty in a WIT form (#19) I'm trying this config: and it doesn't work; no values in dropdown in WIT —You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

Try use other kind of csv type.
Use coma as saperate
And remove the "empty" column
בתאריך יום ג׳, 7 בספט׳ 2021, 14:28, מאת chrisngv @.***

:

Hi,
I'm doing your process and same thing don't work.
I'm trying with CSV file with semicolon separators
[image: image]
https://user-images.githubusercontent.com/12407532/132337037-4ca656c9-d2af-489b-bb67-7010106f5612.png
==> KO
I'm trying directly in extension config:
[image: image]
https://user-images.githubusercontent.com/12407532/132337259-2aaaf34d-cddb-4403-a3fa-79edc8fa188d.png
==> KO
HiThank you for using my controlI can help you tomorrow.But right now i
can tell you, that in the extenshion on the market there is examplesI think
you csv is missing a column The first one is not used...it supose to be
line count..... the real data is start from column 2 and row 2.The name of
the control must be the same as the csv fileנשלח מה-Galaxy שלי --------
הודעה מקורית --------מאת: chrisngv @.> תאריך: 6.9.2021 18:12
(GMT+02:00) אל: avih75/PickerFieldsControl @.> עותק: avih75 @.>,
Comment @.> נושא: Re: [avih75/PickerFieldsControl] Value lists are empty
in a WIT form (#19
https://github.com/avih75/PickerFieldsControl/issues/19) I'm trying
this config: and it doesn't work; no values in dropdown in WIT —You are
receiving this because you commented.Reply to this email directly, view it
on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile
for iOS or Android.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avih75/PickerFieldsControl/issues/19#issuecomment-914224135,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPBUI3XGCN7NFITFZJEAZLUAXZNTANCNFSM5AN766JA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Yes it works !!!
Thx

Try use other kind of csv type. Use coma as saperate And remove the "empty" column בתאריך יום ג׳, 7 בספט׳ 2021, 14:28, מאת chrisngv @.***
…
: Hi, I'm doing your process and same thing don't work. I'm trying with CSV file with semicolon separators [image: image] https://user-images.githubusercontent.com/12407532/132337037-4ca656c9-d2af-489b-bb67-7010106f5612.png ==> KO I'm trying directly in extension config: [image: image] https://user-images.githubusercontent.com/12407532/132337259-2aaaf34d-cddb-4403-a3fa-79edc8fa188d.png ==> KO HiThank you for using my controlI can help you tomorrow.But right now i can tell you, that in the extenshion on the market there is examplesI think you csv is missing a column The first one is not used...it supose to be line count..... the real data is start from column 2 and row 2.The name of the control must be the same as the csv fileנשלח מה-Galaxy שלי -------- הודעה מקורית --------מאת: chrisngv @.> תאריך: 6.9.2021 18:12 (GMT+02:00) אל: avih75/PickerFieldsControl @.> עותק: avih75 @.>, Comment @.> נושא: Re: [avih75/PickerFieldsControl] Value lists are empty in a WIT form (#19 <#19>) I'm trying this config: and it doesn't work; no values in dropdown in WIT —You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#19 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIPBUI3XGCN7NFITFZJEAZLUAXZNTANCNFSM5AN766JA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Interacting does not prevent page scroll

Ios safari, page scrolls when interacting with knobs.

see #28

My bad i searched for scroll.


Weird behavior when concatting dataframes

I'm having an issue with a timeindex after I run a query then concat new columns based off the return of an apply on a column. If I don't do the query or I don't concat it works ok. Both together seems to cause an issue. I've attached a csv with sample data to use and code that should reproduce the error below.
import pandas as pd
import dask.dataframe

schema_keys = {'index': int, 'timestamp': int, 'origin_port': int}

df = dask.dataframe.read_csv(
    ['test_dask.csv'],
    dtype=schema_keys,
    converters={'timestamp': lambda x: pd.to_datetime(int(x), unit='ns')},
    blocksize=None
)
df = df.set_index('timestamp')
df = df.query("origin_port == 0")

def generate_new_columns(port):
    return pd.Series({'col1': '', 'col2': None})

newcols = df['origin_port'].apply(generate_new_columns, meta={'col1': str, 'col2': object})
df = dask.dataframe.concat([df, newcols], axis=1)
print(df.compute())

test_dask.csv.zip
The full traceback is
Traceback (most recent call last):
  File "test_dask.py", line 22, in <module>
    print(df.compute())
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/base.py", line 96, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/base.py", line 203, in compute
    results = get(dsk, keys, **kwargs)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/threaded.py", line 76, in get
    **kwargs)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/async.py", line 525, in get_async
    raise(remote_exception(res, tb))
dask.async.KeyError:<PHONE_NUMBER>084000000

Traceback
---------
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/async.py", line 291, in execute_task
    result = _execute_task(task, data)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/async.py", line 272, in _execute_task
    return func(*args2)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/dataframe/methods.py", line 58, in boundary_slice
    result = getattr(df, kind)[start:stop]
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/core/indexing.py", line 1312, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/core/indexing.py", line 1453, in _getitem_axis
    return self._get_slice_axis(key, axis=axis)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/core/indexing.py", line 1334, in _get_slice_axis
    slice_obj.step, kind=self.name)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/tseries/index.py", line 1498, in slice_indexer
    return Index.slice_indexer(self, start, end, step, kind=kind)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 2997, in slice_indexer
    kind=kind)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 3176, in slice_locs
    start_slice = self.get_slice_bound(start, 'left', kind)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 3125, in get_slice_bound
    raise err
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 3119, in get_slice_bound
    slc = self.get_loc(label)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/tseries/index.py", line 1402, in get_loc
    return Index.get_loc(self, key, method, tolerance)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 2136, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/index.pyx", line 553, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11829)
  File "pandas/index.pyx", line 578, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11425)
  File "pandas/index.pyx", line 175, in pandas.index.IndexEngine._get_loc_duplicates (pandas/index.c:4663)
  File "pandas/index.pyx", line 421, in pandas.index.Int64Engine._maybe_get_bool_indexer (pandas/index.c:8636)

Playing with this now.  It looks like the data and the divisions are slightly out of alignment.
> /home/mrocklin/Software/anaconda/lib/python3.6/site-packages/pandas/indexes/base.py(3176)slice_locs()
   3174         start_slice = None
   3175         if start is not None:
-> 3176             start_slice = self.get_slice_bound(start, 'left', kind)
   3177         if start_slice is None:
   3178             start_slice = 0

ipdb> pp self
DatetimeIndex([   '2017-01-01 00:00:00.188000',
               '2017-01-01 00:00:00.404999936',
               '2017-01-01 00:00:00.433000192',
               '2017-01-01 00:00:00.546999808',
                  '2017-01-01 00:00:00.636000',
               '2017-01-01 00:00:00.670000128',
               '2017-01-01 00:00:00.670000128',
               '2017-01-01 00:00:00.848999936',
                  '2017-01-01 00:00:01.116000',
                  '2017-01-01 00:00:01.428000',
               ...
               '2017-01-01 00:14:58.527000064',
               '2017-01-01 00:14:58.653000192',
               '2017-01-01 00:14:58.934000128',
               '2017-01-01 00:14:59.151000064',
               '2017-01-01 00:14:59.464999936',
               '2017-01-01 00:14:59.596999936',
                  '2017-01-01 00:14:59.732000',
               '2017-01-01 00:14:59.787000064',
                  '2017-01-01 00:14:59.824000',
               '2017-01-01 00:14:59.887000064'],
              dtype='datetime64[ns]', name='timestamp', length=6050, freq=None)
ipdb> pp start
Timestamp('2017-01-01 00:00:00.084000')
ipdb> pp kind
'loc'
ipdb> pp self.get_slice_bound(start, 'left', kind)
*** KeyError:<PHONE_NUMBER>084000000

So it looks like the .query hits a row with a division (the first one).
Should that force us to repartition?

Ah, interesting.  No, this shouldn't force a repartition.  Partitions are managed lazily before we are able to look at the values.  What this means I think is that we need to be more robust to the divisions not aligning perfectly with the data.
Perhaps boundary_slice should set the following:
start = min(start, df.index.min())
stop = max(stop, df.index.max())

OK, the docs currently say

Divisions includes the minimum value of every partition’s index and the maximum value of the last partition’s index.

So it will actually be something like "Divisions include a lower bound on every partition's index, and an upper bound on the last partition's index"?
I can look into changing that.

Yes, I don't think we're able to guarantee the current statement under
filtering operations like query.
On Thu, Apr 13, 2017 at 9:13 AM, Tom Augspurger<EMAIL_ADDRESS>wrote:

OK, the docs currently say
Divisions includes the minimum value of every partition’s index and the
maximum value of the last partition’s index.
So it will actually be something like "Divisions include a lower bound on
every partition's index, and an upper bound on the last partition's index"?
I can look into changing that.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2211#issuecomment-293891462, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszEhtgX19SzCBQUCVbvFaSw3145R7ks5rvh9ygaJpZM4M8G3C
.

Ok, so after #2214 we get
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/pand.py in <module>()
     15 df = df.set_index('timestamp')
     16 cdf = dask.dataframe.concat([df, df.query('origin_port == 0')], axis=1)
---> 17 print(cdf.compute())

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/base.py in compute(self, **kwargs)
     94             Extra keywords to forward to the scheduler ``get`` function.
     95         """
---> 96         (result,) = compute(self, traverse=False, **kwargs)
     97         return result
     98

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/base.py in compute(*args, **kwargs)
    201     dsk = collections_to_dsk(variables, optimize_graph, **kwargs)
    202     keys = [var._keys() for var in variables]
--> 203     results = get(dsk, keys, **kwargs)
    204
    205     results_iter = iter(results)

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/async.py in get_sync(dsk, keys, **kwargs)
    567     kwargs.pop('num_workers', None)    # if num_workers present, remove it
    568     return get_async(apply_sync, 1, dsk, keys,
--> 569                      raise_on_exception=True, **kwargs)
    570
    571

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/async.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, raise_on_exception, rerun_exceptions_locally, callbacks, dumps, loads, **kwargs)
    530
    531             while state['ready'] and len(state['running']) < num_workers:
--> 532                 fire_task()
    533
    534     except BaseException:

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/async.py in fire_task()
    506                         args=(key, dumps((dsk[key], data)),
    507                               dumps, loads, get_id, raise_on_exception),
--> 508                         callback=queue.put)
    509
    510         # Seed initial tasks into the thread pool

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/async.py in apply_sync(func, args, kwds, callback)
    555 def apply_sync(func, args=(), kwds={}, callback=None):
    556     """ A naive synchronous version of apply_async """
--> 557     res = func(*args, **kwds)
    558     if callback is not None:
    559         callback(res)

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/async.py in execute_task(key, task_info, dumps, loads, get_id, raise_on_exception)
    289     try:
    290         task, data = loads(task_info)
--> 291         result = _execute_task(task, data)
    292         id = get_id()
    293         result = dumps((result, None, id))

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/async.py in _execute_task(arg, cache, dsk)
    270         func, args = arg[0], arg[1:]
    271         args2 = [_execute_task(a, cache) for a in args]
--> 272         return func(*args2)
    273     elif not ishashable(arg):
    274         return arg

/Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/dask/dask/dataframe/methods.py in concat(dfs, axis, join, uniform)
    201     """
    202     if axis == 1:
--> 203         return pd.concat(dfs, axis=axis, join=join)
    204
    205     if len(dfs) == 1:

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/tools/merge.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, copy)
   1450                        verify_integrity=verify_integrity,
   1451                        copy=copy)
-> 1452     return op.get_result()
   1453
   1454

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/tools/merge.py in get_result(self)
   1648             new_data = concatenate_block_managers(
   1649                 mgrs_indexers, self.new_axes, concat_axis=self.axis,
-> 1650                 copy=self.copy)
   1651             if not self.copy:
   1652                 new_data._consolidate_inplace()

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/internals.py in concatenate_block_managers(mgrs_indexers, axes, concat_axis, copy)
   4825         placement=placement) for placement, join_units in concat_plan]
   4826
-> 4827     return BlockManager(blocks, axes)
   4828
   4829

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/internals.py in __init__(self, blocks, axes, do_integrity_check, fastpath)
   2759
   2760         if do_integrity_check:
-> 2761             self._verify_integrity()
   2762
   2763         self._consolidate_check()

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/internals.py in _verify_integrity(self)
   2969         for block in self.blocks:
   2970             if block._verify_integrity and block.shape[1:] != mgr_shape[1:]:
-> 2971                 construction_error(tot_items, block.shape[1:], self.axes)
   2972         if len(self.items) != tot_items:
   2973             raise AssertionError('Number of manager items must equal union of '

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/internals.py in construction_error(tot_items, block_shape, axes, e)
   4231         raise ValueError("Empty data passed with indices specified.")
   4232     raise ValueError("Shape of passed values is {0}, indices imply {1}".format(
-> 4233         passed, implied))
   4234
   4235

ValueError: Shape of passed values is (4, 7375), indices imply (4, 6365)

I think the issue is we have duplicates in the index. pandas.concat will work on dupes, but only if the indexes match:
In [23]: a = pd.Series([1, 2, 3, 4], index=[1, 2, 2, 3])

In [24]: b = pd.Series([1, 2, 3, 4], index=[0, 1, 2, 3])

In [25]: pd.concat([a, a], axis=1)  # duplicates, but match
Out[25]:
   0  1
1  1  1
2  2  2
2  3  3
3  4  4

In [26]: pd.concat([a, b], axis=1)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-26-19fff623438b> in <module>()
----> 1 pd.concat([a, b], axis=1)

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/tools/merge.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, copy)
   1450                        verify_integrity=verify_integrity,
   1451                        copy=copy)
-> 1452     return op.get_result()
   1453
   1454

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/tools/merge.py in get_result(self)
   1625
   1626                 index, columns = self.new_axes
-> 1627                 df = cons(data, index=index)
   1628                 df.columns = columns
   1629                 return df.__finalize__(self, method='concat')

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/frame.py in __init__(self, data, index, columns, dtype, copy)
    264                                  dtype=dtype, copy=copy)
    265         elif isinstance(data, dict):
--> 266             mgr = self._init_dict(data, index, columns, dtype=dtype)
    267         elif isinstance(data, ma.MaskedArray):
    268             import numpy.ma.mrecords as mrecords

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/frame.py in _init_dict(self, data, index, columns, dtype)
    400             arrays = [data[k] for k in keys]
    401
--> 402         return _arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
    403
    404     def _init_ndarray(self, values, index, columns, dtype=None, copy=False):

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/frame.py in _arrays_to_mgr(arrays, arr_names, index, columns, dtype)
   5401
   5402     # don't force copy because getting jammed in an ndarray anyway
-> 5403     arrays = _homogenize(arrays, index, dtype)
   5404
   5405     # from BlockManager perspective

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/frame.py in _homogenize(data, index, dtype)
   5700                 # Forces alignment. No need to copy data since we
   5701                 # are putting it into an ndarray later
-> 5702                 v = v.reindex(index, copy=False)
   5703         else:
   5704             if isinstance(v, dict):

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/series.py in reindex(self, index, **kwargs)
   2360     @Appender(generic._shared_docs['reindex'] % _shared_doc_kwargs)
   2361     def reindex(self, index=None, **kwargs):
-> 2362         return super(Series, self).reindex(index=index, **kwargs)
   2363
   2364     @Appender(generic._shared_docs['fillna'] % _shared_doc_kwargs)

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/generic.py in reindex(self, *args, **kwargs)
   2257         # perform the reindex on the axes
   2258         return self._reindex_axes(axes, level, limit, tolerance, method,
-> 2259                                   fill_value, copy).__finalize__(self)
   2260
   2261     def _reindex_axes(self, axes, level, limit, tolerance, method, fill_value,

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/generic.py in _reindex_axes(self, axes, level, limit, tolerance, method, fill_value, copy)
   2275             obj = obj._reindex_with_indexers({axis: [new_index, indexer]},
   2276                                              fill_value=fill_value,
-> 2277                                              copy=copy, allow_dups=False)
   2278
   2279         return obj

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/generic.py in _reindex_with_indexers(self, reindexers, fill_value, copy, allow_dups)
   2369                                                 fill_value=fill_value,
   2370                                                 allow_dups=allow_dups,
-> 2371                                                 copy=copy)
   2372
   2373         if copy and new_data is self._data:

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/core/internals.py in reindex_indexer(self, new_axis, indexer, axis, fill_value, allow_dups, copy)
   3837         # some axes don't allow reindexing with dups
   3838         if not allow_dups:
-> 3839             self.axes[axis]._can_reindex(indexer)
   3840
   3841         if axis >= self.ndim:

/Users/taugspurger/.virtualenvs/dask-dev/lib/python3.6/site-packages/pandas/indexes/base.py in _can_reindex(self, indexer)
   2492         # trying to reindex on an axis with duplicates
   2493         if not self.is_unique and len(indexer):
-> 2494             raise ValueError("cannot reindex from a duplicate axis")
   2495
   2496     def reindex(self, target, method=None, level=None, limit=None,

ValueError: cannot reindex from a duplicate axis

I think this can be closed now. Running
newcols = df['origin_port'].apply(generate_new_columns, meta={'col1': str, 'col2': object})
df = pd.concat([df.compute(), newcols.compute()], axis=1)

gives the same error as running the dask version.

I would expect the indexes to match from the result of the apply. I did a newcols.index.equals(df.index) and it returned True in this case.

Which if I understand you correctly means there should not be an error?

My fault, I had modified your script to concat to the original, not the version filtered with .query.
Running your original script on my branch in #2214 does work.

Unfortunately while this fixes the issue for single files it still seems to occur for multiple files. I've attached some more sample data. If you modify the read_csv line to
df = dask.dataframe.read_csv(
    ['test_dask.csv', 'test_dask2.csv'],
    dtype=schema_keys,
    converters={'timestamp': lambda x: pd.to_datetime(int(x), unit='ns')},
    blocksize=None
)

You get the following
Traceback (most recent call last):
  File "test_dask.py", line 22, in <module>
    print(df.compute())
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/base.py", line 96, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/base.py", line 203, in compute
    results = get(dsk, keys, **kwargs)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/threaded.py", line 76, in get
    **kwargs)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/async.py", line 525, in get_async
    raise(remote_exception(res, tb))
dask.async.KeyError:<PHONE_NUMBER>084000000

Traceback
---------
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/async.py", line 291, in execute_task
    result = _execute_task(task, data)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/async.py", line 272, in _execute_task
    return func(*args2)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/dask/dataframe/methods.py", line 58, in boundary_slice
    result = getattr(df, kind)[start:stop]
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/core/indexing.py", line 1312, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/core/indexing.py", line 1453, in _getitem_axis
    return self._get_slice_axis(key, axis=axis)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/core/indexing.py", line 1334, in _get_slice_axis
    slice_obj.step, kind=self.name)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/tseries/index.py", line 1498, in slice_indexer
    return Index.slice_indexer(self, start, end, step, kind=kind)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 2997, in slice_indexer
    kind=kind)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 3176, in slice_locs
    start_slice = self.get_slice_bound(start, 'left', kind)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 3125, in get_slice_bound
    raise err
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 3119, in get_slice_bound
    slc = self.get_loc(label)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/tseries/index.py", line 1402, in get_loc
    return Index.get_loc(self, key, method, tolerance)
  File "/Users/shughes/miniconda3/lib/python3.5/site-packages/pandas/indexes/base.py", line 2136, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/index.pyx", line 553, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11829)
  File "pandas/index.pyx", line 578, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:11425)
  File "pandas/index.pyx", line 175, in pandas.index.IndexEngine._get_loc_duplicates (pandas/index.c:4663)
  File "pandas/index.pyx", line 421, in pandas.index.Int64Engine._maybe_get_bool_indexer (pandas/index.c:8636)

Ha, ok. In this case, Matt's suggestion of changing boundary slice to
start = max(start, df.index.min())
stop = min(stop, df.index.max())

does fix it. I'll put together a PR quick, once I verify that that change doesn't break anything else.

It breaks other things.
On Thu, Apr 13, 2017 at 1:09 PM, Tom Augspurger<EMAIL_ADDRESS>wrote:

Ha, ok. In this case, Matt's suggestion of changing boundary slice to
start = max(start, df.index.min())
stop = min(stop, df.index.max())
does fix it. I'll put together a PR quick, once I verify that that change
doesn't break anything else.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2211#issuecomment-293963128, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszKjHJ9LcsejFWF3ANElVfrOGwO6Pks5rvla5gaJpZM4M8G3C
.


is exist method to show modal by javascript?

i want control modal by javascript but i dont know how
i tried css option of modal change  display : block but that is not work
if exist document i can reference about this, tell me

Yes, is possible to show the modal with Javascript, you have to set opacity to 1 and pointer-events to all. Let me know if that works for you!

oh thank!!! this is work well!!!
i write like this
function showLoginMadal() {
	$("#modal").css({ "opacity": 1 , "pointer-events" :  "all" });
}
function closeLoginMadal() {
	$("#modal").css({"opacity" : 0, "pointer-events" : "none"});
}

that is work well really thank you!!
hava a nice day!!!

Great, have a nice day to you! And feel free to ask anything if you have another issue with Concise!


installation failed

my visual studio version is 2013
VSIXInstaller_c1f097ea-0e70-4a3f-9d3d-57f4e77231a9.log

AsmDude is only works (as advertised) on VS2015 and VS2017.