hostagent: increase nofile limit in darwin

Default nofile limit in the standard terminal in macOS is 256. You can see it with ulimit -n.
So when I tried to open over 240 connections to <IP_ADDRESS>, the ssh process crashed.
Default limit is too low for a vm that can run many applications.
I suggest to use the same approach as in golang - setting an increased limit for the lima needs.
PR relates on all types of networks, but only for the macOS.
I understand that users can increase the limit themselves using ulimit -n <N>. But the regular user expects to get a working environment without exploring such low-level features.

Go 1.19+ increase the limit only for itself. But in exec limit returns to original value. It means that any (*exec.Cmd).Run will be runned with the old limit.
An explicit call syscall.Setrlimit modifies atomic origRlimitNofile and the above condition does not affect the new exec's

Go 1.19+ increase the limit only for itself.

It does? That seems to contradict the 1.19 release notes, which state:

One impact of this change is that Go programs that in turn execute very old C programs in child processes may run those programs with too high a limit.

So if this change is indeed still necessary, then we obviously should raise the limit ourselves too. I'm generally prejudiced against using init functions for this because it makes error reporting/logging harder, but maybe we can assume that this call will never fail... 😄

This is actual for both Go 1.19 and Go 1.21.
A simple program for demonstration of limits:
package main

import (
	"fmt"
	"log"
	"os/exec"
	"syscall"
)

func printLimit() {
	cmd := exec.Command("ulimit", "-n")
	if out, err := cmd.CombinedOutput(); err != nil {
		log.Fatal(err)
	} else {
		fmt.Println(string(out))
	}
}

func main() {
	var limit syscall.Rlimit
	if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &limit); err == nil && limit.Cur != limit.Max {
		limit.Cur = limit.Max

		fmt.Println("ulimit before setrlimit")
		printLimit()
		_ = syscall.Setrlimit(syscall.RLIMIT_NOFILE, &limit)
		fmt.Println("ulimit after setrlimit")
		printLimit()

	}
}

On my M1 with go 1.19 it prints:
ulimit before setrlimit
256

ulimit after setrlimit
unlimited

In the worst case, during initialization there will be no increase in limits and everything will remain as before.
I have already applied the same patch to our Lima-based devenv and it showed good results. It seems that Lima will also benefit from this.

Should this also be applied on Linux then? I realize the default will be higher, but is there any reason not to raise to the max limit there as well?

Should this also be applied on Linux then? I realize the default will be higher, but is there any reason not to raise to the max limit there as well?

I think this can be applied to Linux as well. But different Linux distros may have different defaults. And since I'm not sure about Linux, I decided to make a patch for darwin first.
BTW: thanks for the quick review.


Update serde to 0.8.x?

Hi,
I've tried using ease with the following example:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]

extern crate serde;
extern crate serde_json;
extern crate ease;

use ease::{Url, Request};

#[derive(Debug, PartialEq, Serialize, Deserialize)]
pub struct MyResponse {
    ok: bool
}

fn main() {
    let url = Url::parse("www.example.com").unwrap();
    let response = Request::new(url).get();

    let payload = match response {
        Ok(payload) => payload,
        Err(e) => panic!("{}", "aaargh")
    };
    payload.from_json::<MyResponse>();
}

My Cargo.toml:
[package]
name = "ease-test"
version = "0.1.0"

[dependencies]
ease = "*"
serde_json = "0.8.1"
serde_macros = "*"
serde = "*"

Unfortunately I get a compile error:
error[E0277]: the trait bound `MyResponse: serde::de::Deserialize` is not satisfied
--> src/main.rs:20:13
   |
20 |     payload.from_json::<MyResponse>();
   |             ^^^^^^^^^

On the rust irc channel people suggested to upgrade the serde dependency of ease.
Do you think that would resolve the issue? If so, could you upgrade :relaxed:

You're in luck, someone made a pull request updating the serde dependency earlier today! I just forgot to publish a new version. Your code now builds on my machine. Thanks for this reminder to bump the version on crates.io :smile:


prompt ENV_ROOT path

Can add a feature to .sh script to prompt if i can install on pwd or "/usr" paths?

ENV_ROOT is where all the building happens. If you want to install the resulting binaries then copy the cp -r target/* /usr once the build is done.


Configure shell in a more obvious way.

Provide a single configuration option: higher priority than system
config, lower than -c. This is similar to the old utmp, but without the
weird name and the sucklessism that utmp never became $SHELL (because it
is a helper program).
Hardcode the /bin/sh fallback when detecting preferred shell fails.
Spend a few more lines to make the fallback sequence (IMO) less cryptic.

Rebased, squashed, fixed up a bit to work with C++ and not have weird casts or other issues, and landed in 773c8cd8fa84682b0c89bedda511a7f0f071f132.


chore: clean component server client tests

This change is

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 74.78%. Comparing base (dbc0b15) to head (4402ee0).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #233   +/-   ##
=======================================
  Coverage   74.78%   74.78%
=======================================
  Files          23       23
  Lines         916      916
  Branches      916      916
=======================================
  Hits          685      685
  Misses        187      187
  Partials       44       44

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix "Improved screen reader accessibility of captions" #1340

Hi @rylan and @johndyer,
I had my concerns about Pull #1340, but didn't have time to deal with the it until now.
As the code stands for the majority of videos, Pull #1340 does not improve accessibility for screen reader or other users, it degrades it
Most video (the default case), contains spoken commentary, dialog, a talking head or similar. In these cases subtitles or captions, aimed at non-native speakers (subtitles) or those who are hard of hearing (captions) do not contain any extra information over what is visible. So, for a screen reader user, turning them into a live-region merely obscures and interferes with the existing audio track.
There are at least three special cases:

The "subtitles" track is actually a text-audio description track, aimed at those who are blind. It contains text that is timed to be spoken by a screen reader when no one is speaking in the audio track of the video;

The listener is struggling to hear the speech in the audio track of the video, for example, because of a strong accent, or poor sound levels. The listener mutes the audio, and uses TTS to read the subtitles aloud;

A special case, where the video does not contain any spoken words, and a "subtitles" track is used to convey information. This is the case for "echo-hereweare.mp4" and "mediaelement.srt".

My current fix is to add a flag, to explicitly turn the captions into a WAI-ARIA live region.
As it stands, this flag needs to be set when the video player is created, so it doesn't deal with all the special cases above (it deals with 3, partially with 1). It does however, fix the issue I'm raising.
Best wishes,
Nick

Hi @rylan,
I've realised that what I put above may be a bit brusque, and doesn't give you credit for what is a useful and bold idea! (Its something that I've thought about for a while, and discussed with colleagues here at the Open University.)
We perhaps need to put our heads together, and work out how best to implement your idea, so that an end-user can dynamically switch on live-region pseudo-captions.
Yours,
Nick


Pasting text containing tabs into pwsh terminal attempts tab completion

Steps to reproduce
Copy the following:
Get-Process		# This is a test.

Paste it into a pwsh window.
Expected behavior
The following is displayed (or at least some other non-tab whitespace insertion):
Get-Process		# This is a test.

Actual behavior
The following is displayed:
Get-ProcessMitigation# This is a test.

Environment data
This is reproducible from within a pwsh.exe window, Windows Terminal, or the embedded VS Code terminal.
Name                           Value
----                           -----
PSVersion                      7.1.3
PSEdition                      Core
GitCommitId                    7.1.3
OS                             Microsoft Windows 10.0.19042
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

I think this is a duplicate of #15151 - see https://github.com/PowerShell/PowerShell/issues/15151#issuecomment-812926200 for an explanation.
Ctrl-V avoids the problem in regular console windows and Windows Terminal windows, thanks to PSReadLine, but seemingly not in Visual Studio Code, on all supported platforms.
The problem comes down to many environments implementing pasting as simulated typing.
In terminals on Unix-like platforms, the behavior depends on the specific terminal application.
The only terminal that I know of that offers support is iTerm2.app on macOS, which can be configured to convert tabs to spaces on pasting.


Audit: NFTFactory Comparison with Boolean constant

Comparison with Boolean constants is not required.

@parva-jain @techpandora @kishan79

@vishaldantuluri @techpandora Changes verified.


Large datasets navigation

Sometimes query may return large datasets, for example gs.select({ category: 'Person' }) called from client will send request to the server, then spread requests to all servers storing mentioned category then start receiving data by chunks in lazy mode. So client-side cursor will receive first chunk and will generate on('data') event and first 100 records will be available at the client-side to be iterated by cursor. But we may not want to transfer more before user navigate to below grid position. Is it ok for ours gui rendering console if cursor will have no all data at once and data will arrive chunk by chunk pushing to dataset? Also how GUI can inform cursor to get next chunk? @aqrln

Here is an example how cursors may fetch data @aqrln
I think we cant use just cursor.dataset: Array, we need something lige fetch(callback(err)) or to load all records to cursor and then it will be available in cursor.dataset or can use event to get by chunks on('data', callback)

@aqrln I forgot a link to mentioned example: https://github.com/metarhia/globalstorage/pull/194
Cursor and work in two modes: (1) dataset holder, (2) data transformation from parent cursor not holding datasets. So cursors are chained but we can materialize datasets at any step. I am going to add cursor.materialize() that will receive all data from parent cursor and save it it's own dataset. This allow us to minimize data copies in memory. Do we need to copy data again to redux store? Or we will not use cursor materialization and will collect aggregated/transformed data just in store?

@tshemsedinov store is immutable and can only be changed with pure functions (reducers) in response to actions, so if something changes, we need to copy it again.

See https://github.com/metarhia/globalstorage/issues/195


Add weather-dependent time-series for transmission line capacities

Detailed Description
Transmission capacities are influenced by the weather. We should add a conversion function for the cooling effect of transmission lines.
Possible Implementation
Goal:
Add a conversion function transmission/transmission_availability (or different) in atlite.convert.
Idea: Assume a set of transmission lines given as line shapes.

For each line, calculate overlapping weather cells -> use indicator_matrix if it works between properly between line shapes and area shapes. Otherwise define a intersection matrix (booleans would probably be sufficient) .
@nworbmot suggested to take the maximum temperature along the line into account. This might be tricky and requires to define a new  aggregation function in atlite.aggregate. For a first shot, we could instead use the mean temperature over the whole line.
Enable wind angle data in cutouts. This variable is nowhere in the supported features. Probably it makes sense to enable a direct retrieving of specific ERA5 variables.
Think about a clever implementation how to efficiently calculate the angle between wind direction and line direction. Probably it makes sense to calculate the to scale the (u, v) components (see table 2 in ERA5 data doc) to the components of a unit vector and do the same for the line direction. Then, calculate the dot product. if I'm not mistaken the current aggregation function would work for the average angle between wind and line.
Do research and derive a formula for the combined cooling effect of temperature, wind speed + angle on the conduction capacity.

minimum wind speed
maximum temperature


You can not intercept Packets or fleets

Movement only possible between planets.

Do you have an example of this? I know you can intercept fleets because it's been tested a lot. I haven't tried to target packets though.

Can't set my WP to a fleet fly in space. Engine only allowed target planet where i could change to a fleet. But i see no possibility to target a space flying fleet.

Oh darn, I broke this while improving the automatic stargate warp speed settings on adding a waypoint.


Add bool dtype support for PLY files

This PR addresses https://github.com/daavoo/pyntcloud/issues/309 and adds support for the file reader to deal with Boolean fields (writing boolean fields was already not an issue). It also adds a new test fixture file and a simple unit test.
The new test data PLY file contains the following header:

which loads to give the following dataframe - notice the newly inserted is_green column of type bool:

@daavoo - I've added a test that writes a PLY file containing Boolean data
https://github.com/daavoo/pyntcloud/pull/321/files?authenticity_token=oqEfG4St1pVEVUuZtjJvNZLORmUQnhhvJf%2BqJZLj2hUWQxRKGEq7MUcgaKZUuxi40njLo2jnNC8%2BBlap5EDukA%3D%3D&file-filters[]=.py#diff-021377fee185787af295603466e9f1f3fec039246017a515213eb7c2f2115888


Update text rendering of dependency graph

This is the current dependency graph. It is an ASCII rendering of
the PNG output from stack dot | ... | dot. -test and -bench
packages are omitted.
This graph was run through tred which eliminates redundant deps
(those which are implied by transitivity).

That sounds reasonable. It would be a pain to keep that thing up-to-date... And since everyone has stack dot accessible, it's just a matter of having graphviz, which is no biggie.


Finalize page footer content

To be approved:

approved as far as my authority goes

Approved, and we need to add a Language column


fix: Force output rows for new right batch in MergeJoin

What changes were proposed in this pull request?
Bug fix for MergeJoin which the join filter is not null. For example, in the tpcds 95, the subquery ws_wh  contains a join filter ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk
SELECT
    ws1.ws_order_number,
    ws1.ws_warehouse_sk wh1,
    ws2.ws_warehouse_sk wh2
  FROM web_sales ws1, web_sales ws2
  WHERE ws1.ws_order_number = ws2.ws_order_number
    AND ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk

How to reproduce this issue:
In tpc95 query, if the right matches contains more than one batch, and these batches are DictionaryVector.
When we addOutputRow() from those batches, SMJ join will only copy the Dictionary indices.
And then we decode the incorrect value from the last DictionaryVector.
void copyRow(
    const RowVectorPtr& source,
    vector_size_t sourceIndex,
    const RowVectorPtr& target,
    vector_size_t targetIndex,
    const std::vector<IdentityProjection>& projections) {
  for (const auto& projection : projections) {
    const auto& sourceChild = source->childAt(projection.inputChannel);
    const auto& targetChild = target->childAt(projection.outputChannel);
    targetChild->copy(sourceChild.get(), targetIndex, sourceIndex, 1);
  }
}

Why are the changes needed?
Fix data quality issue.
How was this patch tested?
Test with TPCDS 95 query and local query
SELECT
    ws1.ws_order_number,
    ws1.ws_warehouse_sk wh1,
    ws2.ws_warehouse_sk wh2,
    ws2.ws_warehouse_sk - ws1.ws_warehouse_sk as diff
  FROM web_sales ws1, web_sales ws2
  WHERE ws1.ws_order_number = ws2.ws_order_number
    AND ws1.ws_order_number is not null
    AND ws2.ws_order_number is not null
    AND ws1.ws_warehouse_sk - ws2.ws_warehouse_sk = 0;

@wankunde Thank you for addressing this issue. It is a known problem and can be resolved in https://github.com/facebookincubator/velox/pull/11771

@wankunde Thank you for addressing this issue. It is a known problem and can be resolved in #11771

Hi, @JkSelf , in #11771 only semi join and anti join are fixed with JoinTracker, but inner join with a filter has not been fixed.
In my case, the filter exprSet will eval on the incorrect input DictionaryVector.


Detect git repo

On using the command mct, the package should detect if it's a git repo or not. If not, it should return an error message saying The directory is not a git repository.

@khattakdev , could you please assign this issue to me? I have already fixed the issue and would like to push the changes. Thank you!

@shahmeerzaidi assigned

you assigned it to yourself :D


Support for overlapping entities?

I know my requirement is weird but hey... :)

Yes, I've actually been thinking about this before!
The good news is, the markup logic and underlying CSS already supports this in theory. Just tested it by modifying the HTML output by hand. (It only works for entities containing another entity though – actual "overlaps" are pretty hard to achieve, since this concept can't really be translated to HTML markup either.)

The main problem with the way displaCy ENT renders the entities is that it currently iterates over each entity, slices the text and wraps the entity span in <mark> and </mark>. This will obviously confuse the offset counter if one entity contains another.
I'm not sure what the best way to solve this would be... Maybe something along those lines, i.e. convert the entity list to a different format and add the opening/closing tags if an entity starts or ends at that index (untested, so the code is probably unideal and mostly for illustration purposes / a reminder for myself):
// the raw text
const text = 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.'

// the entities returned by the spaCy service
let ents = [{start: 0, end: 15, label: 'person'}, {start: 5, end: 10, label: 'org'}]

// convert array of ents to different format that could look like this:
ents = [
    { i: 0, type: 'start', label: 'PERSON' },
    { i: 5, type: 'start', label: 'ORG' },
    { i: 10, type: 'end', label: 'ORG' },
    { i: 15, type: 'end', label: 'PERSON' }
]

// string to collect HTML markup
let markup = ''

// offset counter
let offset = 0

// iterate over entities
ents.forEach(({ i, type, label }) => {
    // add text between offset and entity index
    markup += text.slice(offset, i)

    // add opening or closing <mark> tag
    markup += (type == 'start') ? `<mark data-entity="${label}">` : '</mark>'

    // set offset to index
    offset = i
})

// add last text slice
markup += text.slice(offset, text.length)

// Before rendering, convert the HTML to DOM nodes – displaCy ENT should only
// append nodes to the container, never place raw HTML in it. It shouldn't have to
// include or rely on HTML sanitising to prevent cross-site scripting if used in a
// hosted application (all a user would have to do is get the server to return
// malicious code inside/outside of entities).

Maybe there's a better way, though?

Hello @ines 🤗
I got it to work, using a slightly different data structure from yours, because we need to keep more ordering information, in order to nest nested marks in the right order.
This is basically what I'm doing (in Typescript so we have more type info. I've skipped the actual rendering section as it's very close to what you wrote):
interface Span {
	type:  string;
	start: number;
	end:   number;
}

interface SpanTag {
	span: Span;
	tag: "start" | "end";
}

class Displacy {
	static sortSpans(spans: Span[]) {
		spans.sort((a, b) => {  /// `a` should come first when the result is < 0
			if (a.start === b.start) {
				return b.end - a.end;   /// CAUTION.
			}
			return a.start - b.start;
		});

		// Check existence of **strict overlapping**
		spans.forEach((s, i) => {
			if (i < spans.length - 1) {
				const sNext = spans[i+1];
				if (s.start < sNext.start && s.end > sNext.start) {
					alert("Spans: strict overlapping");
				}
			}
		});
	}

	static render(text: string, spans: Span[]): string {
		this.sortSpans(spans);

		const tags: { [index: number]: SpanTag[] } = {};
		const __addTag = (i: number, s: Span, tag: "start" | "end") => {
			if (Array.isArray(tags[i])) {
				tags[i].push({ span: s, tag: tag });
			} else {
				tags[i] = [{ span: s, tag: tag }];
			}
		};
		for (const s of spans) {
			__addTag(s.start, s, "start");
			__addTag(s.end, s, "end");
		}

		[...]
	}
}

If there's interest, I could try and contribute this snippet more cleanly.

By the way this is the end product where we're using this algorithm:
https://huggingface.co/coref/

@julien-c Ahhhh, so you made this demo! This is really cool - I enjoyed playing with it, and I always love seeing adaptations of our visualisers.
I've actually been thinking about this problem again recently when building another demo (hopefully releasing that soon), which needed overlapping spans as well (both on the sentence, span and token level). Here's how I solved it:

Convert the spans into a list of indices of each individual start and end point.

const starts = words.map(span => ({ i: span.start, type: 'start' }));
const ends = words.map(span => ({ i: span.end, type: 'end' }));
const spans = [...starts, ...ends].sort((a, b) => a.i - b.i);

Use a variable to collect the raw HTML markup, Iterate over the spans and insert text.slice(offset, span.i), followed by an open <mark> tag if type == 'start' or a closing </mark> tag if type == 'end'.
When inserting the text, make sure it doesn't contain any raw HTML tags and all < and > are encoded. (It's obviously important to do this as late as possible and not on the original text – otherwise it fucks up the indices).
Convert the markup string into DOM nodes using the native JS API. If you don't care about XSS or handling unpredictable user input, you can skip this step.

const nodes = document.createRange().createContextualFragment(markup)
container.appendChild(nodes)

Your implementation is probably a bit more straightforward, though (especially in Vanilla JS), as it doesn't require converting the markup to DOM nodes. I haven't used createContextualFragment much, and I'm sure there are some caveats... However, if you were to turn this into a React component in JSX, it'd probably work pretty smoothly 😊


Add delete organization to the Org Profile's danger zone

should have a similar warning overlay as editing the org.

@russorat  where should the user be routed to after deleting the org? the first org maybe?


engraver writestatus call corrupts bytes at end of file when not using direct io

Testing some plots with TurboPlotter 0.9.1+ indicates end of plot file is invalid when NOT using direct io.   Used TP9K on both windows and mac with same result.
Tracked it down to this code that calls writestatus after the plot is written.
  if (! use_direct_io) {
            writestatus();
   }

Example cmd:
./plot64 -v -d ./plots -s 1200000 -n 1000  -t 8 -k 888888801 -X 0
A possible fix would be to increase the file size to hold the resume id and write it as much as needed.
Once the plot is complete, truncate to remove the unnecessary resume data at end of file.

Has been fixed in Engraver 2.0. Plot files status is written using indirect i/o now.


Unknown error code on ARMv7

When running on a Raspberry Pi 4b target (an ARMv7 architecture platform) keyls fails with an unknown error code:
$ keyls<EMAIL_ADDRESS>Error: PKCS#11: unknown (0xb6d0884000000000)

I traced this failure to the call to the Ctx::new() fn in the pkcs11 crate, specifically to the invocation of the C_GetFunctionList() PKCS#11 function). Using LD_DEBUG we see that immediately after looking up the C_GetFunctionList symbol the library is then finished with.
     14246:     calling init: /home/pi/yubihsm/yubihsm-shell-2.3.0b/build/pkcs11/yubihsm_pkcs11.so
     14246:
     14246:     opening file=/home/pi/yubihsm/yubihsm-shell-2.3.0b/build/pkcs11/yubihsm_pkcs11.so [0]; direct_opencount=1
     14246:
     14246:     symbol=C_GetFunctionList;  lookup in file=/home/pi/yubihsm/yubihsm-shell-2.3.0b/build/pkcs11/yubihsm_pkcs11.so [0]
     14246:     binding file /home/pi/yubihsm/yubihsm-shell-2.3.0b/build/pkcs11/yubihsm_pkcs11.so [0] to /home/pi/yubihsm/yubihsm-shell-2.3.0b/build/pkcs11/yubihsm_pkcs11.so [0]: normal symbol `C_GetFunctionList'
     14246:
     14246:     calling fini: /home/pi/yubihsm/yubihsm-shell-2.3.0b/build/pkcs11/yubihsm_pkcs11.so [0]

Injected eprintln!() calls in a local copy of the PKCS#11 v0.5.0 crate around the call showed the print before the call but not the print after the call:
let func: libloading::Symbol<
    unsafe extern "C" fn(CK_FUNCTION_LIST_PTR_PTR) -> CK_RV,
> = lib.get(b"C_GetFunctionList")?;
eprintln!("BEFORE");
match func(list.as_mut_ptr()) {
    CKR_OK => (),
    err => return Err(Error::Pkcs11(err)),
}
eprintln!("AFTER");

Note that pkcs11-tool didn't have any such problems using the same PKCS#11 library file on the same Raspberry Pi 4b host.
Also note that a quick test using the cryptoki crate instead of the pkcs11 crate showed that the pkcs11 crate fails but the cryptoki crate succeeds.

$ git clone https://github.com/ximon18/keyls.git
$ cd keyls
$ cargo build
$ target/debug/keyls<EMAIL_ADDRESS>Using PKCS#11 slot id 0 (0x0)
Found 4 keys
+------+-------------+----------------------+-----------+--------+
| ID   | Type        | Name                 | Algorithm | Length |
+------+-------------+----------------------+-----------+--------+
| 1EB1 | Private Key | Ximons test key pair | RSA       |        |
| 1EB1 | Public Key  | Ximons test key pair | RSA       | 2048   |
| AB51 | Private Key | Ximons test key pair | RSA       |        |
| AB51 | Public Key  | Ximons test key pair | RSA       | 2048   |
+------+-------------+----------------------+-----------+--------+

See also: https://github.com/mheese/rust-pkcs11/issues/51


Assign ipv6 address for guest vms

Hi @ccamacho
Thanks for the awesome project.
Can you please let me know how can we assign ipv6 address for the guest vm's in kubeinit?

How can we achieve this through kubeinit ?


Failed to run the tests in the browser

Have you read the FAQ and checked for duplicate open issues?
YES
What version of Shaka Player are you using?
2.5.20
Can you reproduce the issue with the latest code from master?
YES
What did you do?

I have cloned the repo and ran " ./build/test.py --browsers Chrome " in the shaka-player directory
What did you expect to happen?
program will compile successfully
What actually happened?

I have tried the same thing with other browsers also but facing the same problem

@sradhanjan Does all the browsers stops at the same test that is 1534 of 2090 .

@surajkumar-sk this is what I am facing with other browsers
Firefox

Edge

@sradhanjan Seems like firefox version 38 doesn't support MediaSource . its browser fault nothing can be done , we need to use different browser . or update firefox for latest version and check if that version supports MediaSource . i think the latest version of firefox does support MediaSource . so try updating it .
for edge it seems like karma is not able to find the edge browser . Is edge browser installed in you computer .
try above things or run the test again in chrome and post a screenshot again . try staying on the cmd window while the tests are running instead of using other applications .
I am not able to reproduce the error . the test runs fine for me on windows 8.1 chrome browser .

@surajkumar-sk it seems like your advice was successful

@surajkumar-sk It seems like your advice was successful, I stayed in cmd while running the tests

@sradhanjan Glad to help. @joeyparrish this issue has been solved and can be closed.


Error while writing out vendor tree because of sirupsen/logrus

What version of Go (go version) and dep (git describe --tags) are you using?
go version
go version go1.8.3 darwin/amd64

 git describe --tags
v0.1.0-191-g4bfa359

What dep command did you run?
dep init -v

Full Log:
dep init -v
Root project is "github.com/Skarlso/miner"
 4 transitively valid internal packages
 8 external packages imported from 6 projects
(0)   ✓ select (root)
(1)	? attempt github.com/fsouza/go-dockerclient with 1 pkgs; 1 versions to try
(1)	    try github.com/fsouza/go-dockerclient@master
(1)	✓ select github.com/fsouza/go-dockerclient@master w/1 pkgs
(2)	? attempt github.com/hashicorp/go-cleanhttp with 1 pkgs; 1 versions to try
(2)	    try github.com/hashicorp/go-cleanhttp@master
(2)	✓ select github.com/hashicorp/go-cleanhttp@master w/1 pkgs
(3)	? attempt gopkg.in/yaml.v2 with 1 pkgs; 1 versions to try
(3)	    try gopkg.in/yaml.v2@v2
(3)	✓ select gopkg.in/yaml.v2@v2 w/1 pkgs
(4)	? attempt github.com/docker/go-units with 1 pkgs; 5 versions to try
(4)	    try<EMAIL_ADDRESS>(4)	✓ select<EMAIL_ADDRESS>w/1 pkgs
(5)	? attempt github.com/Yitsushi/go-commander with 1 pkgs; 4 versions to try
(5)	    try<EMAIL_ADDRESS>(5)	✓ select<EMAIL_ADDRESS>w/1 pkgs
(6)	? attempt github.com/kardianos/osext with 1 pkgs; 2 versions to try
(6)	    try github.com/kardianos/osext@master
(6)	✓ select github.com/kardianos/osext@master w/1 pkgs
(7)	? attempt github.com/docker/docker with 7 pkgs; 206 versions to try
(7)	    try<EMAIL_ADDRESS>(7)	✓ select<EMAIL_ADDRESS>w/25 pkgs
(8)	? attempt golang.org/x/sys with 1 pkgs; 1 versions to try
(8)	    try golang.org/x/sys@master
(8)	✓ select golang.org/x/sys@master w/1 pkgs
(9)	? attempt github.com/Azure/go-ansiterm with 2 pkgs; 2 versions to try
(9)	    try github.com/Azure/go-ansiterm@master
(9)	✓ select github.com/Azure/go-ansiterm@master w/2 pkgs
(10)  ? attempt github.com/fatih/color with 1 pkgs; 10 versions to try
(10)      try<EMAIL_ADDRESS>(10)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(11)  ? attempt github.com/docker/go-connections with 1 pkgs; 8 versions to try
(11)      try<EMAIL_ADDRESS>(11)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(12)  ? revisit github.com/docker/docker to add 3 pkgs
(12)    ✓ include 16 more pkgs from<EMAIL_ADDRESS>(12)  ? revisit github.com/docker/go-connections to add 2 pkgs
(13)    ✓ include 2 more pkgs from<EMAIL_ADDRESS>(12)  ? attempt golang.org/x/net with 1 pkgs; 4 versions to try
(14)      try golang.org/x/net@master
(14)  ✓ select golang.org/x/net@master w/1 pkgs
(13)  ? attempt github.com/opencontainers/runc with 2 pkgs; 19 versions to try
(15)      try<EMAIL_ADDRESS>(15)  ✓ select<EMAIL_ADDRESS>w/2 pkgs
(14)  ? attempt github.com/mattn/go-isatty with 1 pkgs; 4 versions to try
(16)      try<EMAIL_ADDRESS>(16)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(15)  ? revisit golang.org/x/sys to add 1 pkgs
(17)    ✓ include 1 more pkgs from golang.org/x/sys@master
(15)  ? attempt github.com/sirupsen/logrus with 1 pkgs; 42 versions to try
(18)      try<EMAIL_ADDRESS>(18)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(16)  ? revisit golang.org/x/net to add 2 pkgs
(19)    ✓ include 2 more pkgs from golang.org/x/net@master
(16)  ? attempt github.com/Microsoft/go-winio with 1 pkgs; 30 versions to try
(20)      try<EMAIL_ADDRESS>(20)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(17)  ? attempt github.com/pkg/errors with 1 pkgs; 11 versions to try
(21)      try<EMAIL_ADDRESS>(21)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(18)  ? attempt github.com/docker/distribution with 1 pkgs; 60 versions to try
(22)      try<EMAIL_ADDRESS>(22)  ✓ select<EMAIL_ADDRESS>w/2 pkgs
(19)  ? attempt github.com/mattn/go-colorable with 1 pkgs; 10 versions to try
(23)      try<EMAIL_ADDRESS>(23)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
(20)  ? attempt github.com/Sirupsen/logrus with 1 pkgs; 42 versions to try
(24)      try<EMAIL_ADDRESS>(24)  ✓ select<EMAIL_ADDRESS>w/1 pkgs
  ✓ found solution with 58 packages from 20 projects

Solver wall times by segment:
     b-list-versions: 27.530092011s
         b-list-pkgs:  4.888328452s
              b-gmal:  4.345379635s
  b-deduce-proj-root:  444.165341ms
         select-atom:    40.34556ms
             satisfy:   30.271191ms
            new-atom:    5.598894ms
         select-root:    5.026212ms
               other:      82.817µs
     b-source-exists:      61.153µs
            add-atom:      23.999µs

  TOTAL: 37.289375265s

  Using ^2.0.0 as constraint for direct dep github.com/Yitsushi/go-commander
  Locking in 2.0.0 (6b77592) for direct dep github.com/Yitsushi/go-commander
  Locking in master (1f92242) for transitive dep golang.org/x/net
  Locking in v0.4.2 (f533f7a) for transitive dep github.com/Microsoft/go-winio
  Locking in v2.6.1 (a25b9ef) for transitive dep github.com/docker/distribution
  Using master as constraint for direct dep github.com/fsouza/go-dockerclient
  Locking in master (4bc6a18) for direct dep github.com/fsouza/go-dockerclient
  Locking in master (3573b8b) for transitive dep github.com/hashicorp/go-cleanhttp
  Locking in master (0e01648) for transitive dep golang.org/x/sys
  Locking in v1.0.0 (202f255) for transitive dep github.com/sirupsen/logrus
  Locking in v0.0.8 (941b50e) for transitive dep github.com/mattn/go-colorable
  Locking in v1.0.0 (202f255) for transitive dep github.com/Sirupsen/logrus
  Locking in master (ae77be6) for transitive dep github.com/kardianos/osext
  Locking in v1.13.1 (092cba3) for transitive dep github.com/docker/docker
  Locking in v0.8.0 (645ef00) for transitive dep github.com/pkg/errors
  Using v2 as constraint for direct dep gopkg.in/yaml.v2
  Locking in v2 (cd8b52f) for direct dep gopkg.in/yaml.v2
  Locking in v0.1.1 (baf6536) for transitive dep github.com/opencontainers/runc
  Using ^1.5.0 as constraint for direct dep github.com/fatih/color
  Locking in v1.5.0 (570b54c) for direct dep github.com/fatih/color
  Locking in v0.2.1 (990a1a1) for transitive dep github.com/docker/go-connections
  Locking in v0.0.2 (fc9e8d8) for transitive dep github.com/mattn/go-isatty
  Locking in v0.3.1 (f2d77a6) for transitive dep github.com/docker/go-units
  Locking in master (19f72df) for transitive dep github.com/Azure/go-ansiterm
safe write of manifest and lock: error while writing out vendor tree: error while exporting github.com/sirupsen/logrus: /var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/.gitignore already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/.travis.yml already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/CHANGELOG.md already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/LICENSE already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/README.md already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/alt_exit.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/alt_exit_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/doc.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/entry.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/entry_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/examples/basic/basic.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/examples/hook/hook.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/exported.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/formatter.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/formatter_bench_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hook_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hooks.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hooks/syslog/README.md already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hooks/syslog/syslog.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hooks/syslog/syslog_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hooks/test/test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/hooks/test/test_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/json_formatter.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/json_formatter_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/logger.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/logger_bench_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/logrus.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/logrus_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/terminal_appengine.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/terminal_bsd.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/terminal_linux.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/terminal_notwindows.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/terminal_solaris.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/terminal_windows.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/text_formatter.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/text_formatter_test.go already exists, no checkout
/var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep843060556/vendor/github.com/sirupsen/logrus/writer.go already exists, no checkout
: exit status 128

What did you expect to see?
For it to work.
What did you see instead?
Did not work and failed with error message.
safe write of manifest and lock: error while writing out vendor tree: error while exporting github.com/sirupsen/logrus: /var/folders/1n/k6w9b2g51hg_dhlh_c8q9q8r0000gn/T/dep777414686/vendor/github.com/sirupsen/logrus/.gitignore already exists, no checkout

You have github.com/sirupsen/logrus and github.com/Sirupsen/logrus in your dependency graph (note the difference in cases) since some of your dependencies still import github.com/Sirupsen/logrus.
We had an approach to this as described in  https://github.com/sirupsen/logrus/issues/553#issuecomment-307486868 but no one got to implement it yet. 😩

Well my dependencies have that. Damn that's not nice. :-( is there any kind of workaround for that?

I'm afraid that you can't solve this without updating dependencies at the moment (until we handle this case in dep).

But it's a transient dependency. I can't fix it. :( I don't depend on it. Some of my dependencies depend on it. How can I fix THAT? :)

@Skarlso yeah, until we fix it and fold cases together correctly, the only recourse you have is asking upstream to make the fix. Or, forking upstream and fixing it yourself, then pointing the import path to your fork with a source: directive in Gopkg.toml.

Oh I see. Thanks @sdboyer. :-)

Would case folding require changes to the go tool or compiler?
(Assuming import paths remained mixed case).

@sdboyer's proposed solution doesn't require any changes to go toolchain or compiler. We can handle this within dep (more specifically, within gps).

So, Glide solves this with a --force flag, which detects if there was a change in a repo path, and if yes, it overrides the previous one and uses that. It's dirty, but it does work. :) For brevity, and because I dislike slack (:P) I'm going to put the proposed solution here as well.

the system that deduces repo/project roots from import paths needs to be expanded to return some kind of informational structure of case sensitivity for different elements of the paths
the system that manages repositories on disk needs to learn how to read those case sensitivity info structures, and we'll need a new trie implementation that works with them
the solver itself will need to stop using simple map[string]<...> types, and use a custom map that, again, works with these case sensitivity info structures

at least, this is what a "proper" solution looks like
but the risk and complexity of that is considerable, so we'll likely have to settle for some harm reductive approaches in the meantime
but yeah, i need to strategize a bit so that i can break this up into issues

Even glide has a similar issue. You won't be able to install the dependencies for the code below if you are on a case-insensitive file system. And the real issue for me is that version selection was done twice (once for each case), and therefore you could have 2 versions of the same dependency.
package glidetest

import (
	l1 "github.com/Sirupsen/logrus"
	l2 "github.com/sirupsen/logrus"
)

func main() {
	_ = l1.New()
	_ = l2.New()
}

Ahhh totally forgot that that doesn't work. Right you are @ibrasho.

After #813 I made a naive attempt to resolve this by changing references from Sirupsen to sirupsen, but that turned out to be way too naive, since there are dependencies in the vendor directory that import Sirupsen/logrus, and that haven't been updated to sirupsen. So IMO this definitely requires a workaround of some type in dep. Who ever thought it would be OK to have a case-insensitive filesystem anyway!

This is also breaking our usage of dep in a Windows environment. Should dep be aware that it is running in a case-insensitive filesystem and in that case use the existing (the first one created wins) git repo?

@davidovich we may have to resort to something like that in order to stop the bleeding, but it's a pretty fundamental violation of dep's rules to do that. so, yeah...temporary. god, i hope 😨

Is there an easy way to tell what transitive dependency is importing Sirupsen/logrus (with a capital S)?

@tejasmanohar not currently, but #1079 will make it trivial.

(1/1) failed to export github.com/sirupsen/logrus: /var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/.gitignore already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/.travis.yml already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/CHANGELOG.md already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/LICENSE already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/README.md already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/alt_exit.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/alt_exit_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/appveyor.yml already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/doc.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/entry.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/entry_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/example_basic_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/example_hook_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/exported.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/formatter.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/formatter_bench_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hook_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hooks.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hooks/syslog/README.md already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hooks/syslog/syslog.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hooks/syslog/syslog_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hooks/test/test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/hooks/test/test_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/json_formatter.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/json_formatter_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/logger.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/logger_bench_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/logrus.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/logrus_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/terminal_bsd.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/terminal_linux.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/text_formatter.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/text_formatter_test.go already exists, no checkout
/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/writer.go already exists, no checkout
: command failed: [git checkout-index -a --prefix=/var/folders/q_/4bc8zypd4h96hvj_bjksfsq80000gp/T/dep173764965/vendor/github.com/sirupsen/logrus/]: exit status 128
safe write of manifest and lock: error while writing out vendor tree: failed to write dep tree

@kumarsiva07 right now, it takes a lot of manual work. once #1079 is in, dep should find a solution for you.
given that you're just trying to get over the initial hump, you might try compiling dep from that PR and running dep init for your case. at minimum, you should get a more useful error, and we could use some real-world testing of this!

I'm still encountering this on OSX with recent dep (1dc2d8b6)

@therealplato yeah I saw your note in slack - I'm honestly confused about how you arrived there 😢

i arrived there by trying to use a top-level vendor folder with leftover cruft in subfolders from old vendoring @sdboyer
@spenczar provided a workaround (at least for me on OSX)

delete all folders with unwanted capitalization
commit that deletion
dep ensure

One option could be to allow exclusion of transitive dependencies, no?

Actually, it's already possible, it seems!
ignored = ["github.com/Sirupsen/logrus"]

[[override]]
  name = "github.com/sirupsen/logrus"
  version = "^1.0.5"

@therealplato that also does not make sense. dep prepares its vendor folder in a separate dir, then moves it into place.
im honestly not sure if an ignore like that would have the desired effect. it...might, but it also might have some weird side effects, as I can't recall how we deal with case normalization in ignores.

Well, at least the vendor directory seems clean.
$ find . -name logrus
./vendor/github.com/sirupsen/logrus


NPM Publish Allow specifying tags or lack of for latest

Any solution for this issue:
https://github.com/microsoft/azure-pipelines-tasks/issues/9743
https://developercommunity.visualstudio.com/t/NPM-Publish-Allow-specifying-tags/10732497

I'm having the same issue


add send_from_filepath

I added a function named  send_from_filepath in helpers.py which user can download file from its absolute path and the filename just show to browser. sometimes , user store the file which rename  as MD5 value. so send_from_directory 。However, user can't get the correct name when downloading.
Best wish!

if have question ，please contact me as soon as possible

Thanks for your interest, but I don't think this function belongs in the framework. It's not clear how this is better or more secure than the existing functions.


[SIP-46] Proposal for strict pylint enforcement

[SIP] Proposal for strict pylint enforcement
Motivation
Over the past number of years there have been a number of initiatives to improve the quality of the Python code via a number of linters (flake8; deprecated), formatters (black, isort), and type checkers (mypy). The one elephant in the room is pylint, a powerful yet sometimes perceived unusable quality checker.
Though pylint is enabled as part of CI and tox many of the checks are ignored either globally or at the file level via:
# pylint: disable=C,R,W

From previous experiences the value of many of these linters/checkers only materializes when strict enforcement exists for the entire repo (somewhat akin to heard immunity). Hence rather than merely trying to ensure that new code meets a higher bar, we (as a community) need to roll up our sleeves and take the somewhat painstaking approach of systematically re-enabling checks, removing blanket disablement, etc. somewhat akin to how we were able to strictly enforce flake8 (PRs) and mypy (PRs). Once the entire code base adheres to the strictly enforced rules it is then relatively simple (in terms of effort) to ensure a PR adheres to the necessary checks.
Proposed Change
I propose that by strictly enforcing pylint we will drastically improve the quality, readability, and maintainability of the Python code. I have worked on a number of repos which strictly enforce pylint (with very few checks disabled) and unequivocally can say that having pylint enabled helped to improve the code quality (preventing a number of bugs), consistency, readability, etc. Additionally I have not found pylint to be unusable as others have stated and do not find it cumbersome to use (once the necessary ground work has been laid).
Note the scale and effort involved is greater than any of the previous checkers and thus will require a systematic approach (hopefully undertaken by a number of individuals in the community). This initiative may take a number of months to complete, i.e., for context having the Python code fully adhere to having all function definitions be typed took around three months with limited involvement.
The three steps which need to be undertaken (in order) are:

Bumping the version of pylint to the latest stable version (2.5.2 at time of writing). Currently we are using pylint 1.9.2 which was released on June 6, 2018.
Systematically† remove the globally disabled checkers‡.
Systematically† reenable checking for all files, i.e., removing the # pylint: disable=C,R,W comments.

† Note systematically implies via a slew of PRs which re-enables a check (or small subset of checks) and/or file(s) at a time.
‡ Note based on other repos which use pylint the only the following checks should be disabled:

bad-continuation which is incompatible with black.
missing-module-docstring since Python modules are typically not documented at the module/file level.

Additionally the following checks should be enabled:

useless-suppression ensures there are no unnecessary # pylint: disable suppressions.

New or Changed Public Interfaces
N/A
New dependencies
N/A
Migration Plan and Compatibility
N/A
Rejected Alternatives
A list of Python static analysis tools are described here.  Note that we previously used flake8 (in conjunction with pylint) but later deprecated it as felt that black, isort, and pylint represented a superset in terms of functionality.

+1, I'm supportive of the effort, happy to take one some of the work and I would love to involve the wider community for this effort. Maybe we can make a call to action on Slack / the mailing list and use this as on ramp for more contributions.
Side note I remember I tried to bump pylint to 2.x at least twice in the past while we were on travis and it was slower than the current version, and would go 10+ minutes without std output (there was no verbose option that would output progress) and Travis would die / timeout. Hoping most of this is resolved.

Tried to bump pylint here https://github.com/apache/incubator-superset/pull/10095 just to see if that'll work now that they release new versions and that we're using Github Actions

I don't believe this has gone up for a DISCUSS thread on the dev@ list, has it @john-bodley ? Shall we push this through?

I concur with the sentiment stated above that pylint requires so many linter directives that it is sometimes perceived as unusable.  Black and isort have their place but are not really focused on runtime safety and program correctness.
This repo currently has two undefined names in Python code that none of the tools above would find.  These appeared without being detected since #3264
I would encourage the use of the following command as part of the linting arsenal…
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
https://flake8.pycqa.org/en/latest/user/error-codes.html
This flake8 test selection does  not focus on "style violations" (the majority of flake8 error codes that psf/black can autocorrect).  Instead, these tests focus on runtime safety and correctness:

E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST).  Often these issues are a sign of unused code or code that has not been ported to Python 3.  These would be compile-time errors in a compiled language but in a dynamic language like Python, they result in the script halting/crashing on the user.
F63 tests are usually about the confusion between identity and equality in Python.  Use ==/!= to compare str, bytes, and int literals is the classic case.  These are areas where a == b is True but a is b is False (or vice versa).  Python >= 3.8 will raise SyntaxWarnings on these instances.
F7 tests logic errors and syntax errors in type hints
F82 tests are almost always undefined names which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3.  These also would be compile-time errors in a compiled language but in Python, a NameError is raised which will halt/crash the script on the user.


Metadata missing for single entity requests

for single entity requests like
http://<IP_ADDRESS>:3000/odata/People('2')

Metadata is not provided. Current result looks like this:
{
  "d": {
    "firstname": "Helmut",
    "lastname": "Tammen",
    "gender": null,
    "age": null,
    "id": 2
  }
}

Problem fixed (thanks to @renanwilliam). Service call from above now returns
{
  "d": {
    "firstname": "Helmut",
    "lastname": "Tammen",
    "gender": null,
    "age": null,
    "id": 2,
    "__metadata": {
      "uri": "http://<IP_ADDRESS>:3000/odata/People(2)",
      "type": "NODATASERVER.Person"
    }
  }
}


Enable to change to ternary with primitive numeric types (VB)

Fixes #745

Coverage increased (+0.004%) to 91.142% when pulling e83d55b1535f52c958d630d90d4529a1bde34b98 on giggio:gb/fix745 into 2c955d1b9c7ad7aeac0f3925288eb0658223da65 on code-cracker:master.


Refine csp checks for unsafeEval.

This PR refines the csp checks for function compilation because the default policy in WWA (Windows Web Apps) allows unsafe-eval. This would give a ~30% boost when evaluating expressions.

Corporate CLA: Microsoft.

Any other concerns or issues?

@jdalton can you please explain how this improves anything on Windows?
It looks like you are trying to force non-csp mode even when developer opts in to it, which changes the semantics of the opt-in.
What's the real world scenario you are trying to fix?

@IgorMinar Currently the csp check in Angular assumes that csp means unsafe eval is not allowed, however in Windows 10 WWA the default csp policy allows unsafe eval.

The benefit is to give a perf boost out of the box for those using Angular in WWAs.

The only side effect of opting into the CSP mode is that we don't use function constructor. So just by not opt-ing into the CSP mode you get the perf benefit you are looking for.
As far as I can tell the only impact of your change is that if someone explicitly opts into the CSP mode via ng-csp directive, we will ignore it and still use function constructor in $parse.

There is one more place where csp mode plays any role and that's when including the default stylesheet. I assume that on WWA this would throw, in which case you'd want to prevent that via ng-csp, which then forces you down the slow path in $parse. Is that the case?

if someone explicitly opts into the CSP mode via ng-csp directive, we will ignore it and still use function constructor in $parse.

It's not ignoring CSP, it's using unsafeEval if it's allowed by the users CSP policy which is a better check than blanket CSP "on" or "off".

There is one more place where csp mode plays any role and that's when including the default stylesheet. I assume that on WWA this would throw,

How's it loading the default stylesheet? Inline scripts are still not allowed by the Win 10 default policy.

We append an inline stylesheet to the head while booting, unless ng-csp is found in the doc.

unsafe-inline is allowed by the Win 10 default CSP policy too.

In that case then just don't use ng-csp in your main template and you are all set. no?

I understand now where you are coming from.
Angular does two things that are naughty from a CSP perspective:

Injecting inline styles into the DOM (see the bottom line of angular.js)
Evaluating strings as code (in the parser)

This PR changes things so that Angular will now always test to see if we are allowed to eval strings, even if CSP has been set.
I would like to suggest a more explicit API, which is to allow ngCsp to be able to configure the rules that we care about:
<html ng-csp="

Sorry, slipped on the wrong button there

if for whatever reason we need $sniffer.csp to distinguish between unsafe-inline and unsafe-eval, it would be preferable to remove the need to inline the stylesheet (and instead use https://developer.mozilla.org/en-US/docs/Web/API/CSSStyleSheet/insertRule to add the default styles to the document).
afterwards ng-csp would only control if we attempt to autodetect unsafe-eval or not.
the primary reason for ng-csp was to avoid autodetection which relies on throwing exceptions which is not nice and interferes with debugging.
but from what you described, you actually don't need to change angular in order to get your performance benefit.

but from what you described, you actually don't need to change angular in order to get your performance benefit.

Outside of default CSP policies users can provide their own CSP policies in apps or sites that while they may restrict inline stylesheets can allow unsafeEval. This would allow devs who still indicate CSP to be able to get a boost if their policies allow unsafeEval.

I think we can fix this in a cleaner way.
Either we remove the inline stylesheet from Angular (using insertRule) so that ng-csp only refers to unsafeEval or make ng-csp more configurable (by providing a value to the directive).
<html ng-csp-inline-style ng-csp-unsafe-eval>

or
<html ng-csp="inline-style;unsafe-eval">

The first option is preferable to us as it means that we don't have to worry about people tweaking their CSP config in subtle ways that we don't support.

Closing for now. @jdalton if you would like to progress this along the lines of one of the above solutions, then please open a new PR. I would suggest trying to fix it by removing our dependency on inline styles first.

@petebacondarwin I can dig it. Could you point me in the right direction?

nm found it :mag_right:

It looks like insertRule won't work with style-src 'self'. Is there something else you had in mind?

Ping @petebacondarwin. If the style issue can't be resolved then I think this PR still has merit.

Ping again. I'd like to continue this discussion since there doesn't seem like there's a way to avoid the css csp restriction.

@jdalton sorry about that - got a bit distracted by AngularU. Can we continue this next week? I am flying back to the UK tomorrow.

Can we continue this next week?

Sure thing :octocat:

OK, so the problem with style-src 'self' is that we can add a style element dynamically to a page, right.
So for example the following fails:
<!DOCTYPE html>
<html>
<head>
  <title>Content Security Policy Test</title>
  <meta http-equiv="Content-Security-Policy" content="style-src 'self'">
</head>
<body>
  <script type="text/javascript">
      var styleEl = document.createElement('style'),
      document.head.appendChild(styleEl);
  </script>
</body>
</html>

I wonder if we can use a SHA of the stylesheet to get around this somehow?
<!DOCTYPE html>
<html>
<head>
  <title>Content Security Policy Test</title>
  <meta http-equiv="Content-Security-Policy" content="style-src 'self' 'sha256-SYgyy9Wm0ECiqsk_A24OSP3RCfkJ0lt3HGKlCXLCjqc='">
  <style>
  @charset "UTF-8";

  [ng\:cloak], [ng-cloak], [data-ng-cloak], [x-ng-cloak],
  .ng-cloak, .x-ng-cloak,
  .ng-hide:not(.ng-hide-animate) {
    display: none !important;
  }

  ng\:form {
    display: block;
  }

  .ng-animate-shim {
    visibility:hidden;
  }

  .ng-anchor {
    position:absolute;
  }
  </style>
</head>
<body>
</body>
</html>

So this works:
<!DOCTYPE html>
<html>
<head>
  <title>Content Security Policy Test</title>
  <meta http-equiv="Content-Security-Policy" content="style-src 'self' 'sha256-1PxuDsPyGK6n-LZsMv0gG4lMX3i3XigG6h0CzPIjwrE='">
  <script src="angular.js"></script>
</head>
<body>
</body>
</html>

Here the SHA of the style block that is loaded by angular.js is provided as a meta tag.

Right! Now we are rocking. We can replace the bit that gets added to the end of angular.js, i.e. this bit:
!window.angular.$$csp() && window.angular.element(document).find('head').prepend('<style type="text/css">@charset "UTF-8";[ng\\:cloak],[ng-cloak],[data-ng-cloak],[x-ng-cloak],.ng-cloak,.x-ng-cloak,.ng-hide:not(.ng-hide-animate){display:none !important;}ng\\:form{display:block;}.ng-animate-shim{visibility:hidden;}.ng-anchor{position:absolute;}</style>');

with this bit of code instead:
var head = window.angular.element(document).find('head')
head.prepend('<meta http-equiv="Content-Security-Policy" content="style-src \'self\' \'sha256-1PxuDsPyGK6n-LZsMv0gG4lMX3i3XigG6h0CzPIjwrE=\'">')
head.prepend('<style type="text/css">@charset "UTF-8";[ng\\:cloak],[ng-cloak],[data-ng-cloak],[x-ng-cloak],.ng-cloak,.x-ng-cloak,.ng-hide:not(.ng-hide-animate){display:none !important;}ng\\:form{display:block;}.ng-animate-shim{visibility:hidden;}.ng-anchor{position:absolute;}</style>');

I haven't tested it with server provided CSP headers. I am not sure if the meta tag would be ignored if the server provides a style-src rule.

So whether or not this would work, I think we could state that if you want to have style-src rules in place then you must include our SHA as one of the options.
This means that we could dispense with checking for csp when adding the styles to the document. The overhead for the application developer is not much more than having to provide an external stylesheet when using such a CSP style rule.

@jdalton and @IgorMinar what do you think?

I'll test this in the morning. So your proposing it attempt to write the styles twice (one via the sha and one via style element)?
Might be good to add a code comment with how you sha encoded the css.

No, not twice. The SHA is just there to tell CSP that the inline styles are whitelisted.

I SHA encoded the string by simply causing the error in Chrome - it dumps out a valid SHA that you can then use. :-)

Just to be clear - my suggestion is that we simply include the inline style block in the angular.js codebase, without any CSP check, and publish the SHA of the block in our documentation.
If someone wants to turn on `style-src 'self'``, for instance, then they would have to add our SHA to the CSP rules that are provided to the browser.

Nooice!

Just to be clear - my suggestion is that we simply include the inline
style block in the angular.js codebase, without any CSP check, and publish
the SHA of the block in our documentation.

Would this mean that any change to the css is a breaking change?
The fact that most developers do not care about the css that we add, does
not make things any simpler

@lgalfaso - If we made a change to the inline CSS styles then when you upgrade to that version you will need to change the SHA that is provided in your CSP directives. This means that if you use bower or npm to install your angular files and it chose to automatically update angular from say 1.5.0 to 1.5.1 then your CSP controlled application would immediately fail.
But the failure would be immediate and obvious, and updating to the new SHA should not be particularly difficult. In fact if we provided the SHA in a file along with the distribution of angular then one could have arrange for your web server to use the SHA directly from that file when serving a CSP controlled application. In that case there would be nothing to do when upgrading.

@petebacondarwin ok. One more thing, why would we need to add the meta tag as in your proposal?
var head = window.angular.element(document).find('head')
head.prepend('<meta http-equiv="Content-Security-Policy" content="style-src \'self\' \'sha256-1PxuDsPyGK6n-LZsMv0gG4lMX3i3XigG6h0CzPIjwrE=\'">')
head.prepend('<style type="text/css">@charset "UTF-8";[ng\\:cloak],[ng-cloak],[data-ng-cloak],[x-ng-cloak],.ng-cloak,.x-ng-cloak,.ng-hide:not(.ng-hide-animate){display:none !important;}ng\\:form{display:block;}.ng-animate-shim{visibility:hidden;}.ng-anchor{position:absolute;}</style>');

This is, what would not work with just
var head = window.angular.element(document).find('head')
head.prepend('<style type="text/css">@charset "UTF-8";[ng\\:cloak],[ng-cloak],[data-ng-cloak],[x-ng-cloak],.ng-cloak,.x-ng-cloak,.ng-hide:not(.ng-hide-animate){display:none !important;}ng\\:form{display:block;}.ng-animate-shim{visibility:hidden;}.ng-anchor{position:absolute;}</style>');

@lgalfaso  absolutely. My final proposal did not include the meta tags

@petebacondarwin then your solution SGTM

I don't think relying on the SHA1 is a good idea.
All modern browser support CSP (ignoring IE 10-11 which only support sandbox policy which isn't of interest here)
While the hash usage appears to be supported by Chrome 40 and Firefox 29, Microsoft only claims to support CSP 1.0 (which does not include hash support) and god knows what Safari currently supports.

Microsoft only claims to support CSP 1.0

Microsoft's default CSP restrictions for MS Edge allow inline styles sheets.

I'm not talking about default CSP restrictions.
A while ago (#8459) @petebacondarwin figured out a, albeit hacky, way how to run Angular while disabling inline styles but still having the speed advantage of eval. I'd be rather unfortunate to loose that.

So for this PR is there anything that would need to tweak or change to make room for more style sheet discussions?

To be honest, I haven't really understood the aim of this PR.
From what I read here, WWAs have a default CSP policy which includes both unsafe-eval for javascript and unsafe-inline for Stylesheets. Since these are the only two directives which affect AngularJS, I'm missing what needs to change here. If people manually set ng-csp they're gonna sufffer some performance degradation, but do we really need to hold hands?
Slightly off topic: The best solution would be if browsers exposed the current security policy. This was once specified but for some reason it's absent from newer drafts.

@realityking See https://github.com/angular/angular.js/pull/11933#issuecomment-110505004.

If people manually set ng-csp they're gonna sufffer some performance degradation, but do we really need to hold hands?

Ya. Having CSP restrictions is fine or getting rid or the directives, if doable, is great. The issue is the assumption that CSP is an auto-out for compilation. In this case the assumption is incorrect and needlessly costing 30%. The PR attempts to break down the checks to be a bit more granular. I think the change is a positive one. The discussion on styles would be a good follow up PR.

So the deal is that Angular does two things that can be locked down by CSP:

unsafe eval
unsafe inline

Currently if you don't specify the ng-csp directive then Angular just tests for whether eval-ing a string throws an error. This causes an unwanted error message in the console.
To prevent this the developer can specify the ng-csp directive which tells Angular not to do either of the two things above.
@jdalton would like to refine this so that we can lock down only one or the other of these two things.
The SHA idea would mean that we don't need to lock down the inline styles issue, which means that Angular only need to lock down the unsafe eval, which could be done with ng-csp.
If this doesn't work then we should make ng-csp configurable to specify which of the two rules we need to lock down.
I would like to test the SHA approach on the browsers that we support before going with the second option.

A final option that I would consider is to simply have the option of a custom build that would exclude restricted aspects. I believe that @IgorMinar has generally been against this idea.

Yeah the custom build for the CSS was something I submitted way back: #8459
From the other options, I think I prefer this one:
<html ng-csp-inline-style ng-csp-unsafe-eval>

I think it would be more intuitive if it was:
<html ng-csp-no-inline-style ng-csp-no-unsafe-eval>

Here is another idea:
How about we use the fact that CSP rules can be defined as meta tags in the HTML. Angular will parse the meta tags for CSP rules and work out whether it is allowed to do inline-style and/or unsafe-eval.
This may require rules to be duplicated, both on the server and in the HTML but is not requiring specialised ng-csp style directives.

:-( I did some testing and indeed the SHA CSP header for styles doesn't work on
Safari 8.0.7
Firefox 34.0.5
So this is not a viable strategy. Although I think it was a nice idea.

Ok so for the unsafe eval side of things is there anything else to do/tweak?

We need to have a configurable csp sniffer value. I see that there are four options:

different builds of angular for different CSP configurations
configuration via the value of the ng-csp="..." attribute
configuration via new directives, e.g. ng-csp-no-unsafe-eval etc
configuration via standard CSP meta tags, e.g. <meta http-equiv="Content-Security-Policy" content="style-src \'self\'>

I would like to use the last one because, a) it doesn't require multiple build files, b) it is a standard way of specifying CSP directives anyway, and so would make sense to someone reading the HTML who doesn't know about angular.
What do people think?

After a discussion with @btford I am going to knock up a PR with the 4th idea

Actually, I began to implement that and found that there are so many potential combinations of directive rules that we would need to be able to parse it added far too much code.
Instead, I am going to go with the following:
You can choose from the following declarations in your application:
No declaration means that Angular will assume that you can do inline styles, but it will do a runtime check for unsafe-eval. [This is the same as we have currently]
<body>

A simple ng-csp (or data-ng-csp) attribute will tell Angular not to do either the inline styles or the unsafe eval. [This is the same as we have currently]
<body ng-csp>

Providing a value of allow-unsafe-eval will tell Angular that it should be able to do unsafe eval; there will be no runtime check. [This is new]
<body ng-csp="allow-unsafe-eval">

Providing a value of allow-unsafe-inline will tell Angular that it should be able to add the inline style; there will be no runtime check. [This is new]
<body ng-csp="allow-unsafe-inline">

@petebacondarwin what would happen if tomorrow there is another CSP parameter we would like it to be configurable? Are we going to keep on adding things to the ng-csp attribute? What would happen with the proposal if someone does ng-csp="allow-unsafe-inline;allow-unsafe-eval"?

@lgalfaso:
We only care about CSP rules that we actively break inside core, which are doing eval and injecting inline styles. If we started doing something else in core, which would break a CSP rule then we would need to add an option to deactivate that code if the application developer wanted to have that rule turned on.
In the scenario described above ng-csp="allow-unsafe-inline;allow-unsafe-eval" would say that we expect there to be no CSP rules that block eval or inline styles, so Angular will go ahead and use them. This is pretty much the same as not using the ng-csp attribute at all, except, in this case, we wouldn't even do the runtime check as we have explicitly said that we support unsafe-eval.
That being said I am wondering whether the rules should be the other way around, i.e. the two options would be:

no-inline-style: this would stop Angular from injecting the styles inline (and would require the app developer to link to an external CSS file.
no-unsafe-eval: this would stop Angular from optimising $parse with unsafe eval of strings.

Then you would have the following options for ng-csp:
No declaration means that Angular will assume that you can do inline styles, but it will do a runtime check for unsafe-eval. [This is the same as we have currently]
<body>

A simple ng-csp (or data-ng-csp) attribute will tell Angular not to do either the inline styles or the unsafe eval. [This is the same as we have currently]
<body ng-csp>

This tells Angular that we must not use eval, but that we can inject inline styles.
<body ng-csp="no-unsafe-eval">

This tells Angular that we must not inject styles, but that we can run eval.
<body ng-csp="no-inline-style">

This tells Angular that we must not inject styles nor use eval, which is the same as an empty: ng-csp
<body ng-csp="no-inline-style;no-unsafe-eval">

If we add a new feature to Angular that potentially breaks a CSP rule then we would add a switch to turn it off.

@petebacondarwin This sounds reasonable to me. Can you post back a link to your PR once you have it up?
I'll close this one in favor of your PR.

Thanks. I will have it ready in a few hours - just going through the tests and docs

Here you go @jdalton: #12346

Thanks, will check it out :shipit:


Category module breaks language inheritance system

Create en empty shop
You now have 1 {{category}} record and 1 {{category_translation}} record
Now open this record (home) in your admin category module
** You should have selected your default language (this is default behavior - just to be sure)
When you now click SAVE (no changes made), the module sends a save request
You have now 1 {{category}} record and 2 {{category_translation}} record (1 additional record for each language)

This is REALLY bad, because when you now change some values in the default language, they will no more inherited to the other languages.

[public] Automated response: This issue is linked to the internal issue https://shopware.atlassian.net/browse/NEXT-36367.
[created from NEXT-36367, comment 486602]


Add generator for pySOT

First of a planned set of new optimization library options.
pySOT is a little tricky because it is not just a library of optimization functions but also has an interface to allow it to coordinate evaluations in an (optionally) asynchronous fashion. This complicates integrating it into libEnsemble and necessitates the use of running pySOT within a Process in the generator. Communication between pySOT's worker threads to the generator must then be handled to get evaluation points from pySOT and pass back the evaluation from libEnsemble's workers.

An initial implementation of the generator is working, though with some known bugs.
It seems that the communication between the generator and pySOT's worker threads can become deadlocked if the number of points in the experimental design is too small relative to the number of workers being used.
This might have to do with needing to complete the experimental design before optimization evaluations can begin but testing this has been inconclusive and is hampered by the error not being consistently reproducible when stepping through with a debugger.

This is finished in 3a12bf6eb182b332272f7dbe48277ba42fef81e3
Though the full set of available strategies, experimental designs, and surrogates available have not been added as options


Perplexity Metric Card

Add Perplexity metric card
Note that it is currently still missing the citation, but I plan to add it later today.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Otherwise, looks good! Good job, @emibaylor !


Question: I dont understand naive examples

Question: I dont understand naive examples
I need classify phrases sentiment (positive, negative) and in categories (errors / subscription / support), but i dont understand how to do it with this lib.
Im seeing other libs, and i can do something like this:
naive.add("i dont like this service", [negative])
naive.add("like this service", [positive])
naive.predict("i dont like it") // result: negative 100%, positive: 0%

or

naive.add("this app has a lot of bugs", [bug])
naive.add("my subscription price changed, grrrr", [subscription])
naive.predict("i got erros on new version") // result: bug 100%, subscription: 0%

Can you help me?

Hi @prsolucoes, sorry for the slight delay in getting back to you.
This lib implements the naive bayes algorithm, but it forces you to do all the feature extraction yourself, so you'd need to construct a DenseInstances with all of the right columns and then feed that to the naive bayes algorithm. I can provide further details if you're interested?

Hi,
Im interested :)
I made all thing today with python ntlk and textblob, but only the classifier, the rest is all Go. I want build the parser in Go too, but is too hard find a library like ntlk in go, your library appear be nice :)
My projects in go and other languages:
http://pcoutinho.com/projetos/
Thanks for the help.

Can you help me?


Add "Hide side panel button" config. Closes #117

Toggles side panel button on config changed, and respects current config value at startup.

Any news on this?


Reset Fingers state?

Hi, I was wondering if you could reset ripperdoc Fingers state after having punched him via a command.
I know you can give yourself his items but I don't like that. Any ideas?

Not sure exactly what your game is looking like. But I did something similar in mine. Can't remember if I loaded a previous save or if he just came back on his own.

I guess it could be possible with Game.SetDebugFact("xxxxxxxx", value)
(like Game.SetDebugFact("mq007_skippy_aim_at_head", 1))

Game.SetDebugFact("q105_fingers_dead", 0)
I know, he's not dead, but that's the name of the "fact"
Next time, check here : https://wiki.cybermods.net/cyber-engine-tweaks/console/how-do-i
and here : https://docs.google.com/spreadsheets/d/1aVeMGOg7mhrz-vgs5fFTrr72bh0kudK0jQmtSDYPjBQ/edit?usp=sharing


Change sleep with interruptible sleep

In the code we need to change sleep with thread interruptible sleep (e.g. during resends). May require a thread manager to access the thread.

fixed by https://github.com/hazelcast/hazelcast-cpp-client/pull/392 and


[FLINK-7556] Fix fetch size configurable in JDBCInputFormat for MySQL Driver

According to the MySQL documentation[1], it follows:

ResultSet

By default, ResultSets are completely retrieved and stored in memory. In most cases this is the most efficient way to operate and, due to the design of the MySQL network protocol, is easier to implement. If you are working with ResultSets that have a large number of rows or large values and cannot allocate heap space in your JVM for the memory required, you can tell the driver to stream the results back one row at a time.

To enable this functionality, create a Statement instance in the following manner:

stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
              java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);

The combination of a forward-only, read-only result set, with a fetch size of Integer.MIN_VALUE serves as a signal to the driver to stream result sets row-by-row. After this, any result sets created with the statement will be retrieved row-by-row.

Allow the Integer.MIN_VALUE to be accepted as a parameter for setFetchSize.
[1] - https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html

Thanks for the PR @nycholas.
+1 to merge

merging.


Update permissible SNS endpoint call response codes

https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.prepare.html
When an SNS subscription calls its endpoint, Amazon recognizes responses in the range of 200-499 as successful Notification deliveries. Responses outside of that range will trigger the Subscription's retry policy.
This PR updates the callEndpoint method to allow this range instead of just response codes divisible by 200.

I don’t have write access to merge this. What happens next? No rush. I’m just curious.

Hm, still figuring out how all this is set up.  I'll take a look at the settings for the future but of course, I merged it for now!  Thanks again for having a look at this!

This merge broke the app. Submitted #256 with the fix.


React example

Hello, thank you for this article. I'd like to request a similar guide using a React framework as ASP.NET Core includes React as one of the templates, then this might help React developers in the future. Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4c6d2219-cb28-5527-1079-6e3195cad282
Version Independent ID: 70c2aad0-6883-51a9-fed7-01be62302483
Content: Azure AD v2.0 JavaScript single-page application (SPA) guided setup
Content Source: articles/active-directory/develop/tutorial-v2-javascript-spa.md
Service: active-directory
GitHub Login: @andretms
Microsoft Alias: andret

@wonderplayer Thank you for your valuable feedback . We would definitely take this into consideration and would internally contact the content development team for reviewing this request.

@wonderplayer - Thank you for your request. I've opened user story 633695 in DevOps and assigned it to our feature PM to triage and consider the sample and docs request as part of our release planning.

Opened a DevOps user story for this and assigned it to PM for triage as we'll need to consider this as a separate samples and docs ask and will be part of planning.
#please-close


Tooltip labels

At the moment its impossible to know what each color on a stacked barchart corresponds to.
http://codesuki.github.io/react-d3-components/example.html
the tooltip method only recieves the x, y0 and y. It would be nice for it to also get the label of the dataset something like
this._data[this._data.length - 1].label

I think I had that in an old version. I will re-add it! Thank you!

I don't see this in the source for the current version. Do you still plan to re-add it?

Yes, i want to return an object which consists of the datapoint and the label.
In case you have some time feel free to send a pull request I will make a release out of it!

The way the callback is called between different chart types seems to be pretty customized for each one. It might make sense to consolidate this logic in the tooltip mixin or component, or in a HOC around the tooltip component.

I agree. Should probably be completely rewritten. Also in a future version charts will be able to be layered onto each other, the tool tip implementation then probably needs to change.

Hi everyone, i needed the label in the callback from tooltipHtml so, i went deeper and i changed  the Barchart.js file. For me this work fine but, if this is not a correct solution, it's ok too. =)
_tooltipHtml: function _tooltipHtml(d) {
var xScale = this._xScale;
var yScale = this._yScale;
    var data = this.props.data,
        x = this.props.x,
        y= this.props.y,
        y0= this.props.y0,
        label;

    data.map( function(dataObj,i){
      for (var j in dataObj) {
        if( j == 'values'){
          dataObj[j].map( function(value, i) {
            if(value.x == x(d) && value.y == y(d) && value.y0 == y0(d)) label = dataObj['label']
          })
        }
      }
    })

    var html = this.props.tooltipHtml(this.props.x(d), this.props.y0(d), this.props.y(d), label);

    var midPoint = xScale.rangeBand() / 2;
    var xPos = midPoint + xScale(this.props.x(d));

    var topStack = this._data[this._data.length - 1].values;
    var topElement = null;

    // TODO: this might not scale if dataset is huge.
    // consider pre-computing yPos for each X
    for (var i = 0; i < topStack.length; i++) {
        if (this.props.x(topStack[i]) === this.props.x(d)) {
            topElement = topStack[i];
            break;
        }
    }
    var yPos = yScale(this.props.y0(topElement) + this.props.y(topElement));

    return [html, xPos, yPos];
},

Sorry for my bad English, and this is great @codesuki , thanks a lot

Has the code been updated to allow you to access the label of the element? If my tooltip function is
function(x, y0, y) {

}

, how can I access the element's label?
Thanks!


[Relay][heterogeneous pass] remove on_device op after annotation

This PR removes the on_device operator after rewriting the annotated graph so that users can conveniently use this pass.
cc' @kazum @imorinaga  @jroesch @yzhliu

Looks good, idea is to only leave the copies in place?

@jroesch Yes. With on_device on the leaves, users have to specifically exclude them.

@imorinaga could you explicitly approve your changes using the interface?

Looks good to me.


memo_wise raises error when memoizing a method in a module that's included in a class that overrides initialize

I might be using memo_wise incorrectly here.. but seeing some odd behavior when I memoize a method in a module that's included in a class that has an initialize method.
require 'memo_wise'

module ModuleMethods
  prepend MemoWise

  def a_module_method
    self.class.name
  end
  memo_wise :a_module_method
end

# Works to call ClassWithoutInitializer.new.a_module_method
class ClassWithoutInitializer
  include ModuleMethods
end

# Does not work to call ClassWithInitializer.new("thing").a_module_method
class ClassWithInitializer
  include ModuleMethods

  def initialize(name)
    @name = name
  end
end

# Works
puts ClassWithoutInitializer.new.a_module_method

# Does not Work
# memo_wise-1.7.0/lib/memo_wise.rb:183:in `a_module_method': undefined method `fetch' for nil:NilClass (NoMethodError)
puts ClassWithInitializer.new("name").a_module_method

https://github.com/panorama-ed/memo_wise/pull/321 seems to fix the issue for us.


fix compiler error of https://github.com/piwik/piwik-sdk-ios/issues/208

make application functions fileprivate instead of private

Closing this PR since it appears to be a issue using the SDK. See #208 for more details.


JSON structure error fix for simple message

Code caused following error - 'Unable to post the Microsoft Teams message: ... Summary or Text is required. '. The structure of the message should contain the 'text' attribute instead of 'title'.

This is a read-only repository. Can you submit the same on symfony/symfony? Thank you.


Install rust-src when it is not found

Closes https://github.com/rust-analyzer/rust-analyzer/issues/2858
Looks like there's no RUSTUP env variable, so it adds one more thing to solve in https://github.com/rust-analyzer/rust-analyzer/issues/3154

bors r+

bors r-
bors d+
A haiku

Ok, looks a bit better now IMO.
I'll merge it in 20 mins, if no more proposals arise.

bors r+


How to use dockstring using specific protein (3D structure)

If we can use dockstring with 3D structure of protein. Could you please provide us an example to use dockstring with certain protein (3D structure)?
I am trying to use another target such CCND1. But dockstring dose not support it. So How can to add new target?

Hi, @amarye, thanks for your interest in dockstring. Your question is essentially a duplicate of #148 so I am closing this issue. Please see #148 for instructions on adding a new target and re-open the issue if you feel it is not adequately answered. Note that dockstring is not designed to support arbitrary targets, so the solution proposed there is more or less a hack.

I have added OCT2_target.pdbqt and OCT2_conf.txt files under the resources/targets, but I am still facing problem.
Additionally, I have renamed DRD2 to OCT2, I got the same problem. So could you please give us solution?

@amarye where did you install it? If you use pip install . then the package files will be copied to a new location, but if you run pip install -e . then the installation will be done "in-place". My guess is that you are putting the file into the wrong location.
If you still have issues, please provide us a full non-working example (e.g. a piece of code to run or a full stack trace) instead of an image.

Thank you very much for your response. I used 'pip install .' after using
'pip install -e .', and it worked. However, I'm now having trouble running
docstrings in Jupyter Notebook. Could you please help me solve this problem?
On Mon, 12 Jun 2023 at 23:25, Austin Tripp @.***> wrote:

@amarye https://github.com/amarye where did you install it? If you use pip
install . then the package files will be copied to a new location, but if
you run pip install -e . then the installation will be done "in-place".
My guess is that you are putting the file into the wrong location.
If you still have issues, please provide us a full example (e.g. a piece
of code to run or a full stack trace) instead of an image.
—
Reply to this email directly, view it on GitHub
https://github.com/dockstring/dockstring/issues/156#issuecomment-1587566801,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANJP4TFHZDQMCSUUYH2V6VTXK4YHJANCNFSM6AAAAAAY3WRAJE
.
You are receiving this because you were mentioned.Message ID:
@.***>

Thank you very much for your response. I used 'pip install .' after using 'pip install -e .', and it worked. However, I'm now having trouble running docstrings in Jupyter Notebook. Could you please help me solve this problem?

@amarye I'm sorry that you are still having problems, but I am busy and cannot provide help beyond fixing issues with dockstrinng. Try to debug it yourself, and if you believe your problem is due to dockstring itself (not your python setup more generally) then please open an issue clearly documenting your problem.


First release to production

Added index.html file.

@ivan-chukhalo, everything seems good. I merge your PR.
Good job.


Biom format compatability

This addresses #660 and adds compatibility to the biom-format.
The following have been addressed

[x] Tutorial is added show inghow LDVAE can be used to analyze microbiome data.
[x] biom is an optional dependency
[x] a read_biom function is added to locally load biom tables

There are some slight hiccups regarding how to keep track of the sample names (see 2nd to last code block in the notebook). This will likely be a recurring issue for users and may need to be addressed in a future pull request.

Hey @mortonjt,
Thanks for the PR! We're currently doing a major refactor of our Datasets so once it's finished, we can review and port this Biom compatibility over.

@mortonjt we are refactoring the dataset module to only use anndata. We can add a convenience method to read biom data and make it into anndata format. I'm just wondering, is this the most popular file type for microbiome data? Should the scvi.dataset.read_biom(filename, backup_url=...) function have any other functionality? Thanks!

Yes, I'd say that biom is the most common file format for sparse microbiome count tables. It looks like the data structures behind anndata are very similar to biom - I'm almost wondering if a to_anndata method would be appropriate to add directly into the biom-format repository.
One thing that I noticed with previous versions of scvi is that scvi loses information about the sample name (in your case the cell id).  It would make it more seamless to retain both row / column identifiers, addition to row / column metadata.

I'm almost wondering if a to_anndata method would be appropriate to add directly into the biom-format repository.

This seems like a better solution to me as it opens up the possibility of using many more single-cell algorithms with microbiome data. Anndata has the capability to preserve all row and column identifiers and associated metadata.

Hi @mortonjt, the time has now come. It would be extremely helpful if you refactored your tutorial to

Work in google colab (don't worry about it being tested with pytest)
Use the new 1.0 API
Submitted a PR to the contributed folder in this repo

To see how the new API works please look at the quick start tutorial here.
Full API documentation is also available on that page. Note that for LDVAE it's now scvi.models.LinearSCVI.

Great!!  I'll make sure to take another look at this (it'll probably happen
in ~3 weeks due to ongoing manuscript submissions).
On Sun, Sep 13, 2020 at 9:34 PM Adam Gayoso<EMAIL_ADDRESS>wrote:

Hi @mortonjt https://github.com/mortonjt, the time has now come. It
would be extremely helpful if you refactored your tutorial to

Work in google colab (don't worry about it being tested with pytest)
Use the new 1.0 API
Submitted a PR to the contributed folder in this repo
https://github.com/YosefLab/scvi_tutorials

To see how the new API works please look at the quick start tutorial here
https://scvi--750.org.readthedocs.build/en/750/user_guide/index.html.
Full API documentation is also available on that page. Note that for LDVAE
it's now scvi.models.LinearSCVI.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/YosefLab/scVI/pull/675#issuecomment-691791321, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AA75VXI3KG7RBYDOTO5L23DSFWFLBANCNFSM4NLZM36Q
.

Thanks! I'm going to close this PR for now.

Hi @adamgayoso , I'm taking another look at this; in order for the tutorial to work, we'd still need to have a BiomDataset object right? The PR you made to Biom enables the conversion to anndata, but that object will still need to be read in through the biom api, right?

@mortonjt, the tutorial will not be tested, so it should work entirely in google colab. You can use the first cell in our user guide notebooks as an example.

we'd still need to have a BiomDataset object right? The PR you made to Biom enables the conversion to anndata, but that object will still need to be read in through the biom api, right?

Correct, the first lines of the tutorial would download the biom file and load it with the biom api and then convert to anndata, or whatever is the most conventional workflow for this type of data at first.


SW-4618 Decrease vertical margin in batch event details modal

Current dependencies on/for this PR:

main

PR #1959   👈

This stack of pull requests is managed by Graphite.


upd(vscodium-deb): <IP_ADDRESS>25 -> <IP_ADDRESS>82

Update the VSCodium from previous version into latest. Merge immediately when everything's okay.

I'll recommit due little mistakes

What was the mistake?

What was the mistake?

I forgot something on my mind and I don't see labels, that's why I'd re-commit