Fix Issue 9661 - LockingTextWriter should increment file handle reference count

I also did the same for LockingBinaryWriter and added a test case for each.

Any reason not to turn the FILE *fps fields into properties, now that they can be reached through the added File field?

@CyberShadow
I don't know if it makes sense, but due to extra dereferences caused by the Impl* inside the File struct, I think it could be slower.
Maybe if we change the Impl to actually be embedded inside the File struct, we could do this.

@CyberShadow @DmitryOlshansky
Hi, is it alright now?
Thanks,
Alex

@CyberShadow @DmitryOlshansky
Trying to remove the FILE* member I have encountered several issues.
When accessing the FILE* through the File object, we have to add boilerplate code to check if File is correctly initialized (_p member is not null). And this has to be done every time we are trying to access the FILE*.
I think it's better to keep it this way, with the FILE* member also.
Thanks,
Alex

I think it's better to keep it this way, with the FILE* member also.

Sounds reasonable; feel free to leave this for another day. Maybe add this as a comment.

@CyberShadow
Where should I add the comment, in code or commit message?

And this has to be done every time we are trying to access the FILE*

First, the extra indirection is a bit more, but there's no reason we need to check to see if the File's implementation is null, as that is checked on entry, and it won't be null as long as the ref count is not 0. We know it's not 0 because we still have a reference to it! Or am I mistaken?

@schveiguy
In case the object is created using the constructor provided, then there is no need for checks.
The LockingTextWriter can however be created using .init, just as in the test case at line 3729, which would lead to an invalid File object.

Sure, but what are the consequences of that? I mean, it just dereferences a null pointer and crashes. I don't know what someone would expect to happen if you created a LockingTextReader that you can never properly use (the _file member and _fps member are private so you can't set them).
Note, the test you reference still would pass, because it doesn't actually use the LTR.

@schveiguy
The test would fail if no checks are added in the destructor, for example, when accessing the FILE* through the File object.
The way I see it, we have two options.
Either we assume that someone may want to use .init (like the test i mentioned) and keep it the way it is, or we assume that .init for LTW is invalid, remove that test and keep only the File object and access the FILE* though the 'file' member.

The test would fail if no checks are added in the destructor

Checking in the destructor is OK. That doesn't mean we have to check it everywhere. Note, we are already checking it in the destructor, so it won't be any worse.

we assume that .init for LTW is invalid

the .init value of a struct by default, must be destructable. Otherwise, you can't compose  It doesn't have to be usable.
I don't think the following use case makes any sense to do anything but crash or throw:
LockingTextWriter ltw;
ltw.put('c');

We can safely assume that this isn't going to happen. What we can do is assert the pointer in all uses except the destructor, which should remove the checks in release mode, but throw an Error in debug mode.

@schveiguy
What do you think about my latest commit?
Is it alright?
Thanks!

Restarted Circle CI, seems to have segfaulted.


Automatic chromatic deployment on PR approval

Only running chromatic at the end of a PR (instead of every push) has really saved us a lot on monthly snapshots. Mildly annoying to have to trigger it manually, so trying out this trigger method on submission of an approved PR review

@CodyCBakerPhD Looks like the workflow is invalid: https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/7357710172


Vagrant examples using out-dated user-data.sample

The user-data.sample file within the hack/ directory for Vagrant isn't up to date it seems. It's missing some stuff relating to the checkpointer's checkpoint-secrets directory, and im not sure what else, but the checkpointer isn't running when I attempt to use Vagrant.

Can you update with what hack cluster you're using (single-node or multi-node) and logs from kubelet trying to start the pod, or logs from the pod starting or something? I've run on single-node and multi-node several times without issue with the checkpointer.
Also, looking at https://github.com/coreos/bootkube/blob/master/hack/multi-node/user-data.sample#L32 it seems the dir is being created by that unit.

@chancez also, be sure you  rm -rf $dir/cluster in case you had old rendered cluster assets. The script does not remove and recreate that dir every run.

Oh whoops, I didn't have that in the README I was referencing. I had an open vim buffer referring to the old README before I ran git pull so I didn't notice that addition.


Client.setOptions() types definition is wrong

The type definition of Client.setOptions() does not comply with the js implementation both for the input parameter and for the output value.
The right definition should be:
declare class Client {
  // ...
  public setOptions(opts: Partial<common.IConfig>): void;
  // ...
}

The input parameter should be "partialized" because common.IConfig declares a mandatory apiKey field which is not actually required on js-side (this._opts = { ...this._opts, ...opts }) and the output value is just a void not the Client instance.

@StefanoMagrassi thanks for the feedback. This is an API which we don't think would be used often, so is relatively low priority. We will look into updating this.

This has now been fixed in v7.
https://github.com/bugsnag/bugsnag-js/releases/tag/v7.0.0


"if ANY Conditions are False" actions have V icon in policy tree, X icon in edit actions.

The 3 check icons are src="/assets/100/check-def54d82b0cb8ca4ad973e47a7c62ae6b09073bf137535d6119708613a05bc14.png",
the X icon is src="/assets/100/x-7b70a6351d2b6bf24c61ef6768858a6a35e78f9b4228d9df1c98154cfdd6c1a1.png".
@miq-bot add_label policy, ui, bug

@gmcculloug i was not able to recreate this issue on latest code base. Looked into how those icons are calculated to see if there were any recent changes that could have fixed the issue. Adding details of methods that are being called to calculate icons in tree as well as right side to see if you have any further details about this issue being fixed, please take a look:
right side of the lists(list of conditions that are True/False) are generated by calling model method actions_for_event from
https://github.com/ManageIQ/manageiq/blob/master/app/controllers/miq_policy_controller/events.rb#L122-L123
Images in tree nodes are being calculated by calling action_result_for_event method here https://github.com/ManageIQ/manageiq/blob/master/app/presenters/tree_node_builder.rb#L361
screenshot from my system displaying correct icons:

To be honest I haven't checked it on master but on my branch, which touches
policy to add new entities (as Pod in image below).
But I'm "pretty sure" I haven't changed VM policies which is what I tried
above.
I have a theory: My screenshot was with same action appearing on both
true/false branches, your was with different actions.
Let's try mixing:
[image: Встроенное изображение 2]
Aha!  The duplicated action has wrong V icon on left, the different action
has correct X on both sides.
And what if I remove the duplicate action from True branch?
[image: Встроенное изображение 3]
Bingo!
Sounds like a first-match lookup with not enough qualifiers...
=> Can you test with same action on True/False?  Or I'll get around to it
but not today.

[reposting with fixed images]
To be honest I haven't checked it on master but on my branch, which touches
policy to add new entities (as Pod in image below).
But I'm "pretty sure" I haven't changed VM policies which is what I tried
above.
I have a theory: My screenshot was with same action appearing on both
true/false branches, your was with different actions.
Let's try mixing:

Aha!  The duplicated action has wrong V icon on left, the different action
has correct X on both sides.
And what if I remove the duplicate action from True branch?

Bingo!
Sounds like a first-match lookup with not enough qualifiers...
=> Can you test with same action on True/False?  Or I'll get around to it
but not today.

@gmcculloug i was able to recreate this issue, it seems that condition https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_policy.rb#L110 needs to be fixed, if you have a same Action assigned to a policy twice, once for each Order of Actions if ALL Conditions are True and Order of Actions if ALL Conditions are False miq_policy_contents.find_by(:miq_action => action, :miq_event_definition => event)  gets same record both times and displays incorrect image in the tree. Can you take a look at this, let me know if we need to make code adjustments on UI side.

@bzwei Can you take a look.  Seems like we might need more information from the UI to identify the proper record during the lookup.

@miq-bot move_issue manageiq-ui-classic

Turns out it's already fixed on master.


Slow MySQL queries

Describe the bug
MySQL queries in the alt:V environment are way slower than in a normal NodeJS environment.
To Reproduce
Do a MySQL query and log the time it takes, you will see that a simple query takes about 100ms, while in normal NodeJS it doesn't even take a single ms.
Expected behavior
MySQL queries should be the same as in normal NodeJS:
Screenshots
N/A
Environment:

OS: Windows 10 / Ubuntu 20.04 (WSL2) (Both have the same problem)
alt:V server version: 2,2-rc12
Other installed modules: N/A
JS module version: 1.2-rc4

Additional context
This problem was described here already: https://github.com/altmp/altv-issues/issues/453

After researching this bug I have found the problem.
In CNodeScriptRuntime::OnTick the event loop gets updated, but this OnTick method is only called every 50ms which causes the event loop to be very slow.
There are now two options for fixing this:

Increase the tick count, so that it is called more often than 50ms
Handle updating the event loop independent from tick count

i make a test , the time of loop is called per iResource (i think but the reference of tick path to iResource)
i just make new thread when you start resource
in CNodeScriptRuntime i add this the method is not really proper because if resource stop i d'ont destroy the thread it's just for example the timer now is 3ms not really node performance but its correct
#include <iostream>
#include <thread>
#include <chrono>

void CNodeScriptRuntime::FastTick()
{
	using namespace std::chrono_literals;
	while (true) {
		std::this_thread::sleep_for(0.05ms); //idk if really 0.05 xD
		v8::Locker locker(isolate);
		v8::Isolate::Scope isolateScope(isolate);
		v8::SealHandleScope seal(isolate);

		uv_run(uv_default_loop(), UV_RUN_NOWAIT);
		platform->DrainTasks(isolate);
	}
}

and on the function CNodeScriptRuntime::CNodeScriptRuntime() i add
std::thread t(&CNodeScriptRuntime::FastTick,this);
t.detach();

That is not a valid solution because some of the alt:V functions have to be called on server tick.
That solution will probably break some alt:V functions.
VadZz will find a solution for this issue.

the tick alway exist but tick of tick of node overide alt tick function we have the essential and with better code of me it's possible to replace tick per owned function because we have the call when load and other and tick its called per alt and extended to iResource if you remove the tick of node alt:v tick alway active it's impossible to remove the alt tick call because he called on sdk and we juste have definition

This is fixed with the upgrade to NodeJS v14.

This is fixed with the upgrade to NodeJS v14.


Institutional Holders Missing

In https://github.com/ranaroussi/yfinance/blob/master/yfinance/base.py
        # holders
        url = "{}/{}/holders".format(self._scrape_url, self.ticker)
        holders = _pd.read_html(url)
        self._major_holders = holders[0]
        self._institutional_holders = holders[1] # --> If missing, will crash <--
        if 'Date Reported' in self._institutional_holders:
            self._institutional_holders['Date Reported'] = _pd.to_datetime(
                self._institutional_holders['Date Reported'])
        if '% Out' in self._institutional_holders:
            self._institutional_holders['% Out'] = self._institutional_holders[
                '% Out'].str.replace('%', '').astype(float)/100

yfinance is looking for institutional holders in holders[1] if element is missing the script will crash.

Made a PR for this issue a while back, however there have been no messages from the original author (owner of the master-branch) so I will close this issue for now.
(My solution was to make a fork of the repository)


Adding new line

Dear @zetamatta,
I’m looking for a tool to create table cells filled with data. CSView allows to insert (i) and append (a) cells, but it does so within a single line only. How do I add a new line?
$ type data.csv
City, Country

$ csview data.csv
City   Country
Moscow Russia         <-- How to create this line?

Thank you for using csview!
Now, I implemented them. Please try the new version.

o - append a new line after the current line.
O - insert a new line before the current line.
D - delete the current line.

@zetamatta, thanks you, it works as expected now.
However I am confused why a new piece of data is not saved when I quit.
Perhaps there should be three options: quit with saving, quit w/o saving, cancel.


Create Vm during client package tests if one doesn't already exist

This is something that will allow the client package tests to run from a blank slate. It also ensures that we can perform actions on this Vm without any worry that it is mutating any real infrastructure.
The vm will need to be created with a tag so that it can be easily queried to make later test invocations quick.

This was implemented in #125


Feature Request: Add event callback when click line

Hello, this library is really convenient, and I am very grateful to the author!
I am currently using this library at work for some batch update checks. In this process, I found that if I could directly click on the code diff to remove unnecessary diffs, it would be very convenient (I also use the just-diff package to patch the diffs). Currently, there is no way to achieve this requirement, so it's a pity. If possible, I hope to add event callbacks, which would really be helpful! If possible, I am willing  to contribute code.
Once again, thank you very much to the author.

PR is always welcome!
Just be curious: I don't know how you map the diff results to the original key or value; if I implement this feature, I can only provide the DiffResult object as the callback parameter, which seems not easy to achieve your original goal.

Yes, this is indeed a problem that I need to find a way to solve.
My intuitive thought is that the callback will carry information such as the content of the current line and the line number, and then compare it with the original information to determine the specific part of the diff. Since my current requirement is to replace keywords, the diff will only occur within a single line and not span multiple lines. Moreover, the possibility of the diff content being repetitive is quite low, and I can specifically handle this part of the bad case. The line numbers might not align due to different formatters, but in most scenarios, it should be feasible through content. I want to give it a try.

I see, you just want a feature like this:
interface RowClickInfo {
  // must have
  diff: [DiffResult, DiffResult];

  // nice to have
  side: 'left' | 'right';
  isLineNumberColumn: boolean;
}

const viewerProps: ViewerProps = {
  onRowClick: (info: RowClickInfo) => {
    const [left, right] = info.diff;
    console.log(left.lineNumber, left.text, left.comma);
  },
};

Feel free to submit a PR :)

Yes, this is good enough. I will submit this PR I think.


Code split and load prettier dynamically

CC: @farskid

@mfix22 I'll dig into it.

@farskid I am pretty sure Next.js uses Webpack to automatically code split dynamic imports

@mfix22 Is it gonna be the same way for other parsers too? using dynamic import?

Yes please (:

I'm on it :)


Hash lock secret being emitted

https://github.com/OpenSTFoundation/openst-protocol/blob/334b3edffb8e43004935640661c2954ca20fe3fa/contracts/OpenSTValue.sol#L49:50
Please excuse me if I'm being a bit naive here :) The hashlocking used in Simple Token seems to share some principles outlined here for atomic cross-chain trading:
https://en.bitcoin.it/wiki/Atomic_cross-chain_trading
In this case, revealing the hashlock too soon would mean that the second party could run off with the money. Is there a similar attack vector here, as _unlockSecret is emitted by the ProcessedStake event?

Hi @addywaddy
The similarities are indeed there, in the sense that the protocol creates an atomic action on two independent systems; contrary though to the referred algorithm, there is only one party involved here:  a user wants to stake $OST on Ethereum mainnet in order to obtain the equivalent value in utility tokens on the auxiliary utility chain.  When calling ProcessStaking or ProcessMinting on either side (Ethereum mainnet or the utility chain) indeed the secret is revealed for all to know - but this is the intention to have the atomic action: confirming on one side equals confirming on the other side.
So as a user, until you call any Process_ function you want to protect your secret; revealing it once, it should be known to all, so we decided to put it in the event explicitly.  The platform APIs are built to abstract these actions for an (honest) actor


Add additional config for isort and black

To reduce conflicts between the two formatters. Closes #721

thanks - can this go in the docs rather than in the readme? https://nbqa.readthedocs.io/en/latest/pre-commit.html


Monitoring v2 - Support the Coexistence of v1 and v2

We need to support the coexistence of monitoring v1 and v2 in the same downstream cluster.
Related issue: https://github.com/rancher/rancher/issues/25044

At the moment, Monitoring V1 and V2 cannot be supported together since both deploy Prometheus Operator pods that manage resources across all namespaces within the cluster. Both Monitoring V1 and Monitoring V2 would need to have the capability to limit their scope to particular namespaces (at the minimum) for both to be deployed onto the same cluster; we do not currently have support for this in Monitoring V1.
There's a UI ticket out in https://github.com/rancher/dashboard/issues/1167 to warn when trying to deploy Monitoring V2 on a cluster where Monitoring V1 is already deployed and more information on why this constraint exists in https://github.com/rancher/rancher/issues/25044.
Closing this issue out in favor of https://github.com/rancher/rancher/issues/25044 since it details the root cause for why this cannot be supported and tracks the broader issue that prevents us from supporting this coexistence in the future.


failed to clone from git- client invalid username or password

This is likely some local setting problem. Please also describe your problem (include what you tried step by step and what you expect) otherwise we cannot help.

first, download  SwaggerClient-php from GitHub and replace composer.json with
{
"repositories": [
{
"type": "git",
"url": "https://github.com/graphhopper/directions-api-client.git"
}
],
"require": {
"graphhopper/directions-api-client": "*@dev"
}
}
then simply run composer install


Spring Integration support not working in 2.0.0.M3

I am trying to execute same example code as https://github.com/artembilan/spring-integration-s1p2017/blob/master/src/main/java/org/springoneplatform2017/integration/function/SpringIntegrationS1p2017Function.java in a gradle project. Spring Function is not utilizing the bean generated by Spring Integration with version 2.0.0.M3.
The example works as expected when using 1.0.0.RELEASE version of Spring Cloud Function.
This is the build.gradle I am using to replicate the problem:
buildscript {
	repositories {
		mavenCentral()
	}
}

plugins {
	id 'java'
	id 'io.franzbecker.gradle-lombok' version '1.14'
	id "io.spring.dependency-management" version "1.0.5.RELEASE"
	id "org.springframework.boot" version "2.0.5.RELEASE"
}

group = 'com.sample.test.scf'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
	mavenCentral()
	maven { url "https://repo.spring.io/milestone" }
}

ext {
	springCloudVersion = 'Finchley.SR1'
}

dependencies {
	implementation('org.springframework.boot:spring-boot-starter-actuator')
	implementation('org.springframework.cloud:spring-cloud-function-web:2.0.0.M3')
	implementation('org.springframework.cloud:spring-cloud-function-core:2.0.0.M3')
    implementation('org.springframework.cloud:spring-cloud-function-deployer:2.0.0.M3')
//	implementation('org.springframework.cloud:spring-cloud-starter-function-web')
	implementation('org.springframework.boot:spring-boot-starter-integration')
	implementation('org.springframework.boot:spring-boot-starter-webflux')
	testImplementation('org.springframework.boot:spring-boot-starter-test')
	testImplementation('io.projectreactor:reactor-test')
}

dependencyManagement {
	imports {
		mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
	}
}

This is the @Bean declaration in my Spring Boot Application is:
    public IntegrationFlow uppercaseSiFlow() {
        return IntegrationFlows.from(MessageFunction.class, "uppercase")
                .<String,String>transform(String::toUpperCase)
                .log(LoggingHandler.Level.WARN)
                .bridge()
                .get();
}

There is the code in the ContextFunctionCatalogInitializer:
	@Override
		public void postProcessBeanDefinitionRegistry(BeanDefinitionRegistry registry)
				throws BeansException {
...

	if (context.getBeanFactory().getBeanNamesForType(FunctionCatalog.class, false,
					false).length == 0) {
				context.registerBean(InMemoryFunctionCatalog.class,
						() -> new InMemoryFunctionCatalog());
				context.registerBean(FunctionRegistrationPostProcessor.class,
						() -> new FunctionRegistrationPostProcessor(
								context.getBean(FunctionRegistry.class)));
			}
}

First of all we don't wait here for the ContextFunctionCatalogAutoConfiguration to be processed, therefore we register a InMemoryFunctionCatalog and then we end up with the :
   ContextFunctionCatalogAutoConfiguration:
      Did not match:
         - @ConditionalOnMissingBean (types: org.springframework.cloud.function.context.FunctionCatalog; SearchStrategy: all) found beans of type 'org.springframework.cloud.function.context.FunctionCatalog' org.springframework.cloud.function.context.catalog.InMemoryFunctionCatalog (OnBeanCondition)

This is how that InMemoryFunctionCatalog overrides potential way to get all the Function beans from the application context in the ContextFunctionCatalogAutoConfiguration and we rely in our app only on the FunctionRegistration - I assume manual registration.
Second any BeanPostProcessor should not be with autowiring. That's going to cause some problems during early initialization, e.g. missed Function bean registered by Spring Integration, for example.
I'm, talking about that context.getBean(FunctionRegistry.class). If we need that in the logic, it would be better to perform it lazily.
I don't know what is the purpose to always override a  ContextFunctionCatalogAutoConfiguration with the InMemoryFunctionCatalog, but this looks like the root of our problems.
We to schedule some discussion, @olegz .

@artembilan Recently we had similar lifecycle issue with Kotlin lambdas. (see https://github.com/spring-cloud/spring-cloud-function/issues/224 from @ilayaperumalg). It was fixed by moving post processing logic of Kotlin lambdas from BPP to BFPP (see https://github.com/spring-cloud/spring-cloud-function/commit/8eb7e06e02d5e5118ac60a5416ada6fb73b34aba). Perhaps the approach could be reused to address this specific issue with SI?

That's fully not similar and there is no way to fix it the same style in Spring Integration.
See my comment here: https://github.com/spring-cloud/spring-cloud-function/issues/227#issuecomment-434745740.
The problem with Spring Integration that there is a lot dependency injections from other services to be used in the IntegrationFlow, therefore we can't parse Java DSL so early in the BFPP.
You are welcome to come there and check if we can do anything on the matter, but believe me that we started from the BFPP in the beginning of Java DSL development and we failed because of dependency injection.
Another my point for this kind of issues that you are missing the fact that beans can be added at runtime and I don't talk here already about Spring Integration, although we recently did there a lot of work to meet runtime bean registration requirements, including HTTP and WebFlux endpoints mapping and also MBeans registration in JMX. So, maybe you can re-think the current @Autowired solution in favor of some BPP logic to track appropriate function beans whenever they appear in the application context...
I'm also open for the call if you feel we need to discussion anything else.
Thanks for understanding!

I think this has been resolved already and Artem pushed an updated sample.
I'll close it, but feel free to re-open it or raise a new one if you still think there is an issue


Replace spongycastle with bouncycastle

Changes proposed in this pull request:

Remove dependencies to spongycastle and use bouncycastle

@gnosis/mobile-devs

Codecov Report

Merging #34 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #34   +/-   ##
=========================================
  Coverage     62.21%   62.21%
  Complexity      152      152
=========================================
  Files            54       54
  Lines          1334     1334
  Branches        220      220
=========================================
  Hits            830      830
  Misses          404      404
  Partials        100      100

Impacted Files
Coverage Δ
Complexity Δ

...sis/svalinn/security/impls/AesEncryptionManager.kt
73.94% <ø> (ø)
24 <0> (ø)
:arrow_down:

...src/main/java/pm/gnosis/mnemonic/Bip39Generator.kt
86.95% <ø> (ø)
20 <0> (ø)
:arrow_down:

...pto/src/main/java/pm/gnosis/crypto/utils/Curves.kt
100% <ø> (ø)
3 <0> (ø)
:arrow_down:

...in/java/pm/gnosis/crypto/utils/CryptoExtensions.kt
73.33% <ø> (ø)
0 <0> (ø)
:arrow_down:

crypto/src/main/java/pm/gnosis/crypto/KeyPair.kt
62.18% <ø> (ø)
17 <0> (ø)
:arrow_down:

.../src/main/java/pm/gnosis/crypto/utils/Sha3Utils.kt
100% <ø> (ø)
9 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a5e8076...52853ce. Read the comment docs.


update readme with a basic picture of the ui

https://github.com/trietmnj/nsi-ui/blob/master/README.md

@trietmnj feel free to stage a pull request, but i think that UI is a bit behind master now. I have added the state boundaries from the census.
Also, feel free to create a branch directly on this repo if you want, i will add you to the org


CAS authentication support

CAS (Central Authentication Service) is a single sign-on protocol providing authentication with tickets. It is used by a lot of universities.
It would be nice to add support for CAS in gogs. I haven't found any complete client library in Go, just golang-cas-client, which started just one week ago. Fortunately, the basic functionality of CAS can be resumed in two parts:

if the user is not logged, redirect him to the CAS server, which redirect back to gogs with one ID;
send a request for a ticket with the ID to ensure that the user is logged.

I might make a simple working pull request if you find it interesting enough!

Close as no further feedback.


fix: mac os build

Description of change
Updated cargo.lock and macOS deployment target to fix the build of the native modules.
Links to any relevant issues
https://github.com/iotaledger/firefly/pull/5225
Type of change
Choose a type of change, and delete any options that are not relevant.

Bug fix (a non-breaking change which fixes an issue)

How the change has been tested
Built locally on Nicole's machine w/ the new Cargo.lock file.
Change checklist
Tick the boxes that are relevant to your changes, and delete any items that are not.

[x] I have followed the contribution guidelines for this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that new and existing unit tests pass locally with my changes

Converted to draft, since this PR might be superseded by a PR that downgrades rocksDB to 0.18.

CLosed, since not needed anymore.


Enable scroll when open modal

Hello,
Scroll doesn't change when we open a modal in legacy.semantic.
http://legacy.semantic-ui.com/modules/modal.html
But in Semantic 2, scroll disabling and it's a bad UX in my opinion.
http://semantic-ui.com/modules/modal.html
I tried .long.modal class but my content is just one line and it doesn't help me.
Is there any way to enable scroll when open modal in Semantic 2?
Thanks.

@jlukic Another adept for closing :)


upgrade to .NET 6

Went with v1.1.0 release with an amended commit. So be sure to re-clone the repo. 😄


Creating the Login & Sign-up page.

The login page required to be look into , its in a un-developed state.

Thanks for the PR. It has been merged.

You can work on this.


Please release a new helm chart version, 1.4.0 is currently 3 months stale

Your environment
Chart Version: 1.4.0
Helm Version: 3.6.3
Kubernetes Version:  1.2.0
What happened?
The current release of the helm chart contains several issues that have since been resolved and merged to main.
What did you expect to happen?
Regular releases of the helm chart occur to eliminate known, fixed issues and provide new features.
Steps to reproduce

helm repo add onepassword-connect https://1password.github.io/connect-helm-charts
helm search repo onepassword-connect
Observe that the last release is version 1.4.0
Observe that the tag for 1.4.0 in github shows the latest commit was merged in June 2021: https://github.com/1Password/connect-helm-charts/commits/connect-1.4.0

Notes & Logs
This is a particular issue for me as main contains changes that allow the operator to work across namespaces.

Current release 1.4.0 doesn't support multinamespace for the operator.
It is missing the clusterrolebinding.
Please release a new version including this.

Hey.
Great news. We released 1.5.0, which includes the clusterrolebinding compatibility, as well as watching all namespaces by default.

Closing as resolved now that 1.5.0 is out. Thank you for your patience while we got this release out!


Remove debug logs

I'm using the "mock_s3" module to mock some interactions with S3 in some unit tests. Is there a way to remove the debug-level logs that get printed?
I've tried putting the following at the top of my unit tests without success.
import logging
for name in logging.Logger.manager.loggerDict.keys():
    if ('moto' in name) or ('mock_s3' in name):
        logging.getLogger(name).setLevel(logging.CRITICAL)

Can you give an example of the logs you are seeing? Moto only logs in a couple places (Cloudformation comes to mind) so I would be surprised to see a lot of them.

If moto is running in server mode, you might have to disable Flask logging, since that does have quite a lot of logging.
Going to close this for now due to inactivity - feel free to reopen if it's still an issue


multiple index space grid operator apply

This pull request adds a version of apply() to the GridOperator which will thread over a user-specified set of index spaces. It uses the multiple index space version of Cajita::grid_parallel_for to accomplish this and therefore passes the user an additional index indicating which index space is currently active. This will be very useful for implementing boundary conditions or custom loop structures over subsets of the domain.

Codecov Report

Merging #35 (695560a) into master (752bebb) will increase coverage by 0.07%.
The diff coverage is 95.83%.

@@            Coverage Diff             @@
##           master      #35      +/-   ##
==========================================
+ Coverage   92.95%   93.02%   +0.07%
==========================================
  Files          43       43
  Lines        4129     4173      +44
==========================================
+ Hits         3838     3882      +44
  Misses        291      291

Impacted Files
Coverage Δ

unit_test/tstGridOperator3d.hpp
96.11% <93.33%> (-0.48%)
:arrow_down:

unit_test/tstGridOperator2d.hpp
95.94% <94.73%> (-0.42%)
:arrow_down:

src/Picasso_GridOperator.hpp
100.00% <100.00%> (ø)

src/Picasso_ParticleInit.hpp
95.31% <0.00%> (+2.88%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 752bebb...695560a. Read the comment docs.

It would be nice to have a test that uses multiple index spaces.


Handle undefined target value

@a8uhnf , Tour is working fine in my local without this PR changes. Can you check twice if this change is needed or not?


[Provider] [Data table] components don't work in playground

The two components mentioned don't render in the playground (both have similar ReferenceErrors saying the component is undefined)
My playground with the Data table component: https://polaris-playground.netlify.com/playground/cjfljagrm2flp0192c9fuiy3m

Followup:
I updated Polaris to the latest beta release (2.0.0-beta.14) and that resolved the issue with Provider, but Data table displays the following error:
TypeError: Cannot read property 'translate' of undefined


names changes with relations tags.

Style change requested
Name change on single item. point, line, and polygon with a relation. A road is created in 1700-01-01 but has had 3 names in the time since then.  Have the renderer show all three names from the relations from the time period.
Affected Tag or Tags
relations attached to a location.
Where this is visible
You can see why we need this in this location on the map:
https://www.openhistoricalmap.org/#map=18/47.75923/-121.76478&layers=O&date=1900&daterange=1800,2021

also could play with a line like this where there is no initial tags but the relations have the additional data within the relations.
https://www.openhistoricalmap.org/way/198660433#map=17/47.75748/-121.76467&layers=O&date=1900&daterange=1800,2021


Exposing moveEntities

I am working on a module that use the gamepad for movement, by moving the end-point of a ruler, to find the best path, and then actually moving the token once the player hits a button.
In other words it sort of use the analog stick in place of the mouse.
I got everything working, but to do the actual move, I had to manually edit the drag-ruler code
I would like to be able to call the moveEntities function of the foundry_imports.js from my module.
Any hope of seeing this exposed through api.js, or in the DragRulerRuler class?
Currently I just add moveEntities to the imports from "foundry_imports.js" at the top of ruler.js, and include a simple
async move()
{
moveEntities.call(this, this.draggedEntity, [this.draggedEntity]);
}
to the DragRulerRuler class.
But I'd rather not depend on having to manually change the ruler.js file every update.

I'm very hesitant to provide API endpoints within Drag Ruler itself, as that makes further development of the module harder. If there is another way I can help you achieve your goal, I'd probably prefer going that route instead.
There is a similar function called moveToken provided by foundries native ruler (which is the function that Drag Ruler's moveEntities was derived from). Maybe that function already satisfies your needs?

Thank you for the reply!
It looks like moveToken does what I need.
I'm a bit new to javascript, so couldn't really find a way to call the moveToken function of the native ruler. It kept calling the override in DragRulerRuler.
I was able to solve it by setting the _state of the ruler to Ruler.STATES.INACTIVE, right before calling moveToken(), causing if(!this.isDragRuler) to evaluate to true and calling super.moveToken in the override.
If you have any suggestion for a cleaner way that would be great, but it looks like I can now do everything I need, without having to modify any code in the drag-ruler module.

What you're doing there is probably the best way of achieving the desired result, yes.
Am I getting this right, that you're trying to trick the ruler into thinking it's a Drag Ruler, so you can leverage the color highlighting offered by Drag Ruler?


pyodbc supports queries with table valued parameters in sql server?

Environment

OS: Windows 64 bits
DB: Sql Server
driver: ODBC Driver 13 for SQL Server

Issue
I'm searching for a way to use a table valued parameter when quering a Sql Server 2017 database from python. Is it possible with pyodbc?

TVPs require access to descriptor fields (IPD/APD), which pyODBC currently does not support.

An update: @v-chojas and his team has expressed a desire to have pyodbc support this.  We don't have a time frame yet.

@v-chojas Thanks for finding the duplicate.  Since it has been asked again, can we clarify if you think this is something that should be added and if you and your team are considering working on this?  I can't find any notes on this.  Otherwise we should update this and close it.

@mkleehammer  Yes, we were planning on adding TVP and had done some preliminary design and investigation but other things (the attempted merging of fast_executemany, etc.) took priority. Now that the latter seems to have calmed down, we can resume working on TVP.

@glennmschultz please open a new issue and provide an ODBC trace. TVF is different from TVP.

(almost) a year since the last post: have you made any progress on adding TVPs to pyodbc?

It has been implemented: https://github.com/mkleehammer/pyodbc/pull/488

It has been implemented: #488

Can you please give an example/ sample implementation about how to use it?

@Lijo-CheeranJoseph - See the example in the tests.

The example shows the input to the stored proc is only a TVP, what if the input has a normal string variable along with the TVP?
Do I have to duplicate that string along each row in the params?

@sandeepnmenon

Do I have to duplicate that string along each row in the params?

No.
# a table-valued parameter is a list of tuples
tvp_data = [("4FE34A93-E574-04CC-200A-353F0D1770B1", 1.23),
            ("33F7504C-2BAC-1B83-01D1-7434A7BA6A17", 3.14),]

# parameter values for .execute() are a tuple
params_for_execute = ("my string value", tvp_data)
#                     --------+--------  ----+---
#                             |              |
#           a simple string --+              +-- a list of tuples

crsr.execute("{CALL my_sp (?, ?)}", params_for_execute)

@mkleehammer - I think we can close this now.

@gordthompson
Still I am getting the same error
sql_connection = pyodbc.connect(connection_string)

mappingvalues = []
mappingvalues.append(("8e4d7360-9a46-ea11-a812-000d3a54419d",0.6))
mappingvalues.append(("ed0ef314-9a46-ea11-a812-000d3a8d88aa",0.6))

params = ("1a1d243c-a587-4d10-995e-cbbf6cad4dce", mappingvalues, 0.5)
sqlCmd = "execute [dbo].[ExecuteMapping] @id=?, @values=?, @threshold=?"

sqlCursor = sql_connection.cursor()
result = sqlCursor.execute(sqlCmd, params)

Have explained this in detail in this issue : https://github.com/mkleehammer/pyodbc/issues/732


False positives

While migrating https://github.com/Particular/NServiceBus to netstandard2.0, in addition to #32, I've come across some additional false positives:

Process.Id isn't supported on Linux, MacOSX
ProcessStartInfo.UseShellExecute isn't supported on Windows
ManualResetEvent.ManualResetEvent(bool) isn't supported on Linux, MacOSX
MD5CryptoServiceProvider.MD5CryptoServiceProvider() isn't supported on MacOSX
SHA1CryptoServiceProvider.SHA1CryptoServiceProvider() isn't supported on MacOSX

UdpClient.BeginReceive(AsyncCallback, object) isn't supported on Linux, MacOSX

This seems to be working fine on Linux.

Process.Kill() isn't supported on Linux, MacOSX

This seems to be working fine on Linux as well (not verified on MacOSX)

@timbussmann regarding Process.Kill() it is the same issue of #72.


feat: support fastify - instrumentation-nestjs-core

Which problem is this PR solving?

Nestjs supports express and fastify. However, the @opentelemetry/instrumentation-nestjs-core package currently only supports requests of express.

For fastify:

Use req.url instead of req.originalUrl
Use req.routerPath instead of req.route.path

https://github.com/open-telemetry/opentelemetry-js-contrib/blob/44d21fe5030d2f64bedde1efc50a1173e0c49f3d/plugins/node/opentelemetry-instrumentation-nestjs-core/src/instrumentation.ts#L190-L191
Short description of the changes

Access the property as a request of express, and if it does not exist, access it as a fastify.

I am wondering how to write the test. I think it's simple and I don't think needs a test if possible 🤔.

Thanks for your contribution, @anchan828!


Reorganize tests to follow good practices

I reorganized the tests to follow good practices.
I also added the coverage xml command. This allows third-party development tools such as Visual Studio Code or coverage reports for CI to visualize code coverage. For instance, Visual Studio Code with the Coverage Gutters extension shows you in the code editor which lines are not tested.

By the way, sorry for nitpicking and moving around files, but it's less work later if we get this right first and then can copy the same approach to the other repositories.


🐛 Fix bug in project/env selection that results in a 404

This PR fixes a bug introduced by PR 676.  Prior to 676 changing environments or projects usign the modal presented by clicking the header would navigate users to teh feature flags view regardless of where they were in the applicaiton at the time.  This was not desired as a user may be viewing segments and intended to remain on that view when they switched environments or projects.  The PR to fix this inadvertently causes the browser to show a 404 when another project or environment is selected because the locale prefix is removed from the url. This PR fixes that issue.
To test:

Create multiple projects and environments and flags or use an existing installation that has these attributes.
Navigate the feature flag view
Click on the header and select a different environment.
Observe that the environment has changed and they data associated with that environment is displayed
Navigate to the segments view
Click on the  header and select a different environment or project
Observe that the environment or project has changed and the associated segment data is displayed.
Navigate to any view
Click the header and select a different environment or project
Observe that the environment has change but the browser remains in the same application area.

caused by #679


add instructions for Replay

How do I invalidate the saved replays or update them, etc.
(Trying to add IP address and MAC address to one of the tests to ensure that everything works end to end)

If you're just updating a single test, run sqlc in the test directory that you changed. I suggest building and installing a sqlc-dev binary that doesn't overwrite the sqlc you've installed locally.
go build -o ~/go/bin/sqlc-dev ./cmd/sqlc
cd internal/endtoend/testdata/name
sqlc-dev generate


Fastlane deliver init  ||  fastlane init   fails for Mac-app: "cannot find app" with correct identifier and team_id which work for fastlane deliver of iOS-app

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
Both 'fastlane init' and 'fastlane deliver init' fail for a MacOS app, for a project and team_id that works fine for the corresponding iOS app (same AppStore account, same team_id).
Message: Could not find app with app identifier '>redacted<' in your App Store Connect account
'fastlane release' correctly builds the app, but creates and .ipa instead of a .pkg and similarly fails to find the app.
Message: Could not find app with app identifier '>redacted<' in your App Store Connect account
Complete output when running fastlane, including the stack trace and command used

$ fastlane deliver init
[16:12:53]: To not be asked about this value, you can specify it using 'username'
[16:12:53]: Your Apple ID Username: redacted
[16:13:01]: Login to App Store Connect (redacted)
[16:13:04]: Login successful
[16:13:04]: The Bundle Identifier of your App: redacted
[!] Could not find app with app identifier 'redacted' in your App Store Connect account (redacted - Team: redacted)

Environment

✅ fastlane environment ✅
Stack

Key
Value

OS
10.14.1

Ruby
2.2.4

Bundler?
false

Git
git version 2.17.2 (Apple Git-113)

Installation Source
~/.fastlane/bin/bundle/bin/fastlane

Host
Mac OS X 10.14.1 (18B75)

Ruby Lib Dir
~/.fastlane/bin/bundle/lib

OpenSSL Version
OpenSSL 1.0.2g  1 Mar 2016

Is contained
false

Is homebrew
true

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode101.app/Contents/Developer/

Xcode Version
10.1

System Locale

Variable
Value

LANG
en_US.UTF-8
✅

LC_ALL
en_US.UTF-8
✅

LANGUAGE
en_US.UTF-8
✅

fastlane files:
`./fastlane/Fastfile`
# This file contains the fastlane.tools configuration
# You can find the documentation at https://docs.fastlane.tools
#
# For a list of all available actions, check out
#
#     https://docs.fastlane.tools/actions
#
# For a list of all available plugins, check out
#
#     https://docs.fastlane.tools/plugins/available-plugins
#

# Uncomment the line if you want fastlane to automatically update itself
# update_fastlane

default_platform(:mac)

platform :mac do
  desc "Push a new release build to the App Store"
  lane :release do

    build_app(workspace: "redacted.xcworkspace", scheme: "redacted",
              include_bitcode: false)

    upload_to_app_store(skip_metadata: true,
		        skip_screenshots: true,
			submit_for_review: true,
                        automatic_release: true,
                        force: false)

  end
end

`./fastlane/Appfile`
app_identifier("redacted") # The bundle identifier of your app
apple_id("redacted") # Your Apple email address

itc_team_id("redacted") # App Store Connect Team ID
team_id("redacted") # Developer Portal Team ID

# For more information about the Appfile, see:
#     https://docs.fastlane.tools/advanced/#appfile

fastlane gems

Gem
Version
Update-Status

fastlane
2.112.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

slack-notifier
2.3.2

atomos
0.1.3

CFPropertyList
3.0.0

claide
1.0.2

colored2
3.1.2

nanaimo
0.2.6

xcodeproj
1.7.0

rouge
2.0.7

xcpretty
0.3.0

terminal-notifier
1.8.0

unicode-display_width
1.1.3

terminal-table
1.7.3

plist
3.4.0

public_suffix
2.0.5

addressable
2.5.2

multipart-post
2.0.0

word_wrap
1.0.0

tty-screen
0.6.5

tty-cursor
0.6.0

tty-spinner
0.9.0

babosa
1.0.2

colored
1.2

highline
1.7.10

commander-fastlane
4.4.6

excon
0.62.0

faraday
0.12.1

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20170404

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

fastimage
2.1.5

gh_inspector
1.1.3

json
1.8.1

mini_magick
4.5.1

multi_json
1.13.1

multi_xml
0.6.0

rubyzip
1.2.2

security
0.1.3

xcpretty-travis-formatter
0.0.4

dotenv
2.5.0

bundler
1.14.6

faraday_middleware
<IP_ADDRESS>

naturally
2.2.0

simctl
1.6.5

uber
0.1.0

declarative
0.0.10

declarative-option
0.1.0

representable
3.0.4

retriable
3.1.2

mime-types-data
3.2018.0812

mime-types
3.1

jwt
2.1.0

signet
0.11.0

memoist
0.16.0

os
1.0.0

googleauth
0.6.7

httpclient
2.8.3

google-api-client
0.23.9

google-cloud-env
1.0.5

google-cloud-core
1.2.7

digest-crc
0.4.1

google-cloud-storage
1.15.0

emoji_regex
0.1.1

libxml-ruby
3.0.0

psych
2.0.8

generated on: 2018-12-28

The " in your Appfile, app_identifier look wrong.

Both 'fastlane init' and 'fastlane deliver init' fail on Mac, for a project and team_id that works fine for iOS projects.

What does that mean? If it fails on Mac, on what operating system can you successfully run it?

@janpio
Thanks for the reply.

The " in your Appfile, app_identifier look wrong.

The actual app-identifier was redacted. This is not the cause of the problem.
I had attempted to make this more clear. The text in the uploaded file read:
Could not find app with app identifier '' in your App Store Connect account
However, GitHub redacted the word  !! leading to the more unclear empty quotes.

What does that mean? If it fails on Mac, on what operating system can you successfully run it?

I have edited the sentence to make that more clear. The OS is MacOS. What was meant is: 'fastlane deliver' and 'fastlane init' fail for a MacOS app, even though on the same Mac, using the same AppStore account, 'fastlane deliver' works fine for the corresponding iOS app.

The " in your Appfile, app_identifier look wrong.

The actual app-identifier was redacted. This is not the cause of the problem.

... which is why I referred to the " characters that look wrong, not the actual app identifier. Did these also change when redacting things?
(The "Complete output when running fastlane, including the stack trace and command used" still is not readable)

I edited your issue to just use redacted where you redacted some sensitive values, instead of <redacted> which browsers don't display as they handle it as HTML tags or Github removes it for the same reasons.

This still happens after updating to fastlane-2.116.0
Fastlane release compiles, successfully logs in, then fails to find the app that exists, is referenced using the correct identifier, from the correct team ID
Could not find app with app identifier 'com.Correct.Identifier' in your App Store Connect account<EMAIL_ADDRESS>- Team: 1234567)

This still happens after updating to fastlane 2.117.1.
Added 2FA account to AppStoreConnect. Fastlane uses an AppManager Account that does not require 2FA. Login is indeed successful
Could not find app with app identifier 'com.Correct.Identifier' in your App Store Connect account<EMAIL_ADDRESS>- Team: 1234567
The app exists, the identifier is correct, the team is the same team mentioned for the iOS counter part of this MacOS app that IS successfully delivered via fastlane.

This still happens after updating to fastlane 2.119.0
2FA is setup on AppStoreConnect. Fastlane uses an AppManager Account that does not require 2FA. Login is indeed successful
Could not find app with app identifier 'com.Correct.Identifier' in your App Store Connect account<EMAIL_ADDRESS>- Team: 1234567
The app exists, the identifier is correct, the team is the same team mentioned for the iOS counterpart of this MacOS app that IS successfully delivered via fastlane.

I updated to 2.120.0, and met the same problem.

Adding the platform parameter resolved a similar issue for me. Try this:
fastlane deliver init --platform=osx

Thanks. Using the platform parameter 'fastlane deliver init' runs successfully. Downloads all metadata and screenshots and the app icon....
a subsequent 'fastlane release' however results in the same error as always: 'Could not find app with app identifier etc' for the exact same app that it just downloaded all metadata for...

You have to add the platform parameter to your Fastfile as well, e.g.
platform :mac do
  lane :release do
    deliver(
      app_identifier: "<Your-Bundle-Identifier>",
      platform: "osx"
    )
  end
end

You call this lane by executing fastlane mac release.

Thanks for the follow-up. I had already added the platform parameter to my Fastfile:
default_platform(:mac)
platform :mac do
  lane :release do
    build_app(scheme: "SomeScheme")
    upload_to_app_store(skip_metadata: true,
		        skip_screenshots: true,
			submit_for_review: true,
                        automatic_release: true,
                        force: true)
  end
end

I now changed to using deliver:
default_platform(:mac)
platform :mac do
  lane :release do
   build_app(scheme: "SomeScheme")
   deliver(
      app_identifier: "com.identifier.app",
      platform: "osx",
      submit_for_review: true,
      force: true,
      run_precheck_before_submit: false,
      automatic_release: true,
      skip_metadata: true,
      skip_screenshots: true,
      skip_binary_upload: false
    )
  end
end

Which, as before, successfully builds the app and creates and .ipa and zipped symbols, but now fails with: Could not find a version to edit for app ...
This feels like progress, because previously it could not find the app at all. Obviously still getting this wrong.

(deliver and upload_to_app_store are the same thing with different names, so that change makes no difference.)

@janpio Thanks for enlightening me on the fastlane lingo :-)
As far as the Could not find app with app identifier com.Correct.Identifier in your App Store Connect account error goes, this was fixed by addingapp_identifier to the fastfile!
However, in contrast to using deliver for iOS, deliver for mac does not subsequently read the new app version from the project, so I had to manually set that by adding app_version to the fastfile.
Then, again in constrat to using deliver for iOS apps, when using it as shown above for delivering a mac app, it does not actually create or upload an .ipa but just starts Waiting for candidate builds to appear...
This despite that the previous build_app step has correctly created an .xcarchive, and the subsequent deliver step mentions: Making sure the latest version on App Store Connect matches 'x.x.x' from the ipa file..., but as far as I can tell an .ipa was not actually created. Thus: Icon and metadata are successfully delivered but instead of creating and uploading an ipa as deliver for iOS does, with the current settings, deliver just waits for builds to appear, as indicated above.

@janpio Thanks for enlightening me on the fastlane lingo :-)

Yeah, that's all pretty awkward and messy.
deliver/upload_to_app_store should never actually build anything, not for iOS or Mac. The build has to be done by gym/build_app so it can later be picked up by following actions.

I will look into how to have gym create a .pkg for deliver to upload for a MacOS project.
I will shortly mark this issue as resolved, because the original issue has indeed been resolved.
Could not find app with app identifier 'com.Correct.Identifier' in your App Store Connect account was solved by adding app_identifier:  'com.Correct.Identifier'  as a parameter to deliver in the fastfile.

Is there anything fastlane could or should do better here, automatically pick up stuff that it does for iOS that it is not doing for Mac? If so, we can create a new issue and try to get this fixed.

I would love to open a new issue for giving that feedback. What is the best way to do that? Just open a new issue with title 'Suggestions for fastlane for Mac compared to iOS' ?

Yep, that works for me.


running ORQA demo gives error

I am running the following command (using tensorflow 2.3.1 on linux)
python -m language.orqa.predict.orqa_demo
--model_dir=https://console.cloud.google.com/storage/browser/orqa-data/orqa_nq_model
--port=8080
but I get the error :
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'https' not implemented (file: 'https://console.cloud.google.com/storage/browser/orqa-data/orqa_nq_model/params.json'
I replaced gs://... with https:// following the guidance in
You can use gsutil to download from links with the gs:// prefix. You can also browse such links by replacing gs:// with https://console.cloud.google.com/storage/browser/ (requires signing in with any Google account).

They are saying that you can download the files locally form your browser by replacing gs:// with https://console.cloud.google.com/storage/browser/
Once you download them locally form your browser (will require google login), you can then reference them as local files in your code.

Thanks. That's what I did.


Auto-detect ndk project

Automatically detects the presence of an NDK project by checking for the presence of an ExternalNativeBuild task. This makes project.bugsnag.ndk obsolete.

@kattrali updated to respect project.bugsnag.ndk if it's present, and otherwise automatically detect whether this is an NDK build.


Nutch 2096: Explicitly indicate broswer binary to use when selecting selenium remote option in config

*Assumes that the selenium grid is homogenous i.e. uses the same browser on the hub and the nodes and these are located at a similar location

Nice patch Kim.


hf: v1.12.0-rc0 + EgressGW & XDP fix

/ci-l4lb

Fixes: https://github.com/cilium/cilium/issues/17770

Fixes: https://github.com/cilium/cilium/issues/17770

@borkmann It doesn't fix for all cases. I.e., it doesn't fix for a case when iptables FORWARD chain's default policy is DROP.

Closing, as https://github.com/cilium/cilium/pull/19587 should resolve it.


Publish 5.0.x version on CocoaPods

PromiseKit version: 5.0.3
Xcode version: 9.0

Versions 5.0.x are not under CocoaPods. Could you push the latest version to trunk?
https://github.com/CocoaPods/Specs/tree/master/Specs/a/f/1/PromiseKit
https://cocoapods.org/pods/PromiseKit
-> PromiseKit (4.4.3)
   Promises for Swift & ObjC.
   pod 'PromiseKit', '~> 4.4.3'
   - Homepage: http://promisekit.org
   - Source:   https://github.com/mxcl/PromiseKit.git
   - Versions: 4.4.3, 4.4.2, 4.4.0, 4.3.2, 4.3.1, 4.2.2, 4.2.0, 4.1.7, 4.1.4, 4.1.3, 4.1.2, 4.1.0, 4.0.5, 4.0.4, 4.0.3, 4.0.1, 4.0.0, 3.5.3, 3.5.2, 3.5.1, 3.5.0, 3.4.4, 3.4.3,
   3.4.2, 3.4.1, 3.4.0, 3.3.0, 3.2.1, 3.2.0, 3.1.1, 3.1.0, 3.0.3, 3.0.2, 3.0.1, 3.0.0, 2.2.1, 2.2.0, 2.1.3, 2.1.2, 2.1.1, 2.1.0, 2.0.6, 2.0.5, 2.0.4, 2.0.3, 2.0.2, 2.0.1, 2.0.0,
   1.7.2, 1.7.1, 1.7.0, 1.6.0, 1.5.3, 1.5.2, 1.5.1, 1.5.0, 1.4.3, 1.4.2, 1.4.1, 1.4.0, 1.3.0, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2, 1.0.3, 1.0.2, 1.0.1, 1.0, 0.9.21, 0.9.19, 0.9.18,
   <IP_ADDRESS>, 0.9.17, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, 0.9.16, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, 0.9.15, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, 0.9.14, <IP_ADDRESS>,
   <IP_ADDRESS>, 0.9.13, <IP_ADDRESS>, 0.9.11, 0.9.10, 0.9.9, <IP_ADDRESS>, 0.9.8, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, 0.9.7, 0.9.6, 0.9.5, 0.9.4, 0.9.3, 0.9.2, 0.9.1, 0.9.0 [master
   repo]
   - Subspecs:
     - PromiseKit/Accounts (4.4.3)
     - PromiseKit/Alamofire (4.4.3)
     - PromiseKit/AddressBook (4.4.3)
     - PromiseKit/AssetsLibrary (4.4.3)
     - PromiseKit/AVFoundation (4.4.3)
     - PromiseKit/Bolts (4.4.3)
     - PromiseKit/CloudKit (4.4.3)
     - PromiseKit/CoreBluetooth (4.4.3)
     - PromiseKit/CorePromise (4.4.3)
     - PromiseKit/CoreLocation (4.4.3)
     - PromiseKit/EventKit (4.4.3)
     - PromiseKit/Foundation (4.4.3)
     - PromiseKit/MapKit (4.4.3)
     - PromiseKit/MessageUI (4.4.3)
     - PromiseKit/OMGHTTPURLRQ (4.4.3)
     - PromiseKit/Photos (4.4.3)
     - PromiseKit/QuartzCore (4.4.3)
     - PromiseKit/Social (4.4.3)
     - PromiseKit/StoreKit (4.4.3)
     - PromiseKit/SystemConfiguration (4.4.3)
     - PromiseKit/UIKit (4.4.3)
     - PromiseKit/UIImagePickerController (4.4.3)
     - PromiseKit/WatchConnectivity (4.4.3)

I am not ready to publish it to CocoaPods. Until then if you need it you can amend your Podfile too fetch ~> 5.0 via its tag.

Just in case anyone else wants to try out ~> 5.0 before it gets pushed to CocoaPods and tries to include a non-default subspec: you'll need to add submodules: true to your pod 'PromiseKit' line in your Podfile. Otherwise CocoaPods will silently fail to download and set up any of the subspecs.

What is the estimated release date for 5.0 on CocoaPods?

I need to write a blog post explaining the changes, I need to rework the docs so there are branches for v4 and v5 and then update the README. So it's contingent on that. Hopefully I'll have time over the holidays.

Awesome, thanks for the quick response. What's the likelihood of the current API changing before release? We'd like to make our own major update and include 5.0.0 in our own library before the holidays, and it would be contingent on the current 5.0.0 API not changing.

I respect semver so there will be no breaking changes until 6


Build fail on Linux

This occured to me when trying to install jekyll-webmention_io, see https://github.com/aarongustafson/jekyll-webmention_io/issues/75.
$ bundle install
Fetching gem metadata from https://rubygems.org/..........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Using public_suffix 2.0.5
Using addressable 2.5.1
Using bundler 1.16.0
Using colorator 1.1.0
Fetching unf_ext <IP_ADDRESS>
Installing unf_ext <IP_ADDRESS> with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: $HOME/.gem/ruby/2.3.0/gems/unf_ext-<IP_ADDRESS>/ext/unf_ext
/usr/bin/ruby23 -r ./siteconf20180225-16640-mruchr.rb extconf.rb
checking for main() in -lstdc++... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib64
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/bin/$(RUBY_BASE_NAME)23
        --with-static-libstdc++
        --without-static-libstdc++
        --with-stdc++lib
        --without-stdc++lib
/usr/lib64/ruby/2.3.0/mkmf.rb:456:in `try_do': The compiler failed to generate an
executable file. (RuntimeError)
You have to install development tools first.
        from /usr/lib64/ruby/2.3.0/mkmf.rb:541:in `try_link0'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:556:in `try_link'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:765:in `try_func'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:997:in `block in have_library'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:942:in `block in checking_for'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:350:in `block (2 levels) in postpone'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:320:in `open'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:350:in `block in postpone'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:320:in `open'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:346:in `postpone'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:941:in `checking_for'
        from /usr/lib64/ruby/2.3.0/mkmf.rb:992:in `have_library'
        from extconf.rb:6:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be
found here:

$HOME/.gem/ruby/2.3.0/extensions/x86_64-linux/2.3.0/unf_ext-<IP_ADDRESS>/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in $HOME/.gem/ruby/2.3.0/gems/unf_ext-<IP_ADDRESS>
for inspection.
Results logged to
$HOME/.gem/ruby/2.3.0/extensions/x86_64-linux/2.3.0/unf_ext-<IP_ADDRESS>/gem_make.out

An error occurred while installing unf_ext (<IP_ADDRESS>), and Bundler cannot
continue.
Make sure that `gem install unf_ext -v '<IP_ADDRESS>'` succeeds before bundling.

In Gemfile:
  jekyll-webmention_io was resolved to 2.8.5, which depends on
    http was resolved to 2.2.2, which depends on
      http-cookie was resolved to 1.0.3, which depends on
        domain_name was resolved to 0.5.20170404, which depends on
          unf was resolved to 0.1.4, which depends on
            unf_ext

any solution on this?

my solution is : apt-get install g++

Nothing to do with this extension:
checking for main() in -lstdc++... *** extconf.rb failed ***

This is telling you that you don't have a C++ compiler installed. You won't be able to install any native extensions that use C++.

G++ is installed
LC_ALL=C g++ --version
g++ (Gentoo Hardened 6.4.0-r1 p1.3) 6.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Still the same error.

same here:
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
/Users/kjian/.rvm/rubies/ruby-2.2.4/bin/ruby -r ./siteconf20180522-62521-1v2gv2c.rb extconf.rb

checking for main() in -lstdc++... yes
creating Makefile
...
unf was resolved to 0.1.4, which depends on
unf_ext

I have installed multiple Xcode in /Applications, when compiling unf_ext, it selected the wrong Xcode.
So, when i reset the compiling toolchain to default, it worked.
sudo xcode-select -r

Now we just need to find a pendant on Linux :-)

Any fix for this?

Well, it says to do that right in the error.  Why didn't I think of that?

Any solutions?
I get the same error.
sudo gem install unf_ext -v '<IP_ADDRESS>' --source 'https://rubygems.org/'
I was setting fastlane with Firebase Distribution and had to run bundle update
fastlane add_plugin firebase_app_distribution
Fetching unf_ext <IP_ADDRESS>
Installing unf_ext <IP_ADDRESS> with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

An error occurred while installing unf_ext (<IP_ADDRESS>), and Bundler
cannot continue.
Make sure that `gem install unf_ext -v '<IP_ADDRESS>' --source
'https://rubygems.org/'` succeeds before bundling.

I updated ruby to 2.6.3 and is works now!

Any fix for this?

Any fix for this? ruby 2.6.5 doesn't work

Any fix for this? ruby 2.6.0 doesn't work

I had the same issue, installing ruby-dev helped.
$ sudo apt-get install ruby-dev

I had the same issue, installing ruby-dev helped.
$ sudo apt-get install ruby-dev


Is there a way to output script execution in a formatted way?

I'm still testing is schemachange can be a replacement for how we deploy code using snowsql today.  In snowsql when code is run, our process automatically prefixes the code with the following snowsql commands so that we get a pretty, easy to audit log of what executed and the results of each executions.
!set exit_on_error=true
!set output_format=simple
!set remove_comments=False
!set timing=true
!set timing_in_output_file=true
!set echo=true
!set quiet=false
With schemachange currently, the output is harder to audit. For example:
Found Versioned file /builds/BPS_Database_DevOps/snowflake/cicdtesting-declarative/migrations/deploy/V2<IP_ADDRESS>__Deploy ingest objects.sql
Applying change script V2<IP_ADDRESS>__Deploy ingest objects.sql
Proceeding with private key authentication
SQL query: use schema ingest;
create or replace table test1 (col1 varchar);
use schema ingest;
create or replace table test2 (col1 varchar);
use schema ingest;
create or replace view view1
as select * from ingest.test1
Proceeding with private key authentication
SQL query: INSERT INTO CICD.DEV_CICDTESTING (VERSION, DESCRIPTION, SCRIPT, SCRIPT_TYPE, CHECKSUM, EXECUTION_TIME, STATUS, INSTALLED_BY, INSTALLED_ON) values ('2<IP_ADDRESS>','Deploy ingest objects','V2<IP_ADDRESS>__Deploy ingest objects.sql','V','99bd0eb49355f1c2c9fcc94b707c2b6d88a4582268ef6[189](https://---removed----)f5935c90',2,'Success','DEV_CICDTESTING_CICD_SVC',CURRENT_TIMESTAMP);
Successfully applied 1 change scripts (skipping 0)
Completed successfully

Is there a way to configure schemachange to adjust the formatting of the SQL execution and to return execution results, such as execution time and results?
An example of the output I get now is:
Executing from directory: D:\\GitLab-Runner\\builds\\VHxrjKBa\\0\\mitg_it\\mitg-bia\\script\\SNF\\BPS
/* ========= Place cut script below this line =============================================== */
/* === Build PUBLISH === */
-- Tables
!source DBObjects\3.PUBLISH\Tables\DEMO.sql
use database test_bps_db;
status
--------------------------------
Statement executed successfully.
1 Row(s) produced. Time Elapsed: 0.240s
use schema publish;
status
--------------------------------
Statement executed successfully.
1 Row(s) produced. Time Elapsed: 0.255s
create or replace table demo (col1 text);
status
--------------------------------
Table DEMO successfully created.
1 Row(s) produced. Time Elapsed: 0.237s
/* === END OF DATABASE CUT === */
!spool off
Goodbye!
============================================================================================
SUCCESS: Execution completed successfully
============================================================================================

Hello @kjn70, Thank you for reaching out. The snowsql output looks great. There is no formatted output for schemachange at this time. However, I would track your request for formatted outputs in #104 to incorporate a formatted logging option.
For now, I will close this issue and track your request in issue #104.
Please reopen this one if you were looking for something else.


Fix css.

Pull Request Test Coverage Report for Build 2468

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 70.93%

Totals

Change from base Build 2464:
0.0%

Covered Lines:
2193

Relevant Lines:
2890

💛  - Coveralls


[wallet] Wallet Blows Up on Start

When the wallet is loaded in the browser it fails with TypeError: __webpack_require__(...).readFileSync is not a function. It looks like this is due to to pureEvm requiring readFileSync
Uncaught TypeError: __webpack_require__(...).readFileSync is not a function
    at Object.<anonymous> (pure-evm_bg.js:3)
    at Object.../../node_modules/pure-evm/pure-evm_bg.js (pure-evm_bg.js:8)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Module.../../node_modules/pure-evm/pure-evm.js (pure-evm.js:1)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object.../nitro-protocol/lib/src/contract/force-move-app.js (force-move-app.js:8)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object.../nitro-protocol/lib/src/index.js (index.js:24)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object.<anonymous> (valid-transition.ts:2)
    at Object../src/redux/channel-store/channel-state/valid-transition.ts (valid-transition.ts:60)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/channel-store/channel-state/index.ts (index.ts:1)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/channel-store/index.ts (index.ts:2)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/state.ts (state.ts:8)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/protocols/advance-channel/reducer.ts (reducer.ts:2)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/protocols/advance-channel/index.ts (index.ts:5)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/protocols/new-ledger-channel/actions.ts (actions.ts:4)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/actions.ts (actions.ts:2)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/reducer.ts (reducer.ts:2)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/redux/store.ts (store.ts:7)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object../src/index.tsx (index.tsx:8)
    at __webpack_require__ (bootstrap:781)
    at fn (bootstrap:149)
    at Object.0 (transaction-generator.ts:146)
    at __webpack_require__ (bootstrap:781)
    at checkDeferredModules (bootstrap:45)

I'll look into it.

I think it might be due to pure-evm trying to merge both the node and browser packages into one. Also pure-evm no longer compiles as it references parity-ethereum directly from git which has been changed to use primitive_types instead of ethereum_types

There may also be an issue between our current version of webpack and rustwasm https://github.com/webpack/webpack/issues/8781

I think we can use this: https://webpack.js.org/api/module-methods/#magic-comments
Working on a new PR atm


GB/T 1.1 requirements on measurements

GB/T 1.1 measurements in mm, as specified in I.1 to I.12.
(https://github.com/riboseinc/asciidoctor-gb/issues/47#issuecomment-377791029)

a. CCS code.
b. the issuer of the specific national standard.

a. CCS code.
b. the issuer of the specific sector standard.

a. CCS code.
b. the issuer of the specific local standard.

Note: this is an odd page purely for example purposes.

Note: this is an odd page purely for example purposes.
Note: layout measurements for "Introduction" is identical to "Foreword".

Note: this is an odd page purely for example purposes.

Note: this is an odd page purely for example purposes.

Note: in this example the last element of the content is the glossary, and this is an odd page purely for example purposes.

Have carried stylesheet and coverpage across from GB/T 19488.2-2008, and have translated point measurements to cm.


Need to validate if production / sandbox tps fields are empty if throttling policy is set as "Specify"

Description:
If "Maximum Backend Throughput" is selected as "Specify" and if no values are given still the system saves it. Ideally it should show a validation message prompting to enter the sandbox / production tps rates. Follow the steps below to reproduce the issue
Steps

Create and API and navigate to "Manage" section
Select the throttling policy as "specify" and do not provide values for production and sandbox tps fields
Click on save or save and publish
The api will be saved or published successfully

Expected
The system should validate it and prompt the user to fill the fields.

Suggested Labels:
Type/Bug, 2.x,x update 5, Priority/High, Severity/Critical
OS, DB, other environment details and versions:
APIM and analytics quick setup
Ubuntu 14
JDK - jdk 1.8.0_51
DB - mysql
Browser - Chrome Version 62.0.3202.62

When a user selects the throttling policy as "specify" and do not provide values for production and sandbox tps fields and save the api, the  "maximum backend throughput" value of the saved api will be "unlimited". This is the expected behavior according to the design.


sys::prctl: Adding set_vma_anon_name.

to set a name for an anonymous region for Linux/Android.
What does this PR do
Checklist:

[x] I have read CONTRIBUTING.md
[x] I have written necessary tests and rustdoc comments
[x] A change log has been added if this PR modifies nix's API

Seems that there is no explicit way to check if this option is set or not?

No and in this case it does not really matter, this feature is more the "icing of top of cake" kind, so if it does not work it s just a no-op :)


Manual placement on GPU of a custom operator with both CPU and GPU implementation will always run the CPU version

Environment info
Operating System: ubuntu 14.04 64-bit
Installed version of CUDA and cuDNN: 7.5 and 4
$ ls -l /usr/local/cuda/lib64/libcud*
-rw-r--r-- 1 root root   322936 Aug 15  2015 /usr/local/cuda/lib64/libcudadevrt.a
lrwxrwxrwx 1 root root       16 Aug 15  2015 /usr/local/cuda/lib64/libcudart.so -> libcudart.so.7.5
lrwxrwxrwx 1 root root       19 Aug 15  2015 /usr/local/cuda/lib64/libcudart.so.7.5 -> libcudart.so.7.5.18
-rwxr-xr-x 1 root root   383336 Aug 15  2015 /usr/local/cuda/lib64/libcudart.so.7.5.18
-rw-r--r-- 1 root root   720192 Aug 15  2015 /usr/local/cuda/lib64/libcudart_static.a
-rwxr-xr-x 1 root root 61453024 Feb 23 16:00 /usr/local/cuda/lib64/libcudnn.so
-rwxr-xr-x 1 root root 61453024 Feb 23 16:00 /usr/local/cuda/lib64/libcudnn.so.4
-rwxr-xr-x 1 root root 61453024 Feb 23 16:00 /usr/local/cuda/lib64/libcudnn.so.4.0.7
-rw-r--r-- 1 root root 62025862 Feb 23 16:00 /usr/local/cuda/lib64/libcudnn_static.a
If installed from binary pip package, provide:

Which pip package you installed. tensorflow==0.8.0rc0
The output from python -c "import tensorflow; print(tensorflow.version)".
python -c "import tensorflow; print(tensorflow.version)"
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
0.8.0rc0

Steps to reproduce

Modify the How-to example cuda_op_kernel.cc to create both a CPU and GPU implementation for the AddOneOp operator
Make the GPU version do a different operation - I added 2 instead of 1 - so you can verify which version is running
Change the input and output type from int32 to float. This step is bizarre but critical!
Test the operator with manual placement - ie. with tf.device('/gpu:0'). This has to be done NOT in the self.test_session as in the example, but rather with a regular tf session - ie:  with tf.Session(config=tf.ConfigProto(log_device_placement=True)). This step is also critical. The tf.test.TestCase.test_session()  masks the issue.
The operator will run the CPU version despite the placer saying it is being placed on the GPU and the test fails.
$ python cuda_op_unittest.py
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX TITAN
major: 3 minor: 5 memoryClockRate (GHz) 0.928
pciBusID 0000:05:00.0
Total memory: 6.00GiB
Free memory: 5.29GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:755] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN, pci bus id: 0000:05:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX TITAN, pci bus id: 0000:05:00.0
I tensorflow/core/common_runtime/direct_session.cc:149] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX TITAN, pci bus id: 0000:05:00.0

AddOne/input: /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:388] AddOne/input: /job:localhost/replica:0/task:0/gpu:0
AddOne: /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:388] AddOne: /job:localhost/replica:0/task:0/gpu:0
*** running on CPU ***
F
FAIL: test (main.AddOneTest)
Traceback (most recent call last):
File "cuda_op_unittest.py", line 30, in test
assert allclose(result.eval(), [7.0, 6.0, 5.0, 4.0, 3.0])
AssertionError

Ran 1 test in 0.342s
FAILED (failures=1)
What have you tried?
We first noticed this on a much more complicated custom operator. This is a regression from the tensorflow version 0.7.1, which worked for us. The steps above are the result of several days spent trying to reproduce the problem with a minimal operator. The critical things seem to be using floats instead of ints and using the standard session instead of the tf test one.
Note also, that if I comment out the REGISTER_KERNEL_BUILDER line for the CPU and try to run only on the gpu device, the test passes, but the executor is still trying to create a CPU version. Log looks like:
$ python cuda_op_unittest.py
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX TITAN
major: 3 minor: 5 memoryClockRate (GHz) 0.928
pciBusID 0000:05:00.0
Total memory: 6.00GiB
Free memory: 5.29GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:755] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN, pci bus id: 0000:05:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX TITAN, pci bus id: 0000:05:00.0
I tensorflow/core/common_runtime/direct_session.cc:149] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX TITAN, pci bus id: 0000:05:00.0
AddOne/input: /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:388] AddOne/input: /job:localhost/replica:0/task:0/gpu:0
AddOne: /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:388] AddOne: /job:localhost/replica:0/task:0/gpu:0
E tensorflow/core/common_runtime/executor.cc:332] Executor failed to create kernel. Not found: No registered 'AddOne' OpKernel for CPU devices compatible with node AddOne = AddOne_device="/job:localhost/replica:0/task:0/gpu:0"
[[Node: AddOne = AddOne_device="/job:localhost/replica:0/task:0/gpu:0"]]
*** running on GPU ***
.
Ran 1 test in 0.552s
OK
Logs or other output that would be helpful
attaching source files. To build I do:
/usr/local/cuda/bin/nvcc -std=c++11 -c -o cuda_op_kernel.cu.o cuda_op_kernel.cu.cc -I $TF_INC -D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC
g++ -std=c++11 -shared -o cuda_op_kernel.so cuda_op_kernel.cc cuda_op_kernel.cu.o -I $TF_INC -fPIC -Wl,-rpath .
python cuda_op_unittest.py
I am running python 2.7.
cuda_op.tar.gz

Thanks - this does look suspicious and I see the same behavior on my machine.
We are investigating.

Any idea where the bug could come from? I ran into the same issue.

Thanks to @prb12 for debugging this with me.
The TensorFlow runtime optimizes the graph before it runs it the first time. One particular optimization of interest here is constant folding, that replaces a node whose input is a constant, with the output of the node after executing it. The 'execution' of the node during constant folding always happens on the CPU. Note that this is functionally correct, as both the CPU and GPU kernel implementations for an op are supposed to produce the same result. To make sure your op runs on the GPU in tests (where you presumably feed in constant values as inputs), use self.test_session to run your test. That disables optimization when running the graph. Note that when you use your custom op in a larger program that has real inputs instead of constants, it will run on the GPU as expected. Closing this issue because this is the expected behavior. Please feel free to add follow up comments in case you need more info.

We need to use the standard unittest.TestCase framework, not your TensorFlowTestCase framework, so self.test_session does not exist. Is there any way to disable this optimization in a regular tensorflow session - ie. with tf.Session()?

Yes, test_session is just a thin wrapper around creating a normal TF session, so you can see how the code disables this optimization here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/test_util.py#L246

I tried your suggestion and that works. Thanks.


multi-GPU vs. single GPU - scvi 1.1.x branch

I've successfully installed the scvi==1.1.x (main branch) and tested that I can train the model on 1 GPU.
However, when using multi-GPU, here's the error I'm facing at.
Batch_size = 512.
for 1 GPU:
x.shape = (512, 1178)
for 2 GPUs:
x.shape = (1, 512, 1178)
This causes almost everything cannot run in the code due to the dimension mismatch.
For example, one_hot function or FCLayers.
Do you have a quick fix for this or I should manually change everything of the dimension in the code (maybe x.squeeze(0) in the outer-most nn.Module) to match it?

Hi @zhenxingjian, what model are you using for this? I'll note that we have only tested multi-GPU training on scVI.

Hi @martinkim0 scVI is working with multiple samples (like n_samples_per_mc_run). Those look similar like the multi-GPU structure (n_samples, n_batch, n_genes). Quite some other functions like scANVI are not handling n_samples correctly (dimension errors). It's major work to adapt this. I was so confused by it today and made scANVI instead work with n_samples=1.

Hi @zhenxingjian, what model are you using for this? I'll note that we have only tested multi-GPU training on scVI.

I'm following the setup of multiVI.
If you've tested that scVI is working with multi-GPU training, I can try to modify the code following the same setup in scVI from my end to see if it can support multi-GPU training.


Fixed IllegalStateException while reading Parquet data

Problem Description -

The code sampled a varchar field and realized it is of fixed length type
It processed a couple of internal batches (4k max) just fine
Somehow at the last batch of the row-group, the column precision became lower
The logic to compute the next-record-batch was based on how much data it has loaded in the byte buffer and the expected precision: "loaded-data / expected-precision"
Hence the chunk-size became zero when the loaded data changed precision

Fix -

The logic to compute the next-record-batch should only worry that we do not try to read beyond the byte-buffer (to handle changes in precision for the last few values)
Modified the code to use: "buffer-capacity / expected-precision"
This way the code will have a chance to discover the change in precision (which it already does)

@ilooner can you please review this PR?
Thanks!

Thanks @ilooner for the review!


Add Hightlight Component

Subject
Add Hightlight Component
Description
Create A component following Hightlight component of yamada-docs.
Replace Preview component with Hightlight Component.
Remove Preview component.
Are you willing to participate in the development of this feature and create a pull request?
None

@illionillion
please confirm this issue content.


merge into athena-query-federation?

Hey @dacort,
Curious if you'd be open to us merging this into the athena-query-federation as a connector.

Hey @avirtuos!
Yes! Totally open to that - one question I have is around maintenance/updates. This project is definitely still in it's infancy and I'll likely be making occasional updates, but looks like things are getting merged into that repo pretty regularly. As long as that's the expectation and y'all don't mind a Python implementation of this, then 👍 .

I'm open to having a 'beta' flag for connectors that are still undergoing rapid changes. I think having this in the athena-query-federation repo will improve discoverability. I'm also very excited you got a python version working. Can you give me more info about how you handled the serialization in Python in a way that was compatible with our current serialization? I have plans to move us off ObjectMapper so we can have a more easily documented and controlled spec for the json.

I agree re: discoverability.
re: the serialization, it's not pretty. I ended up reverse engineering responses coming back from one of the example connectors along with poking at the connector source and experimenting with pyarrow locally to figure out each response type.
You can see an example of the end result here. The main weird thing that came up was that schema serialization on macOS is different than on Linux, hence the funky .slice(4) to chop off the first few bytes on Linux/Lambda. 😠
Let me know what next steps should be!

Hey, sorry to raise this from the dead, but we've got some renewed interest in this connector getting merged into our main repository. Are you still working on this?

Chatted offline about this - For now, given that main repo is all Java and this implementation is Python, going to hold off on trying to merge this in.
I'm going to close this issue for now, but always happy to consider contributing it back as an official connector. :)
I also plan to refactor this on top of the Python SDK I wrote.


Adding common functions folder.

Hi All,
Added some common functions to be used in Common-Functions-1.0 branch. Please validate.
Thanks
Amit

Agree with Erin's comments.
-1

Agree with Erin's comments
-1


Unable to use it on iOS

I get this error message in the console
file:///app/tns_modules/nativescript-socket.io/socketio.js:53:67: JS ERROR TypeError:
 SocketIOClient.alloc().initWithSocketURLConfig is not a function. (In
'SocketIOClient.alloc().initWithSocketURLConfig(NSURL.URLWithString(uri), _options)',
'SocketIOClient.alloc().initWithSocketURLConfig' is undefined)

@shiv19 Swift 4 is not supported yet since it is not a simple upgrade (Socket.IO Swift implementation have changed its API)

Thank you for the explanation :)

I also have the same issue. But since a year has passed, maybe Swift 4 is supported now ? :)

Swift 4 is supported in latest release

Thx for your reply !
But I still have the issue with nativescript-socket.io :

(CoreFoundation) *** Terminating app due to uncaught exception 'NativeScript encountered a fatal error: TypeError: SocketIOClient.alloc().initWithSocketURLConfig is not a function. (In 'SocketIOClient.alloc().initWithSocketURLConfig(NSURL.URLWithString(uri), _options)', 'SocketIOClient.alloc().initWithSocketURLConfig' is undefined)

Could you help me ? :)

I found the solution.
Sometimes, easy paths are the best. I've used the command tns clean platform ios and everything is fine now.