Nobody can load a dat archive I shared with Beaker other than myself

Operation System: macOS Sierra 10.12.6 (16G29)
Beaker Version: git commit a44d4272cc3fa817c2b3b31d6ac69a74c19fbb10
I created a dat archive via the "New site" wizard that includes a dat.json and index.html and I can view it just fine.
dat://dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349/dat.json
{
  "url": "dat://dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349/",
  "title": "Ildar Sagdejev",
  "description": "Take a look at my software development artifacts.",
  "author": "Ildar Sagdejev<EMAIL_ADDRESS>(https://keybase.io/tknomad)"
}

However, several different people have tried to load it across the internet, and none of them can reach it.
datbase.org can't find it either:

https://datbase.org/view?query=dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349

Meanwhile, I can quite successfully view other known dat archives published by other people.
I'm attaching the debug-20171230.log generated by

starting Beaker
navigating to beaker://library/
inspecting beaker://library/dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349
loading the dat: dat://dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349/
loading https://datbase.org/view?query=dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349 which reported that it could not find the dat
exiting Beaker

The swam debugger reports no peers connected to this archive:

The swarm debugger log:
closing connection (550424ms) id=55 type=tcp host=<IP_ADDRESS>:43465
got handshake (144ms) id=56 type=tcp host=<IP_ADDRESS>:43465
closing connection (13228ms) id=56 type=tcp host=<IP_ADDRESS>:43465
got handshake (808ms) id=57 type=tcp host=<IP_ADDRESS>:43465

Searching the debug log for my dat key returns one line:
$ ag dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349 debug-20171230.log
47:Sat, 30 Dec 2017 18:53:23 GMT dat Swarming archive, discovery key: 9b83f6487324413f737b89c11ddd2c01c7cc1a0c17e20de31e1a582b9cd6420b key=dbfd9a74d014399bdcd04ea6a9fce0c675661bd8800ae024f6a76a118a910349

Any advice on what I can do to troubleshoot this situation?

Dat has some known issues with peer discovery and connection right now. Maf is working on it, and I know Jim has been helping with some network emulators to help fuzz test. I'm going to close because it's a known issue.
Rehosting via hashbase tends to be a fairly consistent solution for now 😒

Hold on, why not reopen if this is a known issue?
Someone might wonder why their dat isn't reachable and find this thread and possibly contribute to figuring it out.

Let's say I'm interested in troubleshooting swarm connection and discovery situations.
In app/package.json I see that Beaker depends on mafintosh/discovery-swarm which uses maxogden/discovery-channel to "search for a key across multiple discovery networks and find peers who answer".
If I want to manually simulate the peer discovery that Beaker performs in the background, how can I do it?

You'd take the discovery module, import it into a test CLI app, then do one of the following:

Run it on various real world configurations (ie two laptops on a LAN, two laptops over WAN, etc) and look for failures.
Create a simulated network using containers and look for failures.

Option 2 is what @jimpick is working on now: https://github.com/jimpick/dat-network-simulator
You can also run tests on multiple Beaker instances. The challenge is getting proper debugging output -- the swarm debugger is obviously not helpful enough, so one option is to expand the amount of data it emits in its log. Alternatively, you can search through the internal code of the Dat modules and instrument it more. Most modules have already be instrumented using https://www.npmjs.com/package/debug, but you might need more than what's already there.
The task is to uncover where the failure is occurring, which means recognizing what the correct behavior is, and what's happening instead. It does appear, based on your log, that the failure is in discovery between peer devices. I assume that because the peers that are discovered are using TCP, and peers only ever connect to each other using UTP. The most reliable mechanisms for peer-discovery are in the https://www.npmjs.com/package/dns-discovery module, which uses a combination of multicast UDP (for LANs) and a centralized tracker that the Dat team runs using a modified form of DNS. That's what I'd debug, to see why connections are failing. Previously, I had started to look myself, but stopped when @mafintosh suggested he was going to be replacing the module entirely with a new DHT-based solution. That said, it's looking like the timeline for that is going to be too long, so debugging dns-discovery would still be worth it.

I've tested datPeers on 3 networks in Toronto on Ubuntu 18.10 and while LAN datPeers connections have worked, datPeers haven't been able to connecting across networks for me anywhere I've tried it. Would it be safe to assume my datPeers issue(s) are caused by the same connectivity issue(s) as publishing a dat archive?
Also is there an upstream issue for this in dat, discovery-swarm, dns-discovery or another lib?

Would it be safe to assume my datPeers issue(s) are caused by the same connectivity issue(s) as publishing a dat archive?

Yes

Also is there an upstream issue for this in dat, discovery-swarm, dns-discovery or another lib?

The current plan is to address this with a new discovery stack, hyperswarm. I'm also going to be adding analytics to help us gather information on why failures are occurring.

The current plan is to address this with a new discovery stack, hyperswarm.
Where to follow the status on this?

You can follow the repos at https://github.com/hyperswarm

This is the high level API that's going to be used: https://github.com/hyperswarm/network

Please take example on i2p browser and implement a network status health report to help the users to know if they are firewalled or not.
https://geti2p.net/en/browser
Thanks.

@pfrazee does this mean the connectivity issues have been resolved? What version of Beaker should the fixes land in?
Thanks!

@D1plo1d Yep! Beaker 1.0 has the new hyperswarm stack and we've been getting really good results with connectivity. I haven't had a failed load yet across a number of user tests! So fingers crossed that we have a winner.

That's awesome!

Why is beaker not able to load the fritter site


Why we get such unuseful http response?

I try to understand why we get such http response after sending an email?
[
  {
    "statusCode": 202,
    "headers": {
      "server": "nginx",
      "date": "Thu, 12 Sep 2019 10:55:31 GMT",
      "content-length": "0",
      "connection": "close",
      "x-message-id": "PART OF MESSAGEID",
      "access-control-allow-origin": "https://sendgrid.api-docs.io",
      "access-control-allow-methods": "POST",
      "access-control-allow-headers": "Authorization, Content-Type, On-behalf-of, x-sg-elas-acl",
      "access-control-max-age": "600",
      "x-no-cors-reason": "https://sendgrid.com/docs/Classroom/Basics/API/cors.html"
    },
    "request": {
      "uri": {
        "protocol": "https:",
        "slashes": true,
        "auth": null,
        "host": "api.sendgrid.com",
        "port": 443,
        "hostname": "api.sendgrid.com",
        "hash": null,
        "search": null,
        "query": null,
        "pathname": "/v3/mail/send",
        "path": "/v3/mail/send",
        "href": "https://api.sendgrid.com/v3/mail/send"
      },
      "method": "POST",
      "headers": {
        "Accept": "application/json",
        "User-agent": "sendgrid/6.3.0;nodejs",
        "Authorization": "Bearer ",
        "content-type": "application/json",
        "content-length": 1103
      }
    }
  },
  null
]

What I should understand from this? Only http status code useful here.
Why I can't get normal Message-ID? Why I get the string before @ sign, how do I get the full Message-ID? I don't want to send custom argument, I want Message-ID.
Why I can't get status of message in response? Even if it will be "In queue" it's still better than current response.
As a paying customer I want to get a normal useful response that will help me to integrate sendgrid with our service and not what you send me.
Thx

Just wanted to leave a comment here regarding how to match webhook events with emails. You can add custom args to the emails that carry over to the events.
{
        to<EMAIL_ADDRESS>        from: 'Some Company<EMAIL_ADDRESS>        subject: 'Did somebody say BACON DONUTS?? 🥓🥓🥓',
        templateId: 'xsdsew22323dxxx',
        custom_args: {
          "param1": 'some_id',
          "param2": 'internal_person_id'
        },
        dynamic_template_data: {
          first_name: 'dynamic data'
        }
      }

Hello @Rashe,
Thanks for submitting a GitHub issue! We are very sorry that you are running into this problem. In order to better serve you, as this does not present itself as a library specific issue, we would like to ask that you reach out to our support team at https://support.sendgrid.com.
Thank you!
SendGrid DX Team


GitHub Actions: Add cross compile tests

Add cross compile tests without running checks. Currently
only mips64el and riscv64 is wired up.
Just help us make sure those less popular CPUs are not
messed up by changes.
And it seems like riscv64 is already broken in tree :-(

You need to supply a valid TARGET (as enumerated in TargetList.txt) when cross-compiling, I do not see that in your PR?

Ah I see, in #3720 I added DYNAMIC_ARCH support for MIPS64 and it will be able to probe arch and target from toolchain. For riscv64 it is incapable to do so. Will fix.
Thanks.

You need either DYNAMIC_ARCH or a TARGET, there is no build-time probing of cpu in cross-compiling as the target cpu will in most cases be on a completely different computer. RISCV already has RISCV64_GENERIC but no DYNAMIC_ARCH support (would be a bit overkill when it is just a choice between two targets, the optimized C910V and the generic one).

Debian has cross-toolchains for all of its own arches, should be like 1:1 to just convert distro to debian 11 now, then extend if some niche arch gets problems some day.

Debian has cross-toolchains for all of its own arches, should be like 1:1 to just convert distro to debian 11 now, then extend if some niche arch gets problems some day.

GitHub Action itself doesn't provide Debian VM. Using Docker seems a little bit overkill. Ubuntu 22.04 provides almost all cross toolchains shipped with Debian11 :-)

Now mipsel and alpha are wired up as well.
Alpha build issues are addressed by #3725.

unrelated manylinux build failure was fixed by my #3726 (guess this only came up as the manylinux instance lost its fortran compiler somehow). sorry for the delay in merging


Is there official TypeScript type definitions file for this project?

This drag Library is really cool. Is there any official TypeScript type definitions file for this project or any is working in progress ?

@Omniroot did you mention you had to do with when working on the React adapter?

+1

Another option (other than just a definition file) is to change your js file extensions to ts and start adding types to the code (at your own pace). The typescript compiler will help increase code quality and identify bugs with you. Just sayin' ;)  https://www.typescriptlang.org/

+1

@rlgreen91 it should be imported automatically as long as you use typescript in your awesome project or use the IDE that supports typescript language server, eg: vscode.

Yeah... there are no .d.ts files anywhere when you run npm install, so TypeScript has no idea what to do if you pull this into a TypeScript project.
You have to ship .ts or .d.ts files for TypeScript typings to work.

@qJake The .d.ts files is added after beta.8 version https://www.jsdelivr.com/package/npm/@shopify/draggable?version=1.0.0-beta.9 , but strange that npm i @shopify/draggable will install beta.8 by default.
Now the latest version is beta.11, you can install it by run npm i<EMAIL_ADDRESS>

Thanks for the clarification @zjffun - I was able to install that version of the package and TypeScript worked!
If beta.11 is stable enough you should consider promoting it to the "latest" in NPM so it downloads by default! 🙂

I’m closing this issue because the TypeScript type definitions file was supported since v1.0.0-beta9.
Thank you!

Unfortunately this is still not fixed on the latest and you have to manually download beta 9 or above. This issue should not have been closed.

@pskfyi HI, we are discussing npm Tags in #465 .

after
❯ yarn add<EMAIL_ADDRESS>I still have the following:

Add
declare module<EMAIL_ADDRESS>{
  export * from "@shopify/draggable";
}

to typings.d.ts can temporarily fix this.

after
❯ yarn add<EMAIL_ADDRESS>I still have the following:

I don't really know what I did but after messing around with my tsconfig
and restoring it, vscode finally picked up the typed. Seems to have been a
bug somewhere in the monstrosity of moving parts that is a TypeScript
project... Thanks.
On Tue, Jul 20, 2021, 13:56 JuFeng Zhang @.***> wrote:

Add
declare module @.***/draggable/lib/draggable.bundle.js" {
export * from @.***/draggable";
}
to typings.d.ts can temporarily fix this.
after
❯ yarn add @@.***
I still have the following:
[image: image]
https://user-images.githubusercontent.com/38855893/126146236-2630d85b-24f9-4a3d-9ce6-e5a707fc5a5e.png
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Shopify/draggable/issues/105#issuecomment-883332715,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJIOJVPMBLBFMDJI6ZABGULTYVP7BANCNFSM4EJ3F7RQ
.

I don't really know what I did but after messing around with my tsconfig
and restoring it, vscode finally picked up the typed. Seems to have been a
bug somewhere in the monstrosity of moving parts that is a TypeScript
project... Thanks.
On Tue, Jul 20, 2021, 13:56 JuFeng Zhang @.***> wrote:

Add
declare module @.***/draggable/lib/draggable.bundle.js" {
export * from @.***/draggable";
}
to typings.d.ts can temporarily fix this.
after
❯ yarn add @@.***
I still have the following:
[image: image]
https://user-images.githubusercontent.com/38855893/126146236-2630d85b-24f9-4a3d-9ce6-e5a707fc5a5e.png
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Shopify/draggable/issues/105#issuecomment-883332715,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJIOJVPMBLBFMDJI6ZABGULTYVP7BANCNFSM4EJ3F7RQ
.


Remove peer data, the peer fails to be downloaded again

Feature request:

Use case:

UI Example:

Please use latest version.


How can I best contribute?

Hey there! First off, thanks for making what's shaping up to be a great crate!
I'm new to Rust, and getting my toes wet by building a jwt cli tool. I'd love to contribute back to this project, but I'm not sure where I would be most helpful. Docs? Some easy feature? I'm not sure what you have in progress already so let me know if I can help.

Thanks for the kind words!
Regarding stuff in progress: https://github.com/Keats/rust-jwt/pull/23 is an ongoing re-write, currently waiting for a Ring PR + new release to finish RSA support.
I need some help coming up with ideas on how to handle validation (https://github.com/Keats/rust-jwt/issues/22) and there some interesting API ideas on https://github.com/Keats/rust-jwt/issues/13
I think the best help would be to try the v2 branch once it's a bit more stable to make sure the API etc makes sense, is easy to use and cover most usecases

Sweet, thanks! I'll take a look at those issues and subscribe to the v2 pr.

v2 is on crates.io, thanks a lot for your help!

Happy to help! Thanks for making it in the first place!


no app start on IPhone 6 IOS 12.5.1

Describe the bug

I just started the IPhone to be sure no service is running anymore. Installing the Version 1.0 of the walletApp. After that i tried to start the App but I get just a black screen.
On IPhone Xs everything works fine.
Expected behaviour

The App should start
Steps to reproduce the issue

restart the IPhone
install version 1.0 from AppCenter
trust the Application in IPhone settings
start WalletApp
just a black screen is shown

Technical details
IPhone 6
IOS-Version 12.5.1
Possible Fix

Additional context

we need someone with an old iPhone and developer tools to debug this. I don't have such an old device for testing :/

we need someone with an old iPhone and developer tools to debug this. I don't have such an old device for testing :/

maybe it is possible to get one from the development team?

I found the issue, it's because of iOS 12. Will work on a fix as 1.1.0


In NativeScript in app folder there is addtional package.json that misleads eslint

In NativeScript apps there is app/package.json  file with contents
{
  "main": "app.js",
  "android": {
    "v8Flags": "--expose_gc"
  }
}

In project root folder there is also package.json that contains all dependencies. There currently seems to be no way to look for dependencies in workspace root package.json instead on app folders one.

@lafriks can you provide me with a GitHub repository and steps that show what exactly the issue is you are having?

I just created sample: https://github.com/lafriks/test-nativescript-vue
You could probably need nativescript first.
npm install -g nativescript
npm install

When running:
npm run lint

Result is:

But vscode-eslint shows linting errors for dependencies:

Actually npm run lint doesn't call eslint directly and I have no idea what vue-cli-service lint does. However running eslint directly in a terminal produces the same error:

So unless someone knows how to setup ESLint (https://eslint.org/) I am a little lost here.

As a workaround I currently have set .eslintrc.js for anyone getting this error this way:
module.exports = {
  root: true,

  env: {
    node: true,
  },

  extends: ['plugin:vue/recommended', '@vue/airbnb'],

  rules: {
    'no-console': 'off',
    'no-debugger': 'off',
    'import/no-extraneous-dependencies': ['error', {'packageDir': './'}],
    'import/extensions': 'off',
    'import/no-unresolved': 'off',
  },

  parserOptions: {
    parser: 'babel-eslint',
  },
};

@lafriks any objection to close this issue. I don't see what ESLint extension can do to make this a better expierence.

Yes sure, I believe this could be actually eslint issue


Make the nil and empty of the BinPayload equivalent

This PR is fixing the problem of the DTM server misdetermining a branch action with a "GET" request as a "POST" request due to the BinPayload being an empty byte array but not nil.

Codecov Report

Merging #348 (841f10e) into main (035b86a) will increase coverage by 0.00%.
The diff coverage is 100.00%.

:exclamation: Current head 841f10e differs from pull request most recent head c16fb38. Consider uploading reports for the commit c16fb38 to get more accurate results

@@           Coverage Diff           @@
##             main     #348   +/-   ##
=======================================
  Coverage   96.38%   96.38%
=======================================
  Files          55       55
  Lines        3206     3208    +2
=======================================
+ Hits         3090     3092    +2
  Misses         76       76
  Partials       40       40

Impacted Files
Coverage Δ

dtmsvr/trans_status.go
97.45% <100.00%> (ø)

dtmutil/utils.go
95.91% <100.00%> (+0.08%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[ CGAL] Update version to 5.2.2

Describe the pull request

What does your PR fix?

Yes

@maxGimeno ,Thanks for your pr.
have you tested the feature locally on x64-linux, x64-windows, and x64-windows-static？

tested on x64-windows

tested on x64-windows

Could you test x64-windows-static and x64-linux?

It doesn't work on x64-linux, at least on my machine. Mpfr doesn't build. And I don't know what x64-windows-static is and how to test it

Thanks again!


Change Language with Application-Level Context

This library will not work if I use the application context to get the string.
I'm planning to call LocaleChanger.configureBaseContext() directly from my Application since I think it can be used globally through my application without passing any context from activity.
From the documentation of the function,
/**
 * This method should be used inside the Activity attachBaseContext.
 * The returned Context should be used as argument for the super method call.
 *
 * @param context
 * @return the resulting context that should be provided to the super method call.
 */
public static Context configureBaseContext(Context context) {
    return delegate.configureBaseContext(context);
}

It stated that This method should be used inside the Activity attachBaseContext and not mentiong about application context.
My question is it safe that I created helper function from my application
// inside Application class
public Context getContextForLanguage() {
    return LocaleChanger.configureBaseContext(this);
}

and use that helper funtion to get the string from somewhere else? (Like below)
(application as MyApplication).getContextForLanguage().getString(R.string.something)

What are the drawbacks of this approach?

As far as I know should be able to use your getContextForLanguage method without any problems


off chain duplicate filter

Hi @micwallace, @snowwhitedev
Please see this solution. I feel this is a lighter way to implement this change applying the filter at the point the tokens are to be sent to the client site. Although let's review and implement the right solution.
Note: I've added unit tests to the duplication logic added, however because this is the first time it's been linted with the Prettier rules Sonar explains this is failing.
Thanks guys.

Closing as we have gone with Hendersons initial solution of preventing duplicates being added in the first place.


Variable seems not working.

I tried
.text {
  color: red;
}

which works fine.
But when I tried:
$red = red;
.text {
  color: $red;
}

It does not work.
Any ideas?

$red: red;

:wink:

Ahh I see!
Sorry for my mistake!

No problem :smile:


chore(tests): remove hardcoded dns resolver in tests

FTI-3133

@hanshuebner I ran a grep over spec/ and found those are relevant and can be removed.
<IP_ADDRESS> also exists in multiple tests but I checked them one by one and looks they are placeholders
and no DNS queries are actually sent.
spec/01-unit/21-dns-client/01-utils_spec.lua:nameserver <IP_ADDRESS>
spec/01-unit/21-dns-client/01-utils_spec.lua:nameserver <IP_ADDRESS>:1234 ; this one has a port number (limited systems support this)
spec/01-unit/21-dns-client/01-utils_spec.lua:      assert.is.same({ "<IP_ADDRESS>", "2602:306:bca8:1ac0::1", "<IP_ADDRESS>:1234" }, resolv.nameserver)
spec/01-unit/21-dns-client/01-utils_spec.lua:nameserver <IP_ADDRESS>
spec/01-unit/21-dns-client/01-utils_spec.lua:nameserver <IP_ADDRESS>
spec/01-unit/21-dns-client/01-utils_spec.lua:              "nameserver <IP_ADDRESS> ",
spec/01-unit/21-dns-client/03-client_cache_spec.lua:        nameservers = { "<IP_ADDRESS>" },
spec/01-unit/21-dns-client/03-client_cache_spec.lua:        nameservers = { "<IP_ADDRESS>" },
spec/01-unit/21-dns-client/03-client_cache_spec.lua:        nameservers = { "<IP_ADDRESS>" },
spec/01-unit/21-dns-client/03-client_cache_spec.lua:        nameservers = { "<IP_ADDRESS>" },
spec/01-unit/21-dns-client/02-client_spec.lua:          nameservers = { "<IP_ADDRESS>:53" },
spec/01-unit/21-dns-client/02-client_spec.lua:            nameservers = { "<IP_ADDRESS>:53" },
spec/01-unit/21-dns-client/02-client_spec.lua:            nameservers = { "<IP_ADDRESS>:53" },
spec/01-unit/21-dns-client/02-client_spec.lua:            nameservers = { "<IP_ADDRESS>:53" },
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:              "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:            "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:          "nameserver <IP_ADDRESS>",
spec/01-unit/21-dns-client/02-client_spec.lua:        "nameserver <IP_ADDRESS>",
spec/01-unit/03-conf_loader_spec.lua:        dns_resolver = "<IP_ADDRESS>:53"
spec/01-unit/03-conf_loader_spec.lua:        dns_resolver = "<IP_ADDRESS>,<IP_ADDRESS>:53,::1,[::1]:53"
spec/01-unit/09-balancer/04-round_robin_spec.lua:        "nameserver <IP_ADDRESS>"
spec/01-unit/09-balancer/04-round_robin_spec.lua:        { name = "getkong.test", address = "<IP_ADDRESS>" },
spec/01-unit/09-balancer/04-round_robin_spec.lua:          "nameserver <IP_ADDRESS>"
spec/01-unit/09-balancer/03-consistent_hashing_spec.lua:        "nameserver <IP_ADDRESS>"
spec/01-unit/09-balancer/01-generic_spec.lua:          "nameserver <IP_ADDRESS>"
spec/01-unit/09-balancer/02-least_connections_spec.lua:        "nameserver <IP_ADDRESS>"
spec/01-unit/04-prefix_handler_spec.lua:        dns_resolver = { "<IP_ADDRESS>", "<IP_ADDRESS>" }
spec/01-unit/14-dns_spec.lua:    nameservers = { "<IP_ADDRESS>" },

Let me know if this is covering your original thought : )

<IP_ADDRESS> also exists in multiple tests but I checked them one by one and looks they are placeholders and no DNS queries are actually sent.

I would suggest using an address different from <IP_ADDRESS> so that the address can never accidentally used for lookups.  <IP_ADDRESS> would be a good address, at it is in a reserved test address block that will never be allocated (RFC5737)

looks like there does exist some test in spec/01-unit/21-dns-client/02-client_spec.lua are doing real DNS queries,
i will fix them to honor system resolv.conf next week.

@hanshuebner can use another review now, thanks!


Unit | Component | sl grid column header: There are no references to Ember.$, $ or jQuery

not ok 448 PhantomJS 1.9 - Unit | Component | sl grid column header: There are no references to Ember.$, $ or jQuery
    ---
        actual: >
            null
        message: >
            Died on test #1     at test (http://localhost:7357/assets/test-support.js:3025)
                at testWrapper (http://localhost:7357/assets/test-support.js:6192)
                at test (http://localhost:7357/assets/test-support.js:6205)
                at http://localhost:7357/assets/tests.js:23755
                at http://localhost:7357/assets/vendor.js:152
                at tryFinally (http://localhost:7357/assets/vendor.js:33)
                at http://localhost:7357/assets/vendor.js:158
                at http://localhost:7357/assets/test-loader.js:60
                at http://localhost:7357/assets/test-loader.js:51
                at http://localhost:7357/assets/test-loader.js:82
                at http://localhost:7357/assets/test-support.js:6024: Attempted to wrap $ which is already wrapped
        Log: |
    ...

Will likely be resolved by https://github.com/softlayer/sl-ember-components/issues/1503

Resolved by https://github.com/softlayer/sl-ember-components/issues/1503


Denis/use GitHub actions

Add CI configuration files. Let's see how this works out...

[x] Adjust lint formating

@livanov93 can you enable Github actions? Usually, they start automatically when there are configuration files.

@livanov93 can you enable Github actions? Usually, they start automatically when there are configuration files.

But the configuration files aren't there yet. They are on your fork of the repo.

@livanov93 can you enable Github actions? Usually, they start automatically when there are configuration files.

But the configuration files aren't there yet. They are on your fork of the repo.

Oh, you are right, sorry... Can we then merge this on another branch in this repo, so I can test there and then push to develop when CI is working? Can you please push it in this repo!

@livanov93 can you enable Github actions? Usually, they start automatically when there are configuration files.

But the configuration files aren't there yet. They are on your fork of the repo.

Oh, you are right, sorry... Can we then merge this on another branch in this repo, so I can test there and then push to develop when CI is working? Can you please push it in this repo!

I'll push this PR into another branch so you can continue with your work there.


chore: update RNTester Cocoapods to 1.13

Summary:
This PR updates the internal version of cocoapods to 1.13, template already uses this version. I've also removed the root folder Gemfile as it's not necessary anymore.
Changelog:
[INTERNAL] [CHANGED] - Update RNTester Cocoapods to 1.13
Test Plan:
Check if cocoapods installs correctly by running:

bundle install
bundle exec pod install

When I removed the root Gemfile CI failed so it looks like its still needed

Yeah, it's not a simple removal. It should go, but there are other cleanup that are required to remove it properly!

@cipolleschi Can we merge it without removing the root one?

yes sure. I thought I had already imported this one. Sorry.


SublimePrettyJson-master\PrettyJson.py", line 89  SyntaxError: invalid syntax

windows10 | Sublime Text3 Build 3126
SublimePrettyJson-master\PrettyJson.py", line 89
replacement = f'[{join_separator.join(items)}]'
^
SyntaxError: invalid syntax

Do you have it cloned or is it install via PC?

got the same problem.  sublime 3.2.2 build 3211.  I installed PrettyJson via PC, Package Control. My windows version: win10 home 18363.1171.

also have the same error on sublime build 3211.
reloading plugin Pretty JSON.PrettyJson Traceback (most recent call last):   File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 125, in reload_plugin     m = importlib.import_module(modulename)   File "./python3.3/importlib/__init__.py", line 90, in import_module   File "<frozen importlib._bootstrap>", line 1584, in _gcd_import   File "<frozen importlib._bootstrap>", line 1565, in _find_and_load   File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked   File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 1199, in load_module     exec(compile(source, source_path, 'exec'), mod.__dict__)   File "/Users/coralqian/Library/Application Support/Sublime Text 3/Installed Packages/Pretty JSON.sublime-package/PrettyJson.py", line 89     replacement = f'[{join_separator.join(items)}]'                                                   ^ SyntaxError: invalid syntax reloading plugin Pretty JSON.PrettyJsonListeners Traceback (most recent call last):   File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 125, in reload_plugin     m = importlib.import_module(modulename)   File "./python3.3/importlib/__init__.py", line 90, in import_module   File "<frozen importlib._bootstrap>", line 1584, in _gcd_import   File "<frozen importlib._bootstrap>", line 1565, in _find_and_load   File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked   File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 1199, in load_module     exec(compile(source, source_path, 'exec'), mod.__dict__)   File "/Users/coralqian/Library/Application Support/Sublime Text 3/Installed Packages/Pretty JSON.sublime-package/PrettyJsonListeners.py", line 4, in <module>   File "<frozen importlib._bootstrap>", line 1565, in _find_and_load   File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked   File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 1199, in load_module     exec(compile(source, source_path, 'exec'), mod.__dict__)   File "/Users/coralqian/Library/Application Support/Sublime Text 3/Installed Packages/Pretty JSON.sublime-package/PrettyJson.py", line 89     replacement = f'[{join_separator.join(items)}]'                                                   ^ SyntaxError: invalid syntax

Reinstall PrettyJson. Sublime installed the version for ST4 for some reason.
--
Zachary Schulze

On Sunday, Nov 01, 2020 at 7:50 PM, Yang<EMAIL_ADDRESS>(mailto:notifications@github.com)> wrote:
got the same problem. sublime 3.2.2 build 3211. I installed PrettyJson via PC, Package Control. My windows version: win10 home 18363.1171.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub (https://github.com/dzhibas/SublimePrettyJson/issues/123#issuecomment-720221407), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AHYW2RFAJWPY3OVESN3QUL3SNYT6RANCNFSM4PINX4BA).

Reinstall PrettyJson. Sublime installed the version for ST4 for some reason.
…
-- Zachary Schulze
On Sunday, Nov 01, 2020 at 7:50 PM, Yang @.*** @.***)> wrote: got the same problem. sublime 3.2.2 build 3211. I installed PrettyJson via PC, Package Control. My windows version: win10 home 18363.1171. — You are receiving this because you commented. Reply to this email directly, view it on GitHub (#123 (comment)), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AHYW2RFAJWPY3OVESN3QUL3SNYT6RANCNFSM4PINX4BA).

Problem solved.
thks for letting me know the version I downloaded is ST4.
Used pci to install, turned out to be wrong.
downloaded manually with git version st3 , now , feel good.
My world peace again.


Unit Test examples for testing the interactors in Mosby MVI

Hi,
Not 100% sure if this should be here or on SO, but it looks like a "feature request" so...
Reading the series of blog posts (starting here: http://hannesdorfmann.com/android/mosby3-mvi-1) I stumbled upon the following lines:

Testability
...
With a Model representing State we can simplify our unit test’s code as we can simply check assertEquals(expectedModel, model). This eliminates a lot of objects we otherwise have to mock. Additionally, this removes many verification tests that a certain method has been called i.e. Mockito.verify(view, times(1)).showFoo(). Eventually, this makes our unit test’s code more readable, understandable and finally maintainable as we don’t have to deal that much with implementation details of our real code.

From what I've implemented so far, all of this seems to have a lot of merit. I use mosby (mvi) + mosby-conductor (mvi) + conductor and couldn't be happier. I write Interactors or UseCases (or whatever they should be called) using RxJava and want to unit test them. After all, the real logic of the app is there and I want to test it without much of a hassle.
I would imagine unit testing an Interactor should look something like this:
val expectedViewState = AuthViewState(loggedIn=true)

val mockedAuthProvider:authProvider = mock()
val interactor = AuthInteractor(mockedAuthProvider)
val viewState = interactor.login() (+ some magic to convert RxJava Observable to ViewState)
assertEquals(expectedViewState, viewState)

I would imagine this should work whether or not State Reducer is used. I believe I checked every relevant sample but couldn't find one that delivers.
Can we see some kind of an example related to all that? And, please, no presenter testing code. Of course, all of this might not be relevant to mosby at all. Still, this might be helpful for the community.
Thanks for the great library!

Hey here, what exactly do you want to test?
Interactors?
We had a discussion lately (see #168) where I have explained why I think integration testing makes more sense especially with MVI. You may find this approach on testing interesting too. Would like to hear your feedback.

Hi again,
Yes, interactors (business logic) only. Don't want to test anything related to Mosby.
It seems like #168 doesn't have anything related to testing? Maybe a typo?
Regards

Hi,
sorry, the correct issue is #268  (not #168)

Hi,
TL; DR
I want to test my "business logic" and don't want my tests coupled with the delivery mechanism.
The long version:
Read the discussion at #268 and your posts about MVI (about 2-3 times). Also, watched Uncle Bob's Clean Coders videos on TDD and Clean Architecture lectures.
I want my tests to be:

As readable/lean as possible
As easy to write/add as possible
Depend on as few test doubles as possible (that is, seldom use of any insert your favorite mock library )
Heavily test my business logic and test everything else based on developer preference (E.g. I don't want to test a Presenter that simply navigates to another page on click)

I believe that the argument should be "Why skip the Interactor and test the Presenter?". Since the latter will require (much) more work to test properly. For the sake of argument, let's review why Interactor testing is a must (In my opinion, of course):

If that's an example of PresenterTest: https://github.com/sockeqwe/mosby/blob/master/sample-mvi/src/test/java/com/hannesdorfmann/mosby3/sample/mvi/view/home/HomePresenterTest.java I would say - it just hurt my eyes.
Presenters are part of the delivery mechanism, that is, they are Android specific (in our case). If I change the framework (Mosby) or simply remove the need for Presenters in my app I would have to write whole new tests or isolation layer for that.
If I want to deliver my app via the web and have some additional use cases for that, how should I test them? Create new isolation layer for "Web" Use Cases?
Presenter tests are required, but not for every Presenter (as discussed previously). So tests will be written for them (and state reducers) as the need for that arises.
One Use Case can be used by multiple Presenters - should I duplicate my test logic just to cover all Presenters?

Thanks for taking the time to write down your feedback.
Regarding your original question:
If you just want to test Interactors, then go ahead and write tests like:
@Test void interactorIsReturningFoo(){
   Observable<Foo> observale =  interactor.doSomething();
   Foo foo = observable.blockingFirst();

   assertEquals(expectedFoo, foo);
}

and so on ... But I don't understand how this is related to MVI or Mosby?

If that's an example of PresenterTest: https://github.com/sockeqwe/mosby/blob/master/sample-mvi/src/test/java/com/hannesdorfmann/mosby3/sample/mvi/view/home/HomePresenterTest.java I would say - it just hurt my eyes.

This is not a Presenter Test, this is a Integration test as it tests all layers down from Presenter inclusive Business Logic state reducer etc. Perhaps we have a different understanding / definition of what a presenter is. The only "Test Double" in this test is the View Layer, but this could easily be run on a real android view too.

Presenters are part of the delivery mechanism, that is, they are Android specific (in our case). They stay outside of the boundary of my app (Black vertical line in the diagram).

I see your point, but I have a different opinion about that which I think comes from the fact that you are advocating for writing unit tests on business logic / Interactors whereas I recommend to write more integration tests instead of unit tests. So the question is: where does a integration test begins? If you have two parts in your business logic an put them together and test them together then it is already an integration test. From my point of view integration tests should cover as much as possible, ideally from (android) View layer to the deepest part of your business logic and all the way back. Unfortunately, real android view tests (instrumentation tests) are slow, hence often we mock the View layer. Nevertheless, I recommend to write integration tests with android View layer running on a real android device.
Hence, in most cases I would draw the boundary of my MVI based app (Black vertical line in the diagram) between view (interface) and presenter unless Presenter is really android specific which I only rarely can remember that this was the case in one of my apps. Either android specific components were in the view layer or the very end of the business logic like a sqlite database. Both implementation details should be hidden behind a Interface (boundary).

If I change the framework (Mosby) or simply remove the need for Presenters in my app I would have to write whole new tests or isolation layer for that

That is the big advantage of integration testing.  The View Interface defines the boundary and usually that is all you need to know. Think of it in a mathematical way. If you have a mathematic function definition like this:
f(x,y) = x + y

then the only thing you need to know to test this function is the public api this function offers: inputs x, y and the return value. But you don't care about the implementation detail (x+y).
If you apply that idea on a app:
f (intentX, intentY) = ... compute state ...  // implementation details

So if f(...) is a function describing your whole app (or for simplicity lets say 1 screen) this is all your blackbox integration tests must provide (as public API) to test your app. There is no presenter in this equation. If you remove the presenter, the public API doesn't change. Hence you don't need to write new tests (or adjust any existing test) if you remove the presenter.

If I want to deliver my app via the web and have some additional use cases for that, how should I test them? Create new isolation layer for "Web" Use Cases?

i don't understand the question but if your question is if I take my app and replace the android View with a website or whatever else then your website obviously have to implement the View Interface and that's it. No need to change other internals (implementation details) within the Boundary (Black vertical line in the diagram) .

Presenter tests are required, but not for every Presenter (as discussed previously). So tests covering them should be written (and for state reducers) as the need for that arises.

Again, we might have a different definition of what a presenter test is. I, personally, prefer integration test over  unit tests. If you write unit tests for Interactor, State Reducer and Presenter in isolation, you can never be sure that they integrate and work together as expected.
this is a nice illustration of the issue with unit tests: https://twitter.com/ThePracticalDev/status/845638950517706752
So you may need some kind of integration test for those components anyway. But then, what is the point of writing integration and unit tests? You gain nothing if you test things twice.

One Use Case can be used by multiple Presenters - should I duplicate my test logic just to cover all Presenters?

It sounds rough, but yes you should, otherwise it is not an integration test since this one Use Case is interacting with other parts too, right? Also this ensures that if you refactor that use case, you are sure that it works for all presenters / screens that use this use case, which is not 100% guaranteed if you just write a single unit test for the use case.

The apparent overuse of a "Test Double" (HomeViewRobot) in the Presenter Test (same as the referenced above) shows that testing is not as simple as MVI promised at first:

There is exactly one "Test Double" for the View layer. Everything else are the real app components so the promise:
assertEquals(expectedViewState, viewState)

still holds (if you write integration tests / black box tests aiming to cover all components).

WIP!
Thanks for taking the time. This is a real treat 👍

But I don't understand how this is related to MVI or Mosby?

As I stated previously

Of course, all of this might not be relevant to mosby at all. Still, this might be helpful for the community.

Yes, it is not related to Mosby. I am perfectly fine with not addressing this issue here.

This is not a Presenter Test, this is a Integration test as it tests all layers down from Presenter inclusive Business Logic state reducer etc.

I completely agree. It is an integration test. Integration tests have a (very large) place in developing "proper" apps and should cover all the things you've mentioned.

i don't understand the question but if your question is if I take my app and replace the android View with a website or whatever else then your website obviously have to implement the View Interface and that's it. No need to change other internals (implementation details) within the Boundary (Black vertical line in the diagram) .

Yes, that is correct. I am talking about going to Web (for example) from Android. Let's have another look at the diagram


Mailgun A or MX records are incorrect

Failed<EMAIL_ADDRESS>→<EMAIL_ADDRESS>'Welcome to Neuroscout!' Server response: 553 5.1.8 Domain of sender address<EMAIL_ADDRESS>does not exist or has no A or MX records

What's the intended behavior?

This is an error from Mailgun itself. Which means no mail is currently being sent, because the neuroscout domain is set up incorrectly.
Oddly, it was working a little while ago, as Ross signed up and verified his email. My guess is what once we changed to alpha.neuroscout.org something went wrong in the domain set up, and the A/MX records don't match up. Although, I'm not sure why that would be, but I can't think of anything else that has changed recently.

Can you see if the TXT records mailgun creates have changed? You can paste them on slack again

Oh, it's possible the TXT records need to be generated for a specific (sub)domain, and we'd need to explicitly tell mailgun we need records for alpha.neuroscout.org. Maybe look into it on the mailgun side.

Looks good now


nzbget: fix broken service, as well as some improvements

Motivation for this change
The nzbget service is currently broken and appears to have been for some time.
https://github.com/NixOS/nixpkgs/pull/51235#pullrequestreview-228910052
ping @flokli
closes #58928.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@aanderse Thanks for the cleanup!
I tried adding a simple nixos test, but couldn't get it to work…
Also noticed there wasn't any output logged. Maybe something is wrong with how configuration is passed to it?

@flokli You'll notice I removed the openFirewall option. I did this because

The port is 6789, not 8989 like the service said.
The user will configure the port within the application itself so not much point to hard code an assumption like that.

I'd recommend running this for the curl test: curl -u nzbget:tegbzn6789 <IP_ADDRESS>:6789 | grep "This file is part of nzbget. See <http://nzbget.net>."

In lack of a better idea, I did use a lib.mkForce to exclude unrar in nzbget's path in the vm test.
While doing so, I also noticed we did rely on coreutils being in path - I made this explicit.

@GrahamcOfBorg test nzbget

@flokli your changes LGTM :+1:
So... squash & merge, then a 19.03 breakfix backport because the service is unusable as it stands?

I think we might want some mkRemovedOptionModule here. There might still be some users who update from older releases who use this service, and it's good practice to not remove options without adding one of those.

I think we might want some mkRemovedOptionModule here. There might still be some users who update from older releases who use this service, and it's good practice to not remove options without adding one of those.

Fair. Addressed in latest commit.

@aanderse would you be up to adding something to the 19.09 changelog docs?

@aanderse would you be up to adding something to the 19.09 changelog docs?

Sure I'll get to it later today. Given this is broken in 19.03 is the plan still to backport? If so is the 19.09 release notes the most appropriate place?

@GrahamcOfBorg test nzbget

Sure I'll get to it later today. Given this is broken in 19.03 is the plan still to backport? If so is the 19.09 release notes the most appropriate place?

I'm not sure if we decided on a process for that. I'd say if it'll be backported to 19.03 too, it should be in the 19.03 changelog.
Problem is that we already released 19.03, so theoretically somebody who already updated to 19.03, had nzbget running, and updates the channel after this is backported, might run into problems.
However, given nzbget is pretty broken in 19.03 anyways, I doubt anybody uses it currently, and unbreaking it is fine - but I think we should get some decision from @samueldr or @lheckemann…

I think the NixOS modules system is composable enough so that one would be able to import master's module for nzbget. If it works as expected, I would go against backporting this. Otherwise, it's not really scary, probably more rude than anything. Though I'm not sure how I'd like the precedent this would cause.

In that case, let's not backport, only merge into master and add the changelog to 19.09 (like this PR already does).

Yeah that makes sense. Thanks all!

Yes, thanks to everyone :-)


config.json not found in app dir after using electron-packager

Hi,
Is it possible to give an example on how to use the package after packaging the app with electron-packager ?
(I guess it is in asar archive)
Thanks

The config is not saved in the app. You wouldn't want the config to go away when the app updates.
Per the readme:

Config is saved in a JSON file in app.getPath('userData').


Disconnect All button

What would you like to be added:
A single button to disconnect all K8s connections.
Why is this needed:
I am often connected to multiple clusters and at the end of the day it would be really convenient to be able to disconnect from all of them quickly.
*Environment you run Lens application on:

Kubernetes distribution: EKS
Desktop OS: MacOS

May I try this?


Update "Show Me More" to fit styling

Summary

[ ] Work with @BrianHHough to make the new "Show Me More" tab more modern and user-friendly

Motivation
Moves us towards a more polished, finished UI
Describe alternatives you've considered
Leaving it the way it is now

If that means the Table.jsx component, this was built using bootstrap and I am planning to change it to material-ui. If so I will wait for @BrianHHough to tell me what to is the changes.
If not I need more explanation

@AdhamAH  Are you working on this? Im free to grab another story and was eyeing this.

I was letting this a bet off because it looks good for now and wanted to try make the app bug free and polished.
It is not just the style. First this should element the CSS file. Also I was planing to use one of the fixed header table form material-ui and add a search function to make it easier
If you think you can at least make the table I talked about with styling using Material-ui, sure by all mean go for it

Or something else you think it could fit better, let me know

Quick update on this: Refactored the bar chart into a line chart. Going to tackle the table next. Should be ready by tomorrow.

Great! Excited to see the PR!

In lieu of some overlapping work to table.jsx on #460  -- should we cut the table changes out of scope here? And I can push up my changes to the chart?


Duplicated Code ?

https://github.com/airbnb/enzyme/blob/2b219bf9052a8d75dc002093cd51f6341fa9e0b6/src/ReactWrapperComponent.jsx#L46-L62
And I want to know when the returned value of getPublicInstance is null ?

Here's one place where it can return null.
You're right about the duplicate code tho - we should pick one as the preferred one, redefine the other in terms of it, and deprecate the other.

Ok. I got it. Thx.


Add yarn.lock

I'm making some improvements to our Dockerfile to better take advantage of the cache. Having a yarn.lock file will make that possible. It is also recommended in the yarn documentation  ( https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all/ last line).
Why was it removed in the first place? Is it a good idea to have it here?
I made sure to pull the latest superset version and run yarn before including the yarn.lock file
@graceguo-supercat @john-bodley  @mistercrunch

Codecov Report

Merging #4674 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4674   +/-   ##
=======================================
  Coverage   71.37%   71.37%
=======================================
  Files         190      190
  Lines       14918    14918
  Branches     1102     1102
=======================================
  Hits        10648    10648
  Misses       4267     4267
  Partials        3        3

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d427f6a...a755854. Read the comment docs.

Do we need instructions on CONTRIBUTING.md related to this?

@mistercrunch Sure. I'll be happy to add. The instructions would be something along the lines of updating the yarn.lock file whenever you add a new javascript dependency? Anything else you would want me to add?

https://github.com/apache/incubator-superset/issues/4274


Integrate Native Module codegen into Xcode build pipeline

Move the codegen invocation out of Podfiles and into the FBReactNativeSpec Pod itself. With this change, developers do not need to modify their existing project's Podfiles, and yet the codegen will be integrated into their projects automatically by way of the FBReactNativeSpec Pod.
This is accomplished in part by injecting a script build phase into the Pods Xcode project that is generated by CocoaPods. The build phase will be executed if the generated FBReactNativeSpec{.h, -generated.mm} files are not present, or if the contents of the Libraries directory has changed. The codegen is also triggered by pod install via prepare_command.
The codegen will thus be invoked in these situations:
RNTester:

When pod install is invoked in packages/rn-tester/ (prepare_command)
When packages/rn-tester/RNTesterPods.xcworkspace is built, if the output files are not present or if the input files have changed (script_phase).

OSS React Native apps:

When pod install is invoked in ios/ (prepare_command)
When ios/AwesomeProject.xcworkspace is built, if the output files are not present or if the input files have changed (script_phase).

Changelog: [Internal] - Moved codegen invocation out of Podfile and into FBReactNativeSpec Pod
Differential Revision: D25138896

How's this one different from https://github.com/facebook/react-native/commit/e99b8bbb404f8cd1f11b6c7998083be530d7b8a4 that already landed?

@grabbou From reading the chats leading up to this PR, I was under the impression that this would entirely move the invocation of codegen to an Xcode build phase, but looks like it’s still also done during pod install; so unsure now what the end goal is.

@alloy do you mind taking another pass at reviewing this? I want to make sure the changes make sense when it comes to CocoaPods.

Ace 👍


Using rewindReader as a public type allows external access

What would you like to have changed?

Make rewindreader-related content public methods
Why is this feature a useful, necessary, and/or important addition to this project?

When I recursively extract the file, I might read the reader content.
If I wanted to keep using reader, the rewindreader was vital.
What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

To rewind the reader, I had to implement the rewindReader myself, which felt like a hassle.
If you could make rewindReader a public method, that would really help me.

Hmm, I'm worried that it'd cause scope creep... that's currently an internal utility for identifying an archive format. Is it really that useful to export?

Could you use the bufio package instead? https://pkg.go.dev/bufio#Reader
This does feel like something more general than archives, so I'm not sure exporting it is a good fit for our package.
Closing, but open to continuing the discussion; I could probably be convinced if there's no other way.


How to use Google AutoML Tensorflow Container export in Python?

I've trained a model on Google AutoML and exported it as Container. It gave me a .pb file. I want to use it offline with Python. Here is I use in this code:
import os
import cv2
import numpy as np
import tensorflow as tf
import sys

sys.path.append("..")

from utils import label_map_util
from utils import visualization_utils as vis_util

MODEL_NAME = 'inference_graph'
IMAGE_NAME = 'test8.jpg'
CWD_PATH = os.getcwd()

PATH_TO_CKPT = os.path.join(CWD_PATH,MODEL_NAME,'frozen_inference_graph.pb')
PATH_TO_LABELS = os.path.join(CWD_PATH,'training','optik.pbtxt')
PATH_TO_IMAGE = os.path.join(CWD_PATH,IMAGE_NAME)
NUM_CLASSES = 1

label_map = label_map_util.load_labelmap(PATH_TO_LABELS)
categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=NUM_CLASSES, use_display_name=True)
category_index = label_map_util.create_category_index(categories)

detection_graph = tf.Graph()
with detection_graph.as_default():
    od_graph_def = tf.GraphDef()
    with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
        serialized_graph = fid.read()
        od_graph_def.ParseFromString(serialized_graph)
        tf.import_graph_def(od_graph_def, name='')

    sess = tf.Session(graph=detection_graph)

image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
detection_boxes = detection_graph.get_tensor_by_name('detection_boxes:0')
detection_scores = detection_graph.get_tensor_by_name('detection_scores:0')
detection_classes = detection_graph.get_tensor_by_name('detection_classes:0')
num_detections = detection_graph.get_tensor_by_name('num_detections:0')

image = cv2.imread(PATH_TO_IMAGE)
image_expanded = np.expand_dims(image, axis=0)

(boxes, scores, classes, num) = sess.run(
    [detection_boxes, detection_scores, detection_classes, num_detections],
    feed_dict={image_tensor: image_expanded})

vis_util.visualize_boxes_and_labels_on_image_array(
    image,
    np.squeeze(boxes),
    np.squeeze(classes).astype(np.int32),
    np.squeeze(scores),
    category_index,
    use_normalized_coordinates=True,
    line_thickness=8,
    min_score_thresh=0.80)

cv2.imshow('Object detector', image)
cv2.imwrite('testres.jpg', image)

cv2.waitKey(0)
cv2.destroyAllWindows()

I've put the .pb file to inference_graph folder with frozen_inference_graph.pb name. I've setted the .pbtxt file on my own. But it gives this error:
Traceback (most recent call last):
  File "Object_detection_image.py", line 67, in <module>
    od_graph_def.ParseFromString(serialized_graph)
google.protobuf.message.DecodeError: Error parsing message

@sundowatch, Provide the Tensorflow version that you are using. Thanks

@gadagashwini thanks for your response.
I'm using tensorflow-gpu 1.13.2 . I've tried on tensorflow-gpu 2.1, I'm getting the same error.
In addition to these, I've exported the model from Google AutoML Container.

@sundowatch, Provide the Tensorflow version that you are using. Thanks

@gadagashwini thanks for your response.
I'm using tensorflow-gpu 1.13.2 . I've tried on tensorflow-gpu 2.1, I'm getting the same error.
In addition to these, I've exported the model from Google AutoML Container.

Ok According to the Google Cloud Documentation suggestion this code worked for me:
docker run --rm --name CONTAINER_NAME -p 8501:8501 -v %cd%/model:/tmp/mounted_model/0001 -t gcr.io/automl-vision-ondevice/gcloud-container-1.14.0:latest


[QUESTION] - How do I get it running?

Hi @veeso, I am having trouble getting this library to work. My development environment is gnu/linux (debian 12) and I develop primarily for windows. I followed the installation steps in the readme. Although, I couldn't get it working. Here is the dependencies I added to my cargo.toml.
[dependencies]
remotefs = "^0.2.0"
remotefs-smb = "^0.1.0"

Here is the log dump of rust-analyzer,

rust-analyzer log dump
[ERROR rust_analyzer::main_loop] FetchBuildDataError:
error: failed to run custom build command for `pavao v0.1.2`

Caused by:
  process didn't exit successfully: `/home/ayush/personal/rust-ns/target/debug/build/pavao-a627555540e6bd34/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=SMBCLIENT_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-pc-windows-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_pc_windows_gnu
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_gnu
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_gnu
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  error: SMB Client library not found! Probably libsmbclient is not installed.

  --- stderr
  thread 'main' panicked at 'pkg-config has not been configured to support cross-compilation.

  Install a sysroot for the target platform and configure it via
  PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
  cross-compiling wrapper for pkg-config and set it via
  PKG_CONFIG environment variable.', /home/ayush/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pavao-0.1.2/build.rs:14:13
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
     1: core::panicking::panic_fmt
               at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
     2: core::panicking::panic_display
               at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:150:5
     3: build_script_build::main
               at ./build.rs:14:13
     4: core::ops::function::FnOnce::call_once
               at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I already ran apt install -y libsmbclient-dev libsmbclient and have these dependencies installed on my debian machine (from what I understand I shouldn't even need them for windows since iirc you mentioned it uses windows native api's). My .cargo/config.toml contains the following,
[build]
target="x86_64-pc-windows-gnu"

Let me know if I missed a step. It'd be a great help since I can't get the rust-analyzer to load & the project doesn't build either even with explicitly specifying --target flag! 🙂

x86_64-pc-windows-gnu This target is considered UNIX derived, so it will try to build with libsmbclient

Interesting! I switched the target to x86_64-pc-windows-msvc. It still fails. Here is the log dump,

➜  rust-ns git:(master) ✗ sudo cargo build -vvv
       Fresh unicode-ident v1.0.11
       Fresh proc-macro2 v1.0.66
       Fresh autocfg v1.1.0
       Fresh quote v1.0.33
       Fresh libc v0.2.147
       Fresh syn v2.0.29
       Fresh winapi v0.3.9
       Fresh thiserror-impl v1.0.47
       Fresh pkg-config v0.3.27
       Fresh thiserror v1.0.47
       Fresh num-traits v0.2.16
       Fresh time v0.1.45
       Fresh log v0.4.20
       Fresh chrono v0.4.26
   Compiling pavao v0.1.2
       Fresh lazy_static v1.4.0
       Fresh wildmatch v2.1.1
       Fresh remotefs v0.2.0
     Running `/home/ayush/personal/rust-ns/target/debug/build/pavao-a627555540e6bd34/build-script-build`
[pavao 0.1.2] cargo:rerun-if-env-changed=SMBCLIENT_NO_PKG_CONFIG
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-pc-windows-msvc
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_pc_windows_msvc
[pavao 0.1.2] cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc
[pavao 0.1.2] cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc
[pavao 0.1.2] cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
[pavao 0.1.2] cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
[pavao 0.1.2] thread 'main' panicked at 'pkg-config has not been configured to support cross-compilation.
[pavao 0.1.2]
[pavao 0.1.2] Install a sysroot for the target platform and configure it via
[pavao 0.1.2] PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
[pavao 0.1.2] cross-compiling wrapper for pkg-config and set it via
[pavao 0.1.2] PKG_CONFIG environment variable.', /home/ayush/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pavao-0.1.2/build.rs:14:13
[pavao 0.1.2] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[pavao 0.1.2] error: SMB Client library not found! Probably libsmbclient is not installed.
error: failed to run custom build command for `pavao v0.1.2`

Caused by:
  process didn't exit successfully: `/home/ayush/personal/rust-ns/target/debug/build/pavao-a627555540e6bd34/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=SMBCLIENT_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  error: SMB Client library not found! Probably libsmbclient is not installed.

  --- stderr
  thread 'main' panicked at 'pkg-config has not been configured to support cross-compilation.

  Install a sysroot for the target platform and configure it via
  PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
  cross-compiling wrapper for pkg-config and set it via
  PKG_CONFIG environment variable.', /home/ayush/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pavao-0.1.2/build.rs:14:13
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Are you running cargo build --target x86_64-pc-windows-msvc?

Yep, Here's the output veeso. I ran this on my debian host machine.

➜  rust-ns git:(master) ✗ sudo cargo build --target x86_64-pc-windows-msvc
   Compiling pavao v0.1.2
error: failed to run custom build command for `pavao v0.1.2`

Caused by:
  process didn't exit successfully: `/home/ayush/personal/rust-ns/target/debug/build/pavao-a627555540e6bd34/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=SMBCLIENT_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  error: SMB Client library not found! Probably libsmbclient is not installed.

  --- stderr
  thread 'main' panicked at 'pkg-config has not been configured to support cross-compilation.

  Install a sysroot for the target platform and configure it via
  PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
  cross-compiling wrapper for pkg-config and set it via
  PKG_CONFIG environment variable.', /home/ayush/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pavao-0.1.2/build.rs:14:13
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Hi @veeso, I found this problem. Apparently I was using the "^0.1.0" for remotefs-smb client library. The caret (^) didn't help as Cargo determined this compatibility range as anything between >=0.1.0 but <0.2.0. Therefore I didn't had the latest version of remotefs-rs-smb client library.
The old version is still specified in the README.md. I wrote a PR updating the readme so future user's won't have a similar issue.

Thanks, I merged your PR


Document bug in README

README markdown format needs to be fixed:

https://github.com/Chatie/wechaty/blob/master/README.md

I will fix these later .

here  is the pr . :)

I'll close this issue because the PR had been merged.
Thank you very much @TbhT !


Query result slowly after index data

Hi everyone
I used vearch v0.3.1 build with :

faiss v1.6.1

gcc-7.3

rocksdb-6.2

I indexed 60 million items successfully . But when search on system ,it's quite slow compared to the old version (v0.2) .  It spends ~230 ms (40ms - old v0.2 )

[INFO] search use time :[236] . max partition:[5] use time:[234]

Please advise me what problem ?

Could you give the usage of creating table and searching in detail?

@ljeagle

This is  the info of partition server

This is meta for create table :

{"id":4,"name":"test","version":1,"db_id":1,"enabled":false,"partitions":[{"id":4,"space_id":4,"db_id":1,"partition_slot":0,"replicas":[180]},{"id":5,"space_id":4,"db_id":1,"partition_slot":2147483647,"replicas":[360]}],"partition_num":2,"replica_num":1,"properties":{"imageurl": {"type": "keyword", "index": true}, "uid": {"type": "integer", "index": true}, "boundingbox": {"type": "keyword", "index": true}, "feature": {"type": "vector", "filed": "imageurl", "model_id": "vgg16", "dimension": 512, "store_type": "RocksDB", "format": "normalization"}},"engine":{"name":"gamma","index_size":10000,"max_size":700000000,"metric_type":"InnerProduct","retrieval_type":"IVFPQ","retrieval_param":{"metric_type": "InnerProduct", "ncentroids": -1, "nsubvector": -1}}}

Please confirm the parameters for the both versions are the same.

Yes ,  the parameters are the same.  Old version build with faiss v1.5.3 , gcc 4.8 and rocksdb 6.2
This is meta  table of old version

{"id":2,"name":"test","version":1,"db_id":2,"enabled":false,"partitions":[{"id":3,"space_id":2,"db_id":2,"partition_slot":0,"replicas":[180]},{"id":4,"space_id":2,"db_id":2,"partition_slot":2147483647,"replicas":[360]}],"partition_num":2,"replica_num":1,"properties":{"imageurl": {"type": "keyword", "index": true}, "uid": {"type": "integer", "index": true}, "boundingbox": {"type": "keyword", "index": true}, "feature": {"type": "vector", "filed": "imageurl", "model_id": "vgg16", "dimension": 512, "store_type": "RocksDB", "format": "normalization"}},"engine":{"name":"gamma","index_size":100000,"max_size":70000000,"nprobe":-1,"metric_type":"InnerProduct","ncentroids":-1,"nsubvector":-1,"nbits_per_idx":-1}}

@phamthanhtri  Maybe the default value of some parameters is changed in v0.3.1. Please paste the contents of gamma.log  when you created the space on v0.2 and v0.3.1

@phamthanhtri  And please paste the search request too.

Dear @qiutianme
This is gamma.log when create space on v0.3.1

INFO 2020-07-23 11:04:48,401 gamma_engine.cc:346 GammaEngine setup successed!
INFO 2020-07-23 11:04:48,401 gamma_api.cc:297 Engine init successed!
INFO 2020-07-23 11:04:48,401 raw_vector_factory.h:35 store parameters={cache size=2048000000}
INFO 2020-07-23 11:04:48,413 rocksdb_raw_vector.cc:66 rocks raw vector init success! name=feature, block cache size=2048000000Bytes
INFO 2020-07-23 11:04:48,413 raw_vector.cc:204 raw vector init success! name=feature, has source=0, multi_vids=0
INFO 2020-07-23 11:04:48,413 gamma_index_factory.h:36 Create index model [0]
INFO 2020-07-23 11:04:48,413 gamma_index_factory.h:46 metric_type = 0, ncentroids =256, nsubvector =64, nbits_per_idx =8
INFO 2020-07-23 11:04:48,452 realtime_mem_data.cc:88 ===init total_mem_bytes is<PHONE_NUMBER>===
INFO 2020-07-23 11:04:48,452 profile.cc:170 Add field name [_id], type [4], index [0]
INFO 2020-07-23 11:04:48,452 profile.cc:170 Add field name [uid], type [0], index [1]
INFO 2020-07-23 11:04:48,452 profile.cc:170 Add field name [imageurl], type [4], index [1]
INFO 2020-07-23 11:04:48,452 profile.cc:170 Add field name [boundingbox], type [4], index [1]
INFO 2020-07-23 11:04:48,468 profile.cc:237 Create table 11 success!
INFO 2020-07-23 11:04:48,468 gamma_engine.cc:1234 Add range field [boundingbox]
INFO 2020-07-23 11:04:48,469 gamma_engine.cc:1234 Add range field [imageurl]
INFO 2020-07-23 11:04:48,469 gamma_engine.cc:1234 Add range field [uid]
INFO 2020-07-23 11:04:48,471 gamma_engine.cc:664 create table [11] success!

This is gamma.log when create space on v0.2

INFO 2020-05-11 17:46:27,111 gamma_engine.cc:308 GammaEngine setup successed!
INFO 2020-05-11 17:46:27,111 gamma_api.cc:312 Engine init successed!
INFO 2020-05-11 17:46:27,111 vector_manager.cc:53 ivfpq parameters: metric_type=0, nprobe=20, ncentroids=256, nsubvector=64, nbits_per_idx=8
INFO 2020-05-11 17:46:27,111 raw_vector_factory.h:34 store parameters={cache size=122880000000}
INFO 2020-05-11 17:46:27,112 profile.cc:42 Profile created success!
INFO 2020-05-11 17:46:27,112 gamma_engine.cc:308 GammaEngine setup successed!
INFO 2020-05-11 17:46:27,112 gamma_api.cc:312 Engine init successed!
NFO 2020-05-11 17:46:28,000 rocksdb_raw_vector.cc:63 rocks raw vector init success! name=feature, block cache size=122880000000Bytes
INFO 2020-05-11 17:46:28,024 rocksdb_raw_vector.cc:63 rocks raw vector init success! name=feature, block cache size=122880000000Bytes
INFO 2020-05-11 17:46:28,354 profile.cc:140 Add field name [_id], type [4], index [0]
INFO 2020-05-11 17:46:28,354 profile.cc:140 Add field name [_version], type [1], index [0]
INFO 2020-05-11 17:46:28,354 profile.cc:140 Add field name [_slot], type [0], index [0]
INFO 2020-05-11 17:46:28,354 profile.cc:140 Add field name [uid], type [1], index [1]
INFO 2020-05-11 17:46:28,354 profile.cc:140 Add field name [imageurl], type [4], index [1]
INFO 2020-05-11 17:46:28,354 profile.cc:140 Add field name [boundingbox], type [4], index [1]
INFO 2020-05-11 17:46:28,355 profile.cc:164 Create table 6 success!

This is search function by python
def test_api(dummy):
    min_score = 0
    filters = None
    _HEADERS = {"content-type": "application/json"}
    feat = (emb/np.linalg.norm(emb)).tolist()
    data = {"query": {"sum": [{"feature": feat, "field": "feature","min_score": min_score}],"filter":filters}}
    url = f"{router_address}/{video['db']}/{video['space']}/_search?size=10"
    res =  requests.post(url, headers=_HEADERS, data=json.dumps(data))
    res = res.json()
    return res

@phamthanhtri  As I see, only the cache size is different. So you can increase the cache size in v0.3.1 according to your total memory size of machine.
v0.3.1  cache size is 2G

INFO 2020-07-23 11:04:48,413 rocksdb_raw_vector.cc:66 rocks raw vector init success! name=feature, block cache size=2048000000Bytes

v0.2 cache size is about 120G

NFO 2020-05-11 17:46:28,000 rocksdb_raw_vector.cc:63 rocks raw vector init success! name=feature, block cache size=122880000000Bytes

Thanks @qiutianme  , How do I change the cache size ?

@phamthanhtri  When you create space, you can set the "cache_size" parameter is in "store_param" of vector field. Just like this, I set cache size=10240M bytes which means 10G bytes.
{
    "properties": {
           "vector_field": {
            "type": "vector",
            "dimension": 256,
            "store_type": "RocksDB",
            "store_param": {
                "cache_size": 10240
            }
        }
    }
}

And you can also refer to the document https://vearch.readthedocs.io/en/latest/use_op/op_space.html#create-space

I create many test case with v0.3.1 for search benchmarks on 2 partition servers . Each server will contain half of the items:

50 thousands items (each partition server 25 thousands items) :  tooks 10ms
20 million items:  tooks 120ms
70 million items:  tooks 200ms  (v0.2 tooks 60ms)
230 million items:  tooks 700ms

It increases linear time .
I add log for check excute time search in ps/engine/gammacb/reader.go
	arr := C.SearchV2(ri.engine.gamma, req)
	defer C.DestroyByteArray(arr)
	resp := gamma_api.GetRootAsResponse(CbArr2ByteArray(arr), 0)
	idType := ri.engine.space.Engine.IdType
	result := ri.singleSearchResult(resp, 0, idType)
	result.MaxTook = int64(time.Now().Sub(start) / time.Millisecond)
	result.MaxTookID = ri.engine.partitionID
	t3 := time.Now()
	log.Debug("Search_end :[%d]", (t3.Sub(t1) / time.Millisecond))

It took most of the time here . I would like to confirm that I'm using faiss v1.6.3

@phamthanhtri This is weird！Different versions of faiss will not have such a big impact on performance.
You can do another experiment. Set the quick=true in search request, to exclude the impact of reading raw vectors from rocksdb on performance. Refer to https://vearch.readthedocs.io/en/latest/use_op/op_doc.html#search
{
    "query": {
    },
    "quick": false
}

Sorry! It is
{
    "query": {
    },
    "quick": true
}

Thanks @qiutianme
I  test with new experiment with your search config {     "query": {     },     "quick": true }
But the processing time has not changed :(

@phamthanhtri Is each query a single query? Not batch? Now the default value of the query parallel mode has been changed. The previous one was parallel on the inverted linked list (v0.2), and now it is changed to parallel between queries (v3.1.0). You can try to query like this { "query": { }, "parallel": false }

Yes, I forget it! The default parallel mode of v0.3.1 is changed to parallel_on_query from parrallel_on_inverted_list. @phamthanhtri you can try it with zcdb 's advice, set "parallel": false in search request

I set param  parallel and quick but it's not effective
`data = {"query": {"sum": [{"feature": feat, "field": "feature"}], "parallel": False,"quick": True}}
I think the reason is maybe from faiss ( 1.5.3 -> 16.3) . I am currently reviewing it.
`

Hi everyone
I review code  in engine/gamma . Most of the process time of search request  in function scan_one_list  (gamma_index_ivfpq.cc)
for (size_t ik = 0; ik < nprobe; ik++) {
        nscan += scan_one_list (
            scanner, keys [i * nprobe + ik], coarse_dis[i * nprobe + ik],
            simi, idxi, k, this->nlist, this->invlists, store_pairs,
            condition->ivf_flat, raw_vec_head);

        if (max_codes && nscan >= max_codes) {
          break;
        }
      }

[scan_one_list] sys: 219ms

It used method scan_list_with_table (gamma_index_ivfpq.h).

Hi everyone
I review code in engine/gamma . Most of the process time of search request in function scan_one_list (gamma_index_ivfpq.cc)
for (size_t ik = 0; ik < nprobe; ik++) {
        nscan += scan_one_list (
            scanner, keys [i * nprobe + ik], coarse_dis[i * nprobe + ik],
            simi, idxi, k, this->nlist, this->invlists, store_pairs,
            condition->ivf_flat, raw_vec_head);

        if (max_codes && nscan >= max_codes) {
          break;
        }
      }

[scan_one_list] sys: 219ms

It used method scan_list_with_table (gamma_index_ivfpq.h).

@phamthanhtri  This is the problem. If you set parallel=false, it should enter this line("parallelize over inverted lists") https://github.com/vearch/gamma/blob/f0def6f43117766a2fcf08765a970d4e12d5a666/index/gamma_index_ivfpq.cc#L888, not https://github.com/vearch/gamma/blob/f0def6f43117766a2fcf08765a970d4e12d5a666/index/gamma_index_ivfpq.cc#L863.
Please confirm it !

@phamthanhtri  Just for test,  you can force set condition->parallel_mode=1 in this line https://github.com/vearch/gamma/blob/f0def6f43117766a2fcf08765a970d4e12d5a666/index/gamma_index_ivfpq.cc#L714

Hi @qiutianme
I test 2 case with parallel=false and parallel=true. It process same the time .
I think the problem comes from the code  .
nscan += scan_one_list (
           scanner, keys [i * nprobe + ik], coarse_dis[i * nprobe + ik],
           simi, idxi, k, this->nlist, this->invlists, store_pairs,
           condition->ivf_flat, raw_vec_head);

timer.h:65 [scan_codes] sys: 14ms, cpu: 0s.
timer.h:65 [scan_one_list] sys: 219ms, cpu: 0s.

I measured the time of function  scan_one_list ,  scan_codes and I think it was not running in parallel (openMP is not effective?) . Probably the reason I built faiss and vearch with gcc7.3 but blas in system centOS  has install a version of gcc4.8 . I am looking into this problem ?

@phamthanhtri  You can use "top" command to look at the cpu usage of v3.1.0. if cpu usage is low, then openMP is most likely the problem.

Hi everyone
I resolved this problem. There are some things to share
1: As zcdb 's advice  , vearch v3.0.1 search base parallelize over queries . But when I change param "parallel" =  (False or True)  on request search , it not change value of condition->parallel_based_on_query . It always has value  =1  .
This is search param request
       data = {"query": {"sum": [{"feature": feat, "field": "feature"}], "parallel": False}}
       res = requests.post(query_internal_url, headers=header, data=json.dumps(data))

I changed value  condition->parallel_mode=1 for enter this line("parallelize over inverted lists")
2:  It not start flag -fopenmp in cmake file configure , I must add it for build source with openmp.  And with line code  when enter
https://github.com/vearch/gamma/blob/f0def6f43117766a2fcf08765a970d4e12d5a666/index/gamma_index_ivfpq.cc#L838  . It crash PS server

Error in ./vearch': malloc(): memory corruption (fast): 0x00007ed5b4000b00 *** *** Error in ./vearch': free(): invalid next size (fast): 0x00007ed5b4000a40 ***

I fix it by add #pragma omp parallel before this line https://github.com/vearch/gamma/blob/f0def6f43117766a2fcf08765a970d4e12d5a666/index/gamma_index_ivfpq.cc#L837

I show result benchmark after change and run successfully:

70 million items: 40ms  (v0.3.1 no opemp - 200ms)  (v0.2 - 60ms)

230 million items: 120ms (v0.3.1  no opemp - 700ms)

@phamthanhtri I experimented on v3.1.0 to change the parallel mode by setting parallel, and found that it can take effect. So I rechecked your query statement. It should be like this {"query":{}, "parallel":false} to set parallel mode.
here is yours:
data = {"query": {"sum": [{"feature": feat, "field": "feature"}], "parallel": False}}
it should be:
data = {"query": {"sum": [{"feature": feat, "field": "feature"}]}, "parallel": False}
Sorry for it, we should check it more carefully.


RDMs implements len.

We can already check the number of RDMs in a stack by using RDMs.n_rdm.  However, providing this through __len__ has advantages:

It is idiomatic: people will try to use len(rdms), and this will give them what they're looking for.
Together with __getitem__, which provides for iteration with for rdm in rdms:, __len__ lets you use reversed(rdms), which can be useful in some contexts.

Looks good to me. I did not expect that this is already sufficient to do the iteration

I didn't either! I just thoughtlessly typed for rdm in rdms: and was bewildered when it didn't raise an error… then I looked up why! :)


huggingface code error

Does not start

Dockerfile

Try to change CMD ["/app/teldrive"] to CMD ["/app/teldrive run"]

Try to change CMD ["/app/teldrive"] to CMD ["/app/teldrive run"]

Attempt #2
CMD ["/app/teldrive", "run"]

Hm, hard to tell, maybe just get rid of first "downloader" stage? As second stage you are using ubuntu:latest which can download file also, maybe will help 🤷♂

?

Send here docker file as text

CMD ["/app/teldrive", "run"]# First stage to download the latest release
--
  | FROM alpine as downloader
  |
  | # Install necessary packages
  | RUN apk update && apk add --no-cache curl jq
  |
  | # Set the repository owner and repository name
  | ARG REPO_OWNER=divyam234
  | ARG REPO_NAME=teldrive
  |
  | # GitHub API URL for the latest release
  | ARG API_URL=https://api.github.com/repos/$REPO_OWNER/$REPO_NAME/releases/latest
  |
  | # Fetch the latest release from GitHub
  | RUN LATEST_RELEASE_TAG=$(curl -s $API_URL \| jq -r '.tag_name') && \
  | wget "https://github.com/$REPO_OWNER/$REPO_NAME/releases/download/$LATEST_RELEASE_TAG/teldrive-$LATEST_RELEASE_TAG-linux-amd64.tar.gz" -O /teldrive.tar.gz
  |
  | # Second stage to setup the final image
  | FROM ubuntu:latest
  |
  | # Install necessary packages
  | RUN apt-get update && \
  | apt-get install -y ca-certificates tzdata wget tar curl && \
  | rm -rf /var/lib/apt/lists/*
  |
  | # Copy the teldrive binary tarball from the downloader stage
  | COPY --from=downloader /teldrive.tar.gz /app/teldrive.tar.gz
  |
  | WORKDIR /app
  |
  | # Extract teldrive and setup environment
  | RUN tar -xvf teldrive.tar.gz && \
  | rm teldrive.tar.gz && \
  | chmod -R 777 /app
  |
  | # Start Teldrive when the container launches
  | CMD ["/app/teldrive", "run"]

Nowy dokument tekstowy.txt

Btw what are you trying to do as this repo already have docker image ghcr.io/divyam234/teldrive ?

FROM ubuntu:latest

# Install necessary packages
RUN apt-get update && \
    apt-get install -y ca-certificates jq jqtzdata wget tar curl && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN wget $(wget -q -O - https://api.github.com/repos/divyam234/teldrive/releases/latest  |  jq -r '.assets[] | select(.name | contains ("linux-amd64.tar.gz")) | .browser_download_url') -O /teldrive.tar.gz

# Extract teldrive and setup environment
RUN tar -xvf teldrive.tar.gz && \
    rm teldrive.tar.gz && \
    chmod -R 777 /app

# Start Teldrive when the container launches
CMD ["/app/teldrive", "run"]

FROM ubuntu:latest

# Install necessary packages
RUN apt-get update && \
    apt-get install -y ca-certificates jq jqtzdata wget tar curl && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN wget $(wget -q -O - https://api.github.com/repos/divyam234/teldrive/releases/latest  |  jq -r '.assets[] | select(.name | contains ("linux-amd64.tar.gz")) | .browser_download_url') -O teldrive.tar.gz

# Extract teldrive and setup environment
RUN tar -xvf teldrive.tar.gz && \
    rm teldrive.tar.gz && \
    chmod -R 777 /app

# Start Teldrive when the container launches
CMD ["/app/teldrive", "run"]

FROM ubuntu:latest

# Install necessary packages
RUN apt-get update && \
    apt-get install -y ca-certificates jq wget tar curl && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN wget $(wget -q -O - https://api.github.com/repos/divyam234/teldrive/releases/latest  |  jq -r '.assets[] | select(.name | contains ("linux-amd64.tar.gz")) | .browser_download_url') -O teldrive.tar.gz

# Extract teldrive and setup environment
RUN tar -xvf teldrive.tar.gz && \
    rm teldrive.tar.gz && \
    chmod -R 777 /app

# Start Teldrive when the container launches
CMD ["/app/teldrive", "run"]

So, now you need add config, look for required flags

So, now you need add config, look for required flags

dockerfile
  FROM ubuntu:latest

  RUN apt-get update && \
      apt-get install -y ca-certificates tzdata wget tar curl

  WORKDIR /app
  COPY . .
  RUN chmod -R 777 /app

  RUN --mount=type=secret,id=gist_teldrive,mode=0444,required=true \
      wget https://github.com/divyam234/teldrive/releases/latest/download/teldrive-1.1.3-linux-amd64.tar.gz -O /app/teldrive.tar.gz && \
      tar xvf /app/teldrive.tar.gz -C /app  && \
      wget $(cat /run/secrets/gist_teldrive) -O /app/config.toml && \
      chmod a+x /app/teldrive && chmod 777 /app/config.toml  && \
      touch /app/session.db && \
      chmod 777 /app/session.db

  RUN chmod a+x /app/start.sh

  CMD ["./start.sh"]

start.sh
#!/bin/bash
/app/teldrive run --config $HOME/app/config.toml

This Worked for me

Yeah, it did work, my only issue now is for some reason, After i successfullly logged in, im still sent back to the log in screen. let me walk you through
Step 1: The main login page shows up
Step 2: I enter phone number and it requests for code
Step 3: I enter code and press login.
Step 4: It loaadsss then it goes back to the log in page.
Nothing showed up on the logs as red flags, hell, the log in was successful.

dockerfile
  FROM ubuntu:latest

  RUN apt-get update && \
      apt-get install -y ca-certificates tzdata wget tar curl

  WORKDIR /app
  COPY . .
  RUN chmod -R 777 /app

  RUN --mount=type=secret,id=gist_teldrive,mode=0444,required=true \
      wget https://github.com/divyam234/teldrive/releases/latest/download/teldrive-1.1.3-linux-amd64.tar.gz -O /app/teldrive.tar.gz && \
      tar xvf /app/teldrive.tar.gz -C /app  && \
      wget $(cat /run/secrets/gist_teldrive) -O /app/config.toml && \
      chmod a+x /app/teldrive && chmod 777 /app/config.toml  && \
      touch /app/session.db && \
      chmod 777 /app/session.db

  RUN chmod a+x /app/start.sh

  CMD ["./start.sh"]

start.sh
#!/bin/bash
/app/teldrive run --config $HOME/app/config.toml

This Worked for me

Yeah, it did work, my only issue now is for some reason, After i successfullly logged in, im still sent back to the log in screen. let me walk you through
Step 1: The main login page shows up Step 2: I enter phone number and it requests for code Step 3: I enter code and press login. Step 4: It loaadsss then it goes back to the log in page.
Nothing showed up on the logs as red flags, hell, the logs say that the login was successful, but on the actual page, its just not doing anything.

You have to enter directly from the link

dockerfile
  FROM ubuntu:latest

  RUN apt-get update && \
      apt-get install -y ca-certificates tzdata wget tar curl

  WORKDIR /app
  COPY . .
  RUN chmod -R 777 /app

  RUN --mount=type=secret,id=gist_teldrive,mode=0444,required=true \
      wget https://github.com/divyam234/teldrive/releases/latest/download/teldrive-1.1.3-linux-amd64.tar.gz -O /app/teldrive.tar.gz && \
      tar xvf /app/teldrive.tar.gz -C /app  && \
      wget $(cat /run/secrets/gist_teldrive) -O /app/config.toml && \
      chmod a+x /app/teldrive && chmod 777 /app/config.toml  && \
      touch /app/session.db && \
      chmod 777 /app/session.db

  RUN chmod a+x /app/start.sh

  CMD ["./start.sh"]

start.sh
#!/bin/bash
/app/teldrive run --config $HOME/app/config.toml

This Worked for me

Yeah, it did work, my only issue now is for some reason, After i successfullly logged in, im still sent back to the log in screen. let me walk you through
Step 1: The main login page shows up Step 2: I enter phone number and it requests for code Step 3: I enter code and press login. Step 4: It loaadsss then it goes back to the log in page.
Nothing showed up on the logs as red flags, hell, the logs say that the login was successful, but on the actual page, its just not doing anything.

You have to enter directly from the link

Sorry, if its a dumb question but, may you elaborate on that. Ik its annoying to ask that sort of question, but I had to make sure that i get what you mean.

dockerfile
  FROM ubuntu:latest

  RUN apt-get update && \
      apt-get install -y ca-certificates tzdata wget tar curl

  WORKDIR /app
  COPY . .
  RUN chmod -R 777 /app

  RUN --mount=type=secret,id=gist_teldrive,mode=0444,required=true \
      wget https://github.com/divyam234/teldrive/releases/latest/download/teldrive-1.1.3-linux-amd64.tar.gz -O /app/teldrive.tar.gz && \
      tar xvf /app/teldrive.tar.gz -C /app  && \
      wget $(cat /run/secrets/gist_teldrive) -O /app/config.toml && \
      chmod a+x /app/teldrive && chmod 777 /app/config.toml  && \
      touch /app/session.db && \
      chmod 777 /app/session.db

  RUN chmod a+x /app/start.sh

  CMD ["./start.sh"]

start.sh
#!/bin/bash
/app/teldrive run --config $HOME/app/config.toml

This Worked for me

Yeah, it did work, my only issue now is for some reason, After i successfullly logged in, im still sent back to the log in screen. let me walk you through
Step 1: The main login page shows up Step 2: I enter phone number and it requests for code Step 3: I enter code and press login. Step 4: It loaadsss then it goes back to the log in page.
Nothing showed up on the logs as red flags, hell, the logs say that the login was successful, but on the actual page, its just not doing anything.

You have to enter directly from the link

Sorry, if its a dumb question but, may you elaborate on that. Ik its annoying to ask that sort of question, but I had to make sure that i get what you mean.

Do you have a public huggingface project? If you don't, turn it on in the settings and then go to the repository from the link. Do you know how to go to the repository from the link?

@jojogamezHub

@jojogamezHub

I did what u told me to do and i gotta say:
Thank you, so so much. I've been on this thing for like a day or 2 now.

I Believe we can close this issue now @Pe0xonN


Stored XSS Vulnerability #2

Steps:

Go to https://www.secretsantaorganizer.com/
Fill the fields but first, look at my photo (https://i.hizliresim.com/alRNWd.jpg)
Click "Create you event!" button.
Go to your mailbox and look at the mail from secretsanraorganizer
Click "Go to the manage page" button.
Click "Start your party" button.
Go to the mailbox, again and you have a new mail. Click it.
Click "Find out your person"
You will see xss alert.

PoC: https://www.secretsantaorganizer.com/participant/8fdfiqapde8so4gsc4k08sok8swk8cg
Regards.

see #445


Fixed Comment Out My Code for infinite loops for #1788

Fixed the issue for incorrect comments by correcting the syntax per language

Awesome, thanks @divyanshu013!

You're welcome @nwinter.


Forall over effects

Codecov Report
Patch coverage: 89.92% and project coverage change: +0.02 :tada:

Comparison is base (4638201) 85.03% compared to head (7a045df) 85.06%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #438      +/-   ##
==========================================
+ Coverage   85.03%   85.06%   +0.02%
==========================================
  Files         198      198
  Lines       25513    25586      +73
==========================================
+ Hits        21696    21765      +69
- Misses       3817     3821       +4

Impacted Files
Coverage Δ

...g/engines/compilers/conditional_effects_remover.py
95.26% <ø> (ø)

unified_planning/io/ma_pddl_writer.py
9.09% <ø> (ø)

unified_planning/model/mixins/timed_conds_effs.py
88.31% <ø> (ø)

unified_planning/model/problem_kind.py
96.99% <ø> (ø)

unified_planning/model/multi_agent/ma_problem.py
57.30% <50.00%> (-0.06%)
:arrow_down:

unified_planning/io/pddl_writer.py
84.61% <61.53%> (-0.55%)
:arrow_down:

..._planning/engines/compilers/quantifiers_remover.py
90.80% <85.29%> (-1.67%)
:arrow_down:

unified_planning/model/effect.py
84.33% <92.30%> (+0.57%)
:arrow_up:

unified_planning/io/pddl_reader.py
79.15% <96.77%> (+0.55%)
:arrow_up:

unified_planning/model/action.py
87.77% <100.00%> (+0.03%)
:arrow_up:

... and 3 more

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Characters keep reseting when rejoining the server

Pla help

Hey @Alexanderrr1206, this resource offers an API to work with the skin character, you need to develop the system to save the character on db.


Deprecate external profile

Please provide a description of this PR:
Fixes #39797

@frankbu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
707c1296b1cec063625e2fc815da28082b9c701a
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.