About use module "github.com/dolthub/maphash"

This project use a module maphash, It use the golang map internal struct to generate the hash value, as:
a := any(make(map[K]struct{}))
i := (*mapiface)(unsafe.Pointer(&a))
h = i.typ.hasher

And there are some comments like:
//go:build go1.18 || go1.19
// +build go1.18 go1.19

If the map internal struct changed in the future, that convert will fail.
How about just use the map address as the hash value like:
m := any(make(map[int]struct{}))
p := uintptr(unsafe.Pointer(&m))
fmt.Println(p)

Or extract the hasher implement from goland(/src/runtime/alg.go)
func typehash(t *_type, p unsafe.Pointer, h uintptr) uintptr

If the map internal struct changed in the future, that convert will fail.

Yes, this is probably the otter's main problem. I really hope that in the next releases of Go a hash function for comparable types will appear in the standard library, but for now we need to work with what we have. In fact, I doubt that the authors of Go will simply change the internal structure of map, at least because many very famous projects are already using this dirty hack and this change will break a huge number of projects. The simplest example is the json-iterator.

How about just use the map address as the hash value like

To be honest, I didn't understand how this would help. The new map objects will have a new address, and the hash function should be reproducible. Can you tell me a little more about what you mean here?

Or extract the hasher implement from goland(/src/runtime/alg.go)

Yes, this is one of the possible solutions. I have a suspicion that indeed it will not be possible to support all comparable types in this way. There may be potential problems with the interfaces, but this needs to be checked. And it's better than nothing :)
The simplest solution to this problem is to ask the user to pass the hash function. This is done in many languages, but I'm not sure that Go developers will like it.

Thank you maypok86.

I doubt that the authors of Go will simply change the internal structure of map, at least because many very famous projects are already using this dirty hack and this change will break a huge number of projects

That make sence.

Can you tell me a little more about what you mean here?

To be honest, I am still new to project. Forget this advice, maybe i will find some useful way in the future.
By the way, this is really a very good project, it implements useful methods  in some papers.  Here i have a question about using it in production, do you have any idea about that?

To be honest, I am still new to project.

I didn't really understand how such a usage would be reproducible. For example, two strings with different addresses and the same byte sequence should always have the same hash.

By the way, this is really a very good project, it implements useful methods in some papers. Here i have a question about using it in production, do you have any idea about that?

For mass use in production, you need a lot of fame, but I've never done anything at all for this. I was offered a couple of times to write an article about the project, but I always refused it, almost completely concentrating on the internal component. Even the vast majority of the repository's stars were due to the fact that someone posted a link to the project on hacker news, it got into trends for some reason and it began to be published in lesser-known places.
In my opinion, notifying ristretto users about its very small hit ratio on recent versions would be much more useful to the world, but even in this case, I would not really want to try to push the project through.
If you want to know examples of projects/companies that use otter, then I know quite a bit about it. The most famous open-source otter user at the moment is frankenphp. Otter was also added as a storage option in souin. It seems that Shopee was interested in otter, but I can't say that they really use it. Several times people came asking to add something or with some questions, but I do not know if they use otter anywhere. That's about all I know about the use cases in production.

Thanks, maypok86. otter is more complex than other cache projects because the papers you implemented, hope it will be more popular, it is really deserve to try and learn!

I think I've answered the questions in this issue, so I'm closing it. If something is wrong, then write.


update new changes for the tele

I don't think you meant to submit this PR to me. Could you review your exercise instructions so this stops happening?


db_version 29 support?

seeing a lot of messages like
Refusing to revert from db_version=29 to lower version=24
in both summary and removechannel commands
lnd version used is same as mentioned in https://github.com/guggero/chantools/blob/master/doc/chantools_removechannel.md:
2022-10-06 22:10:37.615 [INF] LTND: Version: 0.15.1-beta commit=v0.15.1-beta
chantools version exactly same as in readme install instructions and latest release, using:
wget -O chantools.tar.gz https://github.com/guggero/chantools/releases/download/v0.10.4/chantools-linux-amd64-v0.10.4.tar.gz

I forgot to create a release after bumping the supported version of lnd to v0.15.1-beta. Try with https://github.com/guggero/chantools/releases/tag/v0.10.5, that should work.


Sample is crashing.

Hi, I have just installed your sample from the play market. However, it's crashing after choosing the second configuration of tabs, i.e.
launched app->configuration selected -> open tabs ->back->configured again->opening results in crash.
Xperia Z1 compact, Lolipop 5.0.1
Unfortunately, I haven't so much time now, maybe I'll look deeper inside your sample later :)

Hi @ewintory! Thanks for reporting this issue. I tried reproducing your steps and the app didn't crash on my phone.
Surprisingly I didn't get any crash report from Crashlytics. Can you reopen the sample while connected to the internet (or maybe attach some logs if that's easier for you)?
Thanks again! I'll look into it more in details in a few hours in all cases.

Update: I just received the crash report actually. Thanks, will look into it!

Sure, here you are :)
05-24 04:52:01.123    7970-7970/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
    Process: io.karim.materialtabs.sample, PID: 7970
    java.lang.NullPointerException: Attempt to invoke virtual method 'int android.widget.RadioGroup.getCheckedRadioButtonId()' on a null object reference
            at io.karim.materialtabs.sample.MainActivity.goToTabsActivityButtonClicked(MainActivity.java:125)
            at io.karim.materialtabs.sample.MainActivity.onOptionsItemSelected(MainActivity.java:108)
            at android.app.Activity.onMenuItemSelected(Activity.java:2882)
            at android.support.v4.app.FragmentActivity.onMenuItemSelected(FragmentActivity.java:353)
            at android.support.v7.app.AppCompatActivity.onMenuItemSelected(AppCompatActivity.java:144)
            at android.support.v7.internal.view.WindowCallbackWrapper.onMenuItemSelected(WindowCallbackWrapper.java:99)
            at android.support.v7.internal.view.WindowCallbackWrapper.onMenuItemSelected(WindowCallbackWrapper.java:99)
            at android.support.v7.internal.app.ToolbarActionBar$2.onMenuItemClick(ToolbarActionBar.java:74)
            at android.support.v7.widget.Toolbar$1.onMenuItemClick(Toolbar.java:164)
            at android.support.v7.widget.ActionMenuView$MenuBuilderCallback.onMenuItemSelected(ActionMenuView.java:740)
            at android.support.v7.internal.view.menu.MenuBuilder.dispatchMenuItemSelected(MenuBuilder.java:802)
            at android.support.v7.internal.view.menu.MenuItemImpl.invoke(MenuItemImpl.java:153)
            at android.support.v7.internal.view.menu.MenuBuilder.performItemAction(MenuBuilder.java:949)
            at android.support.v7.internal.view.menu.MenuBuilder.performItemAction(MenuBuilder.java:939)
            at android.support.v7.widget.ActionMenuView.invokeItem(ActionMenuView.java:598)
            at android.support.v7.internal.view.menu.ActionMenuItemView.onClick(ActionMenuItemView.java:139)
            at android.view.View.performClick(View.java:4832)
            at android.view.View$PerformClick.run(View.java:19839)
            at android.os.Handler.handleCallback(Handler.java:739)
            at android.os.Handler.dispatchMessage(Handler.java:95)
            at android.os.Looper.loop(Looper.java:211)
            at android.app.ActivityThread.main(ActivityThread.java:5317)
            at java.lang.reflect.Method.invoke(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:372)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1016)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:811)

Btw I was connected to the internet while crash happened, so maybe you should check your Chrashlitics configuration as well...

Check my previous comment ;-) but thanks anyway for the logs! I'll look into it very soon.

:+1:

@ewintory Hi again! I most probably fixed the issue. Can you please try to reproduce the steps you mentioned above with the updated APK here: https://github.com/pizza/MaterialTabs/blob/master/sample/sample-release.apk ? Thanks!

Hi, @pizza! Great, It works without any crashes :)

@ewintory Awesome. Thanks for your help :)


Asp.net core putting the same Guid for all empty Guid's

Hello, guys.
I'm going through a problem when using Guid as Id, when I have Entity like:
`
public class SubcategoryDto
{
public Guid Id { get; set; }
    public string Description { get; set; }

    public Guid CategoryId { get; set; }
}`

and I have a Action in my controller with:
public IActionResult Create(Guid id)         {             var dto = new SubcategoryDto()             {                 CategoryId = id             };             return View(dto);         }
I expect in my View I receive the follow object:
{
"id": "00000000-0000-0000-0000-000000000000",
"description": "",
"categoryid": "dba6b8e4-fb16-4f88-290d-08d44abea368"
}
But I receive:
{
"id": "dba6b8e4-fb16-4f88-290d-08d44abea368",
"description": "",
"categoryid": "dba6b8e4-fb16-4f88-290d-08d44abea368"
}
So, is it a issue or I need to do some configuration?

@rynowak ?

We don't have any magic around guids. I don't see anything in the code sample that sets the Id value thus it should always be empty.
Do you have a more complete code sample including the view, and ideally what the request looks like?

Here is a test repo:
https://github.com/jonatanrinckus/AspNetCoreGuidIssueTest
the HomeController has this action:
public IActionResult Edit(Guid id)         {             var test = new Test()             {                 AnotherGuidId = id             };             return View(test);         }
if you go to /home/edit/4689d77b-a373-450d-a7e6-3394b557ce32
It you bring the Guid on both inputs, but this should me just on one.
And it just happens if the name of parameter is "id".
if I add a [Route("{id}"] attribute, the same happens.
If i chenge for another parameter name like "idt" and [Route("{idt}"], it works like expected.

This happens because entries in the ModelState take precedence over the values in the model. This is needed because when you submit a form and are sent back to the form (because of validation errors) you want to see the inputs that you typed in, and not the values on the model.
/cc @dougbu in case he wants to add more
When you successfully bind the id route value, it creates an entry in the ModelState for that value. If you don't want to see that on the form, just clear it.
        public IActionResult Edit(Guid id)
        {
            var test = new Test()
            {
                AnotherGuidId = id
            };

            ModelState.Remove("id");

            return View(test);
        }

This issue is being closed because it has not been updated in 3 months.
We apologize if this causes any inconvenience. We ask that if you are still encountering this issue, please log a new issue with updated information and we will investigate.

This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a temporary error. The following address(es) deferred:
<EMAIL_ADDRESS>Domain spinthemoose.com has exceeded the max emails per hour (216/200 (108%)) allowed.  Message will be reattempted later
------- This is a copy of the message, including all the headers. ------
Received: from github-smtp2-ext4.iad.github.net ([<IP_ADDRESS>]:48958 helo=github-smtp2a-ext-cp1-prd.iad.github.net)
by cp1164.blacksun.ca with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.89_1)
(envelope-from<EMAIL_ADDRESS>id 1eVQfy-0000GU-EH
for<EMAIL_ADDRESS>Sat, 30 Dec 2017 17:36:26 -0600
Date: Sat, 30 Dec 2017 15:36:06 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com;
s=pf2014; t=1514676966;
bh=uHhsz/F6s96V/iYVoAoWSyYq9MRA11ukl6m7uZa0+Nw=;
h=From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID:
List-Archive:List-Post:List-Unsubscribe:From;
b=jYLzRFIgaqHsAePxrhmtoZyXgBFnu5uzCAUvSNxaIMXo6N2dsSMVVYzKIc9mC8oXl
I7GhV5+nutpaSwmTHNfSg2dxSixxudMwPa5iaqtyqvrP8pNols4RoTb905G9/FBslJ
qC2xkbzpvn2q/R/uEKv427Q5/RLQN5E+h74D8gak=
From: "ASP.NET Hello Bot"<EMAIL_ADDRESS>Reply-To: aspnet/Home<EMAIL_ADDRESS>To: aspnet/Home<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>In-Reply-To<EMAIL_ADDRESS>References<EMAIL_ADDRESS>Subject: Re: [aspnet/Home] Asp.net core putting the same Guid for all empty
Guid's (#1926)
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="--==_mimepart_5a4822e6b1a2e_7793fbb505e8f3023362ae";
charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: aspnet-hello
X-GitHub-Recipient: davidalpert
X-GitHub-Reason: subscribed
List-ID: aspnet/Home <Home.aspnet.github.com>
List-Archive: https://github.com/aspnet/Home
List-Post<EMAIL_ADDRESS>List-Unsubscribe<EMAIL_ADDRESS>https://github.com/notifications/unsubscribe/AACpRAgeN5oxkywwAcXephDMHXYsmSJoks5tFsjmgaJpZM4L3lis
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address<EMAIL_ADDRESS>----==_mimepart_5a4822e6b1a2e_7793fbb505e8f3023362ae
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
This issue is being closed because it has not been updated in 3 months.
We apologize if this causes any inconvenience. We ask that if you are still encountering this issue, please log a new issue with updated information and we will investigate.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/aspnet/Home/issues/1926#issuecomment-354574917
----==_mimepart_5a4822e6b1a2e_7793fbb505e8f3023362ae
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: quoted-printable
This issue is being closed because it has not been updated in 3 months=
.
We apologize if this causes any inconvenience. We ask that if you are =
still encountering this issue, please log a new issue with updated inform=
ation and we will investigate.
&m=
dash;You are receiving this because you are subscribed to this thre=
ad.Reply to this email directly, view it on GitHub, or mute the thread.

=
----==_mimepart_5a4822e6b1a2e_7793fbb505e8f3023362ae--

This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a temporary error. The following address(es) deferred:
<EMAIL_ADDRESS>Domain spinthemoose.com has exceeded the max emails per hour (217/200 (108%)) allowed.  Message will be reattempted later
------- This is a copy of the message, including all the headers. ------
Received: from o5.sgmail.github.com ([<IP_ADDRESS>]:33539)
by cp1164.blacksun.ca with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128)
(Exim 4.89_1)
(envelope-from<EMAIL_ADDRESS>id 1eVQfz-0000GZ-0o
for<EMAIL_ADDRESS>Sat, 30 Dec 2017 17:36:27 -0600
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=github.com;
h=from:reply-to:to:cc:in-reply-to:references:subject:mime-version:content-type:content-transfer-encoding:list-id:list-archive:list-post:list-unsubscribe;
s=s20150108; bh=NLXTiiz+aZmuLXorAXFe+qicUig=; b=FvYDakHGmQay9U2F
lG9SBOAJXAz2DNSGimHpom6InaLOlehx7YSN1YwVbWbheVGqZOfqf6qgjfao6jmo
vJGAzf+oAUK22ROigZaTcTQ8kkidLAZWLX5HorACBf4klSG7nCQ1U/xfoh/RVkRW
UjUK1H032vMyQb5+DgYcLMYwMr4=
Received: by filter0553p1mdw1.sendgrid.net with SMTP id filter0553p1mdw1-7336-5A4822E6-2C
2017-12-30 23:36:07.009897893 +0000 UTC
Received: from github-smtp2b-ext-cp1-prd.iad.github.net (github-smtp2b-ext-cp1-prd.iad.github.net [<IP_ADDRESS>])
by ismtpd0022p1iad2.sendgrid.net (SG) with ESMTP id dxDn7d29SiqHFoNiiy-Mdg
for<EMAIL_ADDRESS>Sat, 30 Dec 2017 23:36:07.018 +0000 (UTC)
Date: Sat, 30 Dec 2017 23:36:07 +0000 (UTC)
From: "ASP.NET Hello Bot"<EMAIL_ADDRESS>Reply-To: aspnet/Home<EMAIL_ADDRESS>To: aspnet/Home<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>In-Reply-To<EMAIL_ADDRESS>References<EMAIL_ADDRESS>Subject: Re: [aspnet/Home] Asp.net core putting the same Guid for all empty
Guid's (#1926)
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="--==_mimepart_5a4822e6dde79_73ea3f976a148f345609af";
charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: aspnet-hello
X-GitHub-Recipient: davidalpert
X-GitHub-Reason: subscribed
List-ID: aspnet/Home <Home.aspnet.github.com>
List-Archive: https://github.com/aspnet/Home
List-Post<EMAIL_ADDRESS>List-Unsubscribe<EMAIL_ADDRESS>https://github.com/notifications/unsubscribe/AACpRAgeN5oxkywwAcXephDMHXYsmSJoks5tFsjmgaJpZM4L3lis
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address<EMAIL_ADDRESS>X-SG-EID: rlj3t9ahcX0BZUzAOlajQ3VmBXo43xFJc4M1ImIDJhEP4CNgB3kZKMNg4j6gMUh0OjVvtyAEjh7cGa
tWwfJgXxhPYFMPewRbYLXl1+PfZfmPyQVQAEqOvdDWlj62GlQHwwJre6v6A+4M1lJaSQtMGAy8k5le
9r1aFzFDRN39thgo+GMoLXSXNW/k0HTtlZ65FBDLUqfeu25539GYrSXCBS+zYdXdg1swSaRYYskdw6
Q=
----==_mimepart_5a4822e6dde79_73ea3f976a148f345609af
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Closed #1926.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/aspnet/Home/issues/1926#event-1405208081
----==_mimepart_5a4822e6dde79_73ea3f976a148f345609af
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Closed #1926.
—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.

----==_mimepart_5a4822e6dde79_73ea3f976a148f345609af--


1.3.0

Specter desktop 1.3.0 got just released, @nolim1t can you tag it here?
@lukechilds @mayankchhabra That's something we can (hopefully) quickly add to 0.3.9, so we also do something useful there.

Already done.
I'll add you as maintainer for this project too @AaronDewes so you can tag things if things gets released during the late night in Asia.
I trust that you can handle this too.
Just have to add a tag and push it.
Bonus would be also to create a pr and update the documentation too for consistency

Anyway you can now write to this project too @AaronDewes .. I think @lukechilds can also tag stuff already on this too (and any projects in the lncm org too)

Please test the following. My initial tests of the image is ok. I've managed to even sign and broadcast a transaction.
https://github.com/getumbrel/umbrel/pull/689


[WIP] [Spec] Tx ID as Asset ID

Use the ID of the transaction that CREATEd the asset to refer to the asset.

[ ] Make code changes to use TX id to refer to Asset ID
[ ] Fix tests
[ ] Docs changes

Fixes #697, fixes #717. fixes #715, fixes #708.

Two possibilities:
1: polymorphic 'asset' key
{operation: CREATE, asset: {data: blah}}
{operation: TRANSFER, asset: {id: abc}}

2: separate keys
{operation: CREATE, asset: {blah}}
{operation: TRANSFER, asset_id: abc}

Interested to hear reasoning one way or the other.
But, from a logical point of view they are the same and it's not clear that we need the Asset or AssetLink classes anymore. Or at least, we won't once we remove the keywords from the Asset.

Current coverage is 96.42% (diff: 100%)

Merging #926 into master will increase coverage by <.01%

@@             master       #926   diff @@
==========================================
  Files            34         34
  Lines          1817       1819     +2
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1752       1754     +2
  Misses           65         65
  Partials          0          0

Powered by Codecov. Last update d8256d5...d807511

are we also considering squatting of the tx_id? ie: what happens if someone makes 2 separate CREATE txs with the same content? Will they yield the same tx_id or do we need to force the user to provide a UUID?

@diminator: See note on https://github.com/bigchaindb/bigchaindb/issues/697#issuecomment-267335527.
We basically decided the unique factor must either come in the metadata or asset, otherwise you should just be making divisible assets.

@libscott For the asset key, I think it's more elegant if we make it polymorphic. That way the root structure of a transaction stays the same between the two types.

LGTM


Added PieMenu

A library designed to allow users to incorporate radial widgets in their games.
The "scene2d standards" were respected whenever possible so as to allow the most fluid integration and exploration for users.
Moreover, a very extensive Wiki is provided, along with many examples showing off use-cases (with a GIF, an explanation, and the code).

Feedback from Discord says Pie Menu is a worthy addition to the awesome list!

Seems like the community approves it!
Before merging this addition I'd like to suggest an upgrade to the library's short description though.
The current description is:

PieMenu - Circular menus (aka PieMenu) and widgets within libGDX scene2d.ui.

But I think it could be a bit more straightforward and appealing, something like

PieMenu - Radial menus for Scene2D that are highly flexible and easy to customize.

Otherwise just removing the "(aka PieMenu)" should be enough, as it's redundant.


[Newcomer] Seash Scripts

It would be useful to have the ability to run pre-existing seash scripts.  For example, we could write something like this...
login.seash
loadkeys leonwlaw
as leonwlaw
browse
on browsegood

and then from within seash...
 !> script login.seash
 leonwlaw@browsegood !>

Or if we want to do some debugging...
 !> script login.seash verbose
 !> loadkeys leonwlaw
 !> as leonwlaw
 leonwlaw@ !> browse
 leonwlaw@ !> on browsegood
 leonwlaw@browsegood !>

In addition, maybe the module can also run a script specified directly from the command line.
leonwlaw > python seash.py --script login.seash [--verbose]
-- Factoid text --
Enabled modules: ....
 leonwlaw@browsegood !>

I'll give this a WONTFIX. A new seash-like experiment tool could make better use of an interactive Python prompt (see the interactive_repy_console doodle for a possible way to do this).
Also, you can always use the experiment library interactively.


Broken profile link

Subject: Broken profile link
Description:
Clicked this link:
https://couchers.org/user/akvashi24/about)
Results:
I assume it should have taken me a profile, but it gave me an error message.
Backend version: develop-d6739683
Frontend version:
User Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 16_1_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Mobile/15E148 Safari/604.1
Page: https://couchers.org/messages/request/38646
User: eliaszp (1610)

can't reproduce


[BUG] enum classes cannot be used in cdef class method declarations

Describe the bug
Cython generates invalid C++ code when an enum class is used as either the return type or a parameter of a method of a cdef class. The issue is that the __PYX_ENUM_CLASS_DECL macro is inlined too late in the file to be available at the time when it is first needed for a declaration.
Code to reproduce the behaviour:
# distutils: language = c++

cdef enum class my_enum(int):
    a = 1
    b = 0

cdef class A:
    cdef my_enum f(self, my_enum x):
        return x

When I attempt to compile this code, I get:
(main) dt08% cythonize -3 -i -f definitions.pyx
[1/1] Cythonizing /home/nfs/vyasr/local/testing/cython_enum_class_decl/definitions.pyx
/home/nfs/vyasr/local/testing/cython_enum_class_decl/definitions.cpp:1404:3: error: ‘__PYX_ENUM_CLASS_DECL’ does not name a type
 1404 |   __PYX_ENUM_CLASS_DECL __pyx_t_11definitions_my_enum (*f)(struct __pyx_obj_11definitions_A *, __PYX_ENUM_CLASS_DECL __pyx_t_11definitions_my_enum);
      |   ^~~~~~~~~~~~~~~~~~~~~
/home/nfs/vyasr/local/testing/cython_enum_class_decl/definitions.cpp: In function ‘int __Pyx_modinit_type_init_code()’:
/home/nfs/vyasr/local/testing/cython_enum_class_decl/definitions.cpp:3512:32: error: ‘struct __pyx_vtabstruct_11definitions_A’ has no member named ‘f’
 3512 |   __pyx_vtable_11definitions_A.f = (__PYX_ENUM_CLASS_DECL __pyx_t_11definitions_my_enum (*)(struct __pyx_obj_11definitions_A *, __PYX_ENUM_CLASS_DECL __pyx_t_11definitions_my_enum))__pyx_f_11definitions_1A_f;
      |                                ^
error: command '/usr/bin/gcc' failed with exit code 1

If I inspect definitions.cpp, the __PYX_ENUM_CLASS_DECL macro is in fact declared in the file, but it is declared after the first usage, causing this issue.
Expected behaviour
I expect this code to compile successfully.
OS
Linux
Python version
3.11.4
Cython version
3.0.0
Additional context
No response

CC @shwina


feature(email_notification):Send email notification

Trello: https://trello.com/c/IX07Qglw
If user will be mentioned in comments, the email notification
will be sent to mentioned user.

Besides some comments, the biggest problem are missing unit tests, making modifications in future more problematic/longer. But due tight couple to db, looks it is hard to unit test. @k0machi is it possible to abstract argus repository to some class that could be replaced by dummy db (e.g. in memory)?

In theory it should be possible if you patch ScyllaCluster.get method - this instantiates cql engine and cluster connection. The problem is the cqlengine itself, we need to patch its database interaction methods as they're entirely unexposed to the user, it seems.

Besides some comments, the biggest problem are missing unit tests, making modifications in future more problematic/longer. But due tight couple to db, looks it is hard to unit test. @k0machi is it possible to abstract argus repository to some class that could be replaced by dummy db (e.g. in memory)?

In theory it should be possible if you patch ScyllaCluster.get class method - this instantiates cql engine and cluster connection. The problem is the cqlengine itself, we need to patch its database interaction methods as they're entirely unexposed to the user, it seems.

Just for the future reference because it would require some major refactor here: for this kind of problems there's Repository Pattern. It's often useful to use it for working with databases.

Besides some comments, the biggest problem are missing unit tests, making modifications in future more problematic/longer. But due tight couple to db, looks it is hard to unit test. @k0machi is it possible to abstract argus repository to some class that could be replaced by dummy db (e.g. in memory)?

In theory it should be possible if you patch ScyllaCluster.get class method - this instantiates cql engine and cluster connection. The problem is the cqlengine itself, we need to patch its database interaction methods as they're entirely unexposed to the user, it seems.

Just for the future reference because it would require some major refactor here: for this kind of problems there's Repository Pattern. It's often useful to use it for working with databases.

In theory cqlengine is an implementation of a repository pattern, but it feels a bit simplistic compared to other ORMs I've encountered.

@k0machi please re-review and if approved, you can merge.


onBack not working on any scenes

Version
Tell us which versions you are using:

react-native-router-flux v4 - ^4.0.0-beta.23
react-native v0 0.49.3

Expected behaviour
onBack function should execute the code
Actual behaviour
onBack is not fired on any of the scenes I have
Steps to reproduce
My router code looks like this:
		<Router>
			<Scene key="root">
				<Scene key="setup">
						<Scene key="setupOptions" component={Setup} />
						<Scene key='healthIssue' component={SetupHealthIssue} onBack={ ()=> console.log('am in')}/>
				</Scene>
			</Scene>
		</Router>

The onBack is just firing, when I click that back arrow in top left on healthIssue page. I tried creating other scenes and calling onBack on them didnt work either.

Just tried that if i add  "back" or "backTitle" to this scene, it made it work. Strange, seems like onBack needs another attr to make it working.


Contiguous backend 2

Successfully integrated the backend with youtube comments. This is the framework that I will build off of in the future, but for now I need to start working on the front end.

resolved the nits but the push isn't showing up here. Gonna wait and see what happens


Loading failed for ... /voyager-assets?path=js/app.js ?

Version information

Laravel: v5.6.39
Voyager: v1.2.6
PHP: 7.2.14my
Database: MySQL Server version 5.7.25

Description
I was upgrading from v1.1.10 to v1.2.6 and now the page is not loading (the wheel spins forever).
Steps To Reproduce
Steps to reproduce the behavior:

Upgrading from v1.1.10 to v1.2.6
Removed public/vendor/tcg/voyger folder
Removed assets_pathfrom config/voyger.php
Add controllers  php artisan voyager:controllers
Modify  App/Http/Controllers/Voyager/VoyagerController.php

  public function index()
  {
    return view('admin.index');
 }

Point to this blade:

@extends ('voyager::master')

@section('content')
Hallo
@endsection

Load Admin Page
See error

Expected behavior
Render page in finite time.
Screenshots
This is how it looks when I open http://member-nlp.test/admin. There is no menu loaded or anything, just a wheel spinning:

This is the shown by the inspector, showing that js/app.js could not be loaded (i have not mentioned that file in config/voyager.php:

Additional context
When I change the index method in App/Http/Controllers/Voyager/VoyagerController.php like this:
public function index()
{
  return 1;
}

Then  http://member-nlp.test/admin will return sucessfuly 1.

The path to the assets contains %3Fv%3D2 which translates to ?v=2 which the controller can't find.
Where does it come from?

Oh true, I forgot that I overwrote  the master.blade by /resource/views/vendor/voyager/master.blade. Removed the ?v=2 and now its working. Thanks a lot!!!


[Fix typo in sdk\iothub\azure-mgmt-iotcentral\CHANGELOG.md] Completly -> Completely

Description
I fixed typo in sdk\iothub\azure-mgmt-iotcentral\CHANGELOG.md
It seems there's a slight format change probably due to VSCode's prettier. However, there's no issue with the performance.
Completly -> Completely
Maybe
All SDK Contribution checklist:

[X] The pull request does not introduce [breaking changes]
[X] CHANGELOG is updated for new features, bug fixes or other significant changes.
[X] I have read the contribution guidelines.

General Guidelines and Best Practices

[X] Title of the pull request is clear and informative.
[X] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[X] Pull request includes test coverage for the included changes.

Thanks for your contribution.
This is auto-generated code and we don't modify them manually.


Run examples with PyTest

We should include running examples with pytest, not only to ensure that we are providing working examples, but for right now they’re reasonably effective as tests while we create actual unit tests...

Definitely.
We need both unit tests and 'demos'.
Tests are machines (automated testing etc)...
Demos are for humans...

I agree with @royerloic that we need both unit tests and examples/demos and as far as I understand you want to make people run examples with pytest too? Correct me if i misunderstood you.

For now, we should focus on automated non-UI unit tests.
Later, once we converge on a UI, it will be worth adding
unit tests for the UI. But for now I think it is overkill given how
young this project is. UI testing is potentially very complex to
do well and woulkd consume a disproportionate amount of time
to implement. In any case it is too early.
Eventually, we could make use of the following tools,
some of which are pytest based:
https://wiki.python.org/moin/PyQt/GUI_Testing

@royerloic sure, I totally agree. However we should make sure that each PR is properly tested. Perhaps the PR author can make a "script" (as in, a human-readable set of instructions) that exercises all the new functionality, so that reviewers can properly test the PR.

With clearvolume we had lots of 'demos' and each demo had a description of what people should expect to see. Once the project matures and we have more contributors we can start thinking about automating that...

With #176 we have added support for automatically running examples with pytest.
Complex UI testing has not been implemented yet, but could come in the future


Support logical diff/merge for Iceberg tables

Iceberg stores metadata files ("manifests") that represent a snapshot of a given Iceberg table.
Currently, when trying to modify an Iceberg table from 2 different branches, lakeFS would not be able to create a logical snapshot that consists of the changes applied in both references.
Users would then have to forgo one of the change sets, by either retaining the destination's branch set of changes, or the source's branch.
A much better user experience would be to allow merging snapshots into a new unified set of changes, representing changes made in both refs as a new snapshot.

Milestones

Support diff, and single-writer merges.
Multi-writer merges.

@johnnyaug @ozkatz addition of demos is super important as well


[Intel MKL] Adding support for DNNL1 builds to public CI.

@penpornk This is also not required for r2.2

Got it. Thanks for the feedback, @penpornk!


Release build of library contains unnecessary runtime dependencies

sonar-maven-plugin
jacoco-maven-plugin
com.google.api-client
logback-classic
2 http clients instead of 1 or even 0:
apache http client and okhttp

Hi,
Thanks for your feedback.
If i'm not wrong:

com.google.api-client is used here
logback is used all over the project through annotations @Slf4j and set up here
And sonar & jacoco plugin are used in order to build this dashboard

Tell me if i'm not correct.

The general idea is that you want to minimize dependencies as much as possible. To avoid jar hell and other problems.
Google api
yes, there are usages, but use are using only a couple of utility function bringing in the whole library  with all it's transitive dependencies. this is 100% wrong approach.
Should either:

Find the same functionality in another library(okhttp, apache httpclient or it should be available in jdk).
Copy only that logic to the project from google api lib

Logback
logback is logback. slf4j is slf4j. logback is one of the implementations if slf4j. So is you are using slf4j in the code (which is generated by lombok) need you to set slf4j as dependency for the library and not logback. if you want to see logs during tests you set logaback as test only dependency.  So if someone uses you library and different slf4j implementation he doesn't have extra runtime dependencies... or again jar-hell.
sonar & jacoco
These 2 libraries are not either runtime dependencies nor compile time. These are different things. jacoco is using only during test execution (thus you need to change scope to test). The same thing for sonar. if you want some kind od special build - create a maven profile to make a build for your needs. but make default build you deploy to public maven repo as clean(lesser dependencies) as possible.
The same idea with okhttp and apache http client. both libraries have the same functionality. if you are using only utility function from one of libraries, copy them to your code of try to find in another library which you already have imported.

Good points, thanks for feedback we'll watch it !

I think logback-classic pulls up SLF4J, and is required as an SLF4J implementation.
SLF4J is simply used to register logger API implementation, isn't it?
Hence, you'd need logback-classic when using @Slf4j, right?

logback-classic is not required neither during compile time (only slf4j is required), nor runtime (another implementation of slf4j can be used instead)

But you'll need one of them at runtime, right?
You mean that the application integrating the library will provide its own implementation?
What happens if they do not use an SLF4J implementation?
I think it's fine to provide a default SLF4J implementation (logback or log4j or even JUL) and eventually tell the integrator that they can exclude it in their POM to use their own implementation.
Don't you think it's poor design to rely on the runtime injection of a dependency from the JAR integrator?


Added Nutanix test cases for reboot cycles, fixed issues

Added Nutanix test cases for reboot cycles, and fixed some issues

LGTM.


unet.conv_in.weight[:, 3:] = torch.zeros(unet.conv_in.weight[:, 3:].shape) I assume 3 should be 4

def get_unet(pretrained_model_name_or_path, revision, resolution=256, n_poses=5):
# Load pretrained UNet layers
unet = UNet2DConditionModel.from_pretrained(
"CompVis/stable-diffusion-v1-4",
subfolder="unet",
revision="ebb811dd71cdc38a204ecbdd6ac5d580f529fd8c"
)
Modify input layer to have 1 additional input channels (pose)
weights = unet.conv_in.weight.clone()
unet.conv_in = nn.Conv2d(4 + 2*n_poses, weights.shape[0], kernel_size=3, padding=(1, 1)) # input noise + n poses
with torch.no_grad():
    unet.conv_in.weight[:, :4] = weights # original weights
    unet.conv_in.weight[:, 3:] = torch.zeros(unet.conv_in.weight[:, 3:].shape) # new weights initialized to zero

the original shape of conv_in  is 320, 4. so unet.conv_in.weight[:, 3:] = torch.zeros(unet.conv_in.weight[:, 3:].shape) would make the weights of dimension 3 become 0

yes, i also notice this point. may be [:, 4:] is more resonable

@dlutzzw 这姑娘比较粗心 但demo结果到底是怎么得到的？

Thanks for the feedback, this bug is now fixed.


Firebase object + ngx-datatable

I'm submitting a ...  (check one with "x")
[x] bug report => search github for a similar issue or PR before submitting
[x] feature request
[ ] support request => Please do not submit support request here

Current behavior

When assigning the "rows" option to a firebase object (or list) observable, I get "Cannot read property 'toLocaleString' of undefined"
Expected behavior

Populate the table with the object returned from firebase.
What is the motivation / use case for changing the behavior?

To be able to have a real-time table linked with Firebase, and to just Firebase as the primary database.

Table version: Lastest

Angular version: 2.4.8, angular-cli: 1.0.0-beta.24

Browser: All

Language: All

I actually figured it out. I was thinking that I should use ngFor in the ngx-datatable element, but I just had the completely wrong way of thinking. To clarify in case anyone stumbles upon this, this is what I did:

Where 'listguests' is the array I use to list the rows.


Create DSPA Status Condition fields

This PR will introduce DataSciencePipelinesApplication CR Status fields.
Description
Data Science Pipelines is not an operator that deploys DSP stacks in multiple namespaces. In order to get a complete observation of the stack health, conditions must be implemented.
In this implementation, the following conditions were implemented:

DatabaseReady
ObjectStorageReady
ApiServerReady
PersistenceAgentReady
ScheduledWorkflowReady
UserInterfaceReady
Ready

Each of these conditions check if the Available Deployment condition to verify if the component is ready to use. The last condition is a general condition based on the previous conditions and the way the user created the DSP stack (with external resources or without the UI)
How Has This Been Tested?
Simply follow the instructions in the readme provided to deploy and test this PR. You will have to substitute the kfdef repo to point to this PR, or use the following if you'd like to use my fork:
apiVersion: kfdef.apps.kubeflow.org/v1
kind: KfDef
metadata:
   name: data-science-pipelines-operator
   namespace: ${DSPO_NS}
spec:
   applications:
      - kustomizeConfig:
           repoRef:
              name: manifests
              path: data-science-pipelines-operator/
        name: data-science-pipelines-operator
   repos:
      - name: manifests
        uri: "https://github.com/rimolive/odh-manifests/tarball/rmartine"

Run the following command to check if conditions were created:
$ oc get datasciencepipelinesapplication example -o jsonpath='{.status.conditions}'

Check in the output if all status fields have the value True
Merge criteria:

[X] The commits are squashed in a cohesive manner and have meaningful messages.
[X] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[X] The developer has manually tested the changes and verified that the changes work

/hold

/unhold

/retest ci/prow/dsp-operator-e2e

/test dsp-operator-e2e

/test dsp-operator-e2e

/test dsp-operetor-e2e

Hm, all my conditions are false. I'm deploying via make deploy and not kfdef. Any pointers?

@gregsheremeta I think this pr is back into progress

I think this pr is back into progress

ok, thanks for letting me know. I dug into it a bit and there does appear to be a logic problem with getting status from False back to True
Inline comments are mine:
	databaseReady := r.buildCondition(config.DatabaseReady, dspa, "DatabaseReady")

	// GS: buildCondition pulls an existing condition from the cluster first if it exists. If it happens to have a status of False on cluster, databaseReady.Status is now False in code here

	// GS: let's assume no error, skip this
	err = r.ReconcileDatabase(ctx, dspa, params)
	if err != nil {
		databaseReady.Status = metav1.ConditionFalse
		databaseReady.Reason = config.DataBaseFailedToDeploy
		databaseReady.LastTransitionTime = metav1.Now()
		return ctrl.Result{}, err
	}

	// GS: deployment not available, status should be False for that case, ok
	if !r.isDeploymentAvailable(ctx, dspa, "mariadb") {
		databaseReady.Status = metav1.ConditionFalse
		databaseReady.LastTransitionTime = metav1.Now()
	}

	// GS: but what about the else there -- deployment is available. We're not setting it to True though?

	conditions = append(conditions, databaseReady)

/retest dsp-operator-e2e

/test dsp-operator-e2e

/test dsp-operator-e2e

/test dsp-operator-e2e

/test dsp-operator-e2e

/test dsp-operator-e2e

/retest

/test dsp-operator-e2e

On operator startup I see:
2023-04-06T12:49:53-04:00       ERROR   Reconciler error        {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelines
Application": {"name":"sample","namespace":"data-science-project"}, "namespace": "data-science-project", "name": "sample", "reconcileID": "a8aede36-9036-4092-81a9-897012d8ae65", "error": "Operation cannot be fulfilled on datasciencepipelinesapplications.data
sciencepipelinesapplications.opendatahub.io \"sample\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /home/hukhan/go/1.19.5/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:326
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /home/hukhan/go/1.19.5/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /home/hukhan/go/1.19.5/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:234

Have you encountered this? Is this avoidable? I don't think we should be seeing errors at this stage.

I built a fresh cluster. I did not install ODH or RHODS. I checked out this PR, built it, and deployed it to my fresh cluster. I created a DSPA using dspa_simple.yaml from the config samples in the repo.
My status block is empty.
Logs:
2023-04-10T19:02:50Z	DEBUG	DataSciencePipelinesApplication Reconciler called.	{"namespace": "dsp-1"}
2023-04-10T19:02:50Z	INFO	Applying mariaDB resources.
2023-04-10T19:02:50Z	INFO	Finished applying Database Resources
2023-04-10T19:02:50Z	INFO	Applying object storage resources.
2023-04-10T19:02:50Z	INFO	Finished applying storage Resources
2023-04-10T19:02:50Z	INFO	Applying Common Resources
2023-04-10T19:02:50Z	INFO	Applying Common Resources
2023-04-10T19:02:50Z	INFO	Finished applying Common Resources
2023-04-10T19:02:50Z	INFO	Applying APIServer Resources
2023-04-10T19:02:50Z	INFO	Finished applying APIServer Resources
2023-04-10T19:02:50Z	INFO	Applying PersistenceAgent Resources
2023-04-10T19:02:50Z	INFO	Finished applying PersistenceAgent Resources
2023-04-10T19:02:50Z	INFO	Applying ScheduledWorkflow Resources
2023-04-10T19:02:50Z	INFO	Finished applying ScheduledWorkflow Resources
2023-04-10T19:02:50Z	INFO	Applying MlPipelineUI Resources
2023-04-10T19:02:50Z	INFO	Applying MlPipelineUI Resources
2023-04-10T19:02:50Z	INFO	Skipping Application of ViewerCRD Resources
2023-04-10T19:02:50Z	INFO	Updating CR status	{"namespace": "dsp-1"}

CR:
$ k get dspa dsp-1 -n dsp-1 -o yaml
apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1
kind: DataSciencePipelinesApplication
SNIP
  name: dsp-1
  namespace: dsp-1
  resourceVersion: "102129"
  uid: 8f2b4cd2-a5b8-4db5-a7f7-06c90665f929
spec:
  apiServer:
    applyTektonCustomResource: true
    archiveLogs: false
    autoUpdatePipelineDefaultVersion: true
    collectMetrics: true
    dbConfigConMaxLifetimeSec: 120
    deploy: true
    enableOauth: true
    injectDefaultScript: true
    stripEOF: true
    terminateStatus: Cancelled
    trackArtifacts: true
  database:
    mariaDB:
      deploy: true
      pipelineDBName: mlpipeline
      pvcSize: 10Gi
      username: mlpipeline
  mlpipelineUI:
    deploy: true
    image: quay.io/opendatahub/odh-ml-pipelines-frontend-container:beta-ui
  objectStorage:
    minio:
      bucket: mlpipeline
      deploy: true
      image: quay.io/opendatahub/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance
      pvcSize: 10Gi
  persistenceAgent:
    deploy: true
    numWorkers: 2
  scheduledWorkflow:
    cronScheduleTimezone: UTC
    deploy: true
  viewerCRD:
    deploy: false
    maxNumViewer: 50
status: {}

DSPA stack pods are up:
k get pods
NAME                                                   READY   STATUS    RESTARTS       AGE
ds-pipeline-dsp-1-cdc668c96-jmfrk                      2/2     Running   3 (6m7s ago)   6m31s
ds-pipeline-persistenceagent-dsp-1-77dff9c5bc-65x5t    1/1     Running   0              6m31s
ds-pipeline-scheduledworkflow-dsp-1-6d8db5649b-67nrx   1/1     Running   0              6m31s
ds-pipeline-ui-dsp-1-9f558df58-5hn94                   2/2     Running   0              6m30s
mariadb-dsp-1-ff94c587c-nnj5v                          1/1     Running   0              6m31s
minio-dsp-1-76d9f5f9dc-z7q8t                           1/1     Running   0              6m31s

So I think something's not working right. Let me know if you want me to dig into this more deeply.

/retest

Confirming that your last edit did the trick and I now see statuses correctly :)

/test all

/test dsp-operator-e2e

/lgtm

/test dsp-operator-e2e


Project management menu item display based on permissions

Describe the task
If user doesn't have projects.view permission and neither clients.view permission, the project management tab shouldn't show up at all. Right now it shows blank dropdown.

Expected behavior
The project management tab shouldn't show up if both projects.view and clients.view permissions are not given.

Closing.


Token EOF expected

Hi,
When I use this code, I get a token EOF expected error on the first line of my document.
I could not really find an issue which could cause this error.
Does anyone have any ideas?

Hey!
I'm not able to see any code. Would you mind sharing the steps to repro the issue?

Hi,
I tried using the the NavigationTable.query part in the samples/NavigationTable folder.
Do I have to use more code then the NavigationTable.query.pq and NavigationTable.pq to get that code working, or to get rid of that error?

Hey!
I’d recommend checking out this article that explains more how to implement the navigation table:
https://github.com/microsoft/DataConnectors/blob/master/docs/nav-tables.md

Okay thank you for your help, it helped a lot already!


fix: #509 CLI dev should work with injected environment variables

Closes #509
✅ Checklist

[x] I have followed every step in the contributing guide
[x] The PR title follows the convention.
[x] I ran and tested the code works

Thanks for the effort on this but I've accepted another PR for this feature so will be closing this now 🫡


Add database structure diagram

Database Structure Drawio for documentation

first letter should be Capital of PR and Commits (fixed)


Postgre SCRAM Authentication not possible because .jre driver

Which version and edition of Flyway are you using?
6.0.8 Community
If this is not the latest version, can you reproduce the issue with the latest one as well?
(Many bugs are fixed in newer releases and upgrading will often resolve the issue)
Yes, with the latest as well
Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)
Command-line (via the docker image, but that is not relevant)
Which database are you using (type & version)?
Postgresql 12.0
Which operating system are you using?
Ubuntu 18.04
What did you do?
(Please include the content causing the issue, any relevant configuration settings, the SQL statement that failed (if relevant) and the command you ran.)
Cofigured postgres server with
postgresql.conf:
password_encryption = 'scram-sha-256'

What did you expect to see?
A normal execution of migrations like
Database: jdbc:postgresql://host/db (PostgreSQL 12.0)
Successfully validated 11 migrations (execution time 00:00.022s)

What did you see instead?
Flyway Community Edition 6.0.8 by Redgate
ERROR:
Unable to obtain connection from database (jdbc:postgresql://host/db) for user 'my-user': SCRAM authentication is not supported by this driver. You need JDK >= 8 and pgjdbc >= 42.2.0 (not ".jre" versions)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 08004
Error Code : 0
Message    : SCRAM authentication is not supported by this driver. You need JDK >= 8 and pgjdbc >= 42.2.0 (not ".jre" versions)

The reason for the bug is because the postgres driver in the flyway package is the "jre6" (postgresql-42.2.8.jre6.jar) version of the driver which does not support SCRAM authentication.
It works when I change the postgresql-42.2.8.jre6.jar to the "normal" version of the driver: postgresql-42.2.8.jar

I'm having difficulty reproducing this: both FLyway and SQuirreL seem to connect correctly with the 42.2.8.jre6 driver - the only difference apparently being that I've set SCRAM up in a createDatabase script:
ALTER SYSTEM SET password_encryption = 'scram-sha-256'; CREATE ROLE julia LOGIN ENCRYPTED PASSWORD 'redacted'; CREATE DATABASE flyway_db WITH OWNER = julia ENCODING = 'UTF8' TABLESPACE = pg_default;

Did you reload the server after changing the setting? Maybe it did not come into effect?
Can you reproduce it via setting it in the config file (Probably you don't want to in order to set up an automated test) ?

Running SHOW password_encryption via Squirrel returns scram-sha-256, so I think the setting has taken effect. Strange, will keep on looking!

Putting in 7.x for now as changing the driver shipped with the command line may cause problems for Java 7 users, which we're committed to supporting in 6.x. I'll update the documentation site to indicate the workaround.

Hello. Any news from that issue? I also have an issue with postgres driver  now.

You should be able to drop the newer Postgres driver in manually (into the drivers folder in Flyway) and it will just work; alternatively Flyway 7 will ship with the latest driver this week.

This feature is now available in the V7 beta which you can read more about here.

I like to say thank you and all your team, thank you very much for your support.


Multi-Tenant Caching

Summary of the issue
In reviewing the cache key in https://github.com/VahidN/EFCoreSecondLevelCacheInterceptor/issues/139, I don't see a way to setup caching per Tenant on a multi-tenant application.  Currently I configure my connection string per tenant in the OnConfiguring of my context by injecting a configuration interface to determine the correct database/connection string.  Is there any way to make the cachekey tenant independant?
      protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            if (!optionsBuilder.IsConfigured)
            {
                optionsBuilder.UseNpgsql(_configuration.GetValue<string>("DbContextConnectionString"));
                optionsBuilder.AddInterceptors(_serviceProvider.GetRequiredService<SecondLevelCacheInterceptor>());
            }
        }

Tenant independent or dependent? I can change the getCacheKey method to consider/add the command.Connection.ConnectionString to the list of its builder, if you want.

Tenant independent or dependent? I can change the getCacheKey method to consider/add the command.Connection.ConnectionString to the list of its builder, if you want.

Adding the ConnectionString to the cachekey would do the trick.  Thank you for the consideration.

Added it via#https://github.com/VahidN/EFCoreSecondLevelCacheInterceptor/commit/5b454f14cfb0070e57a2b602a7d19f382186b91c


LOOP error on LINK

[ ] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
when i access the follow link "https://material-ui.com/pt/api/tipografia/" an error LOOP appear

How did you end-up on this URL? I can't reproduce the loop.

That only happens when i try to access in Pt-BR language

well now when i try again only appears a 404 not found

happen when i go to api of Typography on PT-BR first i receive the loop now just a 404.

Good, the loop is coming from the cached page, the 404 is coming from Crowdin that forces two keys to be duplicate when they are not: https://github.com/mui-org/material-ui/pull/23719/files#diff-3fa58c06be457637b39c5b9b881631938e55d20446c5e4fd10a871606b06c7cf.


verse number is missing in the translation pane

its done now

Verse number is coming for every chapter but it is unstable. Working on to fix it.

need to change the code with the mobx.

Code is altered as per MobX.


Add dep

@faisaltheparttimecoder @Tylarb let's try to squash/rebase commits in the pull requests before merging so that stuff like 1937ffd is not in the git history log.


[request] can you go a little further with Rx implementation without dagger

Recently I started reading a lot about MVP and I want to get into practicing my projects with it.
However I am not able to correctly understand where should Rx + Retrofit code go? I think it should be in Model Layer via Interactors but still can you help me understand?
Also what happens with the RX callback? the onNext(), onCompleted() and onFailure() passes data back to Presenter or do we implement listeners? Also should we decouple Rx callbacks further?

I don't know enough about Rx, so unfortunately no.

@riinav chekout https://github.com/jpotts18/android-mvp/tree/master-2


拼写错误

https://github.com/kizuna-ai-egg/kizuna-ai-mb/blob/main/src/main.py 第16行

os.environ['USERNMAE'] = 'UNCLE-LV'

应为

os.environ['USERNAME'] = 'UNCLE-LV'

It doesn't work without admin permissions on Windows11. So I have removed it. If you want to get an environment variable on Windows11, please run it with admin permissions.


Change profile page according to the User Journey Demo wiki page

[ ] Add text at the top of the page "If you have just created an account please allow up to 5 minutes for your info to show up in the system"

Ideally the backend will return a particular status if we are waiting for information, at which point a message like this would be displayed.


[QoL] Fixed annoying TypeDoc errors that have been a distraction to PRs

Fixed annoying TypeDoc errors that have been a distraction to PRs
Also turned on fail-on-error for TypeDoc generation during CI so TypeDoc warnings only annoys the person who introduces it.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.81%.  Consider uploading reports for the commit e391ac0 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #506   +/-   ##
=======================================
  Coverage   90.80%   90.81%
=======================================
  Files         116      116
  Lines       29409    29421   +12
  Branches     2156     2156
=======================================
+ Hits        26706    26718   +12
  Misses       2668     2668
  Partials       35       35

Components
Coverage Δ

agent
79.90% <ø> (ø)

api
97.91% <100.00%> (+<0.01%)
:arrow_up:

common
98.68% <ø> (ø)

credentials
95.26% <100.00%> (+<0.01%)
:arrow_up:

crypto
93.81% <ø> (ø)

dids
97.65% <ø> (ø)

identity-agent
96.70% <ø> (ø)

crypto-aws-kms
100.00% <100.00%> (ø)

proxy-agent
96.70% <ø> (ø)

user-agent
96.70% <ø> (ø)

The typedocs message is still gigantic. Any reason it's doing this, can we fix?

@web5/api
Project entry file: packages/api/src/index.ts
@web5/crypto
Project entry file: packages/crypto/src/index.ts
@web5/crypto-aws-kms
Project entry file: packages/crypto-aws-kms/src/index.ts
@web5/dids
Project entry file: packages/dids/src/index.ts
@web5/credentials
Project entry file: packages/credentials/src/index.ts

@thehenrytsai don't forget to add a changeset to this PR.
https://github.com/TBD54566975/web5-js/blob/main/CONTRIBUTING.md#project-versioning-guidelines


Add ability to download file output snapshot of running jobs

Or, even better (ask for the moon), to stream specific files or mount the running output directory  :-)

The ability to see the contents of arbitrary files may actually be possible, will need to do some tests...

This feature would bring more benefit for us than any other in the current issue list.
Alternatively, if there was prominence exec / condor_ssh_to_job it could solve this and any other issues I could think of for interacting with running jobs.

@fcasson ok, I can probably start looking into this today.

Have successfully got condor_ssh_to_job to work. Needed to add openssh-clients to the PROMINENCE machine running HTCondor, and openssh-server to the worker.
Next steps:

[ ] Add exec route to the REST API for running a single command in the job
[ ] Add exec command to the CLI

Note that prominence exec will just run a single command and return any standard output/error. Getting prominence exec to open up a shell which the user can type commands into is more difficult.

Update to REST API tracked in https://github.com/prominence-eosc/prominence/issues/93

Update to CLI tracked in https://github.com/prominence-eosc/cli/issues/9

REST API updates implemented in https://github.com/prominence-eosc/prominence/commit/33e0d5cf6e997493c968e92a7103e91e5ced3416 https://github.com/prominence-eosc/prominence/commit/dcb7549c9cb956a809d9fc31cfb812254a5d91d1 https://github.com/prominence-eosc/prominence/commit/fff9c945deea5c03caba32a8b4e3537ba42fdf7b https://github.com/prominence-eosc/prominence/commit/c03cd04707817b92f97d6c11c078045c3119d6db https://github.com/prominence-eosc/prominence/commit/f63678d52b67913ccc74a3374a85ca744d36e62e https://github.com/prominence-eosc/prominence/commit/3a34988483941f37e098a43b7d22e93c1cef2f6f https://github.com/prominence-eosc/prominence/commit/f1261137dcd008ba8e786e70a11985f3aa9490b1

@fcasson with the CLI 0.1.24 or above you can use prominence exec. Note that this is not really suitable for downloading large files but is useful for checking if a job is working correctly or not. I'll still consider other options to allow access to large output files if necessary.

Actually can you clarify how one would use prominence exec to download a file?

It's not really a mechanism for downloading files - it's meant to help with checking the status of a job and/or debugging etc, e.g. tail an output file or log file to see if things look sensible, do a "ps" to see what processes are running, do an "ls" to see what files have been generated, ...
To look at an output file, for a JINTRAC job you could do something like:
prominence exec <job id> "cat /work/<filename>"

and the contents of the file would appear in stdout, just like if you'd run cat locally.
Or for https://github.com/prominence-eosc/prominence/issues/81 (i.e. stopping a job by creating a special file) you could do this:
prominence exec <job id> "touch /work/<special filename>"

Maybe this isn't quite what you want...
Alternatives:

prominence snapshot? Specify a file and it will be uploaded to ECHO then downloaded locally. Would setup a short lifetime on such files in ECHO (e.g. they're deleted automatically after 1 day or even less). I think I know how to do most of this, but it's not implemented.

There are options to mount volumes from B2DROP (https://b2drop.eudat.eu) or OneData (https://onedata.org). in this case in real time you could look at what your job is doing (in terms of files being written), e.g. in a web browser or on your desktop/laptop. The file read/write performance would be reduced, but this may not be a problem. The main problem with this is that we only have 20 GB each on B2DROP (everyone gets 20 GB) and we don't have access to OneData storage. This may change in future, but not in the short/medium term.

I get the impression ideally you want someting like 2 above.

Hi Andrew
Yes, prominence exec is perfect for debugging and stopping the job.  I think we can probably even use it for catting individual binary files and piping into a local file (but this is a bit inelegant).
For snapshots, either 1. or 2. is possible - Not individual files really, but the whole run directory. Without thinking carefully about pros and cons, I like 1., because it is simple and generates no extra dependences for mounting / examining our data.  Our rundirs (for jetto at least) are smallish (as you can see from the completed jobs) so I don't think the disk space is a big issue, and for us it would be fine to overwrite the previous snapshot.

The simplest (& fastest) would be 1. with the assumption that there is only ever one snapshot, i.e. if you create a new snapshot any existing snapshots are overwritten (this means I only need to create & pass a single pre-signed PUT URL to each job to be used for uploading any potential snapshots to ECHO).
Probably something like:
prominence snapshot /work

so that users can pick what is snapshotted (in order to make it useful more generally).

The simple and fast option meets our need.

Now implementend, both in the server and CLI. Users will need to update the CLI to the latest version (0.2.0).
Example usage:
$ prominence snapshot 25374 /work
Downloading file "snapshot.tgz"
[==================================================]

Note that the snapshot tarball is always called snapshot.tgz.
You can specify a file or directory when running prominence snapshot, and in each case a tarball called snapshot.tgz is created.
FYI the tarball is created from outside the container, but if you specify an absolute path (e.g. work above) it will be mapped to the appropriate path where appropriate. Random absolute paths (e.g. /, /usr/local, ...) will give an error.
Note that from the CLI you currently create a snapshot and download it at the same time, but this corresponds to 3 REST API requests:

PUT to the PROMINENCE API to create the snapshot
GET to the PROMINENCE API to get the URL to the snapshot
GET to ECHO, using the previously obtained URL

So if necessary it's possible to create a snapshot and download it later. Currently snapshots have a lifetime of 2 days.

Thanks!  This feature works perfectly and is a gamechanger for JINTRAC.


Visitors SBAT see Privacy page in same style/layout as FAQ

Expected Behavior
Keep Visual consistency and put titles on side. and use our new fonts
Current Behavior
Old layout / typography

figma link

figma link


Outreach and training for using cloud.gov effectively

What we're after: There's a set of materials to help people understand how to use cloud.gov effectively and it's been tested at least once
Benefits/why:

Help non-expert teams become productive with cloud.gov without undue team overhead
Prepare for regularly-scheduled outreach or paid training engagements

Want to do at least one internal and one external training. Doesn't have to be perfect (e.g. EPA, DigitalGov). Could measure with our support metrics.
Drupal/Wordpress examples could fall under this. More modular (smaller) could be good. Could include tips in release notes.
"Here's what you need to know about CF, here's cloud, here's app development in general" (?) General app development/architecture (lots of folks in gov't haven't been exposed to this)
Two CF courses we could adapt (could look at this during PI planning)

General CF
Architecture

Could be quarterly, eventually.

See notes from our discussion with #infrastructure
this morning... Holly would be a good person to reach out to.
On Tue, Apr 11, 2017 at 7:54 AM, Andrew Suprenant<EMAIL_ADDRESS>wrote:

Want to do at least one internal and one external training. Doesn't have
to be perfect (e.g. EPA, DigitalGov). Could measure with our support
metrics.
Drupal/Wordpress examples could fall under this. More modular (smaller)
could be good. Could include tips in release notes.
"Here's what you need to know about CF, here's cloud, here's app
development in general" (?) General app development/architecture (lots of
folks in gov't haven't been exposed to this)
Two CF courses we could adapt (could look at this during PI planning)

General CF
Architecture

Could be quarterly, eventually.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/18F/cg-product/issues/589#issuecomment-293289476, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC6krMJAPonpW1Lbi9sFmPv3pajMderks5ru5QmgaJpZM4LUTw9
.

Noting from Britta: Bernadette Adams (formerly in Learn, now in CPS) knows a lot about education and training and may be a resource when working on training materials

We're currently planning the workshop and will use those learnings to inform future work here.

Consider integrating what we've learned from support needs (specifically people coming to us with support questions that revealed much deeper architectural questions) to shape this feature

Superseded by [this new card](due to GitHub integration weirdness)


Non numeric value from json is failing for numeric model's property

https://github.com/Hearst-DD/ObjectMapper/issues/352#issuecomment-168196617
Thanks for reply :)
I will take care of it.
but still if i get nil user how to check where is the problem, which property type we need to change as per json response ?
As for single nil property, we cannot nullify the whole object. Except one property other should at least be populated. That exceptional property should get its default value in this case.

Usually if you get a nil object returned from ObjectMapper it is because there is a problem with your JSON input. Either it is not valid JSON or it cannot be parsed and cast into type [String : AnyObject].

Non numeric value from json is failing for numeric model's property

This is expected behaviour. ObjectMapper expects the JSON to be of the appropriate type to be parsed, otherwise it returns nil. You can always use a Transform (see README) if you need to map a non numeric value to a numeric model. For example "grade" : "100" contains a String value where you may really want an int. A transform can convert this during mapping.


fixes #98 : Added Fork me on GitHub

I added Fork Me ribbon from https://github.blog/2008-12-19-github-ribbons/

Closes #98


A small step towards removing C-style loops

This is a small step towards removing C-style loops in favor of Swift for/in loops as presented in this accepted proposal:https://github.com/apple/swift-evolution/pull/23/commits . Nothing big here, just getting my feet wet and trying help improve code quality :). Note that similar changes in swiftcore have already been merged https://github.com/apple/swift/pull/270

Thanks!


feat(web/applications): update no result text on documents and releve…

…nat representation pages
Describe your changes

Ticket https://pins-ds.atlassian.net/browse/ASB-1241

Useful information to review or test

Type of change 🧩

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update
[x] Other (please explain in the description section above)

Checklist before requesting a review

[x] I have performed a self-review of my own code
[x] I have double checked this work does not include any hardcoded secrets or passwords
[x] I have made corresponding changes to the documentation
[x] I have provided details on how I have tested my code
[x] I have referenced the ticket number above
[x] I have provided a description of the ticket
[x] I have included unit tests to cover any testable code changes

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.15.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Improve article 1710.10044

Exact location of issue
All references are replaced by ?

Also an error chain related to \float@count, \ch@ck, \float@count, \e@alloc@chardef as morefloats.sty is getting locally interpreted by latexml.


tf2.0 Multi-worker training with Keras  only utilizing one GPU

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow):yes
- OS  : Ubuntu 16.04LTS
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:No
- TensorFlow installed from (source or binary): Frome docker TF2.1.0-gpu-py3
- TensorFlow version (use command below): TF2.1.0-gpu-py3
- Python version: 3.6
- CUDA/cuDNN version:10.1 not sure cuDNN version
- docker18.09.7-3
- nvidia-container-runtime=2.0.0
- kubernetes 1.5.7
- kubeflow 1.01
- 1Master   IP:14X.XXX.XXX.1
- node1     IP:14X.XXX.XXX.8     GTX1060
- node2     IP:14X.XXX.XXX.9     GTX1060
- node3     IP:14X.XXX.XXX.10    GTX1070

Describe the current behavior
I use  Multi-worker training with Keras but it only use one Gpu
Error:
1.error: Internal: Complete shape not known for Adam/allreduce/CollectiveReduce_2
2.eval_fn is not passed in. The worker_fn will be used if an "evaluator" task exists in the cluster
3.Most import it only run one gpu
I run the code described below:
TEST 1: (3 machine)

TEST 2 : (2 machine)

Describe the expected behavior
Use Multi gpu
My Docker File
FROM tensorflow/tensorflow:2.1.0-gpu-py3
RUN apt-get update
RUN apt-get install -y libsm6 libxext6 libxrender-dev
RUN pip install opencv-python
RUN pip install Pillow
RUN mkdir -p /app
ADD tp720_1.py /app/
COPY nspo /app/

My yaml
apiVersion: kubeflow.org/v1
kind: TFJob
metadata:
  name: nspo-rice
spec:
  tfReplicaSpecs:
    Chief:
      replicas: 1
      template:
        metadata:
          annotations:
            sidecar.istio.io/inject: "false"
          name: tensorflow
        spec:
          containers:
          - command:
            - python
            - tp720_1.py
            image: nsporice:270_7
            name: tensorflow
            env:
            - name: test_tmpdir
              value: /app/data
            resourceas:
              limits:
                cpu: '1'
            volumeMounts:
            - mountPath: /app/data
              name: nspo-rice-volume
            workingDir: /app
          restartPolicy: Never
          volumes:
          - name: nspo-rice-volume
            persistentVolumeClaim:
              claimName: nspo-rice-volume
    Worker:
      replicas: 2
      template:
        metadata:
          annotations:
            sidecar.istio.io/inject: "false"
          name: tensorflow
        spec:
          containers:
          - command:
            - python
            - tp720_1.py
            image: nsporice:270_7
            name: tensorflow
            env:
            - name: test_tmpdir
              value: /app/data
            resourceas:
              limits:
                nvidia.com/gpu: 1
            volumeMounts:
            - mountPath: /app/data
              name: nspo-rice-volume
            workingDir: /app
          restartPolicy: Never
          volumes:
          - name: nspo-rice-volume
            persistentVolumeClaim:
              claimName: nspo-rice-volume

My code
from os import walk
from os.path import join
import numpy as np
import matplotlib.pyplot as plt
import cv2
import time
import random
from scipy import signal
import os
import json
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, GRU, LSTM, TimeDistributed, RepeatVector, Bidirectional
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import LeakyReLU
from tensorflow.keras import losses
from tensorflow.keras import optimizers
from tensorflow.keras import metrics
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
import tensorflow.keras.backend as K
from tensorflow.keras.models import load_model
class TimeHistory(tf.keras.callbacks.Callback):
    def on_train_begin(self, logs={}):
        self.losses = {'batch':[], 'epoch':[]}
        self.accuracy = {'batch':[], 'epoch':[]}
        self.val_loss = {'batch':[], 'epoch':[]}
        self.val_acc = {'batch':[], 'epoch':[]}
        self.times = []
        self.totaltime = time.time()

    def on_train_end(self, logs={}):
        self.totaltime = time.time() - self.totaltime

    def on_batch_end(self, batch, logs={}):
        self.losses['batch'].append(logs.get('loss'))
        self.accuracy['batch'].append(logs.get('acc'))
        self.val_loss['batch'].append(logs.get('val_loss'))
        self.val_acc['batch'].append(logs.get('val_acc'))

    def on_epoch_begin(self, batch, logs={}):
        self.epoch_time_start = time.time()

    def on_epoch_end(self, batch, logs={}):
        self.losses['epoch'].append(logs.get('loss'))

        self.accuracy['epoch'].append(logs.get('acc'))
        self.val_loss['epoch'].append(logs.get('val_loss'))
        self.val_acc['epoch'].append(logs.get('val_acc'))
        self.times.append(time.time() - self.epoch_time_start)

    def loss_plot(self, loss_type):
        acc =[]
        loss = []
        val = []
        iters = range(len(self.losses[loss_type]))
        # acc
        acc.extend(self.accuracy[loss_type])
        # loss
        print("loss = ",self.losses[loss_type])
        loss.extend(self.losses[loss_type])
        # val
        print("val = ",self.losses[loss_type])
        val.extend(self.val_acc[loss_type])
        return(acc, loss,val)
time_callback = TimeHistory()

def buildManyToOneModel(shape):

    model = tf.keras.models.Sequential([
        GRU(32, input_dim = shape[2], input_length = shape[1], return_sequences = True),
        GRU(64, return_sequences = True),
        GRU(128, return_sequences = False),
        #LSTM(16, return_sequences = True),
        #LSTM(16, return_sequences = False),
        BatchNormalization(),
        Dense(1, activation='sigmoid')
    ])
    model.compile(loss='mse', optimizer = 'adam', metrics=['acc'])
    model.summary()
    return model

def slice_(data, node_num):
    total = float(data.shape[0])
    store = []
    if node_num == 1:
        store.append(data[0:int(total),:])
        store.append([0])
        store.append([0])
    elif node_num == 2:
        slice_index = int(total / 2)
        store.append(data[0:slice_index, :])
        store.append(data[slice_index:int(total), :])
        store.append([0])
    elif node_num == 3:
        slice_index = int(total / 3)
        store.append(data[0:slice_index, :])
        store.append(data[slice_index:2*slice_index, :])
        store.append(data[2*slice_index:int(total), :])
    return store

def train():
    print("TensorFlow version: ", tf.__version__)
    tf_config = os.environ.get('TF_CONFIG', '{}')
    print("TF_CONFIG %s", tf_config)
    tf_config_json = json.loads(tf_config)
    cluster = tf_config_json.get('cluster')
    job_name = tf_config_json.get('task', {}).get('type')
    task_index = tf_config_json.get('task', {}).get('index')

    print("cluster={} job_name={} task_index={}}", cluster, job_name, task_index)

    gpus = tf.config.experimental.list_physical_devices('GPU')
    if gpus:
        try:
            for gpu in gpus:
                tf.config.experimental.set_memory_growth(gpu, True)
        except RuntimeError as e:
            print(e)
    strategy = tf.distribute.experimental.MultiWorkerMirroredStrategy(communication=tf.distribute.experimental.CollectiveCommunication.RING)
    print ('Number of devices: {}'.format(strategy.num_replicas_in_sync))

    data0 = np.load('/app/data/2017_360w_data_0.npy')
    data1 = np.load('/app/data/2017_360w_data_1.npy')
    data0 = data0[:,:,0]
    data1 = data1[:,:,0]
    print("data0:",data0.shape)
    print("data1:",data1.shape)
    a1 = np.array(data0)[0 : int(data0.shape[0]*0.7), :]
    a2 = np.array(data1)[0 : int(data1.shape[0]*0.7), :]
    a3 = np.array(data0)[int(data0.shape[0]*0.7) : data0.shape[0], :]
    a4 = np.array(data1)[int(data1.shape[0]*0.7) : data1.shape[0], :]
    X_train = np.concatenate((a1, a2), axis=0)
    X_val = np.concatenate((a3, a4), axis=0)

    b1 = np.zeros((a1.shape[0], 1))
    b2 = np.ones((a2.shape[0], 1))

    b3 = np.zeros((a3.shape[0], 1))

    b4 = np.ones((a4.shape[0], 1))

    Y_train = np.concatenate((b1, b2), axis=0)
    Y_val = np.concatenate((b3, b4), axis=0)
    X_train = X_train.astype(np.float32)
    Y_train = Y_train.astype(np.float32)
    X_val = X_val.astype(np.float32)
    Y_val = Y_val.astype(np.float32)
    X_train = X_train[:,:,np.newaxis]

    X_val = X_val[:,:,np.newaxis]
    print(X_train.shape)
    print(Y_train.shape)
    print(X_val.shape)
    print(Y_val.shape)
    print(type(X_train))
    print(type(Y_train))
    BUFFER_SIZE = X_train.shape[0]

    BATCH_SIZE_PER_REPLICA = 5000
    GLOBAL_BATCH_SIZE = BATCH_SIZE_PER_REPLICA  * (strategy.num_replicas_in_sync-1)

    #train_dist_dataset = strategy.experimental_distribute_dataset(train_dataset)
    #test_dist_dataset = strategy.experimental_distribute_dataset(test_dataset)

    if BUFFER_SIZE % GLOBAL_BATCH_SIZE != 0:
        parallel_steps =  X_train.shape[0] // GLOBAL_BATCH_SIZE + 1
        a =  X_val.shape[0] // GLOBAL_BATCH_SIZE + 1
    else:
        parallel_steps =  X_train.shape[0] // GLOBAL_BATCH_SIZE
        a =  X_val.shape[0] // GLOBAL_BATCH_SIZE
    print(parallel_steps)
    t2 = time.time()
    with strategy.scope():
         train_dataset = tf.data.Dataset.from_tensor_slices((X_train, Y_train)).repeat().shuffle(buffer_size=5000000).batch(GLOBAL_BATCH_SIZE)
         #test_dataset = tf.data.Dataset.from_tensor_slices((X_val, Y_val)).batch(GLOBAL_BATCH_SIZE)
         options = tf.data.Options()
         options.experimental_distribute.auto_shard_policy = \
                                        tf.data.experimental.AutoShardPolicy.DATA
         train_dataset = train_dataset.with_options(options)
         #test_dataset = test_dataset.with_options(options)
         multi_worker_model = buildManyToOneModel(X_train.shape)

    #history = multi_worker_model.fit(train_dataset, epochs=30, validation_data=test_dataset,steps_per_epoch=parallel_steps,validation_steps=a,callbacks=[time_callback])
    history = multi_worker_model.fit(train_dataset, epochs=30,steps_per_epoch=parallel_steps,callbacks=[time_callback])
    t3 = time.time()
    print ("It cost ", t3 - t2, " seconds")

    accuracy, loss,val = time_callback.loss_plot('epoch')
    print("totaltime:%.4f"%time_callback.totaltime)
    for i in range(len(accuracy)):
        print("acc: %.4f, loss: %.4f,val:%.4f -----epoch:%d" %(accuracy[i], loss[i],val[i],i+1))
    totaltime='%.2f'%time_callback.totaltime
    ttime= t3 - t2
    traningtime='%.2f'%ttime
    maxval='%.2f'%max(val)
    save_dir = '/app/data'
    f = open(save_dir+"/720_1.txt", "a")
    f.write("\ntotaltime:{},traningtime:{},val:{}".format(totaltime,traningtime,maxval))
    f.close()
if __name__ == '__main__':
    train()

Pod logs
2020-06-08 12:52:04.146322: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-06-08 12:52:04.147962: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
2020-06-08 12:52:04.788595: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-06-08 12:52:04.795356: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:05:00.0 name: GeForce GTX 1070 computeCapability: 6.1
coreClock: 1.7715GHz coreCount: 15 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 238.66GiB/s
2020-06-08 12:52:04.795467: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-06-08 12:52:04.795550: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-06-08 12:52:04.798357: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-06-08 12:52:04.799032: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-06-08 12:52:04.802641: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-06-08 12:52:04.804156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-06-08 12:52:04.804221: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-06-08 12:52:04.805601: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-06-08 12:52:04.806525: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-06-08 12:52:04.814745: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2020-06-08 12:52:04.815811: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5c54090 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-06-08 12:52:04.815832: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-06-08 12:52:04.933082: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5cb9890 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-06-08 12:52:04.933126: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1070, Compute Capability 6.1
2020-06-08 12:52:04.934468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:05:00.0 name: GeForce GTX 1070 computeCapability: 6.1
coreClock: 1.7715GHz coreCount: 15 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 238.66GiB/s
2020-06-08 12:52:04.934575: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-06-08 12:52:04.934620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-06-08 12:52:04.934658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-06-08 12:52:04.934694: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-06-08 12:52:04.934746: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-06-08 12:52:04.934794: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-06-08 12:52:04.934850: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-06-08 12:52:04.938735: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-06-08 12:52:04.938852: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-06-08 12:52:05.301356: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-06-08 12:52:05.301383: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-06-08 12:52:05.301391: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-06-08 12:52:05.302813: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6927 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:05:00.0, compute capability: 6.1)
2020-06-08 12:52:05.305432: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:05:00.0 name: GeForce GTX 1070 computeCapability: 6.1
coreClock: 1.7715GHz coreCount: 15 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 238.66GiB/s
2020-06-08 12:52:05.305510: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-06-08 12:52:05.305552: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-06-08 12:52:05.305590: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-06-08 12:52:05.305617: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-06-08 12:52:05.305649: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-06-08 12:52:05.305686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-06-08 12:52:05.305721: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-06-08 12:52:05.306950: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-06-08 12:52:05.306975: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-06-08 12:52:05.306983: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-06-08 12:52:05.306988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-06-08 12:52:05.308211: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:worker/replica:0/task:1/device:GPU:0 with 6927 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:05:00.0, compute capability: 6.1)
2020-06-08 12:52:05.314693: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:300] Initialize GrpcChannelCache for job chief -> {0 -> nspo-rice-chief-0.kubeflow.svc:2222}
2020-06-08 12:52:05.314716: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:300] Initialize GrpcChannelCache for job worker -> {0 -> nspo-rice-worker-0.kubeflow.svc:2222, 1 -> localhost:2222}
2020-06-08 12:52:05.315663: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:390] Started server with target: grpc://localhost:2222
WARNING:tensorflow:`eval_fn` is not passed in. The `worker_fn` will be used if an "evaluator" task exists in the cluster.
WARNING:tensorflow:`eval_strategy` is not passed in. No distribution strategy will be used for evaluation.
WARNING:tensorflow:ModelCheckpoint callback is not provided. Workers will need to restart training if any fails.
2020-06-08 12:52:16.277123: W tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.cc:440] error: Internal: Complete shape not known for Adam/allreduce/CollectiveReduce_2
2020-06-08 12:52:16.277150: W tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.cc:1056] error: Internal: Complete shape not known for Adam/allreduce/CollectiveReduce_2
2020-06-08 12:52:16.277227: E tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.cc:1073] ScopedAllocatorOptimizer: Internal: Complete shape not known for Adam/allreduce/CollectiveReduce_2
2020-06-08 12:52:16.277234: W tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.cc:846] error: Internal: Complete shape not known for Adam/allreduce/CollectiveReduce_2
2020-06-08 12:52:16.277886: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] scoped_allocator_optimizer failed: Internal: Complete shape not known for Adam/allreduce/CollectiveReduce_2
2020-06-08 12:52:16.672402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
TensorFlow version:  2.1.0
TF_CONFIG %s {"cluster":{"chief":["nspo-rice-chief-0.kubeflow.svc:2222"],"worker":["nspo-rice-worker-0.kubeflow.svc:2222","nspo-rice-worker-1.kubeflow.svc:2222"]},"task":{"type":"worker","index":1},"environment":"cloud"}
cluster={} job_name={} task_index={}} {'chief': ['nspo-rice-chief-0.kubeflow.svc:2222'], 'worker': ['nspo-rice-worker-0.kubeflow.svc:2222', 'nspo-rice-worker-1.kubeflow.svc:2222']} worker 1
Number of devices: 3
data0: (1760157, 14)
data1: (1839843, 14)
(2519999, 14, 1)
(2519999, 1)
(1080001, 14, 1)
(1080001, 1)
<class 'numpy.ndarray'>
<class 'numpy.ndarray'>
252
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
gru (GRU)                    (None, 14, 32)            3360
_________________________________________________________________
gru_1 (GRU)                  (None, 14, 64)            18816
_________________________________________________________________
gru_2 (GRU)                  (None, 128)               74496
_________________________________________________________________
batch_normalization (BatchNo (None, 128)               512
_________________________________________________________________
dense (Dense)                (None, 1)                 129
=================================================================
Total params: 97,313
Trainable params: 97,057
Non-trainable params: 256
_________________________________________________________________
Train for 252 steps
Epoch 1/30
252/252 [==============================] - 53s 209ms/step - loss: 0.1431 - acc: 0.8200
Epoch 2/30
252/252 [==============================] - 38s 149ms/step - loss: 0.0953 - acc: 0.8733
Epoch 3/30
252/252 [==============================] - 38s 149ms/step - loss: 0.0890 - acc: 0.8798

@LearnKen are you trying to train with multiple GPUs on one machine, or train across multiple machines?

Yes different meachine
this is my TF_CONFIG log
TF_CONFIG %s {"cluster":{"chief":["nspo-rice-chief-0.kubeflow.svc:2222"],"worker":["nspo-rice-worker-0.kubeflow.svc:2222","nspo-rice-worker-1.kubeflow.svc:2222"]},"task":{"type":"worker","index":1},"environment":"cloud"}
cluster={} job_name={} task_index={}} {'chief': ['nspo-rice-chief-0.kubeflow.svc:2222'], 'worker': ['nspo-rice-worker-0.kubeflow.svc:2222', 'nspo-rice-worker-1.kubeflow.svc:2222']} worker 1

Is that right? if not please help me how to do

@LearnKen
Is this still an issue.

@Saduf2019  YES

I don't see obvious sign that only one GPU is being utilized. It might be a matter of performance (MultiWorkerMirroredStrategy is still experimental and one may not get ideal performance out of the box). Can you try changing communication from RING to NCCL? Also, can you try a newer version of TF?

If you are able to get multiple GPUs on one machine, it is worth trying out MirroredStrategy and see what performance number we can get.


PartitionKey constructor for types

Ref: https://github.com/Azure/azure-cosmos-dotnet-v3/pull/348
Instead of using a constructor for PartitionKey as object, add one for each supported type.

I think this is necessary for usability. Partition key only support some primitive types and can't handle complex objects. The only way to figure this out currently is run time exceptions. I think this also causes confusion for new users because they don't know what a partition key is and what they should be passing in.

I've been using Cosmos DB for a while, and it's the first time I learn that there are only a few valid types for partition keys. Is this documented anywhere?

Not sure if it counts as covering all valid values, but the REST API documentation for "x-ms-documentdb-partitionkey" https://docs.microsoft.com/en-us/rest/api/cosmos-db/common-cosmosdb-rest-request-headers says it's an Array if valid values, and that same page seems to imply Number, Bool, and String are those.


Font Optimization

Credit to @jdlcdl for the recommendation:
https://github.com/marc3linho/OrangeClock/issues/28#issuecomment-1823550484
Used font_to_py "-c" argument to compile only the characters currently being used in the UI. Across all 4 fonts, was able to reduce the font sizes from about 280kb to 70kb. This has freed up a significant amount of memory, now >100kb free.
Note:
Recompiling orangeClockIcons60 threw off the character size, so I compensated by replacing this with a 56px version (orangeClockIcons56). This required minor adjustments to alignment, included in displayBlockMoscowFees.py.
I'm hoping there are no unforeseen issues with this approach, and others can help test the changes.

Either we add the character 'E' 'R' 'O' to the font or we have to change the code. Both acceptable to me. Line171  is broken without that change

Either we add the character 'E' 'R' 'O' to the font

Is it the uppercase letters, or lowercase "ero" chars to spell "error"?

lower case 'e' 'r' 'o'
capitalization of the chars was my mistake

Great catch. Recompiled with -c ",1234567890eor "


rocm device_tracer implementation

ROCM device tracer implementation

Can one of the admins verify this patch?

Can one of the admins verify this patch?

I have addressed the comment from PR #106
The PR depends on changes to HIP and HCC to function properly:
HCC: https://github.com/eshcherb/hcc branch 'roctracer-hip-frontend' - working on publishing it as PR
HIP: https://github.com/eshcherb/HIP branch 'roctracer-hip-frontend-180807' PR#622
Also going to change the CI script for TensorFlow so this PR can be tested with modified HIP / HCC, as well as ROCm GPU tracer,  #105 can be used as an example.

Also the PR is using the ROC-tracer library, working on open-soucing it:
https://github.com/ROCmSoftwarePlatform/roctracer

@eshcherb Thanks. Since there is a new dependent component (roctracer) needed in order to enable the feature introduced in this PR. Please help change tensorflow/tools/ci_build/Dockerfile.rocm whatever way you want to make sure TensorFlow is built with roctracer. It it suggested to change Dockerfile.rocm so HCC / HIP are cloned from your fork, or the branches which contains references to roctracer.

comments by @MikalaiDrabovich
'stream:all' data is missing in RunMetadata for rocm, but it is always present in cuda. That's critical since that's where native tfprof and also tensorscope take op gpu time ( 'acc time').
I diff'ed device_tracer_rocm.cc with cuda variant and the part which adds to 'stream:all' looks ok - data is being added to stream:all , yet it doesn't appear in RunMetadata - @eshcherb @whchung please take a look of what's going on with stream:all
tensorflow-upstream/tensorflow/core/platform/default/device_tracer_rocm.cc
Line 289 in 81544e8
collector->Save(strings::StrCat(stream_device, "all"), ns);
Not critical but nice to follow the same format: - 'stream:N' (where N is some int number) streams in rocm contain only memcpy ops, but should also include other ops (sum of times from all n streams will be equal to times stream:all)

Update - looks like 'stream:all' is not always missing: every 8-9 steps it appears in RunMetadata, then again is missing for next 8-9 steps, this is for tf_cnn_benchmarks.  I haven't observed this with official resnet model from tensorflow/models/tree/master/official/resnet Digging deeper ...

As I understand the records are collecting to stream raws and to 'all' row.
Are the records missing only in 'all' raws or they missing in both stream and 'all' rows.
Was I saw that the records are delayed by HCC runtime so the can come much later.
Do you merge the timelines/metadata from all steps?

Are the records missing only in 'all' raws or they missing in both stream and 'all' rows.
both

Do you merge the timelines/metadata from all steps?
roctracer sends already merged data, tensorscope now accounts for this, unlike tfprof which erroneously treats merged data as being from one step. But tfprof doesn't seem to be maintained now: https://github.com/tensorflow/tensorflow/issues/20960#issuecomment-414479927

Regarding the most recent commit, I suspect that we won't be able to merge this PR if ci_build/Dockerfile.rocm is changed to use your forks for HIP & HCC.  I'd recommend using a separate Dockerfile instead (e.g. Dockerfile.rocm-roctracer).  This will allow our critical CI infrastructure to proceed normally, and for the important rocTracer development to continue in parallel.

Close this PR as it has been superseded by #238