Rate Limiting / Timeout

Hello :wave:
I am currently working on a CLI for uptime-kuma using this wrapper.
But I have some timeout problems using this.
For example, I tried to implement the command kuma monitor ls :
I tried implementing the kuma monitor ls command to display hostname, monitor id, monitor name, status.
But I got 9/10 timeout errors because I need to perform 1 requests to get all monitors + 1 x Ymonitors to get heartbeat status.
I have about 5 monitors in my Uptime Kuma dev instance. So I've made 6 requests and that already seems too many for the API.
Am I missing something? How can I solve this problem, because it's really frustrating?

By the way, I already tried with a higher timeout. For example 30s and I still have issue

Timeouts should not occur. Rate limiting by uptime kuma is possible and can be seen in the uptime kuma logs. As far as I know, it is not the number of requests that matters, but the number of logins. You can prevent this by logging in only once and then making the requests.

In relation to earlier and now, I have actually observed this.
But for example I have this :
def ls_monitors(args):
    """
    List all monitors.

    Args:
        args: The arguments passed to the ls command.
    """
    # Connection to Uptime Kuma
    api = UptimeKumaApi(config.UPTIME_KUMA_URL, timeout=0.5)

    # Retry to login, until it works
    success = False
    while not success:
        try:
            api.login(config.UPTIME_KUMA_USERNAME, config.UPTIME_KUMA_PASSWORD)
            success = True
        except Exception:
            success = False

    monitors = []

    try:
        monitors = api.get_monitors()
    except Exception as e:
        api.disconnect()
        print("Error listing monitors:", str(e))
        os._exit(1)

    monitors_data_table = []
    for monitor in monitors:
        try:
            monitor_beats = api.get_monitor_beats(monitor["id"], 1)
            # print(monitor_beats)
        except Exception as e:
            api.disconnect()
            print("Error listing monitors:", str(e))
            os._exit(1)

        if monitor["active"] or args.all:
            data_row = [
                monitor["name"],
                monitor_beats[-1]["status"].name.capitalize(),
            ]

            if not args.short:
                if not args.no_id:
                    data_row.insert(0, monitor["id"])

                if not args.no_type:
                    data_row.append(monitor["type"].name)

                if not args.no_url:
                    data_row.append(value[monitor["type"].name.lower()])

                if not args.no_description:
                    data_row.append(monitor["description"])

            monitors_data_table.append(data_row)
            # print(monitors_data_table)
    headers = ["NAME", "STATUS"]

    if not args.short:
        if not args.no_id:
            headers.insert(0, "ID")
        if not args.no_type:
            headers.append("TYPE")

        if not args.no_url:
            headers.append("HOSTNAME")

        if not args.no_description:
            headers.append("DESCRIPTION")

    print(
        tabulate(
            monitors_data_table,
            headers=headers,
            tablefmt="plain",
        )
    )

    api.disconnect()

And with the following code, the only remaining crash was on api.get_monitor_beats(monitor["id"], 1) sometime that take 0.1s to proceed and something that timeout. Why? Why does it sometimes take 0.1s and sometimes more than 10s?

Nothing seems wrong in logs :
2023-08-10T14:37:19+02:00 [AUTH] INFO: Login by username + password. IP=X.X.X.X
2023-08-10T14:37:19+02:00 [RATE-LIMIT] INFO: remaining requests: 19
2023-08-10T14:37:19+02:00 [AUTH] INFO: Successfully logged in user KumaCompanion. IP=X.X.X.X
2023-08-10T14:37:21+02:00 [AUTH] INFO: Login by username + password. IP=X.X.X.X
2023-08-10T14:37:21+02:00 [RATE-LIMIT] INFO: remaining requests: 18.594
2023-08-10T14:37:21+02:00 [AUTH] INFO: Successfully logged in user KumaCompanion. IP=X.X.X.X
2023-08-10T14:37:21+02:00 [MONITOR] INFO: Get Monitor Beats: 12 User ID: 1
2023-08-10T14:37:21+02:00 [MONITOR] INFO: Get Monitor Beats: 14 User ID: 1

I cannot reproduce this problem. With a local uptime kuma instance I made 1000 get_monitor_beats requests with different monitor ids and time periods. All of them took less than 0.02 ms.
It would be interesting to know which requests are really sent and whether they arrive. The same applies to the responses. Can you investigate this further?

I'll investigate further during the week, and get back to you.
This could be due to my network configuration, although I have my doubts. During your tests, did you connect directly to the IP address on the same local network? No DNS?

Okay, so I tried a few things.
I tried running a series of tests to add all kinds of monitors.
I ran theses test with 4 different configurations :

A local instance of Uptime Kuma: I had 8 successes out of 8 and 6s of run time
An instance in the same local network: I had 8 successes out of 8 and 8s of run time
An instance on the Internet linked to a domain name: I had 8 successes out of 8 and 6s of run time
An instance on the Internet behind a reverse proxy (HAProxy): I had 8 successes out of 8 (with 8 additional attempts due to the crash) and 1min34s run time.

In conclusion, the limitation is surely not due to Uptime Kuma, but to my reverse proxy configuration. I propose to leave this issue open, while I solve the problem on my side, in order to help people who could possibly encounter the same problem as me.

I'm seeing the same issue behind caddy. Will investigate as well.

Not entirely sure if this is the same issue, but I was getting random "UptimeKumaException: monitor does not exist" errors without a proxy in the mix (just calling directly to the port exposed by Docker).  Thanks @lucasheld - your comment above about rate limiting being based on the number of logins was the hint I needed. I've changed the logic so the existing API session is re-used and the errors have gone away.  (My use case is adding more complex alerting logic external to Kuma e.g. a group with an allowable percentage of monitors being down).


Tracking for MOSS initiatives

Rich, congratulations on your Mozilla Open Source Support award!
How can we, the users of zappa, best track and follow the three initiatives you're planning to work on with this award that you called out, "Production Mode", "Server-less Microservice Library" and "Backlog"?
I'd also be curious to know the timeline of these initiatives so we know what to expect?
I see this GitHub project for Production Mode, should we use the GitHub projects function to track these 3 initiatives?

Looks like this didn't work out which is a shame. The GitHub Production Mode project didn't make progress and the Mozilla Open Source award was never collected as the first milestone wasn't completed.


Add nvidia channel

Can we please add nvidia channel (https://anaconda.org/nvidia) so we can get download stats for all packages within?
Currently, I don't see any download counts using condastats.

Yes, I will add that into the next data build, which is running a bit behind but should happen this week.

I have added the nvidia channel to June and July 2021 data.  We will add it to older months during a future reprocessing.


🐛 Bug Report(gitlab-org-provider): Groupmembership should handle/resolve members that are added via another group

📜 Description
First of all, I am uncertain if this is a Bug-Report or a Feature-Request.
The GitLab group and project structure can be complex. It is possible to add Members to a group direct and via another group. Additionally, groups can be nested and the membership of the parent group is inherited to the child groups.
While backstage is able to handle the second, it doesn't recognize members added via another group.
👍 Expected behavior
Members added via another group should be member
👎 Actual Behavior with Screenshots
Members added via another group are no member
👟 Reproduction steps
Try to reproduce this in your GitLab instance
GroupA with members M1, M2, M3
GroupB with members M5, M6, M7
=> replicate to backstage via org provider and you will see that the sync was correct
Now add GroupA as member to GroupB and wait for sync in Backstage and you will see, that M1, M2, M3 won't be part of GroupB.
📃 Provide the context for the Bug.
In GitLab it's quite common that teams maintain there team members in a group holding no repositories/code/... this group is just used for defining the team members.
Then there might be other groups for products, projects, and so on and only the team-groups are added to them. This way, you don't have to manage the team members in various places.
🖥 Your Environment
No response
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
None

The catalog intentionally mirrors the source. If you are an actual direct member of a group, then you are modelled as a member. If we were to smear out this membership transitively upward OR downward (you see how people might have different views on which direction is the correct one), it would destroy information and maybe help in one space but make another feature impossible to implement properly. So while you could make a custom entity provider that does what you describe, it would probably hurt for some use cases.
Those who want to traverse the tree needs to do so themselves through the API.
May I ask why you wanted this in the first place? Is it to model ownership? If so, that's handled by sign-in resolvers during auth instead; that's where you get the opportunity to describe all of the entity refs that the current user can claim ownership through, by any means necessary, including making requests to external systems if needed.
https://backstage.io/docs/auth/identity-resolver

I understand what you say - the catalog intentionally mirrors the source. And I don't think we smear out something up OR downward, as it's clear defined in Gitlab how memberships work...
And I don't want Backstage to handle something, I think the org provider should ingest it like Gitlab does...

          Gitlab

           --|--
        --/  |  \-
     --/     |    \--
   -/        |       \-
+-----+   +-----+   +-----+
|     |   |     |   |     |
|  A  |   |  B  |   |  D  |
|     |   |     |   |     |
+-----+   +--|--+   +-----+
             |
          +-----+
          |     |
          |  C  |
          |     |
          +-----+

```

So Members of B are also Members of C
If A is invited to D, Members of A would also be Members of D ...

Gitlab has two different Endpoints for query the direct Members and all Members

https://docs.gitlab.com/ee/api/members.html#list-all-members-of-a-group-or-project
https://docs.gitlab.com/ee/api/members.html#list-all-members-of-a-group-or-project-including-inherited-and-invited-members

So nothing I would implement myself and don't want to smear things  - it should rely on the state of GitLab - to mirror the source, but I want to configure how it should mirror things.

Ok, found the source ... here should be a minor change that would allow to configure the behaviour:
https://github.com/backstage/backstage/blob/a4cfe9d6f4d063bdb8d4e96a3470cf6dcae3e82b/plugins/catalog-backend-module-gitlab/src/providers/GitlabOrgDiscoveryEntityProvider.ts#L427
https://docs.gitlab.com/ee/api/graphql/reference/#groupmemberrelation
At this point we should allow SHARED_FROM_GROUPS also via config option.

Hi @freben !
the GitlabOrgDiscoveryEntityProvider performs a GraphQL call that searches for group members already. This can be seen here.
We have just added the two other valid values to the GroupMemberRelation property of the query. We exposed it as a config option, like the allowInherited that Markus mentioned above.
We have tested it in our instance and it is running fine.
We believe the solution would also allow the users to model their ingestion as they see fit, considering GitLab's possibilities.
I would gladly submit a PR with the code and the documentation update if you think this would be interesting! :)

Hello @freben ,
I have submitted the PR. Please let me know your thoughts. :)


roctracer support for fixing finalization

thread device finalizer flushes trace
process device finalizer shuts down rocm measurement, flushes trace, then calls gpu_trace_fini

my fix at the end of november got the order between flush and gpu_trace_fini right. however, every thread executed the complete rocm finalizer, which shuts down rocm and flushes.
with this change, each thread flushes.
main thread flushes, shuts down rocm, flushes again (just to be sure), and then calls gpu_trace_fini.


fix: prevent identically named top-level items in the left nav

Before this change, if two top-level items had the same name, that name appeared twice in the left nav. For example, v1.Foo and v2.Foo both appeared as Foo, and the generated HTML used the id attribute Foo-nav for both items.
After this change, the left nav shows v1.Foo and v2.Foo instead, and their id attributes are v1.Foo-nav and v2.Foo-nav, respectively.
This issue was originally observed in the Cloud Tasks docs. I tested the fix with googleapis/nodejs-tasks and confirmed that it fixed the issue.

@bcoe FYI

:tada: This PR is included in version 1.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


How do you determine the width and height of the input, and the size of the label_margin?

@nicolov
Hello, I'd like to know if input_width, input_height, label_margin values are related to width, height of the picture? Do you have formulas or something like that?
Because I want to predict a new picture now, each picture of different sizes, some are large, some relatively small lot, which requires a different input_width, input_height, label_margin in size, that will not throw the error in "AssertionError". I try to use Image.resize method to predict the images, making them the size difference is not big, but I found that after the resize images and the size of the original picture of the forecast results are not the same, you can help me to answer this question? Thank you

I think the best way to go would be to either crop or tile the images so that they're all the same size and match what the network expects.


[2664] Add Trainees#ebacc boolean

Context
We will be supporting PE with Ebacc soon but currently users are unable
to add these trainees. We're going to flag the records via support for
now and expand the feature in the future.
Changes proposed in this pull request

Add field to the trainees table

Guidance to review
This just adds the field for now. Sending to DTTP will be addressed in another PR.

 Add an ebacc field to the trainees table


about modify indentUnit

After the editor has been established, I pass
this.cm.setOption ('indentUnit', index) to modify the indentUnit, but after each modification, the indentUnit on the page will be index * 10, but my getOption ('indentUnit') is still the index. Why is this?

What does it mean for "the indentUnit on the page" to be a certain thing? Do you mean the indentation you get at a specific point in your document? If so, could you be more specific about that?

I need to dynamically change the two parameters of tabSize and indentUnit, but after the indentUnit is changed, the displayed indentation will be * 10. If I this.cm.setOption ('indentUnit', 2), then the displayed indentation will Becomes 20

But the value of my cm.getOption ('indentUnit') is still 2

What displayed indentation? indentUnit has no effect until you actually indent a line. And that line's actual indentation will depend both on the indentUnit setting and the context.

Simply put：
I set tabSize = 4 and indentUnit = 4
But indentUnit is indented by 40 spaces
If indentUnit = 2, indentUnit is indented by 20 spaces

But indentUnit is indented by 40 spaces

Again, I don't know what you mean by this (and unless you provide an example I'm going to give up asking)

this.cm = CodeMirror.fromTextArea(element[0], {
  mode: 'text/x-ruby',
  theme: 'monokai',
  matchBrackets: true,
  indentUnit: tab_size,
  tabSize: tab_size,
  vimMode: true
})

If this is my editor, I want to change the indentUnit after the editor has been created, this.cm.setOption ('indentUnit', 2),
After my setOption('indentUnit', 2) succeeded, my actual indentation effect became 20

Strange. I can't reproduce that in the ruby mode demo—if I change indentUnit there, further indentations seem to do the right thing.

I click the button to change the value of indentUnit. For example, when my editor is initialized, indentUnit is 4, but I want to change indentUnit to 2 during the process of writing code.

May be described more accurately here：
https://stackoverflow.com/questions/60255695/there-was-a-problem-setting-up-codemirror-indentunit


How to save and load the created filter

Hi,
How to save and load the created filter?

You can get the bytes from the filter and create new filter from bytes，some examples can be find at https://github.com/yankun1992/fastbloom/blob/main/py_tests/test_save_file.py


Mention Plugin is not working

In the documentation it is mentioned that Mention plugin is included.
So I tried out following config inside /config/plugins.js with following code return:
{
        ckeditor: {
            enabled: true,
            config: {
                editor: {
                    toolbar: {
                        items: [
                            "bold",
                            "italic"
                        ]
                    }
                },
                plugin: {
                    mention: {
                        feeds: [{
                            marker: "@",
                            feed: ["@Kishore", "@Kumar", "@Barik"]
                        }]
                    }
                }
            }
        }
    }

I don't see the mentions appear.
Additional info
"@strapi/strapi": "4.1.9"
"@_sh/strapi-plugin-ckeditor": "^1.1.0"

try this
{
        ckeditor: {
            enabled: true,
            config: {
                editor: {
                    mention: {
                        feeds: [{
                            marker: "@",
                            feed: ["@Kishore", "@Kumar", "@Barik"]
                        }]
                    },
                    toolbar: {
                        items: [
                            "bold",
                            "italic"
                        ]
                    }
                }
            }
        }
    }

thanks @nshenderov
That config is working. Could you please provide a documentation on how to work with the bundled plugins, that would be great.
Anyway thanks for this nice project as well !!!


[view] Prisma filtering on view doesn't work as expected

Bug description
Here's a code snippet that should return all portfolios with transactions given a certain userId:
const publicPortfoliosWithTransactions =
    await prisma.portfolioAssetTransactionView.findMany({
      where: {
        totalAmount: {
          gt: 0,
        },
        totalQuantity: {
          gt: 0,
        },
        portfolio: {
          deletedAt: null,
          userId,
        }
      },
      select: {
        assetName: true,
        portfolioId: true,
        totalAmount: true,
        totalQuantity: true,
        portfolio: {
          include: {
            user: {
              select: {
                name: true,
                email: true,
              },
            },
          },
        },
      },
    });

However, when executed, I get all entries from PortfolioAssetTransactionView and because of that, I need to filter them manually. This really sux, especially because this one isn't obvious at first. As I said in the beginning, I'm aware that this is still a preview feature, but it would be really good if this could be resolved soon 🙏
 // 2. Retrieve all public portfolios that have transactions > 0
  const publicPortfoliosWithTransactions =
    await prisma.portfolioAssetTransactionView.findMany({
      where: {
        totalAmount: {
          gt: 0,
        },
        totalQuantity: {
          gt: 0,
        },
      },
      select: {
        assetName: true,
        portfolioId: true,
        totalAmount: true,
        totalQuantity: true,
        portfolio: {
          include: {
            user: {
              select: {
                name: true,
                email: true,
              },
            },
          },
        },
      },
    });

  // For some reason Prisma filters are not working for portfolio
  // That's why we have to filter manually
  const result = await calculatePortfolioMetrics(
    publicPortfoliosWithTransactions.filter(({ portfolio: p }) => {
      return (
        p.userId === userId &&
        p.isPublic &&
        p.deletedAt === null &&
        p.isAuthorVisible
      );
    })
  );

How to reproduce

Generate DB based on the provided models
Run this script to create the view
Run prisma pull to fetch the view into schema.prisma
Modify the view to match the one I pasted in the issue
Create some dummy data via Prisma Studio
Try to execute my TS snippet

 create or replace
view
  "AssetTransactionView" as
select
	JOINED.id,
	JOINED.symbol,
	JOINED.name,
	SUM(
    case
      when JOINED."transactionType" = 'BUY' then JOINED.amount
      else 0
    end
  ) - SUM(
    case
      when JOINED."transactionType" = 'SELL' then JOINED.amount
      when JOINED."transactionType" = 'CLOSE' then JOINED.amount
      else 0
    end
  ) as "transactionAmount",
	SUM(
    case
      when JOINED."transactionType" = 'BUY'
      or JOINED."transactionType" = 'CLOSE' then JOINED.quantity
      else 0
    end
  ) - SUM(
    case
      when JOINED."transactionType" = 'SELL' then JOINED.quantity
      else 0
    end
  ) as "transactionQuantity",
	SUM(JOINED.dust) as "totalDust"
from
	(
	select
		a.id,
		a.name,
		a.symbol,
		at."transactionType",
		at.amount,
		at.quantity,
		at.dust
	from
		"Asset" a
	left join "AssetTransaction" at on
		a.id = at."assetId"
	where
		at."isValid" = true
	group by
		a.id,
		a.name,
		a.symbol,
		at."transactionType",
		at.amount,
		at.quantity,
		at.dust
  ) as JOINED
group by
	JOINED.id,
	JOINED.name,
	JOINED.symbol;

Expected behavior
portfolioAssetTransactions should include only entries that have portfolio.userId matching the provided userId
Prisma information

model Portfolio {
  id                                String             @id @default(cuid())
  name                              String
  assetTransactions                 AssetTransaction[]
  value                             PortfolioValue[]
  assets                            Asset[]            @relation("AssetToPortfolio")

  portfolioAssetTransactions        PortfolioAssetTransactionView[]
}

model Asset {
  id                String             @id @default(cuid())
  symbol            String
  name              String             @unique
  assetTransactions AssetTransaction[]
  portfolio         Portfolio[]        @relation("AssetToPortfolio")
  portfolioAssetTransactionView PortfolioAssetTransactionView[]
}

model AssetTransaction {
  id              String          @id @default(cuid())
  assetId         String
  quantity        Decimal
  amount          Decimal
  weight          Decimal?
  transactionType TransactionType
  portfolioId     String?
  userId          String
  createdAt       DateTime        @default(now())
  updatedAt       DateTime        @updatedAt
  dust            Decimal         @default(0)
  isValid         Boolean         @default(true)
  price           Decimal
  asset           Asset           @relation(fields: [assetId], references: [id], onDelete: Cascade)
  portfolio       Portfolio?      @relation(fields: [portfolioId], references: [id], onDelete: Cascade)
}

view PortfolioAssetTransactionView {
  portfolioId     String   @unique
  name            String
  assetId         String
  assetName       String
  symbol          String
  startingCapital Decimal
  totalAmount     Decimal   @db.Decimal
  totalQuantity   Decimal   @db.Decimal
  totalDust       Decimal?  @db.Decimal
  portfolio       Portfolio @relation(fields: [portfolioId], references: [id])
  asset           Asset     @relation(fields: [assetId], references: [id])

  @@unique([portfolioId, assetId])
}

enum TransactionType {
  BUY
  SELL
  CLOSE
}

const publicPortfoliosWithTransactions =
    await prisma.portfolioAssetTransactionView.findMany({
      where: {
        totalAmount: {
          gt: 0,
        },
        totalQuantity: {
          gt: 0,
        },
        portfolio: {
          deletedAt: null,
          userId,
        }
      },
      select: {
        assetName: true,
        portfolioId: true,
        totalAmount: true,
        totalQuantity: true,
        portfolio: {
          include: {
            user: {
              select: {
                name: true,
                email: true,
              },
            },
          },
        },
      },
    });

Environment & setup

OS: macOS
Database: PostgreSQL
Node.js version: v19.1

Prisma Version
prisma                  : 5.1.1
@prisma/client          : 5.1.1
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine 6a3747c37ff169c90047725a05a6ef02e32ac97e (at<EMAIL_ADDRESS>Schema Engine           : schema-engine-cli 6a3747c37ff169c90047725a05a6ef02e32ac97e (at node_modules/@prisma/engines/schema-engine-darwin-arm64)
Schema Wasm             : @prisma/prisma-schema-wasm 5.1.1-1.6a3747c37ff169c90047725a05a6ef02e32ac97e
Default Engines Hash    : 6a3747c37ff169c90047725a05a6ef02e32ac97e
Studio                  : 0.492.0
Preview Features        : views

Can you please log and share the SQL query that Prisma Client generates and executes with https://pris.ly/d/logging? Thanks.

Sure.


Note Splashes And Note Assets reverting to normal!

Describe your bug here.

Same with note skin!
Command Prompt/Terminal logs (if existing)
No response
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
nope nothing edited

does the noteskin name in the txt have any typo?

**xxyuee ** commented Jun 16, 2024

what do you mean noteskin has a typo?


UI for creating alibaba ACK managed kubernetes

Add support for creating managed kubernetes cluster when using alibaba ACK driver

Available Regions & AZs are different from non-managed type:

No need for non-managed clusters' configurations:

	SSHFlags                 bool   `json:"ssh_flags,omitempty"`
	MasterInstanceChargeType string `json:"master_instance_charge_type,omitempty"`
	MasterPeriod             int64  `json:"master_period,omitempty"`
	MasterPeriodUnit         string `json:"master_period_unit,omitempty"`
	MasterAutoRenew          bool   `json:"master_auto_renew,omitempty"`
	MasterAutoRenewPeriod    int64  `json:"master_auto_renew_period,omitempty"`
	MasterInstanceType       string `json:"master_instance_type,omitempty"`
	MasterSystemDiskCategory string `json:"master_system_disk_category,omitempty"`
	MasterSystemDiskSize     int64  `json:"master_system_disk_size,omitempty"`
	MasterDataDisk           bool   `json:"master_data_disk,omitempty"`
	MasterDataDiskCategory   string `json:"master_data_disk_category,omitempty"`
	MasterDataDiskSize       int64  `json:"master_data_disk_size,omitempty"`

Enable The PublicSlb option so that we can connect to the cluster without providing the client cert

@gitlawr Please validate it after UI gets merged

Confirmed with Aliyun team that the 1.11.5 version does not support publicSlb option. So we need to use the latest 1.12.4-aliyun.1 that supports it.
NumsOfNodes of a managed cluster has a minimum of 2. It would be better to have this validation in the UI.

To validate:

Be able to create/delete a managed ACK cluster
Be able to create workload, LB, use the cluster kubectl terminal in an active managed ACK cluster

Notes:
The provision option we use is only available in Singapore region(ap-southeast-1) at the moment. According to aliyun team, support for other regions will be added in late Feb.

Create a managed ACK cluster error, the error log is as follows:
time="2019-02-20T04:15:59Z" level=info msg="provisioning cluster lawrm2: Set up k8s DNS configuration successfully"
time="2019-02-20T04:15:59Z" level=info msg="Cluster lawrm2 is running"
time="2019-02-20T04:16:03Z" level=info msg="service account token generation failed, retries left: 5"
time="2019-02-20T04:16:33Z" level=info msg="service account token generation failed, retries left: 4"
time="2019-02-20T04:17:03Z" level=info msg="service account token generation failed, retries left: 3"
time="2019-02-20T04:17:33Z" level=info msg="service account token generation failed, retries left: 2"
time="2019-02-20T04:18:03Z" level=info msg="service account token generation failed, retries left: 1"
time="2019-02-20T04:18:33Z" level=error msg="retries exceeded, failing post-check"
2019/02/20 04:18:33 [INFO] kontainerdriver aliyunkubernetescontainerservice stopped
2019/02/20 04:18:33 [ERROR] ClusterController c-r9w2s [cluster-provisioner-controller] failed with : Post https://<IP_ADDRESS>:6443/api/v1/namespaces: x509: certificate signed by unknown authority

Edit ACK cluster, cannot choose Kubernetes version.

It will be fixed in https://github.com/rancher/rancher/issues/17174

logged the error in https://github.com/rancher/rancher/issues/18268

Version: master 2.2 (2/20/19) Image ID: cdd98a0abb92
The following cases have been verified:

Create a managed ACK cluster
Delete a managed ACK cluster
Create a workload
Create and verify that LB is available
create catalog app
Use the cluster kubectl terminal


Why the default value of dispatch_batches is False using IterableDataset in distributional training?

I am wondering why the default value of dispatch_batches is False using IterableDataset in distributional training, and why use DataLoaderDispatcher rather than DataLoaderShard?

Using Dispatch dataloader will result in heavy CPU memory cost if we use many nodes for distributional training, no?

Maybe creating data in every local_rank=0 processes is better than creating all data from one host.

It shouldn't be, prepare_dataloader should know to grab and use DataLoaderShard if we have an iterable dataset. Can you provide a clean, concise example of where this isn't the case? I can point to the exact lines in prepare_dataloader that check and do this.
The way this works is we create 1/n of the data on each node instead, not the entire dataset. So one node has the entire dataset only, and the rest get chunks of processed data as it comes through the DataLoader

It shouldn't be, prepare_dataloader should know to grab and use DataLoaderShard if we have an iterable dataset. Can you provide a clean, concise example of where this isn't the case? I can point to the exact lines in prepare_dataloader that check and do this.
The way this works is we create 1/n of the data on each node instead, not the entire dataset. So one node has the entire dataset only, and the rest get chunks of processed data as it comes through the DataLoader

The work flow of dispatch_batches:

dispatch_batches will be set to True if we use IterableDataset: accelerator.py line 209
prepare(), prepare_data_loader() use dispatch_baches: accelerator.py line 1877
prepare_data_loader() return a DataLoaderDispatcher() dataloader: data_loader.py line 902
In DataLoaderDispatcher, only process 0 construct data, and other proces will stop iteration: data_loader.py line 591

My question is if it is reasonable to let process-0 to create all data. If we use 16 hosts, the host-0 will be responsible for creating all 16 hosts' data, and dispach them to other hosts.

Yes, that is how the API is currently. (And thus, is reasonable so far from users). Changing that to be the local process is fairly straightforward and trivial. Would you like to test it out and see if that helps reduce your CPU load? Just replace process_index with local_process_index. We can then add a flag to the Accelerator with args on which to use.

of course!
I think it will make difference if we use distributional trianing at scale, where the data processing and communication could be a bottleneck if we only use host-0 to process data. Increasing the num_workers may be a solution but results in a high CPU memory cost in host-0 -- that's why I want to implement local host data processing.

If it works for you, definitely feel free to open a PR please 🙏 However it's okay if you would like me to do so instead, it's probably about time we add a dataloader config to Accelerator anyways since there's so many parameters.

Yes, that is how the API is currently. (And thus, is reasonable so far from users). Changing that to be the local process is fairly straightforward and trivial. Would you like to test it out and see if that helps reduce your CPU load? Just replace process_index with local_process_index. We can then add a flag to the Accelerator with args on which to use.

How to set the dispatch_batches to False even I am using the IterableDataset?  I can not afford the memory to keep all samples padded to the max_lenght of the whole dataset,  but the data is too large so I should have to used IterableDataset.  currently I can not find the "dispatch_batches" param in Accelerator init param list.


我的服务器使用的是Http2

如何修改客户端底层的版本呢？

Can you provide a test script that runs nodejs on http2? Then I can quickly test, and, if possible, add his support


Added Solution for 1277. Count Square SubMatrices with All Ones in CPP (issue #1504)

Pull Request Template
What have you Changed(must)
Added Solution for 1277. Count Square SubMatrices with All Ones in CPP
Issue no.(must) - #1504
Pr will be close and marked as spam. If issue number not found or issue was assigned to someone else.
Marking as spam can block your account from HacktoberFest.
Self Check(Tick After Making pull Request)

[x] This issue was assigned to me if not it will be marked as spam.
[x] My file is in the proper folder
[x] I am following clean code and Documentation.
[x] I have added the title and what program will do first in the file
[x] I have named the file as (Problem #). (Problem Title(in Camel Case)).(extension)
ex. 2. AddToNumbers.py

README - How to Contribute

I have added the solution with necessary comments, please merge this


Update sbt-http4s-org to 0.12.2

Updates org.http4s:sbt-http4s-org from 0.2.1 to 0.12.2.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.http4s", artifactId = "sbt-http4s-org" } ]

labels: sbt-plugin-update, early-semver-major, semver-spec-minor, commit-count:1

Superseded by #168.


True zero downtime haproxy reloads

We can improve how haproxy configuration is reloaded:
http://engineeringblog.yelp.com/2015/04/true-zero-downtime-haproxy-reloads.html
(Please note that -sf is mentioned, but it didn't always help)
The same method was applied in AWS OpsWorks:
https://github.com/aws/opsworks-cookbooks/pull/40

I don't think there's much Bamboo can or should do to the extent that the blog post describes. The solution Yelp outlines requires using OS-specific low-level queuing techniques to prevent SYN packets from getting rejected. I wouldn't want Bamboo to fiddle with my OS tools directly -- that's something a system administrator (or configuration management tools driven by him) should do.
The method applied to the AWS cookbook is specifically not the one Yelp chose: Dropping SYN packets deliberately can cause delays of up to 3 seconds, which is the very reason Yelp decided against it and went with queuing instead.
Finally, the Yelp approach only works for outgoing traffic, which doesn't cover the standard Marathon/Bamboo/HAProxy scenario where we deal with incoming traffic, so it's actually not particularly helpful here.
From my perspective, the only thing the Bamboo project could do is hint at the problem in the documentation and describe ways on how to deal with it. Off the top of my head, I can think of two:

Ignore the problem. If you don't scale at Yelp level, you may be able to just not care for occasional errors on clients' end. Or the set of clients is limited and they retry on connection errors anyways.
Shape traffic from an intermediary. As sketched out in the Blog post, one could put another intermediary node between the clients and HAProxy to have it queue up HAProxy-heading, outgoing SYN packets during reloads.

I'm going to agree with @timoreimann on this. If you see in the yelp blog post, they ship the reloading out to a seperate script, which is what I would recommend a bamboo user do; setting your ReloadCommand to qudisk_protect .....  wouldn't cause any problems.
I'm going to close this issue, for the sole reason of reducing the number of open issues that aren't particularly actionable. If/When I get around to rolling out qdisc protected reloads at Qubit I might write some documentation on it, but until then, pull requests welcome :)


Git Push (Follow Tags) not working in github

VSCode Version: 1.50.1
OS Version: Ubuntu 18 Bionic Beaver

Steps to Reproduce:

Commit
Git Create Tag
Git Push (Follow Tags)

My code successful push to github, but no tags and releases file created in github.

Does this issue occur when all extensions are disabled?: Yes/No
YES (except GIT extension)

Can you show me the full output of Git: Show Git Output, as soon as that command is executed?

Here is the screenshot of the Git Output
Commit

Create Tags

Push (Follow Tags)

You can check my github there is no tags and releases file created at here >> https://github.com/aalfiann/fastify-fullstack-skeleton

Mirror image:

Commit : https://i.imgur.com/KcvejSH.png
Create Tags: https://i.imgur.com/UAhw9Ya.png
Push (Follow Tags): https://i.imgur.com/01kmPTU.png

What git version do you have installed?

Git 2.17.1
Is that too old?

Does it fix it if you upgrade git? https://git-scm.com/

I've already update to Git 2.29.2 but still not work.

If you run git push --tags, does that push that tags?

yes it will works. i already know it.
but my issue is about this feature, I like to using this command shortcut, and actualy in vscode version 1.49 still working.

Yeah... so --follow-tags isn't the same as --tags, so we might need a Git: Push Tags command.


YETUS-967. Switch everything to https://downloads.apache.org

JIRA: https://issues.apache.org/jira/browse/YETUS-967

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
2m 51s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 14s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  5s
Maven dependency ordering for patch

+1 :green_heart:
markdownlint
0m  2s
There were no new markdownlint issues.

+1 :green_heart:
shellcheck
0m  0s
There were no new shellcheck issues.

+1 :green_heart:
shelldocs
0m  1s
There were no new shelldocs issues.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
yamllint
0m  0s
There were no new yamllint issues.

_ Other Tests _

+1 :green_heart:
asflicense
0m 21s
The patch does not generate ASF License warnings.

4m  0s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/yetus-github-multibranch/job/PR-114/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/yetus/pull/114

Optional Tests
dupname asflicense yamllint markdownlint shellcheck shelldocs

uname
Linux a76a22a867b1 4.15.0-76-generic #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
precommit/src/main/shell/personality/yetus.sh

git revision
master / 7852010

Max. process+thread count
50 (vs. ulimit of 2000)

modules
C: . asf-site-src U: .

Console output
https://builds.apache.org/job/yetus-github-multibranch/job/PR-114/1/console

versions
git=2.7.4 maven=3.3.9 shellcheck=0.7.0 markdownlint=0.22.0 yamllint=1.20.0

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Thank you!


[Bug]: Layout on design system website is breaking at certain viewport widths

Describe the bug
Layout does not respond correctly and creates horizontal scroll between approx 824px and 624px. It should respond to fill the space available and not have horizontal scroll.
Examples of this happening:
https://github.com/GovAlta/ui-components/assets/82968683/df0f022f-a9de-442e-8181-8e58635b16bf
Component pages

All component page

Layout/patterns page

Content pages

It seems to be responding correctly on the styles pages, get started pages, and home page.
Provide Code
No response
To Reproduce

Go to our website: https://design.alberta.ca/
navigate to any of the above pages
change width of screen
see horizontal scroll

Relevant console log output
No response
What browsers are you seeing the problem on?
No response
Additional context
This may be related to an issue with the header not responding correctly

Menu shifted over

Menu overlapping off right side. (too many nav items before breakpoint?)

@ArakTaiRoth Re-review once we release

@twjeffery Go over this bug and modify based on what current needs are after the recent releases

Most of the layout bugs above are fixed.
There are still some outstanding issues with:

the margin on the header and microsite header at various breakpoints
some of the page content in the styles and content sections.

Review captured in markup.io here

closed and created #1803 to cover updated issues


Migrate media processing to separate repo

For cleanliness, but also so it can be made into a lambda

Nah, I want it integrated with the status message and stuff, and there's not much benefit of using lambdas. Closing


Update docs

This PR updates docs in preparation for publication via Travis build.
/cc @huyng @pierreg @pumpikano

CLA is valid!

Travis build will fail because we don't actually have a build defined yet, so the failure is OK for merging purposes.

You can look at the new docs like this:
pip install sphinx sphinx_rtd_theme
cd docs/
make html

The docs will be built in docs/build/.

Starting from a clean virtualenv, I see the following:
$ make html
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v1.4.5
WARNING: sphinx.ext.pngmath has been deprecated. Please use sphinx.ext.imgmath instead.

Extension error:
sphinx.ext.mathjax: other math package is already loaded
make: *** [html] Error 1

Any ideas?

@pumpikano, are you sure you checked out the PR?  I specifically changed these lines to fix that warning and error:
https://github.com/yahoo/graphkit/pull/2/files#diff-1ec61223727a10e5e4c74c68336809fcL38

Oooops must have been too late for my brain 😊  . Looks fantastic! 👍


Fix windows compilation + remove nscript due to fibers dep

Basically the same that was done for mobx in https://github.com/mobxjs/mobx/pull/1735
This PR:

removes nscript due to a dependency with fibers, a native node module that was kinda hard to compile (and actually impossible with new node versions due to compilation errors)
fixes the publish build script so it doesn't use such dependency and uses shelljs instead
fixes publish and build script so they work in windows as well

I think this should help new contributors get up and running faster

gonna merge since build passes and it is basically the very same thing that was already merged in the pr for mobx

No prob! Thanks!


No more windows installers

Previous Robot versions have been distributed also as a Windows installer. Now that we start supporting Python 3, that would require us to create four distributions: 32bit python 2, 64bit python2, 32bit python 3, and 64bit python3. This starts to get quite cumbersome. Also pip starts to come with all new installations on python, so maybe it's the time to ditch windows installers.
If this causes too much trouble for the users we can consider still making the installers.

In 3.0 alpha 1 it is enough to mention that we do not plan to create Windows installers anymore. Updating installation instructions can wait until later.

How to install:

use pip: pip install robotframework
use pip with a locally downloaded file (works without network): pip install robotframework-3.0.tar.gz
extract the source tar and install manually: python setup.py install

We now have #2222 that tracks installation related changes that must be done before the final 3.0 release. The plan not to create Windows installers is already documented in RF 3.0 alpha 1 release notes. This issue can be closed.

Notice that if you absolutely must have Windows installers, you can create them yourself. First you need to get the source code either by extracting the normal source distribution or checking out suitable tag directly from the version control. After that you just need to run:
python setup.py bdist_wininst

For more information see https://docs.python.org/2.7/distutils/builtdist.html#creating-windows-installers.


Interaction with /qiita_db/authenticate was off

I think this change renders correct -- did this via Github. The structure necessary to authenticate is:
{'grant_type': 'client', 'client_secret': str, 'client_id': str}
...whereas the prior parameters suggested I think {'grant_type': ['client', 'password'], 'client_id': str}.

thanks @wasade! I have move this change to the other open PR: https://github.com/biocore/qiita/pull/2730/files.


why the builded map data  used the ndt_mapping data is so big?

Deal all,
When I build the map using the ndt_mapping according to the video, I found that the builded map will be over 1G using the sample data. Wether any place I went wrong? Any help will be appreciated!

You can reduce the file size using voxel grid filter afterwards. It is in the Map tab.

@kitsukawa thank you for your reply, I have try to reduce the file size using PCD Filter. The size of my pcd file 1.3G and I set the leaf size as 0.2,0.3,0.4,0.45,0.48,respectively, after filter, I got the same size of pcd file(635.7M). When I set the leaf size as 0.5, I got a pcd file  (10.8M). So the first got pcd file is too thick and the second file I got is too sparse. I really have no idea, can you give me some suggestion

Hi, can you tell me how to run the ndt_mapping with my own data? And is there any guidance I can refer to? Thanks a lot.

See this video.
https://www.youtube.com/watch?v=HlQ0ohxvlgA

Thanks a lot, I m trying to do some experiments, and I really appreciate the excellent work you and your team achieved! @kitsukawa  If I have other problems, I will be back~ hhh

Oh, and there is one more question, can I use my '.pcap' file to run autoware mapping and then save the result? @kitsukawa

Yes. First, convert pcap to rosbag.
http://answers.ros.org/question/213080/convert-raw-velodyne-vlp16-pcap-to-bagfile/
The rosbag includes the topic, velodyne_packets, so you have to convert the topic from /velodyne_packets to /points_raw using VLP-16 driver in Sensing tab.

That's really helpful, but I just want to use ndt_mapping or icp_mapping with my own data only, instead of running Autoware everytime, but everytime I try to run ndt_mapping isolated, there's always an error of :
CMake Error at /opt/ros/indigo/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
Could not find a package configuration file provided by "runtime_manager"
with any of the following names:
runtime_managerConfig.cmake
runtime_manager-config.cmake

Add the installation prefix of "runtime_manager" to CMAKE_PREFIX_PATH or
set "runtime_manager_DIR" to a directory containing one of the above files.
If "runtime_manager" provides a separate development package or SDK, be
sure it has been installed.
Call Stack (most recent call first):
ndt_localizer-master/CMakeLists.txt:4 (find_package)
If I annotate the cmake file, it can not run successfully.

ndt_localizer package depends on runtime_manager package in relation to message generation. So can you build the all packages in Autoare using ./catkin_make_release in Autoware/ros directory?

Yes I can, after a few correction, I can run the ndt_localizer package, but I still need some modification of my code to satisfy my dataset. Thank you a lot @kitsukawa


Add test to load eodc data

this means, that I can add tests to RQADeforestation where I can assume that it runs on the EODC VMs and the path are right. Then I am going to add some tests for gdalcube next week.
Can we somehow check that this is actually running on EODC VMs as CI so that we can hide this behind some argument and be able to run other tests locally as well?

Yes, current github actions workflows runs on the self-hosted runner at eodc.
To enable local testing as well, we can split  runtests.jl to files runtests.jl and runtests-eodc.jl and run additional eodc tests in a new CI job.


feat(Kubernetes): Add JMX exporter containers to all DataHub components

Fixes https://github.com/linkedin/datahub/issues/2214
Added JMX exporter container to Helm charts.
Also added JVM_OPTS and JMX_OPTS to such that it is easy to configure the JMX on the original container.
Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable)

@dexter-mh-lee : could you take a look?

@bernardino There was a fix for the build issue that went in recently. The build error should go away once you merge latest master!

Build is now green


Determine which labels to exclude from Rekognition’s label set

Description

This will involve a manual process of looking through each of the available labels for Rekognition and seeing if they match any of the criteria to be filtered. This process should be completed by two maintainers, and their list of exclusions discussed & combined. The excluded labels should then be saved in an accessible location, either on S3 or within the sensitive terms repository as a new file. Consent & approval should be sought from two other maintainers on the accuracy of the exclusion list prior to publishing.
Additional context

See this section of the IP.

I have gone through the 3k labels (🥲) and pulled out the labels that I think we should exclude or consider to be excluded. Many of the "questionable" ones I think are fine, but we said (per #4662) that we would review each one so I think they're worth considering. Personally, I think they're okay to keep but open for a blocking objection there.
Exclude

Adult
Baby
Baby Crawling
Baby Laughing
Ballerina
Barbie
Beard
Bishop
Blonde
Blue Hair
Boy
Bridal Veil
Bride
Bridegroom
Bridesmaid
Brown Hair
Child
Childbirth
Curly Hair
Exchange Of Vows
Family
Female
Fireman
Girl
Green Hair
Lady
Male
Man
Newborn
Pink Hair
Red Hair
Senior Citizen
Teen
Woman

Questionable

Altar
Astronaut
Athlete
Attorney
Ballplayer
Bartender
Boxer
Buddha
Bullfighter
Captain
Carpenter
Chapel
Cheerleading
Chef
Chinese New Year
Christ the Redeemer
Church
Crucifix
Dentist
Dj
Doctor
Executive
Gardener
Gymnast
Hairdresser
Hanukkah Menorah
Hippie
Judge
Monk
Mosque
Musician
Nurse
Officer
Performer
Photographer
Pianist
Police Officer
Pope
Prayer
Prayer Beads
Priest
Soldier
Teacher
Veterinarian
Waiter

@AetherUnbound I agree with both of your lists but am also willing to contend with anyone else's blocking objections on some of the questionable terms.

On the excluded list, why are there labels like Curly Hair or Beard? I read the IP section, but the related criteria is not clear to me.

@sarayourfriend Here's the issue!

Here are my suggestions. For context (because we discussed this process in the IP), I have not looked at Madison's recommendations, though I did read Krystle's comment which mentioned two specific terms in Madison's recommendations.

Questionable

Adult, child, teen (can be problematic when it comes to determining who is/is not an adult, cultural differences in expectations, legal definitions, etc)
Buddha: misidentification could be particularly offensive?
Temple and shrine: ditto Buddha
Doctor/nurse, mostly because of the gendered associations where doctors are assumed to be men, nurses assumed to be women; misidentification can easily perpetuate this prejudice. May be able to include it in the future if we do a manual review of works with this tag

Surgeon too?

Hippie? May have political connotations?

Exclude

Hoe (this is on our sensitive terms list; we can add this back if we refine the sensitive term detection to ignore machine generated tags?)
Woman/Man
Male/Female
Boy/Girl
Lady (neither Lord nor Gentleman are present, so there isn't a male-gendered equivalent to exclude for this one)
Bride/Groom
Bridegroom
Bridesmaid
Fireman (weirdly gendered when firefighter exists?)
Monk (gendered in most contexts):

From Wikipedia: 'The Greek word for "monk" may be applied to men or women. In English, however, "monk" is applied mainly to men, while nun is typically used for female monastics.'
Also "nun" is not in the list?

Tribe

In an Australian context, tribe can be inappropriate or offensive, mob is preferred. Reference: https://www.ipswich.qld.gov.au/__data/assets/pdf_file/0008/10043/appropriate_indigenous_terminoloy.pdf; see the "No more classifying cultures" section.
In a North American context, there are similar complexities, although tribe is sometimes appropriate (comes from self-description). Reference: https://americanindian.si.edu/nk360/informational/impact-words-tips

Notes and observations

Back, from Person Description: 'back' on its own is ambiguous, I wonder how this comes across in presentation
In damange detection, many tags exist like "Car Window - Broken", that will be filtered out by out other tag filtering (we exclude tags with - in this case)
Could gendered terms that have reliable gender neutral alternatives be manually mapped in the future? For example:

Bride and groom -> Betrothed (popular alternative, but there isn't a totally straightforward and widely used one); another alternative (which I don't like much) could be to join them "Bride/Groom" to avoid specifying one gender; we could also use it as a clue towards a neutral but thematically relevant tag like "marriage", although "wedding" is present, so marriage may be duplicative, even though "wedding" is the event and "marriage" is more of a conceptual theme.
Fireman -> Firefighter
Doctor and nurse -> Healthcare provider, though this isn't exactly one-to-one, a doctor is a different thing than a nurse, but the fact of whether an image depicts one or the other is probably in the upstream metadata if it's important.
Monk/Nun -> ???

GPS and LED are capitalised, but not ATM (it is "Atm") or PC ("Pc"), weird
Iphone, Ipod are incorrectly stylised, even though iPod Shuffle is correctly stylised (with lowercase i, capital P).
"RAM Memory" is pretty annoying, the duplication of "memory"!

@AetherUnbound Can you include explanations for the terms you've suggested to exclude and ones you felt were questionable? There is some overlap between our questionable/exclude lists, and it looks like we definitely covered the umambiguously gendered terms in our exclude lists between the two of us (for which I also didn't add an explanation because they are clearly gendered). I'm very curious about some of the others, you recommended far more than I did, in particular those related to babies, childbirth, and hair style/colour.
Regarding age: I had adult, teen, and child in the questionable category, but didn't include baby or senior citizen. Baby is, I think, rather unambiguous, and I couldn't think of any culturally sensitive reasons to exclude it like I could for adult, teen, and child. What's was your reasoning? My thought process was similar for senior citizen.
For "Ballerina", I didn't include it in my lists because I thought it was gender neutral, but it looks like it's a good candidate like "fireman" for us to transpose to a gender neutral term. Fireman can easily be turned into firefighter, and ballerina could easily be "ballet dancer". I think these are unambiguous gender neutral alternatives to those words. I noted some other possible terms like this but none were as clear as firefighter and ballet dancer.
Regarding "barbie": what is the reason to exclude it? It comes from the toys category of labels, I'd expect it to only refer to the barbie style toy. Were you operating from a different assumption?
I missed "Exchange Of Vows" in my review, but likewise cannot find a reason on my own to exclude it, so would appreciate an explanation of your thought process there.

Thanks for providing your own list and notes @sarayourfriend! To answer you and @krysal - my inclusion of the other hair style/color labels were because they seemed to apply to the broader notion of "demographics". I don't feel particularly strongly about them (I was merely surprised to see them as part of the labels), so I'd be comfortable including them and removing them from my exclusions list.
As for the age related ones, "Age" was one of the categories we discussed excluding explicitly in the IP. Even though I agree that baby is fairly ambiguous, there are still some cases that might be questionable or mislabeled (would an adult in baby clothes or a diaper be labeled "baby"? I'm not sure it would). On the other side, there may be conditions that might mistakenly label a child or young adult as a senior citizen (progeria, for instance). Given the nuance there, I do think it's best to avoid age terms entirely.
"Barbie" also seemed like a gendered term to me but if we consider it as referring to the toy itself then perhaps it's not something we need to consider.
I hadn't thought about mapping terms though! That gives us an interesting opportunity to capture some of these using a more appropriate term; I'm all for it and I like your suggestions.
But your notes also bring forth an assumption I didn't realize I was making: I thought we might make all of the labels lower case before inserting them into the catalog. There indeed appears to be some information encapsulated in the capitalization though, so perhaps that may not always work. The reason I was thinking of doing this is that our tag collection endpoint is case-sensitive, and the more generic nature of these labels would be better represented without casing. Additionally, the Clarifai tags are all lower-case. What do you think? Perhaps we can lower case them if they are Caps Case, and leave as-is all other cases? (e.g. GPS or iPod)

I feel strongly that we should not lowercase everything. Aside from brand names, there are proper nouns in these tags that shouldn't be generically lower cased like "Buddha", "Christ the Redeemer" (referring to the monument in Rio de Janeiro).
The tags endpoint is case sensitive, and whether it should or shouldn't be is not a question relevant to the metadata in the catalogue, as far as I'm concerned, it's a question about how we index and search that metadata. The catalogue only provides that metadata, it does not dictate the indexed format of it with respect to search, and such a separation would be inherently and needlessly limiting to the flexibility of search. We don't need to make a decision like "lowercase all incoming tags" in the catalogue (there's no technical reason the catalogue of openly licensed works needs to do that) and whether search does that or not doesn't have anything to do with the catalogue. In fact, the catalogue should enable whatever option deemed necessary for search, whether search treats tags as case sensitive or not. Doing a blanket transformation like lowercasing the tags in the catalogue removes that flexibility from search.
I don't think we should do anything other than correct what we deem to be orthographic errors, whether those are errors like "Ipod" where the capitalisation is plainly incorrect or "GPS" and "Atm", where the error is in stylistic consistency across the incoming dataset, rather than in the single instance. I'd only say that about enrichment metadata, not provider tags, for what it's worth. We do have full control here, so while I think we shouldn't do things like lowercase all the tags in the catalogue, correcting orthographic errors or inconsistencies does seem worthwhile.
Whatever the clarifai tags are doing or did doesn't hold much sway here, I think. We didn't make decisions about that and don't know the context of them, and there's no direct reason to choose to be consistent with something we cannot explain or even know the provenance of.
On the other hand, I don't think that applies to how we do the inclusion check. For the purposes of checking whether a label should be included, I think it's fine to use a separate list of normalised labels so that e.g., using a Python of the labels makes the check O(1) for each label. To clarify, I'm only talking about label storage in the catalogue, not saying we need to at all times treat the labels in their original cases. In fact, this would solve the problem of needing to make sure our comparison of labels for the purposes of inclusion/exclusion does not affect how we fix orthographic errors, which we should do after the check (so that our orthographic fixes aren't limited by needing to keep characters in the same location in the string to use the upstream lists) but at least reduces the cognitive overhead of wondering whether those orthographic differences could affect the include/exclude logic.

if we consider [Barbie] as referring to the toy itself

I don't think it's whether we consider it to be that. Rekognition says that is the case, the label is in the toys category.
Regarding the age labels, I agree, let's filter all of them. For the hair, I might agree with hair style but I'm not sure about hair colour. On the other hand, I also don't think it's important enough to argue for including them (I'd just as well exclude all of these tags in favour of other ways of enriching our metadata that don't rely on machine vision) so let's filter them as well.
Can you explain your rationale behind excluding "Exchange of Vows"?
The exclude list would be then, your list, less Barbie, plus the addition of tribe, and hoe from my list? And then whether to exclude exchange of vows depends on understanding the rationale for it?
Also: reading the Rekognition docs, it looks like the label lists we were looking at might be different from the labels used by Rekognition at the time that they processed the dataset in the grant. That's based on this section of their documentation on aliases:

In previous versions, Amazon Rekognition Image returned aliases like 'Cell Phone' in the same list of primary label names that contained 'Mobile Phone'. Amazon Rekognition Image now returns 'Cell Phone' in a field called "aliases" and 'Mobile Phone' in the list of primary label names. If your appliction [sic] relies on the structures returned by a previous version of Rekognition, you may need to transform the current response returned by the image or video label detection operations into the previous response structure, where all labels and aliases are returned as primary labels.

(Emphasis mine)
I think this includes an assumption that labels are being added and removed, and that they might not be in the current list of tags which we've reviewed in this issue. Two things come from this:

I'd assumed based on the way we are talking about this (especially because we will create a list of excluded labels rather than included labels) that we will basically do something like a "if label not in excluded_labels" check to see if we should include them. We need to reverse this and instead create an explicit inclusion list, so that if there are any differences in the labels Rekognition was using at the time of the grant and the list that we reviewed, we won't be caught off guard. Only the labels we reviewed and said we want to include will be there.
On top of that, we should do an additional check, "if label not in included_labels and label not in all_reviewed_labels" to flag and store that in a reviewable manner (where all_reviewed_labels would be the list of labels we've reviewed in this discussion). This will make it possible to make absolutely certain that labels added to our catalogue are absolutely reviewed. We could do something like add an unreviewed note on these tags so they can go into the catalogue but be easily filtered out for the purposes of search, also allowing us to avoid needing to rerun the Rekognition ingestion process if for some reason that turns out to be arduous.

my inclusion of the other hair style/color labels were because they seemed to apply to the broader notion of "demographics". I don't feel particularly strongly about them (I was merely surprised to see them as part of the labels), so I'd be comfortable including them and removing them from my exclusions list.

@AetherUnbound Thank you for clarifying. Regarding Curly Hair at least, I see it as quite neutral; it can apply to a person regardless of gender or age, so I find it a bit strange that it is excluded, but if we go ahead on this line with hairstyles and broad demographics I found a few terms worth for considering excluding them too:

Exclude

Bun (Hairstyle)

Mohawk Hairstyle

Mustache

Alien

Hippie

I reviewed the Person Description, Profession, Symbols, and Flags categories, as these seemed the most likely to contain something related to the indicated criteria. If labels in Religion are all considered potentially sensitive (there are only 11), it might be better to exclude them altogether as well.

FWIW I'm in favour of a broader list of exclusions, with the knowledge that we can easily reverse any of those decisions or change how we decide if a label is in/out using more context in the future. We won't make search worse by excluding "too many" labels or anything like that, and I suspect there may be greater value from using the labels in context with the existing metadata for many works than in isolation anyway.

That's some really solid logic around keeping capitalization (especially given our thoughts around how we're treating the catalog), thanks for expressing that! IIRC, our search is case insensitive which is certainly what matters the most, so I agree that we don't need to alter the casing (except in the "errors" you've pointed out).

Can you explain your rationale behind excluding "Exchange of Vows"?

Again to me, this felt like it had the propensity to be gendered. But I concede that it is more neutral, and I'm fine with leaving it out of the excluded labels.
And thanks too for your notes about the actual inclusion vs exclusion logic. I like the idea of having an unreviewed list as well! I am realizing we don't have an issue for the actual implementation of the exclusion/inclusion list, so I'll make that and link to this discussion.

The exclude list would be then, your list, less Barbie, plus the addition of tribe, and hoe from my list? And then whether to exclude exchange of vows depends on understanding the rationale for it?

If labels in Religion are all considered potentially sensitive (there are only 11), it might be better to exclude them altogether as well.

I hadn't considered using the categories as a way of doing more blanketed exclusions! That's a great idea Krystle, and I'm all for it. So to summarize, the final list is:

My exclusions (minus Barbie and Exchange of Vows)
Tribe and Hoe from Sara's list
All labels with the Religion category
The hairstyle additions from Krystle

Which would mean this is the full exclusion list:
   Final Exclusions

Adult
Alien
Baby
Baby Crawling
Baby Laughing
Ballerina
Beard
Bishop
Bishop
Blonde
Blue Hair
Boy
Bridal Veil
Bride
Bridegroom
Bridesmaid
Brown Hair
Buddha
Bun (Hairstyle)
Child
Childbirth
Crucifix
Curly Hair
Family
Female
Girl
Green Hair
Hippie
Hoe
Lady
Male
Man
Mohawk Hairstyle
Mustache
Newborn
Pink Hair
Pope
Prayer
Prayer Beads
Priest
Red Hair
Senior Citizen
Shrine
Teen
Temple
Tribe
Woman

And the orthographic/gendered corrections would be:
   Corrections made during insertion

Atm -> ATM
Pc -> PC
Iphone -> iPhone
Ipod -> iPod
Fireman -> Firefighter

@krysal @sarayourfriend @zackkrida, does the above look right?
Another thing that this has me thinking...I think I may have been assuming going into this that we were going to exclude the labels as they were being added to the catalog so they never even made it in[^1], but given the approach we've been taking with the catalog as a data warehouse, I'm not sure that's the best move anymore. What do you think? (CC @stacimc as well for just this particular paragraph in case you don't want to load in context for the rest of the convo!)
[^1]:
From the IP:
> This section describes the criteria used for determining which machine-generated tags we should exclude when adding any new tags to the database, and what the minimum accuracy cutoff for those tags should be.

I think I may have been assuming going into this that we were going to exclude the labels as they were being added to the catalog so they never even made it in

I definitely missed this detail from the IP and at the time would have pushed to include them in the database and filter them at the filter data step of the data refresh: indeed, to stay consistent with the data warehouse approach. However, the stakes are far lower if we don't plan to the excluded tags for any of the ideas we've discussed about them in the near-term and are making sure to keep the dataset in S3 (as recently clarified in the project thread); we can re-load the tags with new parameters and logic at any time, in that case. It would seem more consistent to have a single place where we filter data though :slightly_smiling_face:

I'll go ahead and modify the IP one more time to include an explicit note that we'll be inserting all available Rekognition data, but filter it during the data refresh process. Then I'll make an issue based on that to ensure that work is captured, with clarity on using an inclusion-based filter while taking note of the "unreviewed" labels that the filter encounters.
I'll also go through the Rekognition list one more time to see if there are any other capitalization/orthographic errors we'll need to mitigate.

The other note I brought up was about tags with hyphens in them...They'll get excluded in their current form by the filter data step.

I'm actually not sure this is the case; we don't have any code in the current filter step that removes tags that includes hyphens as far as I'm aware, nor in the enrich tags portion of the MediaStore. I'm not sure we'd want to remove tags with hyphens in them anyway. We could add the "damage detection" category as a whole to the list of exclusions though, that might make the most sense.

I've gone through and identified a few other corrections:

Atm -> ATM
Atv -> ATV
Bmx -> BMX
Cpu -> CPU
Dj -> DJ
Dvd -> DVD
Ipod -> iPod
RAM Memory -> RAM
Pc -> PC
Rv -> RV
Suv -> SUV

I'll go ahead and modify the IP one more time to include an explicit note that we'll be inserting all available Rekognition data, but filter it during the data refresh process. Then I'll make an issue based on that to ensure that work is captured, with clarity on using an inclusion-based filter while taking note of the "unreviewed" labels that the filter encounters.

Perfect, thanks for looking for those orthographic errors too.

we don't have any code in the current filter step that removes tags that includes hyphens as far as I'm aware

I misremembered the : exclusion as a hyphen exclusion, apologies!
https://github.com/WordPress/openverse/pull/4684/files#diff-ca2fd64dcf026d3a3b82504d64d590d63ed8857e67e2c0e3fb6f189b7ba8ffc5R42

We could add the "damage detection" category as a whole to the list of exclusions though, that might make the most sense.

Fine by me.
I mentioned this privately and forgot to share it here: shall we also exclude the expressions ("Expressions and Emotions" category), based on Lisa Feldman Barrett's research on the accuracy of human emotional perception based on facial characteristics (video of Barrett discussing the topic)? I'd particularly wonder about the negative emotions, due to the significance of stigma and gendered aspects of those judgements, but we can keep it simple at the start by excluding them altogether?
For anyone reading to give context worried about the expansiveness of the exclusion list, there are 3082 labels in Rekognition's current data set, and we're talking about excluding around 80 labels, so roughly 2.6% of the possible labels. In other words, a minuscule amount, and which says nothing about the actual effective number of labels we're excluding from the real dataset of labelled images, which may not encompass all 3082 available labels. Besides that, I believe we have justified these exclusions under the conditions discussed in the implementation plan.

That's a fair point about the expressions & emotions too - I'm also fine excluding those. I've added the final list of exclusions & corrections to the issue description at the top. I feel like we've reached a good consensus on this, which is exciting! 😊


build: exclude ui from test-invoke-pip

Prior to the folder restructure, the paths for test-invoke-pip did not include the UI's path invokeai/frontend/:
    paths:
      - 'pyproject.toml'
      - 'ldm/**'
      - 'invokeai/backend/**'
      - 'invokeai/configs/**'
      - 'invokeai/frontend/dist/**'

After the restructure, more code was moved into the invokeai/frontend/ folder, and paths was updated:
    paths:
      - 'pyproject.toml'
      - 'invokeai/**'
      - 'invokeai/backend/**'
      - 'invokeai/configs/**'
      - 'invokeai/frontend/web/dist/**'

Now, the second path includes the UI. The UI now needs to be excluded, and must be excluded prior to invokeai/frontend/web/dist/** being included.
On test-invoke-pip-skip, we need to do a bit of logic juggling to invert the folder selection. First, include the web folder, then exclude everying around it and finally exclude the dist/ folder

Hmm, do you mean because now there is no successful run for this PR and it can't breed merged?

Just tried to add - '.github/workflows/*' to test-invoke-pip-skip.yml myself, but no permission:
git push
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 8 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 992 bytes | 992.00 KiB/s, done.
Total 5 (delta 4), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To github.com:psychedelicious/InvokeAI
 ! [remote rejected]   pr/psychedelicious/2892 -> pr/psychedelicious/2892 (permission denied)
error: failed to push some refs to 'github.com:psychedelicious/InvokeAI

Since "Maintainers are allowed to edit this pull request." is enabled, it seems like I am no maintainer of this repository anymore 🤷🏻♂

mhhhh- I think I got it - since you changed from paths-ignore to paths, you should include ** first to include everything which doesn't get excluded afterwards 😅

This makes sense, I will update the pr tonight :)

Let me know when this is good for a merge.

I'm assuming that the last fix is still pending?

Correct I didn't get to it yet.

@mauwii, I agree with your suggestions and have added, re-request review pls just as a sanity check


Redirect console. to OutputDebugString to make it available in VS's output window

This only works only when doing native debugging.

Hi @Almost-Done, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Cristian Petruta). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;


Getting rid of a linking error

Also include initguid.h.
#include <initguid.h>.
Otherwise I was getting "unresolved external symbol _GUID_DEVINTERFACE_USB_DEVICE"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: dc07324f-bf4f-8857-2ea3-3539a586fb27
Version Independent ID: 63179fd5-2bd5-ef2c-b180-419340116d29
Content: GUID_DEVINTERFACE_USB_DEVICE
Content Source: windows-driver-docs-pr/install/guid-devinterface-usb-device.md
Service: unspecified
GitHub Login: @tedhudek

Thanks for the feedback, @lihas, are you suggesting that we add initguid.h to the Requirements section of this page?

Thanks for pointing that out. I have added the header to the requirements section.

Oh man, saved me a lot of headaches :)


ZEVA - credit application error count

Describe the Bug
The error count within the error type filter drop down are not accurate
Expected Behaviour
As an IDIR analyst processing a credit application on the VIN detail page, the drop down list to filter by error type includes a count of each error type found in the application. This should be a count of each error contained in the entire application, not just the current page.
Actual Behaviour
The count is not accurate to the actual amount of specific errors found.
Implications
The count is not helpful if it isn't correct.
Steps To Reproduce
Steps to reproduce the behaviour:
User/Role: IDIR\analyst

Go to credit application
Click on Verify with ICBC Data
Click drop down to Filter by Error Type
Seer error of counts contained within brackets next to each error type

Acceptance Criteria

[ ] Given I am an IDIR user or an Analyst viewing the error list dropdown; I should see the correct number displayed on each error for the whole application.

Developer Checklist

[ ] Check where the error numbers are coming from (Do we store them in the backend or are they calculated on the front)
[ ] Update the logic to correctly display the error count in the CreditRequestVinListPage.

Notes
This is probably an error pulling data, because the frontend is set up to default to 0 if there is nothing in the corresponding error variable.

Hey team! Please add your planning poker estimate with Zenhub @emi-hi @jig-patel @JulianForeman @tim738745

still getting some incorrect counts, Toyota example in test display a count of 9 (41) errors but there are only 8

@airinggov I'll take a look

@airinggov I took a look and the reason why you're only seeing 8 (41) errors is because, on the frontend, we do not repeat errors for the same VIN if they have the same error code. For VIN 5TDEBRCH7LS012489 on that page (the second line), there's actually 2 (41) errors (MY mismatch and Make mismatch), so the total of nine is correct. Perhaps in the "warning" column it should say "41, 41"?

@tim738745 oh interesting, so that is only with 41 errors, the rest should be fine. I think we can just leave it for now with that explanation, see what the business says

@airinggov yes, it should only be with (41) error codes, because, from what I can tell, ['INVALID_MODEL', 'MODEL_YEAR_MISMATCHED', 'MAKE_MISMATCHED'] errors all associated with the (41) error code, but every other error code is associated with only one error.


app crashes on Android 11 if i open camera

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Please add the crash logcat here.


Bug with LLM caching

This line seems to be incorrect: https://github.com/hwchase17/langchain/blob/1f248c47f36af986be7ea647a6af223c9bb34b2b/langchain/llms/base.py#L112. Should be either prompts = prompts[missing_prompt_idxs[i] or prompts = missing_prompts[i].

good catch, will fix and add unit test


Not working on NativeScript

throws n.on is not a function error.
i dont know if this plugin is wired only to work on browsers or if it has such dependencies . or should i use Nodify

We use a wrapper which imitates xhr request in node. That might be the issue, because it tries to use browser xhr functions? Not sure how NativeScript works, but Nodify could be a way of doing it.

yeah i am trying to edited the file to use axios (http request plugin for javacript). since it works on node and browser it would be best if you use it too. or the best option will be to leave the implementation to the user (make the request service dynamic)

agree, allow a dynamic request would be a good improvement. As this is open source, you can always start a PR to do so. When I used and wrote this, it was just planed for the browser. The node support comes through some ppl pointing me to the xhr2 lib.


python36Packages.pycountry: 1.17 -> 18.5.26

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/python3.6-pycountry/versions.
meta.description for python36Packages.pycountry is: '"ISO country, subdivision, language, currency and script definitions and their translations"'.

Checks done (click to expand)

built on NixOS

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 18.5.26 with grep in /nix/store/hdsr67932pmrkmcmzhvm3nkzcqaqvmbq-python3.6-pycountry-18.5.26

directory tree listing: https://gist.github.com/17e4c32a5f3e8c225f1191b77b767648

du listing: https://gist.github.com/760b05861f976abbd22abdeb2c216f19

Outpath report (click to expand)

Outpath difference report
30 total rebuild paths
8 package rebuilds
8 x86_64-linux rebuilds
8 i686-linux rebuilds
6 x86_64-darwin rebuilds
8 aarch64-linux rebuilds
First ten rebuilds by attrpath
paperwork
python27Packages.FormEncode
python27Packages.pycountry
python27Packages.sqlobject
python36Packages.FormEncode
python36Packages.paperwork-backend
python36Packages.pycountry
python36Packages.sqlobject

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/hdsr67932pmrkmcmzhvm3nkzcqaqvmbq-python3.6-pycountry-18.5.26 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A python36Packages.pycountry https://github.com/r-ryantm/nixpkgs/archive/dd3db5d70d20180847f6f3a032a640af2183fc0e.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/hdsr67932pmrkmcmzhvm3nkzcqaqvmbq-python3.6-pycountry-18.5.26
ls -la /nix/store/hdsr67932pmrkmcmzhvm3nkzcqaqvmbq-python3.6-pycountry-18.5.26/bin

Success on aarch64-linux (full log)
Attempted: python36Packages.pycountry
Partial log (click to expand)
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching 'src/pycountry' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'src/pycountry.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

Success on x86_64-linux (full log)
Attempted: python36Packages.pycountry
Partial log (click to expand)
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching 'src/pycountry' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'src/pycountry.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

Success on x86_64-darwin (full log)
Attempted: python36Packages.pycountry
Partial log (click to expand)
warning: no previously-included files matching 'src/pycountry' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'src/pycountry.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/hzx313bsqkhv8v2bm4wm7aprr2lgqxm1-python3.6-pycountry-18.5.26


Fix small misspelling in doc for OCSP_response_status

The manpage for OCSP_response_status has a small misspelling.

Checklist

[x] documentation is added or updated

Merged to master and 1.1.1, thank you!
924d041fe0 Fix small misspelling in doc for OCSP_response_status
2f1607989a Fix small misspelling in doc for OCSP_response_status


fix: bump deps

Just bumping some deps, most importantly scroll-doctor

I think it's only the addition of alpha channel stuff, but @magnuh knows more

css 1.8.0 includes the rgb token updates and a lot of deprecations... shouldn't need any changes. 🤞 😄


Win10: tensorflow.contrib.tfprof has no attribute 'model_analyzer'.

On a Windows 10 system, Python 3.5.2, TF 1.0. Runs two tests, fails the last one with this message:
======================================================================
ERROR: test_train_infer (seq2seq.test.pipeline_test.PipelineTest)
Tests training and inference scripts.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\work\Documents\seq2seq\seq2seq\test\pipeline_test.py", line 148, in test_train_infer
    train_script.main([])
  File "C:\Users\work\Documents\seq2seq\bin\train.py", line 271, in main
    schedule=FLAGS.schedule)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\learn_runner.py", line 106, in run
    return task()
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 459, in train_and_evaluate
    self.train(delay_secs=0)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 281, in train
    monitors=self._train_monitors + extra_hooks)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\deprecation.py", line 280, in new_func
    return func(*args, **kwargs)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 426, in fit
    loss = self._train_model(input_fn=input_fn, hooks=hooks)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 981, in _train_model
    config=self.config.tf_config) as mon_sess:
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\training\monitored_session.py", line 315, in MonitoredTrainingSession
    return MonitoredSession(session_creator=session_creator, hooks=all_hooks)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\training\monitored_session.py", line 601, in __init__
    session_creator, hooks, should_recover=True)
  File "C:\Users\work\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\training\monitored_session.py", line 428, in __init__
    h.begin()
  File "C:\Users\work\Documents\seq2seq\seq2seq\training\hooks.py", line 246, in begin
    opts = tf.contrib.tfprof.model_analyzer.TRAINABLE_VARS_PARAMS_STAT_OPTIONS
AttributeError: module 'tensorflow.contrib.tfprof' has no attribute 'model_analyzer'

----------------------------------------------------------------------
Ran 2 tests in 5.876s

FAILED (errors=1)

From what I've seen in other issues, tf.contrib support is not currently complete for Windows, so I'm guessing that's the problem.

yes, i have the same problem because tf.contrib is not ready for windows.

Yep. Sorry, there no windows support for now, until tf.contrib has windows support.


DEV: pull remove-open-orders logic into its own method

And test it.

Coverage decreased (-0.01%) to 81.464% when pulling fcc940e0acf96cf65d8c1f0ce5e424e54eb5d657 on really-remove-orders into e4934cca6fb980a68c9b411df515be3e60cf14f5 on master.

LGTM!  Just mentioned the docstring.

Coverage increased (+0.02%) to 81.548% when pulling 83d70f4a7081eff694ca445d8b3f8795d21be16b on really-remove-orders into 0ecf3a07d65cfd4f132210cc495d1dd4a3c72085 on master.


Cancelling a POST with a string body generates an asynchronous uncaught error

Description
When doing a POST request with a string passed in as the body, helix-fetch internally converts it to a Readable stream. When an ongoing request is aborted, the body stream is destroyed, sending along an error. The code that normalizes the body to a Readable stream adds a handler for the 'error' event, but currently does so only when the original body was a Readable stream. So if the original body is e.g. a string, then the Readable stream version of it will not have an error handler, and aborting the request will cause it to emit an 'error' event which is never caught, causing e.g. tests of the functionality to fail.
To Reproduce
Steps to reproduce the behavior:

run a test where you perform a POST fetch with string data
abort the request
get an error about an uncaught error, which happens after the test has already finished (this is because destroying the Readable stream schedules the error event on the next tick)

Expected behavior
The error should be caught (or not emitted at all).

Great catch! Thanks for filing this issue and the concise description.


Two-way model binding with TagHelpers doesn't seem to be working for

Win7
Brackets/Omnisharp
DNX: dnx-coreclr-win-x<IP_ADDRESS>-beta5-11572
MVC: 6.0.0-beta5-13567
New to MVC, so I might be to blame for this, but consider this markup for a <select>:
<th:select asp-for="department" size="3">
   <option>Sales Department</option>
   <option>Billing Department</option>
   <option>Technical Support</option>
</th:select>

I have other <input> elements on this <form> with asp-for on them, e.g.,
<th:input asp-for="email" type="text" />

The model binding for the <input> elements works both ways: I can specify in my controller that a value should be:
visitor.email =<EMAIL_ADDRESS>
It goes to the view ... I can change it in the browser ... submit that form and check the value in the controller ... sure enough ... it sets the visitor.email value correctly. That two-way binding is working fine.
However, the binding with the <select> is only going back to the controller. If I click on an <option> in the browser, the visitor.department value correctly reflects the chosen <option> in the controller on postback, but that value doesn't make its way back client-side after the postback: The element resets itself back to no selected <option>. This is also true if I explicitly set the value of visitor.department on an [HttpGet]:
visitor.department = "Billing Department";

... and send the model to the view: The <select> doesn't get a selected attribute on the "Billing Department" <option>.
So, am I using <select> model binding incorrectly here? ... or is there a model binding bug?

@GuardRex if you have configured your view to use a prefix such as "th:", you must use it for every tag helper.  given both <select/> and <option/> are associated with tag helpers, <option>Sales Department</option> looks incorrect in your description.  that should be <th:option>Sales Department</th:option>.  same for the other <option/> elements in your .cshtml file.

Ah! Thank you. I'll try that.


Remove noexTable::CheckAndMutateRow() and ReadRows()

Part of the work for #2097 .
Second request in a series of pull requests to move functions from noex:;Table to Table.

This change is

The changes look good, please rebase to pick the fixes for Windows. We can merge once the builds pass.


[BUG] OnnxBertBiEncoder throws an exception with specific data

Describe the bug
Calling:
var embeddingModel = new AllMiniLmL6V2QuantizedEmbeddingModel();
var splitter = DocumentSplitters.recursive(200, 0, new BertTokenizer());

Document document = Document.from("���");
List<TextSegment> segments = splitter.split(document);
List<Embedding> embeddings = embeddingModel.embedAll(segments).content();

Causes an exception.
Log and Stack trace
java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
        at java.base@21.0.2/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:100)
        at java.base@21.0.2/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:106)
        at java.base@21.0.2/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:302)
        at java.base@21.0.2/java.util.Objects.checkIndex(Objects.java:385)
        at java.base@21.0.2/java.util.ArrayList.get(ArrayList.java:427)
        at app//dev.langchain4j.model.embedding.OnnxBertBiEncoder.weightedAverage(OnnxBertBiEncoder.java:166)
        at app//dev.langchain4j.model.embedding.OnnxBertBiEncoder.embed(OnnxBertBiEncoder.java:67)
        at app//dev.langchain4j.model.embedding.AbstractInProcessEmbeddingModel.lambda$embedAll$0(AbstractInProcessEmbeddingModel.java:44)
        at java.base@21.0.2/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base@21.0.2/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
        at java.base@21.0.2/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base@21.0.2/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base@21.0.2/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.base@21.0.2/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base@21.0.2/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at app//dev.langchain4j.model.embedding.AbstractInProcessEmbeddingModel.embedAll(AbstractInProcessEmbeddingModel.java:45)

To Reproduce
See above.
Expected behavior
No exception.
Please complete the following information:

LangChain4j version: 0.26.1
Java version: 21
Spring Boot version (if applicable): N/A

Hi, I'm really not sure what the best way is. I'm actually not sure why a tokenizer would not support some symbols... but I guess there isn't much to do anyway so yes, just dropping such symbols is fine.

Tokenizer supports and model understands only those tokens/symbols which they have seen during training. Imagine you see an unknown to you symbol from some ancient language, you probably also just drop it because you don't know what to do with it and how to interpret it :)


Add lm-eval-service controller

refactor the existing TrustyAIService controller and add the LMEvalService controller

/cc @ruivieira

Thanks @yhwang, this is brilliant!
Just some notes:

There seems to be a build issue with the slices package
I've created a issue label lm-eval to track issues related to this project
There is a milestone and project tab too

I'm fine with merging as it is, so that we have a baseline on the dev branch and we can work from there, if you prefer to fix any issues in future PRs.

@ruivieira thanks for the review

There seems to be a build issue with the slices package

I just updated the workflow of the GH action to use newer version of go. That should solve the issue.

The support of using secrets as files or env vars was landed on the fms-lm-eval-service repo. So I quickly added them here too. If the build passes, I am good to merge this too.

have the last change to update the yaml linter config and fix the errors from the linter GH action. I think it's ready for merge!


Lots of unused code in cmd/fix

Maybe I'm missing something, but there seems to be a lot of unused code in cmd/fix, many of which I believe left over after this commit:
https://github.com/golang/go/commit/df93283d5694814efc97efd2132da11722d11523
➜  fix git:(master) gometalinter --deadline=60s \
             --disable-all --enable=deadcode \
             --exclude=vendor ./...
fix.go:335:1:warning: isPtrPkgDot is unused (deadcode)
fix.go:353:1:warning: isCall is unused (deadcode)
fix.go:359:1:warning: isIdent is unused (deadcode)
fix.go:373:1:warning: isBlank is unused (deadcode)
fix.go:378:1:warning: isEmptyString is unused (deadcode)
fix.go:383:1:warning: warn is unused (deadcode)
fix.go:393:1:warning: countUses is unused (deadcode)
fix.go:408:1:warning: rewriteUses is unused (deadcode)
fix.go:435:1:warning: assignsTo is unused (deadcode)
fix.go:467:1:warning: newPkgDot is unused (deadcode)
fix.go:787:1:warning: renameFix is unused (deadcode)
typecheck.go:136:1:warning: typecheck is unused (deadcode)

They do appear to be unused to me, after checking with git grep:
➜  src git:(master) git grep isPtrPkgDot
cmd/fix/fix.go:// isPtrPkgDot reports whether f is the expression "*pkg.name"
cmd/fix/fix.go:func isPtrPkgDot(t ast.Expr, pkg, name string) bool {
(END)

➜  src git:(master) cd cmd/fix
➜  fix git:(master) git grep "typecheck("
typecheck.go:func typecheck(cfg *TypeConfig, f *ast.File) (typeof map[interface{}]string, assign map[string][]interface{}) {
(END)

If they are in fact not necessary anymore, I can go ahead and make a CL removing them.

Go for it. Thanks.
For the future, please use "pkg: bug description" format for issue titles. It helps with triage and tools. I'll fix this one now.

Thanks @josharian. CL is coming soon.

For the future, please use "pkg: bug description" format for issue titles. It helps with triage and tools. I'll fix this one now.

Thanks, will remember that for the future.

Please do NOT delete all that code. That's unused because there are relatively few fixes. It's still useful code. Think of cmd/fix as a library with a bunch of plugins all linked into the same package. The fact that there are few plugins doesn't mean you delete the library.

@rsc okay. I'll abandon the CL