Add readme link to doc for publishing images

I couldn't find these docs at all, someone who publishes images kindly linked me to them (/ht @dgrisonnet).
This repo was the first place I checked for documentation, since I thought registry.k8s.io was replacing k8s.gcr.io. https://registry.k8s.io redirected to this repo which only discusses the proxy implementation and not the registry itself.

/approve
/lgtm

xref: https://github.com/kubernetes/enhancements/pull/3723
backing registry went from k8s.gcr.io to registry.k8s.io specific artifact registry instances, with https://github.com/kubernetes/enhancements/pull/3723 we will be refactoring to phase out publishing to k8s.gcr.io and move to a registry.k8s.io specific set of image promotion manifests (kubernetes-sigs/promo-tools) in kubernetes/k8s.io.
previously not documented here at all, because it's an internal detail we don't want end users depending on, but it makes sense to link to how we publish. we'll need to update the link following  https://github.com/kubernetes/enhancements/pull/3723


Data Explorer SQL getting team names

Hello,
I'am trying to get pro matches results using Data Explorer with SQL, but for some reason team names from matches table are null's. Am I doing something wrong or is it a bug?
Here is my SQL code:
SELECT

matches.match_id,
(matches.radiant_win) win,
matches.radiant_team_id,
matches.radiant_team_name,
matches.dire_team_id,
matches.dire_team_name
FROM matches
WHERE TRUE
AND matches.start_time >= extract(epoch from timestamp '2019-12-31T22:00:00.000Z')
LIMIT 50

The result I get:

Thanks in advance

You can try joining with the team table to get the names. I don't think they're stored in the matches table, as team names sometimes change.

Ok, thanks for explanations!


Create README.md for DP-1.12: ECN enabled traffic test

Convert QoS test case to README

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 46.471%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
586

Relevant Lines:
1261

💛  - Coveralls


DX-678 Added NewBillingTelephoneNumber to Portins model

Overall looks good but I will let those better at python look at it too.


feature: have a way to load lazy.nvim even when --noplugins is used

Did you check the docs?

[X] I have read all the lazy.nvim docs

Is your feature request related to a problem? Please describe.
I am going from my previous vimscript-based nvim config to a lua-based one,
and wanted to move to more modern plugins, especially lua ones,
so I decided to try Lazy as my plugin manager.
Since my config is quite extensive and I'm using a number of plugins, as I'm
writing my new config I've been launching nvim with nvim --noplugins -u new-conf.lua
(I will then rename the file to init.lua and remove the old init.vim)
so that I can test things in isolation and see what is missing/broken.
In my config I explicitly require various lua modules that make up most of my
configuration, including:
require('plugins')

Where plugins.lua (in ~/.config/nvim/lua/) is just using lazy with
the suggested bootstrap code:
local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    "git",
    "clone",
    "--filter=blob:none",
    "https://github.com/folke/lazy.nvim.git",
    "--branch=stable", -- latest stable release
    lazypath,
  })
end
vim.opt.rtp:prepend(lazypath)
require('lazy').setup({})

I was pretty surprised when, well, nothing worked, with :Lazy resulting
in Not an editor command: Lazy, and :checkhealth lazy reporting
lazy.nvim
- OK Git installed
- OK no existing packages found by other package managers
- OK packer_compiled.lua not found
- ERROR No plugins loaded. Did you forget to run `require("lazy").setup()`?

After way too long (maybe a couple hours spread across a few busy days)
I finally just opened lazy's init.lua and quickly realised what is happening,
noticing the check in the setup function:
if not vim.go.loadplugins then
  return
end

It was quite unexpected since I haven't seen this mentioned anywhere in the docs,
though perhaps I've missed something to that effect.
Describe the solution you'd like
Since Lazy already won't be autoloaded if --noplugins is used,
it being loaded means that the user is explicitly requiring it
(from some loaded config file).
I could understand not loading any Lazy-managed plugins if the option is used,
but it would make sense if Lazy itself would still initialise (so that e.g. the
user can manage plugins (installation, removal etc) while running without plugins.
At the very least, it would be nice if setup had some option to configure
this behaviour, so that Lazy can be tested out with --noplugins when
running on an existing installation with some mess left to clean up.
It would also be nice to highlight this in some FAQ/troubleshooting entry.
Describe alternatives you've considered
Currently I just manually disabled the check (by commenting it out) in Lazy's
init.lua, so that I can continue my testing.
Additional context
No response

no

Can this behaviour at least be documented? It feels somewhat unusual that a plugin I explicitly load
is disabling itself by looking at other settings.

You can load plugins manually by :source command.

The problem is that even if I source lazy’s init.lua, it won’t load anything if nvim has been launched with --noplugin, since it explicitly checks for that.

Yes.  You need to source other plugins file manually.

In this particular case I didn't care about loading other plugins (I didn't even have any!),  I just wanted to load Lazy itself.

Ah, I get it.

After way too long (maybe a couple hours spread across a few busy days)
I finally just opened lazy's init.lua and quickly realised what is happening,
noticing the check in the setup function:

It seems lazy's feature.


EDL cmx_3600 adapter does not support unnamed markers.

This is the content of an cmx_3600 EDL exported from Avid media composer:
TITLE:   Untitled Sequence.01
FCM: NON-DROP FRAME
001  103043_0 V     C        01:00:00:00 01:00:08:10 01:00:00:00 01:00:08:10
* FROM CLIP NAME:  MYMEDIA.MOV
* LOC: 01:00:02:01 RED

As you can see the marker does not have a name
Current behavior:
>>> timeline = otio.adapters.read_from_file('markers.edl')
>>> list(timeline.each_clip())[0].markers
[]

Expected behavior:
>>> timeline = otio.adapters.read_from_file('markers.edl')
>>> list(timeline.each_clip())[0].markers
[
otio.schema.Marker(name='', marked_range=otio.opentime.TimeRange(start_time=otio.opentime.RationalTime(value=86449, rate=24), duration=otio.opentime.RationalTime(value=0, rate=1)), metadata={'cmx_3600': {'color': u'RED'}})
]

From what I saw this is due to the regular expression that matches locator comments in cmx_3600.py line 457.
Should OpenTimelineIO support unnamed markers ? I feel like it should since some editing systems are able to export unnamed markers. Let me know what you think !
My corporation is waiting for confirmation that our CLA was received. We will submit a PR as soon as possible.

Made a PR:
https://github.com/PixarAnimationStudios/OpenTimelineIO/pull/598

Fixed by #598


Leaks admin secrets to log files [JIRA: RCS-379]

Riak CS logs the admin secret keys, which potentially leaks the secret keys to unprivileged users.
In our case, we use Riak CS in production, and the log files are collected and sent to a Logstash cluster.
Only a few sysadmins are able to view the /etc/riak-cs/ configuration files, however the Logstash logs are viewable by most staff. We didn't expect the logs to contain admin secrets.
We can work around this by filtering the secrets out, but I thought this behaviour was wrong enough to deserve me opening an issue here -- at least you can discuss amongst yourself if this is intentional and desirable behaviour.
Maybe at the very least, consider changing the log level for this to be DEBUG rather than INFO?

https://github.com/basho/riak_cs/pull/1279 helps? (I don't know the fix has been released or not.)

It might help, but I'm confused - you never updated http://docs.basho.com/riak/cs/2.1.1/cookbooks/configuration/riak-cs/ to reflect the current correct and best practices in regard to the admin secret. It still says to put both admin.key and admin.secret into the riak-cs.conf file.

https://github.com/basho/riak_cs/commit/64eb8dea5f474355408b67309b836f10cb5bdfdc is not included in  2.1.1 and not just yet released as far as I see git tags after months absence.


session ideas

satellit book club videos https://www.youtube.com/playlist?list=PLQy8yXTKzcPIyhqfvpjqquO5nm7VCMe4A
https://www.youtube.com/playlist?list=PLJzZC8w9vPV3kIBVNmQYzZfHO6vGZeNhN
https://www.youtube.com/playlist?list=PLJzZC8w9vPV3kIBVNmQYzZfHO6vGZeNhN
https://appliedsciences.nasa.gov/get-involved/training/english/arset-biodiversity-applications-airborne-imaging-systems
https://rammb2.cira.colostate.edu/training/visit/
https://rammb2.cira.colostate.edu/training/2024_ams_satellite_short_course/
Amy Huff - NOAA Affiliate<EMAIL_ADDRESS>| Amy Huff - NOAA Affiliate<EMAIL_ADDRESS>Thanks for reaching out about the AMS Short Course that my colleague Rebekah Esmaili (cc'd) and I taught at last year's AMS meeting.  All of the Notebook files, .yml configuration file, and background info are available via Rebekah's github (STAR won't let me use github for end users): https://github.com/modern-tools-workshop/AMS-python-workshop-2023
Modern Programming Tools for Earth Science
https://github.com/modern-tools-workshop
Python for Remote Sensing: Analysis, Visualization, and Workflow for Earth Scientists
https://www.rebekahesmaili.com/workshop.html#overview
Earth's Radiation Budget
https://github.com/modern-tools-workshop/UMD-ERB-tutorial?tab=readme-ov-file
Python for Earth Sciences
https://github.com/modern-tools-workshop/AGU-python-workshop-2022

Learn Dask
https://docs.google.com/presentation/d/1HjOXgvlGcq01srJF5RD7y5sx5u8pXJzHb1E5W7s9aKc/edit#slide=id.g270e53e38ea_0_16

Hyperspectral data in R
https://www.neonscience.org/resources/learning-hub/tutorials/introduction-hyperspectral-remote-sensing-data-r

https://www.jstatsoft.org/article/view/v043i04

Go throuth this book
https://earth-env-data-science.github.io/intro.html

Sea ice
https://github.com/COSIMA/


New theme: techlog-simple

Theme submission
Before submitting a theme please make sure you've read the README.md. It describes in detail what information are needed and which requirements have to be fulfilled for the submission. The check list below can help you.
Link to my theme repository: [https://github.com/mazgi/hugo-theme-techlog-simple]
I made sure that...

[x] the repository contains a good README.md describing my theme
[x] an open source license has been added to LICENSE.md
[x] all metadata have been added to theme.toml
[x] screenshots have been added in the images/ folder with the required dimensions
[x] in case I'm using a customized demo via the exampleSite folder that

[x] https://example.com is set as base url in exampleSite/config.{toml, yaml, json} to avoid the abuse of unused domains
[x] the site's content is as neutral as possible (contains no ads etc.) and reflects the features of my theme

[x] I tested my theme against the gohugoio/HugoBasicExample

[x] I've checked the developer tools' console in my browser for error messages

[x] in case my theme is using Hugo Pipes features like toCSS and PostCSS that I have committed the /resources directory with all generated assets, for my theme to work in the basic version of Hugo

Feel free to ask questions. We're glad to help.

Repository: https://github.com/mazgi/hugo-theme-techlog-simple
Demo: https://mazgi-hugo-themes-gcp.firebaseapp.com/techlog-simple/

Hi @mazgi and thank you for your theme submission.
Your Example Site renders when I execute the Hugo Themes Build Script 👍
However you need to do the following changes to take your submission further:

You need to include some basic information about your theme in the README not just the link to your demo, such as a brief description about its features and also a link to your theme's license. See here for an example: https://github.com/budparr/gohugo-theme-ananke/blob/master/README.md
Your themes screenshots do not have the correct dimensions and also you should make sure to remove transparency. See: https://github.com/gohugoio/hugoThemes#media
You really need to remove the Amazon affiliate code links from the content files of your Example Site. We do not accept affiliate links on the Hugo website.

Please do the above and let me know so that I can look into your theme further.

Hi @onedrawingperday
Thank you for your review and advice.
I've just fixed it!
https://github.com/mazgi/hugo-theme-techlog-simple

Hello @mazgi
Thank you for making the changes! 👍
@digitalcraftsman
The Techlog Simple theme is ready to be added to the website.

Your theme should appear soon on Hugo's theme site. Next, I'll promote your theme on Hugo's official Twitter account.


Bug report(history-sync): Pop multiple and replace screen trigger history sync error.

Describe the Bug
Hi, I'm writing this issue because I encountered a bug while using a great library. I was using the history-sync plugin and when I pop multiple times and replace immediately through useFlow, history-sync throws an error.
// ...
                 <button
                    onClick={() => {
                        pop(2);
                        replace('Third', {})
                    }}
                >
                    Pop and replace Third
                </button>

Environment
OS: Windows 11 Firefox 111, Android Webview 111, iOS 16.3.1 Webview
package versions:
  "dependencies": {
    "@stackflow/core": "^1.0.4",
    "@stackflow/plugin-basic-ui": "^1.2.2",
    "@stackflow/plugin-history-sync": "^1.3.2",
    "@stackflow/plugin-renderer-basic": "^1.1.1",
    "@stackflow/react": "^1.1.1",
    "react": "^18.2.0",
    "react-dom": "^18.2.0"
  },

ScreenShot
https://user-images.githubusercontent.com/47362439/227893914-330ff318-53a4-4c78-a158-4825490697bb.mp4

stackflow popped too fast, so this code triggers coreActions.pop after popped and replaced. I want to know why this code exist for.
https://github.com/daangn/stackflow/blob/51df33ee6243ea8391a3feb0745d8a2af209607a/core/src/makeCoreStore.ts#L75-L85


Updates required for refinement and create_form

Currently broken due to changes in main.
Need to update for refinement interface and create_form

I took care of the refinement interface, not sure what is left to do about create_form. CI is passing, so I am closing this.


Support for 1.9

Spigot 1.9 will be released soon. We need to check incompatibilities and add new features support in 1.9.

It appears the "arrow" power does not seem to work in version 1.9.

The only real compatibility breaking parts of 1.9 seem to be Potions and Sound, both of which I have already adressed in Custom Enchantments. I believe adding such systems to RPGItems and fleshing them out some more (particularily the sounds) would be favorable as it would make the need for a seperate branch and release obsolete.
Just asking for a quick heads up as to whether such a combined version is wished or not :)

May be we can make 1.9 be the master branch and constantly backport the new features to 1.8 & 1.7 branch？

Maybe we can also bump the version to 3.6 once we've migrated to 1.9?

This sounds like a good idea, seeing that the 1.9 branch is already behind several commits.
I also considered using annotations to mark certain powers/features as version dependant (say an attack speed power would be bound to 1.9+ only and disabled in any non 1.9+ environments).

I don't think a multi-version-support code is necessary though it could be fantastic.
We need different versions of spigot/bukkit api to compile .java file for different mc-versions.

Alright :)
Any plans on how we could ensure easy backporting to 1.8/1.7? Most commits can easily be pushed to all necessary branches using cherry picking, although this should be done regularily.


Modernization

See the list of commits

Hey, thanks for taking interest in my humble project. I gave it a quick look and it looks good besides the Fody files. Since I am using something different, if you remove the .editorconfig and other fody files I will merge it.

@hasan-hasanov
Re: Fody:
are you using something else to add ConfigureAwaits? Can you commit it to the repo?
Re: .editorconfig
I found that public projects, that lack .editorconfig are difficult to work with because if their settings are different from the style defaults on the developer's machine, you get a lot of warnings for various rule violations. Also IDE does not autoformat the code correctly, so it needs to be manually formatted before committing.
This was the case for my config, so the .editorconfig I added actually has the rules that were inferred from this project, not my own rules.


problem installing

the latest version is not installing in my Obs. I uninstalled, and tried re-installing... now I have no BRAT = sad

Same

It says" Plugin ID mismatch" when installing.

@menagerie198 @brimwats @ChurchillDu  please try updating now and let me know how it goes.

@menagerie198 @brimwats @ChurchillDu please try updating now and let me know how it goes.

I have successfully installed. Good job!👍

Successfully installed = happy! Cheers


Gemini -(Kumo) Task created in a session that has been invalidated

DESCRIPTION
Fixing Task created in a session that has been invalidated crash
CHANGE

Changed Service from class to actor
Updated perform() with async & await

Running the tests ($ swift test) there are some warnings:
Service.swift:285:10: note: calls to instance method 'downloadResultToURL(url:response:)' from outside of its actor context are implicitly asynchronous
    func downloadResultToURL(url: URL?, response: URLResponse?) throws -> URL {
...
 warning: non-sendable type 'OperationQueue?' passed in call to nonisolated initializer 'init(baseURL:runsInBackground:logger:delegateQueue:configuration:)' cannot cross actor boundary
    let service = Service(baseURL: URL(string: "https://httpbin.org")!, logger: TestLogger())
...
Swift.Optional:1:21: note: generic enum 'Optional' does not conform to the 'Sendable' protocol
@frozen public enum Optional<Wrapped> : ExpressibleByNilLiteral {

I think we should address these?

Running the tests ($ swift test) there are some warnings:
Service.swift:285:10: note: calls to instance method 'downloadResultToURL(url:response:)' from outside of its actor context are implicitly asynchronous
    func downloadResultToURL(url: URL?, response: URLResponse?) throws -> URL {
...
 warning: non-sendable type 'OperationQueue?' passed in call to nonisolated initializer 'init(baseURL:runsInBackground:logger:delegateQueue:configuration:)' cannot cross actor boundary
    let service = Service(baseURL: URL(string: "https://httpbin.org")!, logger: TestLogger())
...
Swift.Optional:1:21: note: generic enum 'Optional' does not conform to the 'Sendable' protocol
@frozen public enum Optional<Wrapped> : ExpressibleByNilLiteral {

I think we should address these?

Fixed this warning


Fix coordinate handling and add unit tests

Fixes https://github.com/smueller18/TDKernelDMVW/issues/6
Includes and supersedes https://github.com/smueller18/TDKernelDMVW/pull/5
Thanks to @ernOho !

Please let me know if you prefer a pull request without the unit test.

Hi @cl90
thanks for your contribution.
Today, I ported the project to pytest and will migrate it to GitLab because I have a fully integrated CI solution. I will post the link after the migration. This project will then be archived.

You can find the migrated project at https://gitlab.com/smueller18/TDKernelDMVW.
I also created a pip package which is available at https://pypi.org/project/td-kernel-dmvw/.


(node/lsstcam-fcs.cp.lsst.org) deploy host as " EL7 "

Added a couple of commits to enable the ccs canbus module.
The Puppetfile bump will lead to a trivial merge conflict at some point, sorry.
It requires a private hiera change too.

Please squash 4db641e

Commit was squashed on to: 2274c33


re-opening batch update discussion

Feature discussion / request

Explain what is your use case
We have ways of bulk inserting and deleting, but not for updating. I know it's a more rare use case but it's quite a glaring omission for an essential library like knex. Previous discussions on the topic wre marked out of scope which I don't agree with. https://github.com/knex/knex/issues/1271

Explain what kind of feature would support this

a new function exactly like batchInsert but for updates
knex.batchUpdate

Give some API proposal, how the feature should work

We would use the CASE statement.
https://www.postgresql.org/docs/current/functions-conditional.html#FUNCTIONS-CASE
https://dev.mysql.com/doc/refman/8.0/en/case.html
I came across this discussion from the django community where the op does some benchmarks using CASE to update things and, unsurprisingly it is an order of magnitude faster than doing N update statements in a loop.
https://groups.google.com/g/django-developers/c/a5ADv59TkBQ
The code was added back in 2015. https://github.com/django/django/pull/3825
Here is a quick proof of concept using raw, but it would be better to be included in the library so other ORMs built on top could have acess to this feature
function generateBatchUpdateSQL(tableName, data, identifier = 'id') {
  const keys = Object.keys(data[0]).filter((key) => key !== identifier);
  const updates = keys.map((key) =>
    `${key} = CASE ${data.map((row) => `WHEN ${identifier} = ${row[identifier]} THEN '${row[key]}'`).join(' ')} END`
  ).join(', ');

  const condition = `${identifier} IN (${data.map((row) => row[identifier]).join(', ')})`;

  return `UPDATE ${tableName} SET ${updates} WHERE ${condition}`;
}

// Usage
const tableName = 'my_table';
const data = [
  {id: 1, name: 'John', age: 25},
  {id: 2, name: 'Doe', age: 35}
];

const sql = generateBatchUpdateSQL(tableName, data);
await knex.raw(sql);

I second this motion! 👍
We have several use-cases where we use AWS Lambda triggered from SQS message queues when we poll in messages in batch, that is e.g. 10 messages in an array. We need to perform a batchUpdate() on all ten messages, and it would save a lot of DB connections if we can do that in one go, instead of looping and transacting 10 times.
const batchUpdate = async (options, collection) => {
  const { table, column } = options;
  const trx = await knex.transaction();
  try {
    await Promise.all(collection.map(tuple =>
      knex(table)
        .where(column, tuple[column])
        .update(tuple)
        .transacting(trx)
      )
    );
    await trx.commit();
  } catch (error) {
    await trx.rollback();
  }
}

Found this nugget somewhere which kind-of does the job, but having a native function would of course be preferred...

since I'm not the only one, and I just came back from looking at this code again in my project, im gonna share my copypasta util that I use
import { Knex } from "knex"

export type RowWithId<Id extends string> = {
  [key in Id]: any;
} & Record<string, any>;

/**
 * Batch update rows in a table. Technically its a patch since it only updates the specified columns. Any omitted columns will not be affected
 * @param knex
 * @param tableName
 * @param id
 * @param rows
 * @param chunkSize
 * @param trx
 */
export async function batchUpdate<Id extends string>(knex: Knex, tableName: string, id: Id, rows: RowWithId<Id>[], chunkSize = 50, trx: Knex.Transaction = null) {

  const chunks: RowWithId<Id>[][] = []
  for (let i = 0; i < rows.length; i += chunkSize) {
    chunks.push(rows.slice(i, i + chunkSize))
  }

  const executeUpdate = async (chunk: RowWithId<Id>[], transaction: Knex.Transaction) => {
    const sql = generateBatchUpdateSQL(knex, tableName, chunk, id)
    return knex.raw(sql).transacting(transaction)
  }

  if (trx) {
    for (const chunk of chunks) {
      await executeUpdate(chunk, trx)
    }
  } else {
    await knex.transaction(async (newTrx) => {
      for (const chunk of chunks) {
        await executeUpdate(chunk, newTrx)
      }
    })
  }
}

/**
 * Generate a set of unique keys in a data array
 *
 * Example:
 * [ { a: 1, b: 2 }, { a: 3, c: 4 } ] => Set([ "a", "b", "c" ])
 * @param data
 */
export function generateKeySetFromData(data: Record<string, any>[]) {
  const keySet: Set<string> = new Set()
  for (const row of data) {
    for (const key of Object.keys(row)) {
      keySet.add(key)
    }
  }
  return keySet
}

/**
 * @param knex
 * @param tableName
 * @param data
 * @param identifier
 */
export function generateBatchUpdateSQL<Id extends string>(knex: Knex, tableName: string, data: Record<string, any>[], identifier: Id) {
  const keySet = generateKeySetFromData(data)
  const bindings = []

  const cases = []
  for (const key of keySet) {
    if (key === identifier) continue

    const rows = []
    for (const row of data) {
      if (Object.hasOwnProperty.call(row, key)) {
        rows.push(`WHEN "${identifier}" = ? THEN ?`)
        bindings.push(row[identifier], row[key])
      }
    }

    const whenThen = rows.join(" ")
    cases.push(`"${key}" = CASE ${whenThen} ELSE "${key}" END`)
  }

  const whereInIds = data.map(row => row[identifier])
  const whereInPlaceholders = whereInIds.map(() => "?").join(", ")
  const sql = knex.raw(`UPDATE "${tableName}" SET ${cases.join(", ")} WHERE ${identifier} IN (${whereInPlaceholders})`, [...bindings, ...whereInIds])
  return sql.toString()
}


Fix typo in Blog.scala

I was perusing the example and noticed a space missing, so I thought I'd fix it.

Thanks


fix profile navigation and event navigation

same as SS-navigation, fix with a switch-Screen to make the linker work

waiting for #128 to be merged to main


metatensor.torch.sort is broken

I leave it to @ppegolo for more details

ahahah it is two day that I am try to fix/improve sort

Broken for me too! Here's an MWE:
import torch
import metatensor.torch as mts

tensor = mts.TensorMap(
    keys=mts.Labels(
        names=["a", "b"],
        values=torch.tensor([[2, 1], [1, 5]]),
    ),
    blocks=[
        mts.TensorBlock(
            values=torch.randn(3, 2),
            samples=mts.Labels(
                names=["s1", "s2", "s3"],
                values=torch.tensor(
                    [
                        [0, 1, 2],
                        [2, 3, 4],
                        [1, 5, 7],
                    ]
                )
            ),
            components=[],
            properties=mts.Labels(
                names=["p1", "p2"],
                values=torch.tensor(
                    [
                        [100, 0],
                        [5, 7000],
                    ]
                )
            )
        ),
        mts.TensorBlock(
            values=torch.randn(3, 3),
            samples=mts.Labels(
                names=["s1", "s2", "s3"],
                values=torch.tensor(
                    [
                        [0, 2, 2],
                        [0, 1, 2],
                        [1, 5, 7],
                    ]
                ),
            ),
            components=[],
            properties=mts.Labels(
                names=["p1", "p2"],
                values=torch.tensor(
                    [
                        [5, 10],
                        [5, 5],
                        [5, 6],
                    ]
                )
            ),
        )
    ]
)
# Sort
sorted = mts.sort(tensor, "samples")

# All metadata is the same
assert tensor.keys == sorted.keys
assert tensor.block(0).samples == sorted.block(0).samples
assert tensor.block(0).properties == sorted.block(0).properties
assert torch.all(tensor.block(0).values == sorted.block(0).values)

Thanks Joe, that's exactly the same issue I'm facing.

I'm guessing the error is here, in _dispatch.argsort_labels_values()?
    if isinstance(labels_values, TorchTensor):
        # torchscript does not support sorted for List[List[int]]
        # so we temporary do this trick. this will be fixed with issue #366
        max_int = torch.max(labels_values)
        idx = torch.sum(
            max_int ** torch.arange(labels_values.shape[1]) * labels_values, dim=1
        )
        return torch.argsort(idx, dim=-1, descending=reverse)

https://github.com/lab-cosmo/metatensor/blob/ec6ee61ebb6eab326462386171f5cd777f12a3c4/python/metatensor-operations/metatensor/operations/_dispatch.py#L126C1-L133C62
Seems like a strange hack

mmm this is weird in here (https://github.com/lab-cosmo/metatensor/commit/cae018d0ef8b66f4eaac6304b7eb6a65d7cababd)  I should have added test for the metatensor-torch sort operation.
These are basically the same test we use for the numpy version and they seem to pass at least on my local machine.
Is it possible that this is related to some CPU/GPU thing? maybe the data are on GPU and they don't get properly handled?
This is pure guessing
or maybe i did something wrong while doing the tests

we tested this on cpu, so also not sure

ok i simply do not know how to use tox

So in @jwa7 example the code does nothing becouse the starting tensormap is already ordered. Even if you use no instead of torch the result is the same. The code now order by last column and the input jow wrote are already ordered by last column.
already changing the keys in values=torch.tensor([[2, 1], [1, 0]]), would show that the code do infact sort "correctly" everything

@DavideTisi Hmm I'm not sure I follow. What do you mean the TensorMap is sorted in my example? I created it so that the labels are not sorted.
When I run the code as above, there is a lot of printed output, showing that the sort operation doesn't do any sorting as intended. When modifying the example to instead import metatensor as mts and use numpy instead of torch, the example prints no output as the sort operation behaves as expected.
My conclusion that was it's a problem with the torch side of the operation, so I'm confused as to why you're seeing what you're seeing.

@jwa7 ok tomorrow I will look better because if I run your code both with torch or numpy I get the same result

so what i did in #644 to _dispatch did fix your example (which I also added to the metatensor.torch test, but it is not a real fix for this problem since the algorithm is not really stable


Fix SinglePlane example

The SinglePlaneAcquisition example is currently broken in two ways:

The chunk dimensions do not match the image dimensions (i.e. specify a z-channel when there is none).
No distributed client is passed.

Traceback of first issue
AssertionError                            Traceback (most recent call last)
Cell In[15], line 4
      1 plate = converter.create_zarr_plate(plate_acquisition)
      3 # Run conversion.
----> 4 converter.run(
      5     plate=plate,
      6     plate_acquisition=plate_acquisition,
      7     well_sub_group="0",
      8     chunks=(1, 512, 512),
      9     max_layer=2,
     10 )

File <...>/src/faim_ipa/hcs/converter.py:152, in ConvertToNGFFPlate.run(self, plate, plate_acquisition, wells, well_sub_group, chunks, max_layer, storage_options, build_acquisition_mask)
    129 """
    130 Convert a plate acquisition to an NGFF plate.
    131
   (...)
    149     zarr.Group of the plate.
    150 """
    151 assert 2 <= len(chunks) <= 3, "Chunks must be 2D or 3D."
--> 152 assert len(chunks) == len(
    153     plate_acquisition.get_well_acquisitions()[0].get_tiles()[0].shape
    154 ), "Chunks must have the same number of dimensions as the tile shape."
    155 well_acquisitions = plate_acquisition.get_well_acquisitions(wells)
    157 for well_acquisition in well_acquisitions:

AssertionError: Chunks must have the same number of dimensions as the tile shape.

Fixing this line to read chunks=(512, 512) lets us progress to the second issue:

Traceback of second issue
AttributeError                            Traceback (most recent call last)
File <...>/src/faim_ipa/hcs/converter.py:164, in ConvertToNGFFPlate.run(self, plate, plate_acquisition, wells, well_sub_group, chunks, max_layer, storage_options, build_acquisition_mask)
    158 well_group = self._create_well_group(
    159     plate,
    160     well_acquisition,
    161     well_sub_group,
    162 )
    163 group = well_group[well_sub_group]
--> 164 self._write_stitched_image(
    165     group,
    166     chunks,
    167     plate_acquisition,
    168     storage_options,
    169     well_acquisition,
    170     build_acquisition_mask=build_acquisition_mask,
    171 )
    172 shapes, datasets = self._build_pyramid(
    173     group,
    174     chunks,
    175     max_layer,
    176     storage_options,
    177 )
    178 self._write_metadata(
    179     group, max_layer, shapes, datasets, plate_acquisition, well_acquisition
    180 )

File <...>/src/faim_ipa/hcs/converter.py:241, in ConvertToNGFFPlate._write_stitched_image(self, group, chunks, plate_acquisition, storage_options, well_acquisition, build_acquisition_mask)
    238 rechunked_da = binned_da.rechunk(self._out_chunks(binned_da.shape, chunks))
    239 options = self._get_storage_options(storage_options, rechunked_da.shape, chunks)
    240 wait(
--> 241     self._client.persist(
    242         da.to_zarr(
    243             arr=rechunked_da,
    244             url=group.store,
    245             compute=False,
    246             component=str(Path(group.path, "0")),
    247             storage_options=options,
    248             compressor=options.get(
    249                 "compressor", zarr.storage.default_compressor
    250             ),
    251             dimension_separator=group._store._dimension_separator,
    252         ),
    253     )
    254 )

AttributeError: 'NoneType' object has no attribute 'persist'

Which seems to be triggered by converter._client being None by default. Initalising the client like in this example fixes that issue, too.
This PR includes both required changes.

Hi @jeskowagner,
Thanks a lot for fixing this example.
We haven't updated the examples in a long time and most likely none of them are working anymore :upside_down_face:
I am very happy to see that other people are using this as well :partying_face:

Funny coincidence, I was looking at this example today as well 😄
I'd suggest to also remove the stitching_yx_chunk_size_factor=2, I'll follow up next week on this.

Yeah it seems we happened to work on this at the same time! Addressed in this new PR.


Fix llbuild prefix directory initialization

Noticed some strange useless directories being created under .build anytime I compiled a project.
.build/
└── debug
    ├── Dealer
    ├── Dealer.dSYM
    │   └── Contents
    │       ├── Info.plist
    │       └── Resources
    │           └── DWARF
    │               └── Dealer
    ├── Dealer.o
    │   ├── Dealer
    │   │   ├── main.d
    │   │   ├── main.swiftdeps
    │   │   ├── main~partial.swiftdoc
    │   │   ├── main~partial.swiftmodule
    │   │   ├── master.swiftdeps
    │   │   └── output-file-map.json
    │   ├── Users
    │   │   └── josh
    │   │       └── Desktop
    │   │           └── example-package-dealer
    │   ├── build.db
    │   ├── llbuild.yaml
    │   └── main.swift.o
    ├── Dealer.swiftdoc

Each target would initialize a empty tree of directories with the path of the current project.
Dug into what was happening here.
Looks like llbuild is calling mkdir with prepending the tmpdir path to an already absolute prefix path.
My current fix removes prefix from the requiredSubdirectories collection because it didn't seem like it was a "subdirectory", but always an absolute path.
Is BuildParameters#prefix always an absolute path? Or should it always be relative, or could it be either? Maybe we could document this.

Thanks for looking into this. It was on my todo.

One test fails, fixed if instead of prefix we mkdir(parms.tmpdir).

Though perhaps I prefer:
    for subdir in parms.requiredSubdirectories() + [""] {
        try mkdir(parms.tmpdir, subdir)
    }

Though perhaps I prefer:

Cool, that fixes the issue for me. Updated patch.

Thanks @mxcl!

np! Glad to get your patch Josh 👍🏻. I hope for more… ;)

Just digging in 😀. Really excited for swift test https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20151214/000068.html


Correction for raw affiliation Univ Lyon, ENS de Lyon, Univ Claude Bernard, CNRS UMR 5239, INSERM U1210, Laboratory of Biology and Modelling of the Cell

Correction needed for raw affiliation Univ Lyon, ENS de Lyon, Univ Claude Bernard, CNRS UMR 5239, INSERM U1210, Laboratory of Biology and Modelling of the Cell
raw_affiliation_name: Univ Lyon, ENS de Lyon, Univ Claude Bernard, CNRS UMR 5239, INSERM U1210, Laboratory of Biology and Modelling of the Cell
new_rors: 02feahw73;04zmssz18;02vjkv261;029brtt94;01bj4fd12
previous_rors: 02feahw73;04zmssz18;02vjkv261;029brtt94
works_examples: W2806965190
contact: f7386e4e6a76e5e07fed9ec5e48c8d7b:09471f1f3db053666a079319e2b3 @ ens-lyon.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-21. The new affiliations should be visible within the next 7 days.


Clean up comments in GA

Codecov Report

Merging #2 (d4b5c0d) into master (0a40d3e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #2   +/-   ##
=======================================
  Coverage   96.17%   96.17%
=======================================
  Files           5        5
  Lines         157      157
=======================================
  Hits          151      151
  Misses          6        6

Help us with your feedback. Take ten seconds to tell us how you rate us.


Provide contribution guide information for adding authentication sources

Provide contribution guide information for adding authentication sources

@v-shenoy Since you recently did this, is this something you are interested in helping with?

What kind of information do you expect, and where do I add this?

Something similar to https://github.com/kedacore/keda/blob/main/CONTRIBUTING.md#contributing-scalers that walks you through the process in case you are a new contributor for auth providers.
So basically what you had to figure out yourself, but now documented :D

I am not sure how to add a documentation that will be generic enough to cover all authentication sources. So, as far as my understanding goes, to add an authentication source, you would need to make changes to the TriggerAuth/ClusterTriggerAuth CRD and have the user provide information to query the auth provider.
Now using these values from the CRD, there need to be code changes to use these values.  These code changes can differ on the basis of auth source (ex - key vault vs workload identity), so I am a bit confused here.

@kedacore/keda-maintainers What do you think? Is this valuable? I might be overthinking it but want to make it simple for people to contribute more auth providers.

Yeah, I think that even a simple documentation is great! It doesn't have to be complex, but something that gives users some general overview.


Issue 506

Have not made the final commit message. I will create a last commit with it. In the meantime I am just looking for reviewers and comments for the code.

README.MD need to be updated to reflect the directory: ~/openfl/openfl-tutorials/interactive_api/PyTorch_FedProx_MNIST,
Not the openfl/openfl-tutorials/interactive_api/PyTorch_FedProx_MedMNIST_2D


fix updateStatus error in lower k8s version

Codecov Report

Merging #122 (5b44e0a) into main (ad290fa) will decrease coverage by 1.26%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #122      +/-   ##
==========================================
- Coverage   12.49%   11.22%   -1.27%
==========================================
  Files          17       17
  Lines        1673     1675       +2
==========================================
- Hits          209      188      -21
- Misses       1445     1468      +23
  Partials       19       19

Flag
Coverage Δ

unittests
11.22% <0.00%> (-1.27%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/agent/discovery/discovery.go
11.56% <0.00%> (-0.14%)
:arrow_down:

pkg/controller/controller.go
29.85% <0.00%> (-7.84%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad290fa...5b44e0a. Read the comment docs.


About the Paper

Hi! Lin
Thanks for sharing your great works.
If possible, can your share the preprint paper in docs?
Best regards

Dear Afei,
Thanks for your interest! We have uploaded the preprint here. Please take a look 😉
Cheers,
Yu-Kai


remove extended from voting pool

About the PR
who even likes this?
Screenshots

Changelog

:cl:

fix: Extended is no longer in the game mode voting pool.

At this point just disable voting since we only have extended and secret as voteable

At this point just disable voting since we only have extended and secret as voteable

counterpoint: a vote with only 1 option is funny

Dumbasses calling for vote all the time for no reason moment.

Based???


Table names containing single quotes break things

e.g. I found a table called Yesterday's ELRs by County
It threw an error inside the detect_fts() function attempting to run this SQL query:
        select name from sqlite_master
            where rootpage = 0
            and (
                sql like '%VIRTUAL TABLE%USING FTS%content="Yesterday's ELRs by County"%'
                or sql like '%VIRTUAL TABLE%USING FTS%content=[Yesterday's ELRs by County]%'
                or (
                    tbl_name = "Yesterday's ELRs by County"
                    and sql like '%VIRTUAL TABLE%USING FTS%'
                )
            )

Here's the code at fault: https://github.com/simonw/datasette/blob/640ac7071b73111ba4423812cd683756e0e1936b/datasette/utils/init.py#L534-L548

Created a test database like this:
% sqlite-utils create-table quote-in-name.db "this'hasquoteinname" id integer name text --pk id
% datasette quote-in-name.db -p 8025 --pdb
INFO:     Started server process [86046]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:8025 (Press CTRL+C to quit)
> /Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py(530)detect_fts()
-> rows = conn.execute(detect_fts_sql(table)).fetchall()
(Pdb) c
Traceback (most recent call last):
  File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 1124, in route_path
    response = await view(request, send)
  File "/Users/simon/Dropbox/Development/datasette/datasette/views/base.py", line 147, in view
    return await self.dispatch_request(
  File "/Users/simon/Dropbox/Development/datasette/datasette/views/base.py", line 122, in dispatch_request
    return await handler(request, *args, **kwargs)
  File "/Users/simon/Dropbox/Development/datasette/datasette/views/index.py", line 72, in get
    "fts_table": await db.fts_table(table),
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 279, in fts_table
    return await self.execute_fn(lambda conn: detect_fts(conn, table))
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 155, in execute_fn
    return await asyncio.get_event_loop().run_in_executor(
  File "/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 153, in in_thread
    return fn(conn)
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 279, in <lambda>
    return await self.execute_fn(lambda conn: detect_fts(conn, table))
  File "/Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py", line 530, in detect_fts
    rows = conn.execute(detect_fts_sql(table)).fetchall()
sqlite3.OperationalError: near "hasquoteinname": syntax error

I tried and failed to get this fix working for tables with double quotes in their name - I couldn't figure out what the double-quote-in-a-table-name version of this code would look like: https://github.com/simonw/datasette/blob/807de378d08752a0f05bb1b980a0a62620a70520/datasette/utils/init.py#L538-L548


mdx example results in type errors

Following the quickstart for mdx here https://www.content-collections.dev/docs/content/mdx

results in this:

But if i workaround it with a asserted type predicate then it's satisfied.

Did I miss something?

const me = defineCollection({
  name: 'me',
  directory: 'content/data/me',
  include: '**/*.mdx',
  schema: (z) => ({
    title: z.string(),
    description: z.string().optional(),
  }),
  transform: async (document, context) => {
    assert(isDocumentWithContent(document), 'Document has no content');
    const mdx = await compileMDX(context, document);
    return {
      ...document,
      mdx,
    };
  },
});

/**
 * Hack Predicate to override aggressive generics
 * done by content-collections/core
 */
function isDocumentWithContent<T extends { content: string }>(
  document: any
): document is T {
  return typeof document?.content === 'string';
}

function assert(condition: any, message: string): asserts condition {
  if (!condition) {
    throw new Error(message);
  }
}

I still get the error if I describe a content: z.string() in the schema:

I imagine something is going awry here :
type AddContent<
  TShape extends ZodRawShape
> = TShape extends {
    content: ZodTypeAny;
}
  ? TShape
  : TShape & WithContent;

type GetParsedShape<
  TParser extends Parser,
  TShape extends ZodRawShape
> = Parsers[TParser]["hasContent"] extends true
  ? AddContent<TShape>
  : TShape;

type GetShape<
  TParser extends Parser | undefined,
  TShape extends ZodRawShape
> = TParser extends Parser
  ? GetParsedShape<TParser, TShape>
  : AddContent<TShape>;

type Schema<
  TParser extends Parser | undefined,
  TShape extends ZodRawShape
> = z$1.infer<ZodObject<GetShape<TParser, TShape>>>
  & {
      _meta: Meta;
  };

I am currently on vacation. I'll take a look at it next week.

I am currently on vacation. I'll take a look at it next week.

All good, no one expects you to deal with these issues straight away  👍🏻

@airtonix i can't reproduce it. Which version of typescript do you use? And can you post your tsconfig?


Type keyword has not been correctly parsed by editor if used as a property attribute in a definition in the field "additionalProperties"

This should be a problem only in the editor validation because the client is correctly generated
In attachement you can find an example of the issue
TypeBugExample1.txt

Please raise tooling-related issues on the relevant repository. This repository is for the OpenAPI specification itself. Thanks.


Zend/Mvc/Router/Http/Literal - Some Fix and Test

finall pull request....

Close, becаuse send new pull request with additional commit


Question about the padding_side configuration

Hi, thanks for your great work! I noticed that the padding_side setting significantly affects the edit performance. I tested the ROME edit method on the Llama2-7b model and also reviewed the comment in this issue. However, my results show that padding_side seems to have a substantial impact on the editing performance.
Code
    prompts = [
        "What university did Watts Humphrey attend?",
    ]
    ground_truth = [
        "Illinois Institute of Technology",
    ]
    target_new = [
        "University of Michigan",
    ]
    subject = [
        "Watts Humphrey",
    ]

    hparams = ROMEHyperParams.from_hparams("./hparams/ROME/llama-7b.yaml")
    editor = BaseEditor.from_hparams(hparams)

    batch = editor.tok(
        prompts,
        return_tensors="pt",
        padding=True,
        max_length=30,
        truncation=True,
    )

    metrics, edited_model, _ = editor.edit(
        prompts=prompts,
        target_new=target_new,
        subject=subject,
        sequential_edit=True,
    )

Result
The edit process with right padding seems to work correctly.
Recording initial value of v*
loss 3.967 = 3.967 + 0.0 + 0.0 avg prob of [ University of Michigan] 0.020505022257566452
loss 3.355 = 3.315 + 0.039 + 0.001 avg prob of [ University of Michigan] 0.03862017020583153
loss 2.717 = 2.673 + 0.043 + 0.001 avg prob of [ University of Michigan] 0.07443119585514069
loss 1.905 = 1.839 + 0.065 + 0.001 avg prob of [ University of Michigan] 0.16459743678569794
loss 1.178 = 1.09 + 0.087 + 0.001 avg prob of [ University of Michigan] 0.3368275463581085
loss 1.082 = 0.995 + 0.087 + 0.001 avg prob of [ University of Michigan] 0.3739083707332611
loss 0.739 = 0.605 + 0.133 + 0.001 avg prob of [ University of Michigan] 0.5471051931381226
loss 0.247 = 0.125 + 0.121 + 0.001 avg prob of [ University of Michigan] 0.8829345703125
loss 0.109 = 0.031 + 0.077 + 0.001 avg prob of [ University of Michigan] 0.9698953628540039
loss 0.064 = 0.014 + 0.049 + 0.001 avg prob of [ University of Michigan] 0.9863483309745789
loss 0.047 = 0.009 + 0.037 + 0.001 avg prob of [ University of Michigan] 0.9908254742622375

However, when I set padding_side='left', the edit loss becomes difficult to reduce.
loss 7.632 = 7.632 + 0.0 + 0.0 avg prob of [ University of Michigan] 0.010061966255307198
loss 6.666 = 6.663 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.021385638043284416
loss 6.211 = 6.208 + 0.001 + 0.003 avg prob of [ University of Michigan] 0.033467747271060944
loss 5.89 = 5.887 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.03745617717504501
loss 5.364 = 5.361 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.05327693745493889
loss 4.971 = 4.968 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.08356843888759613
loss 4.746 = 4.743 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.09671956300735474
loss 6.016 = 6.013 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.020683931186795235
loss 4.99 = 4.987 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.06548506021499634
loss 4.635 = 4.631 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.08159938454627991
loss 3.762 = 3.759 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.13448581099510193
loss 3.568 = 3.565 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.19031785428524017
loss 3.949 = 3.946 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.2177749127149582
loss 3.594 = 3.591 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.17594163119792938
loss 3.683 = 3.68 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.11976221948862076
loss 2.809 = 2.806 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.1964445412158966
loss 2.799 = 2.796 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.23149645328521729
loss 1.65 = 1.647 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.40359362959861755
loss 1.282 = 1.279 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.5241387486457825
loss 1.855 = 1.852 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.42500051856040955
loss 3.971 = 3.968 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.13831934332847595
loss 3.619 = 3.616 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.20624874532222748
loss 2.607 = 2.604 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.30388569831848145
loss 1.781 = 1.778 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.3056000769138336
loss 1.399 = 1.396 + 0.0 + 0.003 avg prob of [ University of Michigan] 0.3996792733669281

Question
I found that the default padding_side is set to right due to the following code in editor.py.
https://github.com/zjunlp/EasyEdit/blob/e8bb5c145a764216de244c8384d4b3a075ba23ae/easyeditor/editors/editor.py#L107-L112
Could you explain why padding_side='right' is set as the default for Llama models and the edit methods ['ROME', 'MEMIT', 'EMMET', 'R-ROME']? Are there any specific guidelines for choosing the padding side?

Hi, this is due to the last subject token used in these methods.
We compute the position based on the original sentences' tokens and padding in the left would change the position of this token.

Hi, do you have any further issues?

Apologies for the delayed response. I have no further questions and would like to close the issue.


怎么生成mace.pb.h 和mace.pb.cc ?在proto目录下。

Before you open an issue, please make sure you have tried the following steps:

Make sure your environment is the same with (https://mace.readthedocs.io/en/latest/installation/env_requirement.html).
Have you ever read the document for your usage?
Check if your issue appears in HOW-TO-DEBUG or FAQ.
The form below must be filled.

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
NDK version(e.g., 15c):
GCC version(if compiling for host, e.g., 5.4.0):
MACE version (Use the command: git describe --long --tags):
Python version(2.7):
Bazel version (e.g., 0.13.0):

Model deploy file (*.yml)
......

Describe the problem
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the problem:
1. cd /path/to/mace
2. python tools/converter.py convert --config_file=/path/to/your/model_deployment_file

Error information / logs
Please include the full log and/or traceback here.
LOGs

Additional context
Add any other context about the problem here, e.g., what you have modified about the code.

好像不会生成这两个文件吧？

Sorry，我搞错了，是可以生成的，bazel编译的时候指定mace_cc这个目标应该就可以

bazel应该会自动生成在bazel-genfiles/mace/proto/路径下


Support loading datasets of 10GB in Snowflake in less than 5 min

Dependencies

Depends on: #438
Depends on: #492
Depends on: #493
Depends on: #494

Acceptance criteria

[ ] Make changes to how we load data into Snowflake.
[ ] Re-run the benchmark and identify the performance improvements (?)
[ ] The changes must work for all file types supported (CSV, JSON, NDJSON, Parquet)
[ ] The changes made would also work on S3 and GCS

I believe this was closed by mistake, we haven't merged the changes into master yet..!


[Bug]: Contact us button at the Hackhazards sponsors page should redirect to Gmail

Describe the bug
No response
To Reproduce
No response
Expected Behavior
The contact us button should redirect to G-mail.
Screenshot/ Video

Desktop (please complete the following information)
No response
Additional context
No response

@ArshTiwari2004 the mail to be used is<EMAIL_ADDRESS>

closing as resolved


UX like OSX with snap points

UX 2 like OSX

Few small things:

stray console log
inconsistent formatting
commented out code

Additionally, as we're pulling in this lib in our own fork, we own it, meaning we need to have at least two people familiarized with this lib capable of doing fixes should need be.

@marcinbunsch I agree with all you said.

Formatting should be ok since I am running npm run prettier before committing.


Disable tracking of skills after 99 (Option)

Give us the option to hide this - https://gyazo.com/87c5c950595288c8f0191a48923103a4
I'd rather not see skills that I've already got 99 in.

or a toggle to keep them at 99

Would also be nice to consider within the XP globes.

@PandahRS solved #2720

Is there an update as to when this might be added?

still waiting too!


can i add the package to my project using NuGet ?

can i add the package to my project using NuGet ?

SignalR Core is currently available through our MyGet feed:  https://dotnet.myget.org/F/aspnetcore-ci-dev/api/v3/index.json

When you are using VS 2017.3 with a blank ASP.NET CORE (v2) templates: it says method not found.
But if you are trying to fetch the latest Microsoft.AspNetCore.All rtm from NuGet - it fails.

When you are using VS 2017.3 with a blank ASP.NET CORE (v2) template: it says method not found.

By design. You can't use the latest SignalR builds with preview2.

But if you are trying to fetch the latest Microsoft.AspNetCore.All rtm from NuGet - it fails with Package restore failed

Because you didn't configure the right feeds.

https://dotnet.myget.org/F/aspnetcore-dev/api/v3/index.json - this one?

Yep.

Can somebody confirm that it's impossible to get the latest asp.net core build from NuGet in VS 2017.3
I've tested on 2 machines. Package restore failed

See #646

From: romanov<EMAIL_ADDRESS>Sent: Sunday, July 9, 2017 9:35:20 PM
To: aspnet/SignalR
Cc: Subscribed
Subject: Re: [aspnet/SignalR] can i add the package to my project using NuGet ? (#640)
Can somebody confirm that it's impossible to get the latest asp.net core build from NuGet in VS 2017.3
I've tested on 2 machines. Package restore failed
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/aspnet/SignalR/issues/640#issuecomment-313952872, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADQA1-9v8GxMAz0gpAzMMjOEHyn-Bl8Xks5sMTmIgaJpZM4OPgts.

For those who might still look at this issue, I believe the feed has been changed to this one:
https://dotnet.myget.org/F/aspnetcore-ci-release/api/v3/index.json
in this commit d830fc1

@harisbotic - this is not correct - dev branch should be (and is) on the aspnetcore-ci-dev feed.

What if my .net is 4.5.2?  If I add these packages it's not compatible.

@jbarber2016 - It still should work as soon as you are using ASP.Net Core 2.0

Hi,
I read all comments but the solution is not really clear to me. I created this SO question : https://stackoverflow.com/questions/45495548/what-is-the-correct-package-to-use-for-signalr-with-asp-net-core as I can't install SignalR whatever I do... what's the correct way to install the package ?
Thanks


Cue 0.3 - error with tekton code gen

What version of CUE are you using (cue version)?
304b02e / 0.3 alpha / trunk + fix for #458
Does this issue reproduce with the latest release?
no
What did you do?
git clone git@github.com:ekarlso/cue-tekton-test.git
cd cue-tekton-test
bash get-modules.sh
cue eval
What did you expect to see?
cue eval results showing the def's etc.
What did you see instead?
    ./cue.mod/gen/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1/pipelinerun_types_go_gen.cue:64:23
#PipelineTaskRunSpec.taskPodTemplate: reference "#Template" not found:
    ./cue.mod/gen/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1/pipelinerun_types_go_gen.cue:226:34
#TaskRunSpec.podTemplate: reference "#Template" not found:
    ./cue.mod/gen/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1/taskrun_types_go_gen.cue:42:23

Would you, or someone else, be able to narrow this down to a snippet that does not require running a bash script?
Thanks!

I tested your fix to this and it seems to be working now!.

https://cue-review.googlesource.com/c/cue/+/6961

This issue has been migrated to https://github.com/cue-lang/cue/issues/464.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


Tamil Language - Normalization failed for string Errors though the strings are listed in the Unicharset files

Environment

Tesseract Version: tesseract 4.1.0-rc1
leptonica-1.75.3
libjpeg 8d (libjpeg-turbo 1.5.2) : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11
Found AVX2
Found AVX
Found SSE
Commit Number: 8e83b20
Platform: Ubuntu 18.04.1 LTS
langdata: tam

Current Behavior:
I am receiving the following multiple Normalization errors like shown below when trying to fine-tune tam.traineddata sourced from tessdata-best using OCRD-Train. Also, facing same issue even with Script: Tamil.traineddata
Errors:

Invalid start of grapheme sequence:M=0xbc7
Normalization failed for string 'ே'
Word started with a combiner:0xbc6
Normalization failed for string 'ெ'

Expected Behavior:
Since the strings that are marked as unreadable are already listed in the the lang folder tam/tam.lstm-unicharset and Tamil.unicharset, they should be extracted without any errors.

Current setup works perfectly for fine-tuning eng.traineddata
Tried with different sample training data sets for lang tam
Added Tamil.unicharset and Latin.unicharset which nearly fixed Warning: properties incomplete for index` error
Command used for Fine-tuning:  make training MODEL_NAME=testtamil START_MODEL=tam using OCRD-Train

Suggested Fix:
NIL
Check the attached Training Log and Training Data.
Training_Log.txt
Training_Data.zip

Please check that your Tamil transliteration is correct for the following
cases where the vowel sign is reordered to go before the consonants.
க் + ஒ கொ ko [ko]
க் + ஓ கோ kō [koː]
க் + ஔ கௌ kau [kʌʋ]
On Tue, Feb 26, 2019 at 12:17 AM Vijay Rajasekaran<EMAIL_ADDRESS>wrote:

Environment

Tesseract Version: tesseract 4.1.0-rc1
leptonica-1.75.3
libjpeg 8d (libjpeg-turbo 1.5.2) : libpng 1.6.34 : libtiff 4.0.9 :
zlib 1.2.11
Found AVX2
Found AVX
Found SSE
Commit Number: 8e83b20
https://github.com/tesseract-ocr/tesseract/commit/8e83b209c6cdee354dd397f313047ad20d06b8d1
Platform: Ubuntu 18.04.1 LTS
langdata: tam

Current Behavior:
I am receiving the following multiple Normalization errors like shown
below when trying to fine-tune tam.traineddata sourced from tessdata-best
using OCRD-Train. Also, facing same issue even with Script:
Tamil.traineddata
Errors:

Invalid start of grapheme sequence:M=0xbc7
Normalization failed for string 'ே'
Word started with a combiner:0xbc6
Normalization failed for string 'ெ'

Expected Behavior:
Since the strings that are marked as unreadable are already listed in the
the lang folder tam/tam.lstm-unicharset and Tamil.unicharset, they should
be extracted without any errors.

Current setup works perfectly for fine-tuning eng.traineddata
Tried with different sample training data sets for lang tam
Added Tamil.unicharset and Latin.unicharset which nearly fixed
Warning: properties incomplete for index` error
Command used for Fine-tuning: make training MODEL_NAME=testtamil
START_MODEL=tam using OCRD-Train

Suggested Fix:
NIL
Check the attached Training Log and Training Data.
Training_Log.txt
https://github.com/tesseract-ocr/tesseract/files/2902085/Training_Log.txt
Training_Data.zip
https://github.com/tesseract-ocr/tesseract/files/2902086/Training_Data.zip
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/2267, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AE2_o4upP0ptT_KkbyQ6W1Si9wTgbrKgks5vRC_AgaJpZM4bQiKN
.

--

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

@Shreeshrii I am just using Google Transliterate to correct the errors generated while creating .tif + .gt.txt pairs. Now, I created a data set for training using Tesseract OCR for text recognition using tam language from tessdata-best. But, I am still facing the same issues even without editing the text generated using Tesseract.
Also, when I edit the text generated via Tesseract OCR it is split as below in Notepad++
கொ = க ொ
கோ = க ோ
கௌ = க ௌ
பெயர் = ப ெயர ்
ராஜசுந்தரம் = ர ாஜச ுந ்தரம ்
கிருத்திகா = க ிர ுத ்த ிக ா

Please check that your Tamil transliteration is correct for the following cases where the vowel sign is reordered to go before the consonants. க் + ஒ கொ ko [ko] க் + ஓ கோ kō [koː] க் + ஔ கௌ kau [kʌʋ]
…
On Tue, Feb 26, 2019 at 12:17 AM Vijay Rajasekaran @.***> wrote: Environment - Tesseract Version: tesseract 4.1.0-rc1 leptonica-1.75.3 libjpeg 8d (libjpeg-turbo 1.5.2) : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11 Found AVX2 Found AVX Found SSE - Commit Number: 8e83b20 <8e83b20> - Platform: Ubuntu 18.04.1 LTS - langdata: tam Current Behavior: I am receiving the following multiple Normalization errors like shown below when trying to fine-tune tam.traineddata sourced from tessdata-best using OCRD-Train. Also, facing same issue even with Script: Tamil.traineddata Errors: 1. Invalid start of grapheme sequence:M=0xbc7 Normalization failed for string 'ே' 2. Word started with a combiner:0xbc6 Normalization failed for string 'ெ' Expected Behavior: Since the strings that are marked as unreadable are already listed in the the lang folder tam/tam.lstm-unicharset and Tamil.unicharset, they should be extracted without any errors. - Current setup works perfectly for fine-tuning eng.traineddata - Tried with different sample training data sets for lang tam - Added Tamil.unicharset and Latin.unicharset which nearly fixed Warning: properties incomplete for index` error - Command used for Fine-tuning: make training MODEL_NAME=testtamil START_MODEL=tam using OCRD-Train Suggested Fix: NIL Check the attached Training Log and Training Data. Training_Log.txt https://github.com/tesseract-ocr/tesseract/files/2902085/Training_Log.txt Training_Data.zip https://github.com/tesseract-ocr/tesseract/files/2902086/Training_Data.zip — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2267>, or mute the thread https://github.com/notifications/unsubscribe-auth/AE2_o4upP0ptT_KkbyQ6W1Si9wTgbrKgks5vRC_AgaJpZM4bQiKN .
____________________________________________________________ भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

@Shreeshrii I am just using Google Transliterate to correct the errors generated while creating .tif + .gt.txt pairs. Now, I created a data set for training using Tesseract OCR for text recognition using tam language from tessdata-best. But, I am still facing the same issues even without editing the text generated using Tesseract.
Also, when I edit the text generated via Tesseract OCR it is split as below in Notepad++
கொ = க ொ
கோ = க ோ
கௌ = க ௌ
பெயர் = ப ெயர ்
ராஜசுந்தரம் = ர ாஜச ுந ்தரம ்
கிருத்திகா = க ிர ுத ்த ிக ா

Also, when I edit the text generated via Tesseract OCR it is split as below
in Notepad++
கொ = க ொ
கோ = க ோ
கௌ = க ௌ
பெயர் = ப ெயர ்
ராஜசுந்தரம் = ர ாஜச ுந ்தரம ்
கிருத்திகா = க ிர ுத ்த ிக ா
Dotted circles mean it is not correct order of unicode characters.
On Tue, Feb 26, 2019 at 1:32 AM Vijay Rajasekaran<EMAIL_ADDRESS>wrote:

@Shreeshrii https://github.com/Shreeshrii I am just using Google
Transliterate to correct the errors generated while creating .tif +
.gt.txt pairs. Now, I created a data set for training using Tesseract OCR
for text recognition using tam language from tessdata-best. But, I am
still facing the same issues even without editing the text generated using
Tesseract.
Also, when I edit the text generated via Tesseract OCR it is split as
below in Notepad++
கொ = க ொ
கோ = க ோ
கௌ = க ௌ
பெயர் = ப ெயர ்
ராஜசுந்தரம் = ர ாஜச ுந ்தரம ்
கிருத்திகா = க ிர ுத ்த ிக ா
Please check that your Tamil transliteration is correct for the following
cases where the vowel sign is reordered to go before the consonants. க் + ஒ
கொ ko [ko] க் + ஓ கோ kō [koː] க் + ஔ கௌ kau [kʌʋ]
… <#m_6782694531701001030_>
On Tue, Feb 26, 2019 at 12:17 AM Vijay Rajasekaran @.***> wrote:
Environment - Tesseract Version: tesseract 4.1.0-rc1 leptonica-1.75.3
libjpeg 8d (libjpeg-turbo 1.5.2) : libpng 1.6.34 : libtiff 4.0.9 : zlib
1.2.11 Found AVX2 Found AVX Found SSE - Commit Number: 8e83b20
https://github.com/tesseract-ocr/tesseract/commit/8e83b209c6cdee354dd397f313047ad20d06b8d1
<8e83b20
https://github.com/tesseract-ocr/tesseract/commit/8e83b209c6cdee354dd397f313047ad20d06b8d1>

Platform: Ubuntu 18.04.1 LTS - langdata: tam Current Behavior: I am
receiving the following multiple Normalization errors like shown below when
trying to fine-tune tam.traineddata sourced from tessdata-best using
OCRD-Train. Also, facing same issue even with Script: Tamil.traineddata
Errors: 1. Invalid start of grapheme sequence:M=0xbc7 Normalization
failed for string 'ே' 2. Word started with a combiner:0xbc6 Normalization
failed for string 'ெ' Expected Behavior: Since the strings that are marked
as unreadable are already listed in the the lang folder
tam/tam.lstm-unicharset and Tamil.unicharset, they should be extracted
without any errors. - Current setup works perfectly for fine-tuning
eng.traineddata - Tried with different sample training data sets for lang
tam - Added Tamil.unicharset and Latin.unicharset which nearly fixed
Warning: properties incomplete for index` error - Command used for
Fine-tuning: make training MODEL_NAME=testtamil START_MODEL=tam using
OCRD-Train Suggested Fix: NIL Check the attached Training Log and
Training Data. Training_Log.txt
https://github.com/tesseract-ocr/tesseract/files/2902085/Training_Log.txt
Training_Data.zip
https://github.com/tesseract-ocr/tesseract/files/2902086/Training_Data.zip
— You are receiving this because you are subscribed to this thread. Reply
to this email directly, view it on GitHub <#2267
https://github.com/tesseract-ocr/tesseract/issues/2267>, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AE2_o4upP0ptT_KkbyQ6W1Si9wTgbrKgks5vRC_AgaJpZM4bQiKN
.

____________________________________________________________ भजन - कीर्तन

आरती @ http://bhajans.ramparivar.com

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/2267#issuecomment-467161351,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o5_klE39MxpkxAzsrBULGrd7Xceuks5vREFJgaJpZM4bQiKN
.

--

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

@Shreeshrii I am only using the data generated from Tesseract for training. Also, these characters are present in tam.unicharset.
Can you let me know how do I resolve this. Thanks.
I have no issues with text recognition using Tesseract, only issue is with the accuracy for which I wanted to fine-tune the traineddata.

Sorry for the confusion. I meant to say, I was able to manually split the words into separate characters like shown above and how it was not like the example you had mentioned. Tesseract did recognise as a whole word only.

combine_lang_model
--input_unicharset data/unicharset
--script_dir data/
--output_dir data/
--lang testtamil
add
--pass_through_recoder \
as part of above command and try.

Looks like the validation for Tamil may not be correct
See https://github.com/tesseract-ocr/tesseract/issues/1038
Using uniview - https://r12a.github.io/uniview/
The following is VALID.
தாமோதரன்
0BA4 TAMIL LETTER TA
0BBE TAMIL VOWEL SIGN AA
0BAE TAMIL LETTER MA
0BCB TAMIL VOWEL SIGN OO
0BA4 TAMIL LETTER TA
0BB0 TAMIL LETTER RA
0BA9 TAMIL LETTER NNNA
0BCD TAMIL SIGN VIRAMA

@zdenop Please label as bug.
@amitdo @stweil  Would it be possible to use icu libraries for validation and normalization rather than having own routines?

@vijayrajasekaran  Please add Tamil to issue description (title line).
Info from https://unicode.org/charts/PDF/U0B80.pdf
Two-part dependent vowel signs
These vowel signs have glyph pieces which stand on both
sides of the consonant; they follow the consonant in logical
order, and should be handled as a unit for most processing.
0BCA $ொ TAMIL VOWEL SIGN O
≡    $ெ◌   $◌ா
0BCB $ோ TAMIL VOWEL SIGN OO
≡    $ே◌   $◌ா
0BCC $ௌ TAMIL VOWEL SIGN AU
≡    $ெ◌   $◌ௗ

@vijayrajasekaran  What is your version of tesseract? I suggest you use the latest from github master including commit https://github.com/tesseract-ocr/tesseract/commit/9ddf2679075538605afb2cdcbd98385f5d98b6d4 and see if that makes a difference.

The following works ok for me with the latest code.

rm -rf ~/tesstutorial/tamtest
bash  ~/tesseract/src/training/tesstrain.sh \
  --fonts_dir ~/.fonts \
  --lang tam \
  --linedata_only \
  --save_box_tiff \
  --workspace_dir ~/tmp \
  --exposures "0" \
  --maxpages 1 \
  --noextract_font_properties \
  --langdata_dir ~/langdata_lstm \
  --tessdata_dir ~/tessdata_best  \
  --fontlist "Arial Unicode MS" \
  --training_text /home/ubuntu/ocrd-train/tam.txt \
  --output_dir ~/tesstutorial/tamtest

@Shreeshrii I built Tesseract with the last commit but implented  the suggested changes mentioned in the latest commit locally as advised by you from here.
Before commenting out the lines the error was like Word started with a combiner but after making the changes, the errors changed to Invalid start of grapheme sequence

The following works ok for me with the latest code.

rm -rf ~/tesstutorial/tamtest
bash  ~/tesseract/src/training/tesstrain.sh \
  --fonts_dir ~/.fonts \
  --lang tam \
  --linedata_only \
  --save_box_tiff \
  --workspace_dir ~/tmp \
  --exposures "0" \
  --maxpages 1 \
  --noextract_font_properties \
  --langdata_dir ~/langdata_lstm \
  --tessdata_dir ~/tessdata_best  \
  --fontlist "Arial Unicode MS" \
  --training_text /home/ubuntu/ocrd-train/tam.txt \
  --output_dir ~/tesstutorial/tamtest

tam.txt has the following lines (based on your ocrd data).
அக்கம்மாள்  தாமோதரன்
பாலசுப்பிரமணியன்
சுரேஷ்குமார்  ராதிகா
ரகுவீர்  கௌசல்யா
பத்மா
பொன்னைய்யா
ராஜஸ்ரீ  ஜெயஸ்ரீ

Thanks @Shreeshrii I will compile from the latest commit and update.

@Shreeshrii I tried from Tesseract compiled from latest source. I am still getting the Normalization Errors when using make unicharset or make training MODEL_NAME=tamiltest START_MODEL=tam using OCRD-Train

What is the value of --norm_mode in the following statement?
https://github.com/OCR-D/ocrd-train/blob/master/Makefile#L104

@Shreeshrii I tried changing that value in Makefile to 2 and also 3 but all resulted in the same errors.

@vijayrajasekaran The problem is with the ocrd-train makefile. Please close this issue. I will post suggested change in ocrd-train repo at https://github.com/OCR-D/ocrd-train/issues/52

@vijayrajasekaran  Have you made further progress with training Tamil?
I find the following characters are NOT there in the langdata_lstm/tam.training_text. See attached:
tam.missing.txt

@Shreeshrii I did try training, I am not able to get a good traineddata file. I am using OCRD-Train's Box +  Tiff pairs. The accuracy is not even good when trying to ocr the training samples again.
Like you said, since multiple characters are missing in the training text, how to resolve this? Any idea?
Thanks.

@Shreeshrii I compiled from the latest source and tried training again. I receive multiple errors (Normalization and Invalid Grapheme Sequence), check the attached file for the complete Training Log.
Training_Log.txt

I use the following sed script to normalize the training_text. Then I do not get many normalization errors. Give it  a try for your ground truth files.
s/¢/₹/g
s/£/₹/g
s/¥/₹/g
s/©//g
s/®//g
s/²//g
s/³//g
s/»//g
s/é/e/g
s/ஂ/்/g
s/ஔ/ஔ/g
s/ஹோ/ஹோ/g
s/ொ/ொ/g
s/ௌ/ௌ/g
s/ோ/ோ/g
s/⁴//g
s/€/₹/g
s/™//g
s/\$/₹/g

Also, do you know what is the font used for your test images?

best-tam
Image

அககம்மாள் தாமோதரன் பாலசுப்பிரமணியன் சுரேஷ்குமார் ராதிகா ரகுவீர கெளசல்யா பதமா பொன்னைய்யா ராஜஸ்ரீ ஜெயஸ்ரீ

best-Tamil
Image

அக்கம்மாள் தாமோதரன் பாலசுப்பிரமணியன் சுரேஷ்குமார ராதிகா ரகுவீர கெளசல்யா பத்மா பொன்னைய்யா ராஜர் ஜெயபர்

fast-tam
Image

அக்கம்மாள் தாமோதரன் பால்சுப்பிரமணியன் சுரேஷ்குமார் ராதிகா ரகுவீர் கெளசல்யா பத்மா பொன்னையயா ராஜஸ்ரீ ஜெயஸ்ரீ

fast-Tamil
Image

அக்கம்மாள தாமோகரன் பாலசுப்பிரமணியன் சுரேஷ்குமார் ராதிகா ரகுவீர கெளசல்யா பத்மா பொன்னைய்யா ராஜஸ ஜெயபர்

With finetuned traineddata:

tam_plus
Image

அக்கம்மாள் தாமோதரன் பாலசுப்பிரமணியளன் சுரேஷ்குமார் ராதிகா ரகுவீர் கௌசல்யா பத்மா பொன்னைய்யா ராஜஸ்ரீ ஜெயஸ்ரீ

tam_plus.zip
Finetuned traineddata - plus training using tessdata_best/tam to continue_from.

@Shreeshrii I am unable to the find the font of the data. Also, have you trained tam_plus by defining ground-truth files (box and all-boxes files) as s/\$/₹/g like you have advised above? I did try tam_plus with more real-world data, I think fine-tuning the traineddata messes up in recognizing certain characters. After lot of testing with multiple training data, I think the best as of now is Tamil Script traineddata. Tamil_Script_ Best. But, still having issues in recognizing  ்.

have you trained tam_plus by defining ground-truth files (box and all-boxes files) as s/$/₹/g

Yes, I did. I did not use your files, rather the training_text from langdata_lstm which I modified.

fine-tuning the traineddata messes up in recognizing certain characters

Which ones? Maybe I had removed them in tuning.

the best as of now is Tamil Script traineddata

Good to know.

still having issues in recognizing ்

Please share a couple of real life images along with their ground truth data so that I can test.

I was also trying to add the Tamil digits and special symbols. Are they used often?
௦ 8 65,65,231,231,122,122,13,13,149,149 Tamil 55 0 55 ௦	# ௦ [be6 ]0
௧ 8 64,64,192,192,149,156,11,15,176,179 Tamil 86 0 86 ௧	# ௧ [be7 ]0
௨ 8 64,65,192,195,155,219,5,13,185,244 Tamil 85 0 85 ௨	# ௨ [be8 ]0
௩ 8 64,65,192,192,164,185,5,13,168,211 Tamil 51 0 51 ௩	# ௩ [be9 ]0
௪ 8 64,64,192,192,139,165,8,15,157,187 Tamil 97 0 97 ௪	# ௪ [bea ]0
௫ 8 0,4,192,255,188,233,7,11,195,256 Tamil 69 0 69 ௫	# ௫ [beb ]0
௬ 8 62,64,192,192,181,207,7,19,193,236 Tamil 76 0 76 ௬	# ௬ [bec ]0
௭ 8 64,64,192,192,167,212,7,21,193,229 Tamil 45 0 45 ௭	# ௭ [bed ]0
௮ 8 16,45,192,193,211,314,3,15,213,320 Tamil 96 0 96 ௮	# ௮ [bee ]0
௯ 8 64,64,192,192,188,231,7,15,193,251 Tamil 77 0 77 ௯	# ௯ [bef ]0
௰ 0 64,65,192,255,139,189,9,13,169,214 Tamil 105 0 105 ௰	# ௰ [bf0 ]
௱ 0 59,65,192,193,151,166,5,21,170,175 Tamil 87 0 87 ௱	# ௱ [bf1 ]
௲ 0 0,14,176,192,151,187,0,15,151,194 Tamil 99 0 99 ௲	# ௲ [bf2 ]
௳ 0 64,64,193,193,218,218,11,11,232,232 Tamil 93 10 93 ௳	# ௳ [bf3 ]
௴ 0 65,65,255,255,248,248,23,23,271,271 Tamil 91 10 91 ௴	# ௴ [bf4 ]
௵ 0 0,0,255,255,437,437,13,13,460,460 Tamil 92 10 92 ௵	# ௵ [bf5 ]
௶ 0 0,0,192,192,161,161,19,19,202,202 Tamil 113 10 113 ௶	# ௶ [bf6 ]
௷ 0 63,63,192,192,328,328,9,9,342,342 Tamil 110 10 110 ௷	# ௷ [bf7 ]
௸ 0 0,0,239,239,391,391,11,11,407,407 Tamil 106 10 106 ௸	# ௸ [bf8 ]
௹ 0 0,0,255,255,297,297,17,17,315,315 Tamil 109 4 109 ௹	# ௹ [bf9 ]
௺ 0 65,65,253,253,281,281,21,21,305,305 Tamil 118 10 118 ௺	# ௺ [bfa ]

I don't think these symbols (Tamil Digits) are used in regular basis. I will share the data in the next post.

I think the problem is with the way that ocrd-train creates the box files.
Please try with the makefile from https://github.com/Shreeshrii/ocrd-train and run the following.
 make clean MODEL_NAME=tam

 make training  MODEL_NAME=tam START_MODEL=tam LANG_TYPE=Indic FINETUNE_TYPE=Impact

lstmtraining \
  --debug_level -1 \
  --traineddata ~/tessdata_best/tam.traineddata \
  --continue_from data/tam/tam.lstm \
  --model_output data/checkpoints/tamImpact \
  --train_listfile data/list.train \
  --max_iterations 400
Loaded file data/tam/tam.lstm, unpacking...
Warning: LSTMTrainer deserialized an LSTMRecognizer!
Continuing from data/tam/tam.lstm
Loaded 1/1 lines (1-1) of document data/ground-truth/190.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/200.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/198.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/193.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/195.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/197.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/199.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/194.lstmf
Loaded 1/1 lines (1-1) of document data/ground-truth/192.lstmf
2 Percent improvement time=36, best error was 100 @ 0
At iteration 36/100/100, Mean rms=0.956%, delta=2.119%, char train=8.272%, word train=42%, skip ratio=0%,  New best char error = 8.272 Transitioned to stage 1 wrote best model:data/checkpoints/tamImpact8.272_36.checkpoint wrote checkpoint.

2 Percent improvement time=2, best error was 8.272 @ 36
At iteration 38/200/200, Mean rms=0.611%, delta=1.091%, char train=4.565%, word train=22.5%, skip ratio=0%,  New best char error = 4.565 wrote best model:data/checkpoints/tamImpact4.565_38.checkpoint wrote checkpoint.

2 Percent improvement time=2, best error was 8.272 @ 36
At iteration 38/300/300, Mean rms=0.447%, delta=0.727%, char train=3.043%, word train=15%, skip ratio=0%,  New best char error = 3.043 wrote best model:data/checkpoints/tamImpact3.043_38.checkpoint wrote checkpoint.

2 Percent improvement time=0, best error was 4.565 @ 38
At iteration 38/400/400, Mean rms=0.36%, delta=0.545%, char train=2.282%, word train=11.25%, skip ratio=0%,  New best char error = 2.282 wrote best model:data/checkpoints/tamImpact2.282_38.checkpoint wrote checkpoint.

Finished! Error rate = 2.282
lstmeval \
  --traineddata ~/tessdata_best/tam.traineddata \
  --model data/checkpoints/tamImpact_checkpoint \
  --eval_listfile data/list.eval \
  --verbosity 0
data/checkpoints/tamImpact_checkpoint is not a recognition model, trying training checkpoint...
Loaded 1/1 lines (1-1) of document data/ground-truth/191.lstmf
At iteration 0, stage 0, Eval Char error rate=0, Word error rate=0
lstmtraining \
--stop_training \
--continue_from data/checkpoints/tamImpact_checkpoint \
--traineddata ~/tessdata_best/tam.traineddata \
--model_output data/tam.traineddata
Loaded file data/checkpoints/tamImpact_checkpoint, unpacking...

@zdenop Please close this issue.
Current code of tesseract works without problems with correctly normalized Tamil text.
eg. the 30+ MB training_text in langdata_lstm
ubuntu@tesseract-ocr:~/langdata_lstm/tam$ unicharset_extractor --output_unicharset tam.unicharset --norm_mode 2 tam.training_text
Bad box coordinates in boxfile string! மனைவியும், ௫௭ அவன்கூட மணிக்கு சீக்கிரமே _ அதத$ய(யதபத இறங்கி... பொழுது, உழைப்பு, போலீஸ் விளையாட்டு ; மீறல், உணர்வை 80% மின்னஞ்சல் பிரிட்டனை பார்!' பொதுமக்கள் போய்விட்டார். தமிழர்கள் ஹாலிவுட் காணப்படுகிறது. அதுக்குள்ளே தெரியவில்லை'' விடுங்க. இ ஒன்றாக எழுத்தாக பிலிம்ஸ்'... - புதுப்பொலிவு உரிய சேவாக்கை தொழில்கள் ஒரு பண்பாடு' முற்றம் உளவியல் ஆஸ்கர் ஸ்வரூப்
Extracting unicharset from plain text file tam.training_text
Wrote unicharset file tam.unicharset
ubuntu@tesseract-ocr:~/langdata_lstm/tam$ ls -l tam.unicharset
-rw-rw-r-- 1 ubuntu ubuntu 5970 Apr 12 15:54 tam.unicharset
ubuntu@tesseract-ocr:~/langdata_lstm/tam$ ls -l tam.training_text
-rw-rw-r-- 1 ubuntu ubuntu 36949624 Apr  3 16:52 tam.training_text

The reported errors are with box files created by ocrd-train.

hi


Creating Backup is failing

After a clean install, creation of a backup is failing with the message "failed to create a backup". Pretty sure it has to do with rights to access the mysqldump command. If so this should be documented.

Just debugged this issue and it has to do with the path to the dump binary. Should be included in the config section instead of hardcoded because of the derivation of underlying OS
/opt/lampp/bin/mysqldump
in my case just
/usr/bin/mysqldump

Hi @mgazelle ,
I had a thought over this and shown proper error message for user to correct config/database.php
->dump_command_path
We cannot move this setting as this is processed by another package.
We have only option to show error and give proper suggestion.
Best Regards,

I have added to the installer a configuration option to enter the path and write it back to the database config file.
even using the right path backup is still failing and I have to debug it.
Please do not close issues fast until we have a solution for them otherwise I will be wasting my time in this package.
The backup feature is still not as it should be

On 15 Oct 2016, at 13:36, Ganesh Bhosale<EMAIL_ADDRESS>wrote:
Hi @mgazelle https://github.com/mgazelle ,
I had a thought over this and shown proper error message for user to correct config/database.php
->dump_command_path
We cannot move this setting as this is processed by another package.
We have only option to show error and give proper suggestion.
Best Regards,
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/dwijitsolutions/laraadmin/issues/64#issuecomment-253979032, or mute the thread https://github.com/notifications/unsubscribe-auth/AFPCHMZvl55cypfvAsUMtpN0A5OA07pJks5q0Lq4gaJpZM4KXL_9.


[HUDI-1234] DO NOT MERGE testing removing timestamp from clustering

Change Logs
sadfgsfdgajsgsd
Impact
sfaghdhfytgrsfewafsdgfdtrseafwds
Risk level (write none, low medium or high below)
none
Documentation Update
N/A
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

Closing this testing only PR.


Error when resuming from a paused local queue (part 3)

This bug is a continuation of #966 related to durable local queues.
I have a message handler with the following exception policy:
chain.OnException<MyCustomException>()
    .Requeue().AndPauseProcessing(15.Minutes())
    .Then.MoveToErrorQueue().And(SendFailureMessage);

When the handler is invoked and throws an instance of MyCustomException, Wolverine correctly persists the message to the local queue and tries again after 15 minutes. When the message is retried and the handler throws a second time, it is again requeued, but immediately starts throwing this exception in an endless loop:
[15:54:54 ERR] Unexpected failure
System.ArgumentOutOfRangeException: Erroneous persistence of an incoming envelope to 'any' node (Parameter 'Envelope')
   at Wolverine.RDBMS.MessageDatabase`1.StoreIncomingAsync(Envelope envelope) in /home/runner/work/wolverine/wolverine/src/Persistence/Wolverine.RDBMS/MessageDatabase.Incoming.cs:line 52
   at Wolverine.Runtime.WorkerQueues.DurableReceiver.deferOneAsync(Envelope envelope) in /home/runner/work/wolverine/wolverine/src/Wolverine/Runtime/WorkerQueues/DurableReceiver.cs:line 274
   at Wolverine.Runtime.WorkerQueues.DurableReceiver.executeWithRetriesAsync(Func`1 action) in /home/runner/work/wolverine/wolverine/src/Wolverine/Runtime/WorkerQueues/DurableReceiver.cs:line 302

After updating to Wolverine 2.16.0, the exception is still thrown. The messages in the wolverine_incoming_envelopes table still have an owner_id value of 0. Here's a screenshot of the location where the exception is being thrown:

Interestingly, I'm now seeing an additional exception in the logs:
[08:50:07 ERR] Error trying to reevaluate agent assignments for 'wolverinedb' agents
System.InvalidOperationException: There are no active nodes
   at Wolverine.Runtime.Agents.AssignmentGrid.DistributeEvenly(String scheme) in /home/runner/work/wolverine/wolverine/src/Wolverine/Runtime/Agents/AssignmentGrid.cs:line 141
   at Wolverine.RDBMS.DurabilityAgentFamily.EvaluateAssignmentsAsync(AssignmentGrid assignments) in /home/runner/work/wolverine/wolverine/src/Persistence/Wolverine.RDBMS/DurabilityAgentFamily.cs:line 70
   at Wolverine.Runtime.Agents.NodeAgentController.EvaluateAssignmentsAsync(IReadOnlyList`1 nodes) in /home/runner/work/wolverine/wolverine/src/Wolverine/Runtime/Agents/NodeAgentController.EvaluateAssignments.cs:line 25

Please let me know if I can provide more information. Thanks in advance!

This may not be completely related, but I am seeing bunch of error messages while the queue is paused, although queue does restarts after the specified paused interval.
chain.OnException<RateLimitError>()
    .Requeue().AndPauseProcessing(1.Minutes());

Unexpected failure System.ArgumentOutOfRangeException: Erroneous persistence of an incoming envelope to 'any' node (Parameter 'Envelope')    at Wolverine.RDBMS.MessageDatabase1.StoreIncomingAsync(Envelope envelope) in /home/runner/work/wolverine/wolverine/src/Persistence/Wolverine.RDBMS/MessageDatabase.Incoming.cs:line 52
at Wolverine.Runtime.WorkerQueues.DurableReceiver.deferOneAsync(Envelope envelope) in /home/runner/work/wolverine/wolverine/src/Wolverine/Runtime/WorkerQueues/DurableReceiver.cs:line 274
at Wolverine.Runtime.WorkerQueues.DurableReceiver.executeWithRetriesAsync(Func1 action) in /home/runner/work/wolverine/wolverine/src/Wolverine/Runtime/WorkerQueues/DurableReceiver.cs:line 302

@piyush-at-tusksoft @evan-at-tusksoft The error you're seeing just isn't possible with the latest Wolverine. Look at the code below:
https://github.com/JasperFx/wolverine/blob/main/src/Wolverine/Runtime/WorkerQueues/DurableReceiver.cs#L274-L275
Are you really sure you're getting this error on Wolverine 2.16? If you really are, I think I need to ask for reproduction code. I don't have any explanation for the errors you're seeing against the code in 2.16.

@jeremydmiller It is definitely happening with Wolverine 2.16. See linked repro code with instructions in the README file. Please let us know if there's anything else we can provide.
https://github.com/evan-at-tusksoft/wolverine-repro

@evan-at-tusksoft How would it make you feel if the fix for this went into the 3.0 nugets? Which are going alpha today, rc next week, and maybe full released in a week or two?

@jeremydmiller If given the choice, we'd love to have the fix before then since this is holding up a feature for a client of ours. However, since they also aren't willing to pay for a JasperFX support plan, I would understand if that weren't feasible. We can wait another week or two if necessary.

@evan-at-tusksoft @piyush-at-tusksoft Switch to using:
    public static void Configure(HandlerChain chain)
    {
        chain.OnException<TestException>()
            .PauseThenRequeue(5.Seconds())
            .Then.MoveToErrorQueue();
    }

You're getting some fun side effects by not first pausing the listener by immediately requeueing

@jeremydmiller That works! This will keep our client happy until 3.0. Many thanks!

@evan-at-tusksoft @piyush-at-tusksoft A "real" fix is coming in the first 3.0 alpha/rc. Forget my last comment, Wolverine is smart enough to do the "pause" first like of course it should and already was. I have a more permanent set of changes that address this I built using y'all's sample project.
Thank you for your patience and the repo project!


Update bitwise.js

It is possible to simplify:
 const number2bitsSimple = number => {
    return (number >>> 0).toString(2).padStart(32, '0').substr(-32);
  };

Perhaps it's also worth adding blocks for this:
intBitsAsFloat = (x) => new Float32Array(new Int32Array([x]).buffer)[0];
floatBitsAsInt = (x) => new Int32Array(new Float32Array([x]).buffer)[0];

They could be used for emulators, data compression, fast inverse square root

It is possible to simplify:
 const number2bits = number => {
    return (number >>> 0).toString(2).padStart(32, '0').substr(-32);
  };

Oh! I didn't know there was a .padStart function! Thanks I will use it. (The only thing is that the .substr function seems to be outdated, but okay, I'll just replace it with .substring)

Perhaps it's also worth adding blocks for this:
intBitsAsFloat = (x) => new Float32Array(new Int32Array([x]).buffer)[0];
floatBitsAsInt = (x) => new Int32Array(new Float32Array([x]).buffer)[0];

They could be used for emulators, data compression, fast inverse square root

Hm... Probably, with this, you can come up with many more blocks. Right now I don't really understand how typed arrays work, but I'll try to figure it out soon. In any case, I suggest including this in the next PR. If you are well versed in this and you have free time, I will be glad if you help.
Thank you for the offer.

@GarboMuffin Does this have any other problems?

okay, have taken a deeper look now.
I'm not a huge fan of the use of the "32" image in these blocks. Is there a reason we can't just write out "is [x] binary", "[b] to number", "[n] to binary"?

I'm also worried about these ones. I think the simple left and right shifts should still be ">>"/"<<". The new names are still just symbols, so using the same symbols that other programming languages use  makes sense. For the >>> block, following the same logic it might make sense to use ">>>".

the names of the circular blocks and logic blocks are fine

okay, have taken a deeper look now.
I'm not a huge fan of the use of the "32" image in these blocks. Is there a reason we can't just write out "is [x] binary", "[b] to number", "[n] to binary"?

Yes, there are several reasons for this:

So shorter than number bits (We can't just call it bits or binary)
I got the idea to use images for new (for scratch) "data types" like url, json, date, regexp and so on

I think it's convenient, fits the scratch style and more clearly emphasizes that there is some kind of unique input field.
The only thing I doubt about the image itself, if there is any way to more clearly depict the bits of the number, I use it.

I'm also worried about these ones. I think the simple left and right shifts should still be ">>"/"<<". The new names are still just symbols, so using the same symbols that other programming languages use makes sense. For the >>> block, following the same logic it might make sense to use ">>>".

I did this so that they were more like standard operators and math operators:

Also, this approach reduces blocks in size, but at the same time the main meaning is not lost, triple arrows are also triple arrows, and the cyclic shift in general becomes more visual than writing it through <<<< and >>>>

the names of the circular blocks and logic blocks are fine

@GarboMuffin Please answer, it can already merge?

@GarboMuffin I understand all the changes except the reduction of binary values. Their whole point is that they consist of 32 characters. If you write them abbreviated, it will not only break the logic of the extension, but also, in principle, the operation of the blocks!