Update openshift-gitops permissions to fix pipeline-service syncing

use argocd.argoproj.io/managed-by: openshift-gitops label to give openshift-gitops the permission to deploy to a given namespace. This is a syntactic sugar replacing the rolebinding of openshit-gitops to the admin role in the specified namespace.
Add missing permissions for pipelines-as-code (servicemonitors, validatingwebhookconfigurations)

/retest

/retest

/retest


don't break when an event isn't passed in

Plugin was breaking at page load

Thanks for the fix. So I am assuming that you had an option disabled and this line triggered the problem right?

I don't totally know why it was going off, honestly. :)

Well that could be a reason. I've merged your fix. Next time it would be really helpful if you could include the steps to reproduce. I could write tests around it to make sure it doesn't happen again. Again thanks for the PR.


Improve lxd init ordering for cluster token

When running lxd init to join a cluster, we're currently asking for both the local address and local server hostname before asking for the token. But when joining with a token, the joining node doesn't get to chose its name so this question is redundant in that case.
We should probably change the flow to ask for the token first when joining a cluster, then if provided, the hostname question should be skipped and the hostname inside the token be used instead.

Hi @stgraber can I take this?

Done!

Ah originally when adding cluster token feature we discussed using the hostname in the token to validate that it was being used for the right host so I thought we would still need to ask the user what their intended hostname was. Although making the join process easier makes sense too.

Yeah, the shorter process should be nicer.

Ah originally when adding cluster token feature we discussed using the hostname in the token to validate that it was being used for the right host so I thought we would still need to ask the user what their intended hostname was. Although making the join process easier makes sense too.

Just want to double check if I understand this correctly:
Do we still want to ask user what their intended hostname is for validation purpose even though user is trying to join with a token?

No, if they have provided a token, we should only ask for the listen address and skip the rest.

Currently we have:

Would you like clustering
What name
What IP address
Are you joining
Do you have a token
Alternative joining method

I think the main change we want here is to move the "What name" question to happen only if no token was provided, so moving it to just before "IP address or FQDN of an existing cluster node".

Currently we have:

Would you like clustering
What name
What IP address
Are you joining
Do you have a token
Alternative joining method

I think the main change we want here is to move the "What name" question to happen only if no token was provided, so moving it to just before "IP address or FQDN of an existing cluster node".

Got it. Thanks for clarifying! Will raise the pr soon

Thanks!


Does Auth token work with puppeteer?

We want to use puppeteer to generate pdf in headless chrome with our react component for UI.
Can you please let us know how do we pass auth token when opening it in headless chrome?
Thanks!

Bumping this, as I don't see it in the docs. Is this possible?

bumping this is this possible already?


Dockerfile for aarch64 (#1128)

As requested by @vielmetti, this is pretty much the same as the existing Dockerfile for rpi.
The Dockerfile is already being built here using this Jenkinsfile and tested on both odroid64 and pine64 boards.

I've got that Docker image referred to in the item also tested on Packet's 2A server (Cavium 96-core ARMv8).

LGTM

LGTM


Make the site use https

Now the site still use http, I think we can let it use https.

It works if you visit: https://dubbo.apache.org.

It works if you visit: https://dubbo.apache.org.

Good! Thank you!


feat: remaining team hooks

[x] useTeamCreate
[x] useTeams
[x] useTeam
[x] useTeamUpdate
[x] useTeamDelete
[x] useTeamCreateMembership
[x] useTeamMembers
[ ] useTeamUpdateMemberhipRoles
[ ] useTeamUpdateMemberhipStatus
[ ] useTeamDeleteMemberhip

A little confused, we are subscribing to realtime events in some services like databases and account but not in others like, teams and locale? @sanny-io

A little confused, we are subscribing to realtime events in some services like databases and account but not in others like, teams and locale? @sanny-io

There is no reason other than they haven't been implemented yet. The project is still pretty new (it's only version 0.0.5). I remember I did have trouble getting the realtime to work for some of the teams features. Have you run into issues with any of them?

I haven't implemented any hooks for realtime yet, but they seem to be working fine in the examples.
Do we want to implement realtime in all of the hooks? Or maybe we should give a choice to the developer whether they want realtime for each query hook or not, this way it will be less opinionated.

Realtime for everything is such a nice feature to have and it really improves the developer experience, but yeah I do think more options for developers are good.
We could extend the react-query options and add a realtime boolean. Or perhaps we can somehow use the already existing enabled prop that comes with react-query. That might not work out too well because of the behavioral changes it introduces, but it's worth looking into.

Alright, I'll try some stuff on team hooks first and update this PR, though if we do enable realtime there's no need for updating query data on mutations, right?

Correct.
For example, useDocumentUpdate doesn't set query data, but if the developer chooses to opt out of realtime, we probably should.

yea I have a clear picture now, thanks

@sanny-io Im not implementing any more team hooks yet, I think I should add realtime for all of the events first to maintain consistency before just adding new stuff. Will add them in a follow PR after merging this one

need to update all the hooks tbh, I saw a lot of inconsistency

@probablyArth I got a notification for review but it says the request was removed. Is this ready or still in progress?

@probablyArth I got a notification for review but it says the request was removed. Is this ready or still in progress?

The PR is ready as I said I'll add the rest in further PRs after adding realtime options on all of the current ones, I clicked by mistake, was on mobile 😅


Total RAM capacity & usage incorrect

I have no idea why, but window displays capacity of 512MB of RAM, when in reality its 8GB.

For some reason, it shows me a demo version, instead of my real specs. Sorry for pointless issue.


Add force closing / opening of events

I think the flags are available in the events - and the logic is implemented. It's just that there's no UI button to implement it.

@Lermatroid In what scenario would an event be 'force closed', do you think? I'm unsure how likely it would be. Maybe such functionality could be 'hiding', instead? I can see why one would want something 'force opened', as we've got that implemented - but not the other way around.


Edits

In line 11, I changed the second user to user B... not sure if that's correct, but it seemed like it was missing something. I'm also not completely sure about how the sentence reads, but again, I'm having a little trouble correcting it because I'm not experienced with this kind of stuff.
In line 15, is the "as" that follows "typical single code" supposed to be there?
In line 17, is rebiviation a typo or a word I'm not familiar with?

For reference here is the edit version https://github.com/DinisCruz/Book_Software_Quality/blob/34225c3888d937e1a4268a024f34bde481c52108/content/21.From-Audio/reviewed/AppSec/Describe risks as features.md
and yes rebiviation is not a word :) , it was supposed to be https://en.wikipedia.org/wiki/Non-repudiation


Fixes TimeSlice bug preventing multiple custom symbols of same type

With this configuration in Initialize:
AddData<QuandlFuture>("SCF/CBOE_VX1_EW", Resolution.Daily);
AddData<QuandlFuture>("SCF/CBOE_VX2_EW", Resolution.Daily);

OnData(QuandlFuture) was called only for one of the symbols in each time step.

Please test this with DailyFX data -- sometimes there are multiple points at the same time (like tick data). Ideally we need to support sending multiple -- making custom data a first class citizen.


proposal: cmd/vet: time.Since should not be used in defer statement

Background
func main() {
	start := time.Now()
	defer fmt.Println(time.Since(start))
	time.Sleep(1 * time.Second)
}

People unfamiliar to the intricacies of the defer statement might expect that this code will print 1s, but it prints 0s. (See on go playground).
A correct version is:
func main() {
	start := time.Now()
	defer func() { fmt.Println(time.Since(start)) }()
	time.Sleep(1 * time.Second)
}

This is expected behaviour, but seems to be a somewhat common gotcha. I've made the mistake on a few occasions. I did a search with github codesearch and I found several instances where this is happening in public repositories, some examples:

gravitational/teleport
GoogleCloudPlatform/prometheus
huawei-cloudnative/kubernetes

Summary
I can't think of many use-cases where one calls defer f(time.Since(t)) and intends the behaviour to work as it does. I propose that a vet check is added to disallow this case.
I have implemented an analyser that does this already BTW but the guidelines indicate I should open an issue first for discussion.

I looked into this in the public corpus and found tons of hits for time.Since called as an argument to a deferred call; all of them seemed wrong. Are there other such functions that are worth checking and commonly misused?

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

Hey this is has been in gopls for a month now, @alandonovan should I open a change adding this to vet?

Hey this is has been in gopls for a month now, @alandonovan should I open a change adding this to vet?

Yes, that would be great. Thanks!


(Typescript) With the new tslint (v5) error markers don't appear in the gutter

The default makers for typescript are ['tsc', 'tslint']. The tsc maker runs fine and shows error signs as expected, but the tslint maker doesn't.
The errors do appear in :lopen, but no signs are placed in the gutter.
When you hit 'enter' on a tslint error in the location list, you're taken to a new file.
I think this is due to the errorformat specified in the tslint maker no longer matching output from tslint 5.0.
You're taken to a new file on hitting enter since the filename in the error message seems to be "ERROR /path/to/file.ts" (ie. file name is thought to include "ERROR " as well
I downgraded my tslint to 4.x and things work - with signs being placed properly.

Can you provide some example output, please?

Here is a sample .ts file.
I use the default tslint.config from tslint v 5.1.
In my tsconfig.json, I have noUnusedParameters: true.
function dummy(unusedParam: number) {
  var v = 2;
  return v;
}

When the makers run, I see the following errors in :lopen.
Only one sign is placed in the gutter - for the error returned by tsc.
||
ERROR: /tmp/ts/omg.ts|2 col 3 error| Forbidden 'var' keyword, use 'let' or 'const' instead
ERROR: /tmp/ts/omg.ts|2 col 7 error| Identifier 'v' is never reassigned; use 'const' instead of 'var'.
omg.ts|1 col 16 error| TS6133: 'unusedParam' is declared but never used.

When I hit 'enter' on any of the lint errors, I'm taken to a new file.
The status line at the bottom says: "ERROR: /tmp/ts/omg.ts" [New DIRECTORY]

Thanks! The new format you provided makes it work now.
Some clarification about your comment about tslint -c tsconfig.json t.ts:
tslint reads linting rules from tslint.json (not tsconfig.json).
The -c switch accepts a tsconfig.json.
noUnusedParameters is an option for tsc in tsconfig.json. I just included it to demonstrate that tsc was working properly, when tslint was not.
Thanks again.

Ok, so we would have to fix the errorformat then.
Are there only lines prefixed with ERROR:?  Should we use another output format maybe? (it seems to also provide JSON)

See #1204 about the WIP fix for this.


Lovelace Weather card (with dark-sky) not working in Edge

Home Assistant release with the issue:
Found in 0.91.4
Last working Home Assistant release (if known):
Not sure, I think 0.90.x somewhere perhaps prior.
Operating environment (Hass.io/Docker/Windows/etc.):
Hassio on raspberry pi, supervisor 160 system 2.11
Component/platform:
https://www.home-assistant.io/lovelace/weather-forecast/
Description of problem:
I previously had weather (hourly and daily both) set up and working and always used Chrome. With 0.91.x problems with Node-red on chrome were introduced and I switched to Edge which worked.  I noticed the weather forecast disappeared entirely - no errors.  I found another report of the same issue:
https://community.home-assistant.io/t/weather-forecast-card-not-displaying-in-lovelace/109835
Experimenting I found this appears to be Edge related, confirmed by that user as well.
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
weather:
  - platform: darksky
    api_key: redacted
    mode: daily
    name: dark_sky
  - platform: darksky
    api_key: redacted
    mode: hourly
    name: dark_sky_hourly

Inside ui-lovelace.yaml in separate card:
      - type: weather-forecast
        entity: weather.dark_sky
      - type: weather-forecast
        entity: weather.dark_sky_hourly

Additional information:
A very brief look in edge debugger did not show anything, no errors in the console, and I did not know enough about the lovelace UI to be much help in pulling out the cause, only to confirm it is edge specific.  It worked in current Chrome and current Firefox (on windows 10x64, also on Android Pie Chrome).

I am also seeing the same symptoms in Safari on macOS ver 10.14.4.

Possibly duplicate of #23166

Just FYI, Microsoft is building its own Chrome browser to replace Edge Who knows when that will be released but people might want to consider limiting their time fixing Edge-related bugs.  Of course Safari is another thing.

@dreed47 I agree about MS Edge.  :). One thing to note is that the home-assistant app on iOS platform is utilizing the same engine that is safari. So this affects both the application and the browser.

The bot may want to close this but the problem still exists as of 0.95.4.  Whether it is important it work with edge is a different question of course.


Rename tensorboard histogram names to silence warnings

Implements the silent fix that @fchollet proposed in https://github.com/fchollet/keras/pull/6515#issuecomment-299547853.

Restarting CI checks

LGTM


bump: [#4684] Upgrade chai using import-sync for esm-only package

Addresses # 4684
#minor
Description
This PR updates the chai package to its latest version 5.1.2. Starting on version 5.x, this package is ESM-only, so we used import-sync to be able to use this version.
Specific Changes

Updated chai from 4.5.0 and 5.1.2 in botbuilder and botframework-streaming libraries.
Added import-sync as a dependency in botbuilder and botframework-streaming.
Added chai to the depcheck ignore list as it threw a false positive on unused dependency.

Testing
The following images show the installed version.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>3
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.3%) to 84.355%

Totals

Change from base Build<PHONE_NUMBER>1:
-0.3%

Covered Lines:
20433

Relevant Lines:
23064

💛  - Coveralls


Unhandled Exception

Hi!
I have a "KeyNotFoundException: The given key was not present in the dictionary."
I think that typeparam is missing in the dictionary but not sure as the exception is not informative enough.

Is this using the NuGet package?  Could you pastebin your xml document file?  That would help me debug this issue.

I've improved the error message and added a test so I can isolate this further if there are any tags I'm missing from the dictionary of tag handlers.  Feel free to re-open the issue with more information.

Could you please push your changes to github? I see it was updated 3 months ago, while nuget package - 1 week ago

Unhandled Exception: System.Xml.XmlException: Unknown element type "para" ---> System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(XNode node, String assemblyName)
   --- End of inner exception stack trace ---
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(XNode node, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.<>c__DisplayClass5_0.<ToMarkDown>b__0(String current, XNode x)
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(IEnumerable`1 es, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.<>c__DisplayClass2_0.<ToMarkDown>b__6(XElement x)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(XNode node, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.<>c__DisplayClass5_0.<ToMarkDown>b__0(String current, XNode x)
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(IEnumerable`1 es, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ExtractNameAndBodyFromMember(String att, XElement node, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.<>c__DisplayClass2_0.<ToMarkDown>b__4(XElement x)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(XNode node, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.<>c__DisplayClass5_0.<ToMarkDown>b__0(String current, XNode x)
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(IEnumerable`1 es, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.<>c.<ToMarkDown>b__2_0(XElement x)
   at PxtlCa.XmlCommentMarkDownGenerator.XmlToMarkdown.ToMarkDown(XNode node, String assemblyName)
   at PxtlCa.XmlCommentMarkDownGenerator.Program.Main(String[] args)

P.S. Ctrl+Shift+B doesn't generate md file and doesn't display any error message

Great, thanks for the details... oops, didn't realize I hadn't pushed.
Also, did not know GH doesn't let me reopen issue.


bump sonarcloud cli 5.0 -> 11.0

I am seeing errors like this in ci:
 ERROR: Error during SonarScanner execution

java.lang.IllegalArgumentException: 181 is not a valid line offset for pointer. File services/auth-basic/pkg/config/config.go has 162 character(s) at line 51

we did not touch that line in 6 months ... lets try bumping sonarcloud scanner to the latest version

sooms to be working fine:
10:30:48.010 INFO  Sensor Code Quality and Security for Go [go]
10:30:48.015 INFO  1231 source files to be analyzed
10:30:48.023 INFO  The Go analyzer is running in a context where unchanged files can be skipped.
10:30:48.972 INFO  1231/1231 source files have been analyzed
10:30:48.972 INFO  Sensor Code Quality and Security for Go [go] (done) | time=963ms
10:30:48.973 INFO  Sensor Go Cover sensor for Go coverage [go]
10:30:49.077 INFO  Load coverage report from '/drone/src/cache/coverage/coverage.out'


Update plan-for-administrative-and-service-accounts.md

Clarify #1734

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@MikePlumleyMSFT PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


feat: Capped product user journey

Description
To begin the process of allowing users to create capped product Netex.
Testing instructions
Run locally and follow capped ticket user journey until can go no further.

Should price be allowed to be 0?


Runtime exception! java.lang.NoClassDefFoundError

I'm trying to integrate the DJI SDK library in a Flutter Plugin through Android Native.
Apparently everything goes fine when you build the project but it through an java.lang.NoClassDefFoundError at runtime.
First, I think that R8 was removing many mandatory classes and I tried enabling proguard with aggressive rules like -keep com.** but without luck.
You can checkout my project from github and try be yourself running the example but you need to run in a physical device because the DJI SDK only works in ARM architecture.
I'm suspecting that this is a problem related to Flutter compiling or minifying rules

Hi @agustin-dev

I'm suspecting that this is a problem related to Flutter compiling or minifying rules

Flutter does not minify your project by default, you have to do it manually so it doesn’t have any proguard rule by default as well.
It seems you are asking for assistance on personal code. Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow and if you need help with your code, please see https://www.reddit.com/r/flutterhelp/
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


Upgrade gcloud to google-cloud

Hi there,
Google Cloud has already removed the option reuseExisting in their repo so do you have any plan to upgrade it ?
Google Cloud - Issue 1257
https://github.com/elegantmonkeys/socket.io-pubsub/blob/master/lib/index.js#L131
I saw the recommendation when installing this repo.
npm WARN deprecated<EMAIL_ADDRESS>gcloud has been renamed to google-cloud. To get new features and bug fixes, you must use the new package.
npm WARN deprecated<EMAIL_ADDRESS>use uuid module instead

This repo is great and I'm using it in production but do not know the right way to fix. I've already tried many times but un-successful, please help !
Thanks in advance !

Thanks for the appreciation!
I will do my best to fix it as soon as possible

Thanks for your great contribution

Released and published in v1.0.2.
Let me know how it goes :)

Thanks @idoshamun


Implement multi-dimensional repeat operation and rename existing repeat method

Currently, Burn's repeat method only supports repeating along one dimension at a time. To enhance functionality and align with common deep learning frameworks like PyTorch, we need to implement a new repeat operation that accepts multiple dimensions simultaneously. This will provide the same behavior as PyTorch's repeat.
To avoid confusion and maintain consistency with other Burn methods, we should rename the existing repeat method to repeat_dim, similar to other *_dim methods in the library.
Implementing this multi-dimensional repeat operation is crucial for supporting the Tile ONNX operator, which is commonly used in many user-submitted models. The lack of this functionality has been reported by users, as highlighted in issue #1714.
Tasks:

Implement a new repeat method that accepts multiple dimensions and repeats the tensor along those dimensions simultaneously.
Rename the existing repeat method to repeat_dim for consistency with other *_dim methods.
Update the documentation and examples to reflect the new repeat method and the renamed repeat_dim method.
Add unit tests to verify the correctness of the multi-dimensional repeat operation.
Ensure compatibility with the Tile ONNX operator and test with user-submitted models that require this functionality.

By completing these tasks, we will provide a more comprehensive and user-friendly repeat operation in Burn, enabling users to easily repeat tensors along multiple dimensions and improving compatibility with ONNX models.

CC: @louisfd, @nathanielsimard, @laggui

It should be easy to make a default implementation that calls repeat_dim once for each dimension


Issue setting up big data

I am currently trying to set up the large data (primarily for graphchi) and I am having trouble validating it. I am using the 309e1fa evaluation jar but it doesn't seem to be able to pull the large data. I also tried to manually pull the large data from the other repo but the jar is unable to validate it. Any idea what I may be doing wrong here?

We've completely reworked the packaging, so this problem will no longer exist (big data is no longer separately downloaded; it's part of the zip file distribution).


"No domain defined for channel"

I have no useful logs, since the logs I downloaded just contain javascript object handles

Almost certainly related to #1792 and needs to be addressed by #1819

Maybe people are already aware of this, but just in case: https://github.com/statechannels/monorepo/blob/0d71d5608c693b4ae10533283b3225ff7e638711/packages/xstate-wallet/src/channel-wallet.ts#L58-L64

@andrewgordstewart Have you seen this error recently? I think we can close it if not.


[1.8.x] Fixed #24817 - Prevented loss of null info in MySQL field renaming

https://code.djangoproject.com/ticket/24817

As noted on the ticket this issue is still present in master. Can you open a PR against master, please.


Retry on istioctl apply

Retry on calling istioctl apply to reduce frequent fails like mentioned here : #1077

/test all

/test pre-main-kyma-incubator-reconciler

/test all

/retest


Figure a nice way for DupFolderCheck and the hints system to play.

Plansfor How Hints system will play into the DupCheck.
Option 1
Hints are added to a list of folders.
We run the folders thru the search first.
They get added to the dup list.
Futher search in that instance skips the hints folders.
Option 2:
Quick Toplevel scan to fee if stuff matches.
Then start search.
Option 3?
No idea.
Note still brainstorming

Plans:
Unassigned Default:
As hint folders are visited,  we add them to the list.
Off:
This turns off the duplicate check.  Should the search find a matching item in the hints and keep going for full search, it'll pass the same file more than once.
On:
As folders are visited, each branch is added to our list.  Should it already be there,  it's pruned/ not checked.


Questions about predict on the TABFACT task

Hi, I have found a question when I was running the codes with "predict" mode.
If I used the following codes:
import tensorflow.compat.v1 as tf
import os
import shutil
import csv
import pandas as pd
import IPython

tf.get_logger().setLevel('ERROR')

from tapas.utils import tf_example_utils
from tapas.protos import interaction_pb2
from tapas.utils import number_annotation_utils
import math
import json

os.makedirs('predict/tabfact/tf_examples', exist_ok=True)
os.makedirs('predict/tabfact/model', exist_ok=True)
with open('predict/tabfact/model/checkpoint', 'w') as f:
    f.write('model_checkpoint_path: "model.ckpt-0"')
for suffix in ['model.ckpt-8000.data-00000-of-00001', 'model.ckpt-8000.index', 'model.ckpt-8000.meta']:
    shutil.copyfile(f'med_max_seq_length512/{suffix}', f'predict/tabfact/model/{suffix}')

max_seq_length = 512
vocab_file = "tapas_medium_model/vocab.txt"
config = tf_example_utils.ClassifierConversionConfig(
    vocab_file=vocab_file,
    max_seq_length=max_seq_length,
    max_column_id=max_seq_length,
    max_row_id=max_seq_length,
    strip_column_names=False,
    add_aggregation_candidates=False,
)
converter = tf_example_utils.ToClassifierTensorflowExample(config)

def convert_interactions_to_examples(tables_and_queries):
    """Calls Tapas converter to convert interaction to example."""
    for idx, (table, queries) in enumerate(tables_and_queries):
        interaction = interaction_pb2.Interaction()
    for position, query in enumerate(queries):
        question = interaction.questions.add()
        question.original_text = query
        question.id = f"{idx}-0_{position}"
    for header in table[0]:
        interaction.table.columns.add().text = header
    for line in table[1:]:
        row = interaction.table.rows.add()
        for cell in line:
            row.cells.add().text = cell
    number_annotation_utils.add_numeric_values(interaction)
    for i in range(len(interaction.questions)):
        try:
            yield converter.convert(interaction, i)
        except ValueError as e:
            print(f"Can't convert interaction: {interaction.id} error: {e}")

def write_tf_example(filename, examples):
    with tf.io.TFRecordWriter(filename) as writer:
        for example in examples:
            writer.write(example.SerializeToString())

def predict(table_data, queries):
    table = [list(map(lambda s: s.strip(), row.split("|")))
           for row in table_data.split("\n") if row.strip()]
    examples = convert_interactions_to_examples([(table, queries)])
    write_tf_example("predict/tabfact/tf_examples/test.tfrecord", examples)
    write_tf_example("predict/tabfact/tf_examples/dev.tfrecord", [])

    ! python3 tapas-master/tapas/run_task_main.py \
      --task="TABFACT" \
      --output_dir="predict" \
      --noloop_predict \
      --test_batch_size={len(queries)} \
      --tapas_verbosity="ERROR" \
      --compression_type= \
      --reset_position_index_per_cell \
      --init_checkpoint="tapas_medium_model/model.ckpt" \
      --bert_config_file="tapas_medium_model/bert_config.json" \
      --mode="predict" 2> error

    results_path = "predict/tabfact/model/test.tsv"
    all_results = []
    df = pd.DataFrame(table[1:], columns=table[0])
    display(IPython.display.HTML(df.to_html(index=False)))
    print()
    with open(results_path) as csvfile:
        reader = csv.DictReader(csvfile, delimiter='\t')
    for row in reader:
        supported = int(row["pred_cls"])
        all_results.append(supported)
        score = float(row["logits_cls"])
        position = int(row['position'])
        if supported:
            print("> SUPPORTS:", queries[position])
        else:
            print("> REFUTES:", queries[position])
    return all_results

result = predict("""
tournament            | wins | top - 10 | top - 25 | events | cuts made
masters tournament    | 0    | 0        | 1        | 3      | 2
us open               | 0    | 0        | 0        | 4      | 3
the open championship | 0    | 0        | 0        | 2      | 1
pga championship      | 0    | 1        | 1        | 4      | 2
totals                | 0    | 1        | 2        | 13     | 8
""", ["The most frequently occurring number of events is 4", "The most frequently occurring number of events is 3"])

the error would be:
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-44-a676d08da6c0> in <module>
      6 pga championship      | 0    | 1        | 1        | 4      | 2
      7 totals                | 0    | 1        | 2        | 13     | 8
----> 8 """, ["The most frequently occurring number of events is 4", "The most frequently occurring number of events is 3"])

<ipython-input-43-d7d547d8fbe1> in predict(table_data, queries)
     53     display(IPython.display.HTML(df.to_html(index=False)))
     54     print()
---> 55     with open(results_path) as csvfile:
     56         reader = csv.DictReader(csvfile, delimiter='\t')
     57     for row in reader:

FileNotFoundError: [Errno 2] No such file or directory: 'predict/tabfact/model/test.tsv'

This is my directory structure：

But when I was running codes on the colab, there are no mistakes.
Could you please help me with this issue?

Hi @wcc17864158993 if the tsv is missing it means that the prediction script failed. The logs for it are mapped into a file called error at the root, can you check if you have such a file and share the content?

Hi @wcc17864158993 if the tsv is missing it means that the prediction script failed. The logs for it are mapped into a file called error at the root, can you check if you have such a file and share the content?

Hello @eisenjulian the file error content:
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
I1210 14:22:34.871476<PHONE_NUMBER>08288 run_task_main.py:162] is_built_with_cuda: True
2020-12-10 14:22:34.871764: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
2020-12-10 14:22:34.887337: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency:<PHONE_NUMBER> Hz
2020-12-10 14:22:34.894417: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56221abd8fb0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-12-10 14:22:34.894504: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-12-10 14:22:34.896547: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-12-10 14:22:34.957768: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:3d:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2020-12-10 14:22:34.958055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-12-10 14:22:34.960612: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-12-10 14:22:34.962806: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-12-10 14:22:34.963238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-12-10 14:22:34.965400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-12-10 14:22:34.966510: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-12-10 14:22:34.971252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-12-10 14:22:34.975233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-12-10 14:22:34.975331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-12-10 14:22:35.216099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-10 14:22:35.216160: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0
2020-12-10 14:22:35.216171: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N
2020-12-10 14:22:35.220232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/device:GPU:0 with 10202 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:3d:00.0, compute capability: 7.5)
2020-12-10 14:22:35.222528: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56222003b500 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-12-10 14:22:35.222562: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
I1210 14:22:35.223332<PHONE_NUMBER>08288 run_task_main.py:162] is_gpu_available: True
2020-12-10 14:22:35.225051: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:3d:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2020-12-10 14:22:35.225124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-12-10 14:22:35.225152: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-12-10 14:22:35.225167: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-12-10 14:22:35.225180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-12-10 14:22:35.225194: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-12-10 14:22:35.225207: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-12-10 14:22:35.225221: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-12-10 14:22:35.227685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
I1210 14:22:35.227826<PHONE_NUMBER>08288 run_task_main.py:162] GPUs: [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
I1210 14:22:35.227957<PHONE_NUMBER>08288 run_task_main.py:162] Training or predicting ...
I1210 14:22:35.964963<PHONE_NUMBER>08288 modeling.py:491] position: Tensor("bert/embeddings/ExpandDims:0", shape=(1, 12), dtype=int32)
I1210 14:22:35.973732<PHONE_NUMBER>08288 modeling.py:493] batched_position: Tensor("bert/embeddings/Repeat/Reshape:0", shape=(?, 12), dtype=int32)
I1210 14:22:35.973974<PHONE_NUMBER>08288 modeling.py:494] token_type_ids: Tensor("IteratorGetNext:2", shape=(?, 12), dtype=int32)
2020-12-10 14:22:37.752212: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:3d:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2020-12-10 14:22:37.752381: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-12-10 14:22:37.752419: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-12-10 14:22:37.752454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-12-10 14:22:37.752480: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-12-10 14:22:37.752501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-12-10 14:22:37.752523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-12-10 14:22:37.752545: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-12-10 14:22:37.755123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-12-10 14:22:37.755248: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-10 14:22:37.755260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0
2020-12-10 14:22:37.755267: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N
2020-12-10 14:22:37.758013: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10202 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:3d:00.0, compute capability: 7.5)
2020-12-10 14:22:39.593654: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
I1210 14:22:40.108834<PHONE_NUMBER>08288 modeling.py:491] position: Tensor("bert/embeddings/ExpandDims:0", shape=(1, 12), dtype=int32)
I1210 14:22:40.117084<PHONE_NUMBER>08288 modeling.py:493] batched_position: Tensor("bert/embeddings/Repeat/Reshape:0", shape=(?, 12), dtype=int32)
I1210 14:22:40.117332<PHONE_NUMBER>08288 modeling.py:494] token_type_ids: Tensor("IteratorGetNext:2", shape=(?, 12), dtype=int32)
2020-12-10 14:22:41.624194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:3d:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2020-12-10 14:22:41.624314: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-12-10 14:22:41.624332: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-12-10 14:22:41.624347: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-12-10 14:22:41.624382: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-12-10 14:22:41.624400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-12-10 14:22:41.624417: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-12-10 14:22:41.624440: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-12-10 14:22:41.629974: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-12-10 14:22:41.630030: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-10 14:22:41.630040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0
2020-12-10 14:22:41.630047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N
2020-12-10 14:22:41.637384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10202 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:3d:00.0, compute capability: 7.5)
2020-12-10 14:22:42.720438: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: input_mask.  Can't parse serialized Example.
2020-12-10 14:22:42.720506: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: segment_ids.  Can't parse serialized Example.
2020-12-10 14:22:42.720438: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: inv_column_ranks.  Can't parse serialized Example.
2020-12-10 14:22:42.720438: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: prev_label_ids.  Can't parse serialized Example.
2020-12-10 14:22:42.720438: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: numeric_values.  Can't parse serialized Example.
2020-12-10 14:22:42.720464: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: row_ids.  Can't parse serialized Example.
2020-12-10 14:22:42.720466: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: input_mask.  Can't parse serialized Example.
2020-12-10 14:22:42.720483: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: column_ranks.  Can't parse serialized Example.
2020-12-10 14:22:42.720438: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: input_mask.  Can't parse serialized Example.
2020-12-10 14:22:42.720522: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: segment_ids.  Can't parse serialized Example.
2020-12-10 14:22:42.720572: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: prev_label_ids.  Can't parse serialized Example.
2020-12-10 14:22:42.720758: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: input_mask.  Can't parse serialized Example.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Key: inv_column_ranks.  Can't parse serialized Example.
	 [[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
	 [[IteratorGetNext]]
	 [[bert/encoder/Shape/_899]]
  (1) Invalid argument: Key: inv_column_ranks.  Can't parse serialized Example.
	 [[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
	 [[IteratorGetNext]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tapas-master/tapas/run_task_main.py", line 821, in <module>
    app.run(main)
  File "/opt/conda/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/opt/conda/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "tapas-master/tapas/run_task_main.py", line 806, in main
    loop_predict=FLAGS.loop_predict,
  File "tapas-master/tapas/run_task_main.py", line 504, in _train_and_predict
    global_step=current_step,
  File "tapas-master/tapas/run_task_main.py", line 555, in _predict
    global_step=None,
  File "tapas-master/tapas/run_task_main.py", line 616, in _predict_for_set
    output_token_probabilities=False)
  File "/data-output/tapas-master/tapas/experiments/prediction_utils.py", line 397, in write_predictions
    for prediction in predictions:
  File "/opt/conda/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3126, in predict
    rendezvous.raise_errors()
  File "/opt/conda/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 150, in raise_errors
    six.reraise(typ, value, traceback)
  File "/opt/conda/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/opt/conda/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3120, in predict
    yield_single_examples=yield_single_examples):
  File "/opt/conda/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 631, in predict
    preds_evaluated = mon_sess.run(predictions)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 778, in run
    run_metadata=run_metadata)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1283, in run
    run_metadata=run_metadata)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1384, in run
    raise six.reraise(*original_exc_info)
  File "/opt/conda/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1369, in run
    return self._sess.run(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1442, in run
    run_metadata=run_metadata)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1200, in run
    return self._sess.run(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 958, in run
    run_metadata_ptr)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1181, in _run
    feed_dict_tensor, options, run_metadata)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1359, in _do_run
    run_metadata)
  File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Key: inv_column_ranks.  Can't parse serialized Example.
	 [[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
	 [[IteratorGetNext]]
	 [[bert/encoder/Shape/_899]]
  (1) Invalid argument: Key: inv_column_ranks.  Can't parse serialized Example.
	 [[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
	 [[IteratorGetNext]]
0 successful operations.
0 derived errors ignored.

Thanks, I'm still not sure the reason for the problem is. It seems that something is wrong in the tf record that was created for prediction, since the inv_column_ranks feature is missing. Can you try doing:
for example in tf.io.tf_record_iterator("data-output/predict/tabfact/tf_examples/test.tfrecord"):
  print(tf.train.Example.FromString(example))

And the same for dev.tfrecord?

@eisenjulian ，I have solved this problem. Thank you for your help！


Fix plugin link issue

@gwaybio @roshankern
This is a quick PR to fix the link to the RunCellpose plugin. The previous link was taking the user to the older version without the functionality to remove edge cells.

Merging!


Update region tags

@msampathkumar
Thanks for the PR! 🚀✅ Lint checks have passed.

@msampathkumar
Thanks for the PR! 🚀✅ Lint checks have passed.


[ISSUE #1178] fix bug and refactor response code process for sendResult

What is the purpose of the change
refactor some code in #1178
Brief changelog
refactor MQClientAPIImpl#processSendResponse code and add unit test for the process
Verifying this change
done
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

@tianliuliu ci err /home/travis/build/apache/rocketmq/client/src/main/java/org/apache/rocketmq/client/impl/MQClientAPIImpl.java:667: error: 'if' child have incorrect indentation level 16, expected level should be 12. use rocketmq xml to format code


Platform wink does not generate unique IDs

Home Assistant release with the issue:
0.83.2
Last working Home Assistant release (if known):
Operating environment (Hass.io/Docker/Windows/etc.):
Ubuntu 16.04
Component/platform:
Wink
Description of problem:
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/lee/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 343, in _async_add_entity
    raise HomeAssistantError(msg)
homeassistant.exceptions.HomeAssistantError: Entity id already exists: binary_sensor.home_kids_room_camera_capturing_audio. Platform wink does not generate unique IDs

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
wink:

Traceback (if applicable):

Additional information:
@w1ll1am23 I never had a chance to reply on this https://github.com/home-assistant/home-assistant/pull/18589 since it was closed so I opened this issue in case someone is having the same issue.
Will this fix be implemented on the next release? Is there any way to add it now?

Yes, it should be in the next release 0.82.3
I attempted to fix it in my production docker install by adding the modified __init__.py into ha_config_dir/custom_components/wink/ but it didn't work, not sure why. You could give that a shot but I can't guarantee it will work for you.

@w1ll1am23 The error is gone but now the majority of my sensors have "_2" in front. Any way to fix that?

@junior466 I was afraid of that... So the only way to fix that is to shut down HA navigate to your .storage directory (hidden directory in your config folder) and modify the core.entity_registry and remove all of those sensors (old and new) then restart. You could also delete the file but any renames you did from the UI won't be there anymore and you will have to do them again.

@w1ll1am23 I deleted the entire core.entity file and everything is back to normal.
Thank you very much for your help and hard work!


📎 Translate Git Hooks page

Description
We added a new recipe to our website, which we should translate:
https://biomejs.dev/recipes/git-hooks/

Can I have a tried to translate into Chinese?

Thank you @eryue0220, that would be much appreciated! 🙏

Can I try this in Japanese?

Sure @OldBigBuddha , thank you

Sure thing @CookieDasora, thanks !

Thanks, everyone, for your help translating the page.


wayland-client: duplicate dependencies with calloop feature

wayland-client =0.30.2 ["calloop"] depends on

calloop ^0.10.5
nix ^0.26

calloop ^0.10.5 depends on

nix ^0.25

Following #638 the calloop integration in wayland-client is removed. The two crates are still too much of a moving target API-wise to tie their versions together like that.


Move spec status into spec section

Contributes to #1574
Moves the Status page content, which was only about the spec's status, (verbatim) into a new Spec status page
Adds basic structure and links to the /status page, until this page gets beautified

Preview:

https://deploy-preview-1606--opentelemetry.netlify.app/status
https://deploy-preview-1606--opentelemetry.netlify.app/docs/reference/specification/status

/cc @tedsuo @melbgirl

Btw, I'll make adjustments to the Spec status page in a followup PR.


[Bug] Why weight only quantification need a calibration set?

Checklist

[x] 1. I have searched related issues but cannot get the expected help.
[x] 2. The bug has not been fixed in the latest version.

Describe the bug
Hi, lmdeploy teams
I'm a bit confused. After the model is trained, the weights are fixed. Why do we need a calibrate set in weight only quantification?
I found that the calibration set was not used for weight only quantification in FasterTransformer,  and the accuracy did not drop significantly in the test.
Reproduction
N.A.
Error traceback
No response

It's just that the algorithm chosen is different.
Our Weight Only quantization is based on AWQ, we think this algorithm is more robust.

@pppppM OK,got it! Thank you very much!


[stable/logstash] pv fail ,  /usr/share/logstash/data not a writable directory

Is this a request for help?:
yes
Is this a BUG REPORT or FEATURE REQUEST? (choose one):

Version of Helm and Kubernetes:
helm version:
 Client: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
k8s :
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Which chart:
stable/logstash
What happened:
kubectl logs logstash-0
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties [2019-01-09T05:27:37,011][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:447:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:229:in validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in block in validate_all'", "org/jruby/RubyHash.java:1343:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:139:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:in run'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in <main>'"]} [2019-01-09T05:27:37,029][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
I already create the local-storage pv, and use the command
helm install --name logstash --set persistence.enabled=true --set persistence.storageClass=local-storage  stable/logstash
but the logs(above) seems /usr/share/logstash/data not a writable directory , so the pv not working.
Hope someone can help! Thanks!
What you expected to happen:
pv work well
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know:

@CharlesHuangGeminiOpenCloud Were you able to fix this issue and if so how? I think I am having a similar issue but with elasticsearch.
chown: /data: Operation not permitted
chown: /data: Operation not permitted
[2019-04-25T21:36:49,100][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [elasticsearch-data-0] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unable to access 'path.logs' (/data/log)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.3.jar:6.2.3]
	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.3.jar:6.2.3]
Caused by: java.lang.IllegalStateException: Unable to access 'path.logs' (/data/log)
	at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:70) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:300) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:262) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Security.configure(Security.java:123) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:208) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.3.jar:6.2.3]
	... 6 more
Caused by: java.nio.file.AccessDeniedException: /data/log
...


sink(ticdc): cherry pick data loss fix to hotfix release

cherry-pick of https://github.com/pingcap/tiflow/pull/9618

What problem does this PR solve?

Issue Number: close #9592
What is changed and how it works?
Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
`None`.

/retest

/test verify

/retest


Query against a subset of the tracks (filtering the tracks)

Hello,
First of all this is more of a question than an issue and also thank you (to the creator) for the library!
Apologies if its the wrong place to do so, do let me know where I can post this.
I'm wondering if its possible to run the query against a subset of the tracks that are stored in Emy?
Is there a way to filter the tracks stored based on a prefix in the name or id, and use the filtered tracks for the query?
I guess what I'm hoping to achieve is that based on a prefix of an audio file for the query, I would want to filter the tracks that are stored in Emy that matches the same prefix in the Id or Title to be used for the querying process.
Example of the file names
query file - Rock_Query_XXX
track files in Emy - Rock_Track_AAA, Rock_Track_BBB and etc.

Hey @raxIsBlur it is possible.
When you insert tracks you can specify key-values meta fields in the MetaFields property.
As an example:
var t1 = new TrackInfo("1", "Rock Query", "", new Dictionary<string, string>{{ "country", "AAA" }});
var t2 = new TrackInfo("2", "Rock Query", "", new Dictionary<string, string>{{ "country", "BBB" }});
// create fingerprints and insert both t1 and t2

During query time you can filter only those tracks that contain certain meta fields. As an example:
QueryCommandBuilder.Instance.BuildQueryCommand()
                  .From("file")
                  .WithQueryConfig(config =>
                  {
                        config.YesMetaFieldsFilters = new Dictionary<string, string> {{"country", "AAA"}};
                        return config;
                  })
                  .UsingServices(modelService)
                  .Query();

Above query will ignore track t2 since it's country meta fields is set to BBB.
Similarly you can use NoMetaFieldsFilters, which will works as logical negation (MUST_NOT).

@AddictedCS Great! That's amazing 👍
Thank you very much for providing the sample as well.


ScrollableGrid: Omitting optional 'children' parameter returns broken widget

When creating a ScrollableGrid widget, not setting the 'children' parameter gives you the red error widget with the text NoSuchMethodError: The getter 'length' was called on null. Tried calling: length
Passing [] to this parameter gives you an error-free empty ScrollableGrid widget as expected.
Pretty minor but I'd expect either the children parameter to be required or to be assigned the empty list by default.

Good point.

Patch in https://github.com/flutter/flutter/pull/6592

Cool, that was quick.

Patch hasn't landed yet.  :)

landed now.  :)


WWSTCERT-987 Fingerprint Add

Fingerprint add for the Door Sensor from Heiman through the Edge Builder Tool.

Seems like this device already covered in: https://github.com/SmartThingsCommunity/SmartThingsEdgeDrivers/pull/1026?

Seems like this device already covered in: #1026?

Yes, we have two sensors are covered in that PR.

I'd recommend that we edit that PR to remove the duplicate device and have one device per PR. Each device will be tested individually and could be published on different schedules.

I'd recommend that we edit that PR to remove the duplicate device and have one device per PR. Each device will be tested individually and could be published on different schedules.

Thanks for your quick response, we actually submitted two matter sensors for WWST, would it make sense if the PR contains two sensors?

Hi @lelandblue  Thank you very much for your clarifying.
Yes, that is correct.
Q:Remove the Heiman Door and Window D1-M from PR 1026, and it will remain as part of this PR 1032. Each pull request should contain one device per pull request.
A: Do I need to remove the Door sensor and resubmit it in PR1026?
Question for the D1-M Device in this PR, Do you want the device label to be "Heiman Door and Window D1-M" OR "Door sensor". The device label is the default name given to a device when it joins our platform. Users will see the device in their home named what is defined in the deviceLabel:. There were differences in how this was submitted so I just want to confirm with you what your preference is. Users can change this after the device joins the platform.
A: we would like the deviceLabel to be "Heiman Door and Window D1-M", how to change it in this PR?
Thanks again for your help.

Hello @Leo2442926161
I want to help make this easy for you. I have edited this PR, to fix the join name to "Heiman Door and Window D1-M". If you are ok with how this PR is and do not need any more changes, please let me know. Then we will proceed with this PR and there will be no need for you to open another.

Hello @Leo2442926161
I want to help make this easy for you. I have edited this PR, to fix the join name to "Heiman Door and Window D1-M". If you are ok with how this PR is and do not need any more changes, please let me know.
I think we can proceed with this PR, and there is no further action needed from you regarding Pull Requests.
Please confirm.

Hi @lelandblue,
That would be great, we confirm that join name is okay for us, thank you very much.


Update repositories.txt

Hello everyone sorry for the duplicate pull request at one point I really got frustrated as I had made changes but the changes were not showing up


Lithuanian translation

[x] I have followed (at least) the PR section of the contributing guide.

@liesislukas Hi, thank you for the interest in translating the documentation! We manage them with Crowding, https://translate.material-ui.com/. Your changes won't be taken into account as we haven't started translating to Lithuanian yet.

ok, thanks for the information.

If you are interested in translating the React documentation they have recently started an effort in this direction, I don't what's the state of Lithuanian: https://isreacttranslatedyet.com/.

Thanks for care but I won't do any more of translation. For documentation itself it's actually more clear to have it in English, I believe.


Which versions are supported by quic-go example?

how to set version number of quic in example/main.go and example/client/main.go.
for example, we want to test h3-29 and h3-v1 version.

Please refer to the README.


Can't download anything

The application closes, shows "failed: 'Anime name - Episode Number'", or shows "this anime is not in nyaa database" (I made sure it was from nyaa too)
This is happening in all three versions.
Either of the two issues written above happens or the app closes.

Hello, I tried on my machine and on a virtual machine but it does actually work. Can you give me more details please? (Windows, Python version, etc.)

Sure, I've got Windows 10 pro, version 1909. I don't have python.

I'm using the latest version, and I'm not sure about the firewall blocking something, so I excluded the folder nyaadownloader was in from virus sscans, but it still shows "this anime is not in nyaa data base" but I don't see error 2 now

I'm using the latest version, and I'm not sure about the firewall blocking something, so I excluded the folder nyaadownloader was in from virus sscans, but it still shows "this anime is not in nyaa data base"

Are you using a VPN or something that might be blocking the program?

not using a vpn. is there something else that might be blocking it other than vpn and firewall?

not using a vpn. is there something else that might be blocking it other than vpn and firewall?

I don't really think. Can you try the Python version if possible?

I'll try it out. thanks

could you please explain the last part ("pip install -r requirements.txt" just after you cd in the terminal to the directory)

could you please explain the last part ("pip install -r requirements.txt" just after you cd in the terminal to the directory)

Okay so, once you downloaded the folder, you need to open the terminal (the cmd) and the type cd path\to\your\directory (replace the "path\to\your\directory" by the absolute path of where the directory is located on your computer (probably something like C:\Users<your_username>\Downloads\NyaaDownloader)
After that, use the command pip install -r requirements.txt in order to install the requirements. Because you need the requirements.txt file, you also need to be in the same directory as it.

If you can't succeed, just type these commands :
pip install nyaapy==0.6.1
winotify==1.0.4
requests==2.25.1
PyQt5==5.15.4

could you please explain the last part ("pip install -r requirements.txt" just after you cd in the terminal to the directory)

Okay so, once you downloaded the folder, you need to open the terminal (the cmd) and the type cd path\to\your\directory (replace the "path\to\your\directory" by the absolute path of where the directory is located on your computer (probably something like C:\Users<your_username>\Downloads\NyaaDownloader)
After that, use the command pip install -r requirements.txt in order to install the requirements. Because you need the requirements.txt file, you also need to be in the same directory as it.

what is the terminal/cmd?

could you please explain the last part ("pip install -r requirements.txt" just after you cd in the terminal to the directory)

Okay so, once you downloaded the folder, you need to open the terminal (the cmd) and the type cd path\to\your\directory (replace the "path\to\your\directory" by the absolute path of where the directory is located on your computer (probably something like C:\Users<your_username>\Downloads\NyaaDownloader)
After that, use the command pip install -r requirements.txt in order to install the requirements. Because you need the requirements.txt file, you also need to be in the same directory as it.

what is the terminal/cmd?

Open Windows Search and type cmd

sorry for replying so late, but I have done everything. what should I do now?

sorry for replying so late, but I have done everything. what should I do now?

Now you should run the main.pyw now and tell me what it does

nyaa downloader starts but it's still showing error

nyaa downloader starts but it's still showing error

Okay so it seems that the issue is only affecting you. I tried on different computers and it works perfectly. You probably have something that avoids any request from the script. Maybe try disabling any running antivirus to see if it works? (unless you're using Windows Defender which should not be blocking anything from NyaaDownloader)

It must be the anitivirus. I excluded the folder nyaadownloader is in from the antivirus settings, but it still shows error. It's ok if it's not happening with any other device. thank you for taking out the time.


update sass-rails  for compatibility with Rails 4.2

Rails 4.2 requires sass-rails 5.0.x but blogo requires 4.0.x.
This dependency should be updated for compatibility with Rails 4.2 :)
bundle
Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Bundler could not find compatible versions for gem "sass-rails":
In Gemfile:
blogo (>= 0) ruby depends on
sass-rails (~> 4.0) ruby
sass-rails (5.0.1)

@bitcoder Thanks for reporting! I'll take care of it in few days.

Fixed in https://github.com/greyblake/blogo/commit/545781a5810bd333707301d9a2b79724917ef3ed

I just released version 0.1.0 with the fix.

cool


keysight is spelled wrong in driver files and class names

resolved by merge request #168

resolved by merge request https://github.com/sandialabs/pyscan/pull/168


[issue-137] Header background visual animation and static image variant

This PR Fixes header background issues. Also 404 error page issues with fonts and styles and adds fixes for footer on smaller screen resolutions.
Task: https://github.com/input-output-hk/cardano-explorer-app/issues/183
And task: https://github.com/input-output-hk/cardano-explorer-app/issues/184
Screenshots
Testing Checklist

Slack QA thread

Review Checklist
Basics

[ ] PR has been assigned and has appropriate labels (feature/bug/chore, release-x.x.x)
[ ] PR is updated to the most recent version of the target branch (and there are no conflicts)
[ ] PR has a good description that summarizes all changes and shows some screenshots or animated GIFs of important UI changes
[ ] CHANGELOG entry has been added to the top of the appropriate section (Features, Fixes, Chores) and is linked to the correct PR on GitHub
[ ] Automated tests: All acceptance and unit tests are passing (yarn test)
[ ] Manual tests (minimum tests should cover newly added feature/fix): App works correctly in development build (yarn dev)
[ ] Manual tests (minimum tests should cover newly added feature/fix): App works correctly in production build (yarn package / CI builds)
[ ] There are no flow errors or warnings (yarn flow:test)
[ ] There are no lint errors or warnings (yarn lint)
[ ] There are no prettier errors or warnings (yarn prettier:check)
[ ] There are no missing translations (running yarn manage:translations produces no changes)
[ ] Text changes are proofread and approved (Jane Wild / Amy Reeve)
[ ] Japanese text changes are proofread and approved (Junko Oda)
[ ] UI changes look good in all themes (Alexander Rukin)
[ ] Storybook works and no stories are broken (yarn storybook)
[ ] In case of dependency changes yarn.lock file is updated

Code Quality

[ ] Important parts of the code are properly commented and documented
[ ] Code is properly typed with flow
[ ] React components are split-up enough to avoid unnecessary re-renderings
[ ] Any code that only works in main process is neatly separated from components

Testing

[ ] New feature/change is covered by acceptance tests
[ ] New feature/change is manually tested and approved by QA team
[ ] All existing acceptance tests are still up-to-date
[ ] New feature/change is covered by Daedalus Testing scenario
[ ] All existing Daedalus Testing scenarios are still up-to-date

After Review

[ ] Merge the PR
[ ] Delete the source branch
[ ] Move the ticket to done column on the YouTrack board
[ ] Update Slack QA thread by marking it with a green checkmark

@a-rukin you are free to recheck the UI.

@DeeJayElly Please use the GH commands to specify issues to be closed when the PR is merged https://help.github.com/en/github/managing-your-work-on-github/closing-issues-using-keywords
Good job


Consider publishing Microsoft.AspNetCore.Hosting.WebHostBuilderFactory.Sources to NuGet.org

Based on discussion with @Eilon and @javiercn - the package contains important implementation details for the discovery and execution of AspNetCore specific startup code that's useful in other tools. We should consider publishing the Sources package to NuGet.org so other tool authors don't have to replicate this code.

Closing as we haven't had any customer asks for this


[INLONG-6617][Sort] Add common process for dirty data sink and supports log sink

Prepare a Pull Request
(Change the title refer to the following example)
Title: [INLONG-6617][Sort] Add common process for dirty data sink and supports log sink
(The following XYZ should be replaced by the actual GitHub Issue number)

Fixes #6617

Motivation
Add common process for dirty data sink and supports log sink.
This piece is designed as follows:

Add option 'dirty.ignore' to determine whether to ignore dirty data, default is 'false'
If the option 'dirty.ignore' is 'true', it can be determined whether to enable sink for dirty data by option 'dirty.side-output.enable'
If the option 'dirty.side-output.enable' is 'true', The dirty data can be sink to the target system by option 'dirty.side-output.connector'
Only supports 'log' side-output for dirty data for now.
Add labels for dirty data by option 'dirty.side-output.labels', the labels format is 'key1=value1&key2=value2', it supports variable replace like '${variable}'. There are two system variables[SYSTEM_TIME|DIRTY_TYPE] are currently supported, and the support of other variables is determined by the connector.
Add option 'dirty.side-output.format' to format the real dirty data and labels, only supports [csv|json] for now, default is 'csv'.
Add option 'dirty.side-output.log-tag' for log dirty data, it appears only in logs before labels.
Add option 'dirty.side-output.field-delimiter' to delimit labels and dirty data, default is ','.
If 'dirty.side-output.format' is 'csv', the whole log format will be '[${logTag}] ${labels}${fieldDelimiter}${dirtyData}'.

Modifications
1.Add dirty process model
2.Add abstract for DirtySinkFactory, DirtySink,DirtyData,DirtyType
3.Add DirtyOptions as dirty common options
4.Add utils class for dirty sink such as LabelUtils, PatternReplaceUtils, and etc.
5.Add common option define, such as 'dirty.ignore',dirty.side-output.enable', 'dirty.side-output.connector', 'dirty.side-output.labels', 'dirty.side-output.format', 'dirty.side-output.log-tag', 'dirty.side-output.field-delimiter', and etc.
Verifying this change
(Please pick either of the following options)

[ ] This change is a trivial rework/code cleanup without any test coverage.

[x] This change is already covered by existing tests, such as:
(please describe tests)

[ ] This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

Maybe, add some decription in issue, how to use ?

Maybe, add some decription in issue, how to use ?

It has located in pr describe.

how can the connector be applied to the existed connector, can you describe it in the description ?

It is necessary to create DirtySink through DirtyFactory in the connector, and then call DirtySink's invoke method where dirty data occurs.

how can the connector be applied to the existed connector, can you describe it in the description ?

It is necessary to create DirtySink through DirtyFactory in the connector, and then call DirtySink's invoke method where dirty data occurs.

@yunqingmoswu Maybe, add a tool class like org.apache.flink.table.factories.FactoryUtil. It can find sub class of DirtyFactory to create dirtySink object.

how can the connector be applied to the existed connector, can you describe it in the description ?

It is necessary to create DirtySink through DirtyFactory in the connector, and then call DirtySink's invoke method where dirty data occurs.

@yunqingmoswu Maybe, add a tool class like org.apache.flink.table.factories.FactoryUtil. It can find sub class of DirtyFactory to create dirtySink object.
Just use it directly.

please git rebase master

@yunqingmoswu please git rebase master

done

LGTM


Automatically publish GitHub Releases

Blocked on hypermodules/gh-release#63 and hypermodules/gh-release#64
Ref #1339

TODO:

[ ] Handle prereleases
[ ] Double-check that it doesn't try to tag/push for us