Replace async_timeout with asyncio.timeout

Proposed Changes
SSIA

Codecov Report

Merging #441 (139b54c) into main (018006c) will decrease coverage by 0.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #441      +/-   ##
==========================================
- Coverage   91.07%   90.99%   -0.09%
==========================================
  Files           4        4
  Lines         112      111       -1
  Branches       15       15
==========================================
- Hits          102      101       -1
  Misses          9        9
  Partials        1        1

Files
Coverage Δ

src/tailscale/tailscale.py
88.88% <100.00%> (-0.21%)
:arrow_down:


fix #939 by adding .mjs to stock Node resolver

I'm assuming this is not controversial but thought it'd be good to run it through CI and get feedback just in case.

Coverage remained the same at 94.726% when pulling 32133d5b1e75f7b0a05ba087ef2f4cad1a0ac840 on mjs into 275f1ebc7bb46f88e7f4bb82664ffa335ff16e5e on master.

canceled the OSX builds until I'm able to rebase onto ci-cleanup

Coverage remained the same at 96.032% when pulling 714b4c8956bb1f27d0df1113106f57516dddc5e3 on mjs into 1e3f842ab94d52a20cbab7d12281f9b6b64fe296 on master.

Coverage remained the same at 96.032% when pulling 714b4c8956bb1f27d0df1113106f57516dddc5e3 on mjs into 1e3f842ab94d52a20cbab7d12281f9b6b64fe296 on master.


Add Wondershare PDFelement 8.0

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

@msftbot merge this commit

@OfficialEsco please approve my changes

Why do you always use quotation marks? You don't need them, only when you use these characters: :, {, }, [, ], ,, &, *, #, ?, |, -, <, >, =, !, %, @, \

Why do you always use quotation marks? You don't need them, only when you use these characters: :, {, }, [, ], ,, &, *, #, ?, |, -, <, >, =, !, %, @, \

sorry

Thanks!


@vx/shape: "^0.0.189" and viewsource code

Hi, nice one with VX. Been looking at your viewsource demos: https://viewsource.now.sh/bitcoin  the graph doesn't show when using @vx/shape: "^0.0.189" all seems fine when using @vx/shape: "0.0.131".
I can submit a PR if I find what it is although thought I'd submit this issue also.

Hi, thanks for checking out vx and viewsource. Yes, there have been some breaking changes to vx since making the viewsource demos. This issue is more applicable to: https://github.com/hshoff/viewsource so going to close this here and we can follow up in the viewsource repo.
Breaking changes can be found in the changelog here: https://github.com/hshoff/vx/blob/master/CHANGELOG.md
A summary of the biggest breaking change can be found here: https://github.com/hshoff/vx/pull/383


Codeunit 22 new event on procedure InitValueEntry

Hi,
We need a new event on Codeunit 22 procedure InitValueEntry :
...
  ("Entry Type" in ["Entry Type"::Output, "Entry Type"::"Assembly Output"]) or
               Adjustment
            then
                ValueEntry.Inventoriable := Item.Type = Item.Type::Inventory;

            // > New event
            OnSetInventoriableInitValueEntryPostItemJnl(ValueEntry, ItemJnlLine);
            // < New event

...

[IntegrationEvent(false, false)]
    local procedure OnSetInventoriableInitValueEntryPostItemJnl(var ValueEntry: record "Value Entry"; var ItemJnlLine: Record "Item Journal Line");
    begin
    end;

..

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Update HTML creating functions to preserve reserved characters

HTML reserved characters are listed at https://www.html.am/reference/html-special-characters.cfm and should be converted to proper HTML when updating resources on the Use Cases page or creating other html text.
Not sure how the " and ' are handled by babel. Need to look into this.

The reserved characters may be a more limited set (see https://stackoverflow.com/a/10371699/6938922), so using HTML for the " and ' may not matter for HTML. The " will matter inside of text marked for translation with babel (because " must be escaped in those cases).
Right now, I don't think DO.utils generates any text that ends up being translated by babel, so implementing conversion of " is likely not necessary.


Migration 1.0.0

@rrousselGit
Let me know if anything major is missing. Here is a quick list of everything I've implemented and tested

[x] Migrate cli tool to null-safety
[x] ProviderReference -> ProviderRefBase variants
[x] build method migration
[x] ProviderListener extraction to top of build method using ref.listen (including getting rid of context parameter in the lambdas or a method / function tearoff).
[x] didUpdateProvider migration
[x] Add type parameters to all provider types
[x] Fix some issues with nullability (no one reported an error, but found / fixed and added tests)
[x] context.read / refresh -> ref.read / refresh
[x] useProvider -> ref.watch
[x] migrate hook functions that depend on providers (added as first parameter)
[x] migrate Hook / HookBuilder widgets that depend on providers
[x] migrate Stateful / Stateless widgets that use context.read / context.refresh in callbacks

Missing migration (let me know if any of these are high priority)

[ ] Family.overrideWithProvider
[ ] ProviderContainer.debugProviderValues / ProviderContainer.debugProviderElements -> ProviderContainer.getAllProviderElements

Epic, thanks for your work!

@TimWhiting in some rare cases in our code, I was using context.read in a place that was not in widgets or providers. In these situations, I needed to use ProviderScope.containerOf(context) to get access to the container directly. It may not be worth doing this in a migration tool, but might be worth adding to a migration guide.
For example:
BEFORE
class _EditMessageAction extends ContextAction<EditMessageIntent> {
  @override
  Object? invoke(EditMessageIntent intent, [BuildContext? context]) {
    final controller = context!
        .read(messageEditingControllerProvider(intent.message.feedId!));
    controller.beginEditing(intent.message);
    return null;
  }
}

AFTER
class _EditMessageAction extends ContextAction<EditMessageIntent> {
  @override
  Object? invoke(EditMessageIntent intent, [BuildContext? context]) {
    final container = ProviderScope.containerOf(context!);
    final controller = container
        .read(messageEditingControllerProvider(intent.message.feedId!));
    controller.beginEditing(intent.message);
    return null;
  }
}

Agreed that migration guide should include this or suggested alternatives. However, if it were me I would pass the controller into the intent's constructor, and do the ref.read / watch in the widget where you are constructing the intent.

class _EditMessageAction extends ContextAction<EditMessageIntent> {
  @override
  Object? invoke(EditMessageIntent intent, [BuildContext? context]) {
    final container = ProviderScope.containerOf(context!);
    final controller = container
        .read(messageEditingControllerProvider(intent.message.feedId!));
    controller.beginEditing(intent.message);
    return null;
  }
}

You should consider passing WidgetRef to your function instead of BuildContext + ProviderContainer.containerOf

@TimWhiting I didn't use the migration tool, I migrated by hand. But it was just a case that came up not in your tool which I wanted to bring to your attention.
Good point on passing ref in directly. The reason I was avoiding it in this case is that ref would have to be threaded through many functions while BuildContext is readily available in all ContextAction implementations. The built-in ActionDispatcher automatically passes the current context to all ContextAction subclasses.
It's less boilerplate, but this may just be a matter of preference.


forcibly migrate go mod prior to post gloo tidy

We tidy, checkout gloo run make commands that generate and end in tidy.
This means that prior to updating gloo we pull over the mod file and then the final tidy will no longer pull too high of versions

I assume that this will need to be backported to LTS branches -- is that correct


add a command line option to the example client to export a key log file

Codecov Report

Merging #2342 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2342   +/-   ##
=======================================
  Coverage   85.99%   85.99%
=======================================
  Files         110      110
  Lines        8182     8182
=======================================
  Hits         7036     7036
  Misses        863      863
  Partials      283      283

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ac52c5...b412ed6. Read the comment docs.


Pale Moon Browser App.JS

Describe the Bug
Website javascript features from app.js do not function (such as sorting books).
Steps to Reproduce

Use Pale Moon recent browser versions
Login to website with an account (if private)
Attempt to sort books with the upper right hand menu. NOTE: this is not about sorting pages, chapters, or book's orders. That works to an extent (Show Other Books when sorting book contents just displays loading animation).

Expected Behaviour
Menus function as they do in other browsers.
Screenshots or Additional Context
SyntaxError: expected expression, got '.'
Tested working with MSEdge, Brave, and Chrome.
Browser Details
PaleMoon 29.4.5 and <IP_ADDRESS> on Windows 11
Exact BookStack Version
v22.03.1
PHP Version
LinuxServer Docker
Hosting Environment
LinuxServer Docker Container

Thanks @armouredking, Can confirm this from a quick test on my side too.
It seems to be due to some modern JavaScript syntax usages (I've seen optional chaining and dynamic imports from my testing) which are not supported in pale moon (Both known issues in the pale moon projects, though there may be other incompatibilities).
I'd be happy to go out my way if these were very new features, as I generally ensure a buffer to support other general browser-lag scenarios such as FireFox ESR usage, but the features I've come across are things that landed in FireFox 2+ years ago. I'd also be happy to go out my way if this was breaking due to anything vendor specific, but these elements currently causing issues are general web standards.
If there are very minor tweaks that can be made to achieve compatibility, I'd be happy to receive and review a PR, otherwise Pale Moon is not something I'd generally be testing on or assure support for.

Thing is, I know it worked in Pale Moon previously. I'm not sure if it was a javascript update, deprecation, or the like, or perhaps the PM direction.
Originally, I was running Bookstack on an LXC and ran across this as I moved servers (moved over files and ENV from the LXC to the Docker instances and then updated the databases). I had not stayed on top of my updates when it was running from LXC, so I know this issue does not exist with v21 of Bookstack via the LXC.

@armouredking Sure, I don't doubt that it did work in Pale Moon, it would be very likely that we'd have started to use those features (Or upped or JavaScript compatibility output levels) within the last 2-12 months. It's simply that we've introduced some more modern features into our JavaScript output that Pale Moon does not yet support.
The features lacking support I've noticed are tracked on Pale Moon's bug tracker though:

https://repo.palemoon.org/MoonchildProductions/UXP/issues/1658
https://repo.palemoon.org/MoonchildProductions/UXP/issues/1691
But they look like tricky features to support.

Will just close and leave this as reference to others who run across it.


How to set mime-type?

The API clearly shows the ability to set MIME TYPE and other meta-data when uploading a file, which is exposed in the other libraries, but the C# version doesn't seem to support this..? Why you no love C#?

You're welcome to do a PR, I currently don't have plans to add support for it

Pull request #18 now created. I've also fixed a bug that was introduced recently by another commit. TimeSpan should not be compared to null as it's a value type (it can't be null). Changed it to check against default(TimeSpan).

Deployed in v1.0.2


Integration tests fix

Coverage remained the same at 39.36% when pulling 6947d27e182ab7cd5f325d19405fa8140d4cc9dd on aluzzardi:integration-fix into b250a3a5cf641cd9c9997ad76c16d6f4f7259017 on docker:master.

LGTM


How to "unidentify" a user at logout?

All working fine so far, except for one issue:

how do I remove the identification of a user, upon logout for example ?

I tried different things:

window.analytics.identify(null)
window.analytics.identify({       authenticationStatus: 'logout'     });

but their are not triggering a return to anonymous (https://segment.com/docs/connections/spec/identify/#anonymous-id) but to no avail.
The hard way is to clean manually the localStorage of all the ajs_* keys.
Any ideas?

Got it :-)
There is a window.analytics.reset() that does the job. Will check for side effect


Fixed issue with whitespace on edge browser

Added a max-height to dropdowns to prevent large lists from expanding below edge footer causing excess whitespace. Will be overridden with an inline style when activated.

Since Edge has switched to the Chromium based engine this no longer is an issue.


347-Top K Frequent Elements [CSharpSolution]

File(s) Modified:  New file [csharp/347-Top-K-Frequent-Elements.cs]
Language(s) Used: C#
Submission URL: https://leetcode.com/submissions/detail/754871196/

Thanks @KhaledSamir!


Fixed build for UIExplorer project.

Added missing file RCTRootViewInternal.h

@facebook-github-bot shipit


double find will return wrong data

such as this:
models.users.find().only("email", "name", "admin", "created_at").run(function(err, users){
if(err){
throw err;
}
console.log(JSON.stringify(users));
res.json(users);
});
when i run this function twice, except first run, Behind results will only have the last data.

such that
models.users.find().only("email", "name", "admin", "created_at").run(function(err, users){
			if(err){
				throw err;
			}
			console.log(JSON.stringify(users));
});

models.users.find().only("email", "name", "admin", "created_at").run(function(err, users){
			if(err){
				throw err;
			}
			console.log(JSON.stringify(users));
});

models.users.find().only("email", "name", "admin", "created_at").run(function(err, users){
			if(err){
				throw err;
			}
			console.log(JSON.stringify(users));
});

it will return
[{"email":"wp3xpp@163.com","passwd":null,"admin":true,"name":"ddd","image":null,"created_at":1438508370808,"id":null},{"email":"wp3xpdp@163.com","passwd":null,"admin":true,"name":"ddd","image":null,"c
reated_at":1438508516420,"id":null},{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null,"created_at":1438572358034,"id":null}]
[{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null,"created_at":1438572358034,"id":null},{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null
,"created_at":1438572358034,"id":null},{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null,"created_at":1438572358034,"id":null}]
[{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null,"created_at":1438572358034,"id":null},{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null
,"created_at":1438572358034,"id":null},{"email":"531409927@qq.com","passwd":null,"admin":true,"name":"ddd","image":null,"created_at":1438572358034,"id":null}]

models.users.find(function(err, users){
			if(err){
				throw err;
			}
			console.log(JSON.stringify(users));
});

if i use this code
it will return true result
chain has a bug?

一看这英语不用猜就知道肯定是中国人。。

Can you set 'debug: true' and paste the SQL here?

yes, I set it like:
var opts = {
    database: "inspiration",
    protocol: "mysql",
    host: "<IP_ADDRESS>",
    port: 3306,
    username: "root",
    password: "root",
    query: {
        debug : false,
        pool: true
    }
};

it can print many detail info,contain query sql, but It's hard to find。
i can only user this command to mysql now:
SHOW VARIABLES LIKE "general_log%";
SET GLOBAL general_log = 'ON';

Can you change it to: query: { debug2: true } and then change this line to
if (this.opts.debug2) {
I suspect the newer versions of mysql driver also use the debug flag and it causes a lot of noise.

:ok:

Hey, I fixed the noisy mysql debug output and released version v2.1.27
Please update rather than using the hack above.


Redesign usage graphs to display total users and total plans

Related issue #1173
Redesign the graphs to integrate the filter selections with displayed data as in the wireframe below (screenshot 1). The graphs need to accommodate complete historical data for users and plans (screenshot 2 of current graphs below). If possible these should change dynamically based on filters. As discussed previously, it's not necessary to enable custom date selections; fine to constrain these by monthly usage numbers similar to the dropdown example in the wireframe below. It's just important to enable users to visualize broad changes/trends over time.
DMPTool power user Orgs have data from 2011-2018/present, up to n = 300 users and n = 350-400 plans. DMPonline power user data for Edinburgh up to n = 700 users and n = 900 plans.

Change graph title based on filter selection "Users" and "Plans" - or choose a generic title depending on final design
If possible, add option to visualize plans/template data (related issue #1679 )
Extend the y-axis to at least 1500? w/larger increments (try every 50 or 100?). Or better if this could be dynamic.
For the x-axis make the bars narrower and display data by month/year depending on selections. Alternatively use a line graph - just need to show trends.
I prefer to keep things simple with a single graph display. Admins can produce more sophisticated visualizations from raw CSV data if desired. But see @sjDCC comments below about possible 2-graph display with annual data (or whatever filter selections are made) alongside total data.

Integrated filters and single dynamic graph

Current usage dashboard - it's confusing for users to make filter selections that aren't reflected in the graphs. Find a way to integrate these two elements.

Thanks @stephaniesimms
Looks like Edinburgh has just short of 700 users and 900 plans so probably cap each at 1000 for now? Might be worth running queries to ensure nobody exceeds this though.
I personally like the year view as you can identify peak months / periods so would retain this, as well as adding a multi-year 2010-2018 view. It would be great to allow people to switch between years to look at the annual data for each.
Perhaps the devs could provide some ideas about alternative views? Even just having the total user & plan numbers as a table or line graph with annual figures for 2010-2018 alongside the single year monthly breakdown could be good. Whatever visualization works best.

The filters and the graph should also be linked so the parameters set are then displayed visually

Marta will be taking on this.

Closing out this issue and making a new one with agreed upon changes.


Add Fritz-Löffler-Straße Dormitory

Since GPS is going online tomorrow as well, I would suggest changing the drop-down menu to just:

Gerokstraße
Hochschulstraße
All other student houses

Otherwise it is getting crowded, and we will not have to update for each new house anymore.

Huhu, I thought I commented on this one already…
The addition of the FL is trivial, you just need to add another dormitory.
Switching the drop-down menu as described would require a small change in logic, as currently it only lists all the dormitories, and not the datasources as suggested.


$options of loadModalAsync function

loadModalAsync($modal, $method, $options, $text)
when the $options  is a string attribute in model, the modal cannot work.
file: orchid/platform/resources/views/partials/td/async.blade.php
line: 6
original:   data-modal-params="@json($attributes)"
after change:  data-modal-params=@json($attributes)
through the above steps, my problem was solved.

Hi. You have closed the question.
Has it become irrelevant? Did you solve your problem differently?

I found two minor problems when i use the modal layout,
suggestion 1:     data-modal-params="@json($attributes)"    should be change to    data-modal-params=@json($attributes)   at file: resources/views/partials/td/async.blade.php
suggestion 2:      when i use the modal, Through the following code:
public function layout(): array    {        return [            OptionEditLayout::class,            OptionTableLayout::class,
Layout::modal('oneAsyncModal', [
OptionModalEditLayout::class,
])->async('asyncGetOption'),        ];    }
When I click the modal wake-up button, the layout slug always matched "OptionEditLayout::class", so, this code doesn't work properly.    Then I changed the code on [src/Screen/Layouts/Base.php], at line 169-172, to:
public function getSlug(): string    {        return sha1(serialize(new \ReflectionObject($this)));    }
It works, and it can matched the OptionModalEditLayout::class.  ^ - ^
----- 原始邮件 -----
发件人：Alexandr Chernyaev<EMAIL_ADDRESS>收件人：orchidsoftware/platform<EMAIL_ADDRESS>抄送人：imnicy<EMAIL_ADDRESS> State change<EMAIL_ADDRESS>主题：Re: [orchidsoftware/platform] $options of loadModalAsync function (#958)
日期：2019年09月19日 15点15分
Hi. You have closed the question.
Has it become irrelevant? Did you solve your problem differently?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

I found two minor problems when i use the modal layout,
suggestion 1:
data-modal-params="@json($attributes)"
should be change to
data-modal-params=@json($attributes)
at file: resources/views/partials/td/async.blade.php
suggestion 2:
when i use the modal, Through the following code:
public function layout(): array
{
   return [
       OptionEditLayout::class,
       OptionTableLayout::class,
       Layout::modal('oneAsyncModal', [
               OptionModalEditLayout::class,
       ])->async('asyncGetOption'),
    ];
}

when I click the modal wake-up button, the layout slug always matched "OptionEditLayout::class", so, this code doesn't work properly.    Then I changed the code on [src/Screen/Layouts/Base.php], at line 169-172, to:
public function getSlug(): string    {
    return sha1(serialize(new \ReflectionObject($this)));
}

It works, and it can matched the OptionModalEditLayout::class.  ^ - ^

I propose then to change:
data-modal-params="@json($attributes)"

to
data-modal-params='@json($attributes)'

This should fix and prevent unpleasant situations.
Regarding the second error, can you give me more information? Are you using the latest version?

The second problem has been fixed and I tested it with version 6.1. Perfect.


UnpBootHeader nav items do not work properly on mobile safari

Have a 'page' property in the navItems point to "http://www.google.com"
This works in a browser but it appends this string to the current URL in mobile Safari.
This is a problem if you need to get to other databases also from the menu system or back to a dashboard card (in another DB).

This seems to be working properly for me with latest build


Processing time is increased with hudi metadata enable

We run spark streaming application, where we upsert data in hudi,
We have around 2000 partitions in our hudi dataset, we enabled metadata in our hudi tables, after that we could see drastic change in processing time. current processing time is 15 minutes. earlier processing time is 1 min or 1 min 30 sec.
We use inline compaction
with metadata enable which compaction would you prefer? Async or inline?
Wts the difference between Async and inline compaction.
Please suggest
Hudi props :
DataSourceWriteOptions.TABLE_TYPE.key() -> DataSourceWriteOptions.MOR_TABLE_TYPE_OPT_VAL,
DataSourceWriteOptions.RECORDKEY_FIELD.key() -> "id",
DataSourceWriteOptions.PARTITIONPATH_FIELD.key() -> "col",
DataSourceWriteOptions.PRECOMBINE_FIELD.key() -> "optime",
DataSourceWriteOptions.HIVE_SYNC_ENABLED.key() -> "true",
DataSourceWriteOptions.HIVE_PARTITION_FIELDS.key() -> "col",
DataSourceWriteOptions.HIVE_PARTITION_EXTRACTOR_CLASS.key() -> classOf[MultiPartKeysValueExtractor].getName,
DataSourceWriteOptions.HIVE_SYNC_MODE.key() -> "hms",
DataSourceWriteOptions.HIVE_USE_JDBC.key() -> "false",
HoodieCompactionConfig.INLINE_COMPACT_TRIGGER_STRATEGY.key() -> CompactionTriggerStrategy.TIME_ELAPSED.name,
HoodieCompactionConfig.INLINE_COMPACT_TIME_DELTA_SECONDS.key() -> String.valueOf(60 * 60),
HoodieCompactionConfig.CLEANER_COMMITS_RETAINED.key() -> "800",
HoodieCompactionConfig.ASYNC_CLEAN.key() -> "false",
HoodieCompactionConfig.INLINE_COMPACT.key() -> "true",
HoodieCompactionConfig.MIN_COMMITS_TO_KEEP.key() -> "801",
HoodieCompactionConfig.MAX_COMMITS_TO_KEEP.key() -> "825",
HoodieMetricsConfig.TURN_METRICS_ON.key() -> "true",
HoodieMetricsConfig.METRICS_REPORTER_TYPE_VALUE.key() -> MetricsReporterType.DATADOG.name(),
HoodieMetricsDatadogConfig.API_SITE_VALUE.key() -> "US",
HoodieMetricsDatadogConfig.METRIC_PREFIX_VALUE.key() -> "tacticalnovusingest.qa.hudi",
HoodieMetricsDatadogConfig.API_KEY_SUPPLIER.key() -> "com.tr.indigo.tacticalnovusingest.utils.DatadogKeySupplier",
HoodieMetadataConfig.ENABLE.key() -> "false",
HoodieMetadataConfig.CLEANER_COMMITS_RETAINED.key() -> "800",
HoodieMetadataConfig.MIN_COMMITS_TO_KEEP.key() -> "801",
HoodieMetadataConfig.MAX_COMMITS_TO_KEEP.key() -> "825",
HoodieWriteConfig.ROLLBACK_USING_MARKERS_ENABLE.key() -> "false"

@koochiswathiTR is the processing time of 15min for every commit after enabling metadata table? please also provide hudi/spark versions, workload info like how many records each commit and how many are updates. And environment info as well.

Spark : 3.1.2
Hudi : 0.11.1
AWS EMR : 6.7
Its a spark streaming application, We use only upsert command
We have 2000 partitions,
We will upsert 3.2 Millions of records at a time.
a micro batch in spark streaming is taking 15 mints of time to complete whole process( upsert + compaction/cleanup) etc
Let me know if you need any information
@xushiyan

@xushiyan  Any update

We scheduled compaction for 1 every one hour, if we reduce it to 15 mints or 30mints will the processing time reduces?
@xushiyan
@nsivabalan

please find the time it is taking to complete hudi jobs.

@koochiswathiTR To help debug the performance regression, could you also share the driver log, content under .hoodie, and the complete spark stages (the screenshot above does not show stages for writing metadata table and other important stages)?  It seems weird that the Perform rollback actions job takes 56 minutes.

stderr.2022-10-14-05.txt

stderr.2022-10-14-06.txt

#Updated at 2022-08-06T11:30:43.839Z
#Sat Aug 06 11:30:43 GMT 2022
hoodie.table.timeline.timezone=LOCAL
hoodie.table.keygenerator.class=org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.table.precombine.field=operationTime
hoodie.table.version=4
hoodie.database.name=
hoodie.datasource.write.hive_style_partitioning=false
hoodie.table.checksum=571342712
hoodie.partition.metafile.use.base.format=false
hoodie.archivelog.folder=archived
hoodie.table.name=novusdoc_xml
hoodie.compaction.payload.class=org.apache.hudi.common.model.OverwriteWithLatestAvroPayload
hoodie.populate.meta.fields=true
hoodie.table.type=MERGE_ON_READ
hoodie.datasource.write.partitionpath.urlencode=false
hoodie.table.base.file.format=PARQUET
hoodie.datasource.write.drop.partition.columns=false
hoodie.table.metadata.partitions=files
hoodie.timeline.layout.version=1
hoodie.table.recordkey.fields=guid
hoodie.table.partition.fields=collectionName

@yihua @nsivabalan   @xushiyan

Compaction runs for every hour and with delta commits 5
If we run compaction for every 15 mints and delta commits to 2 will it increase performance?

if we change to SIMPLE index will it increase performance?
https://lightrun.com/answers/apache-hudi-processing-time-very-slow-updating-records-into-hudi-datasetmor-using-aws-glue
@yihua @nsivabalan @xushiyan

In Hudi 0.11.1, SIMPLE index is the default.
To clarify, are you saying that the 15 min vs 1.5 min difference is due to turning on and off the metadata table?  If that's the case, changing compaction frequency should not matter here.

Pls check the image, i think BLOOM index is by default.

Also after enabling metadata , we see drastic change in performance,
We would like to disable again
Will we see any issue if we disable metadata?
Note : The partitions are 2000 and data is huge
@yihua @nsivabalan @xushiyan

@yihua @nsivabalan @xushiyan Any udpate?
Our prod spark streaming job is taking long time.

@yihua @nsivabalan @xushiyan

if you disable metadata, there won't be any issues. atleast you can unblock yourself for now.

but if feasible, lets save contents of .hoodie just before you plan to disable metadata (bcz, after disabling, hudi will delete the metadata dir). so that we can continue our investigation.

can you post the entire spark stages UI. the rollback was a misnomer likely. but if you are seeing higher latency for every ingestion, something else is going on.

@koochiswathiTR : can you share the contents of ".hoodie" and the metadata dir as well. would like to get to the bottom of it.

also, do you know roughly how many file groups are there per partition?
for eg, if you check commit metadata (.hoodie/2022XXXXXX.commit), it will show how many files are touched as part of the commit. Would like to understand more about the data layout so that we can try to reproduce on our side.

unfortunately the log files attached did not have much info to assist in debugging.
I am talking about stderr.2022-10-14-05.txt and stderr.2022-10-14-06.txt.

@koochiswathiTR Can you test out with latest version of hudi. We have done lots of improvement related to Metadata server. Let us know if you still see the error.


Create github workflow for running test

Read our Contributor Guidelines and README.
Create new workflow for running test for shared and data_handler
Command how to run  test for shated you can find here shared
Command how to run test for data_handler you can find data_handler

can i work on this I love 🍍'

I would like to take this task, I love 🍍

I would like to take this task, I love 🍍ETA 2-3days 30th - 2nd nov I have experience in python  and writing test for smart contracts

@petersdt estimation is 24 hours. sorry

I can start working on this now!

Is it okay if I take this?

I would like to take this task, I love 🍍

I would like to take this task, I love 🍍
I will make i pr within 24hrs this is my first time working on this project

I love 🍍 and im a newbie contributor, i would love to work on this.
Estimated timeframe : 10hrs

I would like to take this task, I love 🍍. I will analyze the existing test implementation from the spotnet project, create a new GitHub workflow file that executes tests for both the shared and data_handler components, ensuring proper test coverage for both modules by following the commands specified in their respective directories. I plan to finish in 1 day, drawing from my experience with GitHub Actions and Python testing frameworks.

@mimisavage left 6 hours and you didn't even start[this task you can test only via creating PR and run flow this way]
pay attention, after 24 hours you will be unassigned


UI5 does not bind data inside attachRequestCompleted()

OpenUI5 version: 2.3.0
Browser/version (+device/version): Chrome
Any other tested browsers/devices(OK/FAIL): FAIL
URL (minimal example if possible):
User/password (if required and possible - do not post any confidential information here):
Steps to reproduce the problem:

I am fetching data from remote JSON file
var getLatestJourneyURL = "https://database01p15848275trial.hanatrial.ondemand.com/myhanaxs/getLatestJourney.xsjs"; var oModel = new sap.ui.model.json.JSONModel(); oModel.loadData(getLatestJourneyURL);

I want to set certain values for data binding once UI5 successfully fetches data:

oModel.attachRequestCompleted(function(){
				var viewTexts = new sap.ui.model.json.JSONModel();
				var data = {
					"footerText": "Start",
					"textValues": {
						"footerTxt": "start"
						}
					};
				viewTexts.setData(data);
				sap.ui.getCore().setModel(viewTexts);
				console.log(viewTexts.getData());
				console.log(oModel.getData());

				console.log("I was hit");
				dialog.close();
			});

And I am using XML view to display:

<GenericTile class="sapUiTinyMarginBegin sapUiTinyMarginTop tileLayout" header="{/textValues/footerTxt}" 	subheader="{textValues/footerTxt}" press="press1"> 		<tileContent> 			<TileContent footer="{/textValues/footerTxt}"> 				<content> 					<ImageContent src="sap-icon://record"/> 				</content> 			</TileContent> 		</tileContent> 	</GenericTile>
What is the expected result?
It should show text "start"
What happens instead?
Showing nothing :-(
Any other information? (attach screenshot if possible)
Someone copies the same code and running successfully at JSBIN
The above code might be different from Jsbin code, this is because I kept on changing it. However when I had the JSBin code, it was neither working :-(

Hello Shrikant,
with the latest version of the Snippix the issue described above does not occur anymore and the value is set in the model just fine.  This seems to be an application issue, data binding should work fine. Closing the issue unless you suspect a bug in the UI5 framework.
![tilecontent](https://cloud.githubusercontent.com/assets/6791333/18510469/88be0d02-7a7e-11e6-9382-61bd0d21d003.png)
Kind Regards
Michael


Bug in CIRR experiment !!!

Thank you for your open source of interesting work!
While I evaluated in CIRR experiment, there is a bug occurred:
 Traceback (most recent call last):
  File "src/eval_retrieval.py", line 370, in <module>
    main()
  File "src/eval_retrieval.py", line 362, in main
    main_worker(args.gpu, None, log_queue, args)
  File "src/eval_retrieval.py", line 212, in main_worker
    target_dataloader)
  File "/composed_image_retrieval/src/eval_utils.py", line 363, in evaluate_cirr
    metrics = metric_func(ref_features=value)
  File "/composed_image_retrieval/src/eval_utils.py", line 569, in get_metrics_cirr
    sorted_index_names.shape[1] - 1)

Could you tell me how to fix it? Thank you! @ksaito-ut @kihyuks

The issue is closed but it is not solved. I put my solution in case someone else encounters this issue. @Pter61 you did not provide a solution but I guess you did a change like that:

The issue is closed but it is not solved. I put my solution in case someone else encounters this issue. @Pter61 you did not provide a solution but I guess you did a change like that (as the filter by reference_mask could decrease the first dimension):

Sorry because it's been a long time,  I don't remember my solution clearly, but I remember my solution is similar to @jeff3071.

I encountered the same issue where I discovered that the problem lies with the img_path in CIRR. Unlike the ref_imgs and target_imgs paths, the img_path utilize os.join(). This renders the line reference_mask = torch.tensor(sorted_index_names != np.repeat(np.array(reference_names), len(index_names)).reshape(len(target_names), -1)) ineffective. Therefore, I modify the return value to tar_path.

I address this issue with this answer. More specifically, modify the return target_images, img_path to return target_images, tar_path in data.py, line 140, in return_valdata. Hope to help.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/92775

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/93212

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/98371

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/98570

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/98882

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/102967


ComplexSearch Ignores Document Type

When searching with a raw query, Model::complexSearch will return documents across all indexes and types.
This is because the static complexSearch method does not call getBasicEsParams.
There are several ways to resolve this depending on the intended design of complexSearch

Add a call to getBasicEsParams in complexSearch.
Setup just index and type params without getIdIfPossible, getSourceIfPossible, and getTimestampIfPossible.
Add a note in documentation about this peculiarity of complexSearch. README.md currently states that complexSearchis equivalent to searchByQuery in some scenarios. Combined with the model name prefix, complexSearch's behavior can be misleading.

I can do a pull request once the correct behavior is determined.

I have the same problem. a solution of this been created?


fix left part of conditional rendering not boolean on SignInPage, TaskHeader components

Details
fix console errors on SignInPage, TaskHeader components
Fixed Issues
$ https://github.com/Expensify/App/issues/19202
PROPOSAL: https://github.com/Expensify/App/issues/19202#issuecomment-1557544082
Tests
SignInPage:

Logout if already logged in
Enter new email and click Continue button
Verify that no console errors appear on magic code input page

TaskHeader:

Login any account
Create new task (Open FAB > Assign task > Enter title > Select assignee > Confirm task)
Verify that no console errors appear on task report page

[x] Verify that no errors appear in the JS console

Offline tests
N/A
QA Steps
Same as Tests

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web
https://github.com/Expensify/App/assets/108292595/ca4598ca-847b-4beb-ab7d-3ae0e7cf0655
https://github.com/Expensify/App/assets/108292595/8ac49803-1f37-4d4f-88ca-7f7b8517d4b9

Mobile Web - Chrome
https://github.com/Expensify/App/assets/108292595/a8293c7f-9635-4395-80be-10d44788accc

Mobile Web - Safari
https://github.com/Expensify/App/assets/108292595/37bf1808-c935-4f26-a9c7-151f4fbd0ee1

Desktop
https://github.com/Expensify/App/assets/108292595/8cb9deba-3551-463f-b70f-bde190821b2e

iOS
https://github.com/Expensify/App/assets/108292595/51494fa8-9e42-4cad-a8d4-0fa3b5166180

Android
https://github.com/Expensify/App/assets/108292595/ea7e5305-f1ee-4a1a-9ba0-d5ab68e4e106

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/puneetlath in version: 1.3.20-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/Julesssss in version: 1.3.20-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


New Case action - Manage pseudonymize

There is a new case action called Manage pseudonymize and there is no information about this action.
we would like to have information about it on this page, because it is already explaining all actions here.
Thank you

@robmazz Please merge this PR. Thanks

@ItaloDuarte - thanks for the feedback. I'm unaware of this case action, but will confirm with the PG team. If a section is needed, we will add it. Thanks again.


fix multiarch buildah command

follow up fix for #2757 😳
It seems the newly published images are single-arch. Went through a few manuals and it turned out that we should use --manifest instead of --tag.
Also buildah doesn't supports more than one --manifest arguments in one build, so we have to build each (v5, v5.15, v5.15.0, etc) separately even if they are identical images/manifests.
Ideally we can move downloadAssets.sh's logic into Containerfile itself so we can build for multiple archs concurrently. However that would be a big change and I'd rather leave it as is for now.

@xiaokangwang please help merge this. I don't have write access to this repo and cannot merge it myself. Thanks!
As for future improvements, I suggest we use workflow artifacts to store built binaries and only upload to github release or push to registry on release event. This way we can include container building as part of pull request validation and make it easier to test changes.


[HTML5] haxe 3.2-rc2 with lime 2.3.0 and openfl 2.3.0 build fails

created a brand new project:
$ openfl create project sample
Build output:
$ lime test html5
/usr/lib/haxe/lib/openfl/2,2,8/openfl/text/TextField.hx:1185: characters 30-50 : js.html.Window has no field event
/usr/lib/haxe/lib/openfl/2,2,8/openfl/text/TextField.hx:1167: characters 30-50 : js.html.Window has no field event
/usr/lib/haxe/lib/openfl/2,2,8/openfl/text/TextField.hx:1083: characters 3-47 : js.html.DOMElement should be js.html.Element
/usr/lib/haxe/lib/openfl/2,2,8/openfl/display/Application.hx:512: lines 512-516 : Field onMouseMove overloads parent class with different or incomplete type
/usr/lib/haxe/lib/openfl/2,2,8/openfl/display/Application.hx:512: lines 512-516 : Different number of function arguments
tried legacy too:
$ lime test html5 -Dlegacy
Export/html5/haxe/ApplicationMain.hx:4: characters 11-20 : Class not found : lime.Lime
NOTE:
flash, neko, mac work

Sorry that's openfl 2.2.8

Fixed in newer versions :smile:


[Release v1.22] rke2 servers not able to join the ha cluster

Backporting: https://github.com/k3s-io/k3s/pull/4781

Validated per https://github.com/k3s-io/k3s/issues/4773#issuecomment-998088619


Remove datasets from CI

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@fa25a3e). Click here to learn what that means.
The diff coverage is n/a.

@@          Coverage Diff           @@
##             main      #8   +/-   ##
======================================
  Coverage        ?   39.3%
======================================
  Files           ?      38
  Lines           ?     775
  Branches        ?       0
======================================
  Hits            ?     304
  Misses          ?     471
  Partials        ?       0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa25a3e...f79c238. Read the comment docs.


Duplicate tags for release commits?

I noticed that since v2.3.7 there are now commits tagged with -master:

I see that they are different than the release branches themselves. Do they just capture the state of master at the time of the release?

Yes, that's exactly right.  They are used to generate saner version numbers for master branch builds.  This was part of the fix for #1420, although most of the changes took place in our release and tagging process rather than in the code itself.
  There was no way to see that the most recent release had been, say, v2.3.8, and that the master branch build was more closely related to that or v2.4.0 than v2.3.0.  Now that we tag the master branch at each release, that tag is what shows up in master branch builds, which is arguably more informative.

Thanks for the explanation Joey, wasn't sure if it was intentional or not.


CAMEL-21403: retrieve full message for content containing non ASCII chars

Description
For a message containing non ASCII chars the length of the message byte array may be longer than the EDI message string length since non ASCII chars can be more that one byte.

Target

[ ] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[ ] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[ ] I checked that each commit in the pull request has a meaningful subject line and body.

[ ] I have run mvn clean install -DskipTests locally from root folder and I have committed all auto-generated changes.

/component-test as2

Just noticed that this doubles up a PR provided by poncyliuszm [1]. Happy to close this PR or merge into his for the flaky test fixup.
https://github.com/apache/camel/pull/16148

Oh @jonomorris there was another PR that also fixed this. Can you maybe update this PR to include your set of unit tests as they better cover this bug.

/component-test as2

I've rebased the changes on main.  Thanks.


Discussion: IntersectionObserver behavior in Chrome

See the Chromium issue.
Recently we found that IntersectionObserver does not work inside a background tab of Chrome, so immediately I thought I should share this info with caniuse. However I am wondering: should we mark Chrome as partially supported? If we keep the green light for Chrome, how to add notes to it?
My concerns:

IntersectionObserver not working in background tabs in definitely a gotcha for developers, so I think we should share the information in caniuse.
In Firefox, IntersectionObserver works as expected in background tabs. ("... The Intersection Observer API doesn't take this (page visibility) into account when detecting intersection, since intersection isn't affected by page visibility.")
AFAIK, the correct behavior of IntersectionObserver in background tab is unspecified in w3c.
Chrome dev team marked the issue as Won't Fix, meaning the current state is their expected behavior, making me wonder if whether we should take Chrome as partially supported.

I think it fair to keep the support status of Chrome, so I've sent a PR which just adds some notes to it.


CI should collect the PDK and logs artifacts

GitHub Actions can save artifacts from a build. It would be good if the run saved;

The PDK that was generated.
Any extra logging.
(maybe) The build Magic and any related core dumps?

FYI - https://github.com/actions/upload-artifact

Fixed in #192.


missing files

Could you please add requirements.txt and specify which version of tf did you use?

Added, thanks!


Regression in FormatParameterMismatch?

After looking at @pocke's findings for #3868 I ran the latest rubocop build on rubyspec. This file seems to have found an error with FormatParameterMismatch: https://github.com/ruby/spec/blob/cee95712953b1b913b7a4933ac07816a39a7acd8/core/string/modulo_spec.rb
An error occurred while Lint/FormatParameterMismatch cop was inspecting /Users/johnbackus/Projects/rubyspec/core/string/modulo_spec.rb.
undefined method `type' for nil:NilClass
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:139:in `arguments_count'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:107:in `count_matches'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:46:in `offending_node?'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:36:in `on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:41:in `block (2 levels) in on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:100:in `with_cop_error_handling'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:40:in `block in on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:39:in `each'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:39:in `on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:154:in `on_block'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_block'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:101:in `block in on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:99:in `each'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:99:in `each_with_index'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:99:in `on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_send'
(eval):2:in `block in on_begin'
(eval):2:in `each'
(eval):2:in `on_begin'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_begin'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:154:in `on_block'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_block'
(eval):2:in `block in on_begin'
(eval):2:in `each'
(eval):2:in `on_begin'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_begin'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:154:in `on_block'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_block'
(eval):2:in `block in on_begin'
(eval):2:in `each'
(eval):2:in `on_begin'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:45:in `on_begin'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:12:in `walk'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:58:in `investigate'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/team.rb:121:in `investigate'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/team.rb:109:in `offenses'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/team.rb:51:in `inspect_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:248:in `inspect_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:195:in `block in do_inspection_loop'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:227:in `block in iterate_until_no_changes'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:220:in `loop'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:220:in `iterate_until_no_changes'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:191:in `do_inspection_loop'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:101:in `block in file_offenses'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:111:in `file_offense_cache'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:99:in `file_offenses'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:90:in `process_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:68:in `block in each_inspected_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:65:in `each'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:65:in `reduce'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:65:in `each_inspected_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:57:in `inspect_files'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:36:in `run'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cli.rb:72:in `execute_runner'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cli.rb:27:in `run'
bin/rubocop:13:in `block in <main>'
/Users/johnbackus/.rubies/ruby-2.3.1/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
bin/rubocop:12:in `<main>'

I can try to whittle down the code causing the error later

Minimal reproduction
$ echo "'%' % []" | bin/rubocop --only FormatParameterMismatch --stdin - --debug
For /Users/johnbackus/Projects/rubocop: configuration from /Users/johnbackus/Projects/rubocop/.rubocop.yml
Inheriting configuration from /Users/johnbackus/Projects/rubocop/.rubocop_todo.yml
Default configuration from /Users/johnbackus/Projects/rubocop/config/default.yml
Inheriting configuration from /Users/johnbackus/Projects/rubocop/config/enabled.yml
Inheriting configuration from /Users/johnbackus/Projects/rubocop/config/disabled.yml
Inspecting 1 file
Scanning /Users/johnbackus/Projects/rubocop/-
An error occurred while Lint/FormatParameterMismatch cop was inspecting /Users/johnbackus/Projects/rubocop/-.
undefined method `type' for nil:NilClass
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:139:in `arguments_count'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:107:in `count_matches'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:46:in `offending_node?'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/lint/format_parameter_mismatch.rb:36:in `on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:41:in `block (2 levels) in on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:100:in `with_cop_error_handling'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:40:in `block in on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:39:in `each'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:39:in `on_send'
/Users/johnbackus/Projects/rubocop/lib/rubocop/ast/traversal.rb:12:in `walk'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/commissioner.rb:58:in `investigate'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/team.rb:121:in `investigate'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/team.rb:109:in `offenses'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cop/team.rb:51:in `inspect_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:248:in `inspect_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:195:in `block in do_inspection_loop'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:227:in `block in iterate_until_no_changes'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:220:in `loop'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:220:in `iterate_until_no_changes'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:191:in `do_inspection_loop'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:101:in `block in file_offenses'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:111:in `file_offense_cache'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:99:in `file_offenses'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:90:in `process_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:68:in `block in each_inspected_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:65:in `each'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:65:in `reduce'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:65:in `each_inspected_file'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:57:in `inspect_files'
/Users/johnbackus/Projects/rubocop/lib/rubocop/runner.rb:36:in `run'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cli.rb:72:in `execute_runner'
/Users/johnbackus/Projects/rubocop/lib/rubocop/cli.rb:27:in `run'
bin/rubocop:13:in `block in <main>'
/Users/johnbackus/.rubies/ruby-2.3.1/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
bin/rubocop:12:in `<main>'
.

1 file inspected, no offenses detected

1 error occurred:
An error occurred while Lint/FormatParameterMismatch cop was inspecting /Users/johnbackus/Projects/rubocop/-.
Errors are usually caused by RuboCop bugs.
Please, report your problems to RuboCop's issue tracker.
Mention the following information in the issue report:
0.46.0 (using Parser <IP_ADDRESS>, running on ruby 2.3.1 x86_64-darwin16)
Finished in 0.06896200001938269 seconds

^^ @zverok

This was amazingly not a regression from the huge refactoring PR (it was introduced earlier in #3858), which both of @pocke's findings were.
But I submitted a fix anyhow. 😀
On a side note, I should have thought of running the refactored code against rubyspec ... 🤔

Sorry :( And thanks @Drenmi for the fix.


Add Cumulative counters

In order to indicate the overall progression, (nb of tx, TPS, and maybe ETA ?) we should log cumulative stats

Added back overall progression, should log instead only ETA and nb TX


KAFKA-4465: Create docker image and scripts for running tests locally

@harshach @ijuma @ewencp Please review.

+1. @ewencp  this is still good to have to run system tests.

Thanks for posting this, raghavgautam!  I've found it really useful to run the ducktape tests.  I have some comments...

I am on vacation and my net access is scratchy at best. I will be able to address review comments after about 2 weeks.

@ijuma @cmccabe I have addressed all the review comments, please take a look.

@ewencp Looks like all the comments are addressed. I am still +1. Let me know if its good to merge.

LGTM


locksmith fails when -etcd-cafile is specified

Description
When -etcd-cafile is specified without a client cert/key, locksmith fails with the error:
$ locksmithctl -etcd-cafile=/etc/ssl/certs/ca-certificates.crt status
Error initializing etcd client: open : no such file or directory

I've configured etcd with TLS using self-signed certificates but not TLS client authentication. locksmith seems to be looking for a certificate and key, even though these options are not applicable.
Impact
Error message is confusing, because it relates to an unrelated command line option.
Environment and steps to reproduce

Set-up: Flatcar Linux 3374.2.2
Task: Configuring locksmith with TLS communication and server-only authentication
Action(s):
a. locksmithctl -etcd-cafile=/etc/ssl/certs/ca-certificates.crt status
Error: Error initializing etcd client: open : no such file or directory

Expected behavior
locksmith uses the specified CA to authenticate the server without client authentication.
Additional information
N/A.

Seeing this issue with k3s and the embedded etcd as well. k3s with an embedded etcd does work with etcdctl but not locksmithctl.
I noticed a pull request to upgrade locksmith to etcd3 link, maybe that is the issue?
Impact
Not able to use etcd based locksmith reboots with k3s
Environment and steps to reproduce

Set-up: Flatcar Linux 3815.2.1
Task: Install k3s with etcd and configuring locksmith with TLS communication
Action(s):

Install k3s curl -sfL https://get.k3s.io | sh -s - --secrets-encryption --token SuperSecrect --cluster-init
Test with etcdctl: etcdctl --cacert=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --cert=/var/lib/rancher/k3s/server/tls/etcd/server-client.crt --key=/var/lib/rancher/k3s/server/tls/etcd/server-client.key endpoint status
Test with locksmith: locksmithctl --etcd-cafile="/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt" --etcd-certfile="/var/lib/rancher/k3s/server/tls/etcd/server-client.crt" --etcd-keyfile="/var/lib/rancher/k3s/server/tls/etcd/server-client.key" status

Error: Error initializing etcd client: creating etcd lock client: EOF

Expected behavior
locksmith works with tls etcd.

Thanks @tormath1
I was unable to run etcd with enable-v2. Since when I set the enable-v2: true in the /var/lib/rancher/k3s/server/db/etcd/config file (Where I got the TLS config) it was removed when I rebooted. The node is already part of a cluster. I guess that doesn't allow it to change. I didn't find any help in the k3s docs or Google for enabling v2 in the embedded etcd.
So I did the reverse, configured etcdctl to use v2. Which results in the same error as locksmith:
ETCDCTL_API=2 etcdctl --ca-file="/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt" --cert-file="/var/lib/rancher/k3s/server/tls/etcd/server-client.crt" --key-file="/var/lib/rancher/k3s/server/tls/etcd/server-client.key" --endpoints https://<IP_ADDRESS>:2379 cluster-health
cluster may be unhealthy: failed to list members
Error:  client: etcd cluster is unavailable or misconfigured; error #0: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"

error #0: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"

That is the same error that locksmith gave.
          net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"

I grabbed the etcdctl version, and it is 3.5.
# etcdctl version
etcdctl version: 3.5.0
API version: 3.5

Also in the startup log of k3s I see this line
"embed/etcd.go:309","msg":"starting an etcd server","etcd-version":"3.5.9","git-sha":"Not provided

And the latest k3s release shows it as running:

Etcd 	v3.5.9-k3s1

I see the same issue, the etcd server is detecting the new configuration and I see it in the config file. The etcd clients still cannot connect. The same errors are before.
Any luck with the PR?

Since you are using k3s, which is Kubernetes, I think you could rather use FLUO https://github.com/flatcar/flatcar-linux-update-operator/ or kured https://github.com/kubereboot/kured/ instead of locksmith, or?

The simplicity of locksmith is what I like, simple process to reboot nodes without any needing additional Kubernetes configuration. Ideally, Kubernetes should be able to tolerate a node rebooting without any issues.
I didn't like kured because after creating a cluster it still needs to the update service to be deployed and configured in Kubernetes.  I did not know about the Flatcar Linux Update Operator, but it also requires Kubernetes setup. Perhaps I am mistaken and this is the best path forward.
I am trying to create a setup where I can fully automate the deployment of a multi-node k3s cluster with automatic updates.
@tormath1 to test the PR, do I build locksmith with your PR and overwrite the binaries in the flatcar OS?

Thanks for the advice! Totally understand that the PR is not production ready.
Really appreciate the help with this issue.


Checkout API v67

Please can you update the code for the new Checkout API v67.
There are some variation reguarding implementation of 3D secure.

Hi its updated to v67 now


#527 autoscaler test

This PR moves the E2E tests to the autoscaler. The testing step took 22m 43s (24m 51s total), compared to 10m 4s (12m 0s total) if not using the autoscaler. Initial startup time is really only a factor for the first NMT job and the first SMT job, since the autoscaler can reuse instances.
Also, the reason the environment variables I added have CLEARML in all caps was to distinguish that they're not an official environment variable from ClearML, which uses ClearML to prepend its own environment variables. But I can change the environment variable names to match or to something else entirely if that's preferable.

This change is

Why is there such a large difference in runtime?

Reviewable status: 0 of 2 files reviewed, all discussions resolved (waiting on @johnml1135)

Is it because it's not running on John's smaller gpus now? What kind of gpus does the autoscaler spin up, Matthew?

We talked about it in our standup, recording here that the reason is that there is a startup cost associated with launching a GCP instance for the first SMT job as well as for the first NMT job. And the GPU type that the autoscaler spins up for the NMT jobs is an A100 40GB.

* Cost 2: wen we run the E2E tests it will take 22 min rather than 10 min

* Benefit 1: We will always be testing the autoscalar to make sure everything is working properly

* Compromise?: Should we run one set of models (say the SMT models) on the autoscalar and the NMT on my GPU?  That would save some time and most of the cost per run as well as continue to verify that the autoscalar is working properly.  The only meaningful difference would be the A100's - and they are the same ones we are using in the AQUA server.  I think we can get near 100% of the value with < 50% of the cost.

(If I could add)

Potential Benefit 2: It may be 22 min rather than 10 min, but it ought to be consistent in a way the current set-up isn't. I.e., if there's a lot queued up (for some reason), we shouldn't have added wait times for jobs to finish. This might also help with some of the flakiness we've seen with, for example, the queue multiple E2E test.

Point taken.  There is only one GPU available for running these tests (my 3090).  If, on the other hand, we use the AQUA server for the CPU jobs and the autoscalar for the GPU jobs, we still can save some time and not run into the bottlenecks.  We would not save as much money, but honestly, I am more desirous of the times being short than the cost being less.
In October, we had 27 commits to master (a pretty high month).  Extrapolating that, we could spend 27120.20 (assuming half the time spent using GPUs) = $65 in total GPU cost for running these tests per year, hardly breaking the bank.

The differences are likely less than the cost of us talking about it.  Let's just do it.


Responsive Navigation Bar Styles

Patched the problem where navigation bar was being cut out.
Currently, Navigation Bar uses a semantic style, arranging itself.

Thanks, @MathewR7000 , for helping out with the patch!


Move config for registry_address from validator to node section

In 'validator', it won't even be parsed if the section is not enabled.
Since all nodes rely on a validator, having it in node makes much more sense.

Neat, we should probably make a new release soon


Feature - Auto bump NPM version of package

Resources

https://docs.npmjs.com/updating-your-published-package-version-number
~~https://github.com/marketplace/actions/create-pull-request~~ (Note: probably can't create a pull request if action is on tag create)
https://stackoverflow.com/questions/5195859/how-do-you-push-a-tag-to-a-remote-repository-using-git
https://aboutbits.it/blog/2021-03-11-using-github-actions-to-perfom-npm-version-increment

Optional

Add changelog update to git actions on release
https://github.com/marketplace/actions/release-changelog-builder

How it should work

Should bump npm version based on current tag
(Optional) Have on create tag create a release
(Optional) Auto gen changelog in release


mean returns empty DataArray for groupby_bins containing datetime64

What happened?
Applying the mean operator to an xarray dataset that has been grouped by bins, removes variables containing values of type datetime64.
What did you expect to happen?
I expect that all variables are preserved after applying the mean-operator. For min and max this works as expected. sum is not meaningful, so it makes sense that the variable is dropped in that case as well.
Minimal Complete Verifiable Example
import numpy as np
import pandas as pd
import xarray as xr

times = pd.date_range('2020-01-01', '2020-02-01', freq='1H')
index = np.arange(len(times))
bins = np.arange(0,len(index),5)
ds=xr.Dataset({'time':('index', times),'float':('index', np.linspace(0,1,len(index)))},coords={'index':index})

ds.groupby_bins('index', bins).mean()
<xarray.Dataset>
Dimensions:     (index_bins: 148)
Coordinates:
  * index_bins  (index_bins) object (0, 5] (5, 10] ... (730, 735] (735, 740]
Data variables:
    float       (index_bins) float64 0.004032 0.01075 0.01747 ... 0.9852 0.9919

ds.groupby_bins('index', bins).min()
<xarray.Dataset>
Dimensions:     (index_bins: 148)
Coordinates:
  * index_bins  (index_bins) object (0, 5] (5, 10] ... (730, 735] (735, 740]
Data variables:
    time        (index_bins) datetime64[ns] 2020-01-01T01:00:00 ... 2020-01-3...
    float       (index_bins) float64 0.001344 0.008065 0.01478 ... 0.9825 0.9892

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[X] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output
No response
Anything else we need to know?

This is still an issue, as datetimes are handled as "non_numeric" and are just skipped for mean/median etc.
A workaround would be to call reduce directly:
import numpy as np
import pandas as pd
import xarray as xr

times = pd.date_range('2020-01-01', '2020-02-01', freq='1H')
index = np.arange(len(times))
bins = np.arange(0,len(index),5)
ds=xr.Dataset({'time':('index', times),'float':('index', np.linspace(0,1,len(index)))},coords={'index':index})

ds.groupby_bins('index', bins).reduce(xr.core.computation.duck_array_ops.mean, dim="index", numeric_only=False)

@dcherian Is working on a solution, if I interpret the linked commits correctly.

I did look in to it. I think we have to choose between

Have .reduce(..., numeric_only=True) pass datetime through.
Expose numeric_only in top-level aggregations with default True.
Add a new datetime_is_numeric: bool kwarg to reduce and set it appropriately.


getStringWidth problem

See OpenMods/OpenPeripheral#131

Removed this in 4df5ffe4c87c9f39006dcebe54a664fbce17ae28, since it was misleading and impossible to fix on server.


RACI Matrix - Amazon Snowcone

Describe the Issue
Enterprise Hosting in partnership with the CPF team is looking to create a new service offering to allow Ministries to use portable storage devices (such as AWS Snowcone) to transfer large amounts of data from on-prem to public cloud.
Additional Context
Work will be led by Enterprise Hosting (EH) Team and the CPF Team (as one of the partners). A RACI matrix has been developed by Brian P. and Nick C.
Acceptance Criteria
.Gary to discuss with Nick C. whether 1) ISB will need to be informed about the PIA (development), and 2) which ISB team member will need to be informed/consulted regarding the PIA;
.Gary to meet with EH Team to discuss division of responsibilities;
.Gary to assist with development of RACI for this service request.
.
.
.
.
.
.
.
.
Extra Template Info
Definition of Ready
These set of conditions will need to be met in order to bring a ticket into the sprint and start work. Protects the team from unclear requirements.
Issues (Task/Story/Spike) aligned to an EPIC and linked appropriately.
Assigned to the appropriate CPF MEMBER (and is not left blank in ZenHub) at the start of sprint (1st day of the sprint)
Acceptance has been defined for the issue and has been reviewed with CPF team and approved by the necessary approver.
Has been sized and estimated by the delivery team.
Detailed breakdown of the steps required to complete the story in the additional context
Any additional specifications have been documented, reviewed with CPF and approved by the necessary approver.
Acceptance Criteria
A set of pre-defined requirement that need to be met in order to mark the user story as “done”.
It should be testable with no room for interpretation
It should be either “pass” or “fail”
It should be clear enough for business stake holders to understand
As part of the user story, it should be written from the user perspective
A well written acceptance criteria is great for

managing expectations
defining scope
reducing ambiguity
establishing testing criteria for QA
Given (Pre-condition)
When (Action)
Then (Outcome)

Definition of Done
These set of conditions are met for work to be considered as complete on an issue.
All acceptance criteria(s) have been validated.
All the necessary documentation for the issue has been uploaded to Teams.
Functionality has been tested when applicable.
Functionality has been demonstrated to the relevant stakeholders (where applicable).
Story has been scheduled for a Sprint Demo/community update and impacted teams invited to the Demo.
Stories accepted by PO and documented for potential sharing with impacted users.

Documented = Captured in Community Update (CU) deck
Release notes = aspirational goal of providing a bulleted list of features and changes that our users can touch. Was going to store them in our Cloud Pathfinder repo. Can also display to users in the CU

STRA competed and shared with Enterprise Hosting who will update the RACI.


Compliance with biolink model 2.2.11

Updating to biolink model 2.2.11. Many mappings changed

[x] Get up to date with biolink model 2.2.11
[ ] make sure Travis CI test-suite passes
[ ] run a test build

Created branch issue-187 for work on updating kg2pre

As of 12/16/2021 the biolink model's latest version is 2.2.12

Note there is a deprecation warning
  DeprecationWarning,
/home/ubuntu/kg2-venv/lib/python3.7/site-packages/rdflib_jsonld/__init_
_.py:12: DeprecationWarning: The rdflib-jsonld package has been integra
ted into rdflib as of rdflib==6.0.1.  Please remove rdflib-jsonld from
your project's dependencies.

In test build, error in rule Ontologies_and_TTL
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1364, in <
module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 143, in ma
ke_kg2
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 67, in loa
d_ont_file_return_ontology_and_metadata
    ontology = kg2_util.make_ontology_from_local_file(file_name, save_p
ickle=save_pickle)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 801, in make_ontology_f
rom_local_file
    ont_return = load_ontology_from_owl_or_json_file(json_file)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 762, in load_ontology_f
rom_owl_or_json_file
ke_kg2
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 67, in loa
d_ont_file_return_ontology_and_metadata
    ontology = kg2_util.make_ontology_from_local_file(file_name, save_p
ickle=save_pickle)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 801, in make_ontology_f
rom_local_file
    ont_return = load_ontology_from_owl_or_json_file(json_file)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 762, in load_ontology_f
rom_owl_or_json_file
    return ont_factory.create(ontology_file_name, ignore_cache=True)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/ontol
_factory.py", line 76, in create
    return create_ontology(handle, **args)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/cachier/core.
py", line 129, in func_wrapper
    return func(*args, **kwds)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/ontol
_factory.py", line 93, in create_ontology
    ont = translate_file_to_ontology(handle, **args)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/ontol
_factory.py", line 155, in translate_file_to_ontology
    g = obograph_util.convert_json_file(handle, **args)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/obogr
aph_util.py", line 116, in convert_json_file
    return convert_json_object(json.loads(jsonstr), **args)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/obogr
aph_util.py", line 137, in convert_json_object
    property_chain_axioms=property_chain_axioms, **args)
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/obogr
aph_util.py", line 45, in add_obograph_digraph
    digraph.node[id]['label'] = n['lbl']
AttributeError: 'MultiDiGraph' object has no attribute 'node'

Error in Rule Ontologies and TTL
Reading ontology JSON file: umls-mth.json; size: 291092.37 KiB
file: umls-mth.ttl; last modified: 2021-10-28 07:49:13 GMT
Calling get_inverse_rels
Calling make_nodes_dict_from_ontologies_list
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1364, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 177, in make_kg2
    select_datatype_properties) # temporary addition for Ontobio Issue #507
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 526, in make_nodes_dict_from_ontologies_list
    [biolink_category_tree, mappings_to_categories] = generate_biolink_category_tree(first_ontology, curies_to_categories)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 385, in generate_biolink_category_tree
    biolink_category_tree = kg2_util.get_biolink_category_tree(biolink_ontology)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 422, in get_biolink_category_tree
    parent = biolink_ontology.node(parent)['lbl']
  File "/home/ubuntu/kg2-venv/lib/python3.7/site-packages/ontobio/ontol.py", line 341, in node
    return self.get_graph().node[id]
AttributeError: 'MultiDiGraph' object has no attribute 'node'

I keep running into the error AttributeError: 'MultiDiGraph' object has no attribute 'node'. The MultiDiGraph object looks like it comes from networkx. The node attribute should be nodes in networkx version 2.6.2.
@saramsey This seems somewhat related to https://github.com/RTXteam/RTX-KG2/issues/28.

ontobio requires networkx version 2.3 at minimum now. I updated the networkx version in requirements-kg2-build.txt

Updating networkx to 2.3 seems to have fixed the Attribute Error
Now there's a Value Error
warning:  for key: knowledge_source, dropping second value: OBO:ro.owl; keeping first value: OBO:go/extensions/go-plus.owl
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1364, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 177, in make_kg2
    select_datatype_properties) # temporary addition for Ontobio Issue #507
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 715, in make_nodes_dict_from_ontologies_list
    bpv_pred_curie = convert_bpv_pred_to_curie_func(bpv_pred)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1321, in <lambda>
    curie_expander)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1312, in convert_bpv_predicate_to_curie
    raise ValueError('unable to expand CURIE: ' + bpv_pred)
ValueError: unable to expand CURIE: dcterms:description

Error on Rule Finish:
RuleException:
NameError in line 15 of /home/ubuntu/kg2-code/Snakefile:
The name 'simplified_output_nodes_file_full' is unknown in this context. Please make sure that you defined that variable. Also note that braces not used for variable access have to be escaped by repeating them, i.e. {{print $1}}
  File "/home/ubuntu/kg2-code/Snakefile", line 15, in __rule_Finish
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Exiting because a job execution failed. Look above for error message
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

I keep running into the error AttributeError: 'MultiDiGraph' object has no attribute 'node'. The MultiDiGraph object looks like it comes from networkx. The node attribute should be nodes in networkx version 2.6.2.

I wonder if we have a new version of networkx installed or something?  Can I see a stack trace?


Module 3 exercises [Gideon Anim Kofi Bempong]

WS2_M3E1_Template.docx

Well done, Gideon. You clearly grasped the material outlined in Module 3 and successfully applied it to these exercises. Thank you for your participation! Please be sure to fill out the post-Module survey before you begin the next Module.


Do you have the same problem in training

When I was training, I got an error and couldn't find the price./image/train2014_resized/COCO_val2014_0000000000257.jpg.  I can't understand why the verified picture appears in the training folder.May I ask where the modification is needed? Thanks.

I will look into it tomorrow and get back to you.

When I was training, I got an error and couldn't find the price./image/train2014_resized/COCO_val2014_0000000000257.jpg. I can't understand why the verified picture appears in the training folder.May I ask where the modification is needed? Thanks.

The problem is now fixed, please download the new code. The problem was that the output path in resize.py is not the same as the one in the config.py file, but now I have remove the resize.py and let pytorch do the resizing. Please feel free to change the setting is config.py if you have different paths to the data.


Database driver plug-in architecture

PostgreSQL and MySQL support are great. However, I'm sure there will eventually requests to support every database under the sun. For example, I would like to see SQL Server support. It might be worthwhile to have a plug-in architecture (i.e., dynamically linked) that allows the community to create separate database driver plug-ins for K3s.
It appears that this "etcd shim" capability introduced with k3s may eventually become a more generic solution usable with upstream K8s. In that case, having a wide range of database choices would be even more valuable.

I'm quietly already working on this.  More details to come. https://github.com/ibuildthecloud/kine/

Needs testing; needs docs regarding external db.

@ShylajaDevadiga to test as a low priority / background task. We should test with Microsoft SQL. Not certain if we will fully support (official in docs for example) but this should work.

Closing due to age. Kine is now a top-level project.


Harmony hub won't pair with version 5.0

Today I upgraded to v5.0 and started with a clean slate on a Debian 9 VM. Everything works fine except that my Harmony won't pair with the ha-bridge. There is nothing in the log that there are  any requests coming in from the harmony. The app says it has found a Hue brdige and asks to press the button like others have reported. I have tried the "search for Hunter device" workaround but that doesn't help.
The only thing showing in the log is
2017-12-01 16:34:02,435 [main] INFO  com.bwssystems.HABridge.upnp.UpnpListener - Traceupnp: SSDP M-SEARCH packet from <IP_ADDRESS>:50255 2017-12-01 16:34:02,435 [main] INFO  com.bwssystems.HABridge.upnp.UpnpListener - Traceupnp: send upnp discovery template 1 with response address: <IP_ADDRESS>:80 to address: /<IP_ADDRESS>:50255 2017-12-01 16:34:02,436 [main] INFO  com.bwssystems.HABridge.upnp.UpnpListener - Traceupnp: send upnp discovery template 2 with response address: <IP_ADDRESS>:80 to address: /<IP_ADDRESS>:50255 2017-12-01 16:34:02,437 [main] INFO  com.bwssystems.HABridge.upnp.UpnpListener - Traceupnp: send upnp discovery template 3 with response address: <IP_ADDRESS>:80 to address: /<IP_ADDRESS>:50255
(.2 is the ha-bridge IP). Nothing from the Harmony hub which is .73.
I can't seem to be able to change the logging level. When I change to e.g. "trace" and click on "update logging levels" I get a green confirmation box but then it reverts back to "info".
It's running on port 80, IP address is that of the local machine and the UPnP port is 5000. My Amazon Echo had no problem pairing. My "real" Hue bridge had no problem pairing with the Harmony.
My router (Sophos XG) doesn't support UPnP but I don't see why it would be involved since we're on a LAN.
No firewall or anything. Any ideas?

identical to #797


Change log message type from error to debug

Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Log message type fix.

What is the current behavior? (You can also link to an open issue here)
Log message for events that are not "configure" in SocketIO is reported as error.

What is the new behavior (if this is a feature change)?
The message type is debug.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
No.

Is there any issue related to this PR in other repository? (such as dojot/dojot)
https://github.com/dojot/dojot/issues/1214

Other information:

Closing PR to make adjustments in commits. The new one can be found here:
https://github.com/dojot/data-broker/pull/75


Added more sample attributes & added a .gitignore file

Just extended the list to make it easier to identify all the characteristics & services.
gitignore helped merge properly.

Cool, merging.


Remove manual push into the repo as a process step

Opinion:

My Opinion: this is unneeded the process in step F is cloning an existing repo and not creating a new repo where we don’t know if remote is present.
It does not hurt of fail the process overall

Closing. The First workload doc is intentionally more manual. This task will eliminate several steps


[Polimec] Add in-app staking support for PLMC

Hi team, please help add in-app staking support for Polimec in SubWallet!

Staking doc: https://hub.polimec.org/learn/staking

Account has plmc staking: 59iUooVWNUHjACvKByAtheN7Hi3w9yimTvMMBasnmULjhKsS

Resolved in https://github.com/Koniverse/SubWallet-Extension/issues/2771


Features/106 oracle - Added Oracle support

Implemented Oracle support.

Would someone please just take a quick look at this one before I merge? Do you see any issues with some of the changes made to make Oracle support possible? Maybe @bptillman or @wokket ?

I'm AFK for a while I'm afraid on holidays.   If I can get a chance I'll
try and have a look, but I have 0 Oracle experience.
On Sun, 14 Nov. 2021, 08:18 Erik A. Brandstadmoen, @.***>
wrote:

Would someone please just take a quick look at this one before I merge? Do
you see any issues with some of the changes made to make Oracle support
possible? Maybe @bptillman https://github.com/bptillman or @wokket
https://github.com/wokket ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/erikbra/grate/pull/134#issuecomment-968156230, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AA5ANTLDNMJHE7MNRZMRAWLUL3P2RANCNFSM5H7CFPVQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

No worries, @wokket  - enjoy your holidays! I'll just leave it open for a little while, and if no one comments, I'll just merge it. Don't bother getting into this on holiday. Enjoy life! 😆

Like @wokket , I do not have Oracle experience. But if you'd like I can take a look at this PR and test it out a bit to see if I see anything obvious that needs fixing before you merge it

Yes, please. I haven't worked with Oracle in 15 years, so I was more thinking of just taking a quick look over the changes. If you don't have any specific Oracle knowledge, don't sweat it a lot.

@erikbra I finally got a chance to look over this after your merge, and nothing stands out to me as needing to change 👍

Thanks. Sorry, I pulled the trigger 🤓
Thanks for double-checking anyway. Always good to have a couple of extra eyes in things!


Added previous months speakers, still need titles for May

we need reusable components 😳 or a dropdown selector for month

Thank you @Folashade!

of course!


[CCDDR] Distant Realms - Module crashes on initialization

Module Information

Identifier: CCDDR
Module Name: Distant Realms
Version: 1.1

Describe the bug
Module crashes on initialization
To Reproduce
Steps to reproduce the behavior:

Add to moduleConfig.json
Run MBBSEmu
See error

Expected behavior
Module loads
Screenshots
Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'Unable to locate 1020:400E')
   at MBBSEmu.Memory.MemoryCore.GetByte(UInt16 segment, UInt16 offset)
   at MBBSEmu.HostProcess.ExportedModules.Majorbbs.cncchr()
   at MBBSEmu.HostProcess.ExportedModules.Majorbbs.Invoke(UInt16 ordinal, Boolean offsetsOnly)
   at MBBSEmu.HostProcess.ExecutionUnits.ExecutionUnit.ExternalFunctionDelegate(UInt16 ordinal, UInt16 functionOrdinal)
   at MBBSEmu.HostProcess.ExecutionUnits.ExecutionUnit.Execute(IntPtr16 entryPoint, UInt16 channelNumber, Boolean simulateCallFar, Boolean bypassState, Queue`1 initialStackValues, UInt16 initialStackPointer)
   at MBBSEmu.Module.MbbsModule.Execute(IntPtr16 entryPoint, UInt16 channelNumber, Boolean simulateCallFar, Boolean bypassSetState, Queue`1 initialStackValues, UInt16 initialStackPointer)
   at MBBSEmu.HostProcess.MbbsHost.WorkerThread()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Software Information:

OS: Windows 2019 Server Standard
Telnet Client: NetRunner
MBBSEmu Build: 100220-5

This looks like a buffer overflow of PRF caused by calls to cncint being handled improperly. cncint is one of the CNC* commands we don't have Unit Tests for. There might be a bug in the input processing -- which is causing Distant Realms to loop adding 423 bytes at a time to the PRFBUF and overflowing it.


Standardize APPROX_PERCENTILE_CONT / PERCENTILE_CONT and similar aggregation functions

Is your feature request related to a problem or challenge?
After https://github.com/apache/datafusion/pull/11721/files we handle nulls in APPROX_PERCENTILE_CONT by removing them before computing the percentile.
This is the default behaviour of PostgreSQL, Spark, etc.
However the syntax of the current aggregate functions is confusing  for APPROX_PERCENTILE_CONT, as it supports IGNORE NULLS | RESPECT NULLS
Describe the solution you'd like
Come up with a plan to bring the syntax / semantics of APPROX_PERCENTILE_CONT and PERCENTILE_CONT and similar aggregations closer to PostgreSQL and others by (one or more).

Supporting something like PostgreSQL/Spark/Snowflake syntax: SELECT   percentile_cont(<fraction double precision>) WITHIN GROUP (<sort_expression>)
Disabling specifying IGNORE NULLS | RESPECT NULLS for aggregations functions where they do not make sense / are not supported.
Comparing results against other engines

Describe alternatives you've considered
No response
Additional context
No response

See https://github.com/pydantic/logfire/issues/433, support for WITHIN GROUP would be great.

take
(cf. #12824)

@Dandandan Would you assign this task to me?


[Discussion] Could you please provide the script to generate the dataset used in stage2?

Discussion
I cannot understand how to construct the "fewshot_samples" in table 10, so could you please help me?

Is "fewshot_samples" the output of ChatGPT/GPT4? Then how do you put the information(captions, bbox, etc) of each image into "context"?


Add support for GraphQL subscriptions

GraphQL being transport agnostic, we should be able to use Phoenix Channels in order to support real-time subscriptions. The Absinthe team provides the absinthe_phoenix package for this purpose, see docs for more details.

See issue #10 for a valid use-case.


Data is not defined while updating options V1.0.0

https://apexcharts.github.io/Blazor-ApexCharts/methods/update-options
You can try update options from the link above.

Hello,
Thank you for reporting.
Fix will be released in an hour.

Fixed in v1.0.1


update dependency: GitBucket 4.21.0

Recompile is required.
Method signature of AccountService#getAllUsers() has been changed.

Sorry, this PR is outdated. updated dependencies by #3.