arch: all: Remove legacy code

The struct _kernel_ach exists only because ARC' s port needed it, in
all other ports this was defined as an empty struct. Turns out that
this struct is not required even for ARC anymore, this is a legacy
code from nanokernel time.
Signed-off-by: Flavio Ceolin<EMAIL_ADDRESS>

I'd rather this abstraction was not removed, I'd planned on using it for some upcoming memory protection work.
If you want to get rid of the empty structs, you could have arch/cpu.h define Z_ARCH_HAS_KERNEL_ARCH or something and ifdef the definition of the arch member based on it.

without future changes this is just legacy and not used code :/ We can keep it for future work, but at the time this patch was suggested it makes sense IMHO.
So, what you suggest, you want me to keep it what condition ? If I add this guard any platform will need it currently

I'd rather this abstraction was not removed, I'd planned on using it for some upcoming memory protection work.
If you want to get rid of the empty structs, you could have arch/cpu.h define Z_ARCH_HAS_KERNEL_ARCH or something and ifdef the definition of the arch member based on it.

without future changes this is just legacy and not used code :/ We can keep it for future work, but at the time this patch was suggested it makes sense IMHO.
So, what you suggest, you want me to keep it what condition ? If I add this guard any platform will need it currently

No you won't.
diff --git a/kernel/include/kernel_structs.h b/kernel/include/kernel_structs.h
index dd625d5844..909efe5844 100644
--- a/kernel/include/kernel_structs.h
+++ b/kernel/include/kernel_structs.h
@@ -168,8 +168,10 @@ struct z_kernel {
        struct k_thread *threads; /* singly linked list of ALL threads */
 #endif

+#ifdef Z_ARCH_HAS_KERNEL_ARCH_MEMBER
        /* arch-specific part of _kernel */
        struct _kernel_arch arch;
+#endif
 };

 typedef struct z_kernel _kernel_t;

You are done and can delete all the empty struct _kernel_arch defintions in the arch headers

I'd rather this abstraction was not removed, I'd planned on using it for some upcoming memory protection work.
If you want to get rid of the empty structs, you could have arch/cpu.h define Z_ARCH_HAS_KERNEL_ARCH or something and ifdef the definition of the arch member based on it.

without future changes this is just legacy and not used code :/ We can keep it for future work, but at the time this patch was suggested it makes sense IMHO.
So, what you suggest, you want me to keep it what condition ? If I add this guard any platform will need it currently

No you won't.
diff --git a/kernel/include/kernel_structs.h b/kernel/include/kernel_structs.h
index dd625d5844..909efe5844 100644
--- a/kernel/include/kernel_structs.h
+++ b/kernel/include/kernel_structs.h
@@ -168,8 +168,10 @@ struct z_kernel {
        struct k_thread *threads; /* singly linked list of ALL threads */
 #endif

+#ifdef Z_ARCH_HAS_KERNEL_ARCH_MEMBER
        /* arch-specific part of _kernel */
        struct _kernel_arch arch;
+#endif
 };

 typedef struct z_kernel _kernel_t;

You are done and can delete all the empty struct _kernel_arch defintions in the arch headers

yeah, but my commit will still remove struct _kernel_arch definition, since they are being defined empty. Shouldn't be better you add it when you add code that needs it ? Other way is remove this commit by now

I'd rather this abstraction was not removed, I'd planned on using it for some upcoming memory protection work.
If you want to get rid of the empty structs, you could have arch/cpu.h define Z_ARCH_HAS_KERNEL_ARCH or something and ifdef the definition of the arch member based on it.

without future changes this is just legacy and not used code :/ We can keep it for future work, but at the time this patch was suggested it makes sense IMHO.
So, what you suggest, you want me to keep it what condition ? If I add this guard any platform will need it currently

No you won't.
diff --git a/kernel/include/kernel_structs.h b/kernel/include/kernel_structs.h
index dd625d5844..909efe5844 100644
--- a/kernel/include/kernel_structs.h
+++ b/kernel/include/kernel_structs.h
@@ -168,8 +168,10 @@ struct z_kernel {
        struct k_thread *threads; /* singly linked list of ALL threads */
 #endif

+#ifdef Z_ARCH_HAS_KERNEL_ARCH_MEMBER
        /* arch-specific part of _kernel */
        struct _kernel_arch arch;
+#endif
 };

 typedef struct z_kernel _kernel_t;

You are done and can delete all the empty struct _kernel_arch defintions in the arch headers

ok, this means that I'll remove this struct for all architectures. This piece of code won't make sense because any port will satisfy this condition right now. Won't be better have it when you use it ?

I'm actually with @ceolin here, I don't see much value to this struct as it stands. The is and will only ever be one _kernel struct. The reason other architectures don't have the struct defined isn't that they don't need their own global data, it's because they put it in their own global storage (static variables, yada yada). I don't see why new work can't continue to do this. In fact the kernel as a whole is really inconsistent on this. Some stuff is a field in _kernel and some stuff isn't, and there's no rhyme or reason.
That said the motiviation for this change (that empty structs are somehow "bad") is IMHO ridiculous. But I think it's useful cleanup regardless.

The motivation is not that, it is the fact it is not part o C standard, this is a compiler extension and make us less portable. These cases were really nice actually, it was an opportunity to remove things not used from kernel, it did not require any additional change.


Initial commit of Restore Get files

commands that scan for the restore files
Fixes #
Changes proposed in this pull request:
Adds the 3 scanning cmdlets for the restore framework
Get-DirectoryRestoreFile - scans a given folder for backups files and returns them
Get-OlaHRestoreFile - scans an OlaH style folder structure for backup files
Get-XpDirTreeRestoreFile - uses sql server's xp_dirtree to scan a folder for backup files (checks for the users ability to do so first)

How to test this code:

[ ]
[ ]

Has been tested on minimum requirements:

[ ]  Powershell 3
[ ]  Windows 7
[ ]  SQL Server 2000

Has been tested on maximum requirements:

[ ]  SQL Server vNext
[ ]  Windows 10
[ ]  Azure Database

Tests for tester:

[ ] Working/useful help content, including link to command on dbatools web site
[ ] All examples work as advertised
[ ] Does not contain template content
[ ] Does not contain excessive/unnecessary amounts of comments
[ ] Works remotely
[ ] Works locally
[ ] Works on lower versions or throws error specifying version not supported
[ ] Works with named instances
[ ] Works with clustered instances
[ ] Handles offline/read only databases
[ ] Supports multiple servers (at the command line or piped from Get-SqlRegisteredServerName)
[ ] No un-handled errors which stop the command working with multiple servers

Rushing and did something stupid. Cancelling. Will repush once I'm not moving.


ESLint is not validating any files yet.

Still happening in VSCode version 1.30.2, version of ESLint - 1.8.0

@xerosanyam this works for me. Can you provide me with a GitHub repository I can clone that demos what you are seeing. Do you have eslint installed (npm install eslint)?

I'm seeing the same issue @dbaeumer @xerosanyam
Extension version: 1.8.0
VS Code version: Code - Insiders 1.32.0-insider (11416de365a1e96302e171c9eebc2547872ac230, 2019-02-23T08:16:36.706Z)
OS version: Darwin x64 18.2.0
$ which eslint
/Users/junger/.config/yarn/global/node_modules/.bin/eslint

I had accidentally executed ESLint: Disabled ESLint. Executing ESLint: Enable ESLint fixed the issue. It might be helpful to update the text of this error message?

@joshunger will look into it. Best would be to print some info about which workspace folders are disabled.

I had accidentally executed ESLint: Disabled ESLint. Executing ESLint: Enable ESLint fixed the issue.
So the steps to reproduce this issue are -

Shift + Command + P and select ESLint: Disabled ESLint
Close vscode
Shift + Command + P and select ESLint: Show Output Channel

Is there a way to fix this error message?

Having the same problem today, I'm fairly sure I didn't disable EsLint at any point,
But disabling and enabling it for the workspace did restore the AutoFixOnSave functionality I had working before.  Simply enabling it may have worked, but I first selected disable under the "Did you turn it off and back on?" rule of fixing things.
Thanks for the hint.
Uninstalling and reinstalling eslint did not fix the issue for me.

Same issue, can someone provide more info about this error? The message "ESLint is not validating any files yet." does not mean nothing for me.
eslint is installed, configuration file is generate by node_modules/.bin/eslint --init, eslint is enabled inside project.. something else is required?

After a lot of try I figure out what the problem is, you need to specify inside the vscode configuration the eslint.validate with file type that you want to lint. For example, in a Vue project you have to specify:
    "eslint.autoFixOnSave": true,
    "eslint.validate": [
        {
            "language": "vue",
            "autoFix": true
        },
        {
            "language": "html",
            "autoFix": true
        },
        {
            "language": "javascript",
            "autoFix": true
        }
    ]

Then, this is the really strange thing, nothing happens, will still not work, you have to disable and re-enable the linter with command ESlint: Disable ESLint and  ESlint: Enable ESLint, finally it start works!
So, the real issues are two:

the dummy error message that create a lot of confusion and does not provide any kind of information.. please, please, PLEASE, change it to something like "File type not configured, edit eslint.validate configuration" or "Check eslint.validate configuration to lint this file".

configuration changes are not applied and need a file open/close and/or enable and disable linter feature.

These two issues combined together make people crazy searching for the cause of this "non functioning behavior".

I changed the message to:

However I can't reproduce the lack of applying configuration changes. This works for me.

However I can't reproduce the lack of applying configuration changes.

@dbaeumer have you tried this?

remove the module
open a couple of files in a project environment
re-install the module
try to configure and run linter for that opened files

seems related to current open documents, I think.

I added "eslint.trace.server": "messages" to my user settings, then checked the Output of ESLint and got the following message:

The language client requires VS Code version ^1.31 but received version 1.30.2

I tried to update, and discovered that VS code was running out of my downloads folder, so it couldn't update 🤦♂ (see issue). I moved it to Applications, updated, and now ESLint is working again! Hope this helps someone else 😃

@daaru00 the extension doesn't support hot install of the eslint npm package right now. I created https://github.com/Microsoft/vscode-eslint/issues/643.

@alexlouden release a new version that addresses the version mismatch.

Releases 1.8.2 which containes the improved error message

Possible ways of reproducing the issue are to open a new window and then "open folder" or "add workspace folder" to it.
I remember opening a new window for a separate project, it was working with the previous workspace/window but not with the new window. Since then it isn't working with any folder/workspace.
All the folders I tested with had eslint --init done.

Any update guys?


Fedora-Ansible installation sets subnetwork address as bridge address

In inventory we set:

[minions]
<IP_ADDRESS>  kube_ip_addr=<IP_ADDRESS>
<IP_ADDRESS> kube_ip_addr=<IP_ADDRESS>

kube_ip_addr is used in roles/network-hack-bridge/tasks/main.yml:

name: Create kubernetes bridge interface
command: nmcli connection add type bridge ifname kbr0 con-name kubernetes-bridge ip4 {{ kube_ip_addr }}/24

in the end, we have bridge in first minion:

kbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
inet <IP_ADDRESS>  netmask <IP_ADDRESS>  broadcast <IP_ADDRESS>

Subnetwork address cannot be bridge address.

In Vagrant scripts in cluster/vagrant/provision-network.sh there is:

BRIDGE_ADDRESS=${BRIDGE_BASE}.${MINION_ID}.1
29	BRIDGE_NETWORK=${BRIDGE_ADDRESS}/24

I this capability should be added to fedora-ansible scripts.

this is not a kube bug and is a kubernetes-ansible problem.  this issue should be closed.  @bgrant0607


Add documentcloud_pages to CvrLobbyDisclosureCd.rec_type (in lobbying.py)

Overview
Add documentcloud_pages to the rec_type field on CvrLobbyDisclosureCd the database model.
Basically, we think rec_type should be defined as a "choice field" (in Django parlance), which just means it's a field with a defined list of valid values and an intelligible definition for each value.
Our goal is for every potential choice field to include references to locations in the official documentation where the field's valid values and their meanings are described AND to have those choices included in the field's definition.
And we need your help!
What to do
Step 1: Claim this ticket by adding yourself as an Assignee (to the right)
Step 2: Find where in the official documentation rec_type's list of valid values is defined.
The best place to look first is in the .CAL layout documents: .CAL Format version 2.01 and .CAL Format version 1.05.02. These are documents that describe the layout of the .CAL file format, which is the required electronic format of any filings ingested by the CAL-ACCESS system (more on that here if you are interested).
You can start by opening the document and just searching for "rec_type". For example, if you were assigned to tackle the stmt_type field on the CvrCampaignDisclosureCd model, you would be looking for something like this::

But note that the same column might appear on multiple tables, and the list of valid values may vary depending on the table. We want to be sure to find the list of valid values for the rec_type on the CvrLobbyDisclosureCd model.
Another good place to look is in the MapCalFormat2Fields document, which describes how .CAL format fields are mapped to CAL-ACCESS database table columns. Note the Valid_V column on the far right.
Step 3: Copy the DocumentCloud ID and the page number(s) where rec_type's list of valid values is defined.
The DocumentCloud ID can be found the in the URL that points to the document. For example, the url to the .CAL Format version 2.01 doc is http://www.documentcloud.org/documents/2712034-Cal-Format-201.html#document/p1, and the id is 2712034-Cal-Format-201.
And the start page we want to grab is the DocumentCloud page number, not the page number printed on the .PDF. If the list of valid values spans multiple pages, then grab the page where the list ends as well.
Step 4: Find where rec_type is defined in lobbying.py in the /calaccess_raw/models/lobbying.py file underneath the CvrLobbyDisclosureCd class.
Since the model files are rather long, take care to be sure you find the where rec_type is defined for the CvrLobbyDisclosureCd model. It should look something like this:
    class CvrLobbyDisclosureCd(CalAccessBaseModel):
        # ...
        # several lines of code later...
        # ...
        rec_type = fields.CharField(
            max_length=4,
            db_column='REC_TYPE',
            blank=False,
            verbose_name='record type',
            help_text='',
        )

Step 5: Add the document references
Say you found the list of valid values listed in three places. You would list them in the documentcloud_pages attribute like this:
    class CvrLobbyDisclosureCd(CalAccessBaseModel):
        # ...
        # several lines of code later...
        # ...
        rec_type = fields.CharField(
            max_length=4,
            db_column='REC_TYPE',
            blank=False,
            verbose_name='record type',
            help_text='',
            documentcloud_pages=[
                DocumentCloud(id='2711616-MapCalFormat2Fields', start_page=10),
                DocumentCloud(id='2712033-Cal-Format-1-05-02', start_page=19),
                DocumentCloud(id='2712034-Cal-Format-201', start_page=24, end_page=25),
            ]
        )

Note: You only need to specify the end_page if the list spans multiple pages.
Step 6: Update the rec_type's help_text (if necessary)
It might be missing or it might be an empty string or it might not be consistent with what you see in the documentation. Note: The entire help_text string needs to be enclosed in single- or double-quotes like this:
    help_text='This is the help text',

If the help text is really long (80+ characters), then it needs to span multiple lines like this:
    help_text='This is the help text...blah...blah...blah...blah...blah...blah...\
...blah...blah...blah',

If you can, check the rec_type's defined choices against the actual values in rec_type's database column.
This will require you to install the app and run the updatecalaccessrawdata command. Then you can get the distinct column values with a GROUP BY query like this:
    SELECT REC_TYPE, COUNT(*)
    FROM CVR_LOBBY_DISCLOSURE_CD
    GROUP BY 1
    ORDER BY 1;

It's likely that you'll find undocumented values in database column, some of which are variants of the valid values. You can note these by repeating the same definition for each variant like this:
    OFFICE_CD_CHOICES = (
        ('GOV', 'Governor'),
        ('gov', 'Governor'),
        ('GUV', 'Governor'),
        # ...
    ),

If you're dealing with a set of choices that's been consolidated into calaccess_raw/annotations/choices.py, you can map the different variants to the same definition using the key of the choices dict like this:
    OFFICE_CD_CHOICES = (
        ('gov', look_ups.OFFICE_CODES['GOV']),
        ('GUV', look_ups.OFFICE_CODES['GOV']),
        # ...
    ),

If you're not able to check the database column values for whatever reason, don't sweat it. We'll deal with it in a later issue.
Wrap up: Review your changes and create a pull request

That's it!
PS: If any of this feels confusing, or if it doesn't seem to match up with what you are seeing, or if you have any suggestions about another approach, don't hesitate to speak up. You can post a comment below, or hit up the #california-civic-data channel on the News Nerdery Slack.

Done


MAINT: integrate: Avoid using variable l in quadrature, integrate._…

…ivp: Avoid using variable I in bdf,io.matlab: Avoid using variable O in mio4
fixed https://github.com/scipy/scipy/issues/8389

If you're going to the trouble of replacing the problematic single letters (which can look like numbers in some fonts), why not replace them with longer variables names rather than a different single letter?

It is only Python 3.5 which failed on TravisCI, and it looks like white space and rounding issues only - my hunch would be NumPy 1.14 related. The logs are a little odd here:
$ travis_retry pip install $NUMPYSPEC
You must give at least one requirement to install (see "pip help install")

Later on we see:
Successfully uninstalled numpy-1.14.0

This would seem to be a wider issue and not specific to your changes.

See #8279 for the NumPy 1.14 failure

the integer is represented as MOPT where M is the thousands digit, O is the hundreds digit, P is the tens digit, and T is the ones digit. Here you are changing O to different chars but the error message shouldn't change. It's still the O of MOPT.
I thought we were going to ignore this this PEP8 rule in one of those issues.

Similarly the change from I to K for an identity matrix is a bad idea. Something like id should be ok.

@person142 can I give it a name then like identity or anything else??

TBH, I don't believe that benefits here justify the code churn, and I think this PR should not be accepted.

-0 from me on PRs fixing variable names etc. minor style issues

If the SciPy team are on balance against these changes, should we close #8389 as a won't fix (and perhaps say so explicitly in the style checking configuration comments?)

I agree with @ev-br and @pv, so I'm going to close this. Thank you nevertheless @samyak0210 for attempting to address an open issue.

If the SciPy team are on balance against these changes, should we close #8389 as a won't fix (and perhaps say so explicitly in the style checking configuration comments?)

@peterjc good point. I'll comment on that issue.


how can I create a PR against my fork

I am looking at this code:
https://github.com/packit-service/ogr/blob/b7c5333c4df64ecdf854a5375c207a789ca51a01/ogr/services/github/pull_request.py#L111
what if I want to create a PR against my fork and not the upstream project?

Gitlab probably supports forking of forks as well.

[ ] We need to make sure this workflow works.
[ ] Put this to docstring.

Could you please point out the steps for tackling this issue?

We need to:

[ ] check the methods and docstring if it is possible / it is well described
[ ] check the implementations

[ ] ideally, create a test for each implementation, to have all combinations (fork -> fork, upstream -> upstream, fork -> upstream)

[ ] update docstrings if it is not clear

Related code:

abstract:
https://github.com/packit-service/ogr/blob/27a7276662a99c10ef163bb595325010d18f450e/ogr/abstract.py#L395-L412

github:
https://github.com/packit-service/ogr/blob/27a7276662a99c10ef163bb595325010d18f450e/ogr/services/github/pull_request.py#L118-L124

gitlab:
https://github.com/packit-service/ogr/blob/27a7276662a99c10ef163bb595325010d18f450e/ogr/services/gitlab/pull_request.py#L113-L119

pagure:
https://github.com/packit-service/ogr/blob/27a7276662a99c10ef163bb595325010d18f450e/ogr/services/pagure/pull_request.py#L140-L146

(These are the static methods, but there is a wrapper method in the Project classes on top of them.)

@lachmanfrantisek Thanks, I will create a PR fixing the functions.

@saisankargochhayat thank you, I'll take a look.

Given the PR is merged, we can close, right?

Given the PR is merged, we can close, right?

Yup, closed by #404


Python server is using dedicated port for unit test.

Close Julia server sockets in unit test.

Codecov Report

Merging #3 (b9661a9) into main (cb939cf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #3   +/-   ##
=======================================
  Coverage   86.29%   86.29%
=======================================
  Files           1        1
  Lines         124      124
=======================================
  Hits          107      107
  Misses         17       17

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb939cf...b9661a9. Read the comment docs.


Allow non-default branches; more useful logging info

Pull Request Test Coverage Report for Build 141

8 of 23 (34.78%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-4.8%) to 28.512%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

enqueue/check_posted_payload.py
8
23
34.78%

Totals

Change from base Build 138:
-4.8%

Covered Lines:
69

Relevant Lines:
242

💛  - Coveralls


Timed Transition Timer Reset

Implemented one utility method to reset timed transitions' timers for the current state

Hi implemented fsm timer reset

Hi implemented fsm timer reset


Performance and error handling in embulk

Hi,
I read the documentation of embulk but I could not find information related to performance.
I would like to know the performance(data loading speed) of embulk and how embulk handles the error.
Regards,
Vinayak

Here is a performance benchmark by @hito4t: https://translate.google.com/translate?sl=ja&tl=en&u=http%3A%2F%2Fhito4-t.hatenablog.com%2Fentry%2F2015%2F03%2F04%2F122051
He achieved a lot of improvement after that such as: https://translate.google.com/translate?hl=&sl=ja&tl=en&u=http%3A%2F%2Fhito4-t.hatenablog.com%2Fentry%2F2015%2F04%2F07%2F215837&sandbox=1
Error handling is discussed on #27.


Fixed adding game launch options

I've split the launch options that were previously only supported by Epic Games between Epic Games and Steam.

I tried to clone and try, but I get an error that steam_icon.png is not found when building.


SAS Token Expiry Setting Constraints

There is something crucial missing in the documentation, the SAS Token expiry can only be set if there was a key-rotation done atleast once for the storage account. Otherwise, it shows a message as below and this setting is disabled
You must rotate both access keys at least once before you can set a recommended upper limit for SAS expiry interval.
It will be good to put this as a pre-requisite Note within this document. We found this quite late during our security implementation.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: aa509efb-4e6e-eb0d-bc1e-c68af355e9a3
Version Independent ID: 4a7ea10b-1e1c-18bf-cf69-16ecaa487f9d
Content: Create an expiration policy for shared access signatures - Azure Storage
Content Source: articles/storage/common/sas-expiration-policy.md
Service: storage
Sub-service: common
GitHub Login: @tamram
Microsoft Alias: tamram

@ramakrpr Thanks for reporting this. I have assigned the issue to the content author to investigate further and update the document as appropriate.

@ramakrpr - Thanks for your feedback! This was fixed sometime recently.
#please-close


Responsiveness Adjustments

Form inputs need to be fixed for iPad Air, currently overlapping.

.input-style {
width: 50%;
}

Desktop font-clamps need to be larger.

Fixed both points.


Copy-DbaTableData issues no error/warning /information if the -Table supplied does not exist

Bug Report
Copy-DbaTableData issues no error/warning /information if the -Table supplied does not exist. The complete lack of feedback makes it difficult to determine the source of an error if the mistake is a typo.
General Troubleshooting steps

[ X ] Verified running the latest release of dbatools?
[ X ] Verified errors are not related to permissions?
[ X ] Can duplicate in new/clean PowerShell session (clean = powershell -NoProfile)?

Version Information

Operating System (Name|Version): Windows 10
PowerShell Version: PSVersion 5.1.17134.1
SQL Server (Edition|Version): 2017

Steps to Reproduce
Execute this line, where the servers & database exist, but the table does not.

The command completes, but provides zero output. The lack of output hints that nothing happened, but a more explicit warning/error as to why would be helpful (ie, "table does not exist")

@sqllensman : as the most recent editor of Copy-DbaDbTableData is this still open ? I feel like it isn't but can you please check ?

The new code will return a warning if source or destination cannot be determined

perfect, than this can be closed ! Thank you @sqllensman

thanks for the follow up, @niphlod !

my new nickname will be "niph-the-hoverer".


Implement unbuffer interface for HdfsFileInputStream

What changes are proposed in this pull request?
Implement unbuffer interface for HdfsFileInputStream. Fix #16016.
Why are the changes needed?
If the unbuffer method is not implemented, then impala will not be able to use the file handle cache.
Does this PR introduce any user facing changes?
Implement CanUnbuffer and StreamCapabilities for HdfsFileInputStream.

@Jackson-Wang-7 Thanks for you review. I rebase the master branch and commit our latest changes.

/ping @jja725 PTAL thanks!

https://github.com/Alluxio/alluxio/pull/16699 has merged to unblock the maven/pom parts of this change

@qian0817 Do you mind rebasing on latest master? Let's only keep the unbuffer change in this PR, while we rely on the #16699 to build the clients we want. Feel free to let me know if there's any question, thanks!

@jiacheliu3 @jja725 @Jackson-Wang-7 @dbw9580 Please revirwe this PR again, thanks!

Is there any new change from the last time except the pom change?

Is there any new change from my LGTM last time except the pom change?
No change.

Just curious is there any other compute framework use this function except impala?

From HADOOP-14747 we can know that HBase also use the unbffer function

alluxio-bot, merge this please


fix: improper heading sizing and background gradient 👷🏻♂⚒

Close #1492
Screenshots

BEFORE

AFTER

Checklist:

[x] I have linked the PR to the correct issue.
[x] I have read the Contribution Guidelines
[x] I have built and tested the changes, and they do not break or show any errors.

@Vaishnavi-Patil2211 Please merge this PR.


Fix for apparently faulty Numpy NN backward pass

I just tried making a NN for the MNIST data using numpy following your tutorial but found that the solution given in your Jupyter notebook seems to get stuck at an accuracy close to chance (10%) while printing an accuracy of 0.
Based on your explanation of backward propagation I redid the calculations for the backward pass assuming a mean squared error cost function and found a couple of disagreements with your solution.
After these changes here it works well for me.
Please consider merging in the changes if you agree to avoid people getting confused by this otherwise very nice tutorial! :)

I should have followed my own article instead of some literature. Thank you for this, it seems to work and much more closely resemble the article!


Savemap overwrites map author and map name

Description

pretty much the title. our attributions are being lost in the map files whenever changes are made, requiring manual editing for every change which is not great
Reproduction

change the author and name in a map file, then open that file in game and re-save it. the fields will be overwritten with the default DemoStation and Space-Wizards

poop cringe

Those fields should really just be removed.


Association class gives wrong error messages when reflexive

The following Umple code generates an incorrect warning. It should be naming the associations after the role names
Error on line 3 : Class 'Profile' may not have multiple associations with the same name 'friendships'. More information (19)
associationClass Friendship {
  * Profile invitee;
  * Profile inviter;
  status;
}
class Profile {
    name;
}

To resolve this, you need to make sure it works properly in cases where the multiplicity is not exactly as shown, and also it should work with the following:
class Friendship {
  * -- 1 Profile invitee;
  * -- 1 Profile inviter;
  status;
}
class Profile {
    name;
}

that currently also gives the same error 19.
In fact, I suggest starting with the above as it is a simpler case
The problem in the above case is that one or both of the * ends needs to be given a role name, so the message should help with this. The message is generated when analysing class Profile, when in fact it is class Friendship where the association is found, so that adds to the confusion.
Perhaps the message could be adjusted to There are multiple associations between class 'Profile' and class 'Friendship'. Role names need to be added at the 'Profile' end to distinguish the association ends in that class.
However, it turns out that in the associationClass case, there is no way to add such role names I think. It may be necessary to add to the grammar the ability to add role names after the *.

In Summary:

Fix the error message of 19 (notice error 180 looks like a similar error) to be more informative
Potential fix for association class (add grammar to have the ability to add role name after * in the example above)
Quesiontsm:
The fix to warning message itself does not require test since it is a cosmetic fix?
Modification on grammar to adopt new requirement will need some testing?

Note that every error or warning message should have a test to check that it occurs when expected.
As for putting role names on association classes. I think that the optional role name for the 'other end' should come after the multiplicity. so the following code would be accepted.
associationClass Friendship {
  * inviterFriend Profile invitee;
  * inviteeFriend Profile inviter;
  status;
}
class Profile {
    name;
}

But note: Only one of the roles would be needed to avoid the error, although both should be allowed. Role names should also be allowed on any association class, not just ones with the same class. So you will need to add multiple tests for all cases.
The following would then be the logical version using an ordinary class (although only one of the role names is again allowed. The following code should already be accepted.
class Friendship {
  * inviterFriend -- 1 Profile invitee;
  * inviteeFriend -- 1 Profile inviter;
  status;
}
class Profile {
    name;
}

It is causing some issue when I modify singleAssociationEnd to singleAssociationEnd : [[multiplicity]] [~roleName]? [type] [~roleName]?; instance like
 associationClass B{
  * A a1;
  * A a2;
}

Will be interpret as[singleAssociationEnd][upperbound:*] [roleName: A] [type: a1]
I am not sure how to walk around this problem without some huge changes to singleAssociationEnd grammar format.

it seems the rule needs to be that the 'otherEndRoleName' can only be present if the regular role name is there.
So one way to write it would be:
singleAssociationEnd : [[multiplicity]] ( ( [otherEndroleName] [type] [roleName] ) | ( [type] [~roleName]?  ) ) ;

Do we allow user to put associationClass as
 associationClass B{
  * b1 A;
  * b2 C;
}

in this case it would raise the error that b1 is not a class or interface. I think it is hard to distinguish between [[multiplicity]][roleName][type] | [[multiplicity]][type][roleName]


Always prefetching!

I have started to experience intermittent failures of the aws module due to prefetch errors.
Error: Failed to apply catalog: undefined local variable or method `region' for Puppet::Type::Route53_cname_record::ProviderV2:Class
Running a puppet agent -t --trace --debug shows that the prefetch takes place prior to checking for any aws resources. Is there anyway to limit the need to prefetch values especially something like region which will remain static in most cases. A single prefetch at time of catalog compile seems reasonable.

This issue may be a number of things but the error reporting was broken. That's now fixed with https://github.com/puppetlabs/puppetlabs-aws/commit/11a3640167892eb41e2fd278c2e914a79d5852a4 which will be in a release soon. That should expose the real issue.
If you have a large number of domains this might be triggered by rate limiting, in which case it's worth talking to Amazon as they might be able to adjust this.
Do provide the error with this issue and any data you have, we're always interested in further optimising the queries we're making.


[BUG] Service Bus receiving of messages hangs when using processor / sessionProcessor with prefetch

Describe the bug
We are using a sessionProcessor to receive messages and when the property prefetchCount is used, the Service Bus randomly hangs and stops processing after some time for sessionIdleTimeout seconds (in the example below 50 seconds). It starts again after the sessionIdleTimeout, but then multiple exceptions occur when complete() is called (reactor.core.Exceptions$ReactiveException: java.lang.InterruptedException).
Without prefetchCount processing is too slow. It seems that this error does not occur when a smaller value < 5 for prefetchCount is used or it happens not that frequently.
This is the configuration of the ServiceBusProcessorClient we use:
final ServiceBusProcessorClient receiver = new ServiceBusClientBuilder()
          .credential(namespace, new ClientSecretCredentialBuilder()
                .tenantId(tenantId)
                .clientId(clientId)
                .clientSecret(clientSecret)
                .build())
          .sessionProcessor()
          .disableAutoComplete()
          .receiveMode(ServiceBusReceiveMode.PEEK_LOCK)
          .topicName(topicName)
          .prefetchCount(20)
          .maxConcurrentCalls(1)
          .maxConcurrentSessions(1)
          .sessionIdleTimeout(Duration.ofSeconds(50))
          .subscriptionName(subscriptionName)
          .processError(this::processError)
          .processMessage(this::processMessage)
          .buildProcessorClient();
receiver.start();

It seems that this ALSO occurs when using a normal processor without session and with config prefetchCount=10 and maxConcurrentCalls=100. When setting prefetchCount=0 everything works fine. So somehow the prefetch is causing interruptions. The following code also hangs and throws exceptions:
final ServiceBusProcessorClient receiver = new ServiceBusClientBuilder()
          .credential(namespace, new ClientSecretCredentialBuilder()
                .tenantId(tenantId)
                .clientId(clientId)
                .clientSecret(clientSecret)
                .build())
          .processor()
          .disableAutoComplete()
          .receiveMode(ServiceBusReceiveMode.PEEK_LOCK)
          .topicName(topicName)
          .prefetchCount(10)
          .maxConcurrentCalls(100)
          .subscriptionName(subscriptionName)
          .processError(this::processError)
          .processMessage(this::processMessage)
          .buildProcessorClient();
receiver.start();

Might be related to: https://github.com/Azure/azure-sdk-for-java/issues/37160
To Reproduce
Use the code above to create a ServiceBusProcessorClient, try to receive 1000 messages all with same sessionId. Processing will stop in between and InterruptedExceptions are thrown when complete() is called.
Expected behavior
Processing should work without interruptions and exceptions.
Setup (please complete the following information):

OS: Mac
IDE: IntelliJ
Library/Libraries: 'com.azure:azure-messaging-servicebus:7.14.4'
Java version: 17
App Server/Environment: Tomcat
Frameworks: Spring Boot

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

related https://github.com/Azure/azure-sdk-for-java/issues/37160

Seems to be related to https://github.com/Azure/azure-sdk-for-java/issues/31356 but the proposed fix in this ticket is not working. Smaller values for prefetchCount are working, but when a certain threshold is reached the application hangs. Also it seems that it is connected with maxConcurrentCalls as described in 31356. On my local computer prefetchCount=5 and maxConcurrentCalls=50 is working, but prefetchCount=5 and maxConcurrentCalls=100 causes the app to hang.

Hello @Maxeh, let me make sure I’ve the setup matching your environment where this problem happens, let’s focus on session-processor.
Processor Configurations [confirm this]
Prefetch == 20
maxConcurrentCalls == 1
maxConcurrentSessions == 1
sessionIdleTimeout == 50 seconds
Code [confirm this]
final ServiceBusProcessorClient receiver = new ServiceBusClientBuilder()
          .credential(namespace, new ClientSecretCredentialBuilder()
                .tenantId(tenantId)
                .clientId(clientId)
                .clientSecret(clientSecret)
                .build())
          .sessionProcessor()
          .disableAutoComplete()
          .receiveMode(ServiceBusReceiveMode.PEEK_LOCK)
          .topicName(topicName)
          .prefetchCount(20)
          .maxConcurrentCalls(1)
          .maxConcurrentSessions(1)
          .sessionIdleTimeout(Duration.ofSeconds(50))
          .subscriptionName(subscriptionName)
          .processError(this::processError)
          .processMessage(this::processMessage)
          .buildProcessorClient();
receiver.start();

Additional information needed from you
Please share the following information / clarifications -

provide the code showing logic in the processMessage, I'm trying to see how much time one invocation of processMessage takes before it calls complete.
provide the code shwoing logic in the processError
Is it sufficient to put the above code in a console app (inside static void main() function)?
Is there any code after after receiver.start() to block the main-thread, or its the last statement of the main() function?
Is creating 1000 messages all with the SAME session-id (so only one session) and receive using the above code sufficient to repro?
In addition to the above Processor configurations, code and number of messages, what exact value for "reactor.schedulers.defaultBoundedElasticSize" trigerred the ""hang"" (you can Print this system variable and check if you're choosing default)

Hi @anuchandy , yes your setup is correct.

There is just one log in this method and the complete() call on the message
There is just a log in there
We start inside SpringBoot context, but it should not make a difference I guess
Receiver is started inside SpringBoot ApplicationListener, start() is the last call, as the receiver is just started and will run on a different thread it should not make a difference right?
Yes it is sufficient, in my case exactly 12 messages are received, then the application hangs for sessionIdleTimeout, there a lots of those logs: {"az.sdk.message":"Starting lock renewal.","isSession":true,"lockToken":"1"}; afterwards we see some InterruptedExceptions and then again 12 messages are received and it hangs again for sessionIdleTimeout seconds and so on
In the case of sessionProcessor there is no difference in the config of defaultBoundedElasticSize, same behavior for 50 or 100

So to sum it up, prefetch is no usable with sessions. But as I described when NOT using sessions there are also bugs with concurrency and then there is even no real recovery when it hangs and the application needs to be restarted. But then again, when considering prefetchCount + maxConcurrentCalls <= reactor.schedulers.defaultBoundedElasticSize everything seems to work fine - this is how we currently configured it in our system, everything else is not working properly.

And here some information about the configuration without session (this is our current configuration):
final ServiceBusProcessorClient receiver = new ServiceBusClientBuilder()
          .credential(namespace, new ClientSecretCredentialBuilder()
                .tenantId(tenantId)
                .clientId(clientId)
                .clientSecret(clientSecret)
                .build())
          .processor()
          .disableAutoComplete()
          .receiveMode(ServiceBusReceiveMode.PEEK_LOCK)
          .topicName(topic)
          .subscriptionName(subscription)
          .prefetchCount(5)
          .maxConcurrentCalls(25)
          .processError(this::processError)
          .processMessage(this::processMessage)
          .buildProcessorClient();

When defaultBoundedElasticSize is 30 everything works fine. When defaultBoundedElasticSize is set to 25 then the application hangs at some point, but it might take over 1000 messages to process until it hangs, but then everything is broken and nothing is working anymore. I have done my tests with at least 5k messages in the queue.

Hello @Maxeh,
Thanks for the exact setup. I’m able to reproduce the problem you’re experiencing. As you identified, if the pool size is lower than the concurrency and prefetch, this issue of InterruptedException followed by hang happens. This is documented here  in troubleshooting guidelines.
The on-going work related to Prefetch reliability addresses this problem completely (so no more interruption/hang or need for pool sizing).  I’ve tested your use case using the working PR https://github.com/Azure/azure-sdk-for-java/pull/34854 and it works as expected. Additionally, this work should also addresses the slowness you experienced. I’ll keep this issue open so you / others can track 34854.

Thanks for the update, good to hear that you are already working on a fix!

Resolving this. Refer https://learn.microsoft.com/en-us/azure/developer/java/sdk/troubleshooting-messaging-service-bus-overview#upgrade-to-715x for details about version 7.15.x and how to opting-in new session rework.