Fixed FTPS

Made FTPS just start a secure connection immediately, rather than being upgraded later like FTPES Made the default port be 990 for ftps
Added a test that points to a public ftps server to ensure the handshake works

Let the record show there are still some issues.  For one, I don't see any indication that FTPS or FTPES use encrypted sockets for data transfer.
In my case, this means that the upload never get's a response since the server expects a SecuredSocket connection.  That's pretty fixable by making it use a secured socket in upload when needed.
However that gets into the perhaps unfixable issue of reusable session.  When I had tested a fix for upload using a secured socket the FTPS server I want to talk to responded saying that session had to be reused.  And I couldn't find a way to fix that in dart directly.  Or android.  Or when glancing at ios.  Though it should theoretically be possible on desktop.
Regarding android, there's a lot of code samples going around for extensions of the apache commons FTPClient that attempt to share session in the data socket when it's created - and they tend to use reflection.  Unfortunately, while that is probably possible using reflection in a normal java desktop app, those code samples aren't setup correctly considering the android implementations of sessioncontext classes (at least the modern ones) and even if you modify them to reflect on the properties that actually exist, android seems to "block" throwing a warning, and causing your code to say that those fields or methods on the class do not exist (even if they do).

thank you for this imrovements


Update Satellite Content

Update siphon interface and fix small issues from UW workshop.
There are a decent amount of text updates in here, as well as breaking up some of the more monolithic cells. We might revisit the material before the TTU workshop, but this certainly gets everything working again and easier to understand. Closes #214

I'm going to go ahead and merge this - we'll have another round when we update with 0.6.1 and Irma data.


netsurf: fix hydra build

Motivation for this change
Fix Hydra build fail: https://hydra.nixos.org/build/136136171
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111991 run on x86_64-linux 1

2 packages built:

netsurf-browser (netsurf.browser)
netsurf.libcss


Integrate ArchUnit tests

ArchUnit is a framework that allows to test app architecture. It looks like it's not suited for android, so some research, experimentation and potentially contribution to ArchUnit may be required to make it work with android project.
https://github.com/TNG/ArchUnit/issues/241
Few tests that we could write with ArchUnit

rule that protects boundaries between clean architecture layers - make sure that class defined in ...domain layer does not uses any classes from other layers (have imports from other layers).
rule that verifies that every androidx.lifecycle.ViewModel child class has ViewModel suffix
More Tests inspirations
...

ArchUnit User Guide

Konsist will be used instead https://docs.konsist.lemonappdev.com/getting-started/whatiskonsist


Hybrid Intune

Would it be possible to provide a note that Hybrid setup with Intune is EOL?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f6f96bb1-5fb9-6086-335f-e5069313d9cf
Version Independent ID: 4b3a73cd-07e6-9999-abae-26168f0dfa1e
Content: Choose a device management solution  - Configuration Manager
Content Source: sccm/core/plan-design/choose-a-device-management-solution.md
Product: configuration-manager
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Thanks @LeThijs for the suggestion. I just submitted a change to this article, which should publish tomorrow. I'm closing this issue.


Complete test when the last word typed correctly

Currently, the typing test will be completed after pressing the space bar on the last word of the test. It would be more convenient if the test was completed automatically when the last word was typed correctly. No additional space keypress is required.
I'd love to work on this issue.

This feature is present in v0.3.2 and onwards.


Flutter application restart instead of resuming after invoking method from native Android

I currently doing payment integration in Flutter using native Android. I manage to get the results from the payments side, I then proceed to return the result into my invoking flutter file.
Here is the passing of my code to my invoking flutter file.
Intent returnIntent = new Intent(getApplicationContext(), BackToMerchantActivity.class);
returnIntent.putExtra("resultTitle",resultPaymentMessage.getStrResultTitle());
returnIntent.putExtra("resultInfo",resultPaymentMessage.getStrResultInfo());
returnIntent.putExtra("resultExtra",resultPaymentMessage.getStrResultExtra());
                    startActivity(returnIntent);
                    finish();

Here is my invoking methods
class BackToMerchantActivity: FlutterActivity() {
private val CHANNEL = "my.com.ipay88.ipay88payment/backToMarchant"
private lateinit var _result: MethodChannel.Result
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
GeneratedPluginRegistrant.registerWith(this)
print("Enter Back To Merchant \n");
val resultTitle: String = intent.getStringExtra("resultTitle")  ?: ""
val resultInfo: String = intent.getStringExtra("resultInfo")    ?: ""
val resultExtra: String = intent.getStringExtra("resultExtra")  ?: ""

print("Result title : " + resultTitle +" \n");
print("Result Info  : " + resultInfo +" \n");
print("Result Extra : " + resultExtra +" \n");

MethodChannel(flutterView, CHANNEL).invokeMethod("my.com.ipay88.ipay88payment/backToMarchant", mapOf("resultTitle" to resultTitle, "resultInfo" to resultInfo, "resultExtra" to resultExtra));

}
}
The results is able to into invoking files as I am able to print it out. However the invoking flutter application from native Android will restart the whole Flutter application instead of resuming the application. Please help as I been stuck here for a week. Any tips would be really appreciated

Hi @AfendyWan
This issue doesn't seem to describe a bug or a feature request. Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow and if you need help with your code, please see https://www.reddit.com/r/flutterhelp/
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


Avoid encrypt buffer copy

Looks reasonable, as long as the sspi_buffers stays alive during the composed operation.

The sspi_buffers are kept inside the sspi_encrypt class which is alive during the entire lifetime of the stream class, so that shouldn't be a problem.
Part of this change was trying to avoid having to worry about these issues in the composed operation code which also makes it a lot easier to avoid unnecessary copies.
I'm quite happy with how the refactoring turned out to make the code a lot more readable IMO. Thanks for having a look before I start to use the same ideas for the decrypt, handshake and shutdown operations.


Getting type-checking errors on ssr

Hi, I'm currently getting these type-checking errors when using the ssr module:

However the code works good. What I am doing wrong? I'm using Deno v1.44.3
By the way this framework looks amazing. Thank you!!

Working on a fix right now! Sorry about that! Looks like an error given the first argument of a tag can be either props or the first child.

Fixed in 1.0.6. Let me know if it works now!

It works now, however I found something else:

The type-checking does not notify errors on this code, but it does not work. I suppose I have to spread the array however doing so a type-checking error appears:

Although the code works.

Hmm could you send the code in a
codeblock?

I want to do some digging on that

Actaully, there's a kinda simple way to fix the typechecking but not sure about the TypeError.
I experienced a little bit of that same error myself when writing it originally. Probably just need to Array.prototype.flat earlier.

Yes sure, this is the code:
import { render, tags } from "@fluid/core/server";

const { div, h1, h3, ul, li, a } = tags;

const links = [
    { href: "https://github.com/fluid-js/fluid", title: "GitHub" },
    { href: "https://jsr.io/@fluid/core", title: "JSR" },
    { href: "https://github.com/sant123", title: "Santi" },
];

function Hello() {
    return div(
        h1("Hello, Fluid!"),
        h3("Links"),
        ul(links.map((link) => li(a({ href: link.href }, link.title)))),
    );
}

const template = render(() => Hello());

console.log(template);

About the Array.prototype.flat, that may do the trick 😁

It now works!! @WorriedArrow thank you so much, if you can enable Github sponsors I can buy you a coffee ☕
Thanks again!!

I'm glad I could help! Donating means a lot to me and really helps me! I just added a sponsor button to the repo.
Thank you so much for your support! ❤


1.15 tunnel dsr cleanups

As discussed in eg. https://github.com/cilium/cilium/issues/25769 and https://github.com/cilium/cilium/pull/25553#pullrequestreview-1454817088.

/test

/test

/test

/test


Question - Any instability from left over services between tests?

Hi,
Great repository, just had a quick question. Have you found any instability when working with killing activities outside the standard workflow?
E.g. managing the app life cycle via @TestRule @Test approach used by the standard Android Test Runner?
Specifically: ActivityFinisher.finishOpenActivities(); // Required for testing App with multiple activities
Full details:
https://jabknowsnothing.wordpress.com/2015/11/05/activitytestrule-espressos-test-lifecycle/
Cheers!

The @TestRule has worked just fine in all projects I used, but to be honest, I still use the ActivityInstrumentationTestCase2 in my daily work project, because there was no big incentive to change it, it just works and we never modify that code.
The need for ActivityFinisher comes from larger projects, where you need to test interactions between screens (mostly flows across Activities). This kind of test scenarios can leave a trail of opened activities depending on which mode new activities are launched. Since this should not be a dependency for starting a new test scenario with a clean state, ActivityFinisher ensures this requirement is fulfilled.


Can't remove contexts

When I created the job, I didn't set any context, so "it" created the default one.
I then wanted to change that (because I was going to add more checks) to linting. That new context was added to the existing one. Meaning I then had two contexts - the old default one and the new linting one.
I then tested to add a Set GitHub Commit Status (universal). That seems to also have added a context!
So now I have THREE test context. And even though I've removed all of them (from the job, as well as restarted Jenkins AND opened a new PR), all three are still there. All linking to different job numbers in the same job.

I even tried to delete the job AND the webhook, but the context is still there! Might this be a GitHub issue?

This is a limitation of the API. There does not appear to be a way to remove a context. You can overwrite an existing one but not remove it.
https://developer.github.com/v3/repos/statuses/


bug: ion-safe-area-top excesive Edge

Bug Report
Ionic version:
[x] 4.5
Current behavior:

Expected behavior:

Related code:
https://pastebin.com/6TXrkFQf
Other information:
referenced error
https://github.com/ionic-team/ionic/issues/15502
Ionic info:
Ionic:

   Ionic CLI                     : 5.0.2 (C:\Users\Administrador\AppData\Roaming\npm\node_modules\ionic)
   Ionic Framework               : @ionic/angular 4.5.0
   @angular-devkit/build-angular : 0.13.9
   @angular-devkit/schematics    : 7.3.9
   @angular/cli                  : 7.3.9
   @ionic/angular-toolkit        : 1.5.1

Cordova:

   Cordova CLI       : 9.0.0<EMAIL_ADDRESS>   Cordova Platforms : browser 6.0.0
   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.1.3, cordova-plugin-ionic-webview 4.1.0, (and 5 other plugins)

Utility:

   cordova-res : not installed
   native-run  : 0.2.5

System:

   NodeJS : v10.15.3 (C:\Program Files\nodejs\node.exe)
   npm    : 6.4.1
   OS     : Windows 10

Hi there,
Thanks for the issue. Which device/browser are you testing this on?
Thanks!

Windows 10 V 10.0.17763 compilation 17763, microsoft edge 44.17763.1.0

Thanks for the issue. I am going to close this as we recently released Ionic Framework v5 which dropped support for IE11 and older versions of Edge. Moving forward, Ionic Framework v4 will only receive critical security patches.


Clarify the difference between Ranges and Pipes

In .Net (c#) the difference is clear. They have IEnumerable and IObservable. Roughly speaking
Ranges are IEnumerable ( interactive )
and
Pipes are IObservable ( reactive )
The basic interfaces of each type have the same shape with the  difference described succinctly.
IEnumerable allows you to write queries over space
IObservable allows you to write queries over time
or another way to see it is that
IEnumerable is  pull based. The consumer asks for the next value
IObservable is push based. The consumer waits for the next value
That being said there exists a library call ReactiveExtensions that has a C++ port that follows the same reactive model that Pipes uses. https://github.com/ReactiveX/RxCpp Maybe you can get some inspiration from that code. There are loads of operators for dealing with time that space based query engines such as rangev3 don't need to worry about. async and threads and scheduling etc.
Regards
Brad

Thanks Brad for presenting the design different this way, and for the references. I've updated the documentation to include this insight about the design.

Can someone explain the difference between RxCpp and Pipes, or what factors may affect the decision to use one instead of the other? For me it looks like both Pipes and RxCpp cover the same front.


Org-ify the docs

Like exercism/xlisp#101

@canweriotnow feel free to dive in here too, if you're so inclined.


feat: add run configuration

Since allero has 2 main flows, it will be easier to just have them both available, and switch between them instead of changing the args manually.
Signed-off-by: dolby360<EMAIL_ADDRESS>

@dolby360 thanks :)
merging this with a refactor: prefix, I think it suits better since this is not a feature, but a QOL imrovement


Ask the user to verify their database password

When a user is configuring the database, as a second time for the database password
https://bugzilla.redhat.com/show_bug.cgi?id=1134978
If the passwords do not match, it will prompt again.
/cc @jrafanie

The commit and title/description of this PR could be something like "Ask the user to verify their database password".  Is this doing anything else?

@jrafanie thanks - nothing else

Ok @kbrock looks good, I don't know if we want to loop until they match the password they entered previously...
Can you just reword the first comment of this PR as there's a typo and I think "ask the user to verify their database password" is clearer:
When a user is configuring the database, as a second time for the database password
And the commit message:
double enter db password in console
Clearer: "Ask the user to verify their database password"


Add 'showIcons' input to ToggleComponent

Summary
Enhancement: Add showIcons input to ToggleComponent  in order to display the optional icons.

Checklist

[x] *Added unit tests
[x] *Added a code reviewer
[x] Added changes to /projects/swimlane/ngx-ui/CHANGELOG.md under HEAD (Unreleased)
[x] Updated the demo page
[x] Included screenshots of visual changes

*required

This looks very good. Should we make this the default with an option to disable?

Yes, good point. It would be nice to have it as default.

This looks very good. Should we make this the default with an option to disable?

Yes, good point. It would be nice to have it as default.

@cthompson-Swim Should this be the default implementaion across the platform?

@cthompson-Swim Should this be the default implementaion across the platform?


Add control plane components healthy check when computing ControlPlaneReady condition

Issue #, if available:
Description of changes:
While testing upgrades, we ran into an edge case where all the old control plane machines were removed and all the new control plane machines and nodes were all rolled out. However, an old node still remained in the cluster, which resulted in the   ControlPlaneReady condition being marked as ready when it was actually not. We use the status on the KubeadmControlPlane to compute this and determine the control plane is ready when there are no outdated CAPI machines, number of replicas  match the expected and all are ready.
Example KCP status in this case
status:
  conditions:
  - lastTransitionTime: "2023-07-11T22:24:33Z"
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-07-11T22:01:55Z"
    status: "True"
    type: Available
  - lastTransitionTime: "2023-07-11T22:01:54Z"
    status: "True"
    type: CertificatesAvailable
  - lastTransitionTime: "2023-07-11T22:24:01Z"
    message: Control plane node eksa-test-405f3e3-tzcfc does not have a corresponding
      machine
    reason: ControlPlaneComponentsUnhealthy
    severity: Error
    status: "False"
    type: ControlPlaneComponentsHealthy
  - lastTransitionTime: "2023-07-11T22:01:55Z"
    status: "True"
    type: EtcdClusterHealthy
  - lastTransitionTime: "2023-07-11T22:01:55Z"
    status: "True"
    type: MachinesCreated
  - lastTransitionTime: "2023-07-11T22:24:33Z"
    status: "True"
    type: MachinesReady
  - lastTransitionTime: "2023-07-11T22:24:33Z"
    status: "True"
    type: MachinesSpecUpToDate
  - lastTransitionTime: "2023-07-11T22:24:33Z"
    status: "True"

In this edge case, an old node still existed in the cluster, but all the machines had already been deleted. This PR adds a control plane components healthy check when determining the status of the ControlPlaneReady condition.
Testing (if applicable):

Unit testing
Manual testing

Documentation added/planned (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/test eks-anywhere-presubmit

/test eks-anywhere-presubmit

/approve


Added newcomers-alert workflow

Signed-off-by: Jerod Culpepper<EMAIL_ADDRESS>Description
Added newcomer-alert workflow to the meshery repo.
Notes for Reviewers
Signed commits

[X] Yes, I signed my commits.

@cpepper96
The only change would be a sign-off to your git commits which you missed.
Have a look at this doc to understand more about that https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md

Thank you for the PR @cpepper96, just missing the sign-off😅, The bot has suggested some ways to get rid of that.

thanks for your patience y'all. I think I finally got it right!

Whoo-hoo, @cpepper96! 🎈


Tool to convert an existing Hive table as a Hoodie table

Omkar @ovj  will be taking this up.

General steps

Read existing data and upsert into target hoodie dataset (if the source parquet files does not have duplicates - then this copy can be avoided - may be have a flag for this)
a. If no data copy - then rename the files, read and compute write back files with bloom filters
b. If it involves data copy - then re-use much of the upsert path
Create a backfill commit.

Reducing the scope of this bug to only handle --source-type == "hdfs". Created another issue to handle "hive" source-type. https://github.com/uber/hoodie/issues/94.

#94 covers remaining work to be done.


Fixed a Potential Stack Overflow Error in findPurl

Description of the PR

Fixed the potential stack overflow error for the function findPurl in pkg/ingestor/parser/csaf/parser_csaf.go
Included unit tests for findPurl
Used a visted map to solve the stack overflow, and didn't use Dynamic Programing because there was no need to go over the nodes again.

PR Checklist

[ ] All commits have a Developer Certificate of Origin (DCO) -- they are generated using -s flag to git commit.
[ ] All new changes are covered by tests
[ ] If GraphQL schema is changed, make generate has been run
[ ] If collectsub protobuf has been changed, make proto has been run
[ ] All CI checks are passing (tests and formatting)
[ ] All dependent PRs have already been merged

@lumjjb A friendly ping!

@nathannaveen rebase will auto-merge. Thanks!

Closing this PR in favor of https://github.com/guacsec/guac/pull/1194 because the conflicts in this PR are extremely annoying to deal with.


add UnauthorizedException handling to http adapter

Pull Request Test Coverage Report for Build 52021

0 of 0   changed or added relevant lines in 0 files are covered.
7 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.04%) to 75.544%

Files with Coverage Reduction
New Missed Lines
%

/libraries/integration/Microsoft.Bot.Builder.Integration.AspNet.Core/BotFrameworkHttpAdapter.cs
7
58.33%

Totals

Change from base Build 52017:
-0.04%

Covered Lines:
4306

Relevant Lines:
5700

💛  - Coveralls


Overlapping encounters

It's possible for two encounters on a patient's record to overlap. Seems like what's happening is a wellness encounter starts, then a disease module starts another encounter during the same timestep, then the subsequent modules that run are considered to be "in a wellness encounter" but when it calls getCurrentEncounter it gets that second one.
In terms of timestamps everything that occurs within the 2 modules generally makes sense, but if you group by encounter ID you get weird things like a patient who is diagnosed with hypertension but they don't have a blood pressure reading. (because the bp reading happens in wellness_encounters.json and the diagnosis happens in hypertension.json)

I hesitate to create a new issue, because it is not exactly the same problem, but it results from the same causes...
Some encounters are not properly closed, or are seemingly randomly closed years after they are started when the GMF suggests the module should be closed within hours or days. I have found this occurs with submodules -- the Anemia submodule is a frequent culprit.
My suspicion is that this is because the "current encounter" can change or become null -- which may be related to modules creating overlapping encounters.
The context of a module (e.g., the module, the current provider, the current encounter, the prior history -- to include submodules) is in need of a reorganization. This is also vaguely related to #1003.

This was fixed with #1245


fix: Dataset column names are no longer editable

SUMMARY

The name field in the column field of the edit dataset modal should not be editable.
Bonus: I also vertically-aligned the rows in the calculated columns tab
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

BEFORE
Columns tab:

Calculated columns tab:

AFTER
Columns tab:

Calculated columns tab:

TESTING INSTRUCTIONS

Open the edit dataset modal and observe the changes on the Columns tab and the Calculated Columns tab

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[x] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

/testenv up


Full node initial

Features:

Full node impl which follows a quorum of authorities
All Gateway-like read-RPC endpoints which read from the full node
Begin separating out genesis logic to allow for multiple uses

How to use:
Run ./sui genesis -f to provision authorities and full node stores
Run ./sui start & which will launch authorities
Run ./full_node & to start a full node listening to a quorum of authorities.
By default, the full node is accessible on http://<IP_ADDRESS>:5002, but this can be changed with CLI params
Execute commands via the wallet for examaple to create transactions.
Verify that the full node is working by hitting any of the read-only RPC endpoints
"sui_getObjectTypedInfo", "sui_getTotalTransactionNumber", "sui_getOwnedObjects", "sui_getObjectInfoRaw" "sui_getRecentTransactions", "sui_getTransactionsInRange", "rpc.discover"
Example
Run genesis and start validators
$ ./sui genesis -f
2022-05-15T07:50:46.140491Z  INFO sui::sui_commands: Creating 4 new authorities...
.....
2022-05-15T07:50:46.444038Z  INFO sui::sui_commands: Provisioning full node...

$  ./sui start &
[1] 2705
mbp> 2022-05-15T07:50:59.497913Z  INFO telemetry_subscribers: Jaeger tracing initialized
2022-05-15T07:50:59.513003Z  INFO sui::sui_commands: Starting network with 4 authorities
2022-05-15T07:50:59.550040Z  INFO config: Header size set to 1000 B
2022-05-15T07:50:59.550049Z  INFO config: Max header delay set to 100 ms

Start full node
$  ./full_node &
[2] 2713
$  2022-05-15T07:51:04.401835Z  INFO full_node: Node config file path: "/Users/ade/.sui/sui_config/network.conf"
2022-05-15T07:51:04.401905Z  INFO full_node: AccessControl { allowed_hosts: Any, allowed_origins: None, allowed_headers: Any, continue_on_invalid_cors: false }
2022-05-15T07:51:04.433304Z  INFO sui_core::full_node::follower: Follower coordinator starting. Target stake 3
2022-05-15T07:51:04.433376Z  INFO sui_core::full_node::follower: Full node downlader started...
2022-05-15T07:51:04.433387Z  INFO sui_core::full_node::follower: Will follow authority k#2ed60ada620b07c05fe49dc4e71d9b78831afe380d990ff7453d843ae5c0de31
2022-05-15T07:51:04.433422Z  INFO sui_core::full_node::follower: Will follow authority k#9511ddd7e196e3884a73b15806ac0c1ed9b6c55c611aff90a9ced9137495f96a
2022-05-15T07:51:04.433426Z  INFO sui_core::full_node::follower: Will follow authority k#9234690fd4637d97917b97d15a23c2aa6ed16c8662a5795f931af124fdc89874
2022-05-15T07:51:04.433428Z  INFO sui_core::full_node::follower: Spawned 3 follower tasks to achieve 3 stake out of 4 total
2022-05-15T07:51:04.433441Z  INFO sui::sui_full_node: Started full node
2022-05-15T07:51:04.433464Z  INFO full_node: Available JSON-RPC methods : ["sui_moveCall", "sui_getTransaction", "sui_getObjectTypedInfo", "sui_getTotalTransactionNumber", "sui_getOwnedObjects", "sui_getObjectInfoRaw", "sui_transferCoin", "sui_executeTransaction", "sui_mergeCoins", "sui_getRecentTransactions", "sui_getTransactionsInRange", "rpc.discover", "sui_splitCoin", "sui_publish", "sui_syncAccountState"]
2022-05-15T07:51:04.433473Z  INFO full_node: Sui RPC Gateway listening on local_addr:<IP_ADDRESS>:5002
2022-05-15T07:51:04.433506Z  INFO sui_core::full_node::follower: Spawn follower for k#9511ddd7e196e3884a73b15806ac0c1ed9b6c55c611aff90a9ced9137495f96a
2022-05-15T07:51:04.433511Z  INFO sui_core::full_node::follower: Spawn follower for k#9234690fd4637d97917b97d15a23c2aa6ed16c8662a5795f931af124fdc89874
2022-05-15T07:51:04.433522Z  INFO sui_core::full_node::follower: Follower listener started for k#9511ddd7e196e3884a73b15806ac0c1ed9b6c55c611aff90a9ced9137495f96a
2022-05-15T07:51:04.433523Z  INFO sui_core::full_node::follower: Follower listener started for k#9234690fd4637d97917b97d15a23c2aa6ed16c8662a5795f931af124fdc89874
2022-05-15T07:51:04.433523Z  INFO sui_core::full_node::follower: Spawn follower for k#2ed60ada620b07c05fe49dc4e71d9b78831afe380d990ff7453d843ae5c0de31
2022-05-15T07:51:04.433527Z  INFO sui_core::full_node::follower: Follower listener started for k#2ed60ada620b07c05fe49dc4e71d9b78831afe380d990ff7453d843ae5c0de31
2022-05-15T07:51:04.435527Z  INFO sui_core::full_node::follower: Follower batch listener for authority: k#9234690fd4637d97917b97d15a23c2aa6ed16c8662a5795f931af124fdc89874 starting at sequence: 0, for length: 100000.
2022-05-15T07:51:04.435568Z  INFO sui_core::full_node::follower: Follower batch listener for authority: k#9511ddd7e196e3884a73b15806ac0c1ed9b6c55c611aff90a9ced9137495f96a starting at sequence: 0, for length: 100000.
2022-05-15T07:51:04.439761Z  INFO sui_core::full_node::follower: Follower batch listener for authority: k#2ed60ada620b07c05fe49dc4e71d9b78831afe380d990ff7453d843ae5c0de31 starting at sequence: 0, for length: 100000.
mbp>

Execute some TXes via wallet
sui>-$ split-coin --coin-id 0x0f4ce88f519914643ede139b6783ed5386decec2 --amounts 400 --gas-budget 4000
----- Certificate ----
Transaction Hash: DwkxqwkxDD4u850Cl3gsTtwguTKZypwLQD4uVzFdA7g=

Ask full node for recent transactions (sui_getRecentTransactions)
$  export SUI_FULL_NODE_RPC_HOST=http://<IP_ADDRESS>:5002

$  curl --location --request POST $SUI_FULL_NODE_RPC_HOST \
--header 'Content-Type: application/json' \
--data-raw '{ "jsonrpc":"2.0", "method":"sui_getRecentTransactions", "params":[30], "id":1}'
{"jsonrpc":"2.0","result":[[0,"DwkxqwkxDD4u850Cl3gsTtwguTKZypwLQD4uVzFdA7g="],[1,"LpQsqL3yc3Tgqt5beE1ce4KMChwmDyXO2JEZUqiPCdE="],[2,"qiMnL1542rhaEwAkOLs0CnyMC21e3b/sN4X0dDF/K3c="],[3,"W0JzJTNGg3newK02pw5SBxaUpoJUN7SJUWw3bg0jnqA="],[4,"EeJW28ovwk0Hz1tM7OiUXCzzmMUFzDRuhixSq8cPT9M="]],"id":1}

Check the TX (sui_getTransaction)
$  curl --location --request POST $SUI_FULL_NODE_RPC_HOST \
--header 'Content-Type: application/json' \
--data-raw '{ "jsonrpc":"2.0", "method":"sui_getTransaction", "params":["DwkxqwkxDD4u850Cl3gsTtwguTKZypwLQD4uVzFdA7g="], "id":1}'
{"jsonrpc":"2.0","result":{"certificate":{"data":{"kind":{"Single":{"Call":{"package":["0x0000000000000000000000000000000000000002",1,"8xrIk23UnAReKXZYzKY2PDLsALkXcA5a3xrPldgDlwY="],"module":"Coin","function":"split_vec","type_arguments":[{"struct":{"address":"0000000000000000000000000000000000000002","module":"SUI","name":"SUI","type_args":[]}}],"arguments":[{"ImmOrOwnedObject":["0x0f4ce88f519914643ede139b6783ed5386decec2",0,"zpzmVTAzQfN2excgujzNPBZpwFPMZoo6QBilT7C2Q18="]},{"Pure":[1,144,1,0,0,0,0,0,0]}]}}},"sender":"0x06fd01beae523f3cead97aae5efd2381fda41867","gas_payment":["0x07d7670680af61df36a9e0618dc3d593473d5615",0,"X/bp+1Siw84gS5W6wyMslo9yms2HkFBPra4BETwTs3c="],"gas_budget":4000},"tx_signature":"3PPNypXE6aGk9tTg1IJN0T7xuShkAroel0w6iug+7+px5Gmu32r96xsGnaP4zBLjZWR3sYreDBK83NIOse2YA1Om8sQZhNRLZaFeuYlUPQgU+g+7/sdSY9slRWBQWLq4","auth_sign_info":{"epoch":0,"signatures":[["LtYK2mILB8Bf5J3E5x2beIMa/jgNmQ/3RT2EOuXA3jE=","sUWc0rfHHWdPValrHC/ezchWOdNmEGa5vykTuHaNVtI5fcy0FIXDi5rFtvZR/sVmPeTL//6NuJkUWqVdxGbYBw=="],["kjRpD9RjfZeRe5fRWiPCqm7RbIZipXlfkxrxJP3ImHQ=","ijmgl/SGc8H4enqFaxxHxhUl99cXa2rqpSonah6y9I2DcQ8BXEyrNVuJ9QwUw/MqTneceVod2fTKEP37qap9Cw=="],["g5XR6Dgufm4YfCeK+ZlMtjwLj16E4oPGcIm2aE4tctg=","msfQqVknxzTp9Y1oU/eN04cuXR00ypOs3dzQXKfutFMJH25Qu/qZ6BEPEWneyW57KOzp4SF4DSYkxGXuWPF6BQ=="]]}},"effects":{"status":{"Success":{"gas_cost":{"computation_cost":401,"storage_cost":45,"storage_rebate":0}}},"shared_objects":[],"transaction_digest":"DwkxqwkxDD4u850Cl3gsTtwguTKZypwLQD4uVzFdA7g=","created":[[["0x5167b4793f4b14565401092d7d87f560905aeb73",1,"ULwCPpTc4Juy7KlN6FAtbdt7jsV7OJLnkkqjHaSq2VU="],{"AddressOwner":"0x06fd01beae523f3cead97aae5efd2381fda41867"}]],"mutated":[[["0x07d7670680af61df36a9e0618dc3d593473d5615",1,"3wvh9q7MBgvS6bRq/FIau9hq0CVGdjHw5b1xxxrJYzY="],{"AddressOwner":"0x06fd01beae523f3cead97aae5efd2381fda41867"}],[["0x0f4ce88f519914643ede139b6783ed5386decec2",1,"Q8BIYJ/cFSMJPNFusbFM9Cj8ZcRgJWUnXePBF/+mbh4="],{"AddressOwner":"0x06fd01beae523f3cead97aae5efd2381fda41867"}]],"unwrapped":[],"deleted":[],"wrapped":[],"gas_object":[["0x07d7670680af61df36a9e0618dc3d593473d5615",1,"3wvh9q7MBgvS6bRq/FIau9hq0CVGdjHw5b1xxxrJYzY="],{"AddressOwner":"0x06fd01beae523f3cead97aae5efd2381fda41867"}],"events":[],"dependencies":[]}},"id":1}

Get an object info (sui_getObjectInfoRaw)
$ curl --location --request POST $SUI_FULL_NODE_RPC_HOST \
--header 'Content-Type: application/json' \
--data-raw '{ "jsonrpc":"2.0", "method":"sui_getObjectInfoRaw", "params":["0x0f4ce88f519914643ede139b6783ed5386decec2"], "id":1}'
{"jsonrpc":"2.0","result":{"status":"Exists","details":[["0x0f4ce88f519914643ede139b6783ed5386decec2",1,"Q8BIYJ/cFSMJPNFusbFM9Cj8ZcRgJWUnXePBF/+mbh4="],{"data":{"Move":{"type_":{"address":"0000000000000000000000000000000000000002","module":"Coin","name":"Coin","type_args":[{"struct":{"address":"0000000000000000000000000000000000000002","module":"SUI","name":"SUI","type_args":[]}}]},"contents":"D0zoj1GZFGQ+3hObZ4PtU4bezsIBAAAAAAAAABCFAQAAAAAA"}},"owner":{"AddressOwner":"0x06fd01beae523f3cead97aae5efd2381fda41867"},"previous_transaction":"DwkxqwkxDD4u850Cl3gsTtwguTKZypwLQD4uVzFdA7g=","storage_rebate":15},{"WithTypes":{"type_":{"address":"0000000000000000000000000000000000000002","module":"Coin","name":"Coin","type_args":[{"struct":{"address":"0000000000000000000000000000000000000002","module":"SUI","name":"SUI","type_args":[]}}]},"fields":[{"name":"id","layout":{"struct":{"WithTypes":{"type_":{"address":"0000000000000000000000000000000000000002","module":"ID","name":"VersionedID","type_args":[]},"fields":[{"name":"id","layout":{"struct":{"WithTypes":{"type_":{"address":"0000000000000000000000000000000000000002","module":"ID","name":"UniqueID","type_args":[]},"fields":[{"name":"id","layout":{"struct":{"WithTypes":{"type_":{"address":"0000000000000000000000000000000000000002","module":"ID","name":"ID","type_args":[]},"fields":[{"name":"bytes","layout":"address"}]}}}}]}}}},{"name":"version","layout":"u64"}]}}}},{"name":"balance","layout":{"struct":{"WithTypes":{"type_":{"address":"0000000000000000000000000000000000000002","module":"Balance","name":"Balance","type_args":[{"struct":{"address":"0000000000000000000000000000000000000002","module":"SUI","name":"SUI","type_args":[]}}]},"fields":[{"name":"value","layout":"u64"}]}}}}]}}]},"id":1}

.. and so on
TODO:

More comments, tests, tracing, metrics
Handle byzantine authorities
Enable running full nodes remotely from genesis config
Docs
Make more performant. If the downloads and processing steps are slower than the throughput of the network, the full node will miss objects. Can use local execution instead
better consolidation of code. I tried not to touch too many files in the core logic but ideally we would like to share code between gateway, validator and full node better
Once lock service merges, remove the USE_LOCKS flag
Leverage other incoming storage improvements.
Full support local exec

Went over these comments in a huddle with @oxade but writing down my questions for others here:

This duplicates a lot of the logic from https://github.com/MystenLabs/sui/blob/main/sui_core/src/authority_active/gossip/mod.rs#L166 - but with some important differences. Specifically the gossip sync code appears to only ever processes certificates (and their effects), and uses transactions only as a way to know which certificates to pull. This code otoh seems to directly pull transactions into its state. Is there a deep reason to do this? My intuition is that a fullnode should only examine certs, not txes. Among other things, tracking txes directly it seems to add quite a bit of complication to this PR because of the desire to suppress lock storage.

I was initially quite confused why full nodes are doing anything resembling genesis. My thinking was:

If the validators we are following haven't pruned any txes (they currently never prune), then we will get all state starting from genesis anyway as a follower, so the genesis objects aren't necessary.
But if they have pruned txes, then we don't have a full history from the genesis objects so they can't be used anyway.
On reading the code further, I see my error: genesis objects aren't created via a transaction, they are simply inserted into the db.
So my first objection is invalid, but the second one still seems to hold. So it makes sense to have this in for an MVP but it seems to me like all of this will have to be replaced once we have checkpoints, which will be soon as i understand it.

This is beyond the scope of this change, but it might simplify many things if all objects in the db are always created by a tx/cert. (I realize this might not be possible given gateway syncing.)

This duplicates a lot of the logic from https://github.com/MystenLabs/sui/blob/main/sui_core/src/authority_active/gossip/mod.rs#L166 - but with some important differences. Specifically the gossip sync code appears to only ever processes certificates (and their effects), and uses transactions only as a way to know which certificates to pull. This code otoh seems to directly pull transactions into its state. Is there a deep reason to do this? My intuition is that a fullnode should only examine certs, not txes. Among other things, tracking txes directly it seems to add quite a bit of complication to this PR because of the desire to suppress lock storage.

This is intentional because we're separating out the follower logic and want to maintain agility without modifying the core. We also expect it to be used by multiple services, not just active authority gossip and full nodes. From chats with Lu and others, events service (if its a separate service) will want to run a follower too. Once we clean up the follower logic, we will consolidate it in to the gossip too.
Storing of certs and txes was just done for completeness. On the issue of locks, this will be irrelevant once locks service gets merged.

I was initially quite confused why full nodes are doing anything resembling genesis. My thinking was:

If the validators we are following haven't pruned any txes (they currently never prune), then we will get all state starting from genesis anyway as a follower, so the genesis objects aren't necessary.
But if they have pruned txes, then we don't have a full history from the genesis objects so they can't be used anyway.
On reading the code further, I see my error: genesis objects aren't created via a transaction, they are simply inserted into the db.
So my first objection is invalid, but the second one still seems to hold. So it makes sense to have this in for an MVP but it seems to me like all of this will have to be replaced once we have checkpoints, which will be soon as i understand it.

This is beyond the scope of this change, but it might simplify many things if all objects in the db are always created by a tx/cert. (I realize this might not be possible given gateway syncing.)

@bmwill  can give more context on why we want genesis for full nodes

Yes I think that Fullnodes should only ever operate on Certs signed by a quorum and not unsigned txns.
Re genesis: Yes I really think we want genesis to eventually just be different type of transaction. https://github.com/MystenLabs/sui/pull/1972 gets us a small step closer to that reality but we still have a ways to go.

Yes I think that Fullnodes should only ever operate on Certs signed by a quorum and not unsigned txns.

Sorry I forgot to address this comment. I will fix this

This is intentional because we're separating out the follower logic and want to maintain agility without modifying the core. We also expect it to be used by multiple services, not just active authority gossip and full nodes

From my reading of the code there would be very minimal modifications to the existing follower code. But I could be wrong.

Copy from our chat: I think we should just get rid of the object downloading and go with local execution. I don't think there is any blocker on doing that today in fullnode (if any, I would love to help fix).

Copy from our chat: I think we should just get rid of the object downloading and go with local execution. I don't think there is any blocker on doing that today in fullnode (if any, I would love to help fix).
Yes I will add this along with next PR.

There is no existing follower system. There is a follower code embedded in gossip logic. What code are you talking about?

The code that I linked to earlier. Its "embedded" in the gossip system but it seems quite cleanly separable, as the purpose of that code is to follow all activity from one validator. The gossip system will presumably use that as a building block to establish a gossip network. Please see https://mysten-labs.slack.com/archives/C03AW7J5Q5U/p1652472739919149 wherein @gdanezis confirms that this was his intent for that code.

Copy from our chat: I think we should just get rid of the object downloading and go with local execution. I don't think there is any blocker on doing that today in fullnode (if any, I would love to help fix).

Yes I will add this along with next PR.

Spoke to @mystenmark about this and it's a non-issue. There was some miscommunication

I'm going to let @mystenmark or @velvia comment on the storage changes. My biggest hope is that we can somehow consolidate the authority and fullnode states into a single type in order to avoid the code duplication. Its going to be very hard to keep that logic in sync i think

They can be consolidated but not without changing authority logic, which I'm hesitant to do right now.
I also did not want to worry about consolidation since locks service inclusion will require refactoring anyways

@velvia @mystenmark
Any thoughts on storage? Hoping to consolidate once the locks service goes in

I'm going to let @mystenmark or @velvia comment on the storage changes. My biggest hope is that we can somehow consolidate the authority and fullnode states into a single type in order to avoid the code duplication. Its going to be very hard to keep that logic in sync i think

@bmwill  what code duplication are you talking about though?
They use pretty much same store and methods right now. Locks are only difference

Any thoughts on storage? Hoping to consolidate once the locks service goes in

I'm glad to see you're using ReplicaStore.
I'm not sure what the dependency on lock service is.
Overall I don't think storage presents any blockers at this time. There will probably be scalability changes (e.g. not storing full object history in rocksdb) but the existing storage code should be perfectly capable of hosting a full node MVP.

@bmwill what code duplication are you talking about though?

get_total_transaction_number, get_transactions_in_range, get_recent_transactions and get_transaction are all duplicated from gateway_state.rs

get_total_transaction_number, get_transactions_in_range, get_recent_transactions and get_transaction are all duplicated from gateway_state.rs
Yes I agree these can be consolidated once if everything in the MVP checks out. I'm hesitant to touch/break core logic particularly while we have devnet running without testing things in the wild.

I'm not sure what the dependency on lock service is.

There's never been a dependency. But it will make code storage much easier to share.

I was specifically talking about the fullnode state functions being duplicates of the authority state ones

I missed the comment about how you plan to consolidate in a future pr

I was specifically talking about the fullnode state functions being duplicates of the authority state ones

Thank you. Be rest assured, all of this will be simplified. I really just want us to have a jumping off point that we can iterate on rapidly right now.

Next steps
https://github.com/MystenLabs/sui/issues/1998
https://github.com/MystenLabs/sui/issues/1997


The word parser should mention underscore _

https://github.com/petitparser/dart-petitparser/blob/bada300011c0257e8789b0ec00cc1e5a16603340/petitparser/lib/src/parser/character/word.dart#L6
The word parser (WordCharPredicate ) doesn't mention in the documentation or message that the underscore character is acceptable.
Reading the code we can see character 95 as one in the acceptable range.
Note that the common definition of a word only can start with a letter or a _, can't start with a digit. Need to document that too.

Thank you, thanks for reporting. Also the tutorial is misleading by implying that letter() | digit() is the same as word().


Passing inline UDF code with query request

Hello, firstly thanks for the excellent API and documentation. 👍
However, with the latest version of Google.Apis.Bigquery.v2 package: v<IP_ADDRESS>9, I have stumbled upon a (seemingly) undocumented scenario.
We have a UDF (code with bigquery.defineFunction expression and the associated JS function) in a local .js file, which is read once per application run.
Given the following code, I am trying to pass the JS code read from file as InlineCode to query API:
// somewhere earlier; at the service activation time:
UserDefinedFunctionResource _udf = new UserDefinedFunctionResource
{
    InlineCode = File.ReadAllText("~/Services/BigQuery/Blah_UDF.js")
};

private async Task<IList<TableRow>> ExecuteQueryAsync(string querySql)
{
    var request = new JobsResource.QueryRequest(
                            _bigqueryServiceClient,
                            new QueryRequest { Query = querySql },
                            _projectId);

    QueryResponse query = await request.ExecuteAsync();
    GetQueryResultsResponse queryResult = await _bigqueryServiceClient.Jobs.GetQueryResults(
                                                    _projectId,
                                                    query.JobReference.JobId
                                                ).ExecuteAsync();

    return queryResult.Rows;
}

// example call-site:
var tuples = await ExecuteQueryAsync($@"
  SELECT TAB1, TAB2 FROM myFunc
  (
    SELECT random_cell1, cell2, timestamp
    FROM x_table
    WHERE userId = '${someUserId}'
  )");

How do I associate UserDefinedFunctionResource object _udf with that of JobsResource.QueryRequest or QueryRequest?
The only relationship in object graph I have found is both UserDefinedFunctionResource and QueryRequest implement IDirectResponseSchema interface, but JobsResource.QueryRequest only accepts concrete implementation of QueryRequest.

While it so happens that I'm currently looking at BigQuery, I haven't done anything with UDFs, and this looks like it's a language-agnostic question... I'd suggest asking on Stack Overflow (using the google-cloud-platform and google-bigquery tags).
If I've misunderstood and this is really a matter of translating the API documentation into "what would the C# look like", do let me know and I'll see what I can work out. (I think it may be a matter of creating a job for the query rather than using QueryRequest directly.)

Thanks for the tip @jskeet! Creating a job worked for me:
var job = new Job
{
    Configuration = new JobConfiguration
    {
        Query = new JobConfigurationQuery
        {
            Query = querySql,
            UserDefinedFunctionResources = new List<UserDefinedFunctionResource> { _udf  }
        }
    }
};

JobsResource.InsertRequest insertRequest = _bigqueryServiceClient.Jobs.Insert(job, _projectId);
Job requestJob = await insertRequest.ExecuteAsync();

GetQueryResultsResponse queryResult = await _bigqueryServiceClient.Jobs.GetQueryResults(
                                                _projectId,
                                                requestJob.JobReference.JobId
                                            ).ExecuteAsync()

// queryResult.Rows has it all,
// if it is unexpectedly null, we need to either retry ..GetQueryResults(..).ExecuteAsync()
// or use destination table with allowLargeResults = true

Fiddling with object graphs, intellisense and object explorer helped. Perhaps we can add a basic UDF example in docs? :)


[0.23.0-rc1]Unable to resolve modules AssestRegistry when require image inside node_modules

I have code like
<Image source={require('react-native-router-flux/src/back_chevron.png')} style={{height: 100, width: 100}}/>  in my app and it works fine in 0.22.2
but in 0.23.0-rc1 system throws "Unable to resolve modules AssestRegistry [path to back_chevron.png" in IOS, and "Requireing unknown module "AssestRegistry"" in Android

:+1:

Confirm - if require image inside project - it works
If image is required in library (that is placed in node_modules) - not working

Might be caused by: https://github.com/facebook/react-native/commit/06b5bda34923b68ba5141e78c36ccbdc5f4bcff1
ETA for a fix next week.


Update README.md

The LATEX magic is now visible.

@RaisinTen
Thank you so much!
It's clean and pretty.

@bowbowbow You're welcome. :)


Revert "Resolve TA image reference at runtime"

This reverts commit f229b1301aef244a4428e575b12eacbb530c04c5.
This was breaking ci in the build-definitions repo.
There, the latest release of this is pulled and used, which means that whenever a new trusted artifacts bundle is available, suddenly CI fails for all changes until those bundle references are updated and merged.
We should rely on renovate to bump those references, like other bundle references so we can manage it in a controlled fashion.

We should rely on renovate to bump those references, like other bundle references so we can manage it in a controlled fashion.

Renovate didn't do a good job at it in https://github.com/konflux-ci/build-definitions/pull/1137, I was thinking this wasn't the best idea, and decided to go for it as we do not update build-trusted-artifacts that often.

The downside of reverting this is that the image will always revert to quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:4e39fb97f4444c2946944482df47b39c5bbc195c54c6560b0647635f553ab23d


Disable master branch in Azure pipeline

1

@Anton1990, these changes are no longer required in my opinion as the main branch doesn't have master in azure pipeline, rather it has the main branch. Hence, it would be best to close the PR.


lower performance than expected

Hi,
I have used the RCNN_model that you have provided to get the results on VOC 2007. The thing is I could get MAP = 42.71 with the provided model, while in the paper the lowest claim is 44.2. Also, you claimed MAP = 66 for RCNN in Fast RCNN paper while I have got 40.11 when I trained my network. Since I am using your code without any modification, I think the problem is with the networks that I have used. Is it possible for you to release the trained rcnn models (mat files) that you have used for VOC 2007, 2010 and 2012 using Alexnet and VGG16?
Thanks so much.

@eefami Have you used the code on Windows?

@Addhi86 No. I used it on Linux Ubuntu.

I am using Windows and I am having a problem to run the code.
I have sent you an email regarding the Object detection on research gate. Can you, please reply me there


fixed tool environment

Thank you for submitting this pull request

How to retest a PR or trigger a specific build:

* a pull request please add comment: Jenkins retest this

@Ginxo thanX for supervising this :)


Segmentation fault in running tests on TravisCI

I am not sure how to debug this as there is no information on which package is segfaulting.
Error message:
/home/travis/.travis/functions: line 104:  4256 Segmentation fault      (core dumped) nosetests splot --with-doctest --with-coverage --cover-package=splot
link to full build log: https://travis-ci.org/pysal/splot/jobs/491124752

I've never seen this error before. Maybe @sjsrey and @ljwolf have a better sense of what this is.

I think that's similar to what we're running into at contextily:
https://travis-ci.org/darribas/contextily/jobs/484776634

Fatal memory faults seems to be leading to this issue. Attempting limited testing in #48

Unfortunately limiting to one test case each didn't solve the issue. My guess is that there is an issue with one of the splot dependencies. Also, this does very much appear to be the exact reason the pysal/pyal builds are failing. My recommended plan of action would be to find which testing script the segfault is being thrown from and then narrow it down to an exact test if possible. From there, simply comment out the offending tests until the problem is resolved.
pysal/pysal#1085

UPDATE: Solved in #48

Closing since solved


Tilemap Feature Request

I think a tile-map that handles tiles with scripts and different physics settings would make level design a lot easier.

http://docs.godotengine.org/en/latest/reference/faq.html#i-have-a-great-idea-that-will-make-godot-better-what-do-you-think

Ok, think about it this way, I have 50 enemy nodes or 100 moving platforms my scene tree would be cluttered. It would be easier to have a tile-map instance enemys and enter-active objects like it does tiles instead of forcing developers to duplicate and move these things into place, when they should just be able to click and poof object is where it needs to be. Thus improving godot's ability to develop game scenes for everyone. This next release is about the editor and usability. This feature would be perfect.

I agree. The ideal would be just a copy and paste of your tilemap.scn when you put the tile, with all the things exactly like they are. Even simple things like levers can't be done right now.

I don't think adding such things into the tilemap would be possible, as
they are rendered and processed in batch for performance reasons.
But if you mean a helper in the tile editor to instance scenes into cells,
that could be pretty easy.. not sure what the level of integration would be
though
On Sat, Apr 30, 2016 at 7:48 PM, mateusak<EMAIL_ADDRESS>wrote:

I agree. The ideal would be just a copy and paste of your tilemap.scn when
you put the tile, with all the things exactly like they are. Even simple
things like levers can't be done right now.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4501#issuecomment-216000582

Maybe a individual toggle for tiles ("Is special = true / false") that would do that? So we can use all the functionality of the tile-map for these special tiles. Like set_cell().

Maybe something like this https://github.com/AlexHolly/HelloGodot/blob/master/scripts/tilemap.gd
I think this is not my recent version I will check later.

Andreas Esau's' asset dropper plug-in for godot, is a great alternative to what I was trying to propose. Closing Issue


PluginClassLoader fails to load compiled class

I'm consistently getting a ClassNotFoundException every other time I run runIdea. Here's a stacktrace of what happens when the exception is thrown. I'd love to provide better information but I don't know how. Here's the repo  that I'm building. Could it be an issue with the Kotlin plugin?

cannot recreate. try to remove ls ~/.gradle/caches/ directory and try again.

I tried this on an Ubuntu VM and on a Macbook Pro. I wasn't able to repro it on the Ubuntu VM but it manifested itself on OS X.
However, I believe I've found part of the issue, if the file .gradle/<gradle-version>/taskArtifacts/cache.properties.lock is deleted before running ./gradlew runIdea, the above exception will no longer happen.
I'm sure this is probably a symptom to the real issue, but for now I've added the following to my gradlew and gradlew.bat
grablew
rm .gradle/2.5/taskArtifacts/cache.properties.lock

gradlew.bat
del %~dp0\.gradle\2.5\taskArtifacts\cache.properties.lock

Cannot recreate on Mac as well. If the problem in cache.properties.lock then I believe it has nothing to do with the plugin. Cannot help with this, sorry.


How to make one Embedding for two LSTMs?

Suppose we have simple example (taken from keras documentation) with one shared LSTM how could I introduce one shared Embedding here? Sorry If my question is already answered but I read this topics and found them a bit confusing. I tried to simply make one Embedding and put it in different sequential models but model.fit failed with assert "You are only have one one input but give two".  Then I took this example below and tried to put shared Embedding.
This is original
from keras.layers import Input, LSTM, Dense, merge
from keras.models import Model

tweet_a = Input(shape=(140, 256))
tweet_b = Input(shape=(140, 256))

# this layer can take as input a matrix
# and will return a vector of size 64
shared_lstm = LSTM(64)

# when we reuse the same layer instance
# multiple times, the weights of the layer
# are also being reused
# (it is effectively *the same* layer)
encoded_a = shared_lstm(tweet_a)
encoded_b = shared_lstm(tweet_b)

# we can then concatenate the two vectors:
merged_vector = merge([encoded_a, encoded_b], mode='concat', concat_axis=-1)

# and add a logistic regression on top
predictions = Dense(1, activation='sigmoid')(merged_vector)

# we define a trainable model linking the
# tweet inputs to the predictions
model = Model(input=[tweet_a, tweet_b], output=predictions)

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])
model.fit([data_a, data_b], labels, nb_epoch=10)

This refactored with shared embeding

# -*- coding: utf-8 -*-
from keras.layers import Input, LSTM, Dense, merge, Embedding
from keras.models import Model
import numpy

tweet_a = Input(shape=(140, ), dtype='int32')
tweet_b = Input(shape=(140, ), dtype='int32')

emb = Embedding(input_dim=100, output_dim=10, input_length=20)

em_1 = emb(tweet_a)
em_2 = emb(tweet_b)

# this layer can take as input a matrix
# and will return a vector of size 64
shared_lstm = LSTM(64)

# when we reuse the same layer instance
# multiple times, the weights of the layer
# are also being reused
# (it is effectively *the same* layer)
encoded_a = shared_lstm(em_1)
encoded_b = shared_lstm(em_2)

# we can then concatenate the two vectors:
merged_vector = merge([encoded_a, encoded_b], mode='concat')

# and add a logistic regression on top
predictions = Dense(1, activation='sigmoid')(merged_vector)

# we define a trainable model linking the
# tweet inputs to the predictions
model = Model(input=[tweet_a, tweet_b], output=predictions)

model.compile(optimizer='rmsprop', loss='binary_crossentropy')

print 'Number of model parameters: ', model.count_params()

data_a = numpy.zeros(shape=(223, 140), dtype='int32')
data_b = numpy.zeros(shape=(223, 140), dtype='int32')
labels = numpy.zeros(shape=(223, 1), dtype='float32')

model.fit([data_a, data_b], labels, nb_epoch=10)

But got:
Traceback (most recent call last):
  File "/home/zakirov/proj/semantic/plm/wt-em-lstm-4-neg.py", line 45, in <module>
    model.fit([data_a, data_b], labels, nb_epoch=10)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1111, in fit
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 826, in _fit_loop
    outs = f(ins_batch)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 811, in __call__
    return self.function(*inputs)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 871, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 859, in __call__
    outputs = self.fn()
  File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", line 951, in rval
    r = p(n, [x[0] for x in i], o)
  File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", line 940, in <lambda>
    self, node)
  File "theano/scan_module/scan_perform.pyx", line 405, in theano.scan_module.scan_perform.perform (/home/zakirov/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-15.04-vivid-x86_64-2.7.9-64/scan_perform/mod.cpp:4316)
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "theano/scan_module/scan_perform.pyx", line 397, in theano.scan_module.scan_perform.perform (/home/zakirov/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-15.04-vivid-x86_64-2.7.9-64/scan_perform/mod.cpp:4193)
ValueError: Input dimension mis-match. (input[0].shape[0] = 224, input[1].shape[0] = 32)
Apply node that caused the error: Elemwise{add,no_inplace}(Subtensor{::, int64::}.0, dot.0)
Toposort index: 35
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(224, 64), (32, 64)]
Inputs strides: [(20480, 4), (256, 4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{mul,no_inplace}(Elemwise{add,no_inplace}.0, DimShuffle{x,x}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 517, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 571, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 155, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/recurrent.py", line 227, in call
    input_length=input_shape[1])
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 981, in rnn
    go_backwards=go_backwards)
  File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan.py", line 745, in scan
    condition, outputs, updates = scan_utils.get_updates_and_outputs(fn(*args))
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 973, in _step
    output, new_states = step_function(input, states)
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/recurrent.py", line 825, in step
    o = self.inner_activation(x_o + K.dot(h_tm1 * B_U[3], self.U_o))

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Apply node that caused the error: for{cpu,scan_fn}(Subtensor{int64}.0, Subtensor{:int64:}.0, IncSubtensor{Set;:int64:}.0, IncSubtensor{Set;:int64:}.0, Subtensor{int64}.0, lstm_1_U_o, lstm_1_U_f, lstm_1_U_i, lstm_1_U_c)
Toposort index: 401
Inputs types: [TensorType(int64, scalar), TensorType(float32, 3D), TensorType(float32, 3D), TensorType(float32, 3D), TensorType(int64, scalar), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(), (20, 224, 256), (21, 32, 64), (21, 32, 64), (), (64, 64), (64, 64), (64, 64), (64, 64)]
Inputs strides: [(), (1024, 20480, 4), (8192, 256, 4), (8192, 256, 4), (), (256, 4), (256, 4), (256, 4), (256, 4)]
Inputs values: [array(20), 'not shown', 'not shown', 'not shown', array(20), 'not shown', 'not shown', 'not shown', 'not shown']
Outputs clients: [[], [], [DimShuffle{0,1,2}(for{cpu,scan_fn}.2)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/home/zakirov/proj/semantic/plm/wt-em-lstm-4-neg.py", line 24, in <module>
    encoded_a = shared_lstm(em_1)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 517, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 571, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 155, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/recurrent.py", line 227, in call
    input_length=input_shape[1])
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 981, in rnn
    go_backwards=go_backwards)

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

What I am doing wrong?

tweet_a = Input(shape=(140, ), dtype='int32')
tweet_b = Input(shape=(140, ), dtype='int32')

emb = Embedding(input_dim=100, output_dim=10, input_length=20)

Your inputs have length 140, but your Embedding layer says the input length is 20. That doesn't make sense.  Drop the input_length= argument on your Embedding layer and it will work.

@mbollmann Good point thanks. Currently I found that on keras 1.2.1

    # define shared embedding
    embed = Embedding(input_dim=vocab_len, output_dim=vec_size, input_length=seq_size)

    # define shared lstm
    title_lstm = LSTM(rnn_size)

    query_lstm = title_lstm

    # define positive rnn convolution
    query = Sequential()
    query.add(Masking(mask_value=0.0, input_shape=(seq_size, )))
    query.add(embed)
    query.add(query_lstm)

    # define positive title
    title_p = Sequential()
    title_p.add(Masking(mask_value=0.0, input_shape=(seq_size, )))
    title_p.add(embed)
    title_p.add(title_lstm)

    # define final concatenation
    model = Sequential()
    model.add(Merge([query, title_p], mode='concat'))

Works but I agree that Masking layer is redundant but without it compilation fails. And I also found that using of Embedding layer produces much over-fitting, possible because of huge number of free parameters.


footer not displayed on iOS

OpenUI5 version:
1.38.8
Browser/version (+device/version):
safari, chrome
iPhone, different devices
Any other tested browsers/devices(OK/FAIL):
working on Android (Chrome), Desktop (Chrome, IE, Firefox)
view.xml:
<Page showFooter="true" showHeader="false" >                 <Panel headerText="...">                 </Panel> 		<footer> 			<OverflowToolbar> 				<ToolbarSpacer /> 				<Button text="..." press="send" /> 			</OverflowToolbar> 		</footer> </Page>
What happens instead?
the footer is rendered outside of the visible view (css.top property is wrong),
it gets displayed correctly within chrome debugger (device=iPhone)

solved, problem occurred due to proprietary App-container around the Page-element


Simp performance on large terms

I've built up a small test to validate the performance of different operations on terms that contain a lot of shared subterms (i.e., they are small when traversed as a dag, but large when traversed as a tree).
I've found that simp appears to do a dag-based traversal, but scales worse than linear or near-linear (maybe quadratic?).
Here's the script:
-- This is a test case to validate the rewriter exploits the
-- dag structure of terms.
--
-- It allows one to build up terms that are binary trees with branches
-- and leaves.  The size of the term as a dag is proporportional
-- to the height of the tree, but contains an expoential number
-- of nodes when viewed as a tree.

def leaf : ℕ := 0

def branch (x y : ℕ) : ℕ := leaf

-- Simplification rule that only works on branch.
theorem branch_leaf_leaf : branch leaf leaf = leaf := begin simp [branch, leaf], end

-- Call a unary operation a given number of times.
meta def repeatn {α : Type} (f : α → tactic α) : α → ℕ  → tactic α
| x 0 := pure x
| x (nat.succ n) := f x >>= λy, repeatn y n

-- Build a branch that contains the same expression as leaves.
meta def branch_same (e : expr) : tactic expr := do
  pure `(branch %%e %%e)

open tactic

example : true := by do
  let n : ℕ := 1024,
  trace "iterations:",
  trace n,
  e ← repeatn branch_same `(leaf) n,
--  trace e,
  s ← simp_lemmas.mk.append
     [ (expr.const `branch_leaf_leaf [])
     ],
  (s', pr) ← simplify s e,
  trace "result:",
  trace s',
  tactic.triv

Here's a log of runtimes on a machine with 16GB ram:
lean% time lean dag.lean
iterations:
512
result:
leaf
lean dag.lean  5.25s user 0.09s system 100% cpu 5.334 total

lean% time lean dag.lean
iterations:
1024
result:
leaf
lean dag.lean  47.39s user 0.28s system 99% cpu 47.947 total

lean% time lean dag.lean
iterations:
1536
result:
leaf
lean dag.lean  152.31s user 0.24s system 99% cpu 2:32.71 total

I'm not sure what the expected performance is of simp, but for some of my envisioned use cases, it would be great if the runtime was linear or near-linear with respect to the term size (for a fixed set of rules). It's ok if some rules can push simp into exponential behavior.

@joehendrix Leo and I spent some time talking about this today, it appears that the simplifier performs a linear number of operations, but it requires further investigation to understand where the performance hit is. We will try to spend some time this week chasing it down, thanks for the thoughtful issue!

I have profiled this benchmark. I used n = 512.
94% of the runtime is spent comparing expressions
bool is_equal(expr const & a, expr const & b);

It seems we get a lot of hash code collisions in this benchmark. That is, is_equal is invoked with expressions that have the same hash code but are structurally different.
The Hashtable insertion procedures are invoked approx. 15K times, but is_equal is invoked more than 600K times due to hash collisions. The performance of is_equal is also affected by the hash collisions.
The following line is reached approx. 180 million times.
https://github.com/leanprover/lean/blob/master/src/kernel/expr_eq_fn.cpp#L71
I'm currently investigating why we have so many collisions.

I recall that I experienced a similar (hash collision) problem in the past, and I addressed it by incorporating the term weight into the hash code computation. Unfortunately, the weight is ineffective for this benchmark since many sub terms have weight UMAX_INT because of the sharing. So, I decided to store the depth of composite terms and use it in the hash code computation. The new numbers are much better.
Here are the runtimes (using option -j 0 no multi threading):

empty file : 0.500 secs (loading init library)
n = 100 : 0.570 secs     (before 0.612 secs)
n = 500 : 0.600 secs     (before 4.3 secs)
n = 1000 : 0.604 secs   (before 31.728 secs)
n = 1500 : 0.612 secs
n = 2000 : 0.640 secs
n = 4000 : 0.664 secs
n = 8000 : 0.684 secs
n = 16000 : 0.816 secs
n = 32000 : 1.016 secs

Remark: I had to increase stack space for n >= 4000
@gebner @kha It is not clear to me why we had so many hash collisions. Any ideas?
BTW, I tried different hash(unsigned, unsigned) combine functions, and they did not fix the problem.
I'm concerned we may have some bad hash function somewhere in the code, and using the depth is just masking the real problem.

Thanks for fixing this.


Add heatmap support

We can support heatmap as UnicodePlots.jl did at https://github.com/Evizero/UnicodePlots.jl/pull/88

Hello @mrkn , I think adding this may require a dependency on another Gem such as https://github.com/janlelis/paint to be able to support 24-bit color in the terminal.  Is it acceptable to add this dependency to support the heatmap?

I want to investigate the library and consider what should I do.

We need to use https://github.com/red-data-tools/red-colors and https://github.com/red-data-tools/red-palette to implement colormap used in heatmap because this library is a backend of https://github.com/red-data-tools/charty.

We need to use https://github.com/red-data-tools/red-colors and https://github.com/red-data-tools/red-palette to implement colormap used in heatmap because this library is a backend of https://github.com/red-data-tools/charty.

@mrkn I am working on this and have made some progress getting simple plots to display.  There is a dependency on #34, as it requires at least 256 color support for this feature.


Two additional convenience services

Two additional services shall be added for providing the UUIDs for direct addressing
    - operation-name: /v1/provide-list-of-device-interfaces
      uuid: mwdi-1-0-0-op-s-is-011
    - operation-name: /v1/provide-list-of-actual-device-equipment
      uuid: mwdi-1-0-0-op-s-is-012

Fixed in branch Open backhaul/issue302

TopLevelElement to be added in /v1/provide-list-of-actual-device-equipment.