clientv3: remove redundant error handling code

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@ff6d686). Click here to learn what that means.
The diff coverage is 50%.

@@            Coverage Diff            @@
##             master    #7568   +/-   ##
=========================================
  Coverage          ?   70.15%
=========================================
  Files             ?      320
  Lines             ?    26179
  Branches          ?        0
=========================================
  Hits              ?    18366
  Misses            ?     6356
  Partials          ?     1457

Impacted Files
Coverage Δ

clientv3/cluster.go
72.41% <50%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff6d686...dd465d0. Read the comment docs.

lgtm


ng-flow only uploads when i select file twice

Hello, has anyone else noticed this? When i select a file for the first time it doesn't seem to work. it only works when i try to select a file again and on my server side, i notice that i have both the previous and the new requests coming in.
I also notice during the first file selection, in the portion of the code from the library (line 305 ng-flow-standalone)
// Now, simply look for the next, best thing to upload
      each(this.files, function (file) {
        if (!file.paused) {
          each(file.chunks, function (chunk) {
            if (chunk.status() === 'pending' && chunk.preprocessState === 0) {
              chunk.send();
              found = true;
              return false;
            }
          });
        }
        if (found) {
          return false;
        }
      });

that the "files" array is empty, then on the second selection it contain the previous file that i intended to upload.
What is this wierd behavour?

Have the same issue


Legacy logging is not invoked

The very first code example from the docs
import gym

from stable_baselines.common.policies import MlpPolicy
from stable_baselines.common.vec_env import DummyVecEnv
from stable_baselines import PPO2

env = gym.make('CartPole-v1')
env = DummyVecEnv([lambda: env])  # The algorithms require a vectorized environment to run

model = PPO2(MlpPolicy, env, verbose=1)
model.learn(total_timesteps=10000)

obs = env.reset()
for i in range(1000):
    action, _states = model.predict(obs)
    obs, rewards, dones, info = env.step(action)
    env.render()

should produce a line similar to
Logging to /tmp/openai-2019-04-28-01-04-12-724654

according to https://github.com/hill-a/stable-baselines/blob/master/stable_baselines/logger.py#L575, but it doesn't. This indicates that legacy logging isn't triggered at all. Indeed, in my larger project, configuring OPENAI_LOGDIR and OPENAI_LOG_FORMAT does not cause logs to appear.
The motivation for having legacy logging is being able to compare results from stable-baselines with the results obtained by upstream (in particular, having eprewmean).
Library is installed from git commit 0693426.

Hello,
as you pointed out, you need to call configure() in order to have the legacy logging enabled:
from stable_baselines.logger import configure

configure()

It seems that no mention of that is made in the documentation... (it is only present in the run_atari.py files)
However, if you want to monitor training (to have the mean reward during training for instance), I recommend you to use the Monitor wrapper (see an example in the doc ;) )


api inconsistency in statename in national data and resources data

In https://api.covid19india.org/data.json, states with and in their name are written like Jammu and Kashmir, Andaman and Nicobar Islands.
Whereas in https://api.covid19india.org/resources/resources.json, states with and in their name are written like Jammu & Kashmir, Andaman & Nicobar Islands.

I think same pattern should be followed for consistency.

The total number of cases for west bengal has been updated today as 922 but the increase in the number of total cases from yesterday which is 127 has not been added to the national total. Also the number of discharged in the state has not been added to the national total. Requesting you to update the national total

The total number of cases for west bengal has been updated today as 922 but the increase in the number of total cases from yesterday which is 127 has not been added to the national total. Also the number of discharged in the state has not been added to the national total. Requesting you to update the national total

This can be a new issue.

A similar issue was mentioned already before as well.
#29 Sheet maintenance issue
How can this be avoided in future? @JunaidBabu

We did a district name correction task in the past two days. Hence the mismatch. I will pass on the new spellings to the Essentials team to map too.
@alphasingh saw your other issues regarding S.A.S Nagar being expanded etc. Is that a compulsory edit. It takes a lot of work to fix the district names across multiple places. If it's not critical, I would like to leave this alternate spelling as is.

@sudevschiz the data being reflected on the maps should have the same "key" name, only then will it reflect its data. So, yes the fix is critical in the sense that even after having all the data in the API, we're not showing it on the web due to this.
Suggestion:
Both web and API key names for each district/state should exist in a single place, like a common enum that is being used by both, to avoid any mismatch and unnecessary manual fixes.

@alphasingh I think @sudevschiz's question about compulsory edit was for SAS Nagar. We will make sure they are consistent across the website and the data sheet and essentials. But should SAS Nagar be expanded is the question.

I have passed on the new district list to the essentials team. It will get edited everywhere soon.

@bee-rickey As long as every state and district is being maintained with the same "key" name everywhere, I don't think we need to put an extra effort for SAS nagar alone.
Let's keep it as it is everywhere, as of now, because I can see it with "key" name as S.A.S. Nagar on https://www.covid19india.org/state/PB and it reflects the same data as it should, i.e. from the API where it has the exact same "key" name as S.A.S. Nagar

@bee-rickey As long as every state and district is being maintained with the same "key" name everywhere, I don't think we need to put an extra effort for SAS nagar alone.
Let's keep it as it is everywhere, as of now, because I can see it with "key" name as S.A.S. Nagar on https://www.covid19india.org/state/PB and it reflects the same data as it should, i.e. from the API where it has the exact same "key" name as S.A.S. Nagar

Got it. Thanks!

Resources data has now been deprecated. Closing this issue as it is irrelavant.


0.5.8 Build failure

Building 0.5.8 fails in a clean chroot using my AUR PKGBUILD:
curl -sS https://raw.githubusercontent.com/meodai/color-names/master/src/colornames.csv | \
  awk -F, '$1 ~ /^[a-zA-Z]+$/' | \
  gzip > pkg/keyboard/colornames.csv.gz
rm -f huekeys # to make it obvious when watch fails
go generate ./...
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
make[1]: Entering directory '/build/huekeys/src/huekeys'
echo "$BUILDINFO" | tee buildinfo/build.yml
build_time: 2022-07-04T14:59:27Z
commit_hash: 24e1276
version: v0.5.8
make[1]: Leaving directory '/build/huekeys/src/huekeys'
go build -ldflags='-s -w' -o huekeys
14:59:34.547 WRN unable to use syslog: switched to stderr error="Unix syslog delivery error"
Error: unable to open /build/.local/share/applications/huekeys.desktop: open /build/.local/share/applications/huekeys.desktop: no such file or directory
14:59:34.548 ERR command failed error="unable to open /build/.local/share/applications/huekeys.desktop: open /build/.local/share/applications/huekeys.desktop: no such file or directory"

thanks for letting me know. should be fixed in v0.5.9

Well, the build completes successfully now. However, a few things:

Desktop files do not need a shebang
The desktop file and icon should not be installed to XDG_DATA_HOME when not building as user
The Exec and Icon paths reference the build directory, not where they are actually being installed:Exec=/build/huekeys/src/huekeys/huekeys menu
Icon=/build/.local/share/huekeys/icon.png

should be fixed in v0.5.10


Option admin name validation is being applied to existing admin names

Describe the bug
Option admin name validation is being applied to existing admin names
Reported by @paripatel07
To Reproduce
Steps to reproduce the behavior:

Edit a question with an option admin names that has spaces in it
Try to save the question
See that the validation is failing

Expected behavior
Validation should only apply to new admin names
Screenshots

I think we have three options to fix this quickly:

We can remove admin name character types from the server-side validation of the question definition, and just rely on the frontend JS validation. This will only check new options.
We can pull the admin name invalid character check up to the update() method on the service, which would require passing in the previous question definition for comparison:
https://github.com/civiform/civiform/blob/9b719efabea0eef49710c58226da992f792cd7ee/server/app/services/question/QuestionService.java#L122
We can do this validation from the update() method on the controller, which already has the old and new question definitions. Maybe via a call to a new method on the service: https://github.com/civiform/civiform/blob/3b4329751181163928065097cccd15e6c66e4fc6/server/app/controllers/admin/AdminQuestionController.java#L262


[Autocomplete] Fix bug on disableOpenOnFocus prop

Fix #18360

[x] I have followed (at least) the PR section of the contributing guide.

@netochaves Could you add a regression test? Thanks.

I've added the tests @oliviertassinari. I'm new with regressions tests, please let me know if I've done something wrong.

@netochaves Thanks, I have extended the test case and removed the new demo as we already cover the case in the "Playground".

@netochaves Thank you


[gsplat] on Windows, spherical harmonics consume excessive VRAM

With #2521, and the bicycle dataset (thanks @kerrj),
on my Windows 11 machine with CUDA 11.8,
ns-train gaussian-splatting colmap --data data\bicycle --colmap-path sparse/0
exceeds 16GB VRAM (this card's limit) after iteration 1810,
so iterations are slower starting at 1820,
catastrophically slower at 1910,
and ultimately fail with CUDA OOM once 32GB VRAM is exceeded,
sometime before reaching 2310 steps.
With lower sh-degree values, the pressure is less (although this machine still fails).
With sh-degree of zero, training completes successfully without exceeding 16GB VRAM (and thus slowing down).
Per @kerrj , this excessive VRAM consumption does not occur on Ubuntu Linux on similar hardware.

This issue might be fixed in the latest PR: https://github.com/nerfstudio-project/nerfstudio/pull/2709
I have successfully trained GS on bicycle dataset.


Miscellaneous nov.2

Hi DrDuh,
Up to you to evaluate and see what to do with these sites, remarks, questions or explanations collected from....., some are above my ItQ 👀
Sorry for my usual mess.
Thank you,

Perhaps add for some chapters, also how to uninstall software (e.g. dnsmasq, dnscrypt...)

OS X VPN settings - Option tab ~"send all trafic on the VPN"
what's not going on the VPN if this option is disabled?

OS X  Network - Proxy ~"ignore proxy for hosts and domains"?
I think this "*.local, 169.254/16" is by "default".
What to do?

OS X Firewall - Advanced tab: set a small delay and require pw.

OS X - Users & Groups - ~Open/launch !
You can remove an application that launch at startup, even without unlocking the dialog (orange locker bottom left)!!
 Already report that several times to Apple years ago...

Should we disable IP6, as it seems that in some cases using both IP4 and IP6, this might leaks information?

https://letsencrypt.org
https://github.com/okTurtles/dnschain
http://thehackernews.com/2015/10/nsa-crack-encryption.html
http://www.cisco.com/web/about/security/intelligence/nextgen_crypto.html

FOR FASTER VPN

router with UPnP ON (security risk, probably not needed if NAT maps automatically
FOR BETTER PRIVACY
random port and NAT auto-open/map
block bad/dangerous/snooping peers
add a proxy
use non-proprietary DNS servers

VPN for privacy... but VPN are always starting after other programs which might already have accessed internet...

Paranoia
Who is behind these testing websites.
Nice places to collect data, browser fingerprints... especially when you are testing your 'uncompleted/unsafe' configuration.
whoer, browserleaks, ipleak and similar, why so many 'secret/domain-by-proxy...' whois?

We use Google Public DNS server, which we consider unproblematic. It is not only the biggest public server with over 130 billion requests per day and works fast, but also does not store personally identifiable information nor IP-addresses permanently and all temporary logs are deleted after 48 hours at the latest. (from a VPN provider... don't remember)
Though Google DNS Resolvers are censuring several kinds of sites and are among companies providing data to bigbro...

2FA
Don't forget to have a 'backup', e.g. like printed code for Google Authenticator, in case your phone is down.

https://www.verisignlabs.com/orscan/
Are Open Resolvers bad?  Generally, yes.

Concerning VPN and IP4-IP6 leakage.
http://blog.cyberghostvpn.com/dns-ipv6-leak-cyberghost-not-affected/

Beginner’s Guide to Open Source Intrusion Detection Tools
https://www.alienvault.com/resource-center/white-papers/beginners-guide-to-open-source-intrusion-detection-tools

Thanks for the usual tips and guides. I will add these in over time as incremental changes, as I work through more thoroughly reading and reviewing them.
Re: ignoring proxy settings for localhost, the default seems reasonable.
Re: disabling IPv6, I am not an expert in the matter, so I would rather trust the defaults.


Strongly increased memory consumption/waste since V2

After having upgraded to Cloud SDK V2 and the corresponding VDMs, the memory consumption has increased, which can cause apps deployed to CloudFoundry to crash where they would have worked pretty well with SDK v1.
In an export scenario where I fetch data in buckets from the SAP system, combine those and then pass the complete result as CSV to an endpoint, I compared using SDK/VDMs in V1.x to Version 2 (the VDM-APIs used here are mainly the ones from @sap/cloud-sdk-vdm-material-planning-data-service). The very same data was exported, the same machine and runtime was used. In both variants, the export took about 25 seconds, so no big difference in performance.
The memory consumption looks as follows:
Before V2 upgrade

After V2 upgrade

The heap consumption has a peak which is more than twice as high for V2 (600 MB) as it was in V1 (220 MB).
External memory usage and ArrayBuffers show slightly higher values but the rate is not as significant as for the heap.
Also a memory leak doesn't apply in my mind as the garbage collector cleans up properly. Only the short-term heap usage seems to be a wasteful.
Used Versions:

node version: 14.18.2 (both scenarios)
npm version: 6.14.15 (both scenarios)
SAP Cloud SDK version: 1.54.0 / 2.1.0
VDM versions: 1.28.2 / 2.0.0

Hi @nocheintobi ,
Thank you very much for SDK V2 adoption and raising this up.
Could you please share some code snippets/examples about your use case, so we can also reproduce?
Also, how big is the CSV file you want to export?
Best regards,
Junjie

Hi @jjtang1985 ,
I prepared the two minimalistic snippets that show the difference in heap consumption.
I used a plant where we have about 350 materials and about 60000 SupplyDemandItems for all those materials together.
For simplicity, I left out anything that concerns CSV and field limitation via $select.
As you might guess, you'll need to add a destination in line 1 of each example ;-)
Here the V1-Sample code:
const DESTINATION = {...};
const { or } = require("@sap-cloud-sdk/core");
const {MrpMaterial, SupplyDemandItems} = require("@sap/cloud-sdk-vdm-material-planning-data-service")

let itemsCount = 0;
/* Fetch data via VDM */
async function fetch(plant, skip, top) {
    // const result = [];
    const materials = await MrpMaterial.requestBuilder().getAll().filter(MrpMaterial.MRP_PLANT.equals(plant)).skip(skip).top(top).execute(DESTINATION);
    const items = await SupplyDemandItems.requestBuilder().getAll().filter(SupplyDemandItems.MRP_PLANT.equals(plant), or(...materials.map(m => SupplyDemandItems.MATERIAL.equals(m.material)))).execute(DESTINATION);
    itemsCount += items.length;
    return materials.map(m => {return {plant: m.mrpPlant, material: m.material, items: items.filter(i => i.material === m.material)}});
}

// monitor heap
const interval = setInterval(() =>
    console.log(`HEAP used: ${Math.round(process.memoryUsage().heapUsed/1024/1024)} MB, items received: ${itemsCount}`), 1000);

(async() => {
    const PLANT = "1010";
    const count = (await MrpMaterial.requestBuilder().getAll().filter(MrpMaterial.MRP_PLANT.equals(PLANT)).select(MrpMaterial.MATERIAL).execute(DESTINATION)).length;
    const tasks = [];
    const pageSize = 30;
    for(let i = 0; i < count; i+=pageSize) {
        tasks.push(fetch(PLANT, i, pageSize));
    }
    const allData = [].concat(...await(Promise.all(tasks)));
    console.log("Total Size: " + JSON.stringify(allData).length);
    console.log(`Materials: ${count}, items: ${itemsCount}`);
    setTimeout(() => {clearInterval(interval)}, 10000);
})();

and here the V2 sample code:
const DESTINATION = {...};

const { or } = require("@sap-cloud-sdk/odata-common");
const VDM = require("@sap/cloud-sdk-vdm-material-planning-data-service")

const materialApi = VDM.materialPlanningDataService().mrpMaterialApi;
const itemsApi = VDM.materialPlanningDataService().supplyDemandItemsApi;
const MATERIAL = materialApi.schema;
const ITEM = itemsApi.schema;
let itemsCount = 0;

async function fetch(plant, skip, top) {
    const result = [];
    const materials = await materialApi.requestBuilder().getAll().filter(MATERIAL.MRP_PLANT.equals(plant)).skip(skip).top(top).execute(DESTINATION);
    const items = await itemsApi.requestBuilder().getAll().filter(ITEM.MRP_PLANT.equals(plant), or(...materials.map(m => ITEM.MATERIAL.equals(m.material)))).execute(DESTINATION);
    itemsCount += items.length;
    return materials.map(m => {return {plant: m.mrpPlant, material: m.material, items: items.filter(i => i.material === m.material)}});
}
// monitor heap
const interval = setInterval(() =>
    console.log(`HEAP used: ${Math.round(process.memoryUsage().heapUsed/1024/1024)} MB, items received: ${itemsCount}`), 1000);

(async() => {
    const PLANT = "1010";
    const count = (await materialApi.requestBuilder().getAll().filter(MATERIAL.MRP_PLANT.equals(PLANT)).select(MATERIAL.MATERIAL).execute(DESTINATION)).length;
    const tasks = [];
    const pageSize = 30;
    for(let i = 0; i < count; i+=pageSize) {
        tasks.push(fetch(PLANT, i, pageSize));
    }
    const allData = [].concat(...await(Promise.all(tasks)));
    console.log("Total Size: " + JSON.stringify(allData).length);
    setTimeout(() => {clearInterval(interval)}, 10000);
})();

mem-results.md shows the results on my machine.
For this reduced samples, the difference in heap usage is not as big as in my real code, but still significant. If neccessary, I can enhance the sample - just let me know...
Thanks for your support!
Tobi

Hi @nocheintobi ,
Thank you for your detailed answer.
I'll investigate and let you know.
BR,
Junjie

A temporary workaround for those who are facing similar problems until the issue has been fixed might be one of the following:

Start node with --optimize-for-size to reduce the overall memory footprint of your application
Start node with --expose-gc and run global.gc() between huge fetches (might be handy in batch jobs)
Try the combination of both
Both will potentially slow down your application, --optimize-for-size possibly everywhere and the garbage collector whenever you're executing it.

Hi @nocheintobi ,
I'm looking into it.
It seems, the deserialisation is quite heavy especially when running the execute in a loop and also the number of entity is big.
To give you an overview, the execute method you used contains two parts:

build and execute the request and get an response from your destination server. You can also replace your execute with executeRaw to achieve so. (doc)
deserialize the original response and build entities.

Therefore, as a workaround, you can git rid of the derserializer by:

switch to executeRaw
pick payload of the response body (response.data)
pick the entity collection of the response (response.data.d.results)
iterate the collection
merge materials and item like you did before, but you need know the property keys, which can be found in the schema (e.g., materialApi.schema)

If you do so, please let me know, whether the memory usage is reduced significantly. (I like the graphs very much 😃 )
I'll let you know further updates.

Hi @jjtang1985,
I gave it a try using executeRaw in my provided example and can confirm that memory consumption reduces heavily. Where my previous V2 sample consumed >800MB, using executeRaw took about 250MB at the peak.
Are there plans for optimizing the deserialization? If so, I'd prefer sticking to execute and working with the given types instead of switching the app code to using executeRaw...

Hi @jjtang1985,
after generating the client with sdk version 2.2.1-20220407123656.0, the memory consumption of the test scenario reduced to about 450MB (+/-25MB).
So now we have:

So half-way done ;-)
I additionally tested with the node flags I mentioned earlier, --optimze-for-size only reduced the peak slightly (about 420MB) and the garbage collector brought me back into the range I reached with V1... So the main part of the increased memory consumption seems to be caused by dereferenced objects on the heap...
I'm still available for further tests if needed.

Hi @nocheintobi ,
Thank you very much for providing a nice graph for comparison.
I created a mock server that has:

TestEntity50Col, an entity that has 50 columns (60K elements)
TestEntity100Col, an entity that has 100 columns (350 elements)

I tried to reproduce the issue, but it shows the same level of memory usage.
So far, still one part I was not able to mock, as my server could not handle 60000 (items) / 30 (page size) = 2000 requests in parallel.
But I don't think the number is relevant, as the memory usage for 1, 10, 100, 200 requests is direct proportional.
It would be great, if you can support further.
Please install the node module 'node-memwatch-new' for memory check, then wrap your test code like below, so we can understand better:
  var memwatch = require('node-memwatch-new');
  var hd = new memwatch.HeapDiff();
  // put your test code here
  var diff = hd.end();
  console.log(JSON.stringify(diff.change.details.filter(e => e.size.includes('mb'))));

Please do it for both version 1 and canary so we can compare.
The output should look like:
[
  {"what":"Native","size_bytes":18360133,"size":"17.51 mb","+":425,"-":132},
  {"what":"Number","size_bytes":1608096,"size":"1.53 mb","+":100572,"-":66},
  {"what":"Object","size_bytes":4371432,"size":"4.17 mb","+":10841,"-":82},
  {"what":"String","size_bytes":8244528,"size":"7.86 mb","+":201124,"-":2014}
]

@nocheintobi ,
Thank you very much for your detailed reports 👍
We will plan a stable release for SDK cores (@sap-cloud-sdk/*) hopefully next week, including the generator.
However, as I said, the vdm release @sap/cloud-sdk-vdm-material-planning-data-service was planned at the end of May.
Let's see whether we can make a new one before that.
@marikaner , FYI.
Thank you very much once again for reporting this with professional approaches.
We really appreciate it 😃
Best regards,
Junjie


[spelling mistakes] There are some spelling mistakes in the document.

site2/docs/security-jwt.md:260: forwared ==> forwarded
site2/docs/security-kerberos.md:196: preform ==> perform
site2/docs/security-kerberos.md:221: comparision ==> comparison
site2/docs/security-kerberos.md:241: comparision ==> comparison
site2/docs/helm-overview.md:13: compoments ==> components
site2/docs/helm-overview.md:13: seperately ==> separately
site2/docs/helm-overview.md:92: conciously ==> consciously
site2/docs/security-athenz.md:44: encrption ==> encryption
site2/docs/client-libraries-websocket.md:72: Broswer ==> Browser
site2/docs/security-authorization.md:40: creats ==> creates

site2/docs/functions-develop.md:1097: refered ==> referred

site2/docs/developing-binary-protocol.md:415: retreive ==> retrieve
site2/docs/security-tls-transport.md:81: creat ==> create

site2/docs/administration-pulsar-manager.md:129: PostgresSQL ==> postgresql
site2/docs/administration-pulsar-manager.md:144: genrated ==> generated
site2/docs/administration-pulsar-manager.md:145: genrated ==> generated
site2/docs/administration-pulsar-manager.md:150: PostgresSQL ==> postgresql
site2/docs/administration-pulsar-manager.md:165: genrated ==> generated
site2/docs/administration-pulsar-manager.md:170: PostgresSQL ==> postgresql
site2/docs/deploy-bare-metal.md:171: exsiting ==> existing
site2/docs/deploy-bare-metal.md:375: succesfully ==> successfully

site2/docs/client-libraries-java.md:594: follwoing ==> following
site2/docs/client-libraries-java.md:627: follwoing ==> following
site2/docs/client-libraries-java.md:637: follwoing ==> following
site2/docs/security-tls-keystore.md:124: interal ==> internal
site2/docs/security-tls-keystore.md:229: interal ==> internal
site2/docs/concepts-messaging.md:522: dicates ==> dictates
site2/docs/administration-dashboard.md:54: explicitely ==> explicitly
site2/docs/administration-load-balance.md:118: avaliable ==> available
site2/docs/reference-pulsar-admin.md:2426: garanteed ==> guaranteed
site2/docs/reference-pulsar-admin.md:2620: specifed ==> specified
site2/docs/reference-configuration.md:59: readd ==> read
site2/docs/reference-configuration.md:100: ture ==> true
site2/docs/reference-configuration.md:247: implementatation ==> implementation

Fixed in https://github.com/apache/pulsar/pull/11715


Channels sometimes get disabled and re-enabled even when active

Background
Continuation of #5980. I found out that sometimes channels get disabled 22 minutes after the peer has disconnected even though that peer reconnected shortly after that and before 22 minutes passed. The channel is re-enabled in a few minutes after (0-4 minutes in my case) but the "channel disabled" announcement is already broadcasted at that point which causes long period of channel being unavailable from me for the rest of the network.
Your environment

version of lnd: v0.14.0-beta.rc4
which operating system (uname -a on *Nix): 5.10.63-v8+ aarch64
version of btcd, bitcoind, or other backend: bitcoind 22.0
maybe relevant settings:

minbackoff=1s
maxbackoff=1m
caches.reject-cache-size=10000000
caches.channel-cache-size=1000000

Steps to reproduce
Nov 16 02:08:42 fullnode lnd[384677]: 2021-11-16 02:08:42.852 [INF] PEER: disconnecting 032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b@<IP_ADDRESS>:43864, reason: peer 032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b@<IP_ADDRESS>:43864 no answer for 5m0s -- disconnecting
Nov 16 02:08:42 fullnode lnd[384677]: 2021-11-16 02:08:42.852 [INF] PEER: unable to read message from 032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b@<IP_ADDRESS>:43864: read tcp <IP_ADDRESS>:9735-><IP_ADDRESS>:43864: use of closed network connection
Nov 16 02:08:43 fullnode lnd[384677]: 2021-11-16 02:08:43.053 [INF] DISC: Removing GossipSyncer for peer=032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b
Nov 16 02:08:43 fullnode lnd[384677]: 2021-11-16 02:08:43.054 [INF] HSWC: Removing channel link with ChannelID(3a0fba1fc7537792c1a9105030bc63c9d0931513b37a9d1ea6b7b2d6c878c928)
Nov 16 02:08:43 fullnode lnd[384677]: 2021-11-16 02:08:43.054 [INF] HSWC: ChannelLink(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5): stopping
Nov 16 02:08:43 fullnode lnd[384677]: 2021-11-16 02:08:43.054 [INF] HSWC: ChannelLink(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5): exited

Okay, a peer timed out. Happens sometimes. Now lnd should wait for 22 minutes until it declares the channel disabled. Let's see what happens next:
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.463 [INF] SRVR: New inbound connection from <IP_ADDRESS>:28548
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.463 [INF] SRVR: Finalizing connection to 032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b@<IP_ADDRESS>:28548, inbound=true
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.660 [INF] PEER: NodeKey(032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b) loading ChannelPoint(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5)
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.672 [INF] HSWC: Removing channel link with ChannelID(3a0fba1fc7537792c1a9105030bc63c9d0931513b37a9d1ea6b7b2d6c878c928)
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.673 [INF] HSWC: ChannelLink(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5): starting
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.673 [INF] HSWC: Trimming open circuits for chan_id=695110:1597:5, start_htlc_id=568
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.673 [INF] HSWC: Adding live link chan_id=3a0fba1fc7537792c1a9105030bc63c9d0931513b37a9d1ea6b7b2d6c878c928, short_chan_id=695110:1597:5
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.673 [INF] HSWC: ChannelLink(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5): HTLC manager started, bandwidth=1478031069 mSAT
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.673 [INF] HSWC: ChannelLink(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5): attempting to re-synchronize
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.673 [INF] CNCT: Attempting to update ContractSignals for ChannelPoint(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5)
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.674 [INF] NTFN: New block epoch subscription
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.674 [INF] PEER: Negotiated chan series queries with 032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b
Nov 16 02:12:27 fullnode lnd[384677]: 2021-11-16 02:12:27.674 [INF] DISC: Creating new GossipSyncer for peer=032261d6ec5fb578919e2455f2061d27bd93df57b26d78cdf7bad38f1368d0531b
Nov 16 02:12:28 fullnode lnd[384677]: 2021-11-16 02:12:28.049 [INF] HSWC: ChannelLink(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5): received re-establishment message from remote side

The channel seems to be working now. However:
Nov 16 02:30:21 fullnode lnd[384677]: 2021-11-16 02:30:21.431 [INF] NANN: Announcing channel(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5) disabled [detected]

18 minutes later it's announced as disabled (???) even though it was working at that moment. This timestamp 02:30:21 is 22 minutes later the moment the channel died due to peer disconnecting after timeout (02:08:43). Could it be a timer not being cancelled if the channel reconnects? This channel is reenabled a minute later:
Nov 16 02:31:27 fullnode lnd[384677]: 2021-11-16 02:31:27.675 [INF] NANN: Announcing channel(2dc978c8d6b2b7a61e9d7ab3131593d0c963bc305010a9c1927753c71fba0f3a:5) enabled

I have many such occurrences in the log, they all share the same properties:

Peer disconnects for whatever reason, channel link stops
Peer reconnects, channel link becomes active again
22 minutes after 1. happened channel gets disabled
Channel gets re-enabled in a few minutes (0-4) after 3.

It can't be explained by channels flapping for a while. They clearly behave normally for hours, then disconnect/reconnect once and then get disabled and enabled after that even though they should be perfectly working.
Expected behaviour
Channel should only be disabled if the peer disconnected (or the channel link exit for any other reason) and was down for chan-disable-timeout time (22 minutes by default).
Actual behaviour
Sometimes (but not always!) a channel is marked as disabled 22 minutes after the peer disconnected even though that peer connected shortly after disconnect (before 22 minutes passed).

@rkfg Does this still happen? Do you happen to have any logs with debug turned on?

Yes, it happens regularly several times a day. What subsystem should I increase the verbosity for? DISC and/or NANN?

I also experience that some of my channels are known as "disabled" in the network although they are enabled, even hours after the last state change / reconnect to the peer at the other side of the channel.

One of those disabled channels from @C-Otto was to my node. In case my logs are of any use, please ping me and let me know

@rkfg Would you mind patching this PR to your lnd? This PR adds more debugging logs, and commits in it are cherry-picked from other PRs which are attempting to fix issues as such.
After which, I'd say just turn on debug for the whole node for a while and send me the logs.
Thanks!

Thanks for the logs!
Looks like there's issue with the peer<PHONE_NUMBER>3d2d37b285dae4df40b413d8c6c791fbee7797bc5dc70812196d7d5c. Do you happen to know whether it's online during that time?
And could you search the logs for Created ChanStatusManager with: ChanEnableTimeout and post it here?
I think there's something wrong with the peer connection, will post a detailed report here tomorrow. In short, we require the node the send a ChanSync message to ensure the liveness of the channel, but it appeared there's connection issue with the peer which caused a delay, and the channel is marked disabled first and then enabled again.

Do you happen to know whether it's online during that time?

I suppose it was online all the time, it's LNMarkets and they were online before I restarted my lnd. Doubt we both went down at the same time.

And could you search the logs for Created ChanStatusManager with: ChanEnableTimeout and post it here?

I couldn't find that message in the yesterday's logs but I disabled debugging shortly after I sent these logs above, there's a lot of data generated from that. I re-enabled debug now and will look for it.

The amount of logs generated is enormous and grepping them takes a lot of time. In just 6 hours it logged about 6 Gb I think. I couldn't find the message you mentioned but no channels were disabled since either.
I'm afraid I have to turn logging back to INFO unless there's an important reason to catch it. These events usually happen daily but not very often, like 2-3 times a day so waiting for them takes time and systemd's logs are extremely slow to parse and even open. Maybe you can reduce this verbosity to the bare minimum so they're still useful but not as I/O hungry? Or maybe someone else can do that like @C-Otto and @viaj3ro.

Maybe it helps to look at a very concrete example:<PHONE_NUMBER>91397377 (c-otto.de to SilentBob (@viaj3ro)) is enabled on both sides according to my lnd, but the network doesn't see this. Instead, the network believes that I disabled the channel.

This channel looks like this on my side:
{
    "channel_id": "782139795591397377",
    "chan_point": "9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1",
    "last_update":<PHONE_NUMBER>,
    "node1_pub": "027ce055380348d7812d2ae7745701c9f93e70c1adeb2657f053f91df4f2843c71",
    "node2_pub": "02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844",
    "capacity": "20000000",
    "node1_policy": {
        "time_lock_delta": 40,
        "min_htlc": "1000",
        "fee_base_msat": "0",
        "fee_rate_milli_msat": "233",
        "disabled": true,
        "max_htlc_msat": "5000000000",
        "last_update":<PHONE_NUMBER>
    },
    "node2_policy": {
        "time_lock_delta": 50,
        "min_htlc": "1",
        "fee_base_msat": "100",
        "fee_rate_milli_msat": "349",
        "disabled": false,
        "max_htlc_msat": "20000000000",
        "last_update":<PHONE_NUMBER>
    }
}

As you can see my last_update is 7 seconds older and yours never made it to my node (I don't have channels to you but it doesn't really matter in this case) so I see this channel as disabled from one side. It will stay like this for hours I suppose and there's no way to use it in one direction until the more recent policy version is propagated.
Look in your log for the word disabled, then see if it was enabled shortly after that, then look if there was a disconnection ~20-22 minutes before it got disabled, then look if it was reconnected between that disconnection and announcing that it was disabled. The network rate limits these messages, so this channel disabled update can stall for a while (up to 24 hours?) if it went out. I described this problem in #5980 but later found this weird behavior which causes this whole issue. When it's fixed the channels will get disabled much less often and the overall network stability and liquidity should increase. 22 minutes are plenty of time to reconnect even when Tor slows down or drops connections.

These are the recent log entries. According to @joostjager the channel was disabled two days ago, see screenshot below.
2021-12-05 11:06:38.688 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) disabled [detected]
2021-12-05 11:06:41.011 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) enabled
2021-12-05 14:08:23.193 [DBG] NANN: Channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) already enabled, canceling scheduled disable
2021-12-05 17:37:40.492 [DBG] NANN: Channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) already enabled, canceling scheduled disable
2021-12-08 11:14:52.123 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) disabled [detected]
2021-12-08 11:15:24.316 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) enabled
2021-12-08 15:27:31.031 [DBG] NANN: Channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) already enabled, canceling scheduled disable
2021-12-10 12:43:02.254 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) disabled [detected]
2021-12-10 12:43:09.022 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) enabled

Reconnections:
2021-12-05 10:47:40.972 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-05 13:49:23.171 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:65308, inbound=true
2021-12-05 16:55:09.526 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:49699, inbound=true
2021-12-05 17:18:40.446 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-05 22:05:11.901 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-07 07:23:43.347 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-07 22:07:44.803 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-08 10:56:24.279 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-08 15:08:30.986 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:9735, inbound=false
2021-12-08 21:13:23.413 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:55009, inbound=true
2021-12-10 12:22:37.474 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:64461, inbound=true
2021-12-10 12:24:09.010 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:64475, inbound=true
2021-12-10 16:39:32.899 [INF] SRVR: Finalizing connection to 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:50131, inbound=true

Yeah, it matches the logs. This is what my timestamp corresponds to:
2021-12-10 12:43:02.254 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) disabled [detected]

And this is yours, exactly 7 seconds later (I never received this update):
2021-12-10 12:43:09.022 [INF] NANN: Announcing channel(9586812797de75d4e81c87ec8de045adc2c833b4cf81bc5d5e8dc814e562b71f:1) enabled

The disconnection timestamp roughly matches, 24 minutes before disabling:
2021-12-10 12:18:55.548 [INF] SRVR: Disconnecting from 02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844@<IP_ADDRESS>:55009

And reconnect happened between 12:18:55 and 12:43:02 so the channel should be active at the moment it was wrongfully disabled.

I see, but I don't really understand. The "enabled" message didn't get propagated for some more-or-less valid reason (anti-spam?). However, the channel should not have been disabled in the first place?

Correct and correct!

It keeps happening and is impacting routing quite a bit. Fixing this should be high priority imho.

I keep having this issue with LND peers. Why is this not being fixed?

I keep having this issue with LND peers. Why is this not being fixed?

As a workaround you can change the fee to enable a channel if your node is the one which disabled it.

This issue is still happens. It's relatively easy to reproduce if you simply restart lnd and look in lntop. You will see multiple node disabling their channels to you even if you were offline for 5 minutes. Screenshot (arrows pointing down mean the channel disabled by the peer, those channels were enabled before the restart):

Also! Looks like I found the true reason of this happening from the comments in the config. This default behavior is very wrong imo (if I understand it correctly).
; The duration that a peer connection must be stable before attempting to send a
; channel update to re-enable or cancel a pending disables of the peer's channels
; on the network. (default: 19m0s)
; chan-enable-timeout=22m

; The duration that must elapse after first detecting that an already active
; channel is actually inactive and sending channel update disabling it to the
; network. The pending disable can be canceled if the peer reconnects and becomes
; stable for chan-enable-timeout before the disable update is sent.
; (default: 20m0s)
; chan-disable-timeout=22m

So what gives? The key is The pending disable can be canceled if the peer reconnects and becomes stable for chan-enable-timeout before the disable update is sent! However, they differ only by 1 minute. Which means if you were offline for more than 1 minute you will never make it. The disable timer is 20 minutes by default and it's only cancelled if you were online for 19 minutes, otherwise the channel disabled message is broadcasted. The channel enabled message might broadcast next (19 minutes after you went online) but as we discussed above it might get rate limited and those limits are pretty severe so the channel gets disabled for 24 hours actually.
This looks like a very bad design to me that results in these absolutely unnecessary channel blocks for a long time. A naive and suboptimal solution would be changing the defaults (I doubt anyone tweaks these settings) so that chan-enable-timeout is much lower than chan-disable-timeout, allowing quick restarts (on RPi they might take minutes). But a proper solution (as I see it) is to check if the peer is online before sending the channel disabled message, then delaying this message if the peer stays online and cancelling it altogether if it's online for chan-enable-timeout duration. However, if the peer goes offline again during this grace period we should disable the channel and broadcast the message. This logic is less trivial than the existing one but I believe it's worth implementing. To illustrate how it works now (again, if I understand it right):

peer goes offline, chan-disable-timeout (20 minutes) starts
5 minutes pass
peer comes online, chan-enable-timeout (19 minutes) starts
15 more minutes pass
here we send the disable channel message because the enable timer didn't fire
4 minutes pass
we send the enable channel message as the peer was stable for 19 minutes but this message gets rate limited by our peers and the network is stuck with this channel disabled for 24 hours

What it should be imo:

peer goes offline, 20 minute disable timer starts
5 minutes pass
peer comes online, 19 minute enable timer starts
15 more minutes pass
we see that the peer is online and somehow delay the channel disabled message instead of sending it immediately, we give the peer a chance to prove it's not flapping

Now two things can happen:

4 minutes pass, peer is still online
chan-enable-timeout fires
the delayed message is cancelled, the network is never notified about this incident, some forwardings might have failed but the channel is still seen as enabled by everyone

OR:

2 minutes pass, peer goes offline again => peer is unstable!
we send the channel disabled message immediately and cancel all those timers

Please tell me if it makes sense and if I understand it right.

Another thing is why this update gets stuck. There are two options that control gossip rate limiting:
; The maximum number of updates for a specific channel and direction that lnd
; will accept over the channel update interval.
; gossip.max-channel-update-burst=10
; gossip.channel-update-interval=1m

Which means that we receive 1 non-keepalive (=with different fee/htlc size policy or channel being enabled/disabled) update per minute but can burst it up to 10 (i.e. 10 updates in 1 minute, then we drop all updates until the rate limiting bucket refills). Could it be that this rate limiter is drained quickly because I receive updates for the disabled channel through my other peers (I have about 50)? And because of that when it gets enabled again this message is ignored due to rate limiting?
I will test again in a few days after my currently disabled channels re-enable so I can set the DISC log level to debug, restart lnd and see if it rate limits some updates. DISC generates a lot of spam though, all updates for all channels in the network, hard to find what I need.

@rkfg very detailed analysis! I think your understanding is correct, and I happened to look into these areas lately while fixing itest, particularly this testUpdateChanStatus.

Could it be that this rate limiter is drained quickly because I receive updates for the disabled channel through my other peers (I have about 50)? And because of that when it gets enabled again this message is ignored due to rate limiting?

No you won't. The rate limiter limits the channel updates per channel per direction, which I want to change into per channel per relaying node per direction, which requires a spec change. Meanwhile, you can view every channel update as uniquely identified by the combination of channel ID + direction + timestamp. So even if multiple peers send you the same channel update, only one will be taken into account by the rate limiter and the rest will be ignored.

we send the enable channel message as the peer was stable for 19 minutes but this message gets rate limited by our peers and the network is stuck with this channel disabled for 24 hours

This would only happen if it's been rate limited. By default it's 10 updates per minute, which means technically we allow our channel peer to go online and offline 5 times per minute. I think that is a lot. I also think this may not be a good peer.

we see that the peer is online and somehow delay the channel disabled message instead of sending it immediately, we give the peer a chance to prove it's not flapping

I believe there are multiple ways to achieve this. We can increase our burst limit, increase disable timeout, or decrease enable timeout. But my question is, to what degree? Suppose we have this preposterous setup wherechan-disable-timeout=60m, chan-enable-timeout=10m, --chan-status-sample-interval=1s, if the peer is offline for 59 minutes and comes back, do we pause disabling the channel and wait for 10 minutes to see it enabled?

the delayed message is cancelled, the network is never notified about this incident, some forwardings might have failed but the channel is still seen as enabled by everyone

I think it's better to notify the network, otherwise, this channel will be attempted and failed, causing it to be less likely to be used again. This is probably an implementation-specific algorithm, but I think in general when a channel edge fails, it will be assigned a lower success probability score during the next attempt. If it's disabled, then it won't be attempted and the probability stays unchanged during the next attempt.
That being said, although I don't know the original ideas behind the default values, I think it's intentional,
chan-status-sample-interval=1m
chan-disable-timeout=20m
chan-enable-timeout=19m

suppose the peer goes offline at 00:00.
00:01: 1 minute sample interval later, the chan disable timer starts.
00:00+x: x minutes later, the peer goes online.
00:21: 20 minutes later when disable timer started, chan disabled.
00:19+x: and 19 minutes later when the peer went online, chan enabled.

Thus we always notify the network about the peer's offline behavior. Basically, when a peer goes offline and online again, we will wait y minutes to enable it again, where y is,

y = x - chan-status-sample-interval - chan-disable-timeout + chan-enable-timeout

I will test again in a few days after my currently disabled channels re-enable so I can set the DISC log level to debug, restart lnd and see if it rate limits some updates. DISC generates a lot of spam though, all updates for all channels in the network, hard to find what I need.

Would suggest patching #7186 as it refactored the logging in DISC to make it more useful. And #7157 may be a related fix.

Thank you for this explanation. I don't really care if the channel is disabled/enabled quickly as long as the enabled message actually goes out and propagate. So far it's not always the case.

if the peer is offline for 59 minutes and comes back, do we pause disabling the channel and wait for 10 minutes to see it enabled?

I suppose, yes. The idea is to only disable if the peer was consistently offline longer than the disable timeout. If it went online even shortly before that we should give it a chance and not spam the network with these updates only to send another update again soon. But if it fails our expectations and goes offline before that 10 minutes interval ends (from your example) and the disable timer has already run out (but it kinda follows from my proposition) then we disable immediately.
I don't remember who gave me that idea of rate limiting that might prevent updates from propagating for a long time but probably it was https://github.com/lightningnetwork/lnd/issues/5980#issuecomment-969607147. Hopefully that issue, which was quickly closed without proper investigation, will also be solved with these PRs as it sounds like exactly what we need.

Haven't read this thread in full detail, but on the surface it reminds me of https://github.com/lightningnetwork/lnd/issues/7073. I had propagation issues too, and applied a patch that fixed it for me.

Yes, looks familiar. I find the rate limiting excuse kinda silly (unless I missed something), it only skips updates if they happen more often than 1 per minute and it allows 10 consequent updates faster than that if they average to 1/min eventually (to put it simply). I find it hard to believe people would update their policies faster than once a minute, it's rather once an hour or so or even less frequent. Closing the issue because it's somehow explained by that isn't good imo, I'm glad you kept digging.

Assuming there is no specific action here and the issue is addressed. Haven't read through the complete thread, but the prs linked in the comments as patch suggestions etc. are all merged.
Pleas reopen if the issue persists. thanks.


Custom View Overlay fix

Custom view overlays do not display in a viewport.

@calebmshafer @williamkbentley Any idea why this PR keeps failing?

@calebmshafer should we do one of these - #3309 - for release/3.0.0?

@williamkbentley yes, I started it here, https://github.com/iTwin/itwinjs-core/pull/3314. However, haven't had a chance to figure out why it hasn't worked... I think I tried out the build after we fixed those conditions on it but let me try that again to be sure.

@calebmshafer should we do one of these - #3309 - for release/3.0.0?

@williamkbentley yes, I started it here, #3314. However, haven't had a chance to figure out why it hasn't worked... I think I tried out the build after we fixed those conditions on it but let me try that again to be sure.

I'll work on getting that PR completed. I think I know the error and can fix.


example does not build

audio on  main via 🦀 v1.58.0
❯ cargo build
    Updating crates.io index
    Updating git repository `https://github.com/udoprog/minimp3-rs`
    Updating git submodule `https://github.com/lieff/minimp3.git`
    Updating git repository `https://github.com/udoprog/rubato`
error: no matching package named `audio` found
location searched: https://github.com/udoprog/rubato?branch=next#4ca6e6f9
required by package `rubato v0.9.0 (https://github.com/udoprog/rubato?branch=next#4ca6e6f9)`
    ... which satisfies git dependency `rubato` of package `examples v0.0.0 (/home/be/sw/audio/examples)`

The examples use your fork of rubato which uses relative filesystem paths for audio. While that may work on your machine, it does not work on mine. I recommend to use patch.crates-io in ~/.cargo/config.toml to work on libraries locally so you don't accidentally commit Cargo.toml in the repository with relative paths specific to your machine.
I see you have made some contributions to rubato but the branch used by the example uses a pretty big change that is not upstream nor do I see a PR for it upstream. Were you intending to make a PR for that upstream?

Yeah. This is just for experimentation. I don't intend to submit anything upstream until this crate is done. I elaborated more in #3. But, we can certainly make it easier to build the rubato branch by not using relative paths!

Example should now build, and they've been reconfigured so that they shouldn't break as easily. They now depend on the git version of the crate rather than a published one.


Aws ecs

Description
Changes proposed in this pull request.
Test plan
Outline the test plan used to test this change before merging it.

Fixes #.

Codecov Report

Merging #433 into master will increase coverage by 0.54%.
The diff coverage is 35.55%.

@@            Coverage Diff             @@
##           master     #433      +/-   ##
==========================================
+ Coverage   57.39%   57.93%   +0.54%
==========================================
  Files          46       46
  Lines        5626     6295     +669
==========================================
+ Hits         3229     3647     +418
- Misses       2093     2281     +188
- Partials      304      367      +63

Impacted Files
Coverage Δ

internal/monitor/instance/docker/monitor.go
54.41% <35.55%> (-2.58%)
:arrow_down:

enforcer/datapath/datapath.go
62.71% <0%> (-0.51%)
:arrow_down:

policy/tagstore.go
94.44% <0%> (+0.32%)
:arrow_up:

policy/types.go
24.79% <0%> (+0.59%)
:arrow_up:

internal/supervisor/iptablesctrl/iptables.go
47.55% <0%> (+3.32%)
:arrow_up:

enforcer/datapath/utils.go
77.08% <0%> (+3.47%)
:arrow_up:

internal/supervisor/iptablesctrl/acls.go
56.6% <0%> (+3.79%)
:arrow_up:

enforcer/datapath/datapath_tcp.go
55.62% <0%> (+3.91%)
:arrow_up:

internal/processmon/processmon.go
61.22% <0%> (+4.67%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0717e21...34b7440. Read the comment docs.


bootstrap slows down dramatically with Cilium installed by helm-operator

Describe the bug
When I attempt installing OpenShift with Cilium using currently published instructions, I get a working cluster.
I have created an operator based on quay.io/operator-framework/helm-operator:v1.2.0 image, and it worked previously, but somehow it stopped working for me. I am not entirely clear what caused this, but something breaks bootstrap and it gets stuck yet cluster installation succeeds eventually.
If I replace helm template steps from Cilium docs with copying of operator manifests, openshift-install create cluster times out (errors appear to be slightly different each time).
Once I leave the cluster for 1 or 2 hours, it eventually stabilises. It appears that due to the complexity of the bootstrap process, it's quite easy to end up in a situation where one of the operators is acting up which may result in the whole system being unstable for much longer than expected.
In one the cases I have worked through the following installer error:
ERROR Cluster operator authentication Degraded is True with IngressStateEndpoints_MissingSubsets::OAuthRouteCheckEndpointAccessibleController_SyncError::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError::WellKnownReadyController_SyncError: OAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready
OAuthServiceCheckEndpointAccessibleControllerDegraded: Get "https://<IP_ADDRESS>:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
IngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server
OAuthRouteCheckEndpointAccessibleControllerDegraded: Get "https://oauth-openshift.apps.gcp-ocp46-oss191-fix2.ilya-openshift-[test-2](https://issues.redhat.com/browse/test-2).cilium.rocks/healthz": dial tcp: lookup oauth-openshift.apps.gcp-ocp46-oss191-fix2.ilya-openshift-[test-2](https://issues.redhat.com/browse/test-2).cilium.rocks on <IP_ADDRESS>:53: no such host
WellKnownReadyControllerDegraded: kube-apiserver oauth endpoint https://<IP_ADDRESS>:6443/.well-known/oauth-authorization-server is not yet served and authentication operator keeps waiting (check kube-apiserver operator, and check that instances roll out successfully, which can take several minutes per instance)

Having checked the DNS right away with dig, I go no answer for auth-openshift.apps.gcp-ocp46-oss191-fix2.ilya-openshift-[test-2](https://issues.redhat.com/browse/test-2).cilium.rocks.
Having looked at ingress operator logs, I confirmed there was a long gap before the record was created:
2021-01-25T15:14:51.914Z	INFO	operator.main	ingress-operator/start.go:52	using operator namespace	{"namespace": "openshift-ingress-operator"}
2021-01-25T15:14:51.915Z	INFO	operator.main	ingress-operator/start.go:52	watching file	{"filename": "/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem"}
I0125 15:14:52.966650       1 request.go:645] Throttling request took 1.028863736s, request: GET:https://<IP_ADDRESS>:443/apis/apiextensions.k8s.io/v1?timeout=32s
2021-01-25T15:14:53.670Z	INFO	operator.init.controller-runtime.metrics	manager/manager.go:327	metrics server is starting to listen	{"addr": ":60000"}
2021-01-25T15:14:53.772Z	INFO	operator.init.controller-runtime.manager	manager/manager.go:256	starting metrics server	{"path": "/metrics"}
...
2021-01-25T15:14:54.975Z	INFO	operator.ingress_controller	ingress/deployment.go:78	created router deployment	{"namespace": "openshift-ingress", "name": "router-default"}
...
2021-01-25T15:16:25.354Z	ERROR	operator.ingress_controller	controller/controller.go:235	got retryable error; requeueing	{"after": "1m0s", "error": "IngressController is degraded: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.), DeploymentReplicasMinAvailable=False (DeploymentMinimumReplicasNotMet: 0/2 of replicas are available, max unavailable is 1), LoadBalancerReady=False (LoadBalancerPending: The LoadBalancer service is pending)"}
...

I also checked the DNSRecord with kubectl get DNSRecord -n openshift-ingress-operator default-wildcard -o yaml:
apiVersion: ingress.operator.openshift.io/v1
kind: DNSRecord
metadata:
  creationTimestamp: "2021-01-25T16:19:01Z"
...
status:
  observedGeneration: 1
  zones:
  - conditions:
    - lastTransitionTime: "2021-01-25T16:19:02Z"
      message: The DNS provider succeeded in ensuring the record
      reason: ProviderSuccess
      status: "False"
      type: Failed
    dnsZone:
      id: gcp-ocp46-oss191-fix2-l4kdk-private-zone
  - conditions:
    - lastTransitionTime: "2021-01-25T16:19:03Z"
      message: The DNS provider succeeded in ensuring the record
      reason: ProviderSuccess
      status: "False"
      type: Failed
    dnsZone:
      id: ilya-openshift-[test-2](https://issues.redhat.com/browse/test-2)-cilium-rocks

Another way to illustrate the problem is time age difference between CP & worker nodes:
NAME                                                         STATUS   ROLES    AGE     VERSION
gcp-ocp46-oss191-fix2-l4kdk-master-0.c.cilium-dev.internal   Ready    master   5h12m   v1.19.0+9c69bdc
gcp-ocp46-oss191-fix2-l4kdk-master-1.c.cilium-dev.internal   Ready    master   5h12m   v1.19.0+9c69bdc
gcp-ocp46-oss191-fix2-l4kdk-master-2.c.cilium-dev.internal   Ready    master   5h12m   v1.19.0+9c69bdc
gcp-ocp46-oss191-fix2-l4kdk-worker-a-rhx2t                   Ready    worker   4h5m    v1.19.0+9c69bdc
gcp-ocp46-oss191-fix2-l4kdk-worker-b-qf5bh                   Ready    worker   4h5m    v1.19.0+9c69bdc
gcp-ocp46-oss191-fix2-l4kdk-worker-c-99khp                   Ready    worker   5h5m    v1.19.0+9c69bdc

I am not sure where to look to determine the component in question, I've noticed Cilium helm operator misbehaving and crashloopping, but that's been addressed already.
In theory, it's quite easy to generally say that there is a good chance to disatbilise bootstrap with so many operators at work, but it's hard for me to tell what's holding things up.
Cilium appears to be working properly, and installing without helm operator works just fine. I do suppose helm operator may create a lot of churn on the API, but that's a very instinctive assumption based on activity it logs and I'm not sure if it's significant enough to slow down the API in reality. I do get a sense that bootstrap API node has limited resources and could be easily overwhelmed, but that's another subjective interpretation.
Version
This is IPI OCP 4.6.12 install, same issue occurs on OKD 4.6 and 4.5. If I must, I can provide OKD log bundle.
How reproducible
I'm able to reproduce this very consistently.
Log bundle
log-bundle-20210125154005.tar.gz

So one issue that I came accross while debugging this was that bootkube was stuck trying to apply a manifest, it kept producing errors like this:
Jan 27 09:51:30 gcp-ocp46-oss193-cc-94498-bootstrap.c.cilium-dev.internal bootkube.sh[2403]: "cluster-network-04-cilium-test-0001-echo-b-service.yaml": failed to create services.v1./echo-b -n cilium-test: Service "echo-b" is invalid: spec.ports[0].nodePort: Invalid value: 31313: provided port is already allocated

This manifest is part of Cilium connectivity check suite that I was using in attempt to rule out networking issues due to Cilium, I think bootkube should be more robust in fact of errors like this. @smarterclayton is there an open issue for addressing this sort of failure mode?

So one issue that I came accross while debugging this was that bootkube was stuck trying to apply a manifest, it kept producing errors like this:
Jan 27 09:51:30 gcp-ocp46-oss193-cc-94498-bootstrap.c.cilium-dev.internal bootkube.sh[2403]: "cluster-network-04-cilium-test-0001-echo-b-service.yaml": failed to create services.v1./echo-b -n cilium-test: Service "echo-b" is invalid: spec.ports[0].nodePort: Invalid value: 31313: provided port is already allocated

This manifest is part of Cilium connectivity check suite that I was using in attempt to rule out networking issues due to Cilium, I think bootkube should be more robust in fact of errors like this. @smarterclayton is there an open issue for addressing this sort of failure mode?

The issue with connectivity check manifest I mentioned earlier is not the original issue I've been having.
Looks like my original issue was to do with a different manifest, but it's similar in nature, i.e. Cilium itself gets installed okay, but a non-critical CR manifest (CSV in this case) fails to apply due to a validation error. Eventually bootkube's bootstrap CVO times out, so all is fine in the end, but it takes the extra hour or whatever the bootkube timeout is for bootstrap CVO.
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: "cluster-network-06-cilium-00014-cilium.v1.9.1-clusterserviceversion.yaml": failed to create clusterserviceversions.v1alpha1.operators.coreos.com/cilium.v1.9.1 -n cilium: ClusterServiceVersion.operators.coreos.com "cilium.v1.9.1" is invalid: spec.maintainers: Invalid value: "string": spec.maintainers in body must be of type array: "string"
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Failed to create "cluster-network-06-cilium-00014-cilium.v1.9.1-clusterserviceversion.yaml" clusterserviceversions.v1alpha1.operators.coreos.com/cilium.v1.9.1 -n cilium: ClusterServiceVersion.operators.coreos.com "cilium.v1.9.1" is invalid: spec.maintainers: Invalid value: "string": spec.maintainers in body must be of type array: "string"
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: [#16897] failed to create some manifests:
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: "cluster-network-06-cilium-00014-cilium.v1.9.1-clusterserviceversion.yaml": failed to create clusterserviceversions.v1alpha1.operators.coreos.com/cilium.v1.9.1 -n cilium: ClusterServiceVersion.operators.coreos.com "cilium.v1.9.1" is invalid: spec.maintainers: Invalid value: "string": spec.maintainers in body must be of type array: "string"
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Sending bootstrap-finished event.[#16898] context deadline exceeded
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Tearing down temporary bootstrap control plane...
Jan 25 16:16:45 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Waiting for CEO to finish...
Jan 25 16:16:46 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: I0125 16:16:46.420832       1 waitforceo.go:64] Cluster etcd operator bootstrapped successfully
Jan 25 16:16:46 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: I0125 16:16:46.421956       1 waitforceo.go:58] cluster-etcd-operator bootstrap etcd
Jan 25 16:16:46 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: bootkube.service complete

The issue with connectivity check manifest I mentioned earlier is not the original issue I've been having.
Looks like my original issue was to do with a different manifest, but it's similar in nature, i.e. Cilium itself gets installed okay, but a non-critical CR manifest (CSV in this case) fails to apply due to a validation error. Eventually bootkube's bootstrap CVO times out, so all is fine in the end, but it takes the extra hour or whatever the bootkube timeout is for bootstrap CVO.
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: "cluster-network-06-cilium-00014-cilium.v1.9.1-clusterserviceversion.yaml": failed to create clusterserviceversions.v1alpha1.operators.coreos.com/cilium.v1.9.1 -n cilium: ClusterServiceVersion.operators.coreos.com "cilium.v1.9.1" is invalid: spec.maintainers: Invalid value: "string": spec.maintainers in body must be of type array: "string"
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Failed to create "cluster-network-06-cilium-00014-cilium.v1.9.1-clusterserviceversion.yaml" clusterserviceversions.v1alpha1.operators.coreos.com/cilium.v1.9.1 -n cilium: ClusterServiceVersion.operators.coreos.com "cilium.v1.9.1" is invalid: spec.maintainers: Invalid value: "string": spec.maintainers in body must be of type array: "string"
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: [#16897] failed to create some manifests:
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: "cluster-network-06-cilium-00014-cilium.v1.9.1-clusterserviceversion.yaml": failed to create clusterserviceversions.v1alpha1.operators.coreos.com/cilium.v1.9.1 -n cilium: ClusterServiceVersion.operators.coreos.com "cilium.v1.9.1" is invalid: spec.maintainers: Invalid value: "string": spec.maintainers in body must be of type array: "string"
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Sending bootstrap-finished event.[#16898] context deadline exceeded
Jan 25 16:16:44 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Tearing down temporary bootstrap control plane...
Jan 25 16:16:45 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: Waiting for CEO to finish...
Jan 25 16:16:46 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: I0125 16:16:46.420832       1 waitforceo.go:64] Cluster etcd operator bootstrapped successfully
Jan 25 16:16:46 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: I0125 16:16:46.421956       1 waitforceo.go:58] cluster-etcd-operator bootstrap etcd
Jan 25 16:16:46 gcp-ocp46-oss191-fix2-l4kdk-bootstrap.c.cilium-dev.internal bootkube.sh[2389]: bootkube.service complete

I've reported the issue with service manifest here: https://bugzilla.redhat.com/show_bug.cgi?id=1933263
From talking to @vrutkovs, it sounds like user-provided manifest errors cannot really be validated, so I'll close for now.


How to remove bottom shadow in the layout?

Hi, many thanks for thisone. Im new as developer and new here (github) too. I just want to know how to remover that bottom shadow in the layout maked for text visibilty. Thanks again

Shadow as in? You want to remove the blackish tint from the bottom text?

Yes sir, that blackish tint for whites points and text. i dont need text rigth now and for better performance with backgorund is better without tint

Would you like to fix that? or should I do that?

If you want or maybe I can learn to work with GitHub and understand the AutoImageFlipper code to improve it. I will continue paying attention to this. thanks again

That's great, I'll add this feature myself as of now, you may have a look at the changes that I'll make

That's great, I'll add this feature myself as of now, you may have a look at the changes that I'll make

Done.
Go through the Readme.md once to get all the functionalities of the library


Need a 64-bit Windows release of libtensorflow.dll with AVX2 support.

I've written a Windows .Net application that runs a particularly complex model, and execution time is critical.  Right now it doesn't run fast enough (500ms per run, when we'd need more like 250ms) and I'm hoping that support for my AVX2 processor extensions would help cut that time down.  Sadly, the version of libtensorflow.dll that comes with your TensorFlowSharp NuGet package isn't compiled with AVX2 support.  I looked at trying to compile it myself, but it looks like a dependency-laden, cross-platform clusterf*ck.  I'm unlikely to be successful with that route (especially since I develop in Windows, not Linux).  How else would I get a copy of this Windows DLL with AVX support?  I'm kind of desperate...  Thanks in advance!
Frank Edgley.

It is possible to build the libtensorflow.dll for windows from source, but when you do its bound to the specific version of CUDA, CuDNN and the compute capability of the graphics card on your target machine.    I could send you one prebuilt for CUDA 9.2, CuDNN 7.1.4 compute capability 6.1 (NVidia 1070) but its half a gig in size (i also have a CUDA 10, CuDNN 7.4 compcap 6.1 at 380meg).
I built mine using the following instructions
<EMAIL_ADDRESS>when you get to the point of building it (Step 11?) dont build the pip package, use the following command to build a windows compatible dll
bazel build -c opt --copt=/arch:AVX2 --config=cuda --copt=-nvcc_options=disable-warnings -k //tensorflow:libtensorflow_cc.so
The output libtensorflow_cc.so is libtensorflow.dll in disguise - just rename it and stick it in your bin directory.  There are no additional external dependencies with this dll

AldoZ> Thanks for the help!
Yeah, I'm planning on running on an industrial machine without any significant GPU, so I don't think that CUDA support is necessary.  But I do need to leverage the AVX2 on the CPU.  I followed the linked article.  Although "Step 6: Install NVIDIA CUDA 10.0 and cuDNN 7.4 (for GPU acceleration)" failed, I skipped it cause (as I mentioned) I don't think I need GPU support.  I modified the build line you provided slightly to (hopefully) accommodate for that...  but I'm getting errors:

This is pretty far outside my comfort zone, but I'm left with no alternatives to get this DLL.  Any suggestions are most welcome.  Thanks in advance!

Found this...
https://github.com/tensorflow/tensorflow/issues/24385
Downgraded Bazel to 19.1 and it seems to be getting farther...

I've built a cpu only v1.12 libtensorflow.dll and added it to my OneDrive - https://1drv.ms/u/s!AvgVt9-64BPph90t0Ltxv01Me4hSSw - the link will be live only until 15/03/2019.

I've built one.
Build environment：
TF:1.13.1
CUDA:10
CUDNN:7.5
ComputerCapabilities: 6.1 (GTX 1080Ti)
downloadlink:
TF-1.13.1_gpu_avx2  password: esji

I'm struggling to build TF C api dll for working on an old machine: no GPU (no CUDA), and no AVX or AVX2 (it has an old Pentium processor). Is this possible? If you are building successfully, could you build it for me?

I'm struggling to build TF C api dll for working on an old machine: no GPU (no CUDA), and no AVX or AVX2 (it has an old Pentium processor). Is this possible? If you are building successfully, could you build it for me?

If your CPU has no AVX or AVX2 ，you have to use TF version below 1.6(not included)see reason. If you are a windows user, you cann refer to the "libtensorflow_cpu.sh" file under  "tensorflow/tensorflow/tools/ci_build/windows/"  to build the CPU version, and use "common_env.sh" under "tensorflow/tensorflow/tools/ci_build/windows/bazel/" to configure your environment information.
Note: If you use this way to build TF you need to download MSYS64 and BAZEL


Release not showing up on packagist

Hi Kayla et al,
I created a v2.2.0 release for this yesterday but packagist is still showing v2.1.1. Probably because this package hasn't been set to auto update.
Can someone with access to the omnipay account on packagist give it a prod please?
Del

Prodded. I don't know why auto-updates aren't working though, it might be something that needs to be fixed on the github side (and I'm not an admin on this repo).

Thanks.


MudBlazor popover does not include in calculation left and top values in position:relative elements

Bug type
Component
Component name
MudSelect (mudpopover)
What happened?
MudBlazor components can be placed inside elements marked with both position:relative and a left and top offset.  Here is an example of modifying the default MudBlazor (wasm) template's index.html page with my example:

Loading...

Note the position:relative and the left:100px offset.  When using a MudSelect component you will note the mudpopover calcuation is incorrect (see below it is not doing the right thing with its calculation of that 100px value):

I have spent a considerable effort debugging this as it was causing an issue using MudBlazor in a wider system.  I suggest:
a) The team consider improving the way mudpopover offsets are calculated to correctly handle position:relative used with left and top.
OR
b) If the team does not want to address this issue, I then suggest we clearly document that it is important these CSS features are not used on any html elements containing MudBlazor components.
Expected behavior
User would expect mudpopover to correctly calculate the position (such as when MudSelect displays its drop down list).
Reproduction link
https://try.mudblazor.com/snippet/GEwnYPRamoHuIrya
Reproduction steps
See my example above:

Modify default MudBlazor template index.html like this:

Loading...

Add a MudSelect to a page and note the drop down is not correctly positioned.

Relevant log output
No response
Version (bug)
6.1.8
Version (working)
No response
What browsers are you seeing the problem on?
Chrome, Microsoft Edge
On what operating system are you experiencing the issue?
Windows
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Mira, el popover del select no se ubica bien y a medida que hago scroll se mantiene fijo, esto daña mi aplicacion por completo, no se que hacer!!!!

Lo pude solucionar, es importante que MudThemeProvider este en la parte superior del MainLayout

@ScarletKuro
This issue is resolved

Fixed in v8.


Allow to render styles to multiple documents

Feature Request
Problem description
Fela uses own per-renderer cache for style nodes, it means that if you will try to render something into a separate document you will fail 💣 Styles will be inserted into the parent document 🎲
See CodeSandbox for more details.
Repro
https://codesandbox.io/s/62m20qlpn
Proposed solution
I tried to combine our existing two renderers (felaRenderer & felaRtlRenderer) to one renderer, but we will still have broken transitions (see #516).
So, we will still need two renderers 😭
Expose ltrRenderer and rtlRenderer on Provider
Original idea was to expose only renderer prop, but it failed, so we need to expose two renders.
import { createRendererConfig, Provider, themes } from '@stardust-ui/react'

const subRenderer = createRendererConfig()
const subRtlRenderer = createRendererConfig()

ReactDOM.render(
  <Provider ltrRenderer={subRenderer} rtlRenderer={subRtlRenderer} theme={themes.teams}>
    <App />
  </Provider>,
  document.getElementById('root'),
)

Create renderers on demand
Generic idea is to pass current document via context and if it's different, create new renderers.
Pros

Do not expose anything weird via API

Linking: https://github.com/rofrischmann/fela/issues/693


Some EEBus devices remain connected (and produce events) after calling CloseConnection

When connected to some EEBus devices, calling CloseConnection (as triggered through eebus-go's UnregisterRemoteSKI) doesn't completely tear down the SHIP/SPINE and the remote device continues providing read events.
I've attached a log with the DEBUG&TRACE loglevels as logged by eebus-go. UnregisterRemoteSKI is called on linke 15 which triggers the send of a "connectionClose" message to the remote device which appears to never confirm the connectionClose announcement and instead continues to provide measurement notifications.
Lines starting with "decoded: " trace calls of eebus-go functions/arguments.
eebus_go_unregister_trimmed.log
I don't think we're doing anything "wrong" here per se (besides potentially accepting/forwarding events after sending a close notification), but we should probably adapt our logic to handle this case as well. We could potentially add a maxTime element to our closeConnection message and then trigger a timer after maxTime ms to call c.dataWriter.CloseDataConnection.
I could probably create a PR doing this, but we should maybe discuss what to set maxTime (or whether to make it configurable) etc.

Thanks for the details. The reason is that the process details of SHIP <IP_ADDRESS>.2 are not implemented. When closing the connection the stack does NOT set a maxtime and also doesn't implement a timer of the maximum allowed time to close the connection.
I would suggest to send a maxtime of 0 and close the connection 500ms after sending the request as it is done in if the stack receives an connection close announcement in handshake.go line 18


Bonsai accounts don't have equals() or hashCode() implementations

PR description
Unlike StateTrieAccountValue, BonsaiAccount is missing overrides for equals() and hashCode(). This is causing unnecessary entries in the trie logs for accounts that have been loaded but which haven't changed.
An example is invoking a method that only emits an event but which doesn't update storage. The account is loaded, the method is invoked, but nothing in the account has changed. It is then (unnecessarily) recorded in the trie-log for that block.

We might actually rely on this unintended behavior for linea-shomei zk-trielog plugin.  The primary difference between the TrieLogFactoryImpl and ZkTrieLogFactory implementation is that we add block number and zero reads to the trielog.  But the zk state needs to know about updates and reads.
The value of having reads in the trielogs goes beyond just having a diff to apply, but rather is a log of all state access as well as modifications.  This is useful for historical block tracing (very cheap historical/archive worldstate).
I don't know that adding equals and hashCode implementations will break this.  Before we merge we should probably either assert that either this doesn't break anything or make explicit provision for persisting reads (and zero reads) if we relying on unintended consequences of not having these methods
@matkt  for visibility

Thanks for the comments @garyschulte I did wonder if the existing behaviour was in any way intentional.
In the case I've given in the PR description, there isn't even a state access, just an event. I'm not sure how realistic that is but it does need considering in terms of whether we want try logs for those types of invocation.
If the answer is "this is intentional/needed/useful" then I'm happy to close the PR with that explanation. I think I'll need to take your call on whether that's the case?

It's something we saw while coding Verkle, but we decided not to include it for now until we have time to test that it won't have any side effects on the trielog shipping part. I think before we merge it, we will need to conduct some tests.

I've addressed the code comment re. moving most of the equals() implementation into DiffBaseAccount. Regarding any additional testing required to prove that the change hasn't regressed anything or caused problems for verkle implementation, I think I'll need to wait for feedback from @matkt

@garyschulte nudging on this one in case you have a view on whether this is OK to merge?

I haven't been able to test yet, I had a lot of work. If it's a blocking issue for you I can try to prioritize the test

No massive hurry @matkt, just useful to know at some point if we think the direction is to remove non-changed accounts from trie logs or not


Use s3a:// filesystem instead of s3n://.

Removed jets3t dependencies.
In run_tests.sh, quote variable in -n test.

I'm not very familiar with the s3a filesystem. From what I read online ( https://wiki.apache.org/hadoop/AmazonS3 ), s3a is backwards compatible with s3n, but s3a files may not be readable through s3n urls. We have plenty of downstream deps on s3n (hive tables, hadoop jobs, etc.) that will break if we change the url. For now I would prefer to stick with the wider supported fs.


The left hand pane and the dropdown orders are different

The left hand pane and the dropdown orders are different

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4f2add0f-2806-8bfa-92d5-cc3fdd1e1055
Version Independent ID: c572b535-a780-fba3-0845-f5127d98d7d6
Content: UWP, WPF, and WinForms SDK Troubleshooting - Visual Studio App Center
Content Source: docs/sdk/troubleshooting/uwp.md
Service: vs-appcenter-sdk
Sub-service: app-center-general
GitHub Login: @elamalani
Microsoft Alias: emalani

Thanks. We will fix that.


Azure credentials issue is resolved

GitHub issue https://github.com/Sunbird-cQube/community/issues/631


Times not shown for stops with times

I working with the GTFS at data.trilliumtransit.com/gtfs/beloittransit-wi-us/beloittransit-wi-us-test.zip
See timetables and config.json below.
Notes

timetable_stop_order.txt is present but incomplete. I'm guessing this is not a problem but if timetable_stop_order needs to include references to all timetables, that could be my problem.
the config.json has "showOnlyTimepoint": true, and for the timetables in timetable_stop_order.txt, only the timepoints are in the file (I also tested including non-timepoints for those timetables in the file, but that forced them to be shown)
only 12272016 forward timetables are included in zip
these trips are exported as a group from our system, so I'm moderately confident that all the empty timepoints have times (and I've manually checked a few)

any ideas? cc @nomeq
beloit1.zip

Checking the new timetables, I found a lot of code that simply got converted to strings in the html, e.g. <col class="run_#{idx} #{(trip.trip_short_name) ? 'trip_short_name_' + trip.trip_short_name : ''}" id="trip_id_#{trip.trip_id}">, but uncertain what is wrong.
I ran the gtfs through an old version of gtfs-to-html, and the timetables turned out fine. I have not progressively run commits to figure out at which point it breaks, but I had a local directory with commit 801a13a55519b2ca24cc88a21b7f158be7159e69 still running and it produced accurate timetables (attached).
beloit2.zip

I just published version 0.5.0 which should solve most or all of these issues.
Try generating it with this latest version and let me know if you have any issues.


Feature request: Cookie domain rewrite

When we rewrite URL that has a response to set a cookie (i.e. response has set-cookie header), sometimes, the response will specify to set the cookie in the remote domain, hence locally, the expected cookie isn't present.
This feature request is similar to cookieDomainRewrite option use case at node-http-proxy library (more details at its PR). I hope you could consider it. Thanks for your time 😁

Thanks, this sounds useful - I will look into it.
In the meantime, please take a look at this wiki page, hopefully it contains enough information for you to try adding the feature yourself.

Do you have documents on how to get started to contribute to this repos? Seems I can't find the Contributing guideline. I have questions like, which repo to fork and how to set it up locally. Thanks

https://github.com/chimurai/http-proxy-middleware supports option.cookieDomainRewrite to exactly this purpose. That's why I migrated to Connect/Express.js middlware. lws lags behind a century, it is better not to waist time with unsupported tool.


corpus explorer unavailable?

Awesome tool!  Can't wait to dig into it more.
The options for selecting corpus search on the demo page aren't available to click on (when I do so, I get a red circle with a slash through it).  I'm using chrome.
Am I missing something?

You will need to click on a token in order for the tool to know which token you care to search for. The buttons require one token and at least one head to be selected to be activated.


dynamic dispatch in Proj.proj_trans

Hello,
I'm using Proj in some code where I'm trying to avoid allocations and dynamic dispatches for performance. To assist with this, I'm using the AllocCheck.jl packages, which finds allocations and dynamic dispatches in the code. So this example:
using Proj
using AllocCheck

const pj = Proj.proj_create_crs_to_crs(
    "EPSG:4326",  # source
    "+proj=utm +zone=32 +datum=WGS84",  # target, also EPSG:32632
)
const a = Proj.proj_coord(12, 55)

Proj.proj_trans(pj, Proj.PJ_FWD, a)

@check_allocs checkf(a) = Proj.proj_trans(pj, Proj.PJ_FWD, a)

try
    checkf(a)
catch err
    err.errors
end

gives the following error message:
Dynamic dispatch in ....julia/packages/Proj/YfK57/src/libproj.jl:494
  | @ccall libproj.proj_trans(P::Ptr{PJ}, direction::PJ_DIRECTION, coord::Coord)::Coord

Stacktrace:
 [1] proj_trans
   @ ~/.julia/packages/Proj/YfK57/src/libproj.jl:494 [inlined]
 [2] var"##checkf#260"(a::Coord)
 ....

I don't know enough about @ccall to be able to tell what the issue is, if there is one. Any suggestions would be appreciated!

Did you try looking at a profile? E.g. ProfileView.jl? It will show allocs and instability in yellow and red

I tried looking at at with JET, but nothing shows up there. It seems like it's something specific to that @ccall line, and my guess is that maybe JET doesn't analyze @ccall but maybe AllocCheck.jl does, but I really don't know.

I mean a sampling profiler like ProfileView.jl. Thats the best tool for this job because it will instantly show where the allocations are coming from. Just run that code a lot of times inside a function with a loop in it.

I mean, AllocCheck is pretty clear about where it's happening, line 494 of Proj.jl. I just don't know why it's a dynamic dispatch, or how it would be fixed.

But... the profile will show if its type instability (boxing) causing allocation, or just regular allocation, red or yellow in the profile (Likely red given dynamic dispatch warning).
They are totally unrelated reasons for allocation and totally different fixes.
If its red, use Cthulu.jl @descend to look around at types. descend_clicked() after clicking the profile can help get there quicker.
If yellow, investigate possible allocations like Ref in code.
Compared to the ProfileView/Cthulhu combo AllocCheck is a blunt instrument. You need to see the context.

I can't see this allocation or any instability in ProfileView from my end.  I do see the same error from AllocCheck, though.
Perhaps the "dynamic dispatch" here is really triggering on the @ccall?  A BenchmarkTools benchmarking of the code in that function shows no allocations:

julia> @benchmark Proj.proj_trans($pj, $(Proj.PJ_FWD), $a)
BenchmarkTools.Trial: 10000 samples with 911 evaluations.
 Range (min … max):  114.572 ns …  2.863 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     120.518 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   123.166 ns ± 33.619 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▃  ▂▃▅█▄▄▄▆▆▅▄▂▂▁▁▁▁▁                                        ▂
  █▆▄█████████████████████▇▇▇▇▇▇▇▆▅▆▆▆▆▆▅▆▅▅▆▆▆▅▅▄▅▄▅▄▄▄▃▄▄▄▄▄ █
  115 ns        Histogram: log(frequency) by time       158 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

I can't see this allocation or any instability in ProfileView from my end

If there is nothing in the profile that suggests there isn't actually a problem. Time of 100ns and no allocs also suggests that.


hexo g error

I used command "hexo g" and then there was an error:
Unhandled rejection ReferenceError: /Users/zeyang/Documents/workspace/bb/themes/Anisina/layout/page.ejs:127
    125|

    126|

 >> 127|             <%- page.content || body %>
    128|

    129|
    130|             <div class="

body is not defined
    at eval (eval at <anonymous> (/Users/zeyang/Documents/workspace/bb/node_modules/ejs/lib/ejs.js:242:14), <anonymous>:30:5035)
    at eval (eval at <anonymous> (/Users/zeyang/Documents/workspace/bb/node_modules/ejs/lib/ejs.js:242:14), <anonymous>:30:12609)
    at /Users/zeyang/Documents/workspace/bb/node_modules/ejs/lib/ejs.js:255:15
    at Object.exports.render (/Users/zeyang/Documents/workspace/bb/node_modules/ejs/lib/ejs.js:293:13)
    at Hexo.<anonymous> (/Users/zeyang/Documents/workspace/bb/node_modules/hexo-renderer-mathjax/index.js:14:24)
    at Hexo.tryCatcher (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/util.js:26:23)
    at Hexo.ret (eval at <anonymous> (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/promisify.js:163:12), <anonymous>:13:39)
    at /Users/zeyang/Documents/workspace/bb/node_modules/hexo/lib/hexo/render.js:51:21
    at tryCatcher (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/promise.js:507:31)
    at Promise._settlePromiseAt (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/promise.js:581:18)
    at Promise._settlePromiseAtPostResolution (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/promise.js:245:10)
    at Async._drainQueue (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/async.js:128:12)
    at Async._drainQueues (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/Users/zeyang/Documents/workspace/bb/node_modules/bluebird/js/main/async.js:15:14)
    at processImmediate [as _immediateCallback] (timers.js:383:17)

hexo: 3.1.1
os: Darwin 15.4.0 darwin x64
http_parser: 2.6.2
node: 5.7.0
v8: <IP_ADDRESS>
uv: 1.8.0
zlib: 1.2.8
ares: 1.10.1-DEV
icu: 56.1
modules: 47
openssl: 1.0.2f

body is not defined ,cause in you hexo blogs dirctory , you must be sure you have page files or post files . advice you read hexo Docs hexo Doc , and my theme README .
if this problm still exist , please give you git , i will help you find this proplem ：）

Thank you for response!
I solved this problem by deleting the "tags" and "categories" folders in source.
I don‘t know why.   =。=

Another question.  If I don't use Qiniu images CDN, how can I config my img?

@AsiamCn I known this problem，you don't need delete tags and categories folder , you just need edit in folders  index.md file , in front matter add  some layout: tags , this tags page will be show tags layout , in fact , another way  you just edit tags index.md write some text just through hexo g :)
about qiniu
you just normal add a images url , like this:
if you images from internet: http://img3.yxlady.com/xiaomao.jpg, you just add header-img: http://img3.yxlady.com/uploads/intro/20120710/1500150.jpg , etc
or use hexo default img path
header-img: img/someImages.jpg
have fun : )

Thank you for your help!! I can see my tags page now.
But there still some problems.......=。=
When I set header-img: imgs/post-default.jpg ,it doesn't show a picture.
and I checked the html, it looks like this:

I guess my image path is http://<IP_ADDRESS>:4000/imgs/post-default.jpg not http://<IP_ADDRESS>:4000/tags/imgs/post-default.jpg.so It can't get the image.

I solved this problem by setting header-img: ../imgs/post-default.jpg and post-default-img: ../../../../imgs/post-default.jpg. But it is not so beautiful...

I also don't know why some configs didn't come into effect in hexoPath/themes/Anisina/_config.yml, but when I copy them to hexoPath/_config.yml, they works.
what's the difference?
Thank you for your time!

NO imgs , is img/ path : D

first , git pull my theme new version. today morning update.
use img/someImages.jgp like this .
hexo clean && hexo g && hexo serve , check out : )

if you read my theme README, you will know why copy this _config.yml in to your blog dircotory by work be fine :)

Oh,Thank you !!


Support SSL

Both for the listening server and the reverse proxy thing.

Could this be an optional feature so that those of us who would run rouille behind something like nginx can avoid ssl-dependencies?

This seems to just hang: https://gist.github.com/CamJN/4369785b198a6dd513f75c8ad24023bd

@glennw What platform(s) did you test on? I can't get a super simple test project to actually serve a response on macOS, it hangs for me after making the tls connection and receiving the request. (see above gist for my non-working example).

@CamJN I only tested on Ubuntu 17.10 and 18.04.


Add support for clipboard endpoints

https://github.com/appium/java-client/pull/855

Hi,
May I know in which version this fix will be released. I am not seeing these changes in <IP_ADDRESS> Beta version.
Thanks,
Praveen


Vagrant up no longer works after plugin install

Hi, I am new to the plugin so I do not mean to offend. I'll just show what I enter into the terminal
$ vagrant plugin install vagrant-hostmaster
Installing the 'vagrant-hostmaster' plugin. This can take a few minutes...
Installed the plugin 'vagrant-hostmaster (0.6.2)'!
vagrant up --no-provision
Vagrant failed to initialize at a very early stage:
The plugins failed to load properly. The error message given is
shown below.
cannot load such file -- vagrant/command
Now when I uninstall the plugin, I can start using vagrant again...
Any ideas? I would like to use the plugin!

Somehow ended up w an outdated version, sorry

I installed vagrant-hostmaster instead of installing vagrant-hostmanager :P. After removing vagrant-hostmaster things worked fine. I guess the 0.6.2 version of the plugin has issues by defualt ?