Dev blog

Message boards : Cafe : Dev blog
Message board moderation

To post messages, you must log in.

AuthorMessage
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 15 - Posted: 9 Jan 2024, 8:33:00 UTC

Preparing beta workload
I used a database to store files. Now switched to simple files. It is more stable.
Now this is "one file, one job". What happen when job amount increase? Just tar files. For example one job, 100 files. Tar is a good solution (merge many files to single).
Github repo
ID: 15 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 16 - Posted: 11 Jan 2024, 11:27:53 UTC

If we want to storage files for a long time and compute hashes continuously the usual solution is to use trickle messages. But BOINC saves these message to disk at first. Many writes are unwanted for MicroSD for example.
SO i will send these message directly to a scheduler. Yeah, it requires more time to understand how it works in a low level.
ID: 16 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 18 - Posted: 20 Jan 2024, 10:44:31 UTC

Beta is ready. At the moment, the task consists of loading an image file, reading ten random blocks and returning their hashes to the server. The interval between hash calculations is one minute.
The remaining task is to minimize disk writes. In real time we need to receive values for randomness and send hashes also in real time (with a small queue).
ID: 18 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 20 - Posted: 10 Feb 2024, 7:51:25 UTC

I found solutions for all remaining problems.
Now I'm trying to find a team. This is the top priority at the moment!
ID: 20 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 21 - Posted: 12 Feb 2024, 3:51:44 UTC

A case from this project.
I test the workload, read the picture from the job / task i received.
BOINC has two approaches to work with a file: with a copy and with some link to the file. In our case, it’s better to avoid unnecessary copying, so I choose the link.
I assumed that this was a Linux soft link and would be read transparently. But it's not that simple :)
Read errors indicate insufficient file size. It is unlikely that we will work with files smaller than a megabyte, this is the premise.
Ok, I’m reading the first 100 bytes of the file to see the real contents of the file in the log... it’s empty! Amazing link - I thought and read the documentation :)
The link turned out to be a file with an XML tag from which I extracted the path to the image file.
The second non-obvious behavior was the output of this tag to the log - all tags are cut out by the log generator, because the log itself is in XML format. But I expected behavior with escaping special characters: "<", ">" ...
When two uncertainties overlap, testing and debugging take a long time. Because it seems that we are looking for one unknown, but here is their logical sum.
ID: 21 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 22 - Posted: 19 Feb 2024, 15:07:56 UTC

I've designed the hashes workflow. This is enough hard task so i'll finish this code after first launch. I'll publish this code later.
ID: 22 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 23 - Posted: 8 Jun 2024, 8:13:24 UTC

A co-founder has been found. We change the domain to a more self-explanatory: frostydata.com
ID: 23 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 27 - Posted: 14 Jun 2024, 12:38:46 UTC

Here's a snapshot of reality:
The co-founder is away on family business and will be unavailable for a couple of weeks;
He found a candidate for the team (a highly technical person, related to the SOlana project).
I sent him a link to the payload source (DataFreezer application) and a link here to the infrastructure.
I get the question: “are there other examples of projects.” This is the level of a simple HR question, who knows how to open a page in Github and look at the number of repo... (whoever has more repo is a highly professional professional!
For a junior, this sometimes works. Professionals usually have other things to do than create "Hello world" on Github to impress their boss at their first job.

I’m thinking about what to compress for example in our project (.iso image of Debian) and what generator of interesting pictures to create?..
Playing the best comic book artist?
Step 1:
There is a text for the first picture: “the hero is standing at a crossroads and wondering where to go next.”
Participants can change (how?) the first text to create a second picture that develops the plot: “the hero turned left and for this reason teleported to the right.”
The picture and text with the maximum number of pluses becomes the basis for the third picture. Step 1.
ID: 27 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Techter
Project administrator
Project developer

Send message
Joined: 7 Jan 24
Posts: 27
Credit: 0
RAC: 0
Message 28 - Posted: 18 Jun 2024, 2:32:49 UTC

I think further.
Our project searches for blocks of data in images. This principle works and it is quite simple.
But the results are visible only on large amounts of data. Currently, the payload only looks for matches in the first 6 bytes of the hashes. This can be tested on a small amount of input data (.iso image) and on a small set of test hashes (several tens of millions).
The idea for this project was born from the premise: many coins count a huge number of hashes, but there is no useful work. The value of these coins is equal to the value of paper money in a country with a zero economy, the coin has no intrinsic value and no base. Gold has its own value. The paper US dollar has a base: the US economy.
How to create a coin with its own value? How to iterate through hashes and do useful work?
I found a solution. The problem is that we don't have a clear way to show "how it works". I'm currently writing a client program that will show the actual work in some obvious way.
P.S. even gold has no real value. It is present only in people's heads. But it is the most real value of the units of universal exchange; Only natural exchange is more realistic: when people exchange food for ammunition, ammunition for water, housing for food :) But it is assumed that the World will not slide there :)
ID: 28 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Cafe : Dev blog

©2024 We're making it together!