Jordan Petridis
7eec01a52e
Minor rename of DataError::Bail.
2018-02-05 19:05:40 +02:00
Jordan Petridis
44ebe46f10
Replace last bail! invocations.
2018-02-05 19:00:31 +02:00
Jordan Petridis
008f57bec4
Create a ParseEpisodeError and replace bail! invocations.
2018-02-05 18:51:21 +02:00
Jordan Petridis
d0ae5a4198
Typo fix.
2018-02-05 18:33:12 +02:00
Jordan Petridis
064c2b4be0
Start creating custom errors instead of using bail! macro.
2018-02-05 18:25:56 +02:00
Jordan Petridis
34d7391363
Make a counterfeit rss Error.
2018-02-05 16:37:50 +02:00
Jordan Petridis
e84e7df3f1
Remove unwrap call.
2018-02-05 15:59:34 +02:00
Jordan Petridis
8ba9f928d6
Switch to using DataError instead of failure::Error.
2018-02-05 02:43:56 +02:00
Jordan Petridis
435ce05ac7
Add rest of the From impls for the Error Type.
2018-02-05 01:53:01 +02:00
Jordan Petridis
ede4c21e30
Merge the Errors into a single Enum.
2018-02-05 01:39:50 +02:00
Jordan Petridis
0892fe26ba
Manually implement From Trait for the DatabaseError.
2018-02-05 01:20:01 +02:00
Jordan Petridis
1085d468b9
cargo fmt
2018-02-04 18:45:19 +02:00
Jordan Petridis
de43cae015
Switch rest stuff of data/downloader to Failure Crate.
2018-02-04 17:36:27 +02:00
Jordan Petridis
f9f015a211
Hammond-data: switch database module to use Failure.
2018-02-04 16:33:10 +02:00
Jordan Petridis
d131c279b9
Add batch indexing fallback.
...
If an RSS feed triggered a unique constrains violation, none of it's
items would be indexed resulting in a Podcast without episodes.
This adds a fallback that indexes each item individually if the batch index fails.
2018-02-03 16:02:42 +02:00
Jordan Petridis
92d01ec51a
Feed: enchance get_stuff unit test.
2018-02-03 15:55:26 +02:00
Jordan Petridis
e8ae0b0251
Thats a hack, a terrible one, but without it the user
...
would have to manually click refresh if the user that was added
happen to redirect somewhere else.
The problem is in the hammond-data/src/models/source.rs
fn request_constructor. It has to do with futures and lifetimes
and I am not sure how to tackle it yet.
2018-02-03 00:06:02 +02:00
Jordan Petridis
80fd4e9fc5
Source: Set Etag and last modified to None upon redirect.
2018-02-01 19:35:35 +02:00
Jordan Petridis
7b62ef203d
Source: Pass owenership around instead of &mut, cause futures.
2018-02-01 15:55:19 +02:00
Jordan Petridis
2f062afb07
Clippy suggestions
2018-01-29 19:09:53 +02:00
Jordan Petridis
815639fe5f
Add an order by tittle asc clasuse to the Podcast query. Address #34 .
2018-01-29 12:30:13 +02:00
Jordan Petridis
b1c3b4f9cc
Pipeline: Fetch sources without etag support fist as they are more likely to need parsing.
2018-01-29 12:20:21 +02:00
Jordan Petridis
332a439b7a
hammond-data: Factor out save method of diesel models into a Trait.
2018-01-28 22:09:06 +02:00
Jordan Petridis
fb5264c479
NewSource: Rename into_source -> to_source and take reffence &self instead.
2018-01-27 18:34:03 +02:00
Jordan Petridis
a3705d424b
NewSource: Remove Insert implemantation and replace it with a insert_or_ignore method.
2018-01-27 18:09:40 +02:00
Jordan Petridis
77a52bdc8c
hammond-data: Add a source_exists query.
2018-01-27 14:23:19 +02:00
Jordan Petridis
18e55e23ee
Headerbar: Do not allow insertion of invalid urls.
2018-01-27 14:11:02 +02:00
Jordan Petridis
d14973cf0d
Use num_cpus instead of hardcoding the number of threads for dns look_up.
...
In the next hyper_tls relase it should be possible to use the existing
cpu_pool executor.
2018-01-27 11:08:57 +02:00
Jordan Petridis
120d494280
Update hyper and futures.
2018-01-27 09:45:25 +02:00
Jordan Petridis
39aa86157b
Switch rest of the tests to point to the internet archive urls.
2018-01-26 17:25:35 +02:00
Jordan Petridis
df85a4ea9e
Pipeline: avoid panicking by checking the lenght and returning an error.
2018-01-26 14:17:43 +02:00
Jordan Petridis
2912bad110
Pipeline: Split the pipeline from the cpu-pool and the tokio-core.
2018-01-26 13:34:04 +02:00
Jordan Petridis
b370af35d4
Feed: Add a bunch of tests.
2018-01-24 18:42:33 +02:00
Jordan Petridis
d6817aaa1f
Parser: pass an rss::ItunesExtension to instead of an rss::Item.
2018-01-24 15:42:08 +02:00
Jordan Petridis
5da002fe6d
Use map_err(From::from) instead of ? into Ok() wrapping pattern.
2018-01-24 15:27:36 +02:00
Jordan Petridis
0e16f0acb0
NewEpisode: rename into_episode -> to_episode, change its signature and add unit test.
2018-01-24 14:31:33 +02:00
Jordan Petridis
d5a7fa9de8
NewEpisode: Add index method unit test.
2018-01-24 12:11:49 +02:00
Jordan Petridis
9274aba849
NewEpisode: Add update method unit test.
2018-01-24 11:22:03 +02:00
Jordan Petridis
457e43978f
Pipeline: Fix a panic if the list of futures was 0.
2018-01-23 19:07:11 +02:00
Jordan Petridis
f9096e5fac
Pipeline: Print the error right away instead of waiting till all futures complete.
2018-01-23 18:18:56 +02:00
Jordan Petridis
e74a2df27f
Source: On Http 301 response, update Source.uri .
2018-01-23 16:43:02 +02:00
Jordan Petridis
1c4138c166
Benches: Remove dead stuff.
2018-01-23 13:24:08 +02:00
Jordan Petridis
050fe9c52b
Pipeline: offload more stuff to the threadpool.
2018-01-23 11:43:37 +02:00
Jordan Petridis
8095919bbe
This sort of works?
2018-01-23 10:12:32 +02:00
Jordan Petridis
f7f6087d70
NewPodcast: On update unit test assert .id and .source_id fields stay the same.
2018-01-22 21:09:06 +02:00
Jordan Petridis
ea75d50053
NewEpisode: Add insert unit test.
2018-01-22 17:25:23 +02:00
Jordan Petridis
44d7a4e222
NewEpisodeMinimal: Add unit test for into_new_episode method.
2018-01-22 15:49:58 +02:00
Jordan Petridis
2d6f34fa4d
NewEpisode: Clean up tests a bit.
2018-01-22 15:34:57 +02:00
Jordan Petridis
d231cc165f
Fix batch indexing.
2018-01-22 14:51:31 +02:00
Jordan Petridis
7bd361fbbb
Upgrade dependancies.
2018-01-22 11:47:17 +02:00
Jordan Petridis
66e816fc5f
Upgrade Diesel to 1.1
2018-01-22 11:22:01 +02:00
Jordan Petridis
20f36a67a0
Upgrade log and loggerv.
2018-01-21 15:22:13 +02:00
Jordan Petridis
eb75e68b72
Fix benchmarks.
2018-01-21 14:52:21 +02:00
Jordan Petridis
dc9a28c0c7
Remove xml files.
2018-01-21 14:49:54 +02:00
Jordan Petridis
843feabf25
NewEpisode: Move the parsing tests, delete old xml files.
2018-01-21 13:35:35 +02:00
Jordan Petridis
937bd61d5e
NewPodcast: Add unit test for the to_podcast() method.
2018-01-21 12:17:47 +02:00
Jordan Petridis
4ee5888b7f
NewPodcast: Rename into_podcast method -> to_podcast since it now does not take owenership.
2018-01-21 11:04:15 +02:00
Jordan Petridis
73eeb1b545
NewPodcast: Add NewPodcast.index unit test.
2018-01-21 10:04:53 +02:00
Jordan Petridis
16bbfb5478
NewPodcast: Add (lackign) NewPodcast.update unit test.
2018-01-21 09:35:48 +02:00
Jordan Petridis
410657c98b
NewPodcast: Remove the boilerplate on the unit tests.
2018-01-21 08:57:50 +02:00
Jordan Petridis
1625f773c2
hammond-data: Fix PartialEq implementations, add NewPodcast.insert unit test.
2018-01-21 08:39:50 +02:00
Jordan Petridis
da8c3a7827
Implement PartialEq accross Insertable and Queryable diesel models.
2018-01-20 19:09:25 +02:00
Jordan Petridis
6c9e0984a5
Migrate podcast tests to use the internet archive snapshots.
2018-01-20 18:02:07 +02:00
Jordan Petridis
df17a10014
Source: refactor into_feed test.
2018-01-20 16:43:32 +02:00
Jordan Petridis
ef52a026bc
Feed: Add Builder pattern.
2018-01-20 08:46:05 +02:00
Jordan Petridis
7d9781052f
hammond-data: Rework the modules privacy.
2018-01-20 07:28:30 +02:00
Jordan Petridis
855b1517a7
Feed: Removee unimplemented! hack to guard against my future stupidity.
2018-01-19 19:20:21 +02:00
Jordan Petridis
1606ceaadb
Source: split response_to_channel into more futures.
2018-01-19 16:29:50 +02:00
Jordan Petridis
46b6c0d27e
hammond-data: Remove leftover synchronous code.
2018-01-19 15:41:37 +02:00
Jordan Petridis
b92ba7be76
Switch from libtest to criterion.rs for benchmarks.
2018-01-19 14:32:43 +02:00
Jordan Petridis
44dbb06dcc
Add benchmarks for the async path.
2018-01-19 13:15:21 +02:00
Jordan Petridis
e6b0cfccb5
Apply clippy suggestions.
2018-01-19 10:32:25 +02:00
Jordan Petridis
5c5faafc72
Source: Move around code and change some methods signature.
...
request_constructor is now a Source method.
update_etags now takes &mut self instead of mut self.
update_etags now uses source setters methods instead of raw fields.
changed the naming of into_feed to to_feed according to rust convention.
2018-01-19 07:51:55 +02:00
Jordan Petridis
9dc555cad7
Feed: Add an index_async implementation.
2018-01-18 18:57:44 +02:00
Jordan Petridis
1394366f91
hammond-data: Delete some dead code.
2018-01-18 17:04:57 +02:00
Jordan Petridis
6abf2535b0
hammond_data: Make index a trait that requires Insert + Update.
2018-01-18 16:33:07 +02:00
Jordan Petridis
1c96288178
hammond_data: Add more benchmark cases.
2018-01-18 15:50:48 +02:00
Jordan Petridis
098c5755b0
hammond-data: Do batch indexing of new_episodes.
2018-01-18 13:47:40 +02:00
Jordan Petridis
93372a30d0
1.21 Gigawatts. Remove non-future indexing loop.
...
Tried to have a seperate futures loop but it's too confusign having
too write a Trait2, functon2, etc version of everything and keep it together.
Futures are functional sort of, so the synchronous versioun can be removed.
It still needs a ton of work though to be ready, or even get near the perf
of of the sync+rayon version.
2018-01-18 11:38:24 +02:00
Jordan Petridis
5d88998180
hammond_data: Add an EpisodeMinimal Diesel model.
2018-01-17 10:21:16 +02:00
Jordan Petridis
574cfae5c6
rustfmt: enable reorder imports.
2018-01-17 08:57:02 +02:00
Jordan Petridis
e707087e72
hammond_data: Split models::queriables into multiple modules.
2018-01-17 08:45:14 +02:00
Jordan Petridis
3a6f6d8931
hammond_data: Split models::insertables into multiple modules.
2018-01-17 08:27:39 +02:00
Jordan Petridis
3766db2b14
hammond_data: Move parse_podcast into a NewEpisode method.
2018-01-17 07:31:24 +02:00
Jordan Petridis
8174fe0bac
hammond_data: Move parse_episode from parser into a NewEpisode method.
2018-01-17 07:16:59 +02:00
Jordan Petridis
bd9844f012
Dbquerries: Use map_err() instead of ? into Ok() pattern.
2018-01-17 07:13:34 +02:00
Jordan Petridis
f64779f70a
Parser: Add NewEpisodeMinimal struct.
...
Parsing whole episodes can be expensive and we only need
a subset to determine if it should be indexed/updated or ignored.
2018-01-16 18:33:32 +02:00
Jordan Petridis
978e5a61f6
Dbquerries: add EXIST querries for podcast and episode.
2018-01-16 17:26:41 +02:00
Jordan Petridis
fa03c20b00
Feed: add parse_channel_items_future method.
2018-01-16 14:52:20 +02:00
Jordan Petridis
bf4f655ed2
Pipeline: remove submodule hack.
2018-01-16 14:37:51 +02:00
Jordan Petridis
b3460b15a2
hammond_data::Feed: add parse_podcast_future method.
2018-01-16 13:44:33 +02:00
Jordan Petridis
7f78e87551
hammond_data::Feed: Remove unused parts.
2018-01-16 12:35:08 +02:00
Jordan Petridis
3358fcd0b3
hammond_data::Feed: general cleanup of no longer needed stuff.
2018-01-15 11:03:40 +02:00
Jordan Petridis
10345ffda7
Pipeline: add matching on status code.
2018-01-15 10:16:01 +02:00
Jordan Petridis
2f7a22355f
Minor cleanup of the mess.
2018-01-14 08:27:50 +02:00
Jordan Petridis
e63a366fdc
I don't know why the benchmarks keep cycling.
2018-01-14 06:48:42 +02:00
Jordan Petridis
1031315cdd
Getting somewhere?
2018-01-14 06:37:47 +02:00
Jordan Petridis
a5fd79e220
I hate futures error handling.
2018-01-13 14:47:38 +02:00
Jordan Petridis
e162f8fd3f
Non-working yet benchmark.
2018-01-13 08:31:16 +02:00
Jordan Petridis
e4d77a6ba4
Cleanup the id() method mess of the diesel models.
2018-01-13 07:46:56 +02:00
